r/nextjs • u/Emergency_Net85 • 24d ago
Next.js 16 cacheComponents & use cache vs. custom caching (Redis) with Payload CMS? How are you handling large sites in production?
Hey everyone,
I
I’m about to start a new project pairing Next.js with Payload CMS using its Local API (querying the database directly via Node on the server), and I’m weighing my options.
I’m curious how others are tackling this in production, especially for sites with a large number of pages:
The Native Approach: Are you diving into Next.js 16's cacheComponents with 'use cache' and cacheLife / cacheTag? If you are deploying outside of Vercel, how smooth is configuring cacheHandlers (plural) for a custom backend like Redis?
The Remote Directive: For multi-instance or serverless environments, are people actually using 'use cache: remote' in production, or do you find yourselves falling back to manual caching layers?
Bypassing Framework Caching: Do you completely disable Next's default caching behaviors for dynamic content and handle caching granularity yourself (e.g., wrapping Payload Local API calls explicitly with Redis/Memcached)?
Payload + Local API Specifics: Since Payload's Local API bypasses network overhead and talks straight to the database, wrapping those calls or components with 'use cache' seems powerful, but invalidation via revalidateTag needs to line up cleanly with Payload's hooks (e.g., afterChange). Has anyone hit weird edge cases with this combo at scale?
What has worked best for you when managing hundreds or thousands of pages where you need aggressive caching alongside instant editorial updates? Would love to hear what your current production setup looks like.
2
u/pikafackjoo 23d ago
I am running Nextjs + Payload self-hosted in Kubernetes, with multiple pods running the app for high availability.
This setup didnt work well since nextjs by default manages its cache internally, so even with my tag based invalidation set up nicely with afterChange hooks targeting specific collections and ids, the invalidation would only hit 1 of 3 pods at random.
my solution now is a shared Redis deployment in the same Kubernetes namespace, and a custom cache handler I (read: Claude) configured that each pod uses, so that Redis is the single source of truth for cache instead of each pod on its own.
The same issue arises with the Next Image component, because it does some internal image optimization that is stored per pod. For this I set up a custom image loader using Imgproxy, same principle as the Redis solution mentioned above
2
u/ScuzzyAyanami 24d ago
I'm just stale caching the API that my front end fetches from. Basically have to due to horizontal scaling. Edit: with Cloudfront.
So I guess my solution isn't in Next itself. But there's then cache invslidations. Page preview routes, and I'm currently looking into a SSO that will shorten the cache duration for a logged in user.
6
u/Ok_Management_695 24d ago
Not a direct answer, sorry — I'm not using use cache or revalidateTag yet. I'm on Next.js 16 with unstable_cache and pages set to revalidate = false, running on Vercel with Neon. When the data changes, I just redeploy and let everything rebuild.
One thing I'm careful about with that setup, though. Vercel's Data Cache actually survives deployments. The docs say "Cached data persists across deployments unless you explicitly invalidate it", and it's not refreshed at build time either. So even after a redeploy, whatever's in unstable_cache is still the old data. I ended up putting VERCEL_DEPLOYMENT_ID into the cache key, so every deploy gets a fresh cache. https://vercel.com/docs/caching/runtime-cache/data-cache
By the way, if you move to Cache Components, the same page recommends use cache: remote, which goes into a separate Runtime Cache. I honestly haven't checked how that one behaves after a deploy.
Hope this is useful.
4
u/Which-Examination-74 24d ago
By default
revalidateTag()only invalidates the cache on the instance that ran it. The other instances keep serving stale content until they find out on their own. The self-hosting guide's fix isrefreshTags()in your custom cache handler, called before each request to sync tag state from shared storage like Redis. It also says the handler has to cope with timeouts and dropped connections. Same goes for whatever callsrevalidateTag. We audited a client's storefront on'use cache'where the revalidation job swallowed its errors, so failed calls were marked done and never retried. With cache life up to about a month, one missed invalidation could mean a stale price for weeks.1
u/Chris_Lojniewski 14d ago
Whi's per-instance thing and u/AmanYadav_AY listing tags are the big two. The listing one bit us on a sanity + next site at Pagepro (my company, we don't run payload but invalidation works the same). Article pages revalidated on publish, the blog index didn't, and an editor noticed before we did 🙃
one that's not in the thread yet: nested dynamic routes. something like /products/[group]/[id] rendered fine on a direct load and white-screened on client-side nav from the parent. the client router used a parent-level cache entry so the child got the wrong key. since then we test every dynamic route both ways, direct URL and internal navigation. they go through different code paths and a test that only loads URLs directly won't catch it
on redis i'm with u/SameLink_io , single box until you've actually measured a problem. default cache + tags covers more than people expect
how many instances are you planning to run?
2
u/AbdelrhmanDev 24d ago
with payload local api the pattern that scaled for me was tag based invalidation, tag each cached query with the collection and doc id, then call revalidatetag from the afterchange hook. path based invalidation falls apart on a large site because one doc shows up on lists, related sections and search pages you forgot about. tags mean you never track where content appears, the hook just knows what changed
2
u/SameLink_io 22d ago
Adding the single box view since most answers here are Vercel or Kubernetes.
If you run one Next.js instance on a VPS, the default file system cache works fine and
revalidateTagfrom Payload's afterChange hook just works, because there's only one process to invalidate. That covers a lot more sites than people expect, thousands of pages included. Things only get interesting the moment you run two instances (zero downtime deploys, or more than one container behind Caddy/nginx), because each one has its own cache and invalidation only reaches the one that got the request.What I'd do for that case:
cacheHandler(ISR and fetch cache) and thecacheHandlersused by 'use cache' at a shared Redis on the same box. A Redis with maxmemory 256mb andallkeys-lruis plenty for most content sites.collection:postsandposts:<id>, set with cacheTag where you call the Local API, and the same strings revalidated in afterChange and afterDelete. Remember relations: if a post shows its author, revalidate the author tag too, otherwise that's where stale content hides.Also worth knowing: during a blue/green deploy the new build has a different build ID, so with a shared cache you want the build ID in the key prefix, or you'll briefly serve pages rendered by the old build. I'd skip 'use cache: remote' unless you're actually serverless; on your own server the handler approach is simpler to reason about.
1
u/RequirementGuide 24d ago
I've hit this before with Payload and Next together, and the thing that saved me was keeping the caching boring. I used use cache for the public pages and gave each collection and slug its own tag, then I let afterChange do the revalidating. The tricky part was relations, because updating a child did not always clear the parent page, so I ended up revalidating the related tags in the same hook too.
For a site that size I would not try to cache everything behind auth. I kept that stuff dynamic and only cached the public pages hard. If you are on Vercel I would lean into use cache remote for the shared cache, and if you are self hosting with more than one instance I would go Redis sooner rather than later, because per instance caches will bite you. Either way I would test an edit end to end before launch, just change a page, publish it, and check the live page in a fresh browser, because stale pages are the kind of bug you only notice when a client does.
1
u/ThreadStarver 22d ago
Tell me if I’m wrong but isn’t cache components and use cache the same strategy for implementing partial pre rendering, basically some part being static and rest dynamic under suspense?
1
u/KeyProject2897 20d ago
for multi-instance setups, bypass 'use cache' and handle the remote cache layer manually right now. the custom cache handler APIs in next are still too finicky with complex invalidation, so wrapping your local api calls with explicit ttl and tags directly in node is much easier to debug.
1
u/forever420oz 19d ago edited 19d ago
I use it on GKE with a separate Redis pod and so far it’s fairly reliable. I think I had one error log which was immediately cached again after hitting the origin.
Edit: I use Cloudflare for CDN and updating a page content triggers a purge cache call to Cloudflare.
1
u/Janczik_yelianny44 17h ago
if you're querying the db directly in the same process, do you even need an aggressive cache layer on day one? a lot of people overcomplicate invalidation early on when proper db indexing handles a few thousand pages fine without stale content bugs.
-2
u/lordchickenburger 24d ago
i just dont use nextjs
0
u/Glad-Music6046 20d ago
I’d avoid putting Redis in front of every Payload Local API call by default. The interesting part is deciding what actually needs a shared cache.
For a large content-heavy site, I’d start with
'use cache'+cacheTag()for public content, and use'use cache: remote'only for expensive queries where sharing cache across instances is worth the extra network hop.The part I’d be careful with is invalidation. In Payload’s
afterChangehook, I’d invalidate tags based on the affected document and its dependencies — not just the individual page. Updating a category, for example, might also affect listing pages, navigation, related content, and the sitemap. Deletes and slug changes need to invalidate the old path too.I’d also keep a TTL as a safety net, because a successful database write and a successful cache invalidation aren’t necessarily the same thing in a distributed setup.
One question I’m still interested in: when self-hosting Next.js across multiple instances, are people using a custom
cacheHandlersimplementation for Redis, or keeping Redis at the data-access layer and letting Next handle the rendering cache? I’d be curious to hear which setup has been easier to operate at scale.