Caching: Faster Reads, Carefully Chosen Staleness

A cache keeps a copy of data in a faster or closer place so repeated work can be avoided. That might be your CPU cache, a browser cache, a CDN edge, or Redis beside your application servers. The important word is copy. Once there are two places for the same fact, freshness becomes a design problem.

Caching is powerful because it changes a repeated expensive operation into a quick lookup. It is also a source of some of the most confusing bugs: “I changed the price but I still see the old one.” Good caching starts with that trade-off in the open.

Begin with the slow path

Imagine GET /projects/42 normally reads PostgreSQL. It is correct, but perhaps it is called thousands of times a minute and the project changes only a few times a day. Cache-aside (also called lazy loading) is a sensible first strategy:

async function getProject(id: string) {
  const key = `project:${id}`;
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const project = await projects.findById(id);
  if (!project) return null;
  await redis.set(key, JSON.stringify(project), { EX: 300 });
  return project;
}

On a cache hit, the database is not involved. On a miss, the application reads the source of truth and populates the cache for a later request. This means the cache can be empty after a deploy, eviction, or Redis restart without losing real data. That property makes cache-aside a reassuring default.

Decide what can be stale

Every cached value needs a freshness story. A TTL (time to live) makes a value expire after a period. It is a simple safety net, not a guarantee that the value is fresh until the last second.

For each candidate, ask:

“Cache everything” is not a performance strategy. Start with a measured slow read and a business-acceptable staleness window.

Invalidation is the hard part because writes change facts

When PATCH /projects/42 succeeds, remove or update project:42:

await database.transaction(async (tx) => {
  await projects.update(tx, id, changes);
  await outbox.record(tx, { type: "project.changed", projectId: id });
});
// a worker can invalidate project:42 after the committed event

Deleting the key is often safer than trying to rebuild the representation in the write path. The next read repopulates it from the database. There is still a small race to understand: another request may read the old cache just before or during invalidation. If that is unacceptable, the problem has moved beyond ordinary caching into consistency design.

For changes that must reliably reach a cache after a database commit, an outbox/event pattern avoids the failure mode “database updated, process crashed before cache delete.” The queue note expands that idea.

Cache keys are an interface

Treat key names as a small namespace, not random strings:

project:42
project:42:tasks:open:v1
organisation:8:dashboard:v3

Include identifiers and representation variants that change the answer. Never serve an admin-only cached response to a regular member because both used the same dashboard key. If output depends on locale, permissions, tenant, or feature flag, those inputs must be reflected in the key—or the response should not be cached at that layer.

The failure modes worth knowing early

Cache stampede

If a popular key expires, hundreds of requests may miss together and all hit the database. Mitigations include adding TTL jitter, allowing one request to refill while others wait briefly, or serving a slightly stale value while refresh happens in the background.

Hot keys

One viral item can concentrate traffic on a single cache key or Redis shard. Local in-process caching, replication, and carefully designed sharding may help, but first measure the concentration.

Negative caching

Caching “project 42 does not exist” briefly can protect a database from repeated misses. Keep its TTL short: a newly created project should not remain invisible for minutes.

Eviction and memory pressure

Caches are finite. Redis may evict keys using policies such as LRU-like or TTL-driven eviction. Design as though any key may disappear. A cache should improve performance, not become the only holder of important data.

There are several cache layers

• Browser and HTTP caches can avoid a request entirely using Cache-Control, ETags, and conditional requests.
• CDNs cache public static or cacheable content near users.
• Application caches such as Redis share hot data across API instances.
• Local in-memory caches are very fast but diverge across instances and vanish on restart.

The closest cache is usually fastest; the more copies you have, the harder invalidation becomes. Pick the layer that solves the observed problem with the fewest new moving parts.

A safe first caching plan

1. Measure latency and database load for a specific read.
2. State the acceptable staleness in plain language.
3. Choose a key and TTL.
4. Make cache failure fall back to the source of truth.
5. Add invalidation on the write path where it is worth it.
6. Monitor hit rate, miss rate, latency, evictions, and database load after release.

If you cannot say what happens when Redis is empty, slow, or unavailable, the cache has become an unexplained dependency. A cache is successful when users notice faster responses and nobody has to wonder which copy of the data is real.

When not to cache

Do not cache a value merely because the endpoint is slow. First ask whether the query fetches too much data, lacks an index, makes an N+1 series of database calls, or calls a remote service unnecessarily. Fixing the underlying access path is often simpler and gives correct results without a stale-copy policy.

Also be cautious with permission-sensitive responses. A cached project summary might be safe when it contains public fields; a dashboard that changes per user, role, tenant, feature flag, and locale can become dangerous or expensive to key correctly. In those cases, cache smaller shared pieces or improve the database query instead.

The best cache is one with an explicit owner, an expiry, a failure fallback, and a reason it exists. If a new engineer cannot remove it confidently during debugging, leave a short comment explaining the data source and invalidation path.