Caching Correctness: TTL, Key Design, and Invalidation Rules That Hold Under Load

What Caching Correctness Actually Protects

Caching correctness is the property that a cached value is never served to a request the underlying data would have answered differently. It is a narrow slice of what code governance covers, and it decides whether your read path is a performance optimization or a second source of truth that quietly disagrees with the first. A cache is not a layer sitting on top of the data model. It is a weaker copy of the data model with its own key space, its own retention policy, and its own authorization boundary, and each of those three has to be designed rather than inherited from whatever the client library does by default.

None of these principles are about developer carelessness. They are about the fact that a cache write is a local decision with global consequences. The engineer setting the key has all the context. The engineer reading it six months later from another service has none, and nothing in between forces the two to agree.

1. Cache Only What You Can Enumerate

Search results get cached on the raw query string. Within a week the cache holds four million keys, almost all of them hit exactly once, and the eviction policy is quietly discarding the entity lookups that were actually paying for themselves. Cache only against a bounded key space: an entity identifier, a validated enumeration value, a normalized filter tuple you can write down. If you cannot enumerate the set of keys a call site can produce, the cache will fill with values nobody will ask for twice.

key = f"search:{request.args['q']}:{request.args['filters']}"
if (hit := redis.get(key)) is None:        # near-zero hit rate
    hit = search_index.query(request.args["q"])
    redis.set(key, hit)                    # and no expiry either

facet = Facet.parse(request.args["filters"])   # rejects unknowns
key = f"search:v3:{facet.canonical()}:{page}"  # finite key space
if (hit := redis.get(key)) is None:
    hit = search_index.query(facet)
    redis.setex(key, 300, hit)

The cost is that free-text search stops being cacheable at the application layer, which is the correct outcome: that workload belongs to the search engine's own cache, where the memory budget and eviction policy are tuned for it. Redis key eviction policies will make this decision for you if you do not, and the volatile-* policies only consider keys that have an expiry set at all.

2. Every Cache Write Carries Its Own Expiry

A feature flag payload gets cached during an incident-response deploy. The flag service is decommissioned four months later. The payload is still being served, because nothing ever expired it and memory pressure never rose high enough to evict it. Every set, put, or store call passes an explicit expiration argument at the call site. Not a client default, not a global config, not an eviction policy: an argument, visible in the diff, that a reviewer can argue with. Redis clears an existing expiry when a key is overwritten with a plain SET, so "we set the TTL once at startup" is not the same thing as "it has a TTL."

redis.set(f"flags:{tenant_id}", json.dumps(payload))   # forever

// Java, same bug, different client
jedis.set("flags:" + tenantId, mapper.writeValueAsString(payload));

redis.setex(f"flags:{tenant_id}", 60, json.dumps(payload))

// Go, with jitter so a deploy does not expire every key at once
rdb.Set(ctx, key, payload, 60*time.Second+jitter(10*time.Second))

Short TTLs raise miss rates and push load back onto the source of truth, and every key that expires at the same instant produces a stampede against the same query. Add jitter, and for the handful of keys that are genuinely hot, consider probabilistic early recomputation, where each reader independently decides with rising probability to refresh the value before it expires.

3. The Key Namespace Carries the Schema Version

A struct gains a field. New code deploys, reads a value serialized by the old code, and either throws on a missing attribute or silently defaults it to zero. For the length of the rollout, half the fleet runs new code and half runs old, and both are reading and writing the same keys. Every cached value that is a serialized application object carries a version component in its key namespace, bumped in the same commit that changes the shape.

CACHE_SCHEMA = "v4"   # bumped in the same commit as the struct
key = f"user:{CACHE_SCHEMA}:{user_id}"

# old code reads user:v3:*, new code reads user:v4:*
# no migration, no dual-read, no flush; old keys expire on their TTL

This is the cheapest correctness win in caching and the one most often skipped, because the failure only appears during a rolling deploy and only for the minutes when both versions are live. A version in the namespace turns a schema migration into two disjoint key spaces that never meet. It also removes the temptation to flush the whole cache on deploy, which converts a schema problem into a thundering herd against the database.

The version belongs in the key, not inside the serialized payload. A version field in the value still requires the reader to deserialize something written by code it does not have, which is the failure you were trying to avoid. Keys are cheap. Deserializing a foreign shape to discover you cannot use it is not.

4. Identity Belongs in the Key, Secrets Belong Nowhere

The response cache keys on the request path. /api/v2/invoices returns tenant A's invoices to tenant B for the next five minutes, and the access log shows two clean 200s. This is the caching bug that becomes a breach notification rather than a bug ticket. Any value whose content depends on who asked carries the tenant identifier, user identifier, or role scope in its key, and it carries it in the key rather than in a wrapper the reader is trusted to check.

If two callers with different permissions can produce the same cache
key, the cache is an authorization bypass with a TTL.

wrong:  resp:/api/v2/invoices
right:  resp:v2:invoices:tenant=8812:role=finance_viewer

HTTP has had this rule since before most application caches existed. RFC 9111 requires a shared cache to refuse to reuse a response to a request carrying an Authorization header unless the response explicitly permits it, and it defines the Vary header precisely so the cache key can include the request dimensions the response varied on. Application caches inherit none of that machinery. You are writing the cache key by hand, which means you are writing the authorization boundary by hand.

The related rule is simpler and absolute. Passwords, API keys, private keys, session tokens, and payment instrument values never enter a shared application cache. A Redis instance is a network service with its own auth, its own persistence, and its own backups, and none of those were reviewed as part of your secrets threat model. Resolve those values from the secrets manager on each use and pay the latency.

5. Invalidation Belongs in the Write Path

A user changes their email. The profile service commits the row, returns 200, and the settings page still shows the old address because the entry expires in fifteen minutes. Support gets a ticket, an engineer adds a scheduled job that clears profile keys every minute, and now a second system has to know every key shape in the codebase. Invalidate in the same function that commits the write, by deleting the key.

def update_email(user_id, email):
    db.execute(update(User).where(User.id == user_id)
                          .values(email=email))
    db.commit()
    return 200   # cache holds the old row for another 15 minutes

# elsewhere: a cron that DELs profile:* every minute and knows
# nothing about the six other key shapes built from the same row

def update_email(user_id, email):
    with db.begin():
        db.execute(update(User).where(User.id == user_id)
                              .values(email=email))
    for key in cache_keys_for_user(user_id):  # one place owns shapes
        redis.delete(key)                     # delete, never set

Delete rather than write the new value from the write path. Two concurrent writers that each set their own version race, and the loser's value can survive in the cache indefinitely, whereas two concurrent deletes are idempotent and the next reader repopulates from the source of truth. This is the look-aside model Facebook's memcache paper settled on for the same reason: the write path deletes, the read path fills. AWS documents the same tradeoff space under lazy loading, write-through, and TTL strategies. The cost is one extra miss per write, which is the correct price.

6. A Cache Outage Is a Miss, Not an Error

The cache node fails over. Every read path that treated redis.get as infallible now raises a connection error, and a service whose source of truth is completely healthy returns 500 for ninety seconds. The cache was added to make the system faster and it made it less available instead. Wrap every cache read so that any cache error or timeout is treated as a miss and the call proceeds to the source of truth.

The second half of the rule is the timeout. Client library defaults are frequently measured in seconds, or absent entirely, which means a cache that is up but slow queues requests behind a lookup that was supposed to save you a database round trip. Set an explicit connect timeout and command timeout on every cache client, in the tens of milliseconds, and derive them from the latency of the source of truth: if the fallback query takes 40ms, waiting 200ms for the cache is strictly worse than not having one. This is the same reasoning as the retry, timeout, and circuit breaker guide, applied to a dependency people forget is a network call.

The tradeoff is real and worth naming. Falling through to the source of truth on every cache error means a cache outage becomes a full-load event against the database, which is the moment the database is least able to absorb it. That is an argument for load shedding and a circuit breaker in front of the fallback, not an argument for letting the cache error propagate. A degraded response is recoverable. A 500 is not.

7. Cache Topology Is a Codebase-Wide Contract

The pricing service caches the rate table in a module-level dictionary because it changes twice a year and Redis felt like overkill. It runs on twelve pods. A rate change ships, the invalidation deletes the Redis key, and eleven pods keep serving the old rate until the next deploy. Nothing in the pricing service's own code is wrong. Data that another instance can modify does not belong in a process-local cache, and that is a statement about the whole fleet rather than about one file.

This is where individual competence stops being enough. A reviewer looking at one pull request sees a static map and a comment explaining why it is safe. They cannot see the invalidation path in another repository that assumes the only copy lives in Redis. They cannot see that the same entity is cached under three different key shapes in three services, so a delete from one clears a third of the copies. Netflix's EVCache exists because past a certain number of services the caching tier has to become a shared contract with one key model, rather than a decision each team makes well and independently.

The same is true of the other six rules once you pass a handful of services. TTLs, schema version prefixes, tenant scoping, delete-on-write, client timeouts, the single-key get sitting inside a loop that turns one round trip into two hundred, and the hit and miss metrics that would tell you any of this was broken are all trivially checkable in a single file and completely invisible across a repository boundary. A cache with no hit rate metric is not a cache, it is a hypothesis. That gap between locally reviewable and globally true is what a governance layer exists to close.

How Pandorian Enforces This Guide

Every principle above is statically observable. A cache set with no expiration argument, a key template built from a raw request parameter, a per-user response keyed only on a path, a cache read with no surrounding error handling: each is a code shape rather than a judgment call. Pandorian's enforcement surfaces run the same guideline package at three different moments.

Caught at PR level

A cache write added with no TTL, or a key template interpolating a raw query parameter, surfaces on the pull request that introduces it, with the line, the rule, and a suggested fix. This is where new violations stop, and it is the only surface fast enough to change the shape of the code before it merges.

Full scans

An on-demand scan across the whole codebase establishes the day-one baseline, including everything that predates the guideline: the keys that were never versioned, the process-local caches nobody remembers adding, the read paths with no error handling around the client call. This is the list you triage, not the list you fix in a sprint.

Scheduled scans

A new service that ships with its own cache client and its own key convention, a repository inherited from an acquisition, generated code that reintroduces a pattern the team removed last quarter: drift arrives outside any pull request you were watching. The same scan on a cadence catches it while it is still one service.

A Taste of the Guideline Package

FAQ