Cache Strategy in System Design
An Experienced Engineer’s Walkthrough for Backend Engineers
When reads become heavy, one of the first tools we reach for is a cache: keep a copy of data in fast storage (memory, Redis) so that many reads never touch the database. As a newcomer, it’s easy to think “cache is just get/set” and wire it in without deciding who fills the cache, who invalidates it, and when writes touch cache vs DB. Those choices have a big impact on consistency, complexity, and failure modes — and they have names: Cache-Aside, Read-Through, Write-Through, Write-Behind, plus layering (local + distributed) and invalidation (TTL, active). This article is written as if I’m sitting next to you: we’ll go through each strategy, what the read and write paths look like, when to use which, and which failure modes (penetration, breakdown, avalanche) to guard against. No prior experience with cache patterns is assumed.
Lesson 1: The Core Question — Who Owns the Cache?
Before comparing strategies, one question matters: Who is responsible for loading and invalidating the cache?
- Application (Cache-Aside): The app checks the cache; on miss, the app loads from DB and writes to cache. The app also invalidates (or updates) cache on write. The cache is “aside” — the app talks to both cache and DB. So the app owns the logic: “if not in cache, load from DB and fill cache; on write, update DB then invalidate cache.”
- Cache layer (Read-Through, Write-Through, Write-Behind): The cache (or a library/proxy) proxies reads and optionally writes. On read miss, the cache loads from DB (via a loader you configure); the app only talks to the cache. Writes may go to cache and DB together (Write-Through) or cache first, then async to DB (Write-Behind). So the cache layer owns “when to load” and “when to write to DB.”
That ownership determines consistency (e.g. can cache be stale after a failed write?), latency (who pays the cost of a miss?), and operational complexity (where do you debug when cache and DB disagree?). As an experienced engineer, I always clarify this boundary first; many bugs come from “we thought the cache was filled by X but actually Y fills it.”
Lesson 1 Takeaway
Cache strategy is largely about who fills and invalidates the cache — the app (Cache-Aside) or the cache layer (Read/Write-Through, Write-Behind). Get that clear before you choose a pattern.
Lesson 2: Cache-Aside (Lazy Loading)
Cache-Aside is the most common pattern: the application owns the logic.
Read Path
- Hit: Read from cache; no DB.
- Miss: Load from DB, then write to cache (populate). Next read will hit cache.
Write Path
- Application writes to DB first, then invalidates the cache entry (or updates it; see below). The next read will miss and repopulate from DB, so the reader always sees the latest value from DB after a write.
- Why invalidate instead of update? If you update the cache with the new value and then the DB write fails, your cache now has data that never made it to DB — inconsistent. If you invalidate instead, the next read will load from DB; so even if the DB write fails, you don’t leave a wrong value in cache. Invalidate is simpler and safer. Update is possible if you’re sure the DB write will succeed (e.g. you do both in a transaction), but invalidate is the default for Cache-Aside.
Pros and Cons
- Pros: Simple, robust; app has full control; invalidate-on-write avoids stale data from failed writes.
- Cons: Every miss causes a DB read + cache write; application code must handle both cache and DB.
Lesson 2 Takeaway
Cache-Aside = app reads cache and DB, populates on miss, invalidates on write. Simple and robust; good default for most read-heavy cases.
Lesson 3: Read-Through and Write-Through
Read-Through
- The cache layer (or a library) proxies reads. The app only calls "get key"; the cache returns from memory or, on miss, loads from DB (via a configured loader), stores in cache, and returns. The app is unaware of DB.
- Use when: You want a single interface (cache only) and are okay with the cache layer owning the loader.
Write-Through
- Writes go to the cache; the cache synchronously writes to DB as well. So cache and DB are updated together. Read path is cache-only (cache has the latest).
- Pros: Strong consistency (cache and DB in sync after write).
- Cons: Write latency includes DB write; if DB fails, you need a clear policy (e.g. fail the request or leave cache/DB inconsistent).
Lesson 3 Takeaway
Read-Through = cache proxies read and loads from DB on miss. Write-Through = write updates both cache and DB synchronously; good consistency, higher write latency.
Lesson 4: Write-Behind (Write-Back)
- Writes update the cache first; writes to DB are asynchronous (batched or queued). So write latency is low (only cache write), and throughput can be high.
- Risk: If the process crashes or the queue is lost before flush, data in cache may never reach DB → data loss. Use with care (e.g. for non-critical or recoverable data).
Lesson 4 Takeaway
Write-Behind = cache first, async to DB; high throughput, but risk of data loss on failure. Use only when you accept that trade-off.
Lesson 5: Strategy Comparison Table
| Strategy | Read | Write | Consistency | Complexity |
|---|---|---|---|---|
| Cache-Aside | Cache first, then DB on miss | DB first, then invalidate cache | Eventual | Low |
| Read-Through | Cache proxies; loads from DB on miss | (Separate write strategy) | Depends on write | Medium |
| Write-Through | From cache | Write cache + DB together | Strong | Medium |
| Write-Behind | From cache | Cache first, async to DB | Weak | High |
When to Choose
- Read-heavy, eventual consistency OK → Cache-Aside + TTL.
- Write-heavy, strong consistency → Write-Through or no cache.
- High write throughput, accept loss risk → Write-Behind (use with care).
Lesson 5 Takeaway
Match the strategy to read/write mix and consistency requirements. Cache-Aside is the safe default; Write-Through when you need strong consistency; Write-Behind only when you explicitly accept the risk.
Lesson 6: Layering (Local + Distributed)
- Local cache (e.g. Caffeine, in-process): Very fast, but not shared across instances. Good for hot keys per instance.
- Distributed cache (e.g. Redis): Shared across instances; higher latency than local.
- Layered: Request → local → Redis → DB. Local TTL short (e.g. 1 min), Redis longer (e.g. 1 h). Reduces load on Redis and DB; invalidate both layers on write (or use short TTL to converge).
Lesson 6 Takeaway
Layering (local + distributed) reduces latency and load; keep local TTL short and invalidate (or TTL) on write so data converges.
Lesson 7: Failure Modes and Mitigations
These three failure modes show up in production when you don’t plan for them. As a newcomer, they’re easy to miss until the DB is overloaded or a hot key causes a spike.
- Cache penetration: Someone (or a bot) requests a key that does not exist in DB (e.g. random IDs). Every request misses cache and hits the DB. So the DB gets hammered for “nothing.” The fix: when you load from DB and find no row, still write to cache an “empty” value (e.g. null or a sentinel like
"N/A") with a short TTL (e.g. 1–5 minutes). Then repeated requests for the same non-existent key hit cache and don’t hit DB. As an experienced engineer, I always cache “not found” for keys that are likely to be requested again. - Cache breakdown (thundering herd): A hot key (e.g. “current config”) expires at time T. At time T, many requests for that key arrive; they all miss and all go to the DB at once. So one key causes a spike. The fix: single-flight (only one request does the load; others wait for that result and then use it) or a lock so only one process loads and repopulates; the rest read from cache once it’s filled.
- Cache avalanche: Many keys were written at the same time with the same TTL, so they expire at the same time. Then a large number of keys miss at once and the DB gets a burst of load. The fix: TTL jitter — when you set TTL, add a small random offset (e.g. TTL + random(0, 60) seconds) so expiry times are spread out. Then not all keys expire in the same second.
| Problem | What happens | Mitigation |
|---|---|---|
| Cache penetration | Miss on non-existent key → every request hits DB | Cache empty value (e.g. null or "N/A") with short TTL |
| Cache breakdown | Hot key expires → many requests hit DB at once | Single-flight (one request loads, others wait) or lock |
| Cache avalanche | Many keys expire at same time → DB overload | TTL jitter (e.g. TTL + random offset) so expiry is spread |
Key Rules
- Invalidate on write: After write, invalidate cache; next read repopulates. Avoids cache–DB mismatch from failed write-through.
- Monitor: Hit rate, latency, DB load; tune TTL and strategy by metrics.
Lesson 7 Takeaway
Penetration → cache empty value. Breakdown → single-flight or lock. Avalanche → TTL jitter. Invalidate on write to keep cache and DB aligned.
Key Rules (Summary)
- Invalidate on write (Cache-Aside): After write, invalidate cache; next read repopulates.
- Mitigate penetration, breakdown, avalanche: Empty value cache, single-flight, TTL jitter.
- Monitor: Hit rate, latency, DB load; tune TTL and strategy by metrics.
What's Next
See Cache-Aside, Caching Pitfalls. See Redis, Hot Key for distributed cache design.