Caching Strategies

The latency gap

A single PostgreSQL query over a network costs 1–10 ms. L1 cache access is ~1 ns. Even in-memory Redis over TCP is ~0.1–1 ms—orders of magnitude faster than disk-backed databases. At 35K read QPS, serving from cache instead of DB can mean the difference between 35 database connections and 3,500.

What caching buys you

• Lower latency — p99 drops when hot paths skip the database
• Higher throughput — memory bandwidth exceeds disk IOPS by orders of magnitude
• Cost reduction — fewer DB replicas, smaller instance sizes, less cross-AZ traffic
• Resilience — stale cache can serve traffic during DB degradation (with explicit trade-offs)

Cache-aside (lazy loading)

Application manages the cache. On read: check cache → on miss, read DB → populate cache → return. On write: update DB → delete or update cache entry. Most common pattern in production.

sequenceDiagram
  participant App
  participant Cache
  participant DB
  App->>Cache: GET key
  alt cache hit
    Cache-->>App: value
  else cache miss
    Cache-->>App: null
    App->>DB: SELECT
    DB-->>App: row
    App->>Cache: SET key, TTL
    App-->>App: return value
  end

// Cache-aside pattern (pseudo-code)
public User getUser(long id) {
    String key = "user:" + id;
    User cached = redis.get(key);
    if (cached != null) return cached;

    User user = db.findById(id);
    if (user != null) {
        redis.setex(key, 300, user); // 5-min TTL
    }
    return user;
}

public void updateUser(User user) {
    db.update(user);
    redis.del("user:" + user.getId()); // invalidate, not update
}

Redis eviction

Redis uses maxmemory + maxmemory-policy. Production default: allkeys-lru or allkeys-lfu (Redis 4.0+) for cache-only deployments. Use volatile-lru when mixing cache keys (with TTL) and persistent keys (no TTL).

Invalidation strategies

flowchart LR
  App[App Server]
  DB[(PostgreSQL)]
  CDC[Debezium / CDC]
  Kafka[Kafka Topic]
  Inv[Invalidation Worker]
  Redis[(Redis)]

  App -->|UPDATE| DB
  DB --> CDC
  CDC --> Kafka
  Kafka --> Inv
  Inv -->|DEL user:42| Redis

Thundering herd problem

When a popular cache key expires (or is invalidated), thousands of concurrent requests miss simultaneously and hammer the database—the "thundering herd" or "dog-piling" effect. A key with 10K RPS and sudden invalidation can send 10K simultaneous DB queries.

Redis Cluster

Redis Cluster shards data across 16,384 hash slots using CRC16 mod 16384. Each master owns a slot range; replicas provide failover. Client libraries (Lettuce, Jedis) handle MOVED/ASK redirects. Minimum production topology: 3 masters + 3 replicas across AZs.

flowchart TB
  subgraph nodes [App Fleet]
    A1[App 1\nCaffeine L1]
    A2[App 2\nCaffeine L1]
    A3[App N\nCaffeine L1]
  end
  subgraph redis [Redis Cluster]
    M1[Master 1\nslots 0-5460]
    M2[Master 2\nslots 5461-10922]
    M3[Master 3\nslots 10923-16383]
  end
  DB[(PostgreSQL)]

  A1 & A2 & A3 -->|L2 miss ~1ms| redis
  redis -->|miss ~5ms| DB

L1 / L2 / DB hierarchy

• L1 (in-process) — Caffeine with 10–60s TTL; absorbs 90%+ of repeated reads on same node
• L2 (Redis) — Shared across fleet; 1–5 min TTL; source for L1 population
• DB — Source of truth; only hit on L2 miss (~1–5% of reads with good hit rates)

Cache warming

Cold start after deploy or failover causes mass cache miss. Warm caches by: pre-loading hot keys from DB on startup, running shadow traffic before cutover, or maintaining a "warm key list" refreshed by a background job. Netflix pre-warms EVCache from Cassandra on region bootstrap.

Hot key problem

A single key (celebrity tweet, flash sale SKU) can exceed one Redis node's ~100K ops/sec limit. Solutions: local replication (app holds in-memory copy), key splitting (product:42:shard-{0..7} with random read), read replicas for the hot slot, or multilayer CDN for read-heavy public data.

Cache-Control headers

CDN invalidation

Purge by URL, path prefix, or cache tag. CloudFront invalidation takes 10–60 seconds and costs per path. Prefer versioned URLs (/assets/app.v42.js) over purge for deploys. Use cache tags (Fastly Surrogate-Key) for granular group invalidation without listing every URL.

Edge functions

CloudFront Functions, Lambda@Edge, Fastly Compute@Edge run logic at PoPs—A/B routing, auth token validation, header rewriting, bot filtering, and personalized cache key construction. Latency: <1 ms for CloudFront Functions, ~10–50 ms for Lambda@Edge (includes cold start risk).

sequenceDiagram
  participant User
  participant Edge as CDN Edge PoP
  participant Origin as Origin Server

  User->>Edge: GET /api/products
  alt cache hit
    Edge-->>User: 200 cached response
  else cache miss
    Edge->>Origin: GET /api/products
    Origin-->>Edge: 200 + Cache-Control
    Edge->>Edge: store with TTL
    Edge-->>User: 200 response
  end