Explain caching strategies: CDN, reverse proxy, application caches, database caches,…
- User
Explain caching strategies: CDN, reverse proxy, application caches, database caches, invalidation patterns, cache keys, TTLs, and consistency trade-offs; provide code/config sketches.
- Assistant
Considering consistency trade-offs
I'm thinking about the consistency trade-offs and how to mitigate them through strategies like locking, write-through, event bus, and read-your-writes. It feels important to create a brief checklist that assists in keeping code short yet meaningful. I want to ensure that my answer is informative while being succinct. I'll aim to format this using markdown, but I need to avoid emojis for clarity. Let's get started!
1) Layered caching (where each one fits)
Typical request path:
Client → CDN → Reverse proxy (edge/origin) → App → DB
Each layer caches different “shapes” of data:
- CDN: full HTTP responses (static assets; sometimes HTML/API)
- Reverse proxy (Nginx/Varnish): full HTTP responses close to your app; shields app from bursts
- Application cache (in-process / Redis): objects, computed results, partial fragments, authorization-aware data
- Database-side caching: pages/buffers, prepared statements, materialized views, ORM 2nd-level cache; sometimes query-result caching outside DB
Rule of thumb: cache as close to the client as possible if the response is safe to reuse across users and can tolerate staleness.
2) CDN caching (HTTP semantics + purge)
What you control
Primarily via response headers:
Cache-Control:public,private,max-age,s-maxage,no-store,stale-while-revalidate,stale-if-errorETag/Last-Modified: enables revalidation (304 responses)Vary: tells caches what request headers affect the response (e.g.,Accept-Encoding,Accept-Language)- Some CDNs support surrogate keys/tags for batch invalidation (Fastly/Cloudflare-style), e.g.
Surrogate-Key: product-123 category-9
Common patterns
Static assets (best case): versioned URLs + very long TTL
/app.4f3c2d.jscan be cached “forever”, because content changes imply a new URL.Cache-Control: public, max-age=31536000, immutableHTML/API: shorter TTL + revalidation / SWR (stale-while-revalidate)
Cache-Control: public, s-maxage=60, stale-while-revalidate=300, stale-if-error=86400 ETag: "v3:9caa1..." Vary: Accept-Encoding- Users get up to 60s-fresh content at edge.
- Up to 5 minutes stale is allowed while the CDN refreshes in background.
- If origin is down, serve stale for up to a day.
Purging / invalidation
- Purge by URL: simple, but expensive if many URLs.
- Purge by tag/surrogate key: best for “invalidate all pages related to product-123”.
Cloudflare purge example (sketch):
curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE/purge_cache" \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ --data '{"files":["https://example.com/api/products/123"]}'
3) Reverse proxy caching (Nginx / Varnish)
Reverse proxies cache origin responses and reduce app load. They’re also where you often implement “serve stale on error” reliably.
Nginx
proxy_cachesketchproxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:200m max_size=10g inactive=30m use_temp_path=off; map $http_authorization $bypass_cache { default 1; # auth => bypass by default "" 0; } server { location /api/ { proxy_pass http://app_upstream; proxy_cache api_cache; proxy_cache_methods GET HEAD; proxy_cache_bypass $bypass_cache; proxy_no_cache $bypass_cache; # Key design: include scheme/host/URI; include querystring; avoid cookies unless needed proxy_cache_key "$scheme://$host$request_uri"; # TTLs proxy_cache_valid 200 60s; proxy_cache_valid 404 10s; # negative caching proxy_cache_valid 500 502 503 504 0s; # Stale behaviors proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; add_header X-Cache-Status $upstream_cache_status always; } }Notes:
- Bypass caching when
Authorizationis present unless you have safe per-user caching. proxy_cache_use_stale ... updatinghelps with stampedes: one request refreshes; others get stale.
Varnish “grace” sketch (stale serving)
Varnish excels at stale-while-revalidate behavior:
sub vcl_backend_response { set beresp.ttl = 60s; set beresp.grace = 5m; # serve stale while fetching new }
4) Application caching (in-process + Redis/Memcached)
What you cache here
- Computations (pricing, recommendations, aggregates)
- Read models / DTOs
- Per-user or permissioned objects (not safe at CDN)
- “Fan-out” results (one expensive DB query feeding many requests)
Common patterns
Cache-aside (lazy loading) is most common:
- Read: check cache → on miss, fetch DB → populate cache
- Write: update DB → invalidate cache key(s)
Read-through: cache library loads on miss (less common but nice abstraction)
Write-through: write cache + DB in one flow (simplifies reads; can increase write latency)
Write-behind: write cache first; flush to DB async (fast but riskier)
Redis cache-aside with stampede protection (Node/TypeScript sketch)
import Redis from "ioredis"; const redis = new Redis(process.env.REDIS_URL); async function getProduct(productId: string) { const key = `product:v3:${productId}`; // versioned key namespace const lockKey = `${key}:lock`; const cached = await redis.get(key); if (cached) return JSON.parse(cached); // Basic lock to prevent dogpile const gotLock = await redis.set(lockKey, "1", "NX", "PX", 3000); if (!gotLock) { // Someone else is refreshing; brief wait then retry cache await new Promise(r => setTimeout(r, 50)); const retry = await redis.get(key); if (retry) return JSON.parse(retry); } try { const product = await db.products.findById(productId); // TTL + jitter to avoid synchronized expiry const ttlSec = 60 + Math.floor(Math.random() * 30); await redis.set(key, JSON.stringify(product), "EX", ttlSec); return product; } finally { if (gotLock) await redis.del(lockKey); } }Add-ons commonly needed:
- Negative caching (cache “not found” briefly)
- Soft TTL vs hard TTL (serve stale while refreshing in background)
- Jitter to avoid many keys expiring at once
- Request coalescing (locks or single-flight)
In-process cache (fast, but per-instance)
Good for tiny hot sets; beware multiple app instances (each has its own cache).
5) Database caching (what’s real vs what people assume)
Built-in DB caching
Most serious DBs already cache data pages in memory (buffer pool). You usually don’t “configure query-result caching” anymore (e.g., MySQL Query Cache is removed).
Instead you optimize by:
- Indexing
- Reducing round trips
- Using prepared statements/connection pooling
- Adding read replicas
- Avoiding N+1 queries
Materialized views / derived tables (explicit caching in DB)
Useful for expensive aggregates that don’t need real-time freshness.
Postgres materialized view sketch:
CREATE MATERIALIZED VIEW product_stats AS SELECT product_id, count(*) AS orders, sum(total) AS revenue FROM orders GROUP BY product_id; -- Refresh periodically (or event-driven), can be concurrent REFRESH MATERIALIZED VIEW CONCURRENTLY product_stats;ORM 2nd-level cache
Can help for read-mostly entities, but invalidation complexity rises quickly. Prefer explicit app-level caching for clarity unless your domain is very stable.
6) Invalidation patterns (the hardest part)
A) TTL-only (time-based)
- Simplest, most robust operationally
- Accepts staleness window
- Works great with SWR/grace
Use when “eventual consistency is fine”.
B) Explicit delete/invalidate on write
After updating DB, delete cache keys that might be affected.
Pros: fresher
Cons: hard when one write affects many keys (lists, aggregates, search pages)C) Versioned keys (avoid deletes)
Instead of deleting many keys, bump a version:
product:v3:123→product:v4:123- Old keys expire naturally
You store “current version” in config or a small metadata key.
D) Tag-based invalidation (surrogate keys)
Attach tags to responses/entries and purge by tag:
- Great for CDNs/reverse proxies that support it
- Requires discipline in tagging
E) Event-driven invalidation
Publish domain events on writes (e.g.,
ProductUpdated(123)), consumers invalidate/update caches.Pros: scalable, decoupled
Cons: eventual; needs retries/ordering handling
7) Cache keys (design rules)
A cache key must capture everything that changes the response:
- Resource identity:
productId=123 - Representation:
v3(schema/serialization version) - Locale/currency:
en-US,USD - Permissions/identity if relevant:
userIdor “role bucket” - Query params that affect output (but avoid including tracking params)
Examples:
product:v3:123:locale=en-US:cur=USDsearch:v2:q=shoes:sort=price_asc:filters=...(often hashed due to length)
Avoid high-cardinality explosions:
- Don’t include volatile headers/cookies unless required
- Normalize query params (sorted, remove irrelevant ones)
- Consider bucketing (e.g., “anon vs logged-in”, “plan tier”) instead of per-user, when safe
8) TTLs and freshness controls
TTL selection heuristics
- Immutable assets: 1 year + versioned filenames
- Reference data (categories, config): minutes–hours
- Product pages: 30–300s + SWR/grace
- Highly dynamic (inventory, balances): seconds or avoid caching; prefer conditional requests or partial caching
Soft TTL + hard TTL (common in app caches)
- Soft TTL: after this, serve cached but refresh async
- Hard TTL: beyond this, must block and refresh (or fail)
This reduces tail latency while still converging to fresh data.
9) Consistency trade-offs (what you’re choosing)
Caching typically trades freshness for latency/cost/availability.
Stronger consistency (fresher)
- Short TTLs
- Explicit invalidation on every write
- Write-through caching
- Bypass caches for sensitive endpoints
- “Read-your-writes” techniques (session affinity, per-user bypass after write)
Costs:
- More origin load
- More invalidation complexity
- Higher chance of stampedes without protection
Weaker consistency (eventual)
- Longer TTLs
- SWR/grace
- Background refresh
- Versioned keys
Benefits:
- Excellent performance and resilience
- Simpler ops (TTL-based is hardest to break)
Common mitigations
- Cache stampede protection: locks, single-flight, request coalescing
- Serve stale on error: CDN
stale-if-error, Nginx/Varnish stale configs - Jitter TTLs to avoid synchronized expiry
- Negative caching for 404/empty results (short TTL)
10) Quick “good default” stack (practical baseline)
- CDN: cache static assets aggressively; API only if responses are truly public and vary safely.
- Reverse proxy: cache GETs for anonymous/public endpoints; enable stale-on-error + updating.
- App + Redis: cache expensive reads with cache-aside + jitter + stampede locks.
- DB: rely on buffer cache; use materialized views for heavy aggregates; add read replicas when needed.
- Invalidation: start with TTL + versioned keys; add targeted explicit invalidation for the few hottest/most visible objects.
If you share your stack (Cloudflare/Fastly/CloudFront, Nginx/Varnish, language/framework, Redis/Memcached, DB), I can tailor the snippets to your exact components and show a coherent key/TTL/invalidation scheme for a couple of concrete endpoints.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.