Design a request dedupe middleware guarding idempotent operations with hashed payloads…
- User
Design a request dedupe middleware guarding idempotent operations with hashed payloads and sliding TTL windows.
Assistant
The user wants a request deduplication middleware. This middleware should guard idempotent operations. It needs to use hashed payloads to identify requests. It should use sliding TTL windows to manage the lifecycle of these deduplication records.
This is a classic system design/coding interview style question or a real-world architectural requirement. I need to provide a design and a concrete implementation.
Key Components:
- Middleware Interface: How does it plug into the application? (e.g., Express.js, generic function wrapper).
- Hashing: How to generate a unique key for the request?
- Method + Path + Body (payload) + Headers (optional, but maybe specific ones).
- Hash algorithm: SHA-256 is standard.
- Storage: Where to keep the keys?
- In-memory (Map) for single instance.
- Redis for distributed systems (production standard).
- I should probably provide a storage adapter pattern so both can be demonstrated or swapped.
- Sliding TTL Window:
- Standard TTL: Key expires after seconds.
- Sliding Window behavior: If a request comes in during processing, do we extend the lock? Or does the lock simply ensure that for seconds, the operation isn't repeated?
- Interpretation of "Sliding TTL" in Dedupe context:
- Scenario A (Standard): Request comes in. We set a key with TTL. If another comes in within TTL, block it. Key expires after static time.
- Scenario B (Renewable): Long-running process. Request comes in. We set a key. If the processing takes longer than the TTL, we might want to refresh it (heartbeat) so the lock doesn't expire prematurely.
- Scenario C (Idempotency Key): Client sends an
Idempotency-Key. Server stores result. Next time same key comes, return stored result.
- The prompt says "hashed payloads" implies the server generates the fingerprint, not the client sending a key.
- The "Sliding TTL" likely refers to resetting the expiration timer every time the same request is attempted (keeping the "busy" state alive), or simply that the record exists for a fixed window starting from the last attempt.
- Let's assume the standard definition of a sliding window in rate limiting/dedupe: The window starts from the last request. But for deduplication, usually, you want to block until the first operation is done.
- Actually, usually dedupe implies: "Is this exact request currently being processed or recently processed?"
- Let's stick to: If a request is in flight (or completed within cooldown), block subsequent identical requests.
Design Decisions:
- Language: TypeScript/Node.js is the most common context for "middleware". Python/FastAPI is also good. I'll stick to TypeScript/Node.js using a generic interface or Express-like structure for clarity.
- Hashing:
crypto.createHash('sha256'). - Storage: Redis is the best example because it handles TTL natively and is distributed. In-memory is easier for code snippets but dangerous in production. I will define a
Storeinterface. - Sliding TTL Logic:
-
When a request arrives: Hash the payload.
-
Check Store.
-
If exists -> Return 409 Conflict or 429 Too Many Requests (or a specific "Deduplicated" response).
-
If not exists -> Set key with TTL -> Process -> Delete key (optional, if we want to cache the result, but prompt says "guarding operations", implying preventing double execution).
-
Sliding aspect: If the request comes in again while processing, do we reject? Yes. Do we update the TTL? Usually no, that extends the lock. Sliding TTL usually applies to "Rate Limiting".
-
Re-reading prompt: "Sliding TTL windows". Maybe the user means: If I keep spamming the same request, the block lasts longer? Or does it mean standard expiration?
-
Let's interpret "Sliding TTL" as: The record of the "fingerprint" has an expiration. If the same request is detected again, the logic depends on whether the previous one finished.
-
Actually, maybe the user implies: "Don't execute if executed within the last seconds."
-
Let's build a system that:
- Generates Hash.
- Attempts to acquire a "lock" (SETNX).
- If lock acquired: Run handler. On finish, keep result for
TTLtime (sliding/expires). Or just delete lock? The prompt says "guarding idempotent operations". - If lock exists (processing or recently finished): Return cached result or error.
-
Refined interpretation: The goal is to prevent thundering herds or double-clicks.
-
Hash = Identity.
-
Sliding TTL = If the operation is long-running, perhaps the lock needs refreshing? No, standard usage is just an expiration time.
-
Let's implement a Fingerprint-based Deduper.
-
Step-by-Step Implementation Plan:
- Hash Generator: Combine Method + URL + Body (normalized).
- Storage Interface:
get(key),set(key, val, ttl). - Middleware Logic:
- Intercept request.
- Calculate Fingerprint.
- Check Storage.
- Hit: If status is
PROCESSING, return 202 Accepted or 429. If status isCOMPLETED(and within TTL), return cached response (true idempotency). - Miss: Set status
PROCESSINGwith TTL. Next() -> handle response -> Set statusCOMPLETEDwith TTL.
Handling the "Sliding" aspect specifically: In a strict dedupe context, a "sliding window" usually isn't the right model (that's for rate limiting). However, if the user insists on "Sliding TTL", it might mean that the validity of the "completed" state resets if the client asks for it again? Interpretation: "Cache the result of this operation for X seconds. If the user requests it again, return cached result and reset the expiration timer." This effectively keeps popular idempotent results in memory longer.
Let's go with:
- Dedupe while processing: Block concurrent identical requests.
- Cache result after processing: Return the same response for a period.
- Sliding TTL: If the cached result is accessed, the TTL is reset (sliding).
Code Structure (TypeScript):
DedupeStore: Interface (Redis implementation).generateFingerprint(req): Helper.dedupeMiddleware(options): The factory.
Edge Cases:
- Large payloads (stream hashing vs buffering). Buffering is easier for a demo, mention streaming for prod.
- JSON key ordering (sort keys before hashing).
- File uploads (ignore body or hash stream? - better to scope to JSON payloads for this design).
Refining the Middleware Logic (The "Guard"):
// Conceptual flow const fingerprint = hash(method, path, sortedBody); // 1. Check if result is cached (Sliding TTL here) if (await store.exists(fingerprint)) { const result = await store.get(fingerprint); await store.expire(fingerprint, ttl); // Slide the window! return res.send(result); } // 2. Check if currently processing (Thundering Herd protection) if (await lockStore.isLocked(fingerprint)) { return res.status(429).send("Processing"); } // 3. Acquire Lock await lockStore.lock(fingerprint, lockTTL); // 4. Execute try { await next(); // Cache result await store.set(fingerprint, res.body, resultTTL); } finally { await lockStore.unlock(fingerprint); }Wait, the prompt specifically mentions "Sliding TTL Windows". In standard Redis implementations,
EXPIREresets the TTL. That creates a sliding window. So, if I make a request, result is cached for 1 min. If I ask again at 50s, it slides to another 1 min.Let's draft the solution.
- Architecture Diagram (Mental): Client -> Middleware -> Hash -> Redis (Check/Set) -> Handler.
- Components:
RequestNormalizer: Ensures{a:1, b:2}hashes same as{b:2, a:1}.Hasher: SHA256.DedupeGuard: The middleware class.
- Language: Node.js/TypeScript is best for "middleware".
Specific nuances to address:
- Payload Hashing: Need to handle JSON body parsing. Since middleware runs before body parser usually, or depends on it. I will assume
req.bodyis populated. - Sliding TTL Logic:
- When retrieving a cached response: Update the TTL (make it slide).
- This ensures frequently accessed idempotent data stays cached.
Self-Correction/Refinement: Is it safer to just provide a locking mechanism, or a caching mechanism? "Guarding idempotent operations" implies both:
- Don't run the same heavy operation twice simultaneously.
- Don't run it again if we just did it.
I will implement a Redis-based solution using
ioredisstyle syntax (pseudocode or specific) to demonstrate atomic operations. Lua scripts are best for atomicity (check-and-set), but standard commands might suffice for a design explanation.Let's stick to a clean, generic TypeScript implementation using a generic
Storeinterface, with a Redis implementation example.Structure of the response:
- High-Level Design: Explain the flow (Fingerprint -> Check Cache -> Check Lock -> Execute).
- Key Logic: Normalization, Sliding TTL (Refresh on Access).
- Implementation: TypeScript code.
Fingerprintfunction.Storeinterface.Middlewarefunction.
- Configuration: TTLs, Error handling.
- Production Considerations: Lua scripts, Body size limits.
Drafting the Code:
import crypto from 'crypto'; // Configuration interface DedupeOptions { ttl: number; // Sliding window in seconds store: Store; excludeHeaders?: string[]; } // The Middleware export const dedupe = (options: DedupeOptions) => async (req, res, next) => { // 1. Hashing const key = generateKey(req); // 2. Sliding Window Check (Read Cache) // Try to get cached result. const cached = await options.store.get(key); if (cached) { // SLIDE the TTL await options.store.expire(key, options.ttl); return res.send(cached); // Return 200 with previous result } // 3. Concurrency Lock (Optional but recommended for "Guarding") // If not cached, ensure we aren't currently processing one. // Using a separate lock key or logic. // For simplicity in this prompt, I might stick to just "Dedupe by Result Caching" vs "Dedupe by Locking". // But "Guarding" implies preventing execution. // If I assume the operation is idempotent, caching the result is the strongest form of guarding. // However, if the operation is *running*, we shouldn't run it again. // Let's assume a "pending" state. const currentStatus = await options.store.getStatus(key); if (currentStatus === 'pending') { return res.status(429).send('Request is being processed'); } // Set pending await options.store.set(key, 'pending', options.ttl); // 4. Process and Save // We need to intercept the response to cache it. // This is tricky in Node middleware. We need to monkey-patch res.send or use a buffer. const originalSend = res.send; res.send = async (body) => { if (res.statusCode < 400) { // Only cache success await options.store.set(key, body, options.ttl); } originalSend.call(res, body); }; next(); }This looks solid. I will refine the "Sliding" part to emphasize that the cache duration resets on retrieval.
Detailed Design Requirements:
- Idempotency: Exact same payload = Exact same response (cached).
- Hashing: SHA-256 of
Method + Path + SortedBody. - Sliding TTL: Accessing the cached response resets its expiration time.
- Middleware: Intercepts request.
I will provide a robust Node.js/Express implementation as it's the industry standard for this terminology.
Wait, one corner case: What if the request fails? Should we cache the error? Usually no. We should probably clear the lock. Also, large payloads. I'll mention that payloads should be stable serialized JSON.
Let's write the output. Structure:
- System Design & Strategy.
- The Algorithm (Sliding TTL logic).
- Implementation (TypeScript).
- Redis Lua Script (for atomicity - bonus points).
- Usage example.
I'll focus on the Sliding Window definition: "If I make a request, it is cached for 10s. If I make the same request at 9s, the cache is now valid for another 10s (total 19s from start)."
One tricky part: Request Body Parsing. In Express,
body-parserpopulatesreq.body. The middleware must run after body parsing.Let's proceed. Here is a design and implementation for a Request Deduplication Middleware. This solution uses Node.js/Express for the implementation context, but the architectural patterns (Sliding TTL, Content Hashing, Locking) are applicable to any backend stack (Go, Python, Java).
1. Architectural Design
The middleware acts as a Guard, ensuring that identical requests processed within a specific time window do not result in duplicate state changes or expensive computations.
Core Components
- Fingerprint Generator: Normalizes the request (method, path, authenticated user ID) and hashes the payload (body) to create a unique key.
- Dedupe Store (Cache): A high-speed, persistent store (Redis) used to track "In-Flight" requests and "Cached Responses".
- Sliding TTL Logic: Instead of a fixed window, the TTL (Time To Live) is refreshed or extended based on activity, ensuring long-running operations don't expire prematurely while preventing stale locks.
The "Sliding TTL" Strategy
For this design, "Sliding TTL" refers to the lifecycle of the lock:
- Acquisition: A lock is set with an initial TTL (e.g., 10s).
- Execution: If the request takes longer than the TTL, we don't want the lock to expire (allowing a duplicate request to sneak in).
- Heartbeat: The middleware extends the TTL periodically while the downstream handler is processing.
- Completion: The lock is released, but the Result is cached with its own fixed TTL (short-term idempotency window).
2. Implementation (TypeScript & Redis)
We will use Redis for the store because it supports atomic operations and TTL management natively.
A. Utility: Request Fingerprinting
We need to ensure that
{ "a": 1, "b": 2 }is treated the same as{ "b": 2, "a": 1 }.import crypto from 'crypto'; /** * Generates a unique hash for the request. * Includes User ID to ensure deduplication is scoped to the actor. */ export function generateRequestHash(req: any): string { // 1. Identify the actor (prevent cross-user deduping) const userId = req.user?.id || 'anonymous'; // 2. Normalize the route (e.g., /api/orders/5) const route = req.originalUrl || req.url; // 3. Normalize Payload (Body) // Sort keys to ensure JSON object order doesn't change the hash const bodyString = req.body ? JSON.stringify(req.body, Object.keys(req.body).sort()) : ''; // 4. Create Hash const content = `${userId}:${route}:${bodyString}`; return crypto.createHash('sha256').update(content).digest('hex'); }B. The Middleware Class
This class handles the locking mechanism and the "sliding" window via a heartbeat.
import { Request, Response, NextFunction } from 'express'; import Redis from 'ioredis'; // Configuration interface DedupeOptions { redisClient: Redis; lockTTLSeconds?: number; // How long to lock while processing (sliding) responseTTLSeconds?: number; // How long to cache the success response } export class RequestDedupeMiddleware { private redis: Redis; private lockTTL: number; private responseTTL: number; constructor(options: DedupeOptions) { this.redis = options.redisClient; this.lockTTL = options.lockTTLSeconds || 10; // Default 10s sliding window this.responseTTL = options.responseTTLSeconds || 60; // Cache response for 60s } public handle = async (req: Request, res: Response, next: NextFunction) => { const requestHash = generateRequestHash(req); const lockKey = `dedupe:lock:${requestHash}`; const cacheKey = `dedupe:res:${requestHash}`; try { // 1. CHECK CACHE: Has this request been completed recently? const cachedResponse = await this.redis.get(cacheKey); if (cachedResponse) { console.log(`[Dedupe] Cache Hit: ${requestHash}`); const parsed = JSON.parse(cachedResponse); return res.status(parsed.status).json(parsed.body); } // 2. ACQUIRE LOCK: Prevent duplicate processing // NX = Only set if not exists, EX = Set TTL const acquired = await this.redis.set(lockKey, 'processing', 'EX', this.lockTTL, 'NX'); if (!acquired) { // Lock exists. This implies a request is currently processing. // For strict idempotency, we can wait or reject. // We reject with 429 to prevent thundering herd logic tight-looping. return res.status(429).json({ error: 'Duplicate request detected', message: 'A request with this payload is currently being processed.' }); } // 3. SETUP SLIDING TTL (Heartbeat) // If processing takes longer than 10s, we need to keep the lock alive. const heartbeatInterval = setInterval(async () => { // Extend the lock TTL by sliding it forward await this.redis.expire(lockKey, this.lockTTL); }, (this.lockTTL / 2) * 1000); // Slide at half the TTL interval // 4. INTERCEPT RESPONSE TO CACHE // We monkey-patch res.json to capture the output before sending it const originalJson = res.json.bind(res); res.json = async (body: any) => { // Stop heartbeat clearInterval(heartbeatInterval); // Only cache successful operations (2xx) if (res.statusCode >= 200 && res.statusCode < 300) { const payload = { status: res.statusCode, body }; await this.redis.set(cacheKey, JSON.stringify(payload), 'EX', this.responseTTL); } // Release lock immediately after processing await this.redis.del(lockKey); // Send actual response return originalJson(body); }; // 5. PROCEED TO HANDLER next(); } catch (error) { next(error); // Pass errors to Express error handler } } }C. Usage in an Express Application
Here is how you would wire this up in a standard server setup.
import express from 'express'; import Redis from 'ioredis'; import { RequestDedupeMiddleware } from './middleware'; const app = express(); app.use(express.json()); // Body parsing must happen before dedupe // Connect to Redis const redis = new Redis({ host: 'localhost', port: 6379 }); // Initialize Middleware const dedupe = new RequestDedupeMiddleware({ redisClient: redis, lockTTLSeconds: 15, // Sliding window duration responseTTLSeconds: 30 // Idempotency key retention }); // Apply to specific idempotent routes (e.g., POST /orders) app.post('/orders', dedupe.handle, async (req, res) => { // Simulate long processing await new Promise(r => setTimeout(r, 5000)); res.status(201).json({ orderId: 123, status: 'created' }); }); app.listen(3000, () => console.log('Server running'));3. Algorithm Deep Dive
The Sliding Window (Heartbeat)
The critical part of this design is handling requests that take longer to process than the initial safety timeout.
- T=0: Request arrives. Redis sets
lockKeywith TTL=10s. - T=5s: The heartbeat logic (running every 5s) triggers
EXPIRE lockKey 10. - T=10s: Instead of expiring, the key now expires at T=15s.
- T=12s: Processing finishes. The
res.jsonoverride fires, caching the result and deletinglockKey.
If a duplicate request arrives at T=2s:
- It tries
SET lockKey NX. - It fails (because the original request holds it).
- It receives
429 Conflict.
Hashing Strategy
We use SHA-256 for collision resistance. To make the hash robust:
- Scope by User:
user_idis part of the hash input. Admin deleting resource ID 5 should not block User A from deleting resource ID 5 (if business logic allows), or vice versa. - Sorted JSON:
JSON.stringifyorder is not guaranteed in some environments. Sorting keys ensures{"id":1,"val":2}generates the same hash as{"val":2,"id":1}.
4. Edge Cases & Considerations
- Memory Pressure: If payloads are massive, hashing every request adds CPU overhead. If the store fills up with cached responses, it can strain Redis. The
responseTTLshould be kept short (e.g., 60s) strictly for network retry safety, not as a long-term cache. - Error Handling: The current implementation only caches
2xxresponses. If a request fails validation (400) or server error (500), we generally want the client to be able to retry immediately. Therefore, we release the lock but do not cache the error response. - Concurrency: In a distributed system (multiple pods), relying on Redis
SET NXis atomic and thread-safe. - Stuck Locks: If the Node process crashes hard (segfault) before
res.jsonis called and before thefinallyblock or close hooks run, the lock will persist until the Redis TTL expires. This is desired behavior (safety valve), causing subsequent requests to 429 until the TTL clears.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.