Design a safe third-party API wrapper with retries, circuit breaks, and fallbacks.
- User
Design a safe third-party API wrapper with retries, circuit breaks, and fallbacks.
Assistant
1. Understand the Goal: The user wants a "safe third-party API wrapper" that includes three specific resilience patterns: * Retries: If a request fails, try again (usually with some delay/backoff). * Circuit Breaker: If the API is down or slow, stop sending requests for a while to avoid overwhelming it and wasting resources. * Fallbacks: If everything fails (retries exhausted, circuit open), return a default value or execute an alternative logic.
-
Identify the Key Components:
- Language/Environment: Python is the standard choice for these kinds of design patterns in backend engineering interviews/examples due to readability and library support. I will use Python.
- Core Logic: A class or function that wraps the actual HTTP request.
- Libraries vs. From Scratch:
- Option A (Libraries):
tenacity(retries),pybreaker(circuit breaker). Good for production, but maybe hides the "how it works" if the user wants to see the design. - Option B (From Scratch): Building it manually. Great for demonstrating understanding, but maybe buggy for real use.
- Option C (Hybrid): Implement the logic clearly but mention production-grade libraries.
- Option A (Libraries):
- Decision: I will provide a solution that uses standard logic (implemented from scratch for clarity) but mimics how robust libraries work. I will also briefly mention existing libraries at the end.
-
Drafting the Architecture:
-
The Wrapper Class:
SafeApiClient. -
Configuration: Max retries, backoff factor, circuit thresholds, fallback function.
-
Part 1: Retries (The "Transient Error" Handler)
- Logic: Try block -> Catch Exception -> Check if retryable -> Wait -> Retry.
- Backoff: Exponential backoff is best practice (wait 1s, then 2s, then 4s...).
- Max Attempts: Don't retry forever.
-
Part 2: Circuit Breaker (The "System Overload" Handler)
- States: CLOSED (normal), OPEN (failing, blocking requests), HALF-OPEN (testing the waters).
- Mechanism:
- Count failures.
- If failures > threshold -> OPEN.
- If OPEN -> Reject immediately (raise
CircuitBreakerError). - After timeout -> HALF-OPEN -> Allow one request.
- If success -> CLOSED. If fail -> OPEN.
-
Part 3: Fallbacks (The "Graceful Degradation" Handler)
- Logic:
try... except. - If
CircuitBreakerErrororMaxRetriesExceeded-> Return fallback data.
- Logic:
-
-
Refining the Implementation Details (Python):
-
Dependencies:
requests(standard HTTP),time(sleep),functools(decorators maybe?). -
Structure: I'll use a Decorator pattern. It's the cleanest way to wrap methods.
-
Wait, let's make it a class-based approach for the Circuit Breaker state to persist across calls.
-
Let's write the code mentally:
- Class
CircuitBreaker:__init__: failure_threshold, reset_timeout.call: wraps the function.- State tracking:
failure_count,last_failure_time,state.
- Class
RetryHandler:- Can be combined with the wrapper.
- Class
SafeApiWrapper:- Combines everything.
- Class
-
-
Step-by-Step Implementation Plan:
- Step 1: The Circuit Breaker Class. (Needs state management).
- Step 2: The Retry Logic. (Simple loop with
time.sleep). - Step 3: The Fallback Mechanism. (A simple callback or default value).
- Step 4: Putting it together. (The Client class).
-
Refining the Code (Self-Correction during drafting):
- Issue: If I write a full Circuit Breaker from scratch, the code might get long.
- Solution: Keep the CB logic simple but functional. Use an Enum for states.
- Issue: Thread safety?
- Solution: Note that for simplicity, this example is not thread-safe, but in production, you'd need locks (threading.Lock). I should mention this.
-
Writing the actual Python code (Mental Draft):
class CircuitBreaker: def __init__(self, max_failures, reset_timeout): ... def execute(self, func): if state == OPEN: check time... raise error try: result = func() self.success() return result except Exception as e: self.failure() raise e- Integrating Retries:
def request_with_retry(url): for i in range(max_retries): try: return circuit_breaker.execute(lambda: requests.get(url)) except Exception: sleep(...) raise MaxRetriesError -
Structuring the Explanation:
- Introduction: Briefly explain the three concepts (Retry, CB, Fallback).
- The Implementation (Python): Provide a complete, copy-pasteable, well-commented code snippet.
- Key Concepts Breakdown: specific sections explaining why the code is written that way (Exponential Backoff, Circuit States).
- Production Advice: Mention libraries like
tenacityorpybreakerbecause nobody should write this from scratch in a real enterprise app unless necessary.
-
Final Polish:
- Add type hinting.
- Ensure "Exponential Backoff" is used in retries (it's a crucial detail).
- Ensure the Fallback has access to the error or context if needed (simplified to just returning a value for this example).
-
Output Generation: (Proceeding to generate the response based on these steps). Designing a robust API wrapper requires handling three specific types of failure scenarios:
-
Transient Failures: Network blips or 503 Service Unavailable (handled by Retries).
-
System Overload: The API is down or responding slowly (handled by Circuit Breakers).
-
Total Failure: The API cannot be reached after all attempts (handled by Fallbacks).
Below is a comprehensive design pattern implemented in Python. While I recommend using battle-tested libraries like
tenacity(for retries) andpybreaker(for circuit breaking) in production, building this from scratch is the best way to understand the architecture.Architecture Overview
We will build a class
ResilientHttpClientthat composes these three patterns:- The Circuit Breaker (The Gatekeeper): Checks if the API is healthy. If not, it fails immediately.
- The Retry Strategy (The Persistent One): If the call fails, wait and try again (with Exponential Backoff).
- The Fallback (The Safety Net): If Retries fail or the Circuit is open, return a default value or cached data.
The Implementation
import time import random import logging from functools import wraps from enum import Enum # Configure logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) # --- 1. Circuit Breaker States --- class CircuitState(Enum): CLOSED = "CLOSED" # Everything is normal, requests flow through OPEN = "OPEN" # The API is down, requests are blocked immediately HALF_OPEN = "HALF_OPEN" # Testing if the API is back up # --- 2. Custom Exceptions --- class CircuitBreakerOpen(Exception): """Raised when the circuit is open to prevent API calls.""" pass class ApiError(Exception): """Represents an error from the API or Network.""" pass # --- 3. The Core Components --- class CircuitBreaker: def __init__(self, failure_threshold=5, recovery_timeout=30): self.failure_threshold = failure_threshold self.recovery_timeout = recovery_timeout self.failure_count = 0 self.last_failure_time = 0 self.state = CircuitState.CLOSED def _allow_request(self): if self.state == CircuitState.CLOSED: return True if self.state == CircuitState.OPEN: # Check if recovery time has passed if time.time() - self.last_failure_time >= self.recovery_timeout: self.state = CircuitState.HALF_OPEN logger.warning("Circuit Breaker: Entering HALF-OPEN state") return True return False # Half-open state allows one request through to test the waters return True def record_success(self): if self.state == CircuitState.HALF_OPEN: logger.info("Circuit Breaker: API recovered. Closing circuit.") self.state = CircuitState.CLOSED self.failure_count = 0 def record_failure(self): self.failure_count += 1 self.last_failure_time = time.time() if self.state == CircuitState.HALF_OPEN: logger.warning("Circuit Breaker: API still failing. Re-opening circuit.") self.state = CircuitState.OPEN elif self.failure_count >= self.failure_threshold: logger.error(f"Circuit Breaker: Threshold reached ({self.failure_count}). Opening circuit.") self.state = CircuitState.OPEN def retry_with_backoff(retries=3, base_delay=1, backoff_factor=2): """ Decorator for retrying function calls with exponential backoff. """ def decorator(func): @wraps(func) def wrapper(*args, **kwargs): for attempt in range(retries): try: return func(*args, **kwargs) except ApiError as e: wait_time = base_delay * (backoff_factor ** attempt) logger.warning(f"Retry {attempt + 1}/{retries}: Failed. Waiting {wait_time}s. Error: {e}") time.sleep(wait_time) # If all retries fail, raise the final exception raise ApiError("Max retries exceeded") return wrapper return decorator # --- 4. The Resilient Wrapper --- class ResilientApiClient: def __init__(self, circuit_breaker, fallback_function=None): self.circuit = circuit_breaker self.fallback = fallback_function def call(self, request_func, *args, **kwargs): # 1. Check Circuit Breaker if not self.circuit._allow_request(): logger.error("Circuit Breaker is OPEN. Blocking request.") return self._execute_fallback(*args, **kwargs) try: # 2. Execute Request (Internally handles Retries) # Note: We apply the retry logic specifically to the network call part response = self._execute_with_retry(request_func, *args, **kwargs) # 3. Success: Reset Circuit self.circuit.record_success() return response except ApiError as e: # 4. Failure: Update Circuit and run Fallback self.circuit.record_failure() return self._execute_fallback(*args, **kwargs) @retry_with_backoff(retries=3, base_delay=1) def _execute_with_retry(self, func, *args, **kwargs): try: # Simulate a network call result = func(*args, **kwargs) # Simulate checking HTTP status codes (e.g., 500, 503) # In a real app, check response.status_code return result except Exception as e: # Wrap generic exceptions in our custom ApiError raise ApiError(f"Network Error: {str(e)}") def _execute_fallback(self, *args, **kwargs): if self.fallback: logger.info("Executing Fallback strategy.") return self.fallback(*args, **kwargs) else: raise CircuitBreakerOpen("Service unavailable and no fallback defined.") # --- 5. Usage Example --- # A dummy function simulating an unreliable 3rd party API call_count = 0 def unreliable_api_call(user_id): global call_count call_count += 1 # Simulate the API being down for the first 5 calls if call_count <= 5: raise ConnectionError("Service Unavailable (503)") return {"data": f"User {user_id} details", "status": "ok"} # A fallback function (e.g., return cached data or default object) def get_cached_profile(user_id): return {"data": f"Cached User {user_id}", "status": "cached"} # Setup breaker = CircuitBreaker(failure_threshold=3, recovery_timeout=10) client = ResilientApiClient(circuit_breaker=breaker, fallback_function=get_cached_profile) # Run print("--- Starting Requests ---") for i in range(1, 8): print(f"\nRequest #{i}:") try: # We pass the function we want to call response = client.call(unreliable_api_call, user_id=123) print(f"Response: {response}") except Exception as e: print(f"Final Error: {e}") time.sleep(1) # Pause between requestsBreakdown of the Logic Flow
1. The Circuit Breaker (State Machine)
- Closed (Normal): Requests go through. We track failures. If failures exceed
failure_threshold, we switch to Open. - Open (Failing): Requests are blocked immediately. No network traffic is generated. This prevents your servers from waiting on timeouts (wasting thread resources) when the third party is down. After
recovery_timeout, we switch to Half-Open. - Half-Open (Testing): We allow one request through. If it succeeds, we assume the API is healed and go to Closed. If it fails, we go back to Open.
2. Retries with Exponential Backoff
In
_execute_with_retry, we catch errors and wait.- Exponential Backoff: We wait 1s, then 2s, then 4s. This prevents "thundering the herd"—if the API is overwhelmed, sending 100 retry requests instantly will crash it further.
- Idempotency: Warning: Only retry
GET,PUT, orDELETErequests. Never automatically retryPOSTwithout strict idempotency keys, or you risk charging a customer twice.
3. Fallback Strategy
In
_execute_fallback, we provide graceful degradation.- Cache: Return Redis/Memcached data (even if stale).
- Default Value: Return
None, an empty list, or a generic "Service Unavailable" object. - Queue: Push the payload into a message queue (Kafka/RabbitMQ) to process later when the API is back.
Production Recommendation
While the code above is great for understanding, for a real enterprise system, you should rely on established libraries that handle edge cases (thread safety, context managers) for you.
Python Libraries:
- Retries:
tenacity(highly recommended) orretrying. - Circuit Breaker:
pybreaker. - Comprehensive:
hyx(resilience primitives).
Example using
tenacityandpybreakerconcept:from tenacity import retry, stop_after_attempt, wait_exponential from pybreaker import CircuitBreaker # Define the breaker breaker = CircuitBreaker(fail_max=5, reset_timeout=60) # Define the retry logic @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=1, max=10)) def call_api(): # Logic here pass # Wrap them together def safe_call(): try: return breaker.call(call_api) except Exception: return "Fallback Data"-