Skip to content
TheChatArchiveThe Chat Archive
Grok 4.1 Fast· xaiHugging Face Frontier Reasoning Traces

Write a Python program to create an API gateway with rate limiting.

  1. User

    Write a Python program to create an API gateway with rate limiting.

  2. Assistant
    """
    Simple API Gateway with Rate Limiting using FastAPI and slowapi.
    
    Features:
    - Proxies all HTTP requests (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS) to a backend service.
    - Rate limiting: 5 requests per minute per IP address (in-memory storage for demo).
    - For demo, proxies to https://httpbin.org (echoes requests for testing).
    - Catch-all route for /{path:path}.
    
    Requirements:
    pip install fastapi uvicorn slowapi httpx
    
    Run:
    uvicorn main:app --host 0.0.0.0 --port 8000 --reload
    
    Test:
    # GET
    curl "http://localhost:8000/get?a=1"
    
    # POST
    curl -X POST http://localhost:8000/post -H "Content-Type: application/json" -d '{"test": "data"}'
    
    # Hit rate limit: make 6+ requests quickly
    for i in {1..6}; do curl "http://localhost:8000/get" & done
    
    Customize:
    - Change BACKEND_URL to your backend (e.g., "http://localhost:8001").
    - Adjust rate limit in @limiter.limit("5/minute").
    - For production: Use Redis storage_uri="redis://localhost:6379", add CORS, auth, logging, etc.
    """
    
    from fastapi import FastAPI, Request, Response
    from fastapi.responses import JSONResponse
    from slowapi import Limiter, _rate_limit_exceeded_handler
    from slowapi.util import get_remote_address
    from slowapi.errors import RateLimitExceeded
    import httpx
    from typing import Any
    
    # Rate limiter (in-memory for demo; use Redis for production)
    limiter = Limiter(key_func=get_remote_address, storage_uri="memory://")
    limiter.storage.max_bucket_size = 10000  # Prevent memory issues
    
    # Backend URL (change to your service)
    BACKEND_URL = "https://httpbin.org"
    
    app = FastAPI(title="API Gateway with Rate Limiting")
    app.state.limiter = limiter
    app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
    
    @app.get("/")
    async def root():
        """Health check endpoint (bypasses proxy)."""
        return {"message": "API Gateway is running!", "backend": BACKEND_URL, "rate_limit": "5/minute/IP"}
    
    @app.route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
    @limiter.limit("5/minute")  # Adjust as needed
    async def proxy(path: str, request: Request) -> Response:
        """
        Proxy all requests to backend.
        - Forwards method, path, query params, headers (minus host), body, cookies.
        - Rate limited per IP.
        """
        # Build backend URL
        query_string = str(request.query_params)
        url = f"{BACKEND_URL}/{path}" + (f"?{query_string}" if query_string else "")
    
        # Prepare headers (remove hop-by-hop)
        headers: dict[str, Any] = dict(request.headers)
        headers.pop("host", None)
        headers.pop("content-length", None)
        headers.pop("transfer-encoding", None)
        headers.pop("connection", None)
        headers.pop("upgrade", None)
    
        try:
            async with httpx.AsyncClient(follow_redirects=True, timeout=30.0) as client:
                resp = await client.request(
                    method=request.method,
                    url=url,
                    headers=headers,
                    params=request.query_params,  # httpx handles QueryParams
                    content=await request.body(),
                    cookies=request.cookies,
                )
                # Forward response
                return Response(
                    content=resp.content,
                    status_code=resp.status_code,
                    headers=dict(resp.headers),  # httpx Headers to dict
                    media_type=resp.headers.get("content-type"),
                )
        except httpx.RequestError as e:
            # Handle backend errors gracefully
            return JSONResponse(
                status_code=502,
                content={"error": "Backend request failed", "details": str(e)},
            )
        except Exception as e:
            return JSONResponse(
                status_code=500,
                content={"error": "Gateway error", "details": str(e)},
            )
    
    if __name__ == "__main__":
        import uvicorn
        uvicorn.run(app, host="0.0.0.0", port=8000)
    

    How to Use

    1. Save as main.py.
    2. Install deps: pip install fastapi uvicorn slowapi httpx.
    3. Run: uvicorn main:app --host 0.0.0.0 --port 8000 --reload.
    4. Open http://localhost:8000 (or http://localhost:8000/docs for Swagger UI).
    5. Test proxying and rate limiting as shown above.

    Notes

    • Rate Limiting: Enforced via IP (X-Forwarded-For if behind proxy; customize key_func). In-memory (resets on restart).
    • Production Tips:
      • Redis: storage_uri="redis://localhost:6379".
      • Add CORSMiddleware for browser access.
      • Auth/JWT validation before proxying.
      • Logging: Use structlog or middleware.
      • HTTPS: Use uvicorn with SSL or reverse proxy (nginx).
      • Circuit breaker: Integrate httpx retries or resilience4j-like.
    • This handles most cases; extend for WebSockets, streaming, or file uploads as needed.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.