Skip to content
TheChatArchiveThe Chat Archive
GLMGLM-5 / Pony Alpha· unknownHugging Face Frontier Reasoning Traces

Create a Python application that uses asyncio to handle thousands of concurrent web…

  1. User

    Create a Python application that uses asyncio to handle thousands of concurrent web requests.

  2. GLM Assistant

    1. Understand the Goal: The user wants a Python application that uses asyncio to handle "thousands" of concurrent web requests. This implies a need for high concurrency, non-blocking I/O, and likely a client or server architecture. Since the phrasing "handle... web requests" is slightly ambiguous (could mean making requests as a scraper/client or serving requests as a server), but usually "handle requests" suggests a server, while "make requests" suggests a client. However, often people ask this wanting to build a scraper (client side) or an API server (server side). I should probably provide a comprehensive answer that covers both or clarifies the distinction. Given the context of "thousands of concurrent... requests," it often leans towards an HTTP client (like scraping or load testing) because writing a raw asyncio server for thousands of connections is standard for frameworks like FastAPI/Sanic, but writing the client logic to manage that concurrency is the tricky part where asyncio shines.

    *Decision:* I will provide two examples:
    1.  **The Client (Requester):** A script that fetches data from many URLs concurrently (e.g., a scraper or load tester). This is the most common context for "handling thousands of requests" via `asyncio`.
    2.  **The Server (Handler):** A simple async web server (using `aiohttp` or just standard library `asyncio` streams) that can handle incoming connections.
    
    *Refinement:* The most valuable and distinct demonstration of `asyncio` power is usually the *client-side* semaphore/batching logic to prevent crashing the system or getting banned by the target server. I will focus primarily on the client side (fetching URLs) as the primary example, but briefly mention how to serve them.
    

    2. Key Concepts to Demonstrate: * async/await syntax. * aiohttp (the de-facto standard for async HTTP). * asyncio.gather or asyncio.wait vs. TaskGroup (Python 3.11+). * Rate Limiting/Semaphore: Crucial for "thousands" of requests so you don't run out of sockets or RAM. * Session Management: Reusing a ClientSession is vital for performance (keep-alive).

    1. Drafting the Client Solution (The Main Attraction):

      • Library: aiohttp.
      • Structure:
        • A worker function fetch(session, url).
        • A main coordinator bulk_fetch(urls).
      • Concurrency Control: Using asyncio.Semaphore. If we launch 10,000 requests at once without a semaphore, we might hit OS limits (file descriptors) or get banned.
      • Error Handling: try/except inside the fetch function.
    2. Drafting the Code (Mental or Scratchpad):

    import asyncio
    import aiohttp
    
    # Limit concurrency
    CONCURRENCY_LIMIT = 100
    
    async def fetch(session, url):
        try:
            async with session.get(url) as response:
                return await response.text()
        except Exception as e:
            return None
    
    async def main(urls):
        semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)
        async with aiohttp.ClientSession() as session:
            tasks = []
            for url in urls:
                # Wrap fetch with semaphore logic
                task = asyncio.create_task(bound_fetch(semaphore, session, url))
                tasks.append(task)
            results = await asyncio.gather(*tasks)
    
    1. Refining the Code for "Modern" Python:

      • Use aiohttp.
      • Include a context manager for the semaphore to ensure strict limiting.
      • Add timeouts (essential for network reliability).
      • Use asyncio.gather to collect results.
      • Provide a dummy list of URLs or a way to generate them for the example to run.
    2. Adding a Server Example (Briefly):

      • Just to be thorough, I'll include a tiny aiohttp server snippet to show how to receive requests asynchronously. This covers the ambiguity of "handle".
    3. Structuring the Explanation:

      • Prerequisites: Mention pip install aiohttp.
      • The "Client" Approach (Fetching Data): This is usually what people mean.
        • Explain why standard requests fails (blocking).
        • Explain the Semaphore (limiting open connections).
        • Provide the code.
      • The "Server" Approach (Serving API):
        • Briefly mention FastAPI or show a raw aiohttp server.
      • Best Practices:
        • Session reuse.
        • Timeouts.
        • Exception handling.
    4. Review against "Thousands" constraint:

      • Standard requests would time out or block.
      • Threads would hit GIL issues or context switching overhead at high counts (though threads can do it, async is cleaner).
      • asyncio is perfect here.
      • Self-Correction: I need to ensure I explicitly handle the "thousands" aspect. Launching 10,000 coroutines is fine in Python, but 10,000 simultaneous HTTP connections might hit OS ulimits. I must emphasize the Semaphore.
    5. Final Polish of the Code:

      • Add comments.
      • Make it runnable (include a if __name__ == "__main__": block).
      • Use dummy URLs (httpbin is good for testing) so the user can actually run it.
    6. Constructing the Output: (This aligns with the provided good response).

      • Title: Asynchronous Web Request Client.
      • Dependencies.
      • Code block with aiohttp and Semaphore.
      • Explanation of key components.
      • Server example (using aiohttp server or FastAPI reference).

      (Self-Correction during drafting the response): I'll stick to the Client (scraper/load tester) as the primary answer because "handling web requests" usually implies acting as the client consuming an API or scraping. If they meant a server, they usually say "web server". However, I will clarify this distinction. I will focus the code on fetching thousands of URLs efficiently.

      Code specific checks:

      • aiohttp.ClientSession() must be used in an async context.
      • asyncio.gather vs TaskGroup. TaskGroup is Python 3.11+. I should probably stick to gather for broader compatibility or mention 3.11 features. Let's stick to standard gather with a semaphore wrapper for clarity on the limit logic.

      Let's write the final response.

      • Step 1: Install. pip install aiohttp
      • Step 2: The Code.
        • Define a list of URLs.
        • Semaphore set to, say, 100 or 500 (don't do 5000 instantly).
        • fetch function with error handling.
        • main function managing the session.
      • Step 3: Explanation. Why use aiohttp? Why Semaphore? Why Session?

      Let's also briefly touch upon the Server side just in case. "If you meant creating a server to receive requests..."

      Okay, the plan is solid. I will implement a robust client script using aiohttp. Here is a complete, robust Python application designed to handle thousands of concurrent web requests.

    This solution uses aiohttp (the standard asynchronous HTTP client) and asyncio. It implements a Semaphore to limit concurrency (preventing your operating system or the target server from crashing due to overload) and connection pooling for efficiency.

    Prerequisites

    You will need to install aiohttp:

    pip install aiohttp
    

    The Application

    This script fetches a list of URLs asynchronously. I have included a "Safe Session" wrapper to handle common issues like timeouts and connection errors automatically.

    import asyncio
    import aiohttp
    import time
    from typing import List, Dict, Any
    
    # Configuration
    CONCURRENCY_LIMIT = 100  # Max simultaneous requests
    TIMEOUT_SECONDS = 10     # Timeout for each request
    
    async def fetch_url(
        session: aiohttp.ClientSession, 
        url: str, 
        semaphore: asyncio.Semaphore
    ) -> Dict[str, Any]:
        """
        Fetches a single URL with error handling and concurrency control.
        """
        async with semaphore:
            try:
                async with session.get(url, timeout=TIMEOUT_SECONDS) as response:
                    # Read the response text. 
                    # Note: For large files, consider streaming with 'chunked' instead of reading all at once.
                    data = await response.text()
                    return {
                        "url": url,
                        "status": response.status,
                        "data": data[:100], # Truncated for display
                        "error": None
                    }
            except asyncio.TimeoutError:
                return {"url": url, "status": 0, "data": None, "error": "Timeout"}
            except aiohttp.ClientError as e:
                return {"url": url, "status": 0, "data": None, "error": str(e)}
            except Exception as e:
                return {"url": url, "status": 0, "data": None, "error": f"Unexpected: {str(e)}"}
    
    async def bulk_fetch(urls: List[str]) -> List[Dict[str, Any]]:
        """
        Orchestrates the concurrent fetching of thousands of URLs.
        """
        # 1. Create a semaphore to limit concurrency. 
        # Without this, trying to open 10,000 sockets at once might crash the OS or ban your IP.
        semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)
        
        # 2. Configure the session with connection pooling.
        # Connector limit determines total open connections in the pool.
        connector = aiohttp.TCPConnector(limit=CONCURRENCY_LIMIT, force_close=True)
        
        results = []
        
        print(f"Starting fetch of {len(urls)} URLs with concurrency limit of {CONCURRENCY_LIMIT}...")
        start_time = time.time()
    
        async with aiohttp.ClientSession(connector=connector) as session:
            # 3. Create tasks for all URLs
            tasks = [fetch_url(session, url, semaphore) for url in urls]
            
            # 4. Run tasks concurrently
            # asyncio.gather runs them in parallel, but our semaphore inside fetch_url 
            # ensures we don't exceed CONCURRENCY_LIMIT active requests.
            results = await asyncio.gather(*tasks)
    
        end_time = time.time()
        print(f"Completed in {end_time - start_time:.2f} seconds.")
        
        return results
    
    def generate_test_urls(count: int) -> List[str]:
        """Generates dummy URLs for testing purposes."""
        # Using httpbin.org which is a standard service for testing HTTP requests
        return [f"https://httpbin.org/get?id={i}" for i in range(count)]
    
    # --- Main Entry Point ---
    
    async def main():
        # Generate 1000 test URLs
        urls_to_fetch = generate_test_urls(1000)
        
        # Run the bulk fetch operation
        results = await bulk_fetch(urls_to_fetch)
        
        # Analyze results
        success_count = sum(1 for r in results if r["status"] == 200)
        fail_count = len(results) - success_count
        
        print(f"\nResults Summary:")
        print(f"Success: {success_count}")
        print(f"Failures: {fail_count}")
        
        # Print a sample result
        if results:
            print("\nSample Result:")
            print(results[0])
    
    if __name__ == "__main__":
        # Run the async main loop
        asyncio.run(main())
    

    Key Concepts Explained

    1. asyncio.Semaphore(CONCURRENCY_LIMIT):

      • Why? If you try to launch 5,000 requests simultaneously without a limiter, you will likely hit "Too many open files" errors on your OS or trigger Denial-of-Service protections on the target server.
      • How it works: It acts like a counter. Only CONCURRENCY_LIMIT tasks can pass the async with semaphore: line at once. Others wait until a spot opens up.
    2. aiohttp.ClientSession:

      • Why? Creating a new HTTP connection for every request is slow (DNS lookup + TCP handshake + SSL handshake).
      • How it works: The session keeps a "pool" of open connections (managed by TCPConnector). When one request finishes, the connection is reused for the next URL if they share the same domain.
    3. asyncio.gather:

      • This function takes a list of coroutine objects (the tasks) and schedules them to run on the event loop.
    4. Timeouts:

      • In a synchronous script, a hung request blocks the entire program. In asyncio, a hung request just hangs that specific task. We explicitly set timeout so that unresponsive URLs don't leave connections open indefinitely.

    Server Side (If you meant receiving requests)

    If your goal was to create a Server that can handle thousands of incoming connections (an API), you should use a framework like FastAPI or Aiohttp (server mode).

    Here is a minimal example of an async server using aiohttp:

    from aiohttp import web
    
    async def handle(request):
        # Simulate async work (e.g., DB query)
        await asyncio.sleep(0.1) 
        return web.json_response({"status": "ok", "message": "Hello world"})
    
    app = web.Application()
    app.add_routes([web.get('/', handle)])
    
    if __name__ == '__main__':
        web.run_app(app, port=8080)
    

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.