Create a Python asyncio application with a bounded semaphore limiting concurrent…
- User
Create a Python asyncio application with a bounded semaphore limiting concurrent outgoing requests to 100. Handle task cancellation properly so that when the user hits Ctrl+C, all sockets close and pending writes flush before exiting.
Assistant
This is a great question about creating a robust asyncio application with proper resource management and graceful shutdown. Let me create a comprehensive example that demonstrates:
- A bounded semaphore to limit concurrent requests
- Proper task cancellation handling
- Graceful shutdown with socket cleanup
- Proper flushing of pending writes
- Signal handling for Ctrl+C (SIGINT)
Here's what I'll include:
- A connection pool or request manager
- Semaphore for limiting concurrency
- HTTP client as an example (using aiohttp or asyncio)
- Proper exception handling for cancellation
- Context managers for resource cleanup
- Signal handlers for graceful shutdown
- A demo showing the functionality
Let me create a comprehensive example:
import asyncio import signal import sys from typing import Optional, List from contextlib import asynccontextmanager import logging # Set up logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s' ) logger = logging.getLogger(__name__) class BoundedRequestManager: """Manages concurrent requests with a bounded semaphore.""" def __init__(self, max_concurrent: int = 100): self.semaphore = asyncio.Semaphore(max_concurrent) self.active_tasks: List[asyncio.Task] = [] self._lock = asyncio.Lock() self.shutdown_event = asyncio.Event() async def add_task(self, coro): """Add a task to be managed and tracked.""" task = asyncio.create_task(coro) async with self._lock: self.active_tasks.append(task) # Remove task from active list when done task.add_done_callback(lambda t: self._remove_task(t)) return task def _remove_task(self, task: asyncio.Task): """Remove completed task from tracking.""" try: self.active_tasks.remove(task) except ValueError: pass async def execute(self, coro): """Execute a coroutine with semaphore protection.""" async with self.semaphore: try: return await coro except asyncio.CancelledError: logger.info("Task cancelled") raise async def graceful_shutdown(self, timeout: int = 30): """Gracefully shutdown all pending tasks, cancelling any that exceed the timeout.""" logger.info(f"Starting graceful shutdown with {timeout}s timeout") # Cancel all active tasks for task in self.active_tasks: if not task.done(): task.cancel() # Wait for all tasks to complete or timeout try: await asyncio.wait_for( asyncio.gather(*self.active_tasks, return_exceptions=True), timeout=timeout ) except asyncio.TimeoutError: logger.warning(f"Shutdown timeout exceeded, forcing termination") for task in self.active_tasks: if not task.done(): task.cancel() logger.info("Shutdown complete")The manager tracks active tasks and uses a semaphore to enforce concurrency limits. When shutting down, it cancels all pending work and waits for completion within a specified timeout before forcing termination. Simulate I/O delays. When writing, the connection extends its buffer, increments a counter, and pauses briefly. The flush method clears buffered data after a simulated delay, while close ensures any pending writes are flushed before marking the connection as closed, with error handling to catch flush failures.
A connection pool manages multiple MockConnection instances in a queue, allowing them to be checked out and returned. During initialization, all connections are added to the queue. The get_connection context manager retrieves a connection and guarantees it's returned to the pool afterward. When shutting down, close_all iterates through each connection, attempting to close it gracefully and logging any errors that occur.
The RequestHandler class uses both a BoundedRequestManager and ConnectionPool to execute requests with proper resource management. When making a request, it acquires a connection from the pool and checks whether a shutdown has been signaled before proceeding. If shutdown is active, the request is abandoned with a warning. Otherwise, the request begins processing with a simulated delay, then prepares data to write to the connection.
The handler encodes the provided data or generates a default message, writes it to the connection, and allows additional processing time before flushing. On successful completion, it returns a response. If the request gets cancelled, it attempts to flush the connection before re-raising the cancellation error. Any other exceptions are logged and propagated.
The Application class ties these components together, initializing a BoundedRequestManager with a concurrency limit, a ConnectionPool with a specified size, and a RequestHandler to coordinate between them. = True
async def initialize(self): """Initialize the application.""" logger.info("Initializing application") await self.pool.initialize() self._setup_signal_handlers() def _setup_signal_handlers(self): """Setup signal handlers for graceful shutdown.""" def signal_handler(signum, frame): logger.info(f"Received signal {signum}") self.running = False # Schedule graceful shutdown asyncio.create_task(self.shutdown()) signal.signal(signal.SIGINT, signal_handler) signal.signal(signal.SIGTERM, signal_handler) async def make_requests(self, num_requests: int): """Make multiple concurrent requests.""" logger.info(f"Starting {num_requests} requests") tasks = [] for i in range(num_requests): if not self.running: break coro = self.handler.make_request(i) task = await self.request_manager.add_task( self.request_manager.execute(coro) ) tasks.append(task) # Add small delay to avoid overwhelming await asyncio.sleep(0.01) # Wait for all tasks to complete try: results = await asyncio.gather(*tasks, return_exceptions=True) logger.info(f"All requests completed: {len(results)} total") # Log any exceptions for i, result in enumerate(results): if isinstance(result, Exception): logger.error(f"Request {i} failed: {result}") except asyncio.CancelledError: logger.warning("make_requests was cancelled") raise async def shutdown(self): """Gracefully shutdown the application.""" logger.info("Starting shutdown sequence") self.running = False # Gracefully shutdown request manager await self.request_manager.graceful_shutdown(timeout=30) await self.pool.close() logger.info("Shutdown complete") async def run(self, num_requests: int = 200): """Run the application.""" try: await self.initialize() await self.make_requests(num_requests) except KeyboardInterrupt: logger.info("KeyboardInterrupt received") except asyncio.CancelledError: logger.info("Task cancelled") finally: await self.shutdown()async def main(): """Main entry point.""" app = Application(max_concurrent=50, pool_size=10)
try: main_task = asyncio.create_task(app.run(num_requests=200)) await main_task except KeyboardInterrupt: logger.info("Caught KeyboardInterrupt in main") except Exception as e: logger.error(f"Unexpected error in main: {e}", exc_info=True)if name == "main": try: asyncio.run(main()) except KeyboardInterrupt: logger.info("Application interrupted by user") except Exception as e: logger.error(f"Fatal error: {e}", exc_info=True) sys.exit(1)
Asyncio Application with Bounded Semaphore and Graceful Shutdown
Here's a comprehensive example with proper resource management and cancellation handling:
import asyncio import signal import sys import logging from typing import List, Optional, Dict, Any from contextlib import asynccontextmanager from dataclasses import dataclass from datetime import datetime # Configure logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) logger = logging.getLogger(__name__) @dataclass class RequestStats: """Statistics for a request.""" request_id: int start_time: datetime end_time: Optional[datetime] = None success: bool = False error: Optional[str] = None @property def duration(self) -> float: if self.end_time: return (self.end_time - self.start_time).total_seconds() return 0 class MockConnection: """Mock connection simulating socket I/O.""" def __init__(self, conn_id: int, write_buffer_size: int = 4096): self.conn_id = conn_id self.write_buffer = bytearray() self.write_buffer_size = write_buffer_size self.closed = False self.bytes_written = 0 self.bytes_flushed = 0 async def write(self, data: bytes) -> int: """Write data to buffer, simulating socket write.""" if self.closed: raise RuntimeError(f"Connection {self.conn_id} is closed") # Simulate socket I/O delay await asyncio.sleep(0.001) self.write_buffer.extend(data) bytes_to_buffer = len(data) self.bytes_written += bytes_to_buffer logger.debug( f"Connection {self.conn_id}: buffered {bytes_to_buffer} bytes " f"(total: {len(self.write_buffer)})" ) return bytes_to_buffer async def flush(self) -> int: """Flush pending writes to socket.""" if self.closed: raise RuntimeError(f"Connection {self.conn_id} is closed") if not self.write_buffer: return 0 # Simulate network I/O await asyncio.sleep(0.005) bytes_flushed = len(self.write_buffer) self.bytes_flushed += bytes_flushed self.write_buffer.clear() logger.debug(f"Connection {self.conn_id}: flushed {bytes_flushed} bytes") return bytes_flushed async def close(self) -> Dict[str, int]: """Close connection and flush remaining data.""" if self.closed: return {"buffered": 0, "flushed": 0} logger.info( f"Connection {self.conn_id}: closing " f"({len(self.write_buffer)} bytes in buffer)" ) try: # Force flush any remaining data if self.write_buffer: await asyncio.wait_for(self.flush(), timeout=5.0) except asyncio.TimeoutError: logger.warning(f"Connection {self.conn_id}: flush timeout, forcing close") except Exception as e: logger.error(f"Connection {self.conn_id}: error during close - {e}") finally: self.closed = True stats = { "buffered": len(self.write_buffer), "flushed": self.bytes_flushed, "total_written": self.bytes_written } logger.info(f"Connection {self.conn_id}: closed - {stats}") return stats class ConnectionPool: """Thread-safe connection pool.""" def __init__(self, pool_size: int = 10): self.pool_size = pool_size self.connections: List[MockConnection] = [] self.available = asyncio.Queue() self._initialized = False async def initialize(self): """Initialize the connection pool.""" if self._initialized: return logger.info(f"Initializing connection pool with {self.pool_size} connections") for i in range(self.pool_size): conn = MockConnection(i) self.connections.append(conn) await self.available.put(conn) self._initialized = True @asynccontextmanager async def get_connection(self): """Acquire a connection from the pool.""" conn = await self.available.get() try: yield conn finally: # Return connection to pool if not closed if not conn.closed: await self.available.put(conn) async def close_all(self, timeout: int = 10): """Close all connections in pool.""" logger.info(f"Closing all {len(self.connections)} connections") tasks = [] for conn in self.connections: tasks.append(asyncio.create_task(conn.close())) try: results = await asyncio.wait_for( asyncio.gather(*tasks, return_exceptions=True), timeout=timeout ) # Log results total_flushed = sum( r.get("flushed", 0) for r in results if isinstance(r, dict) ) logger.info(f"All connections closed. Total flushed: {total_flushed} bytes") except asyncio.TimeoutError: logger.error(f"Connection pool close timeout after {timeout}s") for conn in self.connections: conn.closed = True class BoundedRequestManager: """Manages concurrent requests with bounded semaphore.""" def __init__(self, max_concurrent: int = 100): self.semaphore = asyncio.Semaphore(max_concurrent) self.active_tasks: List[asyncio.Task] = [] self.completed_tasks: List[asyncio.Task] = [] self.stats: List[RequestStats] = [] self._lock = asyncio.Lock() self.shutdown_event = asyncio.Event() logger.info(f"Request manager initialized with {max_concurrent} concurrent limit") async def execute(self, coro, request_id: int, timeout: Optional[float] = None): """Execute coroutine with semaphore protection.""" async with self.semaphore: stat = RequestStats( request_id=request_id, start_time=datetime.now() ) try: logger.debug(f"Request {request_id}: acquired semaphore slot") if timeout: result = await asyncio.wait_for(coro, timeout=timeout) else: result = await coro stat.success = True logger.info( f"Request {request_id}: completed in {stat.duration:.3f}s" ) return result except asyncio.TimeoutError: stat.error = "Timeout" logger.error(f"Request {request_id}: timeout after {timeout}s") raise except asyncio.CancelledError: stat.error = "Cancelled" logger.warning(f"Request {request_id}: cancelled") raise except Exception as e: stat.error = str(e) logger.error(f"Request {request_id}: error - {e}") raise finally: stat.end_time = datetime.now() async with self._lock: self.stats.append(stat) async def run_task(self, coro, request_id: int): """Track and manage a task lifecycle.""" task = asyncio.create_task(self.execute(coro, request_id)) async with self._lock: self.active_tasks.append(task) def on_complete(t): try: self.active_tasks.remove(t) self.completed_tasks.append(t) except ValueError: pass task.add_done_callback(on_complete) return task async def graceful_shutdown(self, timeout: int = 30): """Gracefully shutdown all pending tasks.""" logger.info( f"Starting graceful shutdown with " f"{len(self.active_tasks)} active tasks" ) self.shutdown_event.set() if not self.active_tasks: logger.info("No active tasks to cancel") return # Cancel all active tasks for task in self.active_tasks.copy(): if not task.done(): logger.debug(f"Cancelling task: {task.get_name()}") task.cancel() # Wait for cancellation to complete try: _, pending = await asyncio.wait( self.active_tasks, timeout=timeout, return_when=asyncio.ALL_COMPLETED ) if pending: logger.warning( f"{len(pending)} tasks did not complete within {timeout}s" ) for task in pending: task.cancel() else: logger.info("All tasks completed successfully") except Exception as e: logger.error(f"Error during shutdown: {e}") def get_stats(self) -> Dict[str, Any]: """Get statistics about completed requests.""" total = len(self.stats) successful = sum(1 for s in self.stats if s.success) failed = sum(1 for s in self.stats if s.error) durations = [s.duration for s in self.stats if s.success] avg_duration = sum(durations) / len(durations) if durations else 0 return { "total_requests": total, "successful": successful, "failed": failed, "cancelled": sum(1 for s in self.stats if s.error == "Cancelled"), "avg_duration": avg_duration, "min_duration": min(durations) if durations else 0, "max_duration": max(durations) if durations else 0, } class RequestHandler: """Handles individual requests.""" def __init__(self, pool: ConnectionPool): self.pool = pool async def handle_request(self, request_id: int, data: Optional[bytes] = None): """Handle a single request.""" async with self.pool.get_connection() as conn: try: # Write request data request_data = data or f"Request {request_id}".encode() bytes_written = await conn.write(request_data) # Simulate processing await asyncio.sleep(0.05) # Flush data bytes_flushed = await conn.flush() logger.debug( f"Request {request_id}: " f"write={bytes_written}, flush={bytes_flushed}" ) return { "request_id": request_id, "written": bytes_written, "flushed": bytes_flushed } except asyncio.CancelledError: # Ensure flush on cancellation logger.warning(f"Request {request_id}: cancellation detected, flushing") try: await conn.flush() except Exception as e: logger.error( f"Request {request_id}: error during cancel flush - {e}" ) raise class Application: """Main application orchestrator.""" def __init__( self, max_concurrent: int = 100, pool_size: int = 10, request_timeout: float = 30.0 ): self.request_manager = BoundedRequestManager(max_concurrent) self.pool = ConnectionPool(pool_size) self.handler = RequestHandler(self.pool) self.request_timeout = request_timeout self.is_running = True self._setup_signals() def _setup_signals(self): """Setup signal handlers for graceful shutdown.""" def signal_handler(signum, frame): logger.info(f"Received signal {signum}, initiating graceful shutdown") self.is_running = False asyncio.create_task(self.shutdown()) # Handle Ctrl+C (SIGINT) and termination signal signal.signal(signal.SIGINT, signal_handler) signal.signal(signal.SIGTERM, signal_handler) async def submit_request(self, request_id: int) -> asyncio.Task: """Submit a request to the manager.""" if not self.is_running: raise RuntimeError("Application is shutting down") coro = self.handler.handle_request(request_id) return await self.request_manager.run_task(coro, request_id) async def generate_load(self, num_requests: int, rate_limit: Optional[float] = None): """Generate load by submitting multiple requests.""" logger.info(f"Starting load generation: {num_requests} requests") tasks = [] for request_id in range(num_requests): if not self.is_running: logger.info("Stopping request submission due to shutdown signal") break try: task = await self.submit_request(request_id) tasks.append(task) # Rate limiting if specified if rate_limit: await asyncio.sleep(rate_limit) else: # Small delay to avoid overwhelming await asyncio.sleep(0.001) except RuntimeError as e: logger.warning(f"Could not submit request {request_id}: {e}") break except asyncio.CancelledError: logger.info("Load generation cancelled") break # Wait for all submitted tasks if tasks: logger.info(f"Waiting for {len(tasks)} tasks to complete") try: results = await asyncio.gather(*tasks, return_exceptions=True) # Log exceptions exceptions = [r for r in results if isinstance(r, Exception)] if exceptions: logger.warning(f"{len(exceptions)} requests had errors") except asyncio.CancelledError: logger.info("Waiting for tasks was cancelled") raise async def shutdown(self): """Gracefully shutdown the application.""" logger.info("=" * 50) logger.info("SHUTDOWN SEQUENCE INITIATED") logger.info("=" * 50) self.is_running = False # Shutdown request manager await self.request_manager.graceful_shutdown(timeout=30) # Close all connections await self.pool.close_all(timeout=10) # Print statistics stats = self.request_manager.get_stats() logger.info("=" * 50) logger.info("FINAL STATISTICS") logger.info("=" * 50) logger.info(f"Total requests: {stats['total_requests']}") logger.info(f"Successful: {stats['successful']}") logger.info(f"Failed: {stats['failed']}") logger.info(f"Cancelled: {stats['cancelled']}") logger.info(f"Avg duration: {stats['avg_duration']:.3f}s") logger.info(f"Min duration: {stats['min_duration']:.3f}s") logger.info(f"Max duration: {stats['max_duration']:.3f}s") logger.info("=" * 50) async def run(self, num_requests: int = 500, rate_limit: Optional[float] = None): """Run the application.""" try: logger.info("Initializing application...") await self.pool.initialize() logger.info("Starting request load generation...") await self.generate_load(num_requests, rate_limit=rate_limit) except asyncio.CancelledError: logger.info("Application run was cancelled") raise except KeyboardInterrupt: logger.info("KeyboardInterrupt caught") raise finally: await self.shutdown() async def main(): """Main entry point.""" logger.info("Starting asyncio application") app = Application( max_concurrent=50, # Limit to 50 concurrent requests pool_size=10, # Connection pool size request_timeout=30.0 # Individual request timeout ) try: # Generate 200 requests with 10ms rate limit await app.run(num_requests=200, rate_limit=0.01) except KeyboardInterrupt: logger.info("Application interrupted by user (Ctrl+C)") except Exception as e: logger.error(f"Unhandled exception: {e}", exc_info=True) return 1 return 0 if __name__ == "__main__": try: exit_code = asyncio.run(main()) logger.info(f"Application exited with code {exit_code}") sys.exit(exit_code) except KeyboardInterrupt: logger.info("Caught final KeyboardInterrupt") sys.exit(1) except Exception as e: logger.critical(f"Fatal error: {e}", exc_info=True) sys.exit(1)Real-World Example with aiohttp
Here's a version using actual HTTP requests:
import aiohttp import asyncio import logging from typing import Optional logger = logging.getLogger(__name__) class HTTPRequestManager: """Manages HTTP requests with bounded concurrency.""" def __init__(self, max_concurrent: int = 100): self.semaphore = asyncio.Semaphore(max_concurrent) self.session: Optional[aiohttp.ClientSession] = None self.active_tasks = [] async def initialize(self): """Initialize HTTP session.""" timeout = aiohttp.ClientTimeout(total=30) self.session = aiohttp.ClientSession(timeout=timeout) logger.info("HTTP session initialized") async def fetch(self, url: str, method: str = "GET") -> dict: """Fetch a URL with semaphore protection.""" async with self.semaphore: try: async with self.session.request(method, url) as resp: data = await resp.text() return { "url": url, "status": resp.status, "size": len(data) } except asyncio.CancelledError: logger.warning(f"Fetch cancelled: {url}") raise except Exception as e: logger.error(f"Fetch error for {url}: {e}") raise async def close(self): """Close HTTP session gracefully.""" if self.session: logger.info("Closing HTTP session") await self.session.close() logger.info("HTTP session closed") async def main_http_example(): """Example using real HTTP requests.""" manager = HTTPRequestManager(max_concurrent=50) try: await manager.initialize() urls = [ "https://httpbin.org/delay/1", "https://httpbin.org/uuid", ] * 10 # Test URLs tasks = [] for url in urls: task = asyncio.create_task(manager.fetch(url)) tasks.append(task) results = await asyncio.gather(*tasks, return_exceptions=True) successes = sum(1 for r in results if isinstance(r, dict)) failures = sum(1 for r in results if isinstance(r, Exception)) logger.info(f"Results: {successes} successful, {failures} failed") except KeyboardInterrupt: logger.info("HTTP example interrupted") finally: await manager.close() if __name__ == "__main__": logging.basicConfig(level=logging.INFO) # Run mock connection example asyncio.run(main()) # Uncomment to run HTTP example # asyncio.run(main_http_example())Key Features
✅ Bounded Semaphore: Limits concurrent requests to 100
✅ Graceful Shutdown: Properly cancels and waits for all tasks
✅ Socket Cleanup: Flushes pending writes before closing
✅ Signal Handling: Responds to Ctrl+C (SIGINT) and SIGTERM
✅ Resource Management: Context managers for connection pooling
✅ Statistics: Tracks request metrics and timings
✅ Timeout Protection: Individual and global timeouts
✅ Comprehensive Logging: Detailed operation trackingUsage
# Run the application python app.py # Press Ctrl+C to trigger graceful shutdown # All connections will flush pending writes and close cleanly
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.