how to implement tokio AsyncRead proxy with progress callback reporting in Rust?
- User
how to implement tokio AsyncRead proxy with progress callback reporting in Rust?
Assistant
To implement a Tokio
AsyncReadproxy with a progress callback in Rust, we need to:- Wrap an existing
AsyncReadtype (e.g., a file, network stream, or buffer). - Track the number of bytes read so far.
- Call a user-defined progress callback after each successful read operation.
- Handle errors appropriately.
Here's a complete, well-documented implementation:
✅ Key Concepts
AsyncReadtrait: Provides asynchronous read operations.- Progress callback: A
FnMut(usize)closure that reports the number of bytes read so far. - Error handling: Properly propagate errors from the inner reader.
- Unpin requirement: Most
AsyncReadtypes in Tokio areUnpin, so we'll enforce this in our implementation.
🧱 Implementation
use std::io; use std::marker::Unpin; use tokio::io::AsyncRead; /// A proxy for `AsyncRead` that tracks the number of bytes read and calls a progress callback. pub struct ProgressProxy<R> { inner: R, bytes_read: usize, callback: Box<dyn FnMut(usize) + Send + Sync>, } impl<R: AsyncRead + Unpin> AsyncRead for ProgressProxy<R> { fn read(&mut self, buf: &mut [u8]) -> Poll<usize, io::Error> { match self.inner.read(buf) { // Successfully read some bytes Poll::Ready(Ok(n)) => { self.bytes_read += n; (self.callback)(self.bytes_read); Poll::Ready(Ok(n)) }, // An error occurred during read Poll::Ready(Err(e)) => Poll::Ready(Err(e)), // Not ready yet, wait for more data Poll::Pending => Poll::Pending, } } } impl<R> ProgressProxy<R> { /// Creates a new `ProgressProxy` with the given reader and callback. pub fn new( inner: R, callback: Box<dyn FnMut(usize) + Send + Sync>, ) -> Self { ProgressProxy { inner, bytes_read: 0, callback, } } }
🧪 Example Usage
use tokio::fs::File; use std::io; #[tokio::main] async fn main() -> io::Result<()> { // Open a file for reading let file = File::open("example.txt").await?; // Create a progress proxy let mut proxy = ProgressProxy::new(file, Box::new(|bytes| { println!("✅ Read {} bytes", bytes); })); let mut buffer = [0; 1024]; let n = proxy.read(&mut buffer).await?; println!("✅ Total bytes read: {}", n); Ok(()) }
📌 Notes
Unpinrequirement: TheProgressProxyassumes that the inner reader isUnpin. If you're using a non-Unpinreader (e.g., aBufReader), wrap it in aBoxto make itUnpin.- Callback behavior: The callback is called after each successful read. If you want to reduce the frequency (e.g., every 1024 bytes), you can add a check inside the callback.
- Error handling: Errors are propagated directly from the inner reader. The callback is not called on errors.
- Thread safety: The callback must be
Send + Syncto be used in a multi-threaded context.
🔄 Optional: Throttled Progress Reporting
If you want to report progress only at intervals (e.g., every 1024 bytes), you can modify the
readmethod:fn read(&mut self, buf: &mut [u8]) -> Poll<usize, io::Error> { match self.inner.read(buf) { Poll::Ready(Ok(n)) => { self.bytes_read += n; if self.bytes_read % 1024 == 0 { (self.callback)(self.bytes_read); } Poll::Ready(Ok(n)) }, Poll::Ready(Err(e)) => Poll::Ready(Err(e)), Poll::Pending => Poll::Pending, } }
📦 Dependencies
Make sure your
Cargo.tomlincludes:[dependencies] tokio = { version = "1", features = ["full"] }
✅ Summary
This implementation provides a flexible and safe way to wrap any
AsyncReadtype with a progress callback. It's suitable for tasks like:- Streaming large files
- Monitoring network transfers
- Providing real-time feedback during async I/O operations
You can extend it further by adding features like cancellation, timeouts, or custom logging.
- Wrap an existing
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.