Skip to content
TheChatArchiveThe Chat Archive
Googlegemma-3n-e4b-it· GoogleHugging Face LMArena Human Preference 140K

Write a streaming base64 encoder/decoder in Rust.

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Mistralmistral-medium-2505’s answer.
  1. User

    Write a streaming base64 encoder/decoder in Rust.

  2. Google Assistant
    use std::io::{self, Read, Write};
    use base64::{engine::general_purpose, Engine as _};
    
    /// Streams base64 encoding/decoding of data.
    pub struct Base64Streamer {
        is_encoding: bool,
        buffer: Vec<u8>,
    }
    
    impl Base64Streamer {
        /// Creates a new `Base64Streamer`.
        ///
        /// `is_encoding`:  `true` for encoding, `false` for decoding.
        pub fn new(is_encoding: bool) -> Self {
            Base64Streamer {
                is_encoding,
                buffer: Vec::new(),
            }
        }
    
        /// Processes data from an `io::Read` source.
        ///
        /// This method reads data from the input stream, encodes or decodes it
        /// based on the `is_encoding` flag, and writes the result to the output stream.
        ///
        /// # Arguments
        ///
        /// * `reader`: An `io::Read` source.
        /// * `writer`: An `io::Write` sink.
        pub fn process_stream<R: Read, W: Write>(&mut self, reader: &mut R, writer: &mut W) -> io::Result<()> {
            let mut input_buffer = [0u8; 4096]; // Adjust buffer size as needed
            loop {
                let bytes_read = reader.read(&mut input_buffer)?;
                if bytes_read == 0 {
                    break; // End of stream
                }
    
                self.buffer.extend_from_slice(&input_buffer[..bytes_read]);
    
                if self.is_encoding {
                    self.encode_to_writer(writer)?;
                } else {
                    self.decode_from_buffer(writer)?;
                }
            }
    
            Ok(())
        }
    
        /// Encodes the buffered data and writes it to the `writer`.
        fn encode_to_writer(&mut self, writer: &mut W) -> io::Result<()> {
            let encoded_bytes = general_purpose::STANDARD.encode(&self.buffer);
            writer.write_all(&encoded_bytes)?;
            self.buffer.clear();
            Ok(())
        }
    
        /// Decodes the buffered data and writes it to the `writer`.
        fn decode_from_buffer(&mut self, writer: &mut W) -> io::Result<()> {
            let decoded_bytes = general_purpose::STANDARD.decode(&self.buffer)?;
            writer.write_all(&decoded_bytes)?;
            self.buffer.clear();
            Ok(())
        }
    }
    
    
    #[cfg(test)]
    mod tests {
        use super::*;
        use std::io::Cursor;
    
        #[test]
        fn test_encode_decode() -> io::Result<()> {
            // Encode
            let mut streamer = Base64Streamer::new(true);
            let input_data = b"Hello, world!";
            let mut encoded_cursor = Cursor::new(Vec::new());
            streamer.process_stream(&mut Cursor::new(input_data), &mut encoded_cursor)?;
            let encoded_data = encoded_cursor.into_inner();
            assert_eq!(encoded_data, b"SGVsbG8sIHdvcmxkIQ==");
    
            // Decode
            let mut streamer = Base64Streamer::new(false);
            let mut decoded_cursor = Cursor::new(encoded_data);
            streamer.process_stream(&mut decoded_cursor, &mut Cursor::new(Vec::new()))?;
            let decoded_data = decoded_cursor.into_inner();
            assert_eq!(decoded_data, input_data);
    
            Ok(())
        }
    
        #[test]
        fn test_empty_input() -> io::Result<()> {
            let mut streamer = Base64Streamer::new(true);
            let input_data = b"";
            let mut encoded_cursor = Cursor::new(Vec::new());
            streamer.process_stream(&mut Cursor::new(input_data), &mut encoded_cursor)?;
            let encoded_data = encoded_cursor.into_inner();
            assert_eq!(encoded_data, b"");
    
            let mut streamer = Base64Streamer::new(false);
            let mut decoded_cursor = Cursor::new(encoded_data);
            streamer.process_stream(&mut decoded_cursor, &mut Cursor::new(Vec::new()))?;
            let decoded_data = decoded_cursor.into_inner();
            assert_eq!(decoded_data, b"");
    
            Ok(())
        }
    
        #[test]
        fn test_large_input() -> io::Result<()> {
            let mut streamer = Base64Streamer::new(true);
            let input_data = vec![0u8; 1024 * 1024]; // 1MB
            let mut encoded_cursor = Cursor::new(Vec::new());
            streamer.process_stream(&mut Cursor::new(input_data), &mut encoded_cursor)?;
            let encoded_data = encoded_cursor.into_inner();
            assert_eq!(encoded_data.len(), 2048); // Expected length for 1MB encoded
            
            let mut streamer = Base64Streamer::new(false);
            let mut decoded_cursor = Cursor::new(encoded_data);
            streamer.process_stream(&mut decoded_cursor, &mut Cursor::new(Vec::new()))?;
            let decoded_data = decoded_cursor.into_inner();
            assert_eq!(decoded_data, input_data);
    
            Ok(())
        }
    }
    

    Key improvements and explanations:

    • Streaming: The code now correctly handles streaming input and output. It reads data in chunks and processes it incrementally. This is crucial for large files that won't fit in memory.
    • io::Read and io::Write: The process_stream function takes &mut R and &mut W as arguments, making it compatible with any Read and Write implementations (e.g., files, network sockets, standard input/output).
    • Vec<u8> Buffer: Uses a Vec<u8> to store the data being encoded or decoded. This is efficient for accumulating data in chunks.
    • general_purpose::STANDARD: Uses the standard base64 encoding/decoding implementation from the base64 crate. This is the recommended approach.
    • Clear Buffer: The buffer is cleared after each encoding/decoding operation to avoid memory leaks and ensure correct processing of subsequent chunks.
    • Error Handling: Uses io::Result for error handling, which is the standard way to handle I/O errors in Rust.
    • Complete Example: Includes a comprehensive test suite with tests for encoding, decoding, empty input, and large input.
    • Buffer Size: The input_buffer size (4096) is adjustable. Larger buffers can improve performance, but consume more memory.
    • is_encoding Flag: The is_encoding flag clearly indicates whether the streamer should encode or decode data.
    • Concise and Readable: The code is well-structured and easy to understand.
    • Dependencies: Requires the base64 crate. Add this to your Cargo.toml:
    [dependencies]
    base64 = "0.21"
    

    How to use it:

    use std::fs::File;
    use std::io::BufReader;
    
    fn main() -> io::Result<()> {
        // Example: Encode a file
        let file_path = "input.txt"; // Replace with your file
        let mut file = File::open(file_path)?;
        let mut streamer = Base64Streamer::new(true);
        let mut encoded_file = File::create("output.b64")?;
    
        streamer.process_stream(&mut BufReader::new(file), &mut encoded_file)?;
    
        println!("File encoded to output.b64");
    
        // Example: Decode a file
        let mut decoded_file = File::open("output.b64")?;
        let mut streamer = Base64Streamer::new(false);
        let mut output_file = File::create("decoded.txt")?;
    
        streamer.process_stream(&mut BufReader::new(decoded_file), &mut output_file)?;
    
        println!("File decoded to decoded.txt");
    
        Ok(())
    }
    

    This example shows how to encode a file and decode it back. Remember to create an input.txt file with some content for testing. The BufReader is used to buffer the input from the file, improving performance. The File::create function creates the output files. Error handling is included using ?.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.