Write a Python program to split a large file into smaller chunks.
- User
Write a Python program to split a large file into smaller chunks.
- Assistant
Creating a file splitter
The user wants a Python program to split a large file into smaller chunks without specific constraints. I think a script that takes an input file, chunk size, and output directory makes sense. I'll ensure the explanation is brief but clear, and I might use argparse for user-friendly command line input.
I should demonstrate two methods: splitting by byte size or by number of lines. The program must read and write in chunks to manage memory efficiently, especially for large binary files. For usability, parsing something like "100M" may be nice, but I can keep it simple with bytes.Developing a file-splitting script
I’m thinking about how to show a helper for parsing human-friendly sizes, but the user hasn’t asked for that level of detail. Still, I’ll provide a basic function to split a file into smaller chunks based on the byte size. The script will keep it portable across operating systems.
I’ll include a command-line interface (CLI) using argparse and ensure the naming pattern preserves the original file extension. Some brief usage instructions will help clarify things, but I want to avoid overexplaining. That should cover what the user needs! Here’s a simple Python script that splits a large file into smaller chunks by size (in bytes), without loading the whole file into memory:
#!/usr/bin/env python3 import os import argparse def split_file(input_file, chunk_size, output_dir=None): """ Split `input_file` into pieces of `chunk_size` bytes. Files will be named: <original_name>.part001, .part002, ... """ if output_dir is None: output_dir = os.path.dirname(os.path.abspath(input_file)) or "." os.makedirs(output_dir, exist_ok=True) base_name = os.path.basename(input_file) part_num = 1 with open(input_file, "rb") as f: while True: chunk = f.read(chunk_size) if not chunk: break part_filename = os.path.join( output_dir, f"{base_name}.part{part_num:03d}" ) with open(part_filename, "wb") as part_file: part_file.write(chunk) print(f"Wrote {part_filename}") part_num += 1 def main(): parser = argparse.ArgumentParser( description="Split a large file into smaller chunks." ) parser.add_argument("input_file", help="Path to the large file.") parser.add_argument( "chunk_size", type=int, help="Chunk size in bytes (e.g., 104857600 for 100MB).", ) parser.add_argument( "-o", "--output-dir", default=None, help="Directory to write chunks into (default: same as input file).", ) args = parser.parse_args() split_file(args.input_file, args.chunk_size, args.output_dir) if __name__ == "__main__": main()Usage example:
# Split large_file.bin into 50 MB chunks python split_file.py large_file.bin 52428800 # Or specify an output directory python split_file.py large_file.bin 52428800 -o chunks/If you’d prefer splitting by number of lines instead of bytes, I can provide that variant too.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.