Skip to content
TheChatArchiveThe Chat Archive
Kimikimi-k2-0711-preview· Moonshot AIHugging Face LMArena Human Preference 140K

split .csv file into chunks by regexp

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with grok-4-0709’s answer.
  1. User

    split .csv file into chunks by regexp

  2. Kimi Assistant

    To split a .csv file into chunks based on a regular expression, you can use Python with the pandas and re libraries. The idea is to:

    1. Read the CSV file.
    2. Identify rows that match a regex pattern.
    3. Group rows based on the pattern.
    4. Save each group to a separate file.

    ✅ Example Use Case

    Suppose you want to split a CSV file into chunks where each chunk contains rows where a specific column (e.g., "category") matches a regex pattern like ^A\d+, ^B\d+, etc.


    ✅ Python Script

    import pandas as pd
    import re
    import os
    
    # === CONFIGURATION ===
    input_csv = 'data.csv'
    output_dir = 'chunks'
    pattern_column = 'category'
    regex_pattern = r'^(A\d+|B\d+|C\d+)'  # Adjust this to your needs
    # =====================
    
    # Create output directory
    os.makedirs(output_dir, exist_ok=True)
    
    # Read CSV
    df = pd.read_csv(input_csv)
    
    # Function to extract the key from the pattern
    def extract_key(value):
        match = re.match(regex_pattern, str(value))
        return match.group(0) if match else 'unknown'
    
    # Add a key column
    df['chunk_key'] = df[pattern_column].apply(extract_key)
    
    # Group and save
    for key, group in df.groupby('chunk_key'):
        filename = f"{output_dir}/chunk_{key}.csv"
        group.drop(columns=['chunk_key']).to_csv(filename, index=False)
        print(f"Saved: {filename}")
    

    ✅ Notes

    • You can change regex_pattern to match whatever grouping logic you need.
    • If the pattern is more complex (e.g., multiple columns or nested logic), you can adjust the extract_key function accordingly.
    • If you want to split by line content (not a specific column), you can read the file line-by-line and use regex directly.

    ✅ Alternative: Split by Line Content (No CSV Parsing)

    If your CSV is simple and you want to split by regex on any line:

    import re
    
    input_file = 'data.csv'
    output_dir = 'chunks'
    pattern = re.compile(r'^A\d+')  # Example: lines starting with A followed by digits
    
    os.makedirs(output_dir, exist_ok=True)
    
    with open(input_file, 'r') as f:
        header = next(f)  # Save header
        chunk_files = {}
        for line in f:
            match = pattern.search(line)
            key = match.group(0) if match else 'unknown'
            if key not in chunk_files:
                chunk_files[key] = open(f"{output_dir}/chunk_{key}.csv", 'w')
                chunk_files[key].write(header)
            chunk_files[key].write(line)
    
    # Close all files
    for f in chunk_files.values():
        f.close()
    

    Let me know your exact regex or column name if you want a tailored version.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.