Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-2024-05-13· OpenAIAi2 WildChat-4.8M (AI2)

i have a list of zip files which all have some content in them, one of the files in each…

  1. User

    i have a list of zip files which all have some content in them, one of the files in each zip is a .csv file that I want to count how many lines it has. can you write me a code for this in python?

  2. Assistant

    Certainly! You can use Python's zipfile module to handle ZIP files and the io module to read the file content. Below is a code snippet to count the number of lines in the .csv file within each ZIP archive.

    import zipfile
    import io
    
    def count_lines_in_csv_from_zip(zip_files):
        """
        Count the number of lines in the .csv file within each zip file in a list of zip files.
    
        Args:
        zip_files (list): A list of paths to zip files.
    
        Returns:
        dict: A dictionary mapping zip file names to the number of lines in the .csv file within each.
        """
        line_counts = {}
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                # Iterate through the files within each zip file
                for file_name in z.namelist():
                    if file_name.endswith('.csv'):
                        with z.open(file_name) as csv_file:
                            # Read lines from the CSV file
                            lines = io.TextIOWrapper(csv_file).readlines()
                            line_counts[zip_file] = len(lines)
                            break # Assumes only one CSV file per ZIP
    
        return line_counts
    
    # Example usage
    zip_files = ['file1.zip', 'file2.zip']  # Replace with your actual list of zip file paths
    line_counts = count_lines_in_csv_from_zip(zip_files)
    for zip_file, count in line_counts.items():
        print(f'{zip_file}: {count} lines')
    

    How It Works:

    1. zipfile.ZipFile is used to open each ZIP file.
    2. zipfile.ZipFile.namelist() fetches the list of files within the ZIP archive.
    3. The code looks for files ending with .csv.
    4. It opens the CSV file using z.open(file_name) and reads the lines.
    5. The number of lines is determined by len(lines) and stored in a dictionary.

    Make sure to replace the zip_files list with the actual paths to your ZIP files. This script assumes there's only one CSV file in each ZIP. If there can be multiple CSV files and you want to include all of them, you can modify the script accordingly.

  3. User

    how can I get files in a folder?

  4. Assistant

    To get a list of files in a folder, you can use Python's os module or the pathlib module available in Python 3. The os module provides functions to interact with the file system in a portable way, while pathlib offers an object-oriented approach to handling filesystem paths.

    Here is how you can use both methods to get files in a folder:

    Using os module:

    import os
    
    def get_files_in_folder(folder_path):
        return [os.path.join(folder_path, file) for file in os.listdir(folder_path) if os.path.isfile(os.path.join(folder_path, file))]
    
    # Example usage
    folder_path = 'path/to/your/folder'  # Replace with your folder path
    files_in_folder = get_files_in_folder(folder_path)
    print(files_in_folder)
    

    Using pathlib module:

    from pathlib import Path
    
    def get_files_in_folder(folder_path):
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    # Example usage
    folder_path = 'path/to/your/folder'  # Replace with your folder path
    files_in_folder = get_files_in_folder(folder_path)
    print(files_in_folder)
    

    Both approaches will give you a list of file paths in the specified folder. You can then pass this list of files to the function count_lines_in_csv_from_zip to process the ZIP files.

    Combining Both Parts:

    Here is how you can combine these parts into a single script:

    import os
    import zipfile
    import io
    from pathlib import Path
    
    def get_files_in_folder(folder_path):
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def count_lines_in_csv_from_zip(zip_files):
        line_counts = {}
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_name in z.namelist():
                    if file_name.endswith('.csv'):
                        with z.open(file_name) as csv_file:
                            lines = io.TextIOWrapper(csv_file).readlines()
                            line_counts[zip_file] = len(lines)
                            break
    
        return line_counts
    
    # Example usage
    folder_path = 'path/to/your/folder'  # Replace with your folder path
    files_in_folder = get_files_in_folder(folder_path)
    
    # Filter out non-zip files
    zip_files = [file for file in files_in_folder if file.endswith('.zip')]
    
    line_counts = count_lines_in_csv_from_zip(zip_files)
    for zip_file, count in line_counts.items():
        print(f'{zip_file}: {count} lines')
    

    This combined script will:

    1. Retrieve all files in the specified folder.
    2. Filter out non-ZIP files.
    3. Count the lines in the .csv files contained in each ZIP file.
    4. Print the results.
  5. User

    how can I ready the folder path from input?

  6. Assistant

    You can use Python's built-in input function to read the folder path from the user. Here's how you can incorporate this into the full script:

    import os
    import zipfile
    import io
    from pathlib import Path
    
    def get_files_in_folder(folder_path):
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def count_lines_in_csv_from_zip(zip_files):
        line_counts = {}
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_name in z.namelist():
                    if file_name.endswith('.csv'):
                        with z.open(file_name) as csv_file:
                            lines = io.TextIOWrapper(csv_file).readlines()
                            line_counts[zip_file] = len(lines)
                            break
    
        return line_counts
    
    # Read folder path from user input
    folder_path = input("Please enter the folder path: ")
    
    # Validate folder path
    if os.path.isdir(folder_path):
        files_in_folder = get_files_in_folder(folder_path)
    
        # Filter out non-zip files
        zip_files = [file for file in files_in_folder if file.endswith('.zip')]
    
        # Count lines in CSV files within ZIP files
        line_counts = count_lines_in_csv_from_zip(zip_files)
        for zip_file, count in line_counts.items():
            print(f'{zip_file}: {count} lines')
    else:
        print("The provided path is not a valid directory.")
    

    Explanation:

    1. Reading Folder Path from Input: The input function is used to prompt the user to enter the folder path.
    2. Validating Folder Path: The os.path.isdir function checks if the provided path is a valid directory.
    3. Using the Provided Path: If the folder path is valid, the script proceeds with getting the files in the folder, filtering ZIP files, and counting the lines in the .csv files within those ZIP files.
    4. Error Handling: If the provided path is not valid, an error message is printed.

    This approach ensures that the program interacts with the user, making it more flexible and interactive.

  7. User

    I'd also like to group the sum of the number of lines by the month of the file date

  8. Assistant

    To group the sum of the number of lines by the month of the file's date, you can modify the script to extract the dates of the files from the ZIP archives, aggregate the line counts by month, and then sum them up. Let's integrate this feature.

    Below is the modified script to achieve the desired functionality:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    
    def get_files_in_folder(folder_path):
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def count_lines_in_csv_from_zip(zip_files):
        line_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv'):
                        # Extract the date of the file from the zip info
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            lines = io.TextIOWrapper(csv_file).readlines()
                            line_counts_by_month[month_str] += len(lines)
                            break  # Assuming only one CSV file per ZIP
    
        return line_counts_by_month
    
    # Read folder path from user input
    folder_path = input("Please enter the folder path: ")
    
    # Validate folder path
    if os.path.isdir(folder_path):
        files_in_folder = get_files_in_folder(folder_path)
    
        # Filter out non-zip files
        zip_files = [file for file in files_in_folder if file.endswith('.zip')]
    
        # Count lines in CSV files within ZIP files and group by month
        line_counts_by_month = count_lines_in_csv_from_zip(zip_files)
    
        for month, count in sorted(line_counts_by_month.items()):
            print(f'{month}: {count} lines')
    else:
        print("The provided path is not a valid directory.")
    

    Explanation:

    1. Extracting File Date:

      • file_info.date_time gives a tuple with the (year, month, day, hour, minute, second) of the file inside the zip.
      • datetime(*file_info.date_time) creates a datetime object from this tuple.
      • strftime("%Y-%m") formats the datetime to a YYYY-MM string to group by month.
    2. Grouping and Summing by Month:

      • defaultdict(int) is used to create a dictionary that will automatically initialize counts to zero.
      • For each CSV file found, it extracts the lines, sums them up by the corresponding month, and updates the count in the dictionary.
    3. User Interaction:

      • Reads the folder path from the user.
      • Validates if the folder path is correct.
      • Filters for ZIP files.
      • Processes each ZIP file to count line numbers in CSVs, grouping by the month the file within the ZIP was last modified.

    This script will print the total number of lines in CSV files for each month, grouped by the month of the file's date within the ZIP.

  9. User

    there could be multiple csv-s, but the one that i need to read the lines is formatted "OUT"

  10. Assistant

    Incorporating the specific requirement of reading only the CSV files that are named with the pattern *_OUT_*, we can modify the script as follows:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    
    def get_files_in_folder(folder_path):
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def count_lines_in_csv_from_zip(zip_files):
        line_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv') and "_OUT_" in file_name:
                        # Extract the date of the file from the zip info
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            lines = io.TextIOWrapper(csv_file).readlines()
                            line_counts_by_month[month_str] += len(lines)
                            # No break statement, since we need to process all matching CSV files
    
        return line_counts_by_month
    
    # Read folder path from user input
    folder_path = input("Please enter the folder path: ")
    
    # Validate folder path
    if os.path.isdir(folder_path):
        files_in_folder = get_files_in_folder(folder_path)
    
        # Filter out non-zip files
        zip_files = [file for file in files_in_folder if file.endswith('.zip')]
    
        # Count lines in CSV files within ZIP files and group by month
        line_counts_by_month = count_lines_in_csv_from_zip(zip_files)
    
        for month, count in sorted(line_counts_by_month.items()):
            print(f'{month}: {count} lines')
    else:
        print("The provided path is not a valid directory.")
    

    Explanation:

    1. Filtering Specific CSV Files:

      • The condition if file_name.endswith('.csv') and "_OUT_" in file_name ensures that only CSV files containing _OUT_ in their names are processed.
    2. Processing Multiple Matching CSV Files:

      • The break statement has been removed so that the loop will continue processing all files within the ZIP archive rather than stopping after the first matching file.
    3. Line Counting and Grouping:

      • The lines are read and added to the count for the respective month in the line_counts_by_month dictionary.

    When you run this script, it will prompt you to enter the folder path, process the ZIP files in that folder, and print the sum of lines in the relevant CSV files grouped by the month of the file's date.

  11. User

    Now I only want you to count the entries in the csv instead of all lines (meaning no header), also these csv-s can have a column called "error_code", which if exists and contains any value, exclude from the count

  12. Assistant

    To modify the script to count only the entries in the CSV files (ignoring the header) and to exclude entries with an "error_code" value, we can use the csv module for easier row handling and filtering.

    Here's how you can implement this:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    import csv
    
    def get_files_in_folder(folder_path):
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def count_entries_in_csv_from_zip(zip_files):
        entry_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv') and "_OUT_" in file_name:
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            csv_reader = csv.DictReader(io.TextIOWrapper(csv_file))
                            for row in csv_reader:
                                if 'error_code' not in row or not row['error_code'].strip():
                                    entry_counts_by_month[month_str] += 1
    
        return entry_counts_by_month
    
    # Read folder path from user input
    folder_path = input("Please enter the folder path: ")
    
    # Validate folder path
    if os.path.isdir(folder_path):
        files_in_folder = get_files_in_folder(folder_path)
    
        # Filter out non-zip files
        zip_files = [file for file in files_in_folder if file.endswith('.zip')]
    
        # Count entries in CSV files within ZIP files and group by month
        entry_counts_by_month = count_entries_in_csv_from_zip(zip_files)
    
        for month, count in sorted(entry_counts_by_month.items()):
            print(f'{month}: {count} entries')
    else:
        print("The provided path is not a valid directory.")
    

    Explanation:

    1. Using csv.DictReader:

      • csv.DictReader is used to read the CSV file and conveniently access columns by their header names.
    2. Ignoring the Header:

      • By using csv.DictReader, we automatically skip the header row.
    3. Filtering Rows Based on error_code:

      • For each row, check if the error_code column either does not exist ('error_code' not in row) or is empty (not row['error_code'].strip()).
      • Only rows that meet these criteria are counted.

    With this script:

    • The entries in the relevant CSV files are counted, excluding rows that have any non-empty error_code.
    • The counts are grouped by the month of the file's date within the ZIP.
    • The results are printed, showing the number of valid entries for each month.
  13. User

    now instead of az input folder, go through and execute this to all folders in the same level the script is executed

  14. Assistant

    To process all folders at the same directory level as the script itself, you can use the os module to get the current directory and iterate through its subdirectories. Then, apply the same logic to count entries in the CSV files within ZIP files for each subdirectory.

    Here’s the modified script:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    import csv
    
    
    def get_files_in_folder(folder_path):
        """Get all files in a given folder."""
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    
    def count_entries_in_csv_from_zip(zip_files):
        """Count valid entries in CSV files within ZIP files."""
        entry_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv') and "_OUT_" in file_name:
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            csv_reader = csv.DictReader(io.TextIOWrapper(csv_file))
                            for row in csv_reader:
                                if 'error_code' not in row or not row['error_code'].strip():
                                    entry_counts_by_month[month_str] += 1
        return entry_counts_by_month
    
    
    # Get the current directory where the script is executed
    current_directory = Path.cwd()
    
    def process_all_folders_in_directory(directory):
        """Process all subdirectories in the given directory."""
        all_counts = defaultdict(int)
        
        # Iterate through each subdirectory
        for subfolder in directory.iterdir():
            if subfolder.is_dir():
                print(f"Processing folder: {subfolder}")
                files_in_folder = get_files_in_folder(subfolder)
                
                # Filter out non-zip files
                zip_files = [file for file in files_in_folder if file.endswith('.zip')]
                
                # Count entries in CSV files within ZIP files and group by month
                entry_counts_by_month = count_entries_in_csv_from_zip(zip_files)
                
                for month, count in entry_counts_by_month.items():
                    all_counts[month] += count
        
        return all_counts
    
    # Execute the process
    total_counts_by_month = process_all_folders_in_directory(current_directory)
    for month, count in sorted(total_counts_by_month.items()):
        print(f'{month}: {count} entries')
    

    Explanation:

    1. Function Definitions:

      • get_files_in_folder: Retrieves all files in a given folder.
      • count_entries_in_csv_from_zip: Counts valid entries in CSV files within ZIP files.
      • process_all_folders_in_directory: Iterates over all subdirectories in the specified directory, processes them, and aggregates the counts by month.
    2. Current Directory:

      • Path.cwd() gets the current directory where the script is executed.
    3. Processing All Subdirectories:

      • The script iterates through each subdirectory in the current execution directory.
      • For each subdirectory, it retrieves all files, filters ZIP files, and counts CSV entries, aggregating the results by month.
    4. Output:

      • The aggregated results from all subdirectories are printed, showing the number of valid entries grouped by month.

    This approach ensures that the script processes every subdirectory at the same level where it is executed and aggregates the results accordingly.

  15. User

    the csv is separated by ";", does it need to be set somewhere?

  16. Assistant

    Yes, you should specify the delimiter when reading the CSV files using the csv module. By default, csv.DictReader assumes commas (,) as separators. Since your CSV uses semicolons (;), you need to set the delimiter parameter to ';'.

    Here's the updated script with the delimiter specified:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    import csv
    
    def get_files_in_folder(folder_path):
        """Get all files in a given folder."""
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def count_entries_in_csv_from_zip(zip_files):
        """Count valid entries in CSV files within ZIP files."""
        entry_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv') and "_OUT_" in file_name:
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            csv_reader = csv.DictReader(io.TextIOWrapper(csv_file), delimiter=';')
                            for row in csv_reader:
                                if 'error_code' not in row or not row['error_code'].strip():
                                    entry_counts_by_month[month_str] += 1
        return entry_counts_by_month
    
    # Get the current directory where the script is executed
    current_directory = Path.cwd()
    
    def process_all_folders_in_directory(directory):
        """Process all subdirectories in the given directory."""
        all_counts = defaultdict(int)
        
        # Iterate through each subdirectory
        for subfolder in directory.iterdir():
            if subfolder.is_dir():
                print(f"Processing folder: {subfolder}")
                files_in_folder = get_files_in_folder(subfolder)
                
                # Filter out non-zip files
                zip_files = [file for file in files_in_folder if file.endswith('.zip')]
                
                # Count entries in CSV files within ZIP files and group by month
                entry_counts_by_month = count_entries_in_csv_from_zip(zip_files)
                
                for month, count in entry_counts_by_month.items():
                    all_counts[month] += count
        
        return all_counts
    
    # Execute the process
    total_counts_by_month = process_all_folders_in_directory(current_directory)
    for month, count in sorted(total_counts_by_month.items()):
        print(f'{month}: {count} entries')
    

    Key Changes:

    • Delimiter Specification:
      • The csv.DictReader is initialized with delimiter=';' to correctly parse the CSV files that use semicolons as separators.

    Summary:

    • This script will search for _OUT_-patterned CSV files separated by semicolons in ZIP files across all subdirectories.
    • It will count valid entries (excluding headers and rows with error_code values) and group the results by the month of the file's date.
    • The aggregated results are then printed, showing the number of valid entries for each month.
  17. User

    it turned out that it can be either ';' or ','. can I somehow detect this?

  18. Assistant

    Detecting the delimiter in a CSV file can be done by reading the initial part of the file and checking which delimiter appears more frequently. Python's csv.Sniffer class provides a convenient way to deduce the format of a CSV file, including the delimiter.

    Let's update the script to dynamically detect the delimiter of each CSV file:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    import csv
    
    def get_files_in_folder(folder_path):
        """Get all files in a given folder."""
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def determine_delimiter(sample):
        """Determine the delimiter of a CSV file."""
        sniffer = csv.Sniffer()
        return sniffer.sniff(sample).delimiter
    
    def count_entries_in_csv_from_zip(zip_files):
        """Count valid entries in CSV files within ZIP files."""
        entry_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv') and "_OUT_" in file_name:
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            # Read a sample to determine delimiter
                            sample = csv_file.read(1024).decode('utf-8')
                            csv_file.seek(0)  # Reset the file pointer to the beginning
                            delimiter = determine_delimiter(sample)
                            
                            csv_reader = csv.DictReader(io.TextIOWrapper(csv_file), delimiter=delimiter)
                            for row in csv_reader:
                                if 'error_code' not in row or not row['error_code'].strip():
                                    entry_counts_by_month[month_str] += 1
        return entry_counts_by_month
    
    # Get the current directory where the script is executed
    current_directory = Path.cwd()
    
    def process_all_folders_in_directory(directory):
        """Process all subdirectories in the given directory."""
        all_counts = defaultdict(int)
        
        # Iterate through each subdirectory
        for subfolder in directory.iterdir():
            if subfolder.is_dir():
                print(f"Processing folder: {subfolder}")
                files_in_folder = get_files_in_folder(subfolder)
                
                # Filter out non-zip files
                zip_files = [file for file in files_in_folder if file.endswith('.zip')]
                
                # Count entries in CSV files within ZIP files and group by month
                entry_counts_by_month = count_entries_in_csv_from_zip(zip_files)
                
                for month, count in entry_counts_by_month.items():
                    all_counts[month] += count
        
        return all_counts
    
    # Execute the process
    total_counts_by_month = process_all_folders_in_directory(current_directory)
    for month, count in sorted(total_counts_by_month.items()):
        print(f'{month}: {count} entries')
    

    Explanation:

    1. Determine Delimiter:

      • The determine_delimiter function uses Python’s csv.Sniffer to analyze a sample of the CSV content and deduce the delimiter.
      • The sample is read in the line sample = csv_file.read(1024).decode('utf-8') and then the file pointer is reset using csv_file.seek(0).
    2. Reading CSV Content:

      • The determined delimiter is used to initialize the csv.DictReader.

    Key Functions:

    • get_files_in_folder: Retrieves all files in a given folder.
    • determine_delimiter: Uses csv.Sniffer to guess the CSV delimiter.
    • count_entries_in_csv_from_zip: Counts valid entries in CSV files within ZIP files, adjusts for dynamic delimiters.
    • process_all_folders_in_directory: Processes all subdirectories, aggregates results.

    Execution:

    • The script processes each subdirectory at the same directory level where it is executed and prints the number of valid entries for each month.

    With this approach, your script can handle CSV files with either ; or , as delimiters without needing manual intervention.

  19. User

    each csv entry has a 'sid' column which should only be accounted as one if appears multiple times PER FILE

  20. Assistant

    To ensure that each unique sid is only counted once per file, you need to track the sid values you've encountered in each CSV file and only count unique sid values. Here is how you can modify the script to achieve this:

    import os
    import zipfile
    import io
    from pathlib import Path
    from collections import defaultdict
    from datetime import datetime
    import csv
    
    def get_files_in_folder(folder_path):
        """Get all files in a given folder."""
        return [str(file) for file in Path(folder_path).iterdir() if file.is_file()]
    
    def determine_delimiter(sample):
        """Determine the delimiter of a CSV file."""
        sniffer = csv.Sniffer()
        return sniffer.sniff(sample).delimiter
    
    def count_entries_in_csv_from_zip(zip_files):
        """Count valid entries in CSV files within ZIP files."""
        entry_counts_by_month = defaultdict(int)
        
        for zip_file in zip_files:
            with zipfile.ZipFile(zip_file, 'r') as z:
                for file_info in z.infolist():
                    file_name = file_info.filename
                    if file_name.endswith('.csv') and "_OUT_" in file_name:
                        date_time = datetime(*file_info.date_time)
                        month_str = date_time.strftime("%Y-%m")
                        
                        with z.open(file_name) as csv_file:
                            # Read a sample to determine the delimiter
                            sample = csv_file.read(1024).decode('utf-8')
                            csv_file.seek(0)  # Reset the file pointer to the beginning
                            delimiter = determine_delimiter(sample)
                            
                            csv_reader = csv.DictReader(io.TextIOWrapper(csv_file), delimiter=delimiter)
                            unique_sids = set()
                            for row in csv_reader:
                                if 'error_code' not in row or not row['error_code'].strip():
                                    unique_sids.add(row['sid'])
                            entry_counts_by_month[month_str] += len(unique_sids)
        return entry_counts_by_month
    
    # Get the current directory where the script is executed
    current_directory = Path.cwd()
    
    def process_all_folders_in_directory(directory):
        """Process all subdirectories in the given directory."""
        all_counts = defaultdict(int)
        
        # Iterate through each subdirectory
        for subfolder in directory.iterdir():
            if subfolder.is_dir():
                print(f"Processing folder: {subfolder}")
                files_in_folder = get_files_in_folder(subfolder)
                
                # Filter out non-zip files
                zip_files = [file for file in files_in_folder if file.endswith('.zip')]
                
                # Count entries in CSV files within ZIP files and group by month
                entry_counts_by_month = count_entries_in_csv_from_zip(zip_files)
                
                for month, count in entry_counts_by_month.items():
                    all_counts[month] += count
        
        return all_counts
    
    # Execute the process
    total_counts_by_month = process_all_folders_in_directory(current_directory)
    for month, count in sorted(total_counts_by_month.items()):
        print(f'{month}: {count} entries')
    

    Key Changes:

    1. Tracking Unique sid Values:

      • A set named unique_sids is used to track unique sid values within each file.
      • Unique sid values are added to the set using unique_sids.add(row['sid']).
    2. Counting Unique sid Entries:

      • After reading all rows in a CSV file, the length of the unique_sids set is added to the monthly count using entry_counts_by_month[month_str] += len(unique_sids).

    Summary:

    • Ensure that the csv.DictReader dynamically detects the delimiter (; or ,) for each CSV file.
    • Only count unique sid entries per file, and exclude rows with a non-empty error_code.
    • Aggregate counts by the month of the file date, and then print the results.

    This approach will correctly account for each unique sid only once per file, while handling different delimiters and excluding invalid rows.