Skip to content
The Chat Archive
o1-preview-2024-09-12· OpenAI WildChat-4.8M (AI2)

Here is a restructured version of your prompt: Task: Image Resizing and Dataset…

  1. User

    Here is a restructured version of your prompt:

    Task: Image Resizing and Dataset Configuration

    Overview:

    Create a Python code snippet that can be run in a Jupyter Notebook to perform the following tasks:

    1. Image Loading and Pixel Count Calculation:

      • Read images from a folder that may contain non-image files.
      • Calculate the total number of pixels in each image.
      • Display console output for the total pixel count.
    2. Image Scaling:

      • Scale down each image so that its total pixel count is proportional to a specified number (e.g., 1,048,576).
      • Display console output for the scaled image dimensions.
    3. Image Resizing and Storage:

      • Adjust the image's width and height to the nearest multiples of 64.
      • Store the resized images in separate folders based on their corresponding widths and heights (e.g., 1280_832).
      • Display console output for the resized image dimensions and folder paths.
    4. Dataset Configuration File Generation:

      • Create a dataset.toml file based on the resolutions of the resized images.
      • The dataset.toml file should have a specific format (see below).

    Sample dataset.toml Output:

    [general]
    
    [[datasets]]
    batch_size = 1
    enable_bucket = true
    resolution = [1280, 832]
    
      [[datasets.subsets]]
      image_dir = "path/to/save/resized/images/1280_832"
      num_repeats = 1
    
    [[datasets]]
    batch_size = 1
    enable_bucket = true
    resolution = [1152, 864]
    
      [[datasets.subsets]]
      image_dir = "path/to/save/resized/1152_864"
      num_repeats = 1
    

    Requirements:

    • The code should be written in Python and compatible with Jupyter Notebook.
    • The code should handle images with varying dimensions and non-image files in the input folder.
    • The code should generate a dataset.toml file with the specified format.
    • The code should display relevant console output for each step.
  2. Assistant

    Certainly! Below is the complete Python code snippet that performs all the specified tasks. You can run this code in a Jupyter Notebook. Please make sure to replace 'path/to/images' and 'path/to/save/resized/images' with the actual paths on your system.

    import os
    import math
    from PIL import Image
    
    # Specify your input and output folders
    input_folder = 'path/to/images'  # Replace with your images folder path
    output_base_folder = 'path/to/save/resized/images'  # Replace with your output folder path
    target_total_pixels = 1048576  # Target total pixels (e.g., 1024 * 1024)
    
    # Dictionary to store image resolutions for dataset.toml generation
    image_resolutions = {}  # key: (width, height), value: list of image file paths
    
    # Ensure the output base folder exists
    if not os.path.exists(output_base_folder):
        os.makedirs(output_base_folder)
    
    # Iterate over files in the input folder
    for filename in os.listdir(input_folder):
        filepath = os.path.join(input_folder, filename)
        
        # Attempt to open the file as an image
        try:
            with Image.open(filepath) as img:
                # Successfully opened the image
                original_width, original_height = img.size
                total_pixels = original_width * original_height
                
                print(f"Processing image: {filename}")
                print(f"Original dimensions: {original_width}x{original_height}")
                print(f"Total pixels: {total_pixels}")
                
                # Step 2: Scale down the image proportionally to the target total pixels
                scaling_factor = math.sqrt(target_total_pixels / total_pixels)
                scaled_width = int(original_width * scaling_factor)
                scaled_height = int(original_height * scaling_factor)
                
                print(f"Scaled dimensions before adjustment: {scaled_width}x{scaled_height}")
                
                # Step 3: Adjust dimensions to the nearest multiples of 64
                adjusted_width = int(round(scaled_width / 64.0)) * 64
                adjusted_height = int(round(scaled_height / 64.0)) * 64
                
                # Ensure dimensions are at least 64
                adjusted_width = max(64, adjusted_width)
                adjusted_height = max(64, adjusted_height)
                
                print(f"Adjusted dimensions: {adjusted_width}x{adjusted_height}")
                
                # Resize the image
                resized_img = img.resize((adjusted_width, adjusted_height), Image.ANTIALIAS)
                
                # Create the output directory for this resolution if it doesn't exist
                output_folder = os.path.join(output_base_folder, f"{adjusted_width}_{adjusted_height}")
                if not os.path.exists(output_folder):
                    os.makedirs(output_folder)
                    
                # Store the resolution for dataset.toml generation
                res_key = (adjusted_width, adjusted_height)
                if res_key not in image_resolutions:
                    image_resolutions[res_key] = []
                image_resolutions[res_key].append(os.path.join(output_folder, filename))
                
                # Save the resized image
                output_path = os.path.join(output_folder, filename)
                resized_img.save(output_path)
                
                print(f"Saved resized image to: {output_path}")
                print("---------------------------------------")
        except IOError:
            # File is not an image, skip it
            print(f"Skipping non-image file: {filename}")
            print("---------------------------------------")
    
    # Step 4: Generate dataset.toml file
    toml_lines = []
    toml_lines.append("[general]")
    toml_lines.append("")
    
    for res in image_resolutions:
        width, height = res
        output_folder = os.path.join(output_base_folder, f"{width}_{height}")
        
        toml_lines.append("[[datasets]]")
        toml_lines.append("batch_size = 1")
        toml_lines.append("enable_bucket = true")
        toml_lines.append(f"resolution = [{width}, {height}]")
        toml_lines.append("")
        toml_lines.append("  [[datasets.subsets]]")
        # Replace backslashes with forward slashes for paths in toml
        image_dir = os.path.normpath(output_folder).replace('\\', '/')
        toml_lines.append(f'  image_dir = "{image_dir}"')
        toml_lines.append("  num_repeats = 1")
        toml_lines.append("")
    
    # Write the toml content to the file
    toml_content = "\n".join(toml_lines)
    dataset_toml_path = os.path.join(output_base_folder, "dataset.toml")
    
    with open(dataset_toml_path, "w") as toml_file:
        toml_file.write(toml_content)
    
    print(f"Generated dataset.toml at {dataset_toml_path}")
    

    Explanation of the Code:

    • Imports and Initial Setup:

      • The code begins by importing necessary modules: os, math, and Image from PIL.
      • It sets the input_folder and output_base_folder where images will be read from and saved to.
      • target_total_pixels is set to 1,048,576 as specified.
    • Processing Images:

      • The code iterates over each file in the input_folder.
      • It attempts to open each file as an image using Image.open().
      • If the file is not an image, it catches the IOError and skips the file.
      • For each image:
        • It calculates the original dimensions and total pixels and prints them.
        • It calculates the scaling factor to adjust the image to the target total pixels.
        • It scales the image dimensions proportionally and prints the scaled dimensions.
        • It adjusts the dimensions to the nearest multiples of 64 and ensures they are at least 64 pixels.
        • It resizes the image and saves it to the appropriate folder named after its dimensions.
        • It prints the adjusted dimensions and the path where the resized image is saved.
    • Generating dataset.toml:

      • The code collects all the unique resolutions and the paths to the images with those resolutions.
      • It constructs the dataset.toml content as a list of strings, following the specified format.
      • It writes the content to a file named dataset.toml in the output_base_folder.

    Console Output:

    The console output will display messages for each significant step:

    • When processing each image:

      Processing image: example.jpg
      Original dimensions: 1920x1080
      Total pixels: 2073600
      Scaled dimensions before adjustment: 1456x819
      Adjusted dimensions: 1472x832
      Saved resized image to: path/to/save/resized/images/1472_832/example.jpg
      ---------------------------------------
      
    • When skipping non-image files:

      Skipping non-image file: document.txt
      ---------------------------------------
      
    • After generating dataset.toml:

      Generated dataset.toml at path/to/save/resized/images/dataset.toml
      

    Sample dataset.toml Content:

    The generated dataset.toml will look similar to this:

    [general]
    
    [[datasets]]
    batch_size = 1
    enable_bucket = true
    resolution = [1280, 832]
    
      [[datasets.subsets]]
      image_dir = "path/to/save/resized/images/1280_832"
      num_repeats = 1
    
    [[datasets]]
    batch_size = 1
    enable_bucket = true
    resolution = [1152, 864]
    
      [[datasets.subsets]]
      image_dir = "path/to/save/resized/images/1152_864"
      num_repeats = 1
    

    Important Notes:

    • Make sure to install the Pillow library if you haven't already, since it's required for image processing:
      pip install Pillow
      
    • The code replaces backslashes with forward slashes in paths when writing to the dataset.toml file to ensure compatibility across different operating systems.
    • Adjust the input_folder, output_base_folder, and target_total_pixels variables according to your needs.
    • The code handles images with varying dimensions and skips non-image files gracefully.

    Feel free to run this code in your Jupyter Notebook, and it will perform all the tasks as specified.