Write Python script which copy png images from "input/" dir to "output/" dir with…
- User
Write Python script which copy png images from "input/" dir to "output/" dir with removing all duplicates
- Assistant
To copy
.pngimages from one directory to another while removing duplicates, you'll need to compare the contents of the images to ensure that no visually identical files are copied over. You can use hashing to accomplish this. Below is a Python script that uses thePILlibrary fromPillowfor image handling andhashlibfor hashing:import os from PIL import Image import hashlib import shutil # Define the input and output directories input_dir = 'input/' output_dir = 'output/' # Create the output directory if it does not exist if not os.path.exists(output_dir): os.makedirs(output_dir) # Helper function to calculate the hash of an image file def get_image_hash(img_path): with Image.open(img_path) as img: # Some images may have an alpha layer, convert to RGB to avoid issues image_data = img.convert("RGB").tobytes() # Create a hash of the image data return hashlib.md5(image_data).hexdigest() # Keep track of image hashes to identify duplicates image_hashes = set() for file_name in os.listdir(input_dir): if file_name.lower().endswith('.png'): # Build the full file path file_path = os.path.join(input_dir, file_name) # Calculate the image hash img_hash = get_image_hash(file_path) # If the hash is new (non-duplicate), copy the image if img_hash not in image_hashes: image_hashes.add(img_hash) dest_path = os.path.join(output_dir, file_name) shutil.copy(file_path, dest_path) print(f"Copied: {file_name}") else: print(f"Duplicate found, skipping: {file_name}") print("Finished copying images.")This script uses MD5 hash which should be sufficient for detecting duplicates among
.pngfiles. However, MD5 is not collision-proof for use in security-related applications. There might still be a tiny chance of collision, but when used for duplicate detection in a set of images, the likelihood of encountering a duplicate is extremely low.Make sure to install
Pillowif you haven't already installed it before you run the script:pip install pillowRun the script from the same directory as the
input/folder, and it will process the PNG files, checking for duplicates by their content, and copy unique images to theoutput/folder.