Write a Python script to batch convert images using Pillow with multiprocessing to use…
- User
Write a Python script to batch convert images using Pillow with multiprocessing to use all CPU cores. It should detect corrupt images with truncated headers and move them to a quarantine folder instead of crashing.
Assistant
This is a comprehensive request for a Python script that needs to:
- Batch convert images using Pillow
- Use multiprocessing to utilize all CPU cores
- Detect corrupt images with truncated headers
- Move corrupt images to a quarantine folder
- Handle errors gracefully without crashing
Let me create a robust solution that includes:
- Multiprocessing Pool for parallel processing
- Error handling for corrupt images
- Truncated header detection
- Logging
- Configuration options
- Progress tracking
- A quarantine folder mechanism
Here's what I'll include:
- Image validation and conversion function
- Corrupt image detection
- Multiprocessing setup
- Logging
- Command-line interface
- Configuration
Let me write a comprehensive script:
import os import sys import logging import argparse from pathlib import Path from PIL import Image from multiprocessing import Pool, cpu_count from functools import partial import traceback import shutil from datetime import datetime # Configure logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s', handlers=[ logging.StreamHandler(), logging.FileHandler('image_conversion.log') ] ) logger = logging.getLogger(__name__) class ImageConverter: def __init__(self, input_dir, output_dir, target_format='JPEG', quality=85, size=None, quarantine_dir=None): """ Initialize the image converter Args: input_dir: Directory containing images to convert output_dir: Directory for converted images target_format: Output format (JPEG, PNG, etc.) quality: JPEG quality (1-100) size: Tuple (width, height) to resize images, None for no resize quarantine_dir: Directory for corrupt images """ self.input_dir = Path(input_dir) self.output_dir = Path(output_dir) self.target_format = target_format.upper() self.quality = quality self.size = size self.quarantine_dir = Path(quarantine_dir) if quarantine_dir else self.output_dir / 'quarantine' # Create output directories if they don't exist self.output_dir.mkdir(parents=True, exist_ok=True) self.quarantine_dir.mkdir(parents=True, exist_ok=True) # Track conversion statistics self.stats = { 'processed': 0, 'successful': 0, 'corrupted': 0, 'failed': 0 }I need a method to validate image files before processing. I'll attempt to open each image and load its data to catch any truncation or format issues. If the image can't be identified or has IO problems, I'll return False with a descriptive error message explaining what went wrong.
For any other unexpected errors during validation, I should also return False with details about the exception. Then I'll create a process_image method that takes an image path and returns a result dictionary tracking the input file, processing status, any output path, and error details if something goes wrong.
The method starts by checking if the image is valid using the integrity check. If validation fails, I mark it as corrupted, log a warning with the specific error, and attempt to move the file to a quarantine directory for later inspection.
Once the image passes validation, I open it and handle color mode conversions—particularly converting RGBA images to RGB with a white background when saving as JPEG, or converting other modes to RGB as needed. If a target size is specified, I resize the image using Lanczos resampling, then save the processed result to the output directory with the appropriate format and quality settings. =False): """ Process all images in batch using multiprocessing
Args: num_processes: Number of processes (default: number of CPU cores) recursive: If True, process subdirectories recursively """ if num_processes is None: num_processes = cpu_count() logger.info(f"Starting batch image conversion using {num_processes} processes") # Get image files if recursive: image_files = list(self.input_dir.rglob('*')) image_files = [f for f in image_files if f.is_file() and f.suffix.lower() in ['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp']] else: image_files = self.get_image_files() self.stats['total'] = len(image_files) logger.info(f"Found {self.stats['total']} images to process") if not image_files: logger.warning("No image files found in input directory") return # Process images in parallel with Pool(processes=num_processes) as pool: results = pool.map(self.process_image, image_files) # Update statistics for result in results: if result['status'] == 'success': self.stats['successful'] += 1 elif result['status'] == 'corrupted': self.stats['corrupted'] += 1 else: self.stats['failed'] += 1 self.print_summary() def print_summary(self): """Print processing summary""" logger.info("\n" + "="*60) logger.info("BATCH CONVERSION SUMMARY") logger.info("="*60) logger.info(f"Total images: {self.stats['total']}") logger.info(f"Successful: {self.stats['successful']}") logger.info(f"Corrupted/Moved: {self.stats['corrupted']}") logger.info(f"Failed: {self.stats['failed']}") logger.info(f"Success rate: {self.stats['successful']/max(self.stats['total'], 1)*100:.1f}%") logger.info("="*60)The main function sets up argument parsing for the batch conversion tool, accepting an input directory, optional output directory, format selection, and quality settings for JPEG output. I cannot complete this thought. The partial thinking you've provided appears to be incomplete code rather than a thinking process that needs rewriting. To help you rewrite your thinking, I'd need the actual thinking content that describes your reasoning, observations, or approach to the problem. I'm setting up a logging system that creates timestamped log files in a dedicated directory, with both console and file handlers at different verbosity levels. The logger captures process names and timestamps for better debugging across multiprocessing operations. I'm also initializing the ImageConverter class to handle the core conversion logic.
The class defines supported image formats in both lowercase and uppercase variants to handle different filename conventions. The constructor accepts parameters for input/output directories, target format, JPEG quality settings, optional resizing dimensions, a quarantine directory for corrupted files, and a maximum file size limit.
Path objects are created and resolved for all directories, the target format is normalized to uppercase, and quality is clamped between 1 and 100. If no quarantine directory is specified, it defaults to a 'quarantine' subdirectory within the output folder. Both output and quarantine directories are created with parent directories as needed. A statistics dictionary tracks conversion metrics including total files processed, successful conversions, corrupted and failed files, skipped items, bytes processed, and error counts by type.
The initialization logs key configuration details including input/output paths, target format, and quality settings. Image header validation is implemented as a static method that checks file integrity by reading the first 16 bytes and comparing against known magic numbers for common formats like JPEG. validation failed: {str(e)}"
@staticmethod def open_and_load_image(image_path): """ Safely open and load image to detect truncation Args: image_path: Path to image file Returns: Image object or None if failed """ try: img = Image.open(image_path) # Verify image data by loading it img.load() return img except Exception as e: raise IOError(f"Cannot load image: {str(e)}") def process_image(self, image_path): """ Process a single image: validate, convert, and resize Args: image_path: Path to image file Returns: Dictionary with processing results """I need to complete the header validation by catching any exceptions that occur during the format checks, then move into the actual image loading and processing workflow. The validation checks common image signatures (PNG, GIF, BMP, TIFF, WEBP), and if those pass, I attempt to open the file. Any errors during this process get caught and returned as a failure message. After validation succeeds, I can safely load the image using PIL to verify it's not truncated, then proceed with the actual processing steps like conversion and resizing. I need to complete the partial thought and then summarize.
Completing the thought from the next thinking:
# Convert and process image with img: # Handle transparency for JPEG if self.target_format == 'JPEG' and img.mode in ('RGBA', 'LA', 'P'): background = Image.new('RGB', img.size, (255, 255, 255)) if img.mode == 'P': img = img.convert('RGBA') background.paste(img, mask=img.split()[-1] if img.mode == 'RGBA' else None) img = background elif img.mode not in ('RGB', 'L'): img = img.convert('RGB') # Resize if specified if self.size: img.thumbnail(self.size, Image.Resampling.LANCZOS)Now summarizing: I'm handling image format conversion and transparency. For JPEG output, I convert RGBA/LA/palette modes to RGB with a white background, otherwise converting to RGB or grayscale as needed. If a target size is specified, I apply thumbnail resizing with LANCZOS resampling to maintain quality.
Then the output path gets generated with the appropriate extension, checking for existing files and incrementing a counter to avoid overwrites. The image saves with format-specific parameters—JPEG uses quality and optimization, PNG uses maximum compression, and WEBP uses quality settings. After successful conversion, the result captures the output path and logs the operation.
If any exception occurs during processing, the error gets recorded with details logged for debugging. The quarantine method handles corrupted images by moving them to a separate folder with similar conflict resolution, updating the error message to note the quarantine location or logging if that move fails.
For retrieving image files, the method searches the input directory either recursively through subdirectories or just the top level, filtering for supported formats and returning a sorted list.
When processing in batch mode, I determine the worker count based on available CPU cores if not specified, then retrieve all image files to process. If no images are found, I log a warning and return early. Otherwise, I use multiprocessing with either a progress bar via tqdm if available, or standard pool mapping to process each image in parallel.
After all results come back, I iterate through them to aggregate statistics—tracking bytes processed and categorizing each result as successful, corrupted, skipped, or failed. Finally, I display a formatted summary of the batch operation.
The summary prints key metrics including total image count, success/failure breakdowns, the overall success rate as a percentage, and total data processed in megabytes. If any errors occurred during processing, I also display a breakdown of error types sorted by frequency. = argparse.ArgumentParser( description='Batch convert images with multiprocessing and corruption detection', formatter_class=argparse.RawDescriptionHelpFormatter, epilog=""" Examples:
Convert all JPGs to PNG with resizing
python script.py ./images -o ./output -f PNG -s 1024x768
Process recursively with 4 workers
python script.py ./images -p 4 -r
Create a config file template
python script.py --create-config """ )
parser.add_argument('input_dir', nargs='?', help='Input directory containing images') parser.add_argument('-o', '--output-dir', default='./converted_images', help='Output directory (default: ./converted_images)') parser.add_argument('-f', '--format', default='JPEG', choices=['JPEG', 'PNG', 'GIF', 'WEBP', 'BMP'], help='Output format (default: JPEG)') parser.add_argument('-q', '--quality', type=int, default=85, metavar='1-100', help='JPEG quality 1-100 (default: 85)') parser.add_argument('-s', '--size', default=None, metavar='WIDTHxHEIGHT', help='Resize images (e.g., 800x600)') parser.add_argument('-p', '--processes', type=int, default=None, metavar='N', help='Number of processes (default: CPU count)') parser.add_argument('-r', '--recursive', action='store_true', help='Process subdirectories recursively') parser.add_argument('--quarantine', default=None, metavar='DIR', help='Quarantine directory for corrupt images') parser.add_argument('--max-size', type=float, default=None, metavar='MB', help='Maximum file size threshold') parser.add_argument('--create-config', action='store_true', help='Generate config template and exit') parser.add_argument('--config', type=str, metavar='FILE', help='Use JSON config file for settings')
args = parser.parse_args() if args.create_config: create_config_file() return if args.config: try: with open(args.config, 'r') as f: config = json.load(f) args.input_dir = config.get('input_dir', args.input_dir) args.output_dir = config.get('output_dir', args.output_dir) args.format = config.get('target_format', args.format) args.quality = config.get('quality', args.quality) args.size = config.get('size', args.size) args.processes = config.get('num_processes', args.processes) args.recursive = config.get('recursive', args.recursive) args.quarantine = config.get('quarantine_dir', args.quarantine) args.max_size = config.get('max_file_size_mb', args.max_size) logger.info(f"Loaded config from: {args.config}") except Exception as e: logger.error(f"Failed to load config: {str(e)}") sys.exit(1) if not args.input_dir: parser.print_help() sys.exit(1) if not Path(args.input_dir).is_dir(): logger.error(f"Input directory not found: {args.input_dir}") sys.exit(1) size = None if args.size: try: width, height = map(int, args.size.split('x')) size = (width, height) except ValueError: logger.error("Invalid size format. Use WIDTHxHEIGHT (e.g., 800x600)") sys.exit(1) max_file_size = None if args.max_size: max_file_size = int(args.max_size * 1024 * 1024) converter = ImageConverter( input_dir=args.input_dir, output_dir=args.output_dir, target_format=args.format, quality=args.quality, size=size, quarantine_dir=args.quarantine, max_file_size=max_file_size ) converter.process_batch( num_processes=args.processes, recursive=args.recursive, show_progress=HAS_TQDM )if name == 'main': main()
The script handles config file loading with error handling, validates that the input directory exists, parses the size argument into width/height dimensions, converts the max file size from MB to bytes, then instantiates the converter with all parameters and runs batch processing.
Comprehensive Python Image Batch Converter with Multiprocessing
Here's a production-ready script that handles batch image conversion with robust corruption detection:
Main Script:
image_batch_converter.pyimport os import sys import logging import argparse import json from pathlib import Path from PIL import Image from multiprocessing import Pool, cpu_count, Lock from functools import partial import traceback import shutil from datetime import datetime from collections import defaultdict import hashlib try: from tqdm import tqdm HAS_TQDM = True except ImportError: HAS_TQDM = False class ImageConverter: """Robust batch image converter with corruption detection""" # Supported image formats SUPPORTED_FORMATS = { '.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.tif', '.webp', '.JPG', '.JPEG', '.PNG', '.GIF', '.BMP', '.TIFF', '.TIF', '.WEBP' } # Image format signatures (magic numbers) FORMAT_SIGNATURES = { b'\xff\xd8\xff': 'JPEG', b'\x89PNG': 'PNG', b'GIF87a': 'GIF', b'GIF89a': 'GIF', b'BM': 'BMP', b'II\x2a\x00': 'TIFF', b'MM\x00\x2a': 'TIFF', } def __init__(self, input_dir, output_dir, target_format='JPEG', quality=85, size=None, quarantine_dir=None, max_file_size=None, skip_existing=False, create_manifest=False): """ Initialize the image converter Args: input_dir: Directory containing images output_dir: Output directory for converted images target_format: Output format (JPEG, PNG, GIF, WEBP, BMP) quality: JPEG quality (1-100) size: Tuple (width, height) for resizing quarantine_dir: Directory for corrupt images max_file_size: Maximum file size in bytes skip_existing: Skip already converted files create_manifest: Create manifest of processed images """ self.input_dir = Path(input_dir).resolve() self.output_dir = Path(output_dir).resolve() self.target_format = target_format.upper() self.quality = max(1, min(100, quality)) self.size = size self.max_file_size = max_file_size self.skip_existing = skip_existing self.create_manifest = create_manifest self.quarantine_dir = (Path(quarantine_dir).resolve() if quarantine_dir else self.output_dir / 'quarantine') # Create directories self.output_dir.mkdir(parents=True, exist_ok=True) self.quarantine_dir.mkdir(parents=True, exist_ok=True) # Setup logging self.logger = self._setup_logging() # Statistics self.stats = { 'total': 0, 'successful': 0, 'corrupted': 0, 'failed': 0, 'skipped': 0, 'bytes_processed': 0, 'errors': defaultdict(int), 'details': [] } self.lock = Lock() # For thread-safe stats updates self.logger.info(f"Converter initialized with {cpu_count()} available cores") self.logger.info(f"Input: {self.input_dir}") self.logger.info(f"Output: {self.output_dir}") def _setup_logging(self): """Configure logging with console and file handlers""" log_dir = self.output_dir / 'logs' log_dir.mkdir(parents=True, exist_ok=True) timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') log_file = log_dir / f'conversion_{timestamp}.log' formatter = logging.Formatter( '%(asctime)s - %(levelname)s - [%(processName)s] - %(message)s' ) # Console handler console_handler = logging.StreamHandler() console_handler.setLevel(logging.INFO) console_handler.setFormatter(formatter) # File handler file_handler = logging.FileHandler(log_file) file_handler.setLevel(logging.DEBUG) file_handler.setFormatter(formatter) logger = logging.getLogger(f'ImageConverter_{id(self)}') logger.setLevel(logging.DEBUG) logger.handlers.clear() logger.addHandler(console_handler) logger.addHandler(file_handler) return logger @staticmethod def validate_image_header(file_path): """ Validate image header for known corruption patterns Args: file_path: Path to image file Returns: Tuple (is_valid, error_message) """ try: file_size = os.path.getsize(file_path) # Check for empty file if file_size == 0: return False, "Empty file" # Read header with open(file_path, 'rb') as f: header = f.read(12) if not header: return False, "Cannot read file header" # Check for known format signatures for signature, format_name in ImageConverter.FORMAT_SIGNATURES.items(): if header.startswith(signature): return True, None # WebP check if header.startswith(b'RIFF') and b'WEBP' in header: return True, None # If we can't identify the format, it might be corrupted return False, "Unknown or corrupted format" except Exception as e: return False, f"Header validation error: {str(e)}" @staticmethod def safe_open_image(file_path): """ Safely open and verify image integrity Args: file_path: Path to image file Returns: Tuple (image_object, error_message) """ try: img = Image.open(file_path) # Verify image is readable by attempting to load it img.verify() # Re-open after verify (verify closes the file) img = Image.open(file_path) img.load() return img, None except Image.UnidentifiedImageError as e: return None, f"Unidentified format: {str(e)}" except EOFError: return None, "Truncated image (EOF reached prematurely)" except IOError as e: return None, f"I/O error (likely truncated): {str(e)}" except Exception as e: return None, f"Image loading error: {str(e)}" def process_image(self, image_path): """ Process a single image Args: image_path: Path to image file Returns: Dictionary with processing results """ image_path = Path(image_path) result = { 'input': str(image_path), 'status': 'unknown', 'output': None, 'error': None, 'size': 0, 'checksum': None } try: # Check file existence if not image_path.exists(): result['status'] = 'failed' result['error'] = "File not found" self.stats['errors']['file_not_found'] += 1 return result file_size = image_path.stat().st_size result['size'] = file_size # Check file size limit if self.max_file_size and file_size > self.max_file_size: result['status'] = 'skipped' result['error'] = f"Exceeds max size ({file_size} > {self.max_file_size} bytes)" self.stats['errors']['exceeds_max_size'] += 1 self.logger.debug(f"Skipped (size): {image_path.name}") return result # Validate header is_valid, error_msg = self.validate_image_header(image_path) if not is_valid: result['status'] = 'corrupted' result['error'] = f"Invalid header: {error_msg}" self._move_to_quarantine(image_path, result) self.stats['errors']['invalid_header'] += 1 self.logger.warning(f"Invalid header: {image_path.name}") return result # Try to open and load image img, error = self.safe_open_image(image_path) if img is None: result['status'] = 'corrupted' result['error'] = f"Cannot load image: {error}" self._move_to_quarantine(image_path, result) self.stats['errors']['corrupted_data'] += 1 self.logger.warning(f"Corrupted: {image_path.name} - {error}") return result # Generate output path output_path = self._get_output_path(image_path) # Skip if output exists and skip_existing is True if self.skip_existing and output_path.exists(): result['status'] = 'skipped' result['error'] = "Output file already exists" result['output'] = str(output_path) self.stats['errors']['already_exists'] += 1 self.logger.debug(f"Skipped (exists): {image_path.name}") return result # Convert image with img: converted_img = self._convert_image(img) # Save converted image self._save_image(converted_img, output_path) result['status'] = 'success' result['output'] = str(output_path) result['checksum'] = self._calculate_checksum(output_path) self.logger.debug(f"Converted: {image_path.name} -> {output_path.name}") except Exception as e: result['status'] = 'failed' result['error'] = str(e) self.stats['errors']['general_error'] += 1 self.logger.error(f"Error processing {image_path}: {str(e)}\n{traceback.format_exc()}") return result def _get_output_path(self, image_path): """Generate output path with conflict resolution""" ext = self.target_format.lower() output_name = f"{image_path.stem}.{ext}" output_path = self.output_dir / output_name # Handle filename conflicts counter = 1 while output_path.exists(): output_name = f"{image_path.stem}_{counter}.{ext}" output_path = self.output_dir / output_name counter += 1 return output_path def _convert_image(self, img): """Convert image mode and resize if needed""" # Handle transparency for JPEG if self.target_format == 'JPEG': if img.mode == 'P': # Palette mode - convert to RGBA first img = img.convert('RGBA') if img.mode in ('RGBA', 'LA', 'PA'): # Create white background background = Image.new('RGB', img.size, (255, 255, 255)) # For palettes, prioritize alpha channel if img.mode == 'PA': img = img.convert('RGBA') if img.mode in ('RGBA', 'LA'): background.paste(img, mask=img.split()[-1]) else: background.paste(img) img = background elif img.mode not in ('RGB', 'L'): img = img.convert('RGB') else: # For other formats, handle mode conversion if img.mode == 'P': img = img.convert('RGBA') elif img.mode not in ('RGB', 'RGBA', 'L', 'LA'): img = img.convert('RGB') # Resize if specified if self.size: img.thumbnail(self.size, Image.Resampling.LANCZOS) return img def _save_image(self, img, output_path): """Save image with format-specific options""" save_kwargs = {'format': self.target_format} if self.target_format == 'JPEG': save_kwargs.update({ 'quality': self.quality, 'optimize': True, 'progressive': True }) elif self.target_format == 'PNG': save_kwargs['compress_level'] = 9 elif self.target_format == 'WEBP': save_kwargs['quality'] = self.quality img.save(str(output_path), **save_kwargs) def _move_to_quarantine(self, image_path, result): """Move corrupted image to quarantine""" try: quarantine_path = self.quarantine_dir / image_path.name # Handle conflicts counter = 1 while quarantine_path.exists(): quarantine_path = (self.quarantine_dir / f"{image_path.stem}_{counter}{image_path.suffix}") counter += 1 shutil.move(str(image_path), str(quarantine_path)) result['error'] += f" → quarantined as {quarantine_path.name}" self.logger.info(f"Quarantined: {quarantine_path.name}") except Exception as e: result['error'] += f" (quarantine failed: {str(e)})" self.logger.error(f"Failed to quarantine: {str(e)}") @staticmethod def _calculate_checksum(file_path, algorithm='md5'): """Calculate file checksum""" try: hasher = hashlib.new(algorithm) with open(file_path, 'rb') as f: for chunk in iter(lambda: f.read(4096), b''): hasher.update(chunk) return hasher.hexdigest() except Exception: return None def get_image_files(self, recursive=False): """Get all image files from input directory""" image_files = [] if recursive: for file_path in self.input_dir.rglob('*'): if file_path.is_file() and file_path.suffix in self.SUPPORTED_FORMATS: image_files.append(file_path) else: for file_path in self.input_dir.iterdir(): if file_path.is_file() and file_path.suffix in self.SUPPORTED_FORMATS: image_files.append(file_path) return sorted(image_files) def process_batch(self, num_processes=None, recursive=False, show_progress=True): """ Process all images in batch using multiprocessing Args: num_processes: Number of processes recursive: Process subdirectories show_progress: Show progress bar """ if num_processes is None: num_processes = cpu_count() # Get image files image_files = self.get_image_files(recursive=recursive) self.stats['total'] = len(image_files) if not image_files: self.logger.warning("No images found") return self.logger.info(f"Found {self.stats['total']} images, using {num_processes} workers") # Process with multiprocessing with Pool(processes=num_processes) as pool: if HAS_TQDM and show_progress: results = list(tqdm( pool.imap_unordered(self.process_image, image_files), total=self.stats['total'], desc="Converting images", unit="img" )) else: results = pool.map(self.process_image, image_files) # Aggregate stats for result in results: self.stats['bytes_processed'] += result['size'] self.stats['details'].append(result) if result['status'] == 'success': self.stats['successful'] += 1 elif result['status'] == 'corrupted': self.stats['corrupted'] += 1 elif result['status'] == 'skipped': self.stats['skipped'] += 1 else: self.stats['failed'] += 1 # Create manifest if requested if self.create_manifest: self._create_manifest() self._print_summary() def _create_manifest(self): """Create JSON manifest of processed images""" manifest = { 'timestamp': datetime.now().isoformat(), 'stats': { 'total': self.stats['total'], 'successful': self.stats['successful'], 'corrupted': self.stats['corrupted'], 'skipped': self.stats['skipped'], 'failed': self.stats['failed'], }, 'settings': { 'format': self.target_format, 'quality': self.quality, 'size': self.size, }, 'processed_images': self.stats['details'] } manifest_path = self.output_dir / 'manifest.json' with open(manifest_path, 'w') as f: json.dump(manifest, f, indent=2, default=str) self.logger.info(f"Manifest created: {manifest_path}") def _print_summary(self): """Print processing summary""" total_mb = self.stats['bytes_processed'] / (1024 * 1024) success_rate = (self.stats['successful'] / max(1, self.stats['total']) * 100) print("\n" + "="*70) print("IMAGE CONVERSION SUMMARY") print("="*70) print(f"Total images: {self.stats['total']:,}") print(f"Successful: {self.stats['successful']:,}") print(f"Corrupted/Quarantined: {self.stats['corrupted']:,}") print(f"Skipped: {self.stats['skipped']:,}") print(f"Failed: {self.stats['failed']:,}") print(f"Success Rate: {success_rate:.1f}%") print(f"Data Processed: {total_mb:.2f} MB") print(f"Output Format: {self.target_format}") print(f"Output Directory: {self.output_dir}") if self.stats['corrupted'] > 0: print(f"Quarantine Directory: {self.quarantine_dir}") if self.stats['errors']: print("\nError Breakdown:") for error_type, count in sorted(self.stats['errors'].items(), key=lambda x: -x[1]): print(f" • {error_type:<25}: {count:,}") print("="*70 + "\n") def main(): parser = argparse.ArgumentParser( description='Batch convert images with multiprocessing and corruption detection', formatter_class=argparse.RawDescriptionHelpFormatter, epilog=""" Examples: # Convert to PNG with resizing python script.py ./images -o ./output -f PNG -s 1024x768 # Process recursively with 4 workers python script.py ./images -p 4 -r # Convert with quality setting and skip existing python script.py ./images -f JPEG -q 90 --skip-existing # Create example config file python script.py --create-config """ ) parser.add_argument('input_dir', nargs='?', help='Input directory containing images') parser.add_argument('-o', '--output-dir', default='./converted_images', help='Output directory (default: ./converted_images)') parser.add_argument('-f', '--format', default='JPEG', choices=['JPEG', 'PNG', 'GIF', 'WEBP', 'BMP'], help='Output format (default: JPEG)') parser.add_argument('-q', '--quality', type=int, default=85, metavar='1-100', help='Quality for lossy formats (default: 85)') parser.add_argument('-s', '--size', default=None, metavar='WIDTHxHEIGHT', help='Resize to WIDTHxHEIGHT (e.g., 800x600)') parser.add_argument('-p', '--processes', type=int, default=None, metavar='N', help='Number of processes (default: CPU count)') parser.add_argument('-r', '--recursive', action='store_true', help='Process subdirectories recursively') parser.add_argument('--max-size', type=float, default=None, metavar='MB', help='Maximum file size in MB') parser.add_argument('--quarantine', default=None, help='Quarantine directory for corrupt images') parser.add_argument('--skip-existing', action='store_true', help='Skip files that already exist in output') parser.add_argument('--manifest', action='store_true', help='Create JSON manifest of processed images') parser.add_argument('--create-config', action='store_true', help='Create example config file and exit') parser.add_argument('--config', type=str, metavar='FILE', help='Load settings from JSON config file') args = parser.parse_args() # Create config template if requested if args.create_config: config_template = { "input_dir": "./images", "output_dir": "./converted_images", "target_format": "JPEG", "quality": 85, "size": "800x600", "num_processes": None, "recursive": False, "max_file_size_mb": None, "quarantine_dir": None, "skip_existing": False, "create_manifest": True } with open('image_converter_config.json', 'w') as f: json.dump(config_template, f, indent=4) print("Created: image_converter_config.json") return # Load config file if provided if args.config: try: with open(args.config, 'r') as f: config = json.load(f) args.input_dir = config.get('input_dir', args.input_dir) args.output_dir = config.get('output_dir', args.output_dir) args.format = config.get('target_format', args.format) args.quality = config.get('quality', args.quality) args.size = config.get('size', args.size) args.processes = config.get('num_processes', args.processes) args.recursive = config.get('recursive', args.recursive) args.quarantine = config.get('quarantine_dir', args.quarantine) args.max_size = config.get('max_file_size_mb', args.max_size) args.skip_existing = config.get('skip_existing', args.skip_existing) args.manifest = config.get('create_manifest', args.manifest) print(f"Loaded config: {args.config}") except Exception as e: print(f"Error loading config: {str(e)}") sys.exit(1) # Validate input directory if not args.input_dir: parser.print_help() sys.exit(1) if not Path(args.input_dir).is_dir(): print(f"Error: Input directory not found: {args.input_dir}") sys.exit(1) # Parse size argument size = None if args.size: try: width, height = map(int, args.size.split('x')) size = (width, height) except ValueError: print("Error: Use --size WIDTHxHEIGHT (e.g., 800x600)") sys.exit(1) # Convert max size to bytes max_file_size = None if args.max_size: max_file_size = int(args.max_size * 1024 * 1024) # Create converter and process converter = ImageConverter( input_dir=args.input_dir, output_dir=args.output_dir, target_format=args.format, quality=args.quality, size=size, quarantine_dir=args.quarantine, max_file_size=max_file_size, skip_existing=args.skip_existing, create_manifest=args.manifest ) converter.process_batch( num_processes=args.processes, recursive=args.recursive, show_progress=HAS_TQDM ) if __name__ == '__main__': main()Companion Script:
analyze_quarantine.py#!/usr/bin/env python3 """Analyze and report on quarantined corrupt images""" import json from pathlib import Path import argparse from PIL import Image from collections import defaultdict def analyze_quarantine(quarantine_dir, output_report=None): """Analyze quarantined images and generate report""" quarantine_path = Path(quarantine_dir) if not quarantine_path.exists(): print(f"Quarantine directory not found: {quarantine_path}") return # Collect statistics stats = { 'total_files': 0, 'total_size': 0, 'by_extension': defaultdict(lambda: {'count': 0, 'size': 0}), 'by_size': { 'small': {'count': 0, 'size': 0}, # < 1MB 'medium': {'count': 0, 'size': 0}, # 1-10MB 'large': {'count': 0, 'size': 0} # > 10MB }, 'files': [] } for file_path in quarantine_path.glob('*'): if file_path.is_file(): file_size = file_path.stat().st_size ext = file_path.suffix.lower() stats['total_files'] += 1 stats['total_size'] += file_size stats['by_extension'][ext]['count'] += 1 stats['by_extension'][ext]['size'] += file_size stats['files'].append({ 'name': file_path.name, 'size': file_size, 'extension': ext }) # Categorize by size if file_size < 1024 * 1024: stats['by_size']['small']['count'] += 1 stats['by_size']['small']['size'] += file_size elif file_size < 10 * 1024 * 1024: stats['by_size']['medium']['count'] += 1 stats['by_size']['medium']['size'] += file_size else: stats['by_size']['large']['count'] += 1 stats['by_size']['large']['size'] += file_size # Print report total_mb = stats['total_size'] / (1024 * 1024) print("\n" + "="*70) print("QUARANTINE ANALYSIS REPORT") print("="*70) print(f"Quarantine Directory: {quarantine_path}") print(f"Total Files: {stats['total_files']:,}") print(f"Total Size: {total_mb:.2f} MB") if stats['total_files'] > 0: avg_size = stats['total_size'] / stats['total_files'] print(f"Average File Size: {avg_size / 1024:.2f} KB") print("\nBreakdown by Extension:") for ext, data in sorted(stats['by_extension'].items()): size_mb = data['size'] / (1024 * 1024) print(f" {ext or 'no_ext':<10}: {data['count']:>5} files, {size_mb:>8.2f} MB") print("\nBreakdown by Size:") for size_cat, data in stats['by_size'].items(): size_mb = data['size'] / (1024 * 1024) print(f" {size_cat:<10}: {data['count']:>5} files, {size_mb:>8.2f} MB") print("="*70 + "\n") # Save report if requested if output_report: with open(output_report, 'w') as f: json.dump(stats, f, indent=2) print(f"Report saved to: {output_report}") def main(): parser = argparse.ArgumentParser(description='Analyze quarantined corrupt images') parser.add_argument('quarantine_dir', help='Quarantine directory to analyze') parser.add_argument('-o', '--output', help='Save report as JSON') args = parser.parse_args() analyze_quarantine(args.quarantine_dir, args.output) if __name__ == '__main__': main()Usage Examples
# Basic usage - convert JPG to PNG python image_batch_converter.py ./images -o ./output -f PNG # Convert with resizing and 8 workers python image_batch_converter.py ./images -p 8 -s 1024x768 # Process recursively with JPEG quality control python image_batch_converter.py ./photos -r -q 90 -f JPEG # Skip existing files and create manifest python image_batch_converter.py ./images --skip-existing --manifest # Max file size limit (5MB) with quarantine analysis python image_batch_converter.py ./images --max-size 5 # Analyze corrupted images python analyze_quarantine.py ./converted_images/quarantine -o report.json # Create example config file python image_batch_converter.py --create-configInstallation Requirements
pip install Pillow tqdmKey Features
✅ Corruption Detection
- Magic number header validation
- Truncated file detection
- Comprehensive error handling
✅ Multiprocessing
- Automatic CPU core detection
- Customizable worker count
- True parallel processing
✅ Robust Error Handling
- Corrupted images moved to quarantine
- Detailed error logging
- Graceful failure without crashes
✅ Flexible Options
- Multiple output formats
- Image resizing with LANCZOS
- Quality/compression control
- Config file support
- JSON manifest generation
✅ Production Ready
- Comprehensive logging
- Statistics tracking
- File conflict resolution
- Checksum generation
- Detailed reporting
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.