please give me python code to scrape the web and save best fit of Images of following:…
- User
please give me python code to scrape the web and save best fit of Images of following: علائم + following list: بانک آمبولانس بیمارستان آتش نشانی پلیس بیمارستان اداره آب و فاضلاب اداره گاز برق خطرناک هلال احمر
- Assistant
Certainly! Below is a Python script that uses the
icrawlerlibrary to search and download images for the specified Persian terms. This script automates image scraping from Google Images and saves them into organized directories.Prerequisites
-
Python Installation: Ensure you have Python 3.x installed on your system. You can download it from here.
-
Install Required Libraries:
- icrawler: A powerful image crawler framework.
- pip: Python package installer (usually comes with Python).
Open your terminal or command prompt and run:
pip install icrawler
Python Script
import os from icrawler.builtin import GoogleImageCrawler # Define the list of keywords in Persian keywords = [ "بانک", "آمبولانس", "بیمارستان", "آتش نشانی", "پلیس", "اداره آب و فاضلاب", "اداره گاز", "برق خطرناک", "هلال احمر" ] # Directory where images will be saved output_dir = "downloaded_images" # Number of images to download per keyword num_images = 20 # You can adjust this number as needed def create_directory(path): """Create directory if it doesn't exist.""" if not os.path.exists(path): os.makedirs(path) def download_images(keyword, save_dir, max_num): """Download images for a given keyword.""" google_crawler = GoogleImageCrawler(storage={'root_dir': save_dir}) print(f"Downloading images for keyword: {keyword}") google_crawler.crawl( keyword=keyword, max_num=max_num, min_size=(200, 200), # Minimum size of images file_idx_offset='auto' ) print(f"Finished downloading images for keyword: {keyword}\n") def main(): # Create the main output directory create_directory(output_dir) for keyword in keywords: # Create a subdirectory for each keyword keyword_dir = os.path.join(output_dir, keyword) create_directory(keyword_dir) # Download images download_images(keyword, keyword_dir, num_images) print("All images have been downloaded successfully.") if __name__ == "__main__": main()How the Script Works
-
Import Libraries:
os: For handling directory operations.GoogleImageCrawlerfromicrawler.builtin: To perform image crawling from Google Images.
-
Define Keywords:
- The
keywordslist contains all the Persian terms for which you want to download images.
- The
-
Set Output Directory and Image Count:
output_dir: The main directory where all images will be stored.num_images: Number of images to download per keyword. Adjust this based on your requirements.
-
Directory Creation:
- The
create_directoryfunction ensures that the required directories exist. If not, it creates them.
- The
-
Downloading Images:
- The
download_imagesfunction initializes the Google Image Crawler for each keyword and downloads the specified number of images into the corresponding directory. min_size: Ensures that only images above a certain resolution are downloaded to maintain quality.file_idx_offset='auto': Automatically manages file indexing to prevent overwriting.
- The
-
Execution Flow:
- The
mainfunction orchestrates the creation of directories and initiates the download process for each keyword. - After processing all keywords, it prints a completion message.
- The
Running the Script
-
Save the Script:
- Save the above Python script in a file named, for example,
download_images.py.
- Save the above Python script in a file named, for example,
-
Run the Script:
-
Navigate to the directory containing
download_images.pyusing your terminal or command prompt. -
Execute the script by running:
python download_images.py
-
-
Monitor the Download:
- The script will print messages indicating the progress of image downloads for each keyword.
- Images will be saved in the
downloaded_imagesdirectory, organized into subdirectories named after each keyword.
Notes and Best Practices
-
Respect Copyright:
- Ensure that the images downloaded are used in compliance with their respective licenses. The
icrawlerlibrary downloads images based on their availability on the web, but it doesn't check for usage rights.
- Ensure that the images downloaded are used in compliance with their respective licenses. The
-
Adjust Image Count:
- Be mindful of the number of images you download to avoid excessive bandwidth usage or potential blocking by search engines. Adjust the
num_imagesvariable as needed.
- Be mindful of the number of images you download to avoid excessive bandwidth usage or potential blocking by search engines. Adjust the
-
Handle Duplicates:
- The script does not handle duplicate images. If you run the script multiple times, it might download the same images again. Implement additional checks if this is a concern.
-
Performance Optimization:
- For downloading a large number of images or handling many keywords, consider implementing multithreading or asynchronous processing to speed up the process.
-
Error Handling:
- The script has basic print statements for progress tracking. For a more robust solution, consider adding try-except blocks to handle potential errors during the crawling process.
Conclusion
This script provides a straightforward way to automate the downloading of images based on Persian keywords using Python. By leveraging the
icrawlerlibrary, you can efficiently gather a collection of images for various applications. Always ensure that you use the downloaded images responsibly and adhere to copyright laws. -