from PIL import Image import requests from transformers import BlipProcessor,…
- User
from PIL import Image import requests from transformers import BlipProcessor, BlipForConditionalGeneration
Load the BLIP model and processor
model_name = "Salesforce/blip-image-captioning-base" processor = BlipProcessor.from_pretrained(model_name) model = BlipForConditionalGeneration.from_pretrained(model_name)
Load the image
image_url = "path_to_your_image.jpg" # Replace with your image path or URL image = Image.open("/Users/saurabhsingh/Desktop/plant.jpg")
Preprocess the image
inputs = processor(images=image, return_tensors="pt")
Generate captions
outputs = model.generate(**inputs) caption = processor.decode(outputs[0], skip_special_tokens=True)
print("Generated Caption:", caption)
Integrate above code with
import pandas as pd from duckduckgo_search import DDGS
def search(keywords): search_query = keywords
# Fetch search results from DuckDuckGo results = DDGS().text( keywords=search_query, region='wt-wt', safesearch='off', max_results=20 # Limit the number of results ) # Print the results for verification print(results) results_df = pd.DataFrame(results) results_df.to_csv(f'{''.join(keywords.split())}.csv', index=False)search('output')
Output should be the one given by the one caption given by above
- Assistant
To integrate the image captioning code with a web search function, you can modify the code to use the generated caption as the search query. You will also need to ensure that the necessary libraries are imported and used correctly. Below is an example of how you can combine both functionalities:
from PIL import Image import requests from transformers import BlipProcessor, BlipForConditionalGeneration import pandas as pd from duckduckgo_search import DDGS # Load the BLIP model and processor model_name = "Salesforce/blip-image-captioning-base" processor = BlipProcessor.from_pretrained(model_name) model = BlipForConditionalGeneration.from_pretrained(model_name) # Load the image image_path = "/Users/saurabhsingh/Desktop/plant.jpg" # Specify your image path image = Image.open(image_path) # Preprocess the image inputs = processor(images=image, return_tensors="pt") # Generate captions outputs = model.generate(**inputs) caption = processor.decode(outputs[0], skip_special_tokens=True) print("Generated Caption:", caption) def search_with_caption(caption): search_query = caption # Use the generated caption as the search query # Fetch search results from DuckDuckGo results = DDGS().text( keywords=search_query, region='wt-wt', safesearch='off', max_results=20 # Limit the number of results ) # Print the results for verification for result in results: print(result) # Convert results to DataFrame and save to CSV results_df = pd.DataFrame(results) filename = f"{''.join(search_query.split())}.csv" results_df.to_csv(filename, index=False) print(f"Search results saved to {filename}") # Search using the generated caption search_with_caption(caption)Key Points:
-
Image Path: Ensure that
image_pathpoints to the actual image file you are using. You can replace this with a URL if you're downloading the image from the web. -
Search Function: The
search_with_captionfunction takes the generated caption and uses it to search with DuckDuckGo. The results are then saved to a CSV file named after the caption (with spaces removed). -
Output Verification: The search results are printed for verification, and the CSV file contains all relevant details from the DuckDuckGo search.
This code integrates the generation of image captions with a web search process, effectively using the caption as the search query.
-