Write a Python program that crawls data from yjc.ir based on tags or titles or topics.…
- User
Write a Python program that crawls data from yjc.ir based on tags or titles or topics. The program receives input from the user and crawls the data as requested by the user.
- Assistant
To crawl data from yjc.ir based on tags, titles, or topics, you can use the requests and BeautifulSoup libraries in Python. Here's an example program that receives user input and fetches data from the website accordingly.
import requests from bs4 import BeautifulSoup def crawl_data(tag=None, title=None, topic=None): base_url = "https://www.yjc.ir" # Creating the search query based on user input if tag: url = f"{base_url}/fa/tags/{tag}" elif title: url = f"{base_url}/fa/news/{title}" elif topic: url = f"{base_url}/fa/news/{topic}" else: print("No search parameter provided.") return # Sending a GET request to the website response = requests.get(url) if response.status_code == 200: soup = BeautifulSoup(response.content, "html.parser") # Parsing the crawled data based on the website structure if tag or title: news_items = soup.find_all("div", class_="title") for item in news_items: print(item.text.strip()) elif topic: news_items = soup.find_all("div", class_="news-title") for item in news_items: print(item.text.strip()) else: print("Failed to fetch data from the website.") # Taking user input to specify the search search_type = input("Enter the search type (tag, title, topic): ") if search_type == "tag": tag = input("Enter the tag: ") crawl_data(tag=tag) elif search_type == "title": title = input("Enter the title: ") crawl_data(title=title) elif search_type == "topic": topic = input("Enter the topic: ") crawl_data(topic=topic) else: print("Invalid search type.")In this program, the
crawl_data()function takes user input (tag, title, or topic) to construct the search query and fetch the data from yjc.ir accordingly. The function uses the BeautifulSoup library to parse the HTML content of the website and extract the desired information. Finally, it prints the extracted data.The program prompts the user to specify the search type and required parameter, then calls the
crawl_data()function accordingly.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.