Write a program in Python that takes the last 10 sports news from the site yjc.ir and…
- User
Write a program in Python that takes the last 10 sports news from the site yjc.ir and stores it in the redis database. Then, every n hours, it receives the last 10 news of the site and stores it in the database. If the messages are duplicates, do not save them. Get n from the user. Please explain the steps in order. This program stores the content of each news page in the database. For example, links to photos and videos, news text, news URL, news date, etc.
- Assistant
To complete this task, you can follow the following steps:
-
Install the required packages: Make sure you have the
redisandrequestspackages installed in your Python environment to interact with Redis and fetch web pages respectively. You can install them usingpipwith the following command:pip install redis requests -
Import the necessary libraries: In your Python script, start by importing the required libraries.
import redis import requests from bs4 import BeautifulSoup -
Connect to Redis: Establish a connection with the Redis database.
r = redis.Redis(host='localhost', port=6379, db=0) -
Fetch and parse the latest news: Write a function that fetches the last 10 news from the specified website using the
requestslibrary, and then parses the necessary information usingBeautifulSoup. Store the relevant information into a dictionary or a JSON object.def fetch_latest_news(): url = 'https://www.yjc.ir' response = requests.get(url) response.raise_for_status() soup = BeautifulSoup(response.content, 'html.parser') news_elements = soup.find_all('div', class_='news') news_data = [] for news_element in news_elements[:10]: news_title = news_element.find('h2').text news_url = url + news_element.find('a')['href'] news_date = news_element.find('time')['datetime'] # Add any other required information news_data.append({'title': news_title, 'url': news_url, 'date': news_date}) return news_data -
Store the latest news in Redis: Retrieve the last 10 news using the
fetch_latest_newsfunction and store the news data in the Redis database. Before storing, check if the news already exists in the database using the provided URL or any other identifying details.def store_news(): latest_news = fetch_latest_news() for news in latest_news: news_url = news['url'] # Check if the news already exists in Redis if not r.exists(news_url): # Store news data in Redis r.hmset(news_url, news) -
Schedule the program to run periodically: Prompt the user to enter the interval
nin hours, and run the program repeatedly after everynhours.interval = int(input("Enter the interval in hours: ")) while True: store_news() time.sleep(interval * 3600)
This program fetches the last 10 news from the specified website, stores them in the Redis database, and periodically checks for new news in the given interval. Duplicate news is not saved by checking if the news URL already exists in the database. You can extend the
fetch_latest_newsfunction to extract and store additional information as required. -
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.