Write a Python program that checks daily yjc.ir website news by date, tags and topics.
- User
Write a Python program that checks daily yjc.ir website news by date, tags and topics.
- Assistant
To check the daily news on the yjc.ir website by date, tags, and topics, we can use web scraping techniques along with Python's requests and BeautifulSoup libraries.
Here's an example program that retrieves and filters the news based on the specified criteria:
import requests from bs4 import BeautifulSoup # Function to check if the news matches the specified criteria def check_news(news, date, tags, topics): news_date = news.find('span', class_='show-fa-num').text.strip() news_tags = [tag.text.strip() for tag in news.find_all('li', class_='left-inline')] news_topics = [topic.text.strip() for topic in news.find_all('a', class_='show-topic')] if date and not news_date.startswith(date): return False if tags and not set(tags).issubset(news_tags): return False if topics and not set(topics).issubset(news_topics): return False return True # User input for search criteria date = input("Enter the date (format: yyyy-mm-dd): ") tags = input("Enter the tags (comma-separated): ").split(',') topics = input("Enter the topics (comma-separated): ").split(',') # URL for the yjc.ir news page url = 'https://www.yjc.ir/fa/archive/all/0' # Send a GET request to the website response = requests.get(url) # Create a BeautifulSoup object to parse the HTML content soup = BeautifulSoup(response.content, 'html.parser') # Find all the news articles news_list = soup.find_all('div', class_='item-title') # Check and filter the news based on search criteria filtered_news = [news for news in news_list if check_news(news, date, tags, topics)] # Print the filtered news headlines print("\nFiltered News Headlines:") for news in filtered_news: print(news.text.strip())Now, when you run the program, it will prompt you to enter the desired search criteria. You can provide a specific date, comma-separated tags, and comma-separated topics. The program will then scrape the yjc.ir website, filter the news based on the given criteria, and print the filtered news headlines.
Note: Web scraping should be done ethically and responsibly. Make sure to comply with the website's terms of service and do not overwhelm the server with frequent requests.
- User
Have a rest server where a user can submit a task via scraper and the submitted task is stored in a SQL database. (Choose the appropriate database for this task)
- Assistant
To create a REST server where a user can submit a task via a scraper and store the submitted task in a SQL database, we can use Python's Flask framework for creating the server and SQLite as the appropriate database for this task.
Here's an example of how you can set up the server and store the submitted tasks in an SQLite database:
-
Install the required packages:
pip install flask flask_sqlalchemy -
Create a
main.pyfile with the following code:
from flask import Flask, request from flask_sqlalchemy import SQLAlchemy app = Flask(__name__) app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///tasks.db' db = SQLAlchemy(app) # Task Model class Task(db.Model): id = db.Column(db.Integer, primary_key=True) scraper = db.Column(db.String(100)) task = db.Column(db.String(100)) def __init__(self, scraper, task): self.scraper = scraper self.task = task # Create the database tables db.create_all() # Endpoint for submitting a task @app.route('/submit_task', methods=['POST']) def submit_task(): scraper = request.json['scraper'] task = request.json['task'] # Create a new task object new_task = Task(scraper=scraper, task=task) # Add the task to the database db.session.add(new_task) db.session.commit() return "Task submitted successfully!" if __name__ == '__main__': app.run()- Run the server using the following command:
python main.py
The server is now up and running, and you can send a POST request to
http://localhost:5000/submit_taskwith the scraper and task details in JSON format. The server will store the submitted tasks in the SQLite database namedtasks.dbin the same directory as themain.pyfile.To interact with the server, you can use tools like cURL or Postman to send the POST requests.
Make sure to handle authentication, validation, and any other necessary security measures as per your application requirements.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.