Skip to content
TheChatArchiveThe Chat Archive
gpt-3.5-turbo-0613· openaiHugging Face Suzhen/CodeChat

用scrapy框架从网址'https://www.weather.com.cn/textFC/chongqing.shtml'爬取重庆及各区县的天气预报,给出具体代码

  1. User
  2. Assistant

    下面给出了使用Scrapy框架爬取重庆市及各区县天气预报的具体代码:

    import scrapy
    
    class WeatherSpider(scrapy.Spider):
        name = "weather"
        start_urls = [
            'https://www.weather.com.cn/textFC/chongqing.shtml'
        ]
    
        def parse(self, response):
            district_nodes = response.xpath('//div[@class="conMidtab"][1]//tr[@class=""]')
            for node in district_nodes:
                district_name = node.xpath('.//td[1]/a/text()').get()
                district_temperature = node.xpath('.//td[2]/text()').get()
                district_weather = node.xpath('.//td[3]/text()').get()
                
                yield {
                    'district_name': district_name,
                    'district_temperature': district_temperature,
                    'district_weather': district_weather
                }
    

    你可以将以上代码保存为名为 weather_spider.py 的Python文件。然后在命令行中使用以下命令运行爬虫:

    scrapy runspider weather_spider.py -o weather.csv
    

    这将在当前目录下生成一个名为 weather.csv 的CSV文件,其中包含了重庆市及各区县的天气预报数据。

    注意:在运行爬虫之前,确保已经安装了Scrapy库。你可以使用以下命令在终端中安装Scrapy:

    pip install scrapy
    

    此外,还需要确保你已经安装了适用于你的Python版本的Twisted库。你可以使用以下命令安装Twisted:

    pip install twisted
    

    希望对你有所帮助!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.