Skip to content
TheChatArchiveThe Chat Archive
gpt-3.5-turbo-0301· openaiHugging Face Suzhen/CodeChat

给我展示一个爬虫的例子

  1. User

    给我展示一个爬虫的例子

  2. Assistant

    以下是一个简单的Python爬虫代码示例,可以从网站上抓取新闻标题和链接:

    import requests
    from bs4 import BeautifulSoup
    
    url = 'http://www.example.com/news/'
    
    response = requests.get(url)
    html = response.content
    
    soup = BeautifulSoup(html, 'html.parser')
    
    news_list = soup.find_all('div', class_='news-item')
    
    for news in news_list:
        title = news.a.get('title')
        link = news.a.get('href')
        print(title, link)
    

    这个爬虫使用了requests库来获取网站页面,使用了BeautifulSoup库来解析HTML。它找到了页面中所有包含新闻信息的DIV和类'news-item',然后从每个DIV中抓取了新闻标题和链接,并将它们输出到控制台。这个爬虫可以轻松地扩展,以抓取更多信息或不同类型的页面。

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.