Skip to content
The Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)

I have written some code to get the rental data from a certain website here is the…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    I have written some code to get the rental data from a certain website here is the following javascript code.

    const axios = require('axios');
    const cheerio = require('cheerio');
    
    const getData = async () => {
      const response = await axios.get('https://www.realestate.com.au/rent/in-sydney,+nsw/list-1');
      const $ = cheerio.load(response.data);
      
      const properties = [];
    
      $('.listing-result').each((i, el) => {
        const property = {};
    
        property.title = $(el)
          .find('.property-title')
          .text()
          .trim();
        property.address = $(el)
          .find('.property-address')
          .text()
          .trim();
        property.price = $(el)
          .find('.priceText')
          .text()
          .trim();
    
        properties.push(property);
      });
    
      return properties;
    };
    
    getData().then((properties) => {
      console.log(properties);
    });
    

    Why does the following error occur? "AxiosError: Request failed with status code 429"

  2. Assistant

    The HTTP status code 429 Too Many Requests indicates that the server has some sort of rate limit and that your request exceeded it. This could be due to any of the following reasons:

    1. The code which makes this request was run many times in a short period, exceeding the rate limit.

    2. The website rejects requests which it suspects are spam with a 429 status code. This could mean that the user agent is not a common browser, which is a simple check some websites use. To rectify this, you could add the "User-Agent" header to the GET request like so:

    const response = await axios.get('https://www.realestate.com.au/rent/in-sydney,+nsw/list-1', 
         { headers: { 'User-Agent': 'browser.version' }  }
    );
    

    Note that you will have to replace "browser.version" with an actual browser user agent string. Depending on the application this code is being run in, this could be considered user agent spoofing, which may be restricted under the laws of some countries or organizations (or the terms of service of the website that the code attempts to access).

    1. The website is rate limiting requests for some other reason. This could be due to the request being made through a proxy, in which case other users of the proxy may have been making too many requests, although this is unlikely.

    2. The website is rejecting all requests because it is under a denial of service attack. This is unlikely, as the typical behavior of a server under such an attack would be to route traffic to a non-existent server, or deny it completely, giving no valid response at all. However, even if the website did respond with an HTTP status code, it would likely be server error 503 Service Unavailable, given when the server is overloaded.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.