Skip to content
TheChatArchiveThe Chat Archive
o4-mini-2025-04-16· OpenAIHugging Face LMArena Human Preference 140K

help me to fetch a page and get js property from that page

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with grok-4-0709’s answer.
  1. User

    help me to fetch a page and get js property from that page

  2. Assistant

    Here are a few approaches you can take, depending on whether the JavaScript property you need is already in the HTML source (e.g. a global var foo = …;) or is only exposed once the page’s JS runs (e.g. after AJAX calls or client‐side rendering).


    1. If the JS property lives in a <script> tag in the HTML

    Say the page contains somewhere:

    <script>
      window.MY_CONFIG = {
        apiEndpoint: "https://api.example.com",
        version: "1.2.3"
      };
    </script>
    

    You can fetch the raw HTML and use a regex (or a small parser) to extract it.

    Node.js + node-fetch + regex

    // npm install node-fetch
    const fetch = require('node-fetch');
    
    async function fetchConfig(url) {
      const res = await fetch(url);
      const text = await res.text();
    
      // crude regex to pull out the JSON in window.MY_CONFIG = { … };
      const match = text.match(/window\.MY_CONFIG\s*=\s*(\{[\s\S]*?\});/);
      if (!match) throw new Error("MY_CONFIG not found");
    
      const config = JSON.parse(match[1]);
      return config;
    }
    
    (async () => {
      const cfg = await fetchConfig('https://www.example.com');
      console.log(cfg.apiEndpoint, cfg.version);
    })();
    

    2. If the property is set only after the page’s JavaScript runs

    In that case you need a headless browser so the JS will execute, then you can read window.whatever via evaluate().

    A. Node.js + Puppeteer

    // npm install puppeteer
    const puppeteer = require('puppeteer');
    
    async function getProp(url, propPath) {
      const browser = await puppeteer.launch();
      const page = await browser.newPage();
      await page.goto(url, { waitUntil: 'networkidle0' });
    
      // propPath could be 'MY_CONFIG.apiEndpoint' or 'myGlobalVar'
      const value = await page.evaluate(p => {
        // safely dig into window
        return p.split('.').reduce((o, key) => o && o[key], window);
      }, propPath);
    
      await browser.close();
      return value;
    }
    
    (async () => {
      const apiEndpoint = await getProp('https://www.example.com', 'MY_CONFIG.apiEndpoint');
      console.log('API endpoint is', apiEndpoint);
    })();
    

    B. Python + Selenium

    # pip install selenium webdriver-manager
    from selenium import webdriver
    from selenium.webdriver.chrome.service import Service
    from webdriver_manager.chrome import ChromeDriverManager
    
    def get_js_property(url, prop_path):
        service = Service(ChromeDriverManager().install())
        options = webdriver.ChromeOptions()
        options.add_argument('--headless')
        driver = webdriver.Chrome(service=service, options=options)
        driver.get(url)
    
        # e.g. prop_path = 'MY_CONFIG.apiEndpoint'
        script = f"return {prop_path};"
        value = driver.execute_script(script)
        driver.quit()
        return value
    
    if __name__ == '__main__':
        ep = get_js_property('https://www.example.com', 'window.MY_CONFIG.apiEndpoint')
        print('API endpoint =', ep)
    

    3. A middle‐ground: jsdom (for simple pages)

    If there’s no AJAX or dynamic rendering, you can pull HTML and let jsdom simulate window. jsdom won’t run external scripts by default, but you can inline them:

    // npm install node-fetch jsdom
    const fetch = require('node-fetch');
    const { JSDOM } = require('jsdom');
    
    async function getPropViaJsdom(url, propPath) {
      const res = await fetch(url);
      const html = await res.text();
    
      const dom = new JSDOM(html, { runScripts: "dangerously", resources: "usable" });
      await new Promise(r => dom.window.addEventListener("load", r));
    
      // access nested property
      const value = propPath.split('.').reduce((o, k) => o && o[k], dom.window);
      return value;
    }
    
    (async () => {
      const v = await getPropViaJsdom('https://www.example.com', 'MY_CONFIG.version');
      console.log(v);
    })();
    

    Which should you pick?

    • Simple static extraction (JS object literal in HTML): use the regex approach.
    • Full JS execution needed (SPA or dynamic data): use Puppeteer (Node) or Selenium (Python).
    • Minimal “DOM” without a full browser: try jsdom, knowing it may not handle all dynamic scripts.

    Let me know which scenario matches yours or if you’d like code tuned further!

  3. User

    i mean provide console snippet

  4. Assistant

    Here’s a single‐paste snippet you can drop into your browser DevTools Console. It will fetch any URL, scrape out a JS object literal assigned to a global, and log it:

    ;(async ()=> {
      // 1) Edit these two values:
      const url     = 'https://www.example.com';      // ← the page you want to fetch
      const varName = 'window.MY_CONFIG';             // ← the global you’re after
    
      // 2) Fetch the HTML
      const res  = await fetch(url);
      const text = await res.text();
    
      // 3) Build a regex to grab the “= { … };” that follows your varName
      const re   = new RegExp(varName.replace('.','\\.') + '\\s*=\\s*(\\{[\\s\\S]*?\\});');
      const m    = text.match(re);
    
      if (!m) {
        console.error(`❌ ${varName} not found in the HTML of ${url}`);
        return;
      }
    
      // 4) Parse & log
      let obj;
      try {
        obj = JSON.parse(m[1]);
      } catch (err) {
        console.error('❌ Failed to JSON.parse:', err);
        return;
      }
      console.log(`✅ ${varName} =`, obj);
      return obj;  // returns it too, in case you want to assign it
    })();
    

    How it works:

    1. You fetch() the raw HTML of the target page.
    2. A quick regex looks for e.g.
      window.MY_CONFIG = { … };
      
    3. We JSON.parse the { … } chunk and spit it back to you.

    —
    If the JS property you need isn’t inlined (i.e. it only exists after the page’s own scripts run), you’d have to spin up a headless browser (Puppeteer/Selenium). But for simple inline globals, the above console snippet should do the trick!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.