Skip to content
TheChatArchiveThe Chat Archive
Hunyuanhunyuan-turbos-20250416· TencentHugging Face LMArena Human Preference 140K

When tracing a site (built with Shopify) via puppeteer it appears that JS is not…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-opus-4-20250514-thinking-16k’s answer.
  1. User

    When tracing a site (built with Shopify) via puppeteer it appears that JS is not executed. JS execution is enabled and works on other sites. Why does it not work on this one and how to make it work.

    Screenshots lack elements added with JS and the trace doesn't include main thread JS activity.

  2. Hunyuan Assistant

    When Puppeteer fails to execute JavaScript on a Shopify site — while it works fine on others — yet you've confirmed that JavaScript execution is enabled, the issue likely lies in how Shopify serves its content, especially under certain conditions like:

    • Deferred or asynchronous loading
    • Server-side rendering (SSR) or hydration delays
    • Anti-bot measures (like Cloudflare or Shopify bot protection)
    • Navigation timing and resource loading quirks

    Let’s walk through why this happens and how to fix/troubleshoot it.


    ✅ 1. Ensure Full Page Load & JavaScript Execution Time

    Shopify uses progressive hydration and sometimes loads critical parts of the page after the initial HTML is served. If Puppeteer doesn’t wait long enough, it may take a screenshot or create a trace before JS-rendered content appears.

    🔧 Solution: Wait for Network Idle or Specific Elements

    Use page.goto with proper wait conditions:

    await page.goto('https://yourshopifystore.com', {
      waitUntil: 'networkidle', // or 'domcontentloaded' + manual wait
    });
    

    Better yet, wait for a selector that’s rendered by JavaScript:

    await page.goto('https://yourshopifystore.com');
    await page.waitForSelector('.product-grid'); // Replace with actual dynamic element
    

    If needed, add a delay to debug:

    await page.waitForTimeout(5000); // wait 5 seconds
    

    ✅ 2. Check for Lazy-Loaded or Client-Side Rendered Content

    Shopify Liquid templates render server-side, but many themes (e.g., Debut, Dawn, Turbo) rely heavily on JavaScript for:

    • Product recommendations
    • Cart functionality
    • Reviews (like Yotpo, Judge.me)
    • Sliders/carousels
    • Navigation menus

    These often load after DOMContentLoaded.

    🔍 Debug Tip:

    Take a screenshot immediately after goto, then another after a delay. If the second one includes missing elements, it confirms async JS rendering.


    ✅ 3. Enable JavaScript and Disable Caching (Just in Case)

    Double-check your Puppeteer launch config:

    const browser = await puppeteer.launch({
      headless: 'new', // Try 'new' instead of true/false in newer Puppeteer
      args: ['--disable-site-isolation-trials'], // optional
    });
    
    const page = await browser.newPage();
    await page.setJavaScriptEnabled(true); // Usually enabled by default, but safe to ensure
    await page.setViewport({ width: 1280, height: 800 });
    

    Note: setJavaScriptEnabled(true) is usually redundant in recent Puppeteer versions, as JS is enabled by default.


    ✅ 4. Bypass Bot Detection (Cloudflare, Shopify Bot Protection)

    Some Shopify stores use bot protection mechanisms (especially via Cloudflare or Shopify's own anti-scraping). These can serve simplified or empty pages to Puppeteer if it doesn't look like a real browser.

    🔐 Solution: Make Puppeteer More Browser-like

    Set proper headers, including a real-looking user-agent:

    await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36');
    

    You can also enable additional browser features:

    await page.evaluateOnNewDocument(() => {
      Object.defineProperty(navigator, 'webdriver', { get: () => false });
    });
    

    Optional: Use stealth plugins like puppeteer-extra-plugin-stealth


    ✅ 5. Trace Doesn’t Show JS Activity?

    If your trace lacks main thread JS activity, it suggests Puppeteer isn’t actually running JS — possibly due to:

    • Navigation aborting or redirecting too early
    • Site detecting headless and serving fallback page
    • headless: true behaving differently than expected

    🛠️ Try:

    • Use headless: 'new' (Puppeteer v19+)
    • Or run in non-headless for debugging:
    puppeteer.launch({ headless: false })
    

    Watch the browser — do you see the full UI load? Or a stripped-down version?


    ✅ 6. Shopify Store Uses App Blocks / Turbo Pages

    Some Shopify apps inject content dynamically using Turbo Drive / Hotwire, which uses PJAX-like behavior and may not trigger standard navigation events properly.

    🔁 Solution: Listen for request / response or domcontentloaded + polling

    You might need to intercept network responses to ensure all required data is loaded:

    page.on('response', async (response) => {
      const url = response.url();
      if (url.includes('/products.json') || url.includes('/collections/')) {
        console.log(`Dynamic content loaded from ${url}`);
      }
    });
    

    Alternatively, wait for XHR/fetch calls to complete:

    await page.waitForResponse(response => 
      response.url().includes('/api/') && response.status() === 200
    );
    

    ✅ TL;DR — Checklist to Fix Puppeteer JS Execution on Shopify

    StepAction
    ✅Wait for a dynamic element with waitForSelector()
    ✅Use headless: 'new' or test in non-headless mode
    ✅Set realistic user-agent and hide automation flags
    ✅Check for redirects or bot protection (Cloudflare)
    ✅Intercept network activity to confirm JS fetches
    ✅Add waitForTimeout() temporarily for debugging
    ✅Take multiple screenshots to observe rendering over time

    🧪 Bonus: Debugging with DevTools Protocol

    Enable logging to see what’s happening:

    page.on('console', msg => console.log('PAGE LOG:', msg.text()));
    

    Or monitor JS execution:

    page.evaluate(() => {
      window.jsExecuted = true;
    });
    

    Then check window.jsExecuted in a later step.


    If after all this it still doesn’t work, share:

    • The URL you're trying to trace
    • Your Puppeteer code snippet
    • Whether you're using any proxies

    That would help diagnose further.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.