When tracing a site (built with Shopify) via puppeteer it appears that JS is not…
- User
When tracing a site (built with Shopify) via puppeteer it appears that JS is not executed. JS execution is enabled and works on other sites. Why does it not work on this one and how to make it work.
Screenshots lack elements added with JS and the trace doesn't include main thread JS activity.
Assistant
When Puppeteer fails to execute JavaScript on a Shopify site — while it works fine on others — yet you've confirmed that JavaScript execution is enabled, the issue likely lies in how Shopify serves its content, especially under certain conditions like:
- Deferred or asynchronous loading
- Server-side rendering (SSR) or hydration delays
- Anti-bot measures (like Cloudflare or Shopify bot protection)
- Navigation timing and resource loading quirks
Let’s walk through why this happens and how to fix/troubleshoot it.
✅ 1. Ensure Full Page Load & JavaScript Execution Time
Shopify uses progressive hydration and sometimes loads critical parts of the page after the initial HTML is served. If Puppeteer doesn’t wait long enough, it may take a screenshot or create a trace before JS-rendered content appears.
🔧 Solution: Wait for Network Idle or Specific Elements
Use
page.gotowith proper wait conditions:await page.goto('https://yourshopifystore.com', { waitUntil: 'networkidle', // or 'domcontentloaded' + manual wait });Better yet, wait for a selector that’s rendered by JavaScript:
await page.goto('https://yourshopifystore.com'); await page.waitForSelector('.product-grid'); // Replace with actual dynamic elementIf needed, add a delay to debug:
await page.waitForTimeout(5000); // wait 5 seconds
✅ 2. Check for Lazy-Loaded or Client-Side Rendered Content
Shopify Liquid templates render server-side, but many themes (e.g., Debut, Dawn, Turbo) rely heavily on JavaScript for:
- Product recommendations
- Cart functionality
- Reviews (like Yotpo, Judge.me)
- Sliders/carousels
- Navigation menus
These often load after
DOMContentLoaded.🔍 Debug Tip:
Take a screenshot immediately after
goto, then another after a delay. If the second one includes missing elements, it confirms async JS rendering.
✅ 3. Enable JavaScript and Disable Caching (Just in Case)
Double-check your Puppeteer launch config:
const browser = await puppeteer.launch({ headless: 'new', // Try 'new' instead of true/false in newer Puppeteer args: ['--disable-site-isolation-trials'], // optional }); const page = await browser.newPage(); await page.setJavaScriptEnabled(true); // Usually enabled by default, but safe to ensure await page.setViewport({ width: 1280, height: 800 });Note:
setJavaScriptEnabled(true)is usually redundant in recent Puppeteer versions, as JS is enabled by default.
✅ 4. Bypass Bot Detection (Cloudflare, Shopify Bot Protection)
Some Shopify stores use bot protection mechanisms (especially via Cloudflare or Shopify's own anti-scraping). These can serve simplified or empty pages to Puppeteer if it doesn't look like a real browser.
🔐 Solution: Make Puppeteer More Browser-like
Set proper headers, including a real-looking user-agent:
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36');You can also enable additional browser features:
await page.evaluateOnNewDocument(() => { Object.defineProperty(navigator, 'webdriver', { get: () => false }); });Optional: Use stealth plugins like
puppeteer-extra-plugin-stealth
✅ 5. Trace Doesn’t Show JS Activity?
If your trace lacks main thread JS activity, it suggests Puppeteer isn’t actually running JS — possibly due to:
- Navigation aborting or redirecting too early
- Site detecting headless and serving fallback page
headless: truebehaving differently than expected
🛠️ Try:
- Use
headless: 'new'(Puppeteer v19+) - Or run in non-headless for debugging:
puppeteer.launch({ headless: false })Watch the browser — do you see the full UI load? Or a stripped-down version?
✅ 6. Shopify Store Uses App Blocks / Turbo Pages
Some Shopify apps inject content dynamically using Turbo Drive / Hotwire, which uses PJAX-like behavior and may not trigger standard navigation events properly.
🔁 Solution: Listen for
request/responseordomcontentloaded+ pollingYou might need to intercept network responses to ensure all required data is loaded:
page.on('response', async (response) => { const url = response.url(); if (url.includes('/products.json') || url.includes('/collections/')) { console.log(`Dynamic content loaded from ${url}`); } });Alternatively, wait for XHR/fetch calls to complete:
await page.waitForResponse(response => response.url().includes('/api/') && response.status() === 200 );
✅ TL;DR — Checklist to Fix Puppeteer JS Execution on Shopify
Step Action ✅ Wait for a dynamic element with waitForSelector()✅ Use headless: 'new'or test in non-headless mode✅ Set realistic user-agent and hide automation flags ✅ Check for redirects or bot protection (Cloudflare) ✅ Intercept network activity to confirm JS fetches ✅ Add waitForTimeout()temporarily for debugging✅ Take multiple screenshots to observe rendering over time
🧪 Bonus: Debugging with DevTools Protocol
Enable logging to see what’s happening:
page.on('console', msg => console.log('PAGE LOG:', msg.text()));Or monitor JS execution:
page.evaluate(() => { window.jsExecuted = true; });Then check
window.jsExecutedin a later step.
If after all this it still doesn’t work, share:
- The URL you're trying to trace
- Your Puppeteer code snippet
- Whether you're using any proxies
That would help diagnose further.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.