Design a JavaScript rendering watchdog: bot rendering diffs, hydration error capture,…
- User
Design a JavaScript rendering watchdog: bot rendering diffs, hydration error capture, blocked resource alerts, pre‑render fallback rules, and recovery runbooks.
Assistant
1. Understand the Goal: The user wants a comprehensive "JavaScript rendering watchdog." This isn't just a simple script; it's a system design problem. The key components requested are: * Bot rendering diffs (detecting differences between what a bot sees vs. a user, or comparing renders). * Hydration error capture (specifically for frameworks like React/Next.js/Vue/Nuxt where client-side JS fails to match server HTML). * Blocked resource alerts (detecting CSS/JS/fonts that fail to load and break the layout). * Pre-render fallback rules (logic to serve static HTML if dynamic rendering fails). * Recovery runbooks (automated or manual steps to fix the issues).
-
Target Audience/Context: This is likely for a Site Reliability Engineer (SRE), Frontend Platform Lead, or SEO Specialist working with complex Single Page Applications (SPAs) or Server-Side Rendered (SSR) apps. The tone should be technical, architectural, and code-focused.
-
Deconstruct Components & Brainstorming:
-
Core Architecture: I need a central "Agent" or "Monitor" class. It needs to hook into the browser lifecycle (
PerformanceObserver,window.onerror, DOM mutation) and potentially run in a headless environment (Puppeteer/Playwright) for the bot diffing. -
1. Bot Rendering Diffs:
- Concept: Compare the HTML output of a headless browser (simulating Googlebot) vs. the raw HTML response (or a different user agent).
- Implementation: Use a Node.js service (Lambda/Cloud Run) running Puppeteer. It takes a snapshot, compares it (visual or DOM diff), and flags discrepancies.
- Code Snippet: A Puppeteer script comparing
contentvsplaintext.
-
2. Hydration Error Capture:
- Concept: When React/Vue hydrates, it warns if the DOM differs. We need to catch these warnings and send them to a logger (Sentry/Datadog).
- Implementation: Monkey-patch
console.erroror use framework-specific error boundaries. Detect specific strings like "Hydration failed" or "Text content did not match." - Code Snippet: A wrapper for
console.errorwith regex filtering.
-
3. Blocked Resource Alerts:
- Concept: Critical CSS or JS failing to load (404, CORS, timeout) leads to a broken UI (Layout Shift or Missing Content).
- Implementation:
PerformanceObserverlistening forresourcetiming errors. Specifically looking attransferSize(0) or decoded body size vs. encoded size. - Code Snippet: A
PerformanceObserversetup.
-
4. Pre-render Fallback Rules:
- Concept: If the JS execution fails or times out, serve a static HTML fallback so the user/SEO isn't left with a blank screen.
- Implementation: Middleware (Edge functions like Cloudflare Workers/Next.js Middleware). Check health signals. If fail, rewrite URL to a cached/prerendered version.
- Code Snippet: A basic Edge Middleware logic.
-
5. Recovery Runbooks:
- Concept: What to do when alerts fire.
- Implementation: Not code, but structured procedures. "If Hydration Error -> Check Date/Time mocking -> Check random IDs." "If Blocked Resource -> Check CDN status."
-
-
Structuring the Response:
- Introduction: Define the scope and the system architecture (Client-side Agent + Server-side Auditor).
- Module 1: The Client-Side Watchdog (The "Agent"): Focus on Hydration and Resources.
- Code: A class
RenderWatchdogusingPerformanceObserverand error interception.
- Code: A class
- Module 2: Bot Rendering Diff Service (The "Auditor"): Focus on headless browser diffing.
- Code: Puppeteer/Playwright script for DOM comparison.
- Module 3: Pre-render Fallback (The "Safety Net"): Edge logic.
- Code: Pseudo-code for Edge Middleware.
- Module 4: Recovery Runbooks: Specific troubleshooting steps.
- Summary/Architecture Diagram (Text): How it all fits together.
-
Refining the Code Snippets:
- Hydration: React 18+ has
onRecoverableErroron the root. I should mention that, but generic console interception is more framework-agnostic. Let's stick to the genericconsole.errorpatch for broad applicability but mention React specifics. - Resource Blocking:
PerformanceObserveris the modern way. Need to filter forinitiatorType. - Bot Diff: Needs to be robust. Compare text content length or specific selectors.
- Hydration: React 18+ has
-
Drafting the Content (Mental Sandbox):
- Self-Correction during drafting: Just logging errors isn't a "watchdog." A watchdog needs to act. I should add a "Circuit Breaker" concept. If too many errors occur, the client should stop trying to hydrate and just show the static shell, or trigger a refresh.
-
Final Polish: Ensure distinct sections, clear headings, and production-grade advice (e.g., using UUIDs for tracing).
- Structure Check:
- Architecture Overview
- Part 1: Hydration & Resource Monitor (Browser JS)
- Part 2: Bot Rendering Diff (Node/Headless)
- Part 3: Fallback Middleware (Edge)
- Part 4: Runbooks (Process)
- Structure Check:
-
Writing the Output: (This aligns with the provided good response, ensuring code is copy-pasteable and concepts are explained clearly.) Self-correction: Make sure to distinguish between 'Hydration Mismatch' (React warning) and 'Render failure' (White screen).
(Self-Correction Details): Initial thought: Just write a script that runs in the browser. Correction: That misses the "Bot" aspect. Bots (Googlebot) often use their own specific rendering flow. I need a server-side component simulating the bot to compare against user reality. Initial thought: Just list the errors. Correction: The prompt asks for "Recovery runbooks" and "Fallback rules." I need to provide logic (pseudo-code or actual code) for routing fallbacks, not just monitoring.
- Final Output Generation. (Proceed to generate the response based on these structured thoughts.) This is a comprehensive system design for a JavaScript Rendering Watchdog. This system operates on two levels: the Client-Side Agent (running in the user's browser to detect hydration/runtime errors) and the Synthetic Auditor (running in a headless environment to detect bot rendering diffs).
Below is the architecture, implementation, and runbooks.
System Architecture
- Client-Side Agent (Browser): Monitors the "Hydration Phase" and Resource loading. Sends telemetry to the logging endpoint.
- Synthetic Auditor (Node/Headless): Periodically renders pages using a User-Agent spoofing a search bot (e.g., Googlebot) and compares the DOM output against a baseline or the client-rendered output.
- Edge Gateway / Middleware: Intercepts requests to serve Pre-render fallbacks if the origin is failing.
Module 1: The Client-Side Watchdog Agent
This script should be injected inline in the
<head>to capture early errors and hydration mismatches.A. Hydration Error & Mismatch Capture
Modern frameworks (React, Vue) emit specific console errors or custom events when hydration fails. We capture the specific DOM nodes that differ.
class HydrationMonitor { constructor() { this.errors = []; this.init(); } init() { // 1. Generic Global Error Handler window.addEventListener('error', (event) => { this.report('runtime_error', { message: event.message, filename: event.filename, lineno: event.lineno, colno: event.colno, }); }); // 2. Framework Specific: React Hydration Mismatch Parser const originalError = console.error; console.error = (...args) => { const msg = args[0]; if (typeof msg === 'string' && msg.includes('Hydration failed')) { // Attempt to extract the specific node causing issues const nodeSnapshot = this.captureDomSnapshot(); this.report('hydration_mismatch', { raw: msg, domContext: nodeSnapshot }); } originalError.apply(console, args); }; } captureDomSnapshot() { // Returns a simplified view of the body to avoid massive logs return document.body.innerHTML.substring(0, 1000) + '...'; } async report(type, payload) { // Use sendBeacon to ensure data is sent even if page unloads const data = { type, url: window.location.href, payload, timestamp: Date.now() }; navigator.sendBeacon('/api/watchdog/telemetry', JSON.stringify(data)); } } // Initialize immediately window.__watchdog = new HydrationMonitor();B. Blocked Resource & Critical CSS Alerts
We use
PerformanceObserverto detect resources that failed to load or took too long, impacting the Critical Rendering Path.class ResourceMonitor { constructor() { this.observer = new PerformanceObserver((list) => { list.getEntries().forEach((entry) => { if (entry.initiatorType === 'script' || entry.initiatorType === 'css') { // Check for failed resources (transferSize 0 usually implies blocked/failed in CORS or adblock) if (entry.transferSize === 0 && entry.decodedBodySize > 0) { // Likely cached, ignore } else if (entry.transferSize === 0 && entry.duration === 0) { this.alertBlockedResource(entry); } } }); }); this.observer.observe({ entryTypes: ['resource'] }); } alertBlockedResource(entry) { window.__watchdog.report('blocked_resource', { name: entry.name, initiatorType: entry.initiatorType }); } }
Module 2: Bot Rendering Diff Auditor (Synthetic)
This runs in a CI/CD pipeline or a scheduled Lambda function. It detects "Cloaking" issues (where the bot sees something different than the user) or Bot-specific crashes.
// run-audit.js (Node/Puppeteer) const puppeteer = require('puppeteer'); async function auditRenderDiff(url) { const browser = await puppeteer.launch(); // 1. Render as Googlebot const botPage = await browser.newPage(); await botPage.setUserAgent('Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'); await botPage.goto(url, { waitUntil: 'networkidle0' }); const botHtml = await botPage.content(); const botText = await botPage.evaluate(() => document.body.innerText); // 2. Render as Regular User const userPage = await browser.newPage(); await userPage.goto(url, { waitUntil: 'networkidle0' }); const userHtml = await userPage.content(); const userText = await userPage.evaluate(() => document.body.innerText); // 3. Compare Semantics (Not exact bytes, whitespace doesn't matter) // A simple text diff check: if (similarity(botText, userText) < 0.95) { triggerAlert('SEO_DIFF_DETECTED', { url, botLength: botText.length, userLength: userText.length }); } // 4. Check for "Shell" content (Common SPA issue where bot sees empty div) if (botText.trim().length < 100 && userText.length > 500) { triggerAlert('BOT_SHELL_RENDER', { url }); } await browser.close(); }
Module 3: Pre-render Fallback Rules (Middleware)
This acts as a "Circuit Breaker" at the Edge (e.g., Cloudflare Workers, Varnish, or Next.js Middleware). If the origin server returns a 5xx, or if the JS bundle is corrupted, serve a static pre-rendered version.
// middleware.js (Concept for Edge/Next.js) import { NextResponse } from 'next/server'; const PRERENDER_BUCKETS = { '/product/a': 'https://static-cdn.com/prerender/product-a.html', // ... dynamic mapping }; export async function middleware(req) { const url = req.nextUrl; // 1. Check if request is from a Bot const isBot = req.headers.get('user-agent')?.includes('bot'); // 2. If we detect server instability (via external health check flag) OR it's a bot // We intercept to ensure stability. if (isBot && process.env.ORIGIN_HEALTH === 'UNSTABLE') { const prerenderUrl = PRERENDER_BUCKETS[url.pathname]; if (prerenderUrl) { // Fetch static HTML and return const response = await fetch(prerenderUrl); return new Response(response.body, { headers: { 'Content-Type': 'text/html', 'X-Watchdog-Fallback': 'true' } }); } } return NextResponse.next(); }
Module 4: Recovery Runbooks
When the watchdog barks, follow these procedures.
Alert 1:
hydration_mismatch- Diagnosis: The HTML sent by the server differs from what the client-side React/Vue generated.
- Immediate Fix: No code fix is instant.
- Check if the specific component uses browser-only globals (window, document) during render.
- Check for Date/Time mismatches (Server time zone vs Client time zone).
- Recovery: Patch the component to use
useEffectoronMountedfor browser-specific logic. Deploy hotfix.
Alert 2:
blocked_resource- Diagnosis: Ad-blockers or strict corporate firewalls are blocking a script crucial to the UI (often analytics or bundle chunks with "ad" in the name).
- Immediate Fix: None (This is client-side infrastructure).
- Long-term Recovery:
- Rename bundles to remove "ad", "tracker", "marketing" strings.
- Bundle critical CSS/JS inline to prevent external request blocking for above-the-fold content.
Alert 3:
bot_shell_render- Diagnosis: Googlebot received an empty
<div id="root"></div>. - Immediate Fix:
- Check if the Dynamic Rendering service is up.
- If using SSR, check server CPU load (timeout during rendering).
- Recovery: Switch traffic to a static pre-render bucket using the Middleware rules (Module 3).
Alert 4:
runtime_error(Spike)- Diagnosis: A new deployment broke JS execution for a specific browser version.
- Immediate Fix: Rollback Deployment.
- Recovery:
- Analyze
filenameandlinenofrom the telemetry. - Add Polyfills for missing browser features.
- Add Error Boundaries around the crashing component to isolate the failure.
- Analyze
Summary Dashboard Metrics
- Hydration Success Rate: % of page loads without mismatch warnings.
- Time to Interactive (TTI) Variance: Watch for increases indicating blocked resources.
- Bot vs. User Content Similarity Score: Calculated by the Auditor.
-