Run a sitewide redirect hygiene audit: chain/loop detection, hreflang/canonical…
- User
Run a sitewide redirect hygiene audit: chain/loop detection, hreflang/canonical integrity, and regression watchdogs.
Assistant
The user wants me to perform a "sitewide redirect hygiene audit." This request implies checking for:
- Chain/Loop Detection: Are there redirects that point to other redirects (chains) or back to themselves (loops)?
- Hreflang/Canonical Integrity: Do the redirects conflict with
hreflangtags orrel="canonical"headers? For example, redirecting a URL that is declared as a canonical or hreflang alternative elsewhere. - Regression Watchdogs: Identifying patterns that suggest recent breakage or unintended changes (e.g., a spike in 404s, sudden mass redirects to a single page).
Crucial limitation: I am an AI model (text-based). I cannot actually crawl a website, access server logs, or run live network requests against a domain unless the user provides the data or specific URLs.
Strategy:
- Acknowledge limitations: State clearly that I cannot perform a live audit on an external site without a target.
- Offer a Python script: Since I cannot run it live, I will write a comprehensive Python script (using
requestsandBeautifulSoupor similar logic) that the user can run to perform this audit. This is the most helpful response. - Provide a manual checklist/guide: Explain how to perform this audit using existing tools (Screaming Frog, Ahrefs, etc.) if they don't want to code.
- Simulate the logic (Mental Model): I will structure the Python script to look for exactly what was asked (chains, loops, canonical mismatches).
Drafting the Python Script:
- Input: List of URLs or a sitemap.
- Logic:
- Request URL.
- Follow history to check for chains (status codes 301, 302).
- Detect loops (URL A -> URL B -> URL A).
- Check final destination vs. canonical tags in the response.
- Check hreflang (this is harder to check strictly via script without a map of all pages, but I can check if the current URL matches the hreflang target).
- Output: Report of issues.
Drafting the "Watchdog" advice:
- How to spot regressions? Look at trends.
- Check for redirecting to 404s (soft 404s).
- Check for mass redirecting to the homepage (generic handling).
Refining the Response: Since the user didn't provide a URL, the most "executive" way to answer is to provide the tooling to do the job immediately. I will provide a Python script that acts as the auditor.
Plan:
- Disclaimer: "I cannot access the live web."
- Solution: "Here is a Python script to automate this audit."
- Code: A robust script using
requestsandurllib.- Feature 1: Loop/Chain detector (max redirects < 5, detect circular refs).
- Feature 2: Canonical check (Does
rel=canonicalmatch the final resolved URL?). - Feature 3: Hreflang integrity (basic check: do hreflang URLs return 200 and not redirect?).
- Explanation of the "Regression Watchdogs" (what metrics to look for).
Self-Correction during drafting: A full hreflang audit requires checking all language versions against each other. A single-threaded script checking one URL can't fully validate hreflang reciprocity (e.g., Page A says "en-gb" is Page B, but does Page B say "en-us" is Page A?). I should focus the script on checking if the hreflang targets resolve correctly and don't redirect. Reciprocity is a second step.
Let's refine the "Regression Watchdog" part. This usually implies monitoring. I will explain what to set up (alerts in Google Search Console or server monitoring).
Final Output Structure:
- Context: Can't run live, but here is the tool.
- The Audit Script (Python): Complete code.
- Audit Methodology (The "Why"): Explaining the specific checks (Chains, Canonicals, Hreflang, Watchdogs).
- Tool-based Alternative: Mentioning Screaming Frog configs.
Let's write the Python script mental draft:
get_redirect_chain(url): returns list of steps.- Check if start URL in steps -> Loop.
- Check len(steps) > 1 -> Chain.
get_head(url): check headers.get_content(url): parse HTML for<link rel="canonical">.- Compare canonical vs final_url.
Okay, ready to generate the response. Because I do not have access to your specific domain or server configuration, I cannot run this audit on your live site.
However, I have constructed a Python-based Audit Bot that you can run locally. This script will act as your regression watchdog, identifying redirect chains, loops, and canonical mismatches.
Following the script, I have provided a strategic checklist for the "Watchdog" logic (monitoring and regression prevention).
Phase 1: The Automated Audit Script
Prerequisites: Python 3,
requests,beautifulsoup4.pip install requests beautifulsoup4import requests from urllib.parse import urljoin, urlparse from bs4 import BeautifulSoup import time class RedirectAuditor: def __init__(self, start_url, max_depth=10): self.start_url = start_url self.max_depth = max_depth # Max redirects to follow before breaking (Loop detection) self.session = requests.Session() self.session.headers.update({'User-Agent': 'RedirectHygieneBot/1.0'}) self.visited = set() def get_canonical(self, url, response): """Extracts the canonical tag from HTML.""" if 'text/html' not in response.headers.get('Content-Type', ''): return None try: soup = BeautifulSoup(response.text, 'html.parser') canonical_link = soup.find('link', rel='canonical') if canonical_link and canonical_link.get('href'): return urljoin(url, canonical_link['href']) except Exception: pass return None def audit_single_url(self, url): """Checks for chains, loops, and canonical integrity.""" if url in self.visited: return self.visited.add(url) chain = [] current_url = url print(f"\n[AUDITING] {url}") for i in range(self.max_depth): try: response = self.session.get(current_url, allow_redirects=False, timeout=10) # 1. Integrity Check: Canonical canonical = self.get_canonical(current_url, response) if canonical and canonical != current_url: # If this is a redirect, the canonical should usually point to the destination # or self-referencing. Pointing elsewhere while redirecting is a hygiene risk. print(f" ⚠️ WARN: Redirect page {current_url} has canonical pointing to {canonical}") # 2. Redirect Logic if 300 <= response.status_code < 400: location = response.headers.get('Location') if not location: print(f" ❌ FAIL: Status {response.status_code} but no Location header.") break # Resolve relative URLs next_url = urljoin(current_url, location) # LOOP DETECTION if next_url in chain: print(f" 🔄 LOOP DETECTED: {next_url} redirects back to itself via {chain}") return # HTTPS REGRESSION if urlparse(next_url).scheme == 'http': print(f" 🔓 INSECURE: Redirecting to HTTP {next_url}") chain.append(current_url) current_url = next_url else: # Final destination reached if len(chain) > 1: print(f" ⛓️ CHAIN DETECTED ({len(chain)} hops):") for step in chain: print(f" -> {step}") print(f" [FINAL] {current_url}") else: print(f" ✅ OK: {response.status_code} (Direct or 1 hop)") return except requests.RequestException as e: print(f" 💥 ERROR: Could not reach {current_url} - {e}") return print(f" 🔥 DEEP CHAIN: Exceeded max depth ({self.max_depth})") # EXECUTION BLOCK if __name__ == "__main__": # 1. Define your test list (Replace with your actual sitemap or top URLs) target_urls = [ "http://yourdomain.com/", # Test HTTP->HTTPS "http://www.yourdomain.com/", # Test WWW->Non-WWW "http://yourdomain.com/old-page", # Known old URL "https://yourdomain.com/active-page" ] auditor = RedirectAuditor("http://yourdomain.com") print("--- STARTING REDIRECT HYGIENE AUDIT ---") for url in target_urls: auditor.audit_single_url(url) time.sleep(1) # Be polite to the server
Phase 2: Strategic Analysis (The "Watchdog" Logic)
If you are using an enterprise crawler (like Screaming Frog, Sitebulb, or Ahrefs) or setting up a CI/CD pipeline, here are the specific logic patterns you must configure to act as a "Regression Watchdog."
1. Chain & Loop Detection
- The Logic: A redirect chain occurs when URL A redirects to URL B, which redirects to URL C (
A -> B -> C). - The Watchdog Rule: Flag any redirect chain length > 1.
- Why? Every "hop" dilutes link equity (PageRank) and slows down page load time (LCP).
- Regression Alert: If you update a CMS plugin and suddenly see
A -> Temp_B -> C, the plugin is inserting intermediate tracking URLs.
- The Loop Logic:
A -> B -> A. The script above detects this by storing the history of visited URLs in the current chain.- Critical Failure: Loops make pages completely inaccessible to crawlers and users.
2. Hreflang & Redirect Collision
- The Logic: Hreflang tags should point to the final, canonical version of a page.
- The Watchdog Rule:
- Scenario A: URL A redirects to URL B. URL A has an
hreflangtag in its<head>.- Audit Result: FAIL. A redirecting page should not serve HTML content (it should serve 301 headers immediately). If it serves HTML with hreflang, it is a "soft redirect."
- Scenario B: Page X en-us has an hreflang pointing to Page Y es-es. However, Page Y es-es redirects back to Page X en-us.
- Audit Result: FAIL. Hreflang targets must return a
200 OKstatus. Redirecting targets invalidate the hreflang directive.
- Audit Result: FAIL. Hreflang targets must return a
- Scenario A: URL A redirects to URL B. URL A has an
3. Canonical Integrity (The "Soft 404" Trap)
- The Logic: A
rel="canonical"tag is a strong suggestion to Google. - The Watchdog Rule:
- Rule 1: If URL A redirects to URL B, URL B must canonicalize to URL B (self-referencing). If URL B canonicalizes to URL C, you have created a chain (
A -> B -> rel=canonical C). - Rule 2: A redirecting URL (301/302) should generally not contain an HTML body. If your server returns a 301 but also returns HTML with a canonical tag, the headers usually win, but this is messy code that implies a backend misconfiguration.
- Rule 1: If URL A redirects to URL B, URL B must canonicalize to URL B (self-referencing). If URL B canonicalizes to URL C, you have created a chain (
4. Protocol & Subdomain Regression (The "Bare" Check)
- The Watchdog Rule:
- Any redirect passing through
http://(non-secure) is a failure. - Any redirect passing through
wwwif your canonical isnon-www(or vice versa) is a failure. - Why? Your SSL certificate might cover
domain.combut notwww.domain.com(or vice versa), causing browser errors in the middle of the redirect chain.
- Any redirect passing through
Summary of Recommendations
- Map the Redirect Map: Export your
.htaccessor CDN redirect rules. - Flatten the Map: If
A -> B -> C, change the rule soA -> Cdirectly. - Fix Hreflang Targets: Ensure no hreflang URL in your sitemap returns a 3xx status code. They must all be 200.
- Automate: Put the Python script (or a similar logic) into your deployment pipeline so that if a developer pushes code that breaks redirects, the build fails.