JAVASCRIPT · TECHNICAL SEO & CI
sitemap-scan — High-Concurrency Recursive XML Sitemap Crawler & Status Auditor
sitemap-scan is a fast Node.js CLI and library that recursively parses <sitemapindex> and <urlset> XML sitemaps to verify HTTP status codes, redirect chains, and canonical headers. Teams run it in CI/CD pipelines after deployment to catch broken 404 URLs and stale sitemap entries before search engines crawl them.
$ npx sitemap-scan https://example.com/sitemap.xml
Copy
GitHub ↗
Concurrency
50 Parallel Workers
97-URL Scan Time
0.68 sec
Detects
404 / 301 / Non-Canonical
CI Output
JSON / Markdown / Exit 1
01. Interactive Workbench
02. Reference Implementation
03. Architecture & Specs
Interactive `<urlset>` & `<sitemapindex>` Validator
Target Sitemap URL or XML Snippet
Audit Policy
Strict — Require HTTP 200 + Matching <link rel='canonical'>
Redirect Audit — Flag 301/302 Redirect Chains inside <loc>
Freshness Audit — Validate ISO-8601 <lastmod> Timestamps
Export Broken Links Report as JSON / GitHub Issue Body
Execute Pipeline ▶
Reset
Deterministic Execution Output & Telemetry
STATUS: READY // 0 ERRORS
JavaScript (Concurrent Sitemap Scanner)
export async function scanSitemap(sitemapUrl, concurrency = 25) {
const xml = await (await fetch(sitemapUrl)).text();
const urls = [...xml.matchAll(/<loc>([^<]+)<\/loc>/g)].map(m => m[1].trim());
const results = await mapPool(urls, concurrency, async (url) => {
const res = await fetch(url, { method: 'HEAD', redirect: 'manual' });
return { url, status: res.status, ok: res.status === 200 };
});
return { total: urls.length, passed: results.filter(r => r.ok).length, results };
}
Production Engineering Specifications
Repository: https://github.com/bhubbard/sitemap-scan
Live Studio: https://code.brandonhubbard.com/sitemap-scan/
Portal: https://code.brandonhubbard.com/
Install: npx sitemap-scan https://code.brandonhubbard.com/sitemap.xml
Design System: Brandon Hubbard Swiss Modernist (#0A0A0C / 1px Exposed Structural Grid)