Web & Data Engineering

SEO Site Extractor

Type a website, hit Extract, and get one Obsidian-friendly Markdown file with the SEO profile of every page — titles, meta, Open Graph, headings, link counts, and site-wide findings. Everything runs in your browser; pages are fetched through a public CORS proxy. Reads server-rendered HTML only — no JavaScript is executed, so client-rendered (SPA) pages may look incomplete.

Crawl a site

Max pages

Use sitemap / robots.txt Discover URLs from sitemaps first; fall back to link crawling. Include query strings Treat ?a=1 URLs as distinct pages (off = ignore queries).

Advanced: CORS proxy

Proxy URL The target URL is appended (URL-encoded), or substituted for {url} if present. Paste your Cloudflare Worker URL (e.g. https://seo-proxy.you.workers.dev/?url=) for reliable, rate-limit-free crawls with accurate per-page statuses. Saved in this browser.

How it works

Discovery. The crawler reads /robots.txt for Sitemap: directives, then /sitemap.xml and /sitemap_index.xml, recursing into child sitemaps to collect every <loc> URL. With no sitemap (or that toggle off) it BFS-crawls from the homepage, following same-origin <a href> links only.
Extraction. Each page's HTML is parsed with the browser's native DOMParser — no library — pulling title, meta description, canonical, robots, lang, Open Graph, Twitter card, hreflang, JSON-LD @types, the H1/H2/H3 outline, and link / image / word counts.
Output. One Markdown file with YAML frontmatter, a Findings section flagging site-wide issues, an Index, and a section per page. Headings inside each page render as nested bullets so the report's own heading hierarchy stays clean and [[#path]] wikilinks resolve in Obsidian.
CORS. Static pages can't fetch other origins directly, so every request goes through a public CORS proxy. For heavy crawls or rate limits, paste your own Cloudflare Worker or Caddy reverse-proxy URL into the Advanced: CORS proxy field above (saved in your browser) — see assets/tools/seoextractor/proxy/ for a ready-to-deploy Worker. The Worker also reports accurate per-page HTTP statuses via X-Proxy-Status.
Nothing is uploaded to Cool Web Tools — the crawl runs from your browser. Be considerate: only crawl sites you own or have permission to audit.