Open SEO Crawler
Self-hosted SEO crawler: run site audits and AI crawler checks, then read the issues and pages.
Connector by puneetindersingh · GitHub account verified by the MCP Registry
Install
Works with Claude Desktop. Read the repository before you give any tool access to your data.
MCP bundle (.mcpb): open it with Claude Desktop.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Use cases: seo, web-scraping
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this server can do and what it asks for, from its tool list, install config and code. Checked Oct 25, 2026.
- Tools
- 10 read · 3 write · 0 delete
- Secrets
- None asked for
- Install scripts
- None found
- Runs as
- Downloaded binary
Tools by what they do 13
Read from the source code; the running server may list different tools.
Read 10 · Look things up without changing anything.
bot_checkcheck_robotscompare_crawlscrawl_statuscrawl_summarydetect_cmsget_briefget_pageget_pageslist_crawls
Write 3 · Create, change, send or run things.
export_xlsxstart_crawlstop_crawl
Network hosts it talks to 25
ahrefs.com· found in codeapple.com· found in codebing.com· found in codebooks.toscrape.com· found in codechromewebstore.google.com· found in codecommoncrawl.org· found in codedev.to· found in codedeveloper.amazon.com· found in codedeveloper.chrome.com· found in codedevelopers.cloudflare.com· found in codedevelopers.google.com· found in codeexample.comundefined· found in codegit-scm.com· found in codegithub.com· found in install command, codegoogle.com· found in codemoz.com· found in codeopenai.com· found in codeother.com· found in codeperplexity.ai· found in codeplatform.openai.com· found in codequotes.toscrape.com· found in coderaw.githubusercontent.com· found in codesitemaps.org· found in codesupport.apple.com· found in codeweb.dev· found in code
Files it touches
- Your home directory (~)
~/.site-crawler-crawls~/.site-crawler-users.json
Tools (13)
Read from the source code; the running server may differ. Descriptions are the server’s own words; hidden characters are shown as [U+…].
bot_checkFetch pages as Googlebot, Bingbot, AI search, AI assistant and training crawlers and compare each with what a browser gets: blocked, CDN challenge, noindex, redirected, thinner or different content. Requests come from this computer with each crawler's user agent.
check_robotsFetch and parse a site's robots.txt: rules, sitemaps, and which search and AI crawlers it blocks.
compare_crawlsDiff two saved crawls of a site: pages added, removed and changed, and how the issue counts moved.
crawl_statusProgress of a crawl started in this session: state (crawling, sitemap, near_dup, bot_check, saving, done, stopped, error), pages so far, queue, and the saved crawl name once done.
crawl_summaryWhat the Summary tab shows, as data: pages, status codes, indexable pages, every issue grouped into errors, warnings and info with page counts and sample URLs, duplicates, redirect chains, sitemap gaps, near duplicates and the bot readability results.
detect_cmsIdentify the CMS behind a URL, with the crawl settings and exclude patterns the app suggests for it.
export_xlsxWrite a saved crawl to an .xlsx file on this computer (All Pages and Issues Summary sheets).
get_briefThe app's AI brief for a crawl: a Markdown report grouped by severity with capped URL lists and a page table, under an audit prompt. Good as one read of a whole crawl.
get_pageEverything the crawl holds for one URL (meta, headings, canonical, hreflang, schema types, links, images, issues) plus the internal links pointing at it.
get_pagesRows of a saved crawl, filtered and paged. Filter by issue (a Summary heading such as "Missing H1", or any text from the issue), severity, status (404, 3xx), URL text or indexability. Default fields: url, status_code, title, title_len, meta_description, meta_len, h1, word_count, depth, indexable, canonical, canonical_kind, response_time, issues. fields="all" returns every field (large).
list_crawlsSaved crawls from the last 30 days, newest first (crawls from the app and from here).
start_crawlCrawl a website with the same engine and defaults as the app (1500 pages, 5 workers, 0.4 s delay per host, depth 10, robots.txt respected; then sitemap analysis, near-duplicate content and the AI crawler readability check). Returns a job id at once; poll crawl_status. Set max_pages low (20 to 100) for a quick look. The finished crawl is saved and also shows under Load in the app.
stop_crawlStop a running crawl like the Stop button. Pages fetched so far are kept, checked and saved.
Compare and alternatives
Featured in
Signals
- Stars
- 71
- Forks
- 23
- Open issues
- 4
- Contributors
- 1
- Last commit
- Oct 23, 2026 (6 days ago)
- Last release
- Oct 16, 2026 (12 days ago)
From GitHub (fetched Oct 24, 2026).
Details
- Registry name
io.github.puneetindersingh/open-seo-crawler- Version
- 1.0.0
- Auth
- No auth
- Packages
- mcpb https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb
- License
- MIT
- Website
- github.com/puneetindersingh/open-seo-crawler#use-it-from-an-ai-assistant-mcp-or-the-command-line
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
More from puneetindersingh
- 0.869score
seo-content-brief
puneetindersingh
Generate competitive SEO content briefs with per-section word counts, competitor scoring, keyword placement guidance, and page-type templates.
SkillAFreshness: A★ 20.1KClaude Code +9 - 0.502score
claude-seo-content-brief
puneetindersingh
Generate competitive SEO content briefs with per-section word counts, competitor scoring, keyword density guidance, and page-type templates.
SkillCFreshness: C★ 1Claude Code +9
Cite this page
Open SEO Crawler MCP server. Agent Mart. https://agentmart.dev/connectors/puneetindersingh-open-seo-crawler (last updated 2026-10-24)