Skip to content
Agentmart
Submit a tool

doc-scraper

Crawl documentation sites into local corpora agents can search, read, and diff fully offline.

Connector by Sriram-PR · GitHub account verified by the MCP Registry

IndexedApache-2.0Local (stdio)★ 107
Write a review

Install

Works with Claude Desktop. Read the repository before you give any tool access to your data.

Download the bundle

MCP bundle (.mcpb): open it with Claude Desktop.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Crawl documentation sites into local corpora agents can search, read, and diff fully offline.

Use cases: web-scraping, documentation

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this server can do and what it asks for, from its tool list, install config and code. Checked Oct 25, 2026.

Tools
9 read · 2 write · 0 delete
Secrets
None asked for
Install scripts
None found
Runs as
Downloaded binary

Tools by what they do 11

Read from the source code; the running server may list different tools.

Read 9 · Look things up without changing anything.

  • describe_server
  • diff_crawl
  • get_freshness
  • get_job_status
  • get_page
  • list_pages
  • list_sites
  • read_page
  • search_docs

Write 2 · Create, change, send or run things.

  • cancel_crawl
  • crawl_site

Network hosts it talks to 60

  • 100.64.0.1 · found in code
  • 169.254.169.254 · found in code
  • 93.184.215.14 · found in code
  • a.com · found in code
  • academy.subquery.network · found in code
  • accessible-astro.incluud.dev · found in code
  • agones.dev · found in code
  • alpha.com · found in code
  • alumni.cottonwoodhigh.school · found in code
  • aspire.dev · found in code
  • astro-snapshot.johng.io · found in code
  • astro-tips.dev · found in code
  • b.com · found in code
  • beta.com · found in code
  • biomejs.dev · found in code
  • blog.palashsh.me · found in code
  • bonjourr.fr · found in code
  • botonic.io · found in code
  • brycerussell.github.io · found in code
  • buildtracker.dev · found in code
  • buildwithfern.com · found in code
  • capgo.app · found in code
  • cdn.jsdelivr.net · found in code
  • cdn.redoc.ly · found in code
  • chaos-mesh.org · found in code
  • cho.sh · found in code
  • click.palletsprojects.com · found in code
  • clutch.sh · found in code
  • cn.vuejs.org · found in code
  • codesweetly.com · found in code
Show 30 more hosts
  • console-table.netlify.app · found in code
  • countrystatecity.in · found in code
  • covid-fr.misterfishup.com · found in code
  • crawlee.dev · found in code
  • crawler.siteone.io · found in code
  • cryptography.io · found in code
  • d.io · found in code
  • dev.mysql.com · found in code
  • developer.hashicorp.com · found in code
  • developers.home-assistant.io · found in code
  • dinosaur.leedom.me · found in code
  • docs.alpaca.markets · found in code
  • docs.antora.org · found in code
  • docusaurus.example.io · found in code
  • e.com · found in code
  • elsewhere.org · found in code
  • fastapi.tiangolo.com · found in code
  • flask.palletsprojects.com · found in code
  • github.com · found in install command, code
  • internal.test · found in code
  • other.com · found in code
  • other.io · found in code
  • public.test · found in code
  • rust-cli.github.io · found in code
  • rust-lang.github.io · found in code
  • sitemaps.org · found in code
  • sriram-pr.github.io · found in code
  • unused.invalid · found in code
  • vitepress.dev · found in code
  • vuejs.org · found in code

Files it touches

  • /etc/doc-scraper.yaml
  • /etc/passwd

Tools (11)

Read from the source code; the running server may differ. Descriptions are the server’s own words; hidden characters are shown as [U+…].

  • cancel_crawlCancel a crawl job

    Cancel a running or pending crawl job by job ID. The previously stored corpus and crawl history are left untouched. Has no effect on jobs already in a terminal state.

  • crawl_siteStart a crawl

    Start a background crawl for a configured site. Returns immediately with a job ID. The crawl builds in a staging area and replaces the stored corpus only when it completes, so a cancelled or failed crawl leaves the previous corpus intact.

  • describe_serverDescribe this serverread-only

    Description is built at runtime.

  • diff_crawlDiff crawls over timeread-only

    Return added/removed/changed pages between the latest crawl and a baseline crawl: the crawl before it by default, or with since, the most recent crawl whose crawl_ended_at <= since. Hash-based verdicts from the SQLite history index. To see what a crawl changed, run crawl_site, wait for it to finish, then call diff_crawl without since.

  • get_freshnessCheck crawl freshnessread-only

    Return the most recent crawl summary for a site (last_crawl with its start and end times, total_pages and mode, plus age_seconds), output/state dir presence and any running job. Use this to decide whether to query the existing crawl or run crawl_site first.

  • get_job_statusCheck a crawl jobread-only

    Get the status and page counts of a crawl job. Set wait_seconds to wait for the job to finish instead of polling: the call returns as soon as the job completes, fails or is cancelled, or when the wait runs out, with the status at that moment.

  • get_pageFetch a page liveread-only

    Fetch an HTML page live over the network and return its content as markdown. This is an on-demand fetch independent of any crawl: it does not read the stored crawl output, and it ignores the site's configured content_selector and scope. Large pages are truncated at max_bytes; follow next_offset to read the rest (each call fetches the page again).

  • list_pagesList crawled pagesread-only

    List crawled pages for a site, paginated and sorted by URL. Returns metadata only (URL, title, depth, crawled_at, content_length). Pass any URL returned here to read_page to get its stored markdown; get_page re-fetches a URL live rather than returning the crawled copy.

  • list_sitesList configured sitesread-only

    List all configured sites available for crawling

  • read_pageRead a crawled pageread-only

    Return a page's markdown from the stored crawl output, without any network access. This is the counterpart to get_page: read_page serves the crawled copy that already had the site's content_selector applied, while get_page re-fetches the URL live. Use list_pages to discover URLs, then read_page to read them. Pass a search_docs result's anchor to read only that section. Large pages are truncated at max_bytes; follow next_offset to read the rest.

  • search_docsSearch crawled docsread-only

    Full-text search across all crawled documentation, ranked by relevance (BM25 with stemming), with zero network access. Results carry the page URL, its section heading path, and a snippet with match terms marked [like this]; follow up with read_page for the full page. Supports FTS5 syntax: quoted phrases, OR, and trailing * for prefix matching. Searches the stored index only; run crawl_site first for uncrawled sites.

Signals

Stars
107
Forks
17
Open issues
8
Last commit
Oct 22, 2026 (6 days ago)
Last release
Oct 25, 2026 (3 days ago)

From GitHub (fetched Oct 24, 2026).

Details

Registry name
io.github.Sriram-PR/doc-scraper
Version
2.24.0
Auth
No auth
Packages
mcpb https://github.com/Sriram-PR/doc-scraper/releases/download/v2.24.0/doc-scraper.mcpb
License
Apache-2.0
Repository
github.com/Sriram-PR/doc-scraper
Website
sriram-pr.github.io/doc-scraper/
First seen
Oct 24, 2026
Last checked
Oct 26, 2026
Source
Official MCP Registrygithub-graphql
Cite this page

doc-scraper MCP server. Agent Mart. https://agentmart.dev/connectors/sriram-pr-doc-scraper (last updated 2026-10-26)