PyreCrawl

Install

Works with Claude Code, Codex, VS Code, Cursor, Windsurf, Claude Desktop. Read the repository before you give any tool access to your data.

npx agentmarket install connector/sanggonboy-pyrecrawl

About

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this server can do and what it asks for, from its tool list, install config and code. Checked Oct 26, 2026.

Tools

8 read · 0 write · 0 delete · 5 not yet classified

Secrets

None asked for

Install scripts

None found

Runs as

Python package

Tools (13)

Read from the source code; the running server may differ. Descriptions are the server’s own words; hidden characters are shown as [U+…].

• batch_scrape

  Scrape MANY URLs in ONE call (parallel, deduped, cache-aware). Use when the user provides multiple URLs or you have a list of pages to fetch. More efficient than calling `scrape` N times. Args: urls: Target URLs (deduped automatically; empties dropped). prefer: "auto" | "fast" | "stealth" | "llm". timeout: per-URL timeout in seconds. max_concurrency: parallel workers (default 4). include_html: include raw HTML per result (large; off by default). Returns {requested, unique, succeeded, failed, results[]}. Per-URL failures are isolated — other URLs still succeed. Returns: …

• cache

  Inspect the response cache: stats, clear, enable, or disable. Use this to check cache hit rates before large batch jobs, or to clear stale cached responses when a site's content has changed. Args: action: "stats" (default) | "clear" | "disable" | "enable".

• crawl

  Multi-page crawl: discover URLs on `root`, then scrape each. Use this when the user wants to crawl an entire site section or docs, or needs multiple pages scraped in bulk. For single pages use `scrape`; for research questions use `deep_research`. Args: root: start URL. max_pages: hard cap on pages scraped. css_selector: scope each page's html/markdown to the matched element (non-llm: lxml re-scope of the fetched HTML; llm: native crawl4ai css_selector). prefer: "auto" | "fast" | "stealth" | "llm" (llm = Crawl4AI BFS deep-crawl). include_paths: regex — keep…

• deep_research

  Search the web, then pull the top sources as EVIDENCE (no LLM synthesis). PRIMARY RESEARCH TOOL — use when the user asks to research, investigate, deep-dive, fact-check, or learn about a topic. Returns a ``citations`` list with stable [n] numbers and an ``evidence`` list of per-source markdown — the agent does the synthesis from evidence. Multi-pass mode: set iterations=2-3 to auto-run additional searches with refined queries (alternatives, criticism, latest developments) and append deduplicated evidence. Each pass adds up to ``scrape_top`` new sources. Args: query: search string. l…

• document

  Extract text from a PDF/DOCX/PPTX URL → markdown (no browser). Use when the user shares a link to a document (PDF, Word, PowerPoint) and wants its text content. Also useful after `search_papers` to get full text from a paper's pdf_url. Content-type sniffed and routed to pypdf / python-docx / python-pptx. Optional deps — install with `pip install 'pyrecrawl[docs]'`.

• extract

  Scrape + structured extraction using a CSS-based JSON schema. Use when the user wants structured data (tables, lists, product info) extracted from a page. Define a CSS schema to target specific elements. The schema is a JsonCssExtractionStrategy schema: { "name": "PageItems", "baseSelector": "div.item", "fields": [{"name": "title", "selector": "h2", "type": "text"}, ...] } Returns parsed JSON in `data`.

• health

  Verify PyreCrawl is working: engine versions, dependencies, update status. Use this at the start of a session or before a large scraping job to confirm all engines are installed and up to date.

• map_site

  Enumerate all internal URLs reachable from `root`. Use when the user wants to map a site's structure or find all pages before crawling. Often paired with `crawl` or `batch_scrape`. Args: root: Website root (e.g. "https://example.com/docs"). include_pattern: Optional regex; only URLs matching are returned. limit: Hard cap on returned URLs.

• monitor

  Track a URL over time and report meaningful content changes. Use when the user wants to watch a page for updates (price changes, new blog posts, status updates). Ask me to 'set up monitoring for <url>' for a guided setup playbook. Args: url: target URL. action: "check" | "history" | "forget". prefer: ladder preference, same as ``scrape``. css_selector: scope the diff to one element (so banner / nav changes don't trigger false positives). Snapshots persist under ``PYRECRAWL_MONITOR_DIR`` (default ``~/.pyrecrawl/monitors/``). ``check`` returns ``status`` of ``new`` | `…

• scrape

  Scrape a single URL → LLM-ready markdown. Use this when the user shares a URL and wants its content (read, analyze, summarize, extract). Auto-escalates through fast→stealth→llm when blocked. Args: url: Target URL (http/https). prefer: "auto" | "fast" | "stealth" | "llm". auto = fast first, escalate to stealth on block/short page. fast = cheap HTTP only (no JS). stealth = real Chromium + Cloudflare solver. llm = full Crawl4AI browser + BM25 fit-markdown. timeout: per-attempt timeout in seconds. include_html: include raw HTML in t…

• search

  Web search via DuckDuckGo HTML (no API key required). Use this for targeted searches where you need anti-bot bypass (Cloudflare protection on DDG). For simple searches, the built-in web_search may suffice. For research questions, prefer `deep_research` (search + scrape + citations). Returns [{url, title, snippet}, ...]. The smart ladder bypasses DDG's bot detection if needed. Returns: {query, results: [{url, title, snippet}], count} or {error, query} on failure.

• search_papers

  Search academic papers via arXiv or Crossref — no API keys. Args: query: free-text search, e.g. "transformer attention scaling laws". limit: max results (1-25 arXiv / 1-20 crossref). source: "arxiv" (CS/physics/math preprints, default) or "crossref" (all fields, DOI-backed). category: optional arXiv category filter, e.g. "cs.LG", "cs.CV". Returns papers with id/url/pdf_url/title/authors/summary/published. Feed pdf_url into the `document` tool to extract full text. Returns: {query, source, papers: [{title, authors, abstract, url, pdf_url?, ...}]} or {error, qu…

• session

  Drive a persistent browser session — cookies & JS state kept across calls. Use for login walls and multi-step flows the one-shot ladder can't handle (e.g. user needs to log in first, then scrape a protected page). For simple pages use `scrape`; for research use `deep_research`. One-shot scrape has no session memory; here each action runs against the same live page. Args: session: named session; reuse the same name to keep state. action: "open" (url) — navigate, returns url/title/status "click" (selector) — click an element "fill" (selector, text) — type int…

Signals

Stars

22

Forks

2

Open issues

0

Contributors

3

Weekly downloads

77

Last commit

Sep 21, 2026 (1 month ago)

Last release

Sep 21, 2026 (1 month ago)

Last package publish

Sep 21, 2026 (1 month ago)

From GitHub (fetched Oct 24, 2026) and the package registry.

Details

Registry name

io.github.SanggonBoy/PyreCrawl

Version

0.8.1

Auth

No auth

Packages

pypi pyrecrawl

License

MIT

Repository

github.com/SanggonBoy/PyreCrawl

Website

pypi.org/project/pyrecrawl/

First seen

Oct 24, 2026

Last checked

Oct 24, 2026

Source

Official MCP Registrygithub-graphql