249 listed
Agent infrastructure
The services agents run on: memory, code sandboxes, browsers, voice, search and scraping APIs, vector databases, LLM gateways, inference, durable execution and hosting. Facts such as self-hosting and compliance come from each vendor's own pages, with links.
Showing 1–17 of 17
- 0.888score
Firecrawl
Firecrawl
API to search, scrape and interact with the web for AI agents, open source and available as a hosted service.
InfrastructureAFreshness: A★ 194.9Kscraping - 0.879score
Crawl4AI
Crawl4AI
Open-source web crawler that turns pages into LLM-ready Markdown, with an optional hosted cloud API.
InfrastructureAFreshness: A★ 86.2Kscraping - 0.843score
ScrapeGraphAI
ScrapeGraphAI
Python library and API that uses LLMs to extract structured data from websites and local documents.
InfrastructureAFreshness: A★ 32.2Kscraping - 0.824score
Crawlee
Apify
Web scraping and browser automation library for Node.js, covering link discovery, data extraction and storage.
InfrastructureAFreshness: A★ 26.2Kscraping - 0.735score
Spider
Spider
Rust crawling engine for scraping and crawling at scale, runnable locally or through the Spider Cloud API.
InfrastructureAFreshness: A★ 2.8Kscraping - 0.686score
Jina Reader
Jina AI (Elastic)
Converts any URL into LLM-friendly input and searches the web, through the r.jina.ai and s.jina.ai endpoints.
InfrastructureCFreshness: C★ 12.2Kscraping - 0.597score
Apify
Apify
Marketplace of ready-to-run web scraping and automation tools called Actors, runnable through an API.
InfrastructureAFreshness: Ascraping - 0.596score
Scrapfly
Scrapfly
Web scraping, cloud browser, screenshot, extraction and crawler APIs under one API key, with managed proxies.
InfrastructureAFreshness: Ascraping - 0.595score
Nimble
Nimble
Web search, extraction, crawling and research-agent APIs that return live web data to AI agents.
InfrastructureAFreshness: Ascraping - 0.595score
Oxylabs Web Scraper API
Oxylabs
Web Scraper API that returns structured data from public websites, with CAPTCHA and JavaScript rendering handled.
InfrastructureAFreshness: Ascraping - 0.595score
ScrapingBee
ScrapingBee
Web scraping API that handles proxies, headless Chrome and anti-bot defenses, returning JSON or Markdown extractions.
InfrastructureAFreshness: Ascraping - 0.594score
Bright Data Web Scraper API
Bright Data
Pre-built scrapers that return structured data from sites such as LinkedIn, Amazon and Instagram.
InfrastructureAFreshness: Ascraping - 0.594score
ScraperAPI
ScraperAPI
Scraping API that handles proxies, browsers and CAPTCHAs for public websites, with structured data endpoints.
InfrastructureAFreshness: Ascraping - 0.592score
Diffbot
Diffbot
APIs that turn public web pages and entities into structured, verified facts for AI applications.
InfrastructureAFreshness: Ascraping - 0.592score
Olostep
Olostep
Search, scrape, crawl and answer API for AI agents that returns clean Markdown, HTML and structured JSON.
InfrastructureAFreshness: Ascraping - 0.592score
ZenRows
ZenRows
Web data infrastructure for agents: Fetch, Extract, Crawl, Batch and Browser Sessions over API, CLI, MCP or SDK.
InfrastructureAFreshness: Ascraping - 0.592score
Zyte API
Zyte
Web scraping API with automatic ban handling, browser rendering, sessions and extraction, plus a Python client.
InfrastructureAFreshness: Ascraping