35 listed
Protocols & benchmarks
The standards and benchmarks behind agent tooling: MCP, A2A, Agent Skills, AGENTS.md and other protocols, plus benchmarks such as SWE-bench and Terminal-Bench. Protocol pages count the listings that use them; benchmark pages link to the official leaderboards.
Showing 25–35 of 35
- 0.649score
SWE-Bench Pro
Scale AI
Harder, long-horizon software engineering tasks from public and proprietary codebases, with a held-out set.
ReferenceBFreshness: B★ 552 - 0.632score
Agentic Commerce Protocol (ACP)
OpenAI and Stripe
Open checkout standard from OpenAI and Stripe for buyers and AI agents to complete purchases with businesses.
ReferenceCFreshness: C★ 1.6K - 0.623score
GDPval
OpenAI
Real work tasks from professionals across many occupations, graded by experts against human deliverables.
ReferenceAFreshness: A - 0.611score
Agent Network Protocol (ANP)
Agent Network Protocol (ANP)
Open protocol suite for agent identity, discovery and messaging on the open web, built on W3C DIDs.
ReferenceAFreshness: A★ 1.5K - 0.587score
BrowseComp
OpenAI
1,266 hard-to-find questions with short answers for testing how persistently agents can search the web.
ReferenceDFreshness: D★ 4.7K - 0.581score
MCPMark
EVAL SYS, with LobeHub and NUS TRAIL
Stress test for realistic MCP use across Notion, GitHub, filesystem, PostgreSQL and Playwright servers.
ReferenceCFreshness: C★ 467 - 0.577score
WebArena
Carnegie Mellon University researchers
Self-hosted web environment with realistic sites (shopping, forums, code hosting, maps) for testing web agents.
ReferenceDFreshness: D★ 1.6K - 0.557score
OpenAI Apps SDK (ChatGPT plugins)
OpenAI
OpenAI's toolkit for building ChatGPT plugins from MCP servers, skills and optional in-chat UI, now documented under Plugins.
ReferenceDFreshness: D★ 2.4K - 0.542score
MLE-bench
OpenAI
75 Kaggle competitions turned into offline tasks for testing machine learning engineering agents.
ReferenceDFreshness: D★ 1.8K - 0.508score
LiveCodeBench
Researchers at UC Berkeley, MIT and Cornell
Continuously refreshed coding benchmark built from new contest problems to limit training-data contamination.
ReferenceFFreshness: F★ 964 - 0.501score
ARC-AGI-2
ARC Prize Foundation
Grid puzzles that are easy for people but hard for AI, testing abstraction and reasoning on new problems.
ReferenceFFreshness: F★ 746