Everything in the catalog

Harder, long-horizon software engineering tasks from public and proprietary codebases, with a held-out set.

Open checkout standard from OpenAI and Stripe for buyers and AI agents to complete purchases with businesses.

Real work tasks from professionals across many occupations, graded by experts against human deliverables.

Open protocol suite for agent identity, discovery and messaging on the open web, built on W3C DIDs.

1,266 hard-to-find questions with short answers for testing how persistently agents can search the web.

Stress test for realistic MCP use across Notion, GitHub, filesystem, PostgreSQL and Playwright servers.

Self-hosted web environment with realistic sites (shopping, forums, code hosting, maps) for testing web agents.

OpenAI's toolkit for building ChatGPT plugins from MCP servers, skills and optional in-chat UI, now documented under Plugins.

75 Kaggle competitions turned into offline tasks for testing machine learning engineering agents.

Continuously refreshed coding benchmark built from new contest problems to limit training-data contamination.

Grid puzzles that are easy for people but hard for AI, testing abstraction and reasoning on new problems.