SWE-bench Verified
Human-validated set of 500 real GitHub issues from Python repositories for testing coding agents.
Reference by SWE-bench team, created with OpenAI Preparedness
IndexedMIT★ 6,077
About
Whether an agent can write a patch that resolves a real GitHub issue so the repository's hidden tests pass.
Use cases: llm-evaluation, code-generation
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
About the benchmark
- Benchmark site
- www.swebench.com/verified.html
- Maintainer
- SWE-bench team, created with OpenAI Preparednessgithub.com
- Launched
- 2024-08-13github.com
- Leaderboard
- www.swebench.com/verified.html
We link to the official leaderboard instead of copying scores. Benchmarks get gamed and saturate; we never use them to rank listings.
Signals
- Stars
- 6,077
- Forks
- 1,031
- Open issues
- 27
- Last commit
- Sep 2, 2026 (1 month ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- MIT
- Repository
- github.com/SWE-bench/SWE-bench
- Website
- www.swebench.com/
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
Cite this page
SWE-bench Verified benchmark. Agent Mart. https://agentmart.dev/reference/swe-bench-verified (last updated 2026-10-24)