SWE-Bench Pro
Harder, long-horizon software engineering tasks from public and proprietary codebases, with a held-out set.
Reference by Scale AI
IndexedMIT★ 552
About
Whether an agent can complete multi-file, long-horizon code changes across many repositories, judged by the project tests.
Use cases: llm-evaluation, code-generation
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
About the benchmark
- Benchmark site
- labs.scale.com/papers/swe_bench_pro
- Maintainer
- Scale AIarxiv.org
- Launched
- 2025-09arxiv.org
- Leaderboard
- labs.scale.com/leaderboard/swe_bench_pro
We link to the official leaderboard instead of copying scores. Benchmarks get gamed and saturate; we never use them to rank listings.
Signals
- Stars
- 552
- Forks
- 102
- Open issues
- 37
- Last commit
- Sep 22, 2026 (1 month ago)
- Last release
- Sep 22, 2026 (1 month ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- MIT
- Repository
- github.com/scaleapi/SWE-bench_Pro-os
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
Cite this page
SWE-Bench Pro benchmark. Agent Mart. https://agentmart.dev/reference/swe-bench-pro (last updated 2026-10-24)