AutoGen vs judgeval
AutoGen and judgeval are both agent frameworks for Python. AutoGen is published by Microsoft and judgeval by JudgmentLabs. On GitHub, AutoGen has 61,549 stars and judgeval has 1,070. Both were last active in Oct 2026. The table lines up where they differ (publisher, license, GitHub stars and weekly downloads) and has the install commands for both. Everything here comes from the listings' public sources and is refreshed daily. AutoGen describes itself as: Archived Microsoft Python framework for multi-agent apps; Microsoft directs new users to Microsoft Agent Framework. judgeval: Python SDK for tracing agents and scoring their traces with prompt-based judges, to find failures in production. Licenses: AutoGen CC-BY-4.0, judgeval Apache-2.0.
Updated Oct 26, 2026 · Built from catalog data; how we rank and grade
AutoGen
Microsoft
Archived Microsoft Python framework for multi-agent apps; Microsoft directs new users to Microsoft Agent Framework.
judgeval
JudgmentLabs
Python SDK for tracing agents and scoring their traces with prompt-based judges, to find failures in production.
Who should pick which
- Pick AutoGen if you need several agents working together.
- Pick judgeval if you want a stable release (AutoGen is marked experimental).
- AutoGen has the larger community: 61,549 GitHub stars against 1,070.
From listing data only. We don't pick a winner on security beyond the grades; read how grades work.
Side by side
| Fact | AutoGen | judgeval |
|---|---|---|
| Type | Harness | Harness |
| Publisher | Microsoft | JudgmentLabs |
| License | CC-BY-4.0 | Apache-2.0 |
| Pricing | Not stated | Not stated |
| Runs | Not stated | Not stated |
| Auth | Not stated | Not stated |
| Hosts supported | Not stated | Not stated |
| Tools | Not listed yet | Not listed yet |
| Security grade | –Security: not graded yet | –Security: not graded yet |
| Freshness grade | AFreshness: A | AFreshness: A |
| GitHub stars | 61,549 | 1,070 |
| Weekly downloads | 88,975 | 77,683 |
| Last release | Sep 30, 2025 | Oct 21, 2026 |
| Last activity | Oct 16, 2026 | Oct 23, 2026 |
Facts from each listing's public sources, checked daily. Tool lists arrive with tool scans.
Install
AutoGen
pip install autogen-agentchat
judgeval
pip install judgeval
Also consider
- 0.853score
LlamaIndex
LlamaIndex
Open-source Python toolkit for RAG and agents over your own data, with event-driven workflows and 300+ integration packages.
HarnessAFreshness: A★ 52.6K↓ 1.4M/wk - 0.851score
Strands Agents
AWS
Open-source Python and TypeScript SDK for model-driven agents, with tools, MCP, multi-agent patterns and streaming.
HarnessAFreshness: A★ 8.9K↓ 8.9M/wk - 0.841score
GitHub Copilot SDK
GitHub
Multi-language SDKs for embedding the GitHub Copilot agent runtime, with planning, tool calls and file edits, in your apps.
HarnessAFreshness: A★ 10.6K↓ 1.2M/wk
Questions
- Is AutoGen or judgeval more popular?
- AutoGen has more GitHub stars (61,549 against 1,070). Stars measure attention, not quality, so check freshness and docs too.
- Are AutoGen and judgeval open source?
- Yes: AutoGen is CC-BY-4.0 and judgeval is Apache-2.0.
Related
Cite this page
AutoGen vs judgeval. Agent Mart. https://agentmart.dev/compare/autogen-vs-judgeval (last updated 2026-10-26)