judgeval vs LlamaIndex
judgeval and LlamaIndex are both agent frameworks for Python. judgeval is published by JudgmentLabs and LlamaIndex by LlamaIndex. On GitHub, LlamaIndex has 52,641 stars and judgeval has 1,070. Both were last active in Oct 2026. The table lines up where they differ (publisher, license, GitHub stars and weekly downloads) and has the install commands for both. Everything here comes from the listings' public sources and is refreshed daily. judgeval describes itself as: Python SDK for tracing agents and scoring their traces with prompt-based judges, to find failures in production. LlamaIndex: Open-source Python toolkit for RAG and agents over your own data, with event-driven workflows and 300+ integration packages. Licenses: judgeval Apache-2.0, LlamaIndex MIT.
Updated Oct 26, 2026 · Built from catalog data; how we rank and grade
judgeval
JudgmentLabs
Python SDK for tracing agents and scoring their traces with prompt-based judges, to find failures in production.
LlamaIndex
LlamaIndex
Open-source Python toolkit for RAG and agents over your own data, with event-driven workflows and 300+ integration packages.
Who should pick which
- Pick judgeval if you want a stable release (LlamaIndex is marked beta).
- LlamaIndex has the larger community: 52,641 GitHub stars against 1,070.
From listing data only. We don't pick a winner on security beyond the grades; read how grades work.
Side by side
| Fact | judgeval | LlamaIndex |
|---|---|---|
| Type | Harness | Harness |
| Publisher | JudgmentLabs | LlamaIndex |
| License | Apache-2.0 | MIT |
| Pricing | Not stated | Not stated |
| Runs | Not stated | Not stated |
| Auth | Not stated | Not stated |
| Hosts supported | Not stated | Not stated |
| Tools | Not listed yet | Not listed yet |
| Security grade | –Security: not graded yet | –Security: not graded yet |
| Freshness grade | AFreshness: A | AFreshness: A |
| GitHub stars | 1,070 | 52,641 |
| Weekly downloads | 77,683 | 1,391,530 |
| Last release | Oct 21, 2026 | Sep 21, 2026 |
| Last activity | Oct 23, 2026 | Oct 14, 2026 |
Facts from each listing's public sources, checked daily. Tool lists arrive with tool scans.
Install
judgeval
pip install judgeval
LlamaIndex
pip install llama-index
Also consider
- 0.753score
LangChain
LangChain
Python framework of interoperable components and integrations for building agents and LLM apps, with LangGraph and Deep Agents layered above it.
HarnessAFreshness: A★ 148.4K↓ 37.3M/wk - 0.748score
LangGraph
LangChain
Low-level Python orchestration framework for long-running stateful agents, with durable execution, human review points and short- and long-term memory.
HarnessAFreshness: A★ 43.8K↓ 13.1M/wk - 0.732score
CrewAI
CrewAI
Open-source Python framework for multi-agent workflows, with role-based Crews and event-driven Flows.
HarnessAFreshness: A★ 60.1K↓ 644.6K/wk
Questions
- Is judgeval or LlamaIndex more popular?
- LlamaIndex has more GitHub stars (52,641 against 1,070). Stars measure attention, not quality, so check freshness and docs too.
- Are judgeval and LlamaIndex open source?
- Yes: judgeval is Apache-2.0 and LlamaIndex is MIT.
Related
Cite this page
judgeval vs LlamaIndex. Agent Mart. https://agentmart.dev/compare/judgeval-vs-llamaindex (last updated 2026-10-26)