Skip to content
Agentmart
Submit a tool

Alternatives to agent-evals

agent-evals is an agent skill by devops-skills: Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. If it doesn't fit your setup, these 12 agent skills cover similar ground. They are ordered by how closely they match agent-evals and by our rank, which weighs adoption, freshness and documentation; money never changes the order. bmad-eval leads the list (supports Python). 1 of them is published by the company behind the product. Each entry shows data reasons to switch, and you can open a side-by-side comparison with agent-evals.

Updated Oct 26, 2026 · Built from catalog data; how we rank and grade

You're replacing

agent-evals

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates.

12 alternatives

Ordered by how closely each matches agent-evals (shared product, category or stated alternative) and by rank.

  1. 1
    bmad-eval

    bmad-code-org

    Runs the evals of a BMad skill through the agent harness the user works in and reports what they show: baseline against the bare model, variant against a…

    • Supports Python

    ★ 54.5KLast active Oct 14, 2026Claude Code, Claude apps, Codex, Cursor +6Compare →

  2. 2
    eval-improve

    malloydata

    Make the smallest safe edit that closes a diagnosed gap the model repo owns: a Malloy model edit, or a package skill under skills/.

    • Supports Python

    ★ 118Last active Oct 18, 2026Claude Code, Claude apps, Codex, Cursor +6

  3. 3
    eval-answer

    malloydata

    Score one analytical answer against a verified golden, and score which of the entities the golden depends on retrieval delivered to the answerer.

    • Supports Python

    ★ 118Last active Oct 18, 2026Claude Code, Claude apps, Codex, Cursor +6

  4. 4
    mllm-eval

    Aperivue

    Use when designing or auditing how an LLM or multimodal LLM is evaluated on a clinical task (report generation, VQA, text extraction).

    • Supports Python and bash

    ★ 349Last active Oct 4, 2026Claude Code, Claude apps, Codex, Cursor +6

  5. 5
    decision-eval

    avelikiy

    Spawns the decision-scorer agent after architect proposes 2+ variants in an ADR.

    • Active in the last 30 days
    • Open source (MIT)

    ★ 105Last active Oct 23, 2026Claude Code, Claude apps, Codex, Cursor +6

  6. 6
    eval-harness

    sangrokjung

    Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

    • Open source (MIT)

    ★ 863Last active Aug 25, 2026Claude Code, Claude apps, Codex, Cursor +6

  7. 7
    eval

    PrimeIntellect-ai

    Launch and monitor prime-rl evals — the `uv run eval` entrypoint, its config and CLI shorthands, run directory, resume, logs and metrics.

    • Repository has 2.2K stars
    • Active in the last 30 days

    ★ 2.2KLast active Oct 23, 2026Claude Code, Claude apps, Codex, Cursor +6Compare →

  8. 8
    agent-plugin-eval

    fabricioctelles

    Audit, score, and compare repositories containing portable Agent Plugins against the official Agent Plugins specification.

    • Supports Python

    ★ 119Last active Sep 18, 2026Claude Code, Claude apps, Codex, Cursor +6

  9. 9
    wandb-eval-tables

    wandbOfficial

    Convert W&B Table artifacts into non-destructive EvalTable previews with scan-first planning, typed input/output/score columns, bounded batches, verification…

    • Published by the vendor
    • Supports Python

    ★ 71Last active Aug 6, 2026Claude Code, Claude apps, Codex, Cursor +6

  10. 10
    eval-creator-ci

    pskoett

    [Beta] CI-only eval regression runner using gh-aw (GitHub Agentic Workflows).

    ★ 320Last active Sep 5, 2026Claude Code, Claude apps, Codex, Cursor +6

  11. 11
    eval-creator

    pskoett

    [Beta] Creates permanent eval cases from promoted learnings and runs regression checks against them.

    ★ 320Last active Sep 5, 2026Claude Code, Claude apps, Codex, Cursor +6

  12. 12
    eval-skills

    dzhng

    Eval and improve a skill against golden cases — run the target skill blind in a fresh, context-free subagent on each example input, grade the artifact against…

    • Repository has 1.4K stars
    • Open source (MIT)

    ★ 1.4KLast active Jul 5, 2026Claude Code, Claude apps, Codex, Cursor +6

Related