Best agent skills for LLM evaluation

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates.

• Repository has 48.4K stars
• Active in the last 30 days
• Open source (MIT)

★ 48.4KLast active Oct 24, 2026Claude Code, Claude apps, Codex, Cursor +6

Runs the evals of a BMad skill through the agent harness the user works in and reports what they show: baseline against the bare model, variant against a…

• Largest repository here (54,505 stars)
• Active in the last 30 days
• Ships helper scripts

★ 54.5KLast active Oct 14, 2026Claude Code, Claude apps, Codex, Cursor +6

Launch and monitor prime-rl evals — the `uv run eval` entrypoint, its config and CLI shorthands, run directory, resume, logs and metrics.

• Repository has 2.2K stars
• Active in the last 30 days
• Open source (Apache-2.0)

★ 2.2KLast active Oct 23, 2026Claude Code, Claude apps, Codex, Cursor +6

Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI.

• Repository has 1.3K stars
• Active in the last 30 days
• Open source (Apache-2.0)

★ 1.3KLast active Oct 22, 2026Claude Code, Claude apps, Codex, Cursor +6

Plan Fireworks post-training from a task-shaped goal — recommend a method, cookbook entry, data plan, and eval from what the user already said, ask only…

• Published by the vendor
• Active in the last 30 days
• Open source (Apache-2.0)

★ 231Last active Oct 21, 2026Claude Code, Claude apps, Codex, Cursor +6

Make the smallest safe edit that closes a diagnosed gap the model repo owns: a Malloy model edit, or a package skill under skills/.

• Active in the last 30 days
• Open source (MIT)
• Ships helper scripts

★ 118Last active Oct 18, 2026Claude Code, Claude apps, Codex, Cursor +6

Score one analytical answer against a verified golden, and score which of the entities the golden depends on retrieval delivered to the answerer.

• Active in the last 30 days
• Open source (MIT)
• Ships helper scripts

★ 118Last active Oct 18, 2026Claude Code, Claude apps, Codex, Cursor +6

Use when designing or auditing how an LLM or multimodal LLM is evaluated on a clinical task (report generation, VQA, text extraction).

• Active in the last 30 days
• Open source (MIT)
• Ships helper scripts

★ 349Last active Oct 4, 2026Claude Code, Claude apps, Codex, Cursor +6

Analyze a finished coder-eval run and write analysis.md — cluster failures into systemic patterns and recommend fixes.

• Active in the last 30 days
• Open source (Apache-2.0)

★ 154Last active Oct 21, 2026Claude Code, Claude apps, Codex, Cursor +6

Generate and run a coder-eval activation suite for a Claude Code skill.

• Active in the last 30 days
• Open source (Apache-2.0)

★ 154Last active Oct 21, 2026Claude Code, Claude apps, Codex, Cursor +6

Eval-driven skill tuning.

• Repository has 1.2K stars
• Open source (MIT)
• Ships helper scripts

★ 1.2KLast active Sep 11, 2026Claude Code, Claude apps, Codex, Cursor +6

Spawns the decision-scorer agent after architect proposes 2+ variants in an ADR.

• Active in the last 30 days
• Open source (MIT)

★ 105Last active Oct 23, 2026Claude Code, Claude apps, Codex, Cursor +6

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

• Open source (MIT)

★ 863Last active Aug 25, 2026Claude Code, Claude apps, Codex, Cursor +6

This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias…

• Repository has 18.2K stars
• Open source (MIT)
• Ships helper scripts

★ 18.2KLast active Jun 29, 2026Claude Code, Claude apps, Codex, Cursor +6

Suede AI eval design and coverage audit: AI-SPEC, failure-mode rubric with severity scoring, concrete pass/fail eval cases, coverage and infrastructure…

• Active in the last 30 days
• Open source (MIT)

★ 125Last active Oct 24, 2026Claude Code, Claude apps, Codex, Cursor +6