Skip to content
Agentmart
Submit a tool

eval

Run real tests or evals against a plan file, store results per version and compare with the previous baseline.

Skill by smlfg

IndexedMIT
Write a review

Install

Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.

npx agentmarket install skill/smlfg-eval

Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

We use your email only for this newsletter. Privacy

About

Run real tests or evals against a plan file, store results per version and compare with the previous baseline. Use when the user wants to evaluate a version of a system rather than review its code.

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Permissions

What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.

Tools
No tools found
Secrets
None asked for
Install scripts
None found

Network hosts it talks to 1

  • codeload.github.com · found in install command

Files it touches

  • ~/.claude/skills/eval

What's inside

Skill name
eval
Description
Run real tests or evals against a plan file, store results per version and compare with the previous baseline. Use when the user wants to evaluate a version of a system rather than review its code.
License
MIT
Instructions
21 lines in SKILL.md · view on GitHub

Files (1)

  • SKILL.md

SKILL.md (first lines)

# Eval

1. Read the plan file the user names.
2. **Always run actual tests** (pytest or the project's eval framework). Never evaluate by code review alone.
3. Write results into that version's eval folder (e.g. `evals/<version>/`).
4. Compare against the previous version's baseline and call out regressions first.
5. Commit the results with the message `eval: <version> - <summary>` (only if the user allows commits in this repo).

## Example

```text
evals/v0.4/results.json   42/50 pass (v0.3: 39/50)
Regressions: case_17 (date parsing), case_31 (empty input)
Improvements: case_02, case_09, case_22, case_40, case_44
```

## Use case

You iterate on a prompt pipeline and keep claiming "v4 feels better". `/eval plan.md` runs the same suite on v4, writes the numbers next to v3's, and tells you exactly which cases got worse.

From smlfg/HAI-skills-registry, MIT.

More from smlfg/HAI-skills-registry

Signals

Stars
0
Forks
0
Open issues
0
Last commit (this folder)
Oct 15, 2026 (15 days ago)

From GitHub (fetched Oct 24, 2026).

Details

License
MIT
Repository
github.com/smlfg/HAI-skills-registry
Website
human-agent-interface.com
First seen
Oct 24, 2026
Last checked
Oct 24, 2026
Source
GitHubgithub-graphql

More from smlfg

Cite this page

eval skill. Agent Mart. https://agentmart.dev/skills/smlfg-eval (last updated 2026-10-24)