Agent Eval
Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.
Connector by RudrenduPaul · GitHub account verified by the MCP Registry
Install
Works with Claude Code, Codex, VS Code, Cursor, Windsurf, Claude Desktop. Read the repository before you give any tool access to your data.
npx agentmarket install connector/rudrendupaul-agent-eval
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this server can do and what it asks for, from its tool list and install config. Checked Oct 25, 2026.
- Tools
- 0 read · 1 write · 0 delete
- Secrets
- None asked for
- Install scripts
- Not scanned yet
- Runs as
- Python package
Tools by what they do 1
Read from the source code; the running server may list different tools.
Write 1 · Create, change, send or run things.
run
Tools (1)
Read from the source code; the running server may differ. Descriptions are the server’s own words; hidden characters are shown as [U+…].
runRun the agent-regress CLI as a subprocess and return its parsed --json output, so an agent can get a statistically grounded verdict on whether an LLM agent's behavior actually shifted between two versions instead of eyeballing two numbers. Call this after you already have two pre-computed sets of per-run scores (one JSON array of numbers per agent version, produced by whatever harness ran each version) and you need to know whether the difference between them is a real regression/improvement or just run-to-run noise. Do not call it to generate those scores in the first place -- it only compare…
Signals
- Stars
- 0
- Forks
- 0
- Open issues
- 1
- Weekly downloads
- 63
- Last commit
- Sep 25, 2026 (1 month ago)
- Last release
- Sep 25, 2026 (1 month ago)
- Last package publish
- Aug 11, 2026 (2 months ago)
From GitHub (fetched Oct 24, 2026) and the package registry.
Details
- Registry name
io.github.RudrenduPaul/agent-eval- Version
- 0.1.8
- Auth
- No auth
- Packages
- pypi agent-regress-cli
- License
- Apache-2.0
- Repository
- github.com/RudrenduPaul/agent-eval
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
More from RudrenduPaul
- 0.600score
Neuronscope
RudrenduPaul
Traces which neurons and attention heads drive a language model's output via MCP tools.
ConnectorAFreshness: ALocal★ 1↓ 45/wkClaude Code +5 - 0.594score
Evolveguard
RudrenduPaul
Wraps the evolveguard CLI as a single generic MCP tool for agent-skill file safety checks.
ConnectorAFreshness: ALocal★ 1↓ 36/wkClaude Code +5 - 0.594score
Skillguard
RudrenduPaul
Security scanner for third-party AI agent-skill files (SKILL.md, hooks, scripts).
ConnectorAFreshness: ALocal★ 2↓ 27/wkClaude Code +5 - 0.593score
Agenticworkspace
RudrenduPaul
Wraps the AgenticWorkspace CLI as a single generic MCP tool for repo workspace scans.
ConnectorAFreshness: ALocal↓ 62/wkClaude Code +5 - 0.593score
Llmscout
RudrenduPaul
Zero-config SEO and GEO checker: 21 technical-SEO and generative-engine checks for a live site.
ConnectorAFreshness: ALocal↓ 61/wkClaude Code +5 - 0.592score
Agent Observability
RudrenduPaul
AI agent observability with deterministic record/replay for debugging agent failures.
ConnectorAFreshness: ALocal↓ 60/wkClaude Code +5
Cite this page
Agent Eval MCP server. Agent Mart. https://agentmart.dev/connectors/rudrendupaul-agent-eval (last updated 2026-10-24)