“LLM evaluation”

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

Use this skill when you need to design LLM evaluation datasets, judges, metrics, and human-review boundaries; triggers include LLM Eval 设计 and llm evaluation…

Use this skill when you need to design LLM evaluation datasets, judges, metrics, and human-review boundaries; triggers include llm evaluation design.

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Platform for monitoring, troubleshooting and evaluating LLM applications and AI agents

MCP as a Judge: a behavioral MCP that strengthens AI coding assistants via explicit LLM evaluations

Indicative US 409A and UK EMI share value estimates from cap table and financial inputs.

Deterministic finance tools for AI agents — IRR, NPV, MOIC, DCF, WACC and sensitivity.

Open-source framework that builds, evaluates and self-improves multi-agent LLM workflows from a goal.

JavaScript and TypeScript framework of interoperable components for building LLM apps and agents, with LangGraph.js and Deep Agents above it.

Ruby library for LLM-powered apps with a unified LLM interface, prompt tools, output parsers, RAG and assistants.

AGENTEVAL

tanishbhandari24

Evaluate AI agents while you build them — benchmarks, traces, failure autopsy, and regression compare.

BenchClaw — P2PCLAW Agent Benchmark

agnuxo1

Benchmark any AI agent (Claude, GPT, Gemini, Qwen, Kimi, DeepSeek, Grok, Llama…) on the P2PCLAW network.

CLAUDE.md from HolmesGPT/holmesgpt: 913 lines covering Repository Overview, Development Commands, Architecture Overview, Testing Framework and more.

Subagent "modeling-test-specialist" from NVIDIA/TensorRT-LLM: Runs model-level and module-level tests for TensorRT-LLM.

Hobby project for LLM code editing in large codebases, run via Docker with a web UI and a SWE-bench harness.

Retool platform for building agents that combine code, LLM decisions and human review, connected to external systems of record

Inference infrastructure for AI teams: hosted open-source and fine-tuned models, plus tracing and evaluation.