“LLM evaluation”

Use when building, running, or debugging evaluations for LLM or agent components: golden sets, regression evals, LLM-as-judge, rubric scoring…

Inspects the plan, milestones, tasks, and generated session schedules to certify they meet formatting guidelines, safety constraints, and scheduling…

Evaluate agent behavior with versioned cases and explicit verifiers.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).

Evaluate, test, and recommend tools and platforms with weighted scoring, TCO analysis, and strategic adoption planning.

Design LLM-as-judge rubrics, direct scores and pairwise comparisons.

Provides qualitative-first, evidence-traceable developmental review of scholarly works and audit low-stakes research-assessment rubrics with optional local…

Evaluate whether a the reference deployment skill actually works — blind runner + separate judge against a bar, rubric scoring, pass-rate on re-runs, and…

Open-source platform for tracing, evaluating and improving AI agents

Evaluate prompt, model, retrieval, and agent changes with repeated graded trials.

Automated regression testing and continuous evaluation for LLM prompts, ensuring deterministic outputs and preventing prompt drift.

Use when evaluating MCP-enabled agents, tool-using LLM systems, multi-server workflows, tool-selection quality, trajectory quality, task completion, or MCP…

Use Langfuse for LLM observability and evaluation and look up Langfuse documentation.

Open-source LLM observability and evaluation platform with agent tracing and test suites

Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR).

Evaluate and improve Claude Code commands, skills, and agents.

Use when creating or updating agent evaluation suites.

Systematic evaluation of Retrieval-Augmented Generation (RAG) pipelines measuring faithfulness, answer relevance, context recall, and precision.

Systematically evaluate scholarly work using the ScholarEval framework, providing structured assessment across research quality dimensions including problem…

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or…

Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer…

Continuously assess agent quality and detect behavioral regressions using LLM-as-a-Judge rubrics, pairwise evaluation, and golden datasets.

Calibrate a reviewer persona with few-shot rubric examples so skepticism stays consistent and doesn't drift lenient over long runs.

Build evaluation frameworks for agent systems