“LLM evaluation”

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).

Evaluate, test, and recommend tools and platforms with weighted scoring, TCO analysis, and strategic adoption planning.

Design LLM-as-judge rubrics, direct scores and pairwise comparisons.

Provides qualitative-first, evidence-traceable developmental review of scholarly works and audit low-stakes research-assessment rubrics with optional local…

Evaluate whether a the reference deployment skill actually works — blind runner + separate judge against a bar, rubric scoring, pass-rate on re-runs, and…

Evaluate prompt, model, retrieval, and agent changes with repeated graded trials.

Automated regression testing and continuous evaluation for LLM prompts, ensuring deterministic outputs and preventing prompt drift.

Use when evaluating MCP-enabled agents, tool-using LLM systems, multi-server workflows, tool-selection quality, trajectory quality, task completion, or MCP…

Use Langfuse for LLM observability and evaluation and look up Langfuse documentation.

Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR).

Evaluate and improve Claude Code commands, skills, and agents.

Use when creating or updating agent evaluation suites.

Systematic evaluation of Retrieval-Augmented Generation (RAG) pipelines measuring faithfulness, answer relevance, context recall, and precision.

Systematically evaluate scholarly work using the ScholarEval framework, providing structured assessment across research quality dimensions including problem…

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or…

Continuously assess agent quality and detect behavioral regressions using LLM-as-a-Judge rubrics, pairwise evaluation, and golden datasets.

Calibrate a reviewer persona with few-shot rubric examples so skepticism stays consistent and doesn't drift lenient over long runs.

Build evaluation frameworks for agent systems

Expert in Langfuse - the open-source LLM observability platform.

Evaluate a Codex skill's routing accuracy and task quality with a human-approved, isolated workflow.

Use this skill when designing, running, interpreting, or reporting Agent Skill evaluations, selecting cases or judges, and analyzing trigger, benchmark, or…

Use this skill when designing, running, interpreting, or reporting Agent Skill evaluations, selecting cases or judges, and analyzing trigger, benchmark, or…

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents…

Use when evaluating agent outputs, designing benchmarks, or building self-evaluation rubrics.