“LLM evaluation”

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

Use this skill when you need to design LLM evaluation datasets, judges, metrics, and human-review boundaries; triggers include LLM Eval 设计 and llm evaluation…

Use this skill when you need to design LLM evaluation datasets, judges, metrics, and human-review boundaries; triggers include llm evaluation design.

This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias…

Design and analyze reproducible LLM evaluations for investment research.

ML model evaluation and experiment tracking -- LLM benchmarking with lm-evaluation-harness (MMLU, GSM8K, HumanEval, 60+ tasks), experiment tracking and…

This skill should be used when the user asks to "evaluate LLM output quality", "set up LLM-as-judge", "build an eval rubric", "compare model outputs…

Use when designing or auditing how an LLM or multimodal LLM is evaluated on a clinical task (report generation, VQA, text extraction).

Prompt engineering and LLM evaluation.

LLM/AI output evaluation patterns — RAGAS, DeepEval, LLM-as-judge, hallucination detection, prompt-injection testing, OWASP LLM Top 10, distribution shift…

End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — container setup, vLLM/SGLang/ATOM serving, lm-eval / lighteval / evalscope benchmarks.

End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — container setup, vLLM/SGLang/ATOM serving, lm-eval / lighteval / evalscope benchmarks.

End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — docker container setup OR host (no-docker) runtime, vLLM/SGLang/ATOM serving, lm-eval / lighteval…

Systematic evaluation (evals) for LLM and AI products.

This skill should be used when building agent evaluation systems: deterministic checks, regression suites, multi-dimensional rubrics, quality gates…

Evaluates LLM and agent products with Braintrust—offline/online evals, SDK/CI workflows, eval platform design, Zapier/Notion product ops, and Loop-style…

Add or modify RLM evaluations, including Harbor agent tasks, LLM-as-judge utilities, rubrics, cases, or long-context benchmarks while keeping model cost…

Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks…

Design and evaluate LLM product features, retrieval, validated outputs, latency, and cost controls.

Master LLM-as-a-Judge evaluation techniques including direct scoring, pairwise comparison, rubric generation, and bias mitigation.

Author continuously-running online evaluations in PostHog AI observability, grounded in real failure modes you've identified.

Use when validating, benchmarking, or monitoring LLM application performance.

Build evaluation frameworks for agent systems.

Design or improve evaluation for an LLM-based feature (RAG, agents, classification, extraction, chat).