“LLM evaluation”

This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias…

Open-source LLM evaluation framework, similar to Pytest, for unit testing LLM apps and agents

Use when designing or auditing how an LLM or multimodal LLM is evaluated on a clinical task (report generation, VQA, text extraction).

End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — container setup, vLLM/SGLang/ATOM serving, lm-eval / lighteval / evalscope benchmarks.

End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — container setup, vLLM/SGLang/ATOM serving, lm-eval / lighteval / evalscope benchmarks.

End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — docker container setup OR host (no-docker) runtime, vLLM/SGLang/ATOM serving, lm-eval / lighteval…

This skill should be used when building agent evaluation systems: deterministic checks, regression suites, multi-dimensional rubrics, quality gates…

Evaluates LLM and agent products with Braintrust—offline/online evals, SDK/CI workflows, eval platform design, Zapier/Notion product ops, and Loop-style…

Add or modify RLM evaluations, including Harbor agent tasks, LLM-as-judge utilities, rubrics, cases, or long-context benchmarks while keeping model cost…

Toolkit for evaluating LLM applications with objective metrics, test data generation and feedback loops

Design and evaluate LLM product features, retrieval, validated outputs, latency, and cost controls.

Master LLM-as-a-Judge evaluation techniques including direct scoring, pairwise comparison, rubric generation, and bias mitigation.

Use when validating, benchmarking, or monitoring LLM application performance.

Build evaluation frameworks for agent systems.

Design or improve evaluation for an LLM-based feature (RAG, agents, classification, extraction, chat).

A.I.G Scanner — AI security scanning for infrastructure, AI tools / skills, AI Agents, and LLM jailbreak evaluation via Tencent Zhuque Lab AI-Infra-Guard.

Use when maintaining repository Markdown, Copilot instructions, Codex AGENTS.md, repo skills, prompts, or LLM evaluation tasks.

INVOKE THIS SKILL for LLM-as-judge evaluation workflows on Arize: creating/updating evaluators, running evaluations on spans or experiments, tasks…

Evaluate a call recording with a multimodal LLM.

Evaluation — an observability concept covering evaluation, score, drift, shadow, benchmark.

Renamed to /model-assessment in v6 (removed in v7).

Use when designing, configuring, running, or troubleshooting NeMo Agent Toolkit evaluations, datasets, evaluator selection, ATIF surfaces, quality gates…

Evaluate one design or a user-approved maturity-mapped batch through a transparent evidence-based rubric.

Use when the user wants proof that finished work meets its acceptance criteria, not just that tests pass: 'verify it is actually done', 'check it against the…