“LLM evaluation”

Expert in Langfuse - the open-source LLM observability platform.

Evaluate a Codex skill's routing accuracy and task quality with a human-approved, isolated workflow.

Use this skill when designing, running, interpreting, or reporting Agent Skill evaluations, selecting cases or judges, and analyzing trigger, benchmark, or…

Use this skill when designing, running, interpreting, or reporting Agent Skill evaluations, selecting cases or judges, and analyzing trigger, benchmark, or…

Open-source CLI and library for evaluating and red-teaming LLM apps and agents

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents…

Use when evaluating agent outputs, designing benchmarks, or building self-evaluation rubrics.

Use for evaluating DSPy programs with Evaluate, answer_exact_match, SemanticF1, custom metrics, baselines, and program comparisons.

Evaluate and benchmark skills, prompts, or model changes with test cases, assertions, and quantitative comparison.

Aplicar model evaluation cuando el encargo requiera casos representativos y criterios.

Use when designing or implementing an evaluation loop for AI agent outputs — reflection loops, evaluator-optimizer pipelines, LLM-as-judge scoring, or…

Output is generated text, summarization, QA, agent traces or structured responses.

Use when comparing workers or models on a bounded task set.

Evaluate, select, and contract with vendors and SaaS tools.

Evaluate existing capabilities during rebuild programmes.

Evaluate third-party logistics providers.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).

Evaluate against Nielsen's 10 heuristics (or custom set), severity ratings, prioritized findings.

LLM & VLM evaluation suite for home security AI applications

Evaluate AI agent outputs — hallucination detection, quality scoring, behavioral testing, and CI/CD pipelines.

Evaluate agent and skill behavior or routing.

Evaluate the reliability of sources and flag conflicts.

L3 recipe that runs a Torch LLM PTQ end-to-end for AMD Quark — for PyTorch / HuggingFace transformers models (safetensors input): quantize → validate →…

Private evaluation dependency for Agent Skill behavior claims.