“LLM evaluation”

Use for evaluating DSPy programs with Evaluate, answer_exact_match, SemanticF1, custom metrics, baselines, and program comparisons.

Evaluate and benchmark skills, prompts, or model changes with test cases, assertions, and quantitative comparison.

Aplicar model evaluation cuando el encargo requiera casos representativos y criterios.

Use when designing or implementing an evaluation loop for AI agent outputs — reflection loops, evaluator-optimizer pipelines, LLM-as-judge scoring, or…

Output is generated text, summarization, QA, agent traces or structured responses.

Use when comparing workers or models on a bounded task set.

Evaluate, select, and contract with vendors and SaaS tools.

Evaluate existing capabilities during rebuild programmes.

Evaluate third-party logistics providers.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).

Evaluate against Nielsen's 10 heuristics (or custom set), severity ratings, prioritized findings.

LLM & VLM evaluation suite for home security AI applications

Evaluate AI agent outputs — hallucination detection, quality scoring, behavioral testing, and CI/CD pipelines.

Evaluate agent and skill behavior or routing.

Evaluate the reliability of sources and flag conflicts.

L3 recipe that runs a Torch LLM PTQ end-to-end for AMD Quark — for PyTorch / HuggingFace transformers models (safetensors input): quantize → validate →…

Private evaluation dependency for Agent Skill behavior claims.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents…

Independent assessment without debate context — adversarial loop prevents gaming

Testing and benchmarking LLM agents including behavioral testing,

Use for search evaluation, a POC or bake-off, Parallel vs Exa/Tavily comparisons, SimpleQA/DeepSearchQA benchmarks, or diagnosing search evaluation results.

Use after completing any non-trivial task.

Settle a "should we switch TTS engines / is X better than Edge TTS" question by measuring produced audio files, not by reading library docs or trusting…

Evaluate technology candidates for a single decision-area using a 6-dimension rubric.