“LLM evaluation”

Evaluate Anvia agents and retrieval — deterministic metrics, semantic similarity, LLM judges, RAG quality, and CLI eval runs.

Use after completing any non-trivial task

Use when comparing basic, enhanced, GraphRAG, or agentic RAG designs, evaluating domain-specific RAG quality, tuning retrieval components, or deciding whether…

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

Run evals on the requirement-to-story map — deterministic traceability gate plus an LLM-judge review of every requirement→story→eval link — and write a dated…

Standard metrics, acceptance thresholds, and report format for evaluating ML models.

Scores binary predictions (Brier, log loss, ECE, MCE, AUROC, accuracy, reliability bins, per-cohort) or saved categorical outputs joined to retained targets…

Evaluate real Pi and Codex sessions against centralized criteria for personal and repo-local skills, identify whether skills helped, were neutral, or made…

Evaluate AI and agent features for grounded quality, safety, tool authority, and fallback.

Evaluation rules for five-class ESI prediction.

Evaluate third-party vendors for engineering fit.

Benchmarks LLM inference and drives GPU kernel optimization with Magpie.

lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).

Design honest evaluation across language/strata with sample-size warnings and exact metric.

Run and interpret the project's quality matrix after correctness and route gates are closed.

Merge MMI and DDD evaluation results — and the UX evaluation when it exists — into a unified improvement plan.

MLflow 3 GenAI agent evaluation.

Independently evaluate a native proposal with explained quality scores, coverage-limited novelty and a previously frozen blind baseline.

An ExpectationEvaluation records one open or closed window for an Expectation.

统一机会评估主链路、八维评分与 Gate；默认输出原生文档，不调用生图。

Use before research or engineering-research decisions: deciding whether to continue a direction, run an experiment, change route, interpret results, write a…

Evaluate the UX of an existing UI from its inventory — Nielsen's heuristics, WCAG 2.2, task efficiency, consistency, navigation — as a UX index with…

Evaluates a third-party repo, skill or plugin before importing it.

Use Hugging Face Evaluate to load metrics, comparisons, and measurements; compute and combine results; run evaluator pipelines; create custom modules…