“LLM evaluation”

Evaluate Anvia agents and retrieval — deterministic metrics, semantic similarity, LLM judges, RAG quality, and CLI eval runs.

Use after completing any non-trivial task

Use when comparing basic, enhanced, GraphRAG, or agentic RAG designs, evaluating domain-specific RAG quality, tuning retrieval components, or deciding whether…

Scores binary predictions (Brier, log loss, ECE, MCE, AUROC, accuracy, reliability bins, per-cohort) or saved categorical outputs joined to retained targets…

Evaluate real Pi and Codex sessions against centralized criteria for personal and repo-local skills, identify whether skills helped, were neutral, or made…

Use Hugging Face Evaluate to load metrics, comparisons, and measurements; compute and combine results; run evaluator pipelines; create custom modules…

Run model generation, pairwise Gemini scoring, and eval-db persistence.

Build an automated evaluation pipeline that tests a Python LLM application end-to-end with pixie test — real code paths, real LLM calls, instrumented external…

Build and maintain an LLM-curated knowledge base from papers, articles, transcripts, notes and project findings.

Self-rate a completed non-trivial task on accuracy, completeness, clarity, actionability, and conciseness, with evidence per axis and a 1-5 scorecard.

Build, validate, and package reusable evaluation datasets for agent and LLM systems from prompts, traces, tool-call logs, metadata, and expected outcomes.

Eval-driven skill tuning.

Use after completing any non-trivial task.

Evaluates a named skill by running its artifact checks and optionally invoking the skill against fixture inputs.

Evaluate a Zira vision model — see how accurately it detects each class (precision, recall, the images it gets wrong, drawn on the images) by running it on a…

Master context engineering for AI agent systems.

Add and manage evaluation results in Hugging Face model cards.

This skill should be used when the user asks to "create a hook", "add a PreToolUse/PostToolUse/Stop hook", "validate tool use", "implement prompt-based…

Isolated DSPy optimizer companion (GEPA and MIPROv2) for governed self-evolution.

Design and build use cases, POCs and production systems with TypeSafe's Jev decision model (System One): choose a design mode and a use-case archetype, build…

LLM-as-a-judge evaluations over OpenAI-compatible APIs (including NVIDIA Nemotron & LM Studio).

Use when the user wants to migrate code that calls OpenAI, Gemini/Google AI, or the Anthropic API to Amazon Bedrock — a pure model/SDK rewrite.

Score prompts for prompt injection with TypeSafe's Jev before they reach an LLM: a reference gate function a backend can call (allow / review / block with a…

复现结果评估 — 从论文分析中提取声称指标，与实验实际结果对比，用 LLM 给出复现评估（full/partial/failed）。