ai-system-testing

Files (1)

• SKILL.md

SKILL.md (first lines)

<objective>
AI-powered features are fundamentally different from deterministic software. The same input can produce different outputs, correctness is subjective, and failure modes include hallucination, harmful content, and subtle quality degradation. This skill covers how to test AI features rigorously despite these challenges.
</objective>

---

## Discovery Questions

Check `.agents/qa-project-context.md` first. If it exists, use it as context and skip questions already answered there.

**AI features under test:**
- What AI-powered features does the application have? (Chat, summarization, classification, code generation, recommendations, search)
- Which LLM provider is used? (OpenAI, Anthropic, Google, open-source models)
- Are prompts hardcoded, template-based, or dynamically constructed?
- Is RAG (retrieval-augmented generation) involved? What is the knowledge source?

**Determinism requirements:**
- Which outputs must be deterministic (classification, structured extraction) vs. creative (chat, summarization)?
- Is temperature fixed or variable? What temperature is used in production?
- Are outputs cached? For how long?
- Is there a fallback when the AI service is unavailable?

**Quality requirements:**
- How is output quality defined? (Accuracy, relevance, completeness, safety, tone)
- Who currently evaluates AI output quality? (Humans, automated metrics, nobody)
- Is there a golden dataset of expected inputs and acceptable outputs?
- What is the tolerance for incorrect or irrelevant responses?

**Safety requirements:**
- Does the AI handle user-generated input? (Prompt injection risk)
- Are there content policy requirements? (No harmful content, no PII generation)
- Is the AI used in regulated domains? (Healthcare, finance, legal)
- What happens when the AI produces harmful or incorrect output?

---

## Core Principles

### 1. Nondeterminism is inherent, not a bug

LLMs are stochastic systems. The same prompt can produce different outputs across runs. Testing must account for this by asserting on properties and boundaries rather than exact strings. If your test does `expect(output).toBe("The answer is 42")`, it will break on the next run when the model responds with "42 is the answer."

### 2. Test properties and boundaries, not exact outputs

Good AI test assertions check: Does the response contain the required information? Is it within the expected length range? Does it include no prohibited content? Does it match the expected format? Bad assertions check: Is the response identical to a saved snapshot?

### 3. Evals are the test suite for AI

Evaluation frameworks (evals) are the AI equivalent of a test suite. They define a set of inputs, run them through the system, and score the outputs against quality criteria. Invest in evals the same way you invest in test infrastructure.

### 4. Safety testing is non-negotiable

AI systems can produce harmful content, leak private data, or be manipulated through adversarial inputs. Safety tests are not optional features -- they are the equivalent of security tests for traditional software.

---

## Tooling

Pick the layer that fits the job. Don't reach for hand-rolled TS unless none of these match.

| Tool | Best for | Notes |
|------|----------|-------|
| **Promptfoo** | Prompt regression, A/B tests, redteam scans | Open-source CLI; YAML-defined eval suites; weekly redteam additions; MCP target support; ~0.121.x current. https://github.com/promptfoo/promptfoo |
| **DeepEval** | Pytest-style agent and LLM evals; agentic metrics | v3.9.9 added first-class TaskCompletion / ToolCorrectness / ArgumentCorrectness — exactly the "tool call validation" patterns this skill teaches. https://github.com/confident-ai/deepeval |
| **Ragas** | RAG-specific eval (faithfulness, context precision/recall, answer relevance) | v0.4.3 added DSPy-based prompt optimization for grounding metrics. https://github.com/explodinggradients/ragas |
| **TruLens** | Production tracing + RAG triad + non-LLM SchemaValidation | v2.8 (April 2026) added programmatic SchemaValidation feedback — cheaper and more deterministic than LLM-as-judge for structured outputs. https://github.com/truera/trulens |
| **Inspect AI** | Government-backed agent eval harness; 200+ pre-built evals | UK AI Security Institute. Date-based releases (`release/2025-11-28`). https://inspect.aisi.org.uk |
| **Garak** | Adversarial prompt scanner / red-team probes | v0.15.0 added GOAT (multi-turn jailbreaks), Agent Breaker (tool-aware), system-prompt-extraction, ModernBERT refusal detector. https://github.com/NVIDIA/garak |
| **PyRIT** | Microsoft AI Red Team's orchestration framework | v0.13.0 (April 2026) — orchestrated multi-turn attacks; complements Garak. https://pypi.org/project/pyrit/ |
| **Braintrust** | Commercial evals + prompt playground | Hosted, paid; SDK works alongside any of the above. |

**Public benchmarks** (HELM, LMSYS Chatbot Arena, Inspect AI's catalog) are for *model selection*, not for app regression testing — they don't know your domain. Use them when picking a base model; use the tools above for everything after.

### Tool-call validation with DeepEval (replaces hand-rolled assertions)

```python
# pip install deepeval
from deepeval import evaluate
from deepeval.metrics import TaskCompletionMetric, ToolCorrectnessMetric, ArgumentCorrectnessMetric
from deepeval.test_case import LLMTestCase, ToolCall

case = LLMTestCase(
    input="What is the weather in Prague?",
    actual_output="It is currently 18°C and partly cloudy.",
    expected_tools=[ToolCall(name="get_weather", arguments={"city": "Prague"})],
    tools_called=[ToolCall(name="get_weather", arguments={"city": "Prague"})],
)

evaluate(
    test_cases=[case],
    metrics=[TaskCompletionMetric(threshold=0.7), ToolCorrectnessMetric(), ArgumentCorrectnessMetric()],
)
```

For prompt regression at the suite level, Promptfoo's YAML config is the lowest-friction entry point:

```yaml
# promptfooconfig.yaml
prompts: [file://prompts/summarize.txt]
providers:
  - openai:gpt-4o
  - anthropic:claude-sonnet-4-6
tests:
  - vars: { document: "..." }
    assert:
      - type: contains-all
        value: ["actionable insight", "key finding"]
      - type: llm-rubric
        value: "Output is 3 sentences or fewer and contains no opinions"
```

For red-team / safety, run `garak --model_type openai --probes encoding.InjectAscii85,goat,system_prompt_extraction` against your deployed prompt before launch.

---

## Prompt Regression Testing

### Version prompts like code

From JaiminVadadoriya/Distributed-Cloud-Storage---Sync-Platform, MIT.

More from JaiminVadadoriya/Distributed-Cloud-Storage---Sync-Platform