test-audit

Files (2)

• SKILL.md
• SWEEP.md

SKILL.md (first lines)

# Test Audit

Status: Active — promoted 2026-10-15 per the `[DOM-14]` promotion tier from
multi-repo test-surface audits (`simplebroker` `208c3a8` whole-test-surface
audit across "2026-10-06-test-audit-remediation-plan" and its four lane
appendices; `weft` "2026-10-07-test-audit-repairs-plan" and
`test_test_audit_policy.py`; originally adapted from `openclaw/openclaw`
`80930af`).

Three modes, one value bar. **Authoring** gates every new or changed test at
write time. **Audit** runs focused passes for tests that re-assert source,
duplicate stronger proof, couple behavior to implementation, or keep test-only
production seams alive. **Sweep** prunes one whole subsystem's test surface.
Optimize for confidence, not deletion count.

This skill owns *judgment* about test value and the repeatable audit/sweep
procedure. `docs/agent-context/runbooks/testing-patterns.md` owns the standing
testing rules and patterns, and the repository owns *mechanics*: read its
`AGENTS.md` and `docs/agent-context/runbooks/testing-patterns.md` first, and
use its canonical test runner, parallelism, fixture, and CI rules wherever this
skill says "run tests". Repository rules win on any conflict.

## Governing References

- `docs/agent-context/runbooks/testing-patterns.md` — standing test rules
  (Rules 1–10) and patterns (Patterns 1–9)
- `docs/agent-context/engineering-principles.md` — §8 (independent review),
  §10 (regression tests), §12 (enumerable contracts), and §14 (governed
  suppression registries)
- `docs/specs/01-development-documentation-operating-model.md` — `[DOM-10]`
  (verification standard) and `[DOM-11]` (independent review workflow)
- `skills/call-agent/SKILL.md` — Tier-1 cross-family independent review for
  deleted coverage and preservation checks

## Which Sections Apply

| Mode | When | Read |
|---|---|---|
| Authoring | adding or changing one or a few tests | Value bar through Retention bar |
| Audit | a focused pass over existing tests | everything in this file |
| Sweep | pruning a whole subsystem's test surface | this file, then `skills/test-audit/SWEEP.md` |

## Value Bar

Tests justify their maintenance cost by protecting behavior, a credible
regression, or an independently meaningful contract. Apply the same
contract-based judgment to new tests and existing ones.

The *owner* of a contract is the production code responsible for it; its
*boundary* is the smallest interface through which that code's behavior is
observable without substituting it. "Owner boundary" throughout this skill
means that interface.

The one question behind every rule here: **if the claimed production behavior
were removed or deliberately broken, would this test still pass?** If yes, it
does not establish protection against that regression; it may still protect
other cases. Assess the gap rather than declaring the whole test worthless. The
[sensitivity check](#exercise-production-behavior) is how to answer it when
reading is not enough.

## Shape of a Good Test

A test makes one coherent behavioral claim. Its name states the behavior; its
setup is only what the claim needs; its assertions check observable results.
Retries, lifecycles, idempotency, and state machines may require several
actions to establish that one claim. Assert semantic properties, not brittle
counts or incidental ordering, unless the count or order is the contract. A
contract includes its failure modes: what the code rejects, and how, is as
much the claim as what it accepts. A reader should be able to tell from the
test alone what production behavior is promised and what change would break
it.

Tests must own and clean up their mutable state, pending work, and resources,
including on failure (`docs/agent-context/runbooks/hardening-plans.md` §13).
They must not depend on test execution order or uncontrolled time. Use explicit
synchronization or module-owned clock seams when timing matters
(`docs/agent-context/runbooks/testing-patterns.md` Pattern 9), so passing runs
and sensitivity checks are reproducible.

## Authoring Gate

Before adding any test, answer four questions; a missing answer means do not
add it yet:

1. What observable behavior, invariant, or independent contract does it protect?
2. What credible regression makes it fail?
3. Why does existing coverage not already catch that failure? Prefer the
   smallest reliable boundary that proves the claim. Another layer needs its
   own distinct risk, such as wiring, boundary straddling, or real dependency
   semantics (`docs/agent-context/runbooks/testing-patterns.md` Rule 9). A
   broader test is not stronger merely because it traverses more code. Prefer
   extending a table-driven case or shared fixture over a near-duplicate test
   when the cases share a contract. Share setup when it improves clarity
   without coupling unrelated cases.
4. Does it need a production seam (export, flag, wrapper, injection hook,
   test-environment branch) that exists only for testing? Prefer the real
   boundary. A seam is justified when it controls a real dependency, such as
   time, randomness, or external failures, without bypassing the production
   behavior under test. Explain that need; do not expose private logic merely
   to assert its implementation.

Make the gate reasoning clear in the test and review context; explain
non-obvious choices explicitly. Use concise comments where they preserve
useful context, such as a historical edge case, protocol requirement, or
required concurrency interleaving. No fixed comment length or reporting
location is required.

Then check the test against every [junk pattern](#junk-patterns); a match fails
the gate unless the [retention bar](#retention-bar) names the contract it
independently guards. If a behavior-preserving refactor would require changing
the assertion or expected result, check for implementation coupling. Updating
an import or setup alone is not evidence of a bad test. Rewrite coupled
assertions unless they protect an independent contract in the retention bar.

Bug regression tests must fail on the pre-fix code for the intended reason and
pass after the owner-boundary repair. Demonstrate it with the
[sensitivity check](#exercise-production-behavior), using the pre-fix behavior
as the control; without that demonstration, its ability to catch the
regression is unverified. Start at the owner boundary; add another layer only

From VanL/agent-theory, CC0-1.0.