tdd

Files (8)

• CREDITS.md
• NOTICE
• SKILL.md
• openai.yaml · agents/
• characterization.md
• mocking.md
• mutate.sh · scripts/
• tests.md

SKILL.md (first lines)

# Test-Driven Development

TDD is the red → green loop. This skill is the reference that makes that loop produce tests worth keeping: what a good test is, where tests go, the anti-patterns, and the rules of the loop. Every section applies on every cycle: consult them before and during the loop, not after.

When exploring the codebase, read the glossary (`docs/agents/domain.md` gives the layout; by default `GLOSSARY.md` at the root, if it exists) so test names and every identifier the change introduces match the project's domain language, and respect ADRs in the area you're touching.

For a bug whose cause is not yet known, load and follow the `diagnosing-bugs` skill through the host's skill mechanism first: it finds the cause and brings the repro back here as the failing test.

## What a good test is

Tests verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't. A good test reads like a specification: "user can checkout with valid cart" tells you exactly what capability exists, and it survives refactors because it doesn't care about internal structure.

Before you write a test, name the **break** it catches: the production change that turns it red. If you cannot name one, the test observes no behavior yet; find the behavior first.

Read [tests.md](tests.md) for good and bad test examples before writing the first test, and read [mocking.md](mocking.md) before adding any mock or test double. Before any behaviour-preserving change, read [characterization.md](characterization.md): it pins code that has no tests, and checks that existing tests cover each change.

## Seams: where tests go

A **seam** is where a module's interface lives (the `codebase-design` term): tests cross it the way callers do and observe behavior without reaching inside. Tests live at seams, never against internals.

**Test only at pre-agreed seams.** Before writing any test, write down the seams under test and confirm them with the user. Seams named in the spec or issue you are working from are already confirmed. No test is written at an unconfirmed seam. You can't test everything, so agreeing the seams up front is how testing effort lands on the critical paths and complex logic instead of every edge case. The seams bound the code as well as the tests: when the change needs code behind a seam not on the list (a TUI line when only the Session API is confirmed), confirm that seam before writing the code, so every line the run adds has its red case at a confirmed seam.

Ask: "What's the public interface, and which seams should we test?"

When the shape of that interface is itself in question (how deep the module is, where the seam belongs, what the interface should expose), load and follow the `codebase-design` skill through the host's skill mechanism for the vocabulary. It is the shared source of the module, interface, depth, seam, adapter, leverage and locality terms, and it is a reference to consult, not a session to run.

## Anti-patterns

- **Implementation-coupled**: mocks internal collaborators, tests private methods, or verifies through a side channel (querying the database instead of using the interface). The tell: the test breaks when you refactor but behavior hasn't changed.
- **Tautological**: the assertion recomputes the expected value the way the code does (`expect(add(a, b)).toBe(a + b)`, a snapshot derived by hand the same way, a constant asserted equal to itself), so it passes by construction and can never disagree with the code. Expected values must come from an independent source of truth: a known-good literal, a worked example, the spec.
- **Change detector**: the test goes red only on a deliberate decision (a constant's value, the exact wording of a message, the source text of a script or doc) and stays green through real bugs. Assert the behavior that depends on the decision: "a failing call is tried 5 times, then gives up" in place of `expect(MAX_RETRIES).toBe(5)`.
- **Horizontal slicing**: writing all tests first, then all implementation. Bulk tests verify _imagined_ behavior: you test the _shape_ of things rather than user-facing behavior, the tests go insensitive to real changes, and you commit to test structure before understanding the implementation. Work in **vertical slices** instead: one test → one implementation → repeat, each test a **tracer bullet** that responds to what the last cycle taught you.

## Rules of the loop

- **Red before green.** Write the test first and run it. It is red when the assertion fails for the break you named. An error, a typo, a failed import, a build failure, or a panic is not red yet: fix it and run again. When the test calls a name that does not exist yet, fix it with a stub: add the name so it compiles and returns the zero value (an empty field, a function returning nil or an empty list), then run again until the assertion fails. A test that passes at once pins behavior that already exists. If the code predates this run, rewrite the test to name a break that is not there yet, unless the spec asks for this case over existing behavior: then it is a characterization test, and it counts once you break the code it covers ([characterization.md](characterization.md), "When you have enough") and watch it go red. If this run wrote the code ahead of the test, the test counts only once you break that code (the mutation check below) and watch it go red: do that before counting the cycle, then undo the break. Then write only enough code to pass it: every branch you add is one the failing case runs. A behavior the case does not reach (another input form, another case of the same rule, a later slice's rule) waits for its own red test, even when the natural implementation would cover it in the same edit. Don't anticipate future tests or add speculative features.
- **Green is the affected seam.** During a cycle, run the new test and the tests for the affected seam, plus lint on changed files and the narrowest supported typecheck. Use the full command when the runner cannot select that scope or shared configuration makes the effect project-wide. Before a completed behavior is committed or handed off, run the repository's full required gate, including its full suite. A cycle is not a commit boundary: several cycles can complete one behavior. Reuse a passing check only while its relevant files, dependencies, configuration, and environment are unchanged; a later phase or commit alone does not invalidate it. Repository rules and hooks still apply. Report every failure by name, including failures you did not cause.
- **One slice at a time.** One seam, one case, one minimal implementation per cycle. A case is one input and the assertions on it, whatever file holds it: in a table-driven test or a script of many cases, add one case per cycle and watch that case go red, since a runner that stops at its first failure shows only the first new case red and hides the rest.
- **Mutate before you finish.** Break every file this run changed in small, realistic ways, one at a time: a wrong constant or argument, the wrong branch, a missing side effect, an empty return, a missing check for empty, zero, null, or bad input. Run the tests for that seam, then undo the change. Each mutation must turn at least one test red as the red rule above defines it, an assertion failing for that break: a mutation that only panics or fails to compile proves nothing, so pick another. Change the logic and leave every import and variable in use (`false && cond` in place of a deleted `cond`, `x += 0` in place of `x = x`): a compiler that rejects unused names, as Go's does, or a vet step that rejects self-assignment, reports the mutation `broken`. One that stays green marks behavior no test protects: add the test that turns it red, or, when no input reaches the mutated code or the mutation changes no observable result (an equivalent mutation), delete that code, since the red rule admits no branch a failing case does not run. An equivalent mutation of code that has to stay (a `==` that no input tells apart from `>=`, a value the output never shows) is a **survivor**: write down the input you tried to tell the two apart and why none can, and list it by name in your report. Rewriting the code so the mutation no longer applies is a behaviour change: it needs its own red case, and the rewritten code is mutated again. The check is done when every mutation is red or is a listed survivor. The mutation runner requires a POSIX shell, Perl, and standard Unix utilities (`cat`, `cp`, `date`, `head`, `mktemp`, and `rm`). Run them with `sh <this skill's directory>/scripts/mutate.sh [-b BUILD] [-t SECONDS] TEST MUTATION...` ([`scripts/mutate.sh`](scripts/mutate.sh), bundled with this skill). Each MUTATION is a file: the target path on line 1, then the exact text to find, a line holding only `====`, then the replacement. A missing separator is rejected as `missing`; bytes outside the replacement, including the final newline, are preserved. Write them into a `mktemp -d` directory, one heredoc per file. Call it through `sh`, because an installed copy can lose its execute bit. A batch runs BUILD and TEST once per mutation, so a batch of dozens can outlast your shell tool's time limit: run a large batch in the background and read its output when it exits; until then leave the working tree alone, since the batch rewrites each target in place and restores it from a copy, so an edit made meanwhile is lost or skews the results. A mutation that removes a loop or recursion guard can run TEST forever. An explicit `-t SECONDS` also bounds the unchanged baseline; `-t 0` disables TEST timeouts. Without an explicit limit, mutation tests get five times the unchanged run plus a minute. A timeout kills TEST and processes still in its process group. Test commands that detach descendants into another group or session must clean them up themselves. Count a `timeout` on such a guard as red, since never ending is the break it names. It applies each one alone, restores the file, and reports it `test-failed`, `GREEN`, `timeout` (TEST ran past the limit), `broken` (the `-b` build step failed) or `missing` (its text was not found). A failed backup stops before that mutation. A failed restore exits 2 and retains the backup at the path printed on stderr; recover it before continuing. Failed TEST output is printed on stderr with the mutation name. Inspect that output before counting `test-failed` as red: identify the assertion and confirm it failed for the intended break. Missing diagnostics require rerunning with a test command that exposes them. A runtime error or failed import is not red. Exit 0 means every TEST failed; it does not certify the failure causes. It first runs BUILD and TEST on the unchanged code and stops with exit 2, printing the failing command's output, when either fails, since every mutation would then read red: fix TEST until it passes there (TEST runs through `sh`, so quote a `|` in a `-run A|B` pattern).
- **Refactoring is not part of the loop.** It comes after the slice is green, usually on what review found (the `code-review` skill), with the tests left unchanged apart from a mechanical rename or move. A refactor runs no red-green cycle: the seam's existing tests are its characterization tests. Each step runs the affected checks and characterization.md's break check; the completed refactor runs the full required gate before commit or handoff.

From svyatov/supermatt, MIT.

More from svyatov/supermatt