add-regression-test

Files (2)

• SKILL.md
• reproduction-modes.md · references/

SKILL.md (first lines)

> **Tool mapping (OpenCode):** This skill uses:
> - `question` tool for user prompts (not `AskUserQuestion`)
> - `task` tool for subagents (not `Agent` or `Task` subagent syntax)
> - `skill({ name: "..." })` to load other skills (not `Skill(skill="...", args="...")`)
> - `codegraph_explore`, `codegraph_search`, `codegraph_context` for codebase exploration (preferred)
> - `todowrite` for phase/progress tracking
> - `grep`, `glob`, `read`, `bash` as fallback when `.codegraph/` is not initialized
>
> **Context rule:** This skill supersedes any prior skill instructions. Follow ONLY these instructions now. Read the user's goal and any mode/include/skip from the conversation context.

Read `mode=`, `include=`, and `skip=` from the conversation context. Default to the mode specified in this skill if not provided.

# Add Regression Test

A regression test is only valuable if it would have failed before the fix and passes after. Anything else is a confidence test pretending to be a regression test. The whole skill is built around proving both halves.

---

## Phase 1 — Identify the Fix

Use `todowrite` to mark this phase as in_progress. Mark it completed when done.

Determine what changed:

- If the fix is staged or committed but not pushed: `git diff <base>..HEAD -- <files>` and `git log <base>..HEAD --oneline`.
- If the fix is in the working tree: `git diff` (unstaged) plus `git diff --cached` (staged).
- If the user names a commit: `git show <sha>`.

Read the changed code. Identify:
- The function / route / component the fix lives in
- The input shape that triggers the bug (from the diff context, the bug report, or the user)
- The expected externally-observable outcome (return value, DB row, HTTP response, emitted event, rendered DOM)

If the fix is purely structural (renames, refactor, reformatting) with no behavior delta — stop. Tell the user there is nothing to regression-test.

**Exit:** the bug's input shape and expected outcome are written down.

---

## Phase 2 — Detect Test Harness

Use `todowrite` to mark this phase as in_progress. Mark it completed when done.

Apply the harness-detection rules from `write-tests` (or use that skill's references file directly): config files, lockfile, scripts. If no harness exists, hand off to `skill({ name: "write-tests" })` Phase 3 to install one — do not introduce a runner here.

**Exit:** the runner command and the test-file convention are known.

---

## Phase 3 — Reproduce the Failure

Use `todowrite` to mark this phase as in_progress. Mark it completed when done.

This is the load-bearing step. **MANDATORY — READ [`references/reproduction-modes.md`](references/reproduction-modes.md)** before choosing a mode.

Pick one mode:

- **A. Fix is unstaged in working tree** → write the test, `git stash` only the fix files, run the test (must fail), `git stash pop`, run the test (must pass).
- **B. Fix is staged but not committed** → write the test, `git stash --keep-index` to set aside the test, then `git stash pop`-trick won't work cleanly; instead use mode C.
- **C. Fix is committed locally** → write the test on top, then `git revert --no-commit <fix-sha>` in a temporary worktree (or via `git stash` of the fix files), confirm the test fails, restore.
- **D. Fix is squashed into a larger commit and can't be cleanly isolated** → revert just the offending hunks via `git checkout <pre-fix-sha> -- <file>` in a worktree, confirm test fails, restore.

In every mode, the gate is the same: **the test must fail without the fix code present.** A test that passes both before and after is broken — go back and tighten the assertion.

**Exit:** the test failed against the pre-fix code with output that names the actual bug behavior (not "expected X, got undefined" — the failure message should resemble the original bug report).

---

## Phase 4 — Verify the Pass

Use `todowrite` to mark this phase as in_progress. Mark it completed when done.

Restore the fix, run the test. It must pass.

If it doesn't pass: either the assertion is wrong (too strict, asserting unrelated behavior) or the fix is incomplete. Surface to the user — do not loosen the assertion to make it green.

**Exit:** test passes with the fix in place.

---

## Phase 5 — Wire It In

Use `todowrite` to mark this phase as in_progress. Mark it completed when done.

- Place the test in the project's convention (colocated `<name>.test.ts`, `tests/`, `__tests__/`, etc. — match the nearest sibling).
- Name it after the bug's *symptom*, not the fix: `it("returns 404 when slug contains uppercase letters")`, not `it("normalizes slug case")`. The next reader needs to recognize the bug from the test name.
- If a bug tracker reference exists (issue #, ticket id), include it in a one-line comment above the test.

Run the full suite — the new test plus its neighbors must all pass.

**Exit:** test is in place; full suite is green.

---

## Phase 6 — Report

Use `todowrite` to mark this phase as in_progress. Mark it completed when done.

```
Regression test added: <path>:<line>
  Pinned: <one-line bug description>
  Verified failed against: <pre-fix sha or "stashed working tree">
  Verified passes against: HEAD
  Run: <project test command>
```

---

## NEVER

- **NEVER skip the failure verification.**
  **Instead:** stash, revert, or use a temporary worktree to confirm the test fails without the fix, every time.
  **Why:** a test written against a fix that's already in the file passes by construction. Without the failing-state proof, you have a confidence test, not a regression test — it cannot detect the bug coming back.

- **NEVER assert on internal call sequences or implementation details.**
  **Instead:** assert on the externally-observable outcome — return value, DB row state, HTTP response, rendered DOM, emitted event.
  **Why:** implementation-coupled tests pass for the wrong reasons and break on every refactor. The bug is observable; the test should be too.

- **NEVER loosen an assertion to make a failing test pass.**

From Acauhi99/opencode-agent-system, MIT.

More from Acauhi99/opencode-agent-system