mutation-testing

Files (4)

• LICENSE
• README.md
• SKILL.md
• hero.png · images/

SKILL.md (first lines)

> **Project:** [honnibal/claude-skills](https://github.com/honnibal/claude-skills). "Claude skills I'm experimenting with. Please review carefully before use."
>
> **License:** [MIT License](https://github.com/honnibal/claude-skills/blob/main/LICENSE). Full license text is in [`LICENSE`](LICENSE) alongside this file.
>
> **Snapshot:** Frozen copy of [`mutation-testing.md.txt`](https://github.com/honnibal/claude-skills/blob/main/mutation-testing.md.txt) as of 2026-05-22. The maintained version lives upstream and may have evolved since this snapshot.

# Mutation Testing

You are now in mutation testing mode. Your job is to assess the strength of the
project's test suite by introducing deliberate bugs (mutations) into the source
code, one at a time, and checking whether any test fails. A mutation that the
tests don't catch reveals a gap in coverage.

## Scope

$ARGUMENTS

- If the user names specific files or directories, restrict mutations to those.
- If no argument is given, look at the project's source layout and use
  `AskUserQuestion` to agree on a starting scope. Don't try to mutate
  everything at once — pick a module or package and work through it.
- Test files themselves are out of scope for mutation. Only mutate production
  code.
- Prioritise code that has meaningful logic (branching, arithmetic, state
  changes) over boilerplate, config, or trivial accessors.

## Pre-flight

Before mutating anything:

1. **Clean working tree.** Run `git status` on the files in scope. If there are
   uncommitted changes to any file you plan to mutate, stop and ask the user to
   commit or stash first. You need a clean baseline so every mutation can be
   reverted cleanly with `git checkout -- <file>`.
2. **Find the test runner.** Look for `pytest.ini`, `pyproject.toml
   [tool.pytest]`, `setup.cfg [tool:pytest]`, `tox.ini`, a `Makefile` test
   target, or a `tests/` directory. Ask the user if you can't determine how to
   run tests. Confirm that the test suite passes on the unmodified code before
   starting — if it doesn't, stop and tell the user.
3. **Map the code.** Read the files in scope. Build a mental model of each
   module's structure and data flow so you can choose mutations that are
   meaningful rather than trivially dead.

## Workflow

Work through the files in scope one at a time. For each file:

1. **Choose mutations.** Read the file and identify 3–8 candidate mutations
   using the catalogue below. Prefer mutations that test interesting behaviour
   — a bug a real developer might introduce — over mechanical operator swaps
   on dead code. For each candidate, write a one-line description of what the
   mutation does and what behaviour it should break.

2. **Apply, test, revert.** For each mutation:
   a. Apply the mutation using `Edit`. Change as little as possible — usually a
      single line.
   b. Run the test suite (or the relevant subset if the suite is large). Use a
      timeout — if the tests hang, that still counts as "caught" (the mutation
      broke something).
   c. Record the result:
      - **Killed** — a test failed. Note which test, and briefly assess how
        helpful the failure message is. Would a developer reading this failure
        immediately understand what went wrong, or would they have to dig?
      - **Survived** — no test failed. This is a gap. Note what behaviour is
        untested.
   d. Revert the mutation: `git checkout -- <file>`. Confirm the file is back
      to its original state before moving on.

3. **Never leave a mutation in place.** After each test run, revert immediately.
   If something goes wrong and you're unsure of the file state, run
   `git diff <file>` to check, and `git checkout -- <file>` to restore.

Use `TaskCreate` to track progress across files when there are more than a
handful.

## Mutation Catalogue

Choose mutations from these categories, ordered roughly from most to least
likely to reveal meaningful test gaps.

### 1. Delete or skip a side effect

Remove or comment out a line that modifies state — an assignment, a method
call that updates an object, an append to a list, a cache write, a database
call. This tests whether the suite verifies that the side effect actually
happened.

```python
# Original
self.count += 1
results.append(item)

# Mutation: delete the line entirely
```

### 2. Negate or invert a condition

Flip a boolean condition: `if x` → `if not x`, `x > 0` → `x <= 0`,
`x and y` → `x or y`, `x is None` → `x is not None`.

```python
# Original
if user.is_active and user.has_permission:

# Mutation
if user.is_active or user.has_permission:
```

### 3. Change a boundary or comparison

Off-by-one and boundary errors: `<` → `<=`, `>=` → `>`, `== 0` → `== 1`,
`range(n)` → `range(n - 1)`.

```python
# Original
if retry_count < max_retries:

# Mutation
if retry_count <= max_retries:

From hugobowne/show-us-your-agent-skills, MIT.

More from hugobowne/show-us-your-agent-skills