qa-swarm

Files (5)

• PANEL.md
• PROOF.md
• SKILL.md
• SPEC.md
• TRIAGE.md

SKILL.md (first lines)

# QA swarm

The swarm takes a branch to a draft PR that the user can merge after one read.
The bar: the user finds nothing that a reviewer, a review bot, or CI would have
found. Fresh reviewers from two model families review the change, you fix what
holds, and a second, narrow round reviews your fixes. Later fixes get final
reads until one comes back clean. Past two rounds or two final reads, an
**arbiter** decides whether the next review runs, so the user decides only
when reviewers contradict each other or the loop stops converging.

For a draft spec, run the [spec pass](SPEC.md) instead of the steps below.

## Rules

- **Fresh eyes.** Every reviewer is a new agent that did not see the build.
  You frame, triage, and fix. A fix you push is new code, and the next round
  reviews it cold.
- **Two model families.** Every panel runs the review lane on the other family
  and the fresh-eyes lane on the author's family, because they miss different
  things. A finding from both carries more weight.
- **Claims are hypotheses.** Check every finding against the code at HEAD
  before you act on it, whoever raised it.
- **Observe.** A green test suite is test evidence. A proof is something you
  ran and watched in the running system, per [`PROOF.md`](PROOF.md).
- **Scope.** The PR's goal is the scope. A real problem outside it is a
  follow-up.
- **Draft.** The PR stays a draft. The user reads the code and marks it ready
  for review.
- **Comments.** Panel findings stay in the ledger. Every GitHub comment goes
  through the `leaving-pr-comments` skill.
- **Fleet.** The `fleet` CLI is on `PATH` or at `~/dev/fleet/fleet`. When it is
  available, it holds the proof, counts the rounds, and runs the arbiter.
  Otherwise, the ledger holds them.

## 1. Frame

Fetch the base branch, then write down:

- **Goal**: the outcome a user gets, in one or two sentences, and the issue it
  closes. Say what a user can do after the change that they cannot do on the
  base, and leave the design out. The goal is the oracle: behavior it does not
  ask for gets extra scrutiny, and so does behavior that no user can observe
  once the PR and the PRs below it merge, such as a parameter for a reader
  that no PR in the stack adds.
- **Approach**: the design the author chose, in a few bullets: the contracts
  it adds or changes and where the logic lives. It is a claim for the panel to
  judge, never a part of the goal.
- **Base** and the diff, `git diff origin/<base>...HEAD`.
- **Danger areas**: the hunks in the danger areas of the `merge-risk` skill.
- **Split**: the PR is one slice, cut before the build by step 1 of
  `full-send` or by the map's stack ticket. A PR with mixed concerns or more
  than about 400 changed production lines goes to the user with a proposed
  split before the first round; do not restack it yourself. In a stack, run
  the swarm on each PR, bottom up.
- **Proofs**: the proofs the PR owes per [`PROOF.md`](PROOF.md), with the
  persisted inputs and boundary formats the changed path touches.

Done when all six are written down.

## 2. Open the draft

1. Run the [design pass](PANEL.md#design-pass) when no PR exists yet, and fix
   what holds before anything else.
2. Run the **lane gate** that the user's agent instructions or the repo's
   agent docs name. When they name none, the repo's full local gate is the
   lane gate: the formatter, lint, typecheck, and tests that its agent docs,
   scripts, and CI configuration name. A step that times out fails. Rerun a
   per-file step, such as a linter, a formatter, or `semgrep`, on the changed
   files only:
   `git diff --name-only --diff-filter=d origin/<base>...HEAD`. Every other
   step that times out, and a per-file step that times out again, records a
   gap. Every later run of the lane gate follows the same rule. Done when
   every step is green or recorded as a gap.
3. Run every owed proof before review, the live run first, and save its
   evidence. For UI changes, the [UI walk](PROOF.md#ui-walk) captures the
   changed journey, and its screenshots are the PR's
   [UI evidence](../pr-description/references/ui-evidence.md). Record a gap for each owed proof you
   cannot run.
4. Push. When no PR exists, open a draft with `gh pr create --draft` and write
   the description with the `pr-description` skill. Once the PR number exists,
   register the owed proofs with `fleet proof require`, or `--none` with the
   reason when the PR owes none, then save the evidence
   and gaps from step 3 under it, as [Record](PROOF.md#record) describes.
5. Keep the proof and the ledger in storage that outlives the worktree and
   stays out of source history. The **ledger** holds every finding, thread,
   and check failure with its source, location, bucket, and evidence. With the
   `fleet` CLI, get the commit proof directory with
   `fleet proof path --repo <owner/repo> --pr <number> --commit <full SHA>`
   and save the verification output and captures there with `fleet proof save`.
   Keep the canonical ledger in its parent PR directory as `ledger.md`. Copy an
   existing Git-metadata ledger there once and keep its findings. Without the
   CLI, keep an existing durable ledger location, or create one directory per
   PR outside the repository.

Done when the PR exists, its head is your HEAD, and the description carries
the observed behavior and any verification gaps.

## 3. Rounds

A round works on one HEAD. Run its steps in order.

1. **Review.** Read [`PANEL.md`](PANEL.md) and pick the round's lenses. With
   the `fleet` CLI, start the panel on HEAD with one `--lens` per lens:
   `fleet proof review --repo <owner/repo> --pr <number> --commit <full SHA> --lens <lens> --lens <lens> ...`,
   plus `--scope fixes --base <previous round's SHA>` for round two and later.
   The CLI reserves the round, so a resumed session cannot reset the count,
   starts one reviewer per lens in the background, and returns. Then wait in
   one call with `fleet proof review --wait`, and call it again while it exits
   2. `fleet proof show` prints each lens with its model and raw answer. Exit 1
   names a lens that failed twice: run the same `review --lens` command, which
   reruns only the failed lenses. The same command resumes an unfinished
   round. The CLI saves the joined answers as the round's findings; keep your
   triage in the ledger. Do not edit the files under review while the panel
   runs. A timeout, an error, or no reply is not approval.
   Without the CLI, write the round number, scope, and HEAD to the ledger,
   dispatch the panel per the Lanes of [`PANEL.md`](PANEL.md), mark the round
   finished after the panel, and count the rounds from the ledger on resume.
   Past round two, `fleet proof review` asks the arbiter first, a cheap model that
   reads the findings so far. It reserves the round unless a later finding

From Silthus/agent-skills, MIT.

More from Silthus/agent-skills