spec-agent-audit

Files (1)

• SKILL.md

SKILL.md (first lines)

# Spec agent audit

Readonly meta-audit of agent definition files in **`agents/`**. The agent files are the artifact under review; `.specs/`, `ENGINEERING-RECIPES.md`, and the orchestrator gate matrix are the governing truth. Findings only — never edit agent files.

## Epistemic stance

- Audit against the on-disk agent files and the governing specs/recipes — not chat summaries or agent self-description in chat.
- If parent context contradicts the on-disk agent file or the orchestrator matrix → **disk wins**; ignore chat.
- You are epistemically isolated from the agents you are auditing — do not trust an agent's own claim that it "follows the pattern"; verify the file content.

## Scope

All files in **`agents/`** (currently 20). Per-agent or batch. Pass `Audit round: 1|2` on every delegate.

### Anti-loop

- Max **2** audit rounds per agent file per audit pass.
- Round 2 = **delta only** (open Blocking/Advisory from Round 1 + regressions from fixes).
- After Round 2 with open Blocking → escalate to **user** (fix, waive, or reject). No Round 3.

## Checklist A — Memory & self-learning patterns

Governed by skill **`spec-agent-memory`**. For each agent file, verify:

1. The `## Skills` line references **`spec-agent-memory`**.
2. A memory path is declared as `.agents/memory/<agent-name>/` and the directory name matches the `id:` field in frontmatter.
3. A read-at-start + update-at-end flow is present (read `_project/MEMORY.md`, `_project/specs-index.md`, role `MEMORY.md` at start; update role `MEMORY.md` and `specs-index.md` at end / gate boundaries).
4. For **`eng-orchestrator`** only: the `## Active plan` block is required and `specs-index.md` updates are explicit.
5. The ≤200-line `MEMORY.md` cap is honored in guidance; topic files (`<topic>.md`) are used for deep dives and linked from `MEMORY.md`.
6. No instruction to store secrets, tokens, or credentials in memory files.

## Checklist B — Debugging patterns

Governed by agent **`debugger`** and the spec-gap handoff rule. For each agent file, verify:

1. **`debugger` agent specifically:**
   - BUG file template present (`.specs/maintenance/BUG-NNN-*.md`) with Observed / Expected / Reproduction / Root cause / Fix scope / Spec gap / Regression tests sections.
   - Spec-gap vs implementation-defect classification is present.
   - Fix-in-place is gated on orchestrator/hotfix scope; otherwise hand off to the matching implementer.
   - Durable diagnosis is always written (BUG file or checkpoint/CHANGELOG backfill).
2. **All other implementers, reviewers, and verifier:**
   - Spec-gap handoff rule present: do not invent product behavior; HANDOFF to orchestrator → `requirements-analyst` / `adr-recorder` / user when behavior is undefined.
   - Evidence capture present: git SHA **or** explicit uncommitted path list, plus test output path when tests are run.
   - No instruction to delete or weaken tests to clear failures.

## Checklist C — Description alignment

Compare the frontmatter `description:` against the body. Verify:

1. Body role boundaries match the description (e.g., `verifier` description says "epistemically isolated — does not trust implementer/reviewer narratives or fix code"; the body must enforce both halves).
2. `readonly: true` appears in frontmatter **iff** the body forbids edits (expected: `verifier`, `code-reviewer`, `spec-guardian`, `challenger`). A readonly agent whose body says "fix code" is a Blocking contradiction.
3. Skills listed in `## Skills` match the described role — no missing required skills, no extra skills that contradict the role boundary.
4. Gate rules in the agent file do not contradict the orchestrator gate matrix (`eng-orchestrator.md` "Gate matrix (recipe × tier)" and the shared production gates).
5. Approval authority in the agent file matches `eng-orchestrator.md` "Approval authority" — only the user sets `APPROVED`; agents must not self-approve.

## Checklist D — Contradicting instructions

1. **Internal contradictions** within a single agent file:
   - `readonly: true` vs any "edit / fix / write code" instruction.
   - Any "self-approve", "approve REQ/ARCH", or "waive Critical" language from a non-user role.
   - Conflicting skill references (e.g., naming two skills that claim opposing authority over the same artifact).
   - Duplicate roster or list entries within the file.
2. **Cross-agent contradictions** across `agents/`:
   - Approval authority, anti-loop round caps (max 2, R2 delta-only), and "disk wins" rule consistent with `eng-orchestrator.md`.
   - Gate pass conditions and delegate targets consistent with the orchestrator gate matrix.
   - No two agents claim ownership of the same gate/role boundary (e.g., `verifier` vs `spec-guardian` boundaries must match `spec-guardian.md` "vs `verifier`" table).
3. **Skill-authority conflicts:**
   - `spec-pipeline` is an alias stub for `spec-recipes`; recipes §0 wins; no other agent file reverses this.
   - Any agent file that overrides a skill's stated authority without citing the orchestrator or recipes as the higher source.

## Severity

| Severity | Examples |
|----------|----------|
| **Blocking** | Missing memory path; readonly agent that edits; self-approve language; gate rule contradicting orchestrator matrix; missing `spec-agent-memory` reference |
| **Advisory** | Missing read-at-end update step; weak debugging evidence step; skill reference mismatch; spec-gap handoff rule absent for an implementer |
| **Nit** | Wording, duplicate non-roster entries, formatting drift from the spec-* skill shape |

Do not inflate Nits into Blocking. Do not "approve" agent files; report findings and `audit_passed: yes|no`.

## Allowed inputs

- `agents/*.md` paths (the artifacts under review)
- `skills/spec-agent-memory/SKILL.md`, `skills/spec-handoff/SKILL.md`, `skills/spec-recipes/**` (governing patterns)
- `agents/eng-orchestrator.md` (gate matrix and approval authority — the authoritative cross-agent source)
- `Audit round: 1|2`, prior finding IDs (R2), recipe/tier/phase if relevant

## Forbidden as primary truth

- Chat / conversation summaries about what an agent "is supposed to do"
- Agent self-description in chat; trust only the on-disk file
- If parent context contradicts the on-disk agent file → **disk wins**

## Report format

```markdown
## Agent audit — [agent-id | batch]
**Round:** 1 | 2
**Files reviewed:** agents/*.md

### Findings
1. **ID:** A1 | **Severity:** Blocking|Advisory|Nit | **Where:** agents/<file>.md:<line>
   - Issue / what's missing / contradiction / remediation hint

### Per-agent summary
- <agent-id>: pass | findings [IDs]

### Gate
- **audit_passed:** yes | no
- **Blocking open:** [IDs]
- **Recommend:** human-fix | proceed
```

## Rules

- Readonly: do not edit agent files, specs, or skills.
- Prefer **file paths + line numbers** over narrative in findings.
- Cite the governing source for each contradiction (orchestrator section, skill, or recipes §).
- Set HANDOFF **Blockers** if any Blocking finding is open.

From sushilti80/specforge-engineering-team, MIT.

More from sushilti80/specforge-engineering-team