audit-merges

Files (1)

• SKILL.md

SKILL.md (first lines)

# Audit Merges

Retrospective, read-only **code-substance audit** of every merge to `main`
across one or more repos over a time window. Where `sweep-prs` audits the PR
_lifecycle_ (green builds, red main, thread hygiene), this skill re-reads the
**merged code itself** — quality, correctness, safety, and whether deferred
work was actually tracked. Built for high-automation repos where most merges
are agent-authored or auto-merged and human review coverage is thin.

## Invocation

- `/audit-merges` — current repo, last 7 days.
- `/audit-merges since 2026-07-22` — explicit window start; `7d` / `30d` /
  natural phrasing ("since last Wednesday") all work.
- `/audit-merges since 2026-07-22 lgtm-hq/py-lintro lgtm-hq/lgtm-ci` —
  explicit repo list. Invoked outside any repo with no repo argument → ask
  which repos, don't guess.

## Ground Rules

- **Read-only while auditing.** No pushes, no thread replies, no issue
  creation, no workflow triggers until findings are confirmed (Remediation).
- **Evidence or it didn't happen.** Every finding carries severity, PR
  link(s), concrete evidence (file:line, thread quotes, run IDs), and a
  recommended action. No speculative nitpicks — if you can't point at it,
  drop it.
- **Judge against the house standards.** Load the repo's matching standards
  skills before reading any diff: `stand-general` always, plus `stand-py` /
  `stand-rust` / `stand-ts` / `stand-odin` by language, `stand-ci` for
  workflow changes, and `analyze-code` for repo-shape risk emphasis.

## Phase 1 — Enumerate and triage

Per repo, gather in bulk with one paginated GraphQL `search` query per repo
(`type: ISSUE`, query `repo:OWNER/NAME is:pr is:merged base:main
merged:>=DATE` — filter to the default branch, resolving it per repo when it
is not `main`) whose nodes fetch — inside a `... on PullRequest` inline
fragment, since `search` returns `SearchResultItem` nodes — the checks
rollup (`statusCheckRollup`) and review threads (`reviewThreads`) in the
same query — prefer server-side
search filters over client-side truncation. Paginate **every** connection
(`search`, `reviewThreads`, thread comments, rollup contexts) via
`hasNextPage`/`endCursor` — nested cursors do not advance with the search
cursor, so when a PR node reports `hasNextPage` on a nested connection,
drain it with per-PR follow-up queries (`repository { pullRequest(...) }`)
before moving on. GitHub search caps at 1,000 results per query, so split
the date window when a repo's merge volume can exceed it.

1. Full merged-PR list for the window — paginate past 100; verify the total
   against `search(type: ISSUE)` `issueCount` (or the REST search API) and
   treat any mismatch or truncated connection as a gathering failure, not a
   smaller audit.
2. Default-branch workflow-run health across the window (`gh run list
--branch <default-branch> --created "START..END" --limit 1000 --json
databaseId,headSha,createdAt,event,conclusion,workflowName,url` — an
   explicit date range plus a limit well above the repo's run volume; never
   the bare default, which returns a recent time-unbounded subset). If the
   returned count equals the limit, the window is truncated: raise the
   limit or split the date range until the count comes back under it, so
   every failure can be mapped to the merge that caused it. Attribute
   `push` runs to merges via each PR's `mergeCommit` (squash commits have
   one parent); classify `schedule`/manually dispatched runs separately —
   they are not caused by a merge.

Then triage:

- **Bot PRs** (Renovate, release bots): light pass — merged green, `main`
  stayed green after. Exception: in repos whose _content_ is the product
  (skills, configs, infra), also read what the bot changed.
- **Substantive PRs** (human- or agent-authored): full deep-read against the
  checklist below.

## Phase 2 — Deep-read checklist

Apply per substantive PR, diff in hand. The diff of record is the **landed**
change: `mergeCommit` against its first parent, not the PR-branch diff —
conflict resolutions, merge-queue rebases, and manual merge edits only show
up in the landed tree.

1. **Broken main, never fixed** — the merge caused a `main` failure and no
   later merge fixed it. Map every red `main` run to a cause and a
   resolution (or its absence).
2. **Quality regressions** — deviations from the loaded `stand-*` skills:
   workarounds, lint/type suppressions, inline shell in workflows, unpinned
   actions, dropped types, copy-paste drift, reduced maintainability.
3. **Merged over red signals** — failing or still-running required checks at
   merge time, or unresolved review threads from any reviewer (CodeRabbit,
   Greptile, CodeQL, humans). Current API state is mutable: judge checks by
   comparing run timestamps for the evaluated commit against `mergedAt`,
   and since GitHub exposes no thread-resolution history, mark
   thread-timing findings the current state cannot prove as unverifiable
   rather than asserting them.
4. **Hollow resolutions** — threads resolved with no fix and no reasoned
   disagreement, just closed to clear the gate (review-thread hygiene canon:
   resolve only via fix or disagreement reply).
5. **Unsafe changes** — secrets, token permissions, supply chain (unpinned
   deps/actions, new registries), script injection (`${{ }}` into `run:`),
   over-broad IAM, destructive CI or infra steps.
6. **Introduced bugs** — critical read of the diff: logic errors, unhandled
   error paths, off-by-ones, concurrency hazards, silently changed behavior.
7. **Untracked deferrals** — "follow-up", "later", "TODO", "out of scope",
   "in a separate PR" in code, PR body, or thread replies with **no
   corresponding issue filed**. Cross-check the repo's issues in **all**
   states (`gh issue list --state all`) — a closed follow-up still proves
   the deferral was tracked; known intentionally-parked work is not a
   finding.
8. **Auditor's judgment** — anything else that should be addressed:
   convention breaks, doc drift, test assertions that assert nothing,
   coverage theater (`analyze-tests` mindset on PRs touching tests).

## Orchestration

Multi-repo audits fan out **one background sub-agent per repo**
(sub-agents-first). Each agent:

- loads that repo's standards skills (Ground Rules above),
- gathers per Phase 1, deep-reads per Phase 2,
- writes structured findings to a scratchpad file: stats header (total
  merges, bot/human split, `main` failure count), one section per finding

From lgtm-hq/ai-skills, MIT.

More from lgtm-hq/ai-skills