dx-audit

Files (44)

• SKILL.md
• evals.json · evals/
• evaluation-scenarios.md · evals/
• standards-map.md · references/
• _sections.md · rules/
• _template.md · rules/
• api-argument-order.md · rules/
• api-async-consistency.md · rules/
• api-naming-consistency.md · rules/
• api-no-hidden-side-effects.md · rules/
• api-predictable-return-shape.md · rules/
• api-sensible-defaults.md · rules/
• api-stable-contract.md · rules/
• cli-agent-input-hardening.md · rules/
• cli-delta-polling.md · rules/
• cli-exit-codes.md · rules/
• cli-flag-naming.md · rules/
• cli-help-and-version.md · rules/
• cli-idempotent-resume.md · rules/
• cli-order-independent-flags.md · rules/
• cli-pipes-tty-and-json.md · rules/
• cli-responsive-and-progress.md · rules/
• cli-safe-mutations.md · rules/
• cli-schema-introspection.md · rules/
• cli-structured-io.md · rules/
• cli-suggest-corrections.md · rules/
• config-optional-with-defaults.md · rules/
• config-validate-and-discover.md · rules/
• config-xdg-and-precedence.md · rules/
• err-fail-fast-validation.md · rules/
• err-name-cause-and-value.md · rules/
• err-no-raw-stack-as-message.md · rules/
• err-stable-error-codes.md · rules/
• err-suggest-the-fix.md · rules/
• onboard-exports-resolve-typed.md · rules/
• onboard-minimal-install.md · rules/
• onboard-no-required-env.md · rules/
• onboard-tree-shakeable.md · rules/
• onboard-zero-config-quickstart.md · rules/
• types-discriminated-unions.md · rules/
• types-helpful-generics.md · rules/
• types-no-leaked-any.md · rules/
• types-prefer-inference.md · rules/
• types-public-jsdoc.md · rules/

SKILL.md (first lines)

# DX Audit

Audit or improve what developers import, run, configure, or read when something fails.

- **IS:** a bounded review of public APIs, developer-facing errors, CLI commands, exported types, install and first-run behavior, and config, with fixes only when asked.
- **IS NOT:** a repo-wide quality sweep (`tidy`), end-user UI (`ui-design` Audit mode), agent trust review (`ax-audit`), public site or docs agent scores (`agent-ready`), docs prose or a README (`ghostwriter`), repository architecture (`codebase-architecture`), or building a new CLI (`scaffold-cli`).

## Modes

Pick the narrowest mode the request supports, and write a one-line scope receipt before reading code:

| Mode | When | Output |
|------|------|--------|
| Targeted (default) | a named or changed public surface | findings report, read-only |
| Fix | the user says fix, improve, simplify, or implement | localized edits inside the receipt, then verification |
| Exhaustive | the user explicitly asks for the whole package or every public surface | every material finding, partitioned by surface |

```text
Scope: <mode>; surfaces: <commands/exports/config>; prefixes: <err-, cli->; excludes: <UI, docs, architecture, private internals>
```

"DX", "gold standard", and "review holistically" do not by themselves widen a targeted audit into exhaustive. When several skills are invoked together, this one owns only the surfaces above.

## Audit progress

```text
DX audit progress:
- [ ] 1. Lock the public surface and write the scope receipt
- [ ] 2. Gather local evidence and run the safe probes
- [ ] 3. Select prefixes, then open candidate rules
- [ ] 4. Rank root causes
- [ ] 5. Report, or fix when asked
- [ ] 6. Verify on the same scope
```

### 1. Lock the public surface

Start from `git diff` against the normal base and keep only changed files reachable from a public entry point: `package.json` `exports` or `bin`, a command registry, an exported type, a documented config loader, or an observed error path. With no useful diff, use the command, export, error, or config the user named. A private helper enters scope only through a public caller.

### 2. Gather evidence, then stop

1. Local instructions, the manifest, and the diff or named entry point.
2. Direct public dependencies and the nearest tests that pin behavior.
3. Safe probes against the local build:
   - CLI: `--help`, `--version`, one success path, one invalid-input path, and the same command with stdout piped (`| cat`) to see non-TTY behavior. Never trigger a real mutation to test DX; use `--dry-run` where it exists.
   - Package: `pnpm dlx publint` and `pnpm dlx @arethetypeswrong/cli --pack .` after a build. These inspect packaging, but packing may invoke lifecycle scripts. Inspect those scripts first or use a disposable checkout before calling `--pack`.
4. The prior release contract, only when the diff changes a public export, signature, or return shape.

Stop when the behavior is proven, disproven, private, or out of scope. External research is for an explicit comparison request or a named uncertainty local evidence cannot resolve; `references/standards-map.md` carries the standards this skill already leans on.

### 3. Dispatch rules candidate-first

Read `rules/_sections.md`, then select prefixes by surface:

| Priority | Prefix | Category | Default impact | Rules |
|----------|--------|----------|----------------|-------|
| 1 | `api-` | Public API and SDK | CRITICAL | 7 |
| 2 | `err-` | Developer-facing errors | CRITICAL | 5 |
| 3 | `cli-` | CLI UX for humans and agents | HIGH | 13 |
| 4 | `types-` | Exported type ergonomics | HIGH | 5 |
| 5 | `onboard-` | Install and first run | HIGH | 5 |
| 6 | `config-` | Config ergonomics | MEDIUM | 3 |

| Surface | Prefixes |
|---------|----------|
| Public API entry point | `api-`, `types-`, reached `err-` paths |
| CLI command | `cli-`, reached `err-` paths |
| Exported declarations | `types-`, plus `api-` when behavior changed |
| Install, `package.json`, first run | `onboard-` |
| Config loader | `config-`, reached `err-` paths |

Applicability outranks priority: a CLI-only audit never loads `api-` because API rules rank higher.

Targeted mode: list the filenames for the selected prefixes (the names are the checklist), look for concrete evidence, then open only the rule files a finding needs. Exhaustive mode: read every rule in the selected prefixes.

Capability gates, applied inside a selected prefix:

- Structured JSON input, schema introspection, and compact polling snapshots apply when automation or agent use is promised, requested, or already supported.
- Dry-run and confirmation apply to destructive, expensive, or hard-to-reverse mutations.
- Progress and resume apply to operations that can block, outlive one command, or be retried after ambiguous output.
- `stdin` applies when the command semantically accepts file or stream data.
- Stable-contract comparison applies only when a public contract changed.

### 4. Rank root causes, not instances

- Order CRITICAL, HIGH, MEDIUM by each cited rule's frontmatter `impact`, copied exactly. Impact is the rule's declared consequence, not a confidence score: exit 0 on failure in one subcommand is still HIGH, because the CI gate it defeats is the same. Open the rule file before citing it, since an id cited from memory drifts from the file, and drop the citation if no such file exists.
- Merge repeated instances of one root cause into one finding with up to three representative locations. Missing JSDoc, error codes, or `--json` across a surface is one finding, not one per symbol.
- A missing feature with no current consumer path is not a defect.
- Targeted mode: every CRITICAL finding, then the highest-value remainder up to five total; summarize the rest by category.

### 5. Report or fix

Audit requests are read-only. A fix request authorizes localized changes inside the receipt, not a redesign of adjacent docs, UI, or architecture.

```markdown
## DX Audit

Scope: `tool status` CLI; `err-`, `cli-`; 4 files inspected.

### Findings
- [HIGH] `cli-idempotent-resume` at `src/start.ts:42`: retrying the same target creates a second job.
  Fix: return the existing job id and state unless the caller passes `--fresh`.

### Deferred
- 2 lower-impact config candidates were outside the locked CLI scope.
```

List only files with findings; a clean surface gets one pass line naming the surfaces and rule files checked. In fix mode, replace `Deferred` with `Changed` and `Verification` (the exact commands or probes that passed). Length follows findings, not the template.

### 6. Verify on the same scope

Re-open every touched or cited location, rerun the same probes and focused project checks, and reapply the same candidate rules. Match the evidence to the claim: a clean build does not prove CLI behavior, a runtime probe does not prove exported types, and only `attw` or a consumer-style `tsc` import proves a types resolution.

## Reference files

| File | Read when |
|------|-----------|
| `rules/_sections.md` | Every audit, for prefix applicability and priority |
| `rules/<prefix>-*.md` | A candidate has evidence, or exhaustive mode |

From mblode/agent-skills, MIT.

More from mblode/agent-skills