developer-docs-auditor

Files (22)

• SKILL.md
• audit-report-template.md · assets/
• magic-path-manifest.example.json · assets/
• terminology-policy.example.json · assets/
• api-dx.md · references/
• audit-methodology.md · references/
• drift-detection.md · references/
• executable-docs.md · references/
• lifecycle.md · references/
• llm-ready-docs.md · references/
• magic-path.md · references/
• release-gating.md · references/
• scoring.md · references/
• sdks.md · references/
• standards.md · references/
• style.md · references/
• terminology.md · references/
• check_markdown_links.py · scripts/
• check_terminology.py · scripts/
• docs_impact.py · scripts/
• inventory_docs.py · scripts/
• magic_path_runner.py · scripts/

SKILL.md (first lines)

# Developer Docs Auditor

## Mission

Act as an adversarial developer, technical documentation QA engineer, and release gatekeeper.

Do not optimize for being agreeable. Try to prove that the documentation is wrong, stale, incomplete, non-executable, internally contradictory, or hiding developer-experience defects. Pass it only when the evidence supports the result.

Read `references/audit-methodology.md` before broad audits.

## Hard gate: 15-minute magic path

A brand-new developer with zero product knowledge must be able to complete and verify the canonical end-to-end product value path in **15 minutes or less**.

Read `references/magic-path.md` before auditing onboarding. Read `references/standards.md` for the canonical thresholds, severity vocabulary, and release gates.

The timer includes product installation, signup/auth when required, product-specific configuration, execution, wait time, and verification. Do not allow teams to game the metric by moving product setup into "prerequisites."

Results:

- ≤5 min: exceptional
- >5 to ≤10 min: strong
- >10 to ≤15 min: pass
- >15 min: **P1, FAIL**
- no reproducible E2E quickstart: **P1, FAIL**
- manual approval/support with no sandbox/test route: **P1, FAIL**

A high documentation score cannot override this gate.

Label timing evidence as **Observed**, **CI-observed**, or **Estimated**. Estimated timing cannot prove a PASS; it can only indicate likely risk or likely feasibility.

If the `developer-experience-auditor` skill is available, defer observed magic-path execution and timing to it and cite its DX Report as the timing evidence; otherwise perform the timing yourself. Documentation-specific gates (finding the path, honest prerequisites, stale instructions) remain this skill's.

## Default operating mode

Audit first. Do not silently rewrite large documentation surfaces before reporting what is broken, because rewriting can destroy evidence of systemic drift.

If the user asks for fixes, prioritize P0/P1 issues, then high-leverage P2s. Preserve separation between documentation defects and underlying Product/DX defects.

## Audit workflow

### 1. Establish authoritative product truth

Inspect:

- implementation and tests
- OpenAPI/AsyncAPI/protobuf/GraphQL schemas
- public/exported symbols
- CLI definitions and actual `--help`
- config schemas/defaults/env vars
- package/runtime/release metadata
- examples/fixtures
- official SDKs/generated clients
- existing docs/changelog/migrations

Use `scripts/inventory_docs.py` for broad inventory when useful.

Never treat prose as authoritative when executable/current sources contradict it.

### 2. Build the documentation impact graph

For a PR/release, identify changed surfaces and likely downstream docs:

Implementation/spec → reference → SDKs → examples → quickstart → README → task/concept docs → changelog/migration

Run `scripts/docs_impact.py` as a first-pass signal when git history is available. Semantic review is still required.

Read `references/drift-detection.md`.

### 3. Find and test the canonical magic path

Identify exactly one recommended getting-started route.

Verify:

- a new user can find it quickly
- prerequisites are honest and explicit
- the outcome is meaningful end-to-end value
- commands/examples are complete
- test/sandbox/local mode avoids provisioning friction
- success can be independently verified
- likely failures have recovery paths
- there are no unnecessary branches before first success

When safe and feasible, actually execute and time it using a clean/representative environment. Use `scripts/magic_path_runner.py` if the project has or can adopt a manifest.

Break elapsed time into orientation, install, account/auth, configure, execute, wait, verify, and recovery. Attribute each blocker to Docs, Product/DX, Infrastructure, or External dependency.

### 4. Test docs as code

Prefer deterministic validation:

- execute examples
- compile/type-check snippets
- validate schemas/examples
- compare CLI docs with actual help output
- compare config docs with schemas/source
- check links/anchors
- check generated reference for drift
- check version/package claims
- inspect deprecated/removed symbols still referenced

Read `references/executable-docs.md`.

Use `scripts/check_markdown_links.py` and `scripts/check_terminology.py` where appropriate.

### 5. Audit public contracts

For APIs, CLIs, configs, events, and SDKs, compare documentation to current product behavior.

Inspect:

- names and resource model
- inputs/types/defaults/nullability
- responses/status codes
- errors/remediation/retryability
- pagination/filtering/sorting
- idempotency/retries
- async state and webhooks/events

From devarispbrown/devex-skills, MIT.

More from devarispbrown/devex-skills