docs-review

Files (4)

• SKILL.md
• DETECTION-PLAYBOOK.md · references/
• QUALITY-FILTER.md · references/
• REVIEW-VOICE.md · references/

SKILL.md (first lines)

# Documentation Review

A specialised lens on [`pr-review`](https://github.com/priyank766/OpenSource-SKILL/blob/main/skills/pr-review/SKILL.md), for diffs that change documentation
and for diffs that break it without touching it.

**Read first:** [`DETECTION-PLAYBOOK.md`](references/DETECTION-PLAYBOOK.md) ·
[`QUALITY-FILTER.md`](references/QUALITY-FILTER.md) ·
[`REVIEW-VOICE.md`](references/REVIEW-VOICE.md)

> **Where the time goes.** Finding the flaw is the review. The playbook above is the
> general flaw-hunting procedure; this file is the domain-specific lens on top of it.
> Scoring and writing the comment are the last ten minutes, not the work.

**Severity floor: 8.5** — higher than the suite default of 8.0. Docs attract more low-value
opinion per line than any other part of a codebase: everyone can read prose, so everyone has a
view on it, and a review thread fills with comma placement while the broken `pip install` line
three paragraphs down goes unmentioned. Raising the floor is the only defence. At 8.5, a finding
ships only if it is a correctness error or something that stops a reader completing the task.

---

## The governing rule

**Documentation is code that is never executed, so it rots silently. The job is to execute it.**

Nothing in CI runs the README. No test fails when a tutorial's second command starts erroring.
The drift is invisible until a user hits it, and by then it has cost them an hour and cost the
project an issue. Wrong docs are worse than missing docs: a missing doc costs a search, a wrong
doc costs a debugging session against a description of software that no longer exists.

So the review is not a read. It is a run. Every fenced block gets executed, every command gets
typed from an empty directory, every symbol named in prose gets grepped for in the source. What
you cannot execute, you verify against the code. What you can neither execute nor verify, you do
not comment on.

Grammar is not the job.

---

## Why this exists

Docs PRs get either rubber-stamped or nitpicked, rarely reviewed. Both failure modes are
expensive. Rubber-stamping ships instructions nobody tried; nitpicking spends a maintainer's
evening on style and still ships instructions nobody tried.

### What each role gets out of it

| Role | Time this gives back |
| :--- | :--- |
| **Maintainer** | The verification you would have to do yourself, already done and pasted: the exact command, the exact error. No thread about hyphenation. A docs PR becomes a two-minute merge instead of a deferred tab. |
| **Contributor** | Your quickstart tested from a clean container before a maintainer discovers it fails on a machine without your `~/.config` already populated. |
| **Project owner** | Fewer "the README doesn't work" issues, which are the highest-volume and lowest-information class of report a project receives, and the one most likely to make a first-time user leave. |

---

## Procedure

```mermaid
flowchart TD
    A[Start] --> B[1. Classify the diff: docs-touching or docs-invalidating]
    B --> C[2. Build a clean environment]
    C --> D[3. Execute every code sample]
    D --> E[4. Walk the quickstart from empty state]
    E --> F[5. Grep every referenced symbol against source]
    F --> G[6. Links, anchors, versions]
    G --> H[7. Reverse direction: what did the code diff invalidate?]
    H --> I[8. Changelog vs actual diff]
    I --> J[9. Score with the 10-point filter]
    J -->|"score below 8.5"| K[Discard, or batch as one suggested-changes review]
    J -->|"score 8.5 or above"| L[10. Report with command and actual error]
    L --> M[11. Get explicit approval, then post]
```

---

## 1. Classify the diff

Two different reviews live here. Run whichever applies; run both when both apply.

```bash
git diff --name-only upstream/main...pr-<N>
```

Prose paths (`docs/`, `*.md`, `*.rst`, `*.mdx`, `site/`, docstrings) mean **forward direction**:
does what this PR now says match reality? Source paths only, with no docs touched, mean
**reverse direction**: what does this PR make untrue elsewhere? Section 7 covers that, and it is
the class of defect reviewers miss most.

---

## 2. Build a clean environment

Your machine is contaminated. It has the package installed, the credentials cached, the repo
already cloned, the tool on `PATH` from some other project. Every one of those hides a missing
prerequisite step.

```bash
# Throwaway container, repo mounted read-only (`:ro`) since you run untrusted README
# commands in here; build into /tmp inside the container if a step needs to write.
docker run --rm -it -v "$PWD":/w:ro -w /w python:3.12-slim bash
docker run --rm -it -v "$PWD":/w:ro -w /w node:22-slim bash
docker run --rm -it -v "$PWD":/w:ro -w /w rust:1-slim bash

# No container available: at minimum a fresh venv and a scratch HOME
python -m venv /tmp/docscheck && . /tmp/docscheck/bin/activate
env -i HOME=$(mktemp -d) PATH=/usr/bin:/bin bash --noprofile --norc
```

`env -i` is the cheap version of the whole exercise: it strips the environment variables that
your shell has been quietly supplying to every command in the tutorial.

---

## 3. Execute every code sample

### Extract the fenced blocks

```bash
# All fenced blocks with their language tag, numbered, from one file

From priyank766/OpenSource-SKILL, Apache-2.0.

More from priyank766/OpenSource-SKILL