test-health

Files (7)

• SKILL.md
• artifact-formats.md · references/
• test-count-parsers.md · references/
• trend-schema.md · references/
• artifact-parser.js · scripts/
• count-parser.js · scripts/
• trend.js · scripts/

SKILL.md (first lines)

# Test Health — Holistic Coverage Measurement

## Trigger

- Keywords: test health, coverage measurement, test metrics, coverage trend, test inventory, holistic test audit

## When NOT to Use

| Scenario | Alternative |
|----------|------------|
| Run tests | `/verify` |
| Review test sufficiency only | `/codex-test-review` |
| Generate unit tests | `/codex-test-gen` |
| Feature-doc coverage only | `/check-coverage` |
| Context-aware test execution + triage | `/test-deep` |

## Workflow

```mermaid
flowchart TD
    U[User: /test-health] --> M{Mode?}
    M --> |quick| Q[Quick Mode]
    M --> |--full| F[Full Mode]

    Q --> Q1[Test Inventory]
    Q1 --> Q2[Consume Coverage Artifacts]
    Q2 --> Q3[Trend Delta]
    Q3 --> QR[Quick Dashboard]

    F --> A[Phase A: /check-coverage]
    A --> B[Phase B: Coverage Collection]
    B --> C[Phase C: /codex-test-review]
    C --> D[Phase D: Aggregate Dashboard]
    D --> T[Trend Snapshot]
    T --> FR[Full Dashboard]
```

## Modes

| Mode | Trigger | Content | Duration |
|------|---------|---------|----------|
| `quick` (default) | `/test-health` | Test inventory + consume artifacts + trend delta | <15s |
| `full` | `/test-health --full` | Phase A→B→C→D (feature coverage + instrumentation + qualitative + aggregation) | 2-5min |

## Quick Mode Workflow

1. **Test Inventory**: Count test files by layer using Glob (see `references/test-count-parsers.md` for layer classification). If `--scope <path>` specified, limit Glob to that directory. If verify-runner cache exists (`.claude/cache/verify/`), read historical logs for test counts.
2. **Coverage Artifacts**: Scan for existing coverage artifacts (see `references/artifact-formats.md`). If `--scope` specified, scan within scope only. Never execute project commands in quick mode.
3. **Trend Delta**: Read previous snapshot, compute delta (see `references/trend-schema.md`). Skip if `--no-trend` flag is set.
4. **Output**: Quick Dashboard.

## Full Mode Workflow

### Phase A: Feature Coverage

Resolve docs path using `bash scripts/resolve-feature.sh` (same cascade as other skills) — the shim over the wrapper, which emits the full shape with `scan_error: true` rather than a bare `{}` however the CLI fails: nonzero exit, signal, partial write, or a payload that is not the agreed shape. It cannot cover `node` itself being unavailable — the shim would exit 127 with no JSON — so treat an empty or non-JSON reply as a failure too. **Gate on `scan_error !== false` before reading anything else** — never on `=== true`, because an
empty or non-JSON reply carries no such field at all and the stricter test is false for it. Only once
the flag is exactly `false` does any other field mean what it says: the failure payload sets
`has_tech_spec` false along with everything else, so branching on that field first reports an
unreadable corpus as a feature with no documents, and the coverage of a real feature disappears
behind a reassuring advisory.

| Payload | Phase A |
|---------|---------|
| `scan_error !== false` (including an empty or non-JSON reply) | Skip, advisory `"Phase A skipped: feature docs could not be read (scan_error) — coverage is unknown, not absent"` |
| `scan_error: false`, `has_tech_spec: true` | Dispatch `/check-coverage <docs_path>` via Skill tool |
| `scan_error: false`, feature unresolved or no tech spec | Skip, advisory `"Phase A skipped: no feature docs detected"` |

### Phase B: Test Inventory + Coverage Collection

1. Count test files by layer (same as quick mode)
2. If `--collect` flag: execute project coverage command (`test:coverage` or `coverage` from `package.json`)
3. Otherwise: consume existing coverage artifacts (same as quick mode)
4. Parse test runner stdout for test counts (see `references/test-count-parsers.md`)

### Phase C: Qualitative Review

Dispatch `/codex-test-review` via Skill tool for 5-dimension quality assessment.

### Phase D: Aggregate + Trend

1. Aggregate all dimensions into full dashboard
2. Write trend snapshot (see `references/trend-schema.md`)
3. Output Full Dashboard

## Coverage Collection Strategy (Consume-First)

| Priority | Method | Trigger | Output |
|----------|--------|---------|--------|
| 1 | Consume existing artifact | Default (quick + full) | `source_type: instrumented_artifact` |
| 2 | Run project coverage command | `--collect` flag only (opt-in) | `source_type: collected_now` |
| 3 | Heuristic proxy (test/source file ratio) | No artifact and no `--collect` | `source_type: heuristic` |

**Prohibited**: Never auto-install coverage tools (c8, nyc, istanbul, pytest-cov, tarpaulin, jacoco).

## Output: Quick Dashboard

```markdown
## Test Health (Quick)

### Test Inventory
| Layer | Files | Tests | Source |
|-------|-------|-------|--------|
| Unit  | 25    | 47    | cached_stdout |
| Integration | 1 | 12  | cached_stdout |
| E2E   | 0     | —     | file_count |

### Code Coverage
| Metric | Value | Tool | Freshness |
|--------|-------|------|-----------|
| Lines  | 82.3% | c8   | current   |
| Branches | 76.0% | c8 | current   |

### Trend (vs previous)
| Metric | Previous | Current | Delta |
|--------|----------|---------|-------|
| Line coverage | 80.2% | 82.3% | +2.1% |
| Test count | 57 | 59 | +2 |

From sd0xdev/sd0x-harness, MIT.

More from sd0xdev/sd0x-harness