ui-verification

Files (17)

• SKILL.md
• evals.json · evals/
• billing.html · evals/files/
• onboarding.html · evals/files/
• axe-scan.md · probes/
• console-network.md · probes/
• failure-injection.md · probes/
• focus-walk.md · probes/
• layout-shift.md · probes/
• target-size.md · probes/
• theme-locale-matrix.md · probes/
• viewport-stress.md · probes/
• web-vitals.md · probes/
• evidence-output.md · references/
• rule-coverage.md · references/
• session-setup.md · references/
• visual-review.md · references/

SKILL.md (first lines)

# UI Verification

Owns the browser session. Every other UI skill in this repo reasons about source and infers what the user will see; this one loads the page and measures it.

- **IS:** booting the app, driving it with a browser, and running the probes that decide a rule at runtime: computed boxes, injected failures, observed layout shift, a scripted Tab walk, an axe scan per theme. Output is findings keyed to rule ids with reproducible evidence, plus the clearing re-run after a fix.
- **IS NOT:** finding defects by reading source or deciding their tier and ship verdict (`ui-design` Audit mode owns both); building or restyling UI (`ui-design` Build); authoring a durable test suite (write Playwright tests); building or maintaining a whole app's persistent verify CLI, doctor command, and feature map (`app-verification`, call it by name); persistent pixel-baseline approval workflows (Chromatic, Percy); field performance (RUM or CrUX; Lighthouse is a lab tool).

The division of labour is the point. A static audit reports what the code will probably do; it cannot see a 40px control whose hit area a pseudo-element already expands to 44, or a retry button wired to nothing. This skill reproduces or kills each of those, so a finding arrives with a measurement instead of a confidence.

## Contents

- [When to run](#when-to-run)
- [Probe catalogue](#probe-catalogue)
- [Progress checklist](#progress-checklist)
- [1. Establish the session](#1-establish-the-session)
- [2. Select the probe set](#2-select-the-probe-set)
- [3. Run the probes](#3-run-the-probes)
- [4. Decide each finding](#4-decide-each-finding)
- [5. Clearing re-run](#5-clearing-re-run)
- [6. Report](#6-report)
- [Honesty rules](#honesty-rules)
- [Gotchas](#gotchas)
- [Related skills](#related-skills)

## When to run

| Situation | What this skill does |
|---|---|
| A `ui-design` audit emitted findings and the user wants them confirmed | Run only the probes those rule ids map to, in `references/rule-coverage.md` |
| No audit ran; the user points at a route or a running app | Detect features, run the full battery on the resolved routes |
| A fix just landed for a previously reproduced finding | Run the clearing re-run only (step 5) |
| The user asks for captures across themes or widths | `probes/theme-locale-matrix.md`, then visual review below |
| The user asks to review a UI change or named screenshot states | Read [references/visual-review.md](references/visual-review.md) for matched before/after inspection and suspicious duplicates |
| A `design-system.md` claims a scale and someone needs it checked | Read the claimed values off computed styles on a real page; a theme value the build overrides never reaches the browser |

Nothing here assigns a tier or a ship verdict. Hand reproduced findings back with their evidence and let `ui-design`'s `references/ship-readiness.md` tier them; two skills tiering the same finding is how the tiers drift apart.

## Probe catalogue

Each file is one probe: what it measures, the driver calls, the false positives it must guard, and the shape of the evidence it returns.

| Probe | Measures | Primary for |
|---|---|---|
| [probes/axe-scan.md](./probes/axe-scan.md) | axe-core violations per route and theme, including computed contrast | contrast, accessible names, landmarks |
| [probes/target-size.md](./probes/target-size.md) | Bounding box and effective hit area of every visible interactive element | `interaction-target-size` |
| [probes/focus-walk.md](./probes/focus-walk.md) | Scripted Tab traversal, focus-ring pixel delta, dialog trap and restoration | `focus-*`, `interaction-focus-visible`, `interaction-keyboard-operable` |
| [probes/layout-shift.md](./probes/layout-shift.md) | Attributed `layout-shift` entries with the data response held open | `states-layout-shift`, `perf-image-dimensions-and-priority` |
| [probes/viewport-stress.md](./probes/viewport-stress.md) | Horizontal overflow and clipped text at 320px and up, and under tripled strings | `layout-long-content-safety`, `mobile-viewport-scaling` |
| [probes/failure-injection.md](./probes/failure-injection.md) | What renders when the data request returns 500, `[]`, malformed, or nothing | `states-no-error-state`, `states-no-empty-state`, `async-*`, `microcopy-*` |
| [probes/theme-locale-matrix.md](./probes/theme-locale-matrix.md) | The same route captured across viewport, theme, pseudo-locale, and direction | `dark-i18n-untested`, `dark-i18n-rtl-untested` |
| [probes/console-network.md](./probes/console-network.md) | Console errors, page errors, failed requests, hydration mismatch warnings | hydration mismatch, silent runtime failures |
| [probes/web-vitals.md](./probes/web-vitals.md) | LCP with its attributed element, CLS, INP on a scripted interaction | perf attribution, never a budget verdict |

## Progress checklist

```text
Verification progress:
- [ ] Step 1: Establish the session (references/session-setup.md): driver, build mode, base URL, auth, resolved routes
- [ ] Step 2: Select the probe set from handed-over rule ids (references/rule-coverage.md) or from the routes
- [ ] Step 3: Run each probe; record evidence artifacts before interpreting any of them
- [ ] Step 4: Decide each finding reproduced / not-reproduced / unknown; repeat timing-sensitive or inconsistent results when needed
- [ ] Step 5: For each fix applied, re-run the identical probe and record clearedBy
- [ ] Step 6: Emit the verification block per finding (references/evidence-output.md), then render
- [ ] Step 7: List probes skipped and why. A probe that could not run is never a pass
```

## 1. Establish the session

Read [references/session-setup.md](./references/session-setup.md). It resolves four things, and getting any of them wrong invalidates every probe downstream:

- **Driver.** Playwright when the repo can run it, the Chrome DevTools browser tools when it cannot. Four probes need request interception and network control, so they do not exist under a driver without them.
- **Build mode.** Perf and layout-shift probes run against a production build; dev-mode numbers measure the bundler. Failure injection and focus probes run fine in dev.
- **Auth and seeded data.** A route behind a login with no seeded session returns `unknown`, never a pass.
- **Routes.** Map the diff to URLs. A component with no reachable route falls back to the repo's Storybook, and to `unknown` if there is none.

Stop here if the app will not boot. A verification run with no session produces no findings, which is a reportable outcome and not a clean bill of health.

## 2. Select the probe set

**Handed a list of rule ids** (the normal case, from a `ui-design` audit): open `references/rule-coverage.md`, take the probe each id maps to, and run only those. Ids with no probe stay source-only findings and pass through untouched, marked as such.

**Given only routes:** detect features the way `ui-design`'s feature playbooks do (form, list, modal, dashboard, checkout), then run the battery that surface earns. `probes/axe-scan.md`, `probes/console-network.md`, and `probes/viewport-stress.md` run on every route regardless: they are cheap, and they are the three that find things nobody suspected.

Budget the matrix before running it. Routes multiplied by viewports multiplied by themes grows fast, and a run that takes twenty minutes gets skipped next time. Two viewports (360 and 1280) and two themes cover the ground; add widths only where a probe already found an edge.

## 3. Run the probes

Each probe file carries its own recipe. Three rules hold across all of them:

- **Capture evidence before interpreting it.** Write the screenshot, the JSON measurement, and the console log to disk first. A finding whose evidence was never written is unverifiable by the person reading the report, which puts it back where the static audit left it.
- **One probe, one route, one viewport, one theme.** Never fold two conditions into one run: when the result surprises you, you need to know which axis produced it.
- **Repeat uncertain measurements.** Retry timing-sensitive or inconsistent results under controlled conditions. A deterministic failure with a captured trigger needs no duplicate run. A result that flips remains inconclusive until its conditions are understood.

For UI-change and named-state captures, inspect the images yourself before reporting, following `references/visual-review.md`. Record visual observations separately from measured probe results.

## 4. Decide each finding

Three outcomes, and the middle one is the one that earns this skill its keep.

| Outcome | Meaning | What it does to the handed-over finding |
|---|---|---|
| `reproduced` | The probe measured the defect | Stays a `fail`, now carrying `observed` from the measurement rather than from the source read |
| `not-reproduced` | The tested conditions did not exhibit the defect | Withdraw only if the probe exercised the alleged trigger; otherwise retain the candidate with the remaining coverage gap |
| `unknown` | The probe could not run or could not decide | Finding survives as `unknown` with the probe's reason. Never converts to a pass |

A `not-reproduced` result is a real deliverable, not a wasted run. It is what removes the false positives a large rule corpus asserts with `file:line` confidence, and it feeds the rejection section `ui-design` already requires.

Where a probe finds something no static rule predicted, emit it as a new finding against the rule id the probe is primary for. Where no rule covers it (an axe violation with no ui-design counterpart, a console error), emit it under `axe:<violation-id>` or `runtime:<signature>` and say plainly that it came from the browser and not the corpus.

## 5. Clearing re-run

A fix is not verified by reading the diff. Re-run the identical probe: same route, same viewport, same theme, same seed, same injected failure. Record it as `clearedBy` on the finding, keep the before-artifact, and never overwrite it with the after.

Three outcomes worth naming:

- The probe now passes: the finding is `applied` and cleared.
- The probe still fails: the fix did not work. Report it as applied-unverified and say so; do not report the fix and stay silent about the re-run.
- The probe passes but a different probe on the same route now fails: the fix caused a regression, which is a new finding, not a footnote on the old one.

From mblode/agent-skills, MIT.

More from mblode/agent-skills