evaluate-ux

Files (1)

• SKILL.md

SKILL.md (first lines)

# UX Evaluation

Shared files: a path written `@rules/…`, `@skills/…`, `@templates/…` or `@docs/…` is relative to the
plugin root, `${CLAUDE_PLUGIN_ROOT}` — not to the project being worked on. Read it from there.

## Desired Outcome

A UX evaluation of the existing UI that a migration decision can rest on, and that a second run on
the same UI would reproduce:

1. **Findings** — each one a defect from the vocabulary of @rules/ux-evaluation.md §5, on the axis
   that owns it, citing a screen, action, input, component, feature or token, a criterion, a
   calibrated severity and a concrete recommendation.
2. **Five axis scores and the UXI** — Heuristics, Accessibility, Efficiency, Consistency,
   Navigation, each bounded by its metrics and by its own findings' severity, combined by the
   formula of §3 into one index and a band that says whether to carry the UI over or redesign it.
3. **An honest method statement** — static expert review from code, plus runtime evidence where it
   was captured; never presented as a usability test with real users.

## Decision Criteria

- **Measure, do not estimate** (@rules/ux-evaluation.md §1). Every count, ratio and contrast value
  comes from `tools/lib/ui_metrics.py`; findings quote it as `metrics.<path>` with its value.
- **One defect, one finding, on the axis that owns it** (§2, §5). A redundant e-mail field is one
  `redundant-input` on E — not also an H8, an H6 and an H4.
- **Scores are fenced twice** (§2): by the metric cap and by the worst severity among the axis's own
  findings. Judgement moves a score inside the fence, never outside it.
- **The parent computes the UXI**, after every axis evaluator has returned.
- **Findings before scores.** The report leads with what costs users most; the index summarizes.
- **Security and architecture are routed, not scored** — view-only authorization, client-only
  validation and view-layer business logic belong to `/architect:investigate-security` and the
  redesign (§1 Out of scope).

## Prerequisites

| Input | Required/Recommended | Source | If missing |
|-------|---------------------|--------|------------|
| `reports/before/{project}/ui-inventory.json` + `ui-design-tokens.json` | Required | `/architect:analyze-ui` | Stop: "run /architect:analyze-ui first" |
| `reports/01_analysis/actors-roles-permissions.md` | Recommended (the pipeline runs `analyze` first) | `/architect:analyze` | Judge per role from `access.roles` alone |
| `reports/02_evaluation/ux-evaluation.json` | Optional | A previous run of this skill | No baseline: every finding is judged fresh |
| `reports/01_ux/personas.md`, `reports/01_ux/journey-maps.md` | Optional (read-only) | `/product:*` | No persona weighting; say so in the method statement |
| A running instance of the UI | Optional | User (`--base-url`) | Static evaluation only |

## Invocation

```
/architect:evaluate-ux [--base-url=<url>] [--storage-state=<path>] [--confirm-versions|--no-confirm-versions] [--auto]
```

| Argument/Flag | Required | Description |
|---------------|----------|-------------|
| `--base-url=<url>` | Optional | A running instance (`http://localhost:8080`). Adds runtime evidence per screen route (Step 4) |
| `--storage-state=<path>` | Optional | A Playwright storage-state file holding an authenticated session. Repeatable; write `<role>=<path>` to capture role-guarded screens with that role's session (`--storage-state=admin=work/ux-evaluation/runtime/admin.json`). Credentials are never asked for or written into a report |
| `--confirm-versions` / `--no-confirm-versions` | Optional | Whether to confirm the resolved Playwright / axe-core versions before installing them for the runtime pass (@rules/dependency-versions.md §4) |
| `--auto` | Optional | Ask nothing: every task is a primary task, carried-forward Open Questions stay open, and runtime capture runs only if `--base-url` is given. `/architect:pipeline` invokes it this way, static only |

## Execution

Write each step's output as soon as it completes; intermediate files go under `work/ux-evaluation/`.

### Step 0: Register the phase

Stamp `evaluate-ux` `in_progress` with `"plugin": "architect"` and `started_at` in
`work/pipeline-progress.json` (@skills/common/progress-registry.md).

### Step 1: Load the model, the baseline and the open questions

1. Locate `reports/before/*/ui-inventory.json` (one per project; `project_name`'s when there are
   several) and validate it:
   ```bash
   python3 "${CLAUDE_PLUGIN_ROOT}/tools/lib/ui_inventory.py" <project_dir>
   ```
   A non-zero exit stops the skill — an evaluation of an inventory that is not well-formed carries
   its defects into every score.
2. If `reports/02_evaluation/ux-evaluation.json` exists, it is the **baseline**: copy it to
   `work/ux-evaluation/baseline.json`. Evaluators receive it (Step 5) and keep a baseline judgement
   for an unchanged subject unless new evidence contradicts it.
3. Read the actor matrix and, when they exist, the product personas and journeys (read-only; never
   write under `reports/01_ux/`).
4. Read the Open Questions store (`work/context.md` § Open Questions) and pick up every `deferred` /
   `unasked` entry whose impact names this skill or the UX of a screen (@rules/open-questions.md §7).

### Step 2: Compute the metrics

```bash
python3 "${CLAUDE_PLUGIN_ROOT}/tools/lib/ui_metrics.py" <project_dir> > work/ux-evaluation/metrics.json
```

This is the `metrics` block of the evaluation, embedded verbatim. Its `caps` are the ceiling of each
axis score.

### Step 3: Primary tasks and carried-forward questions

In one `AskUserQuestion` batch (at most four questions, @rules/open-questions.md §3):

- **Which tasks are primary** — `multiSelect: true`, up to four candidate tasks from
  `metrics.tasks`, ranked by `steps`, then by `required_inputs`; "Other" names different ones. No
  option is marked recommended: which tasks matter most is the user's call. The Efficiency evaluator
  weighs these first.
- **The carried-forward Open Questions** of Step 1, re-asked with their recorded options; answers
  are written back to the store under the same `OQ-` id.

Under `--auto`, every task is primary and the carried-forward questions stay as they are; the method
statement says both.

### Step 4: Runtime evidence (only with `--base-url`)

Skip this step without `--base-url`, and always under `/architect:pipeline`.

1. **Driver.** Prefer the Playwright MCP tools when the session has them; otherwise install
   `playwright` and `@axe-core/playwright` into `work/ux-evaluation/runtime/` with versions looked
   up per @rules/dependency-versions.md (confirmed per `--confirm-versions`) and run a script there.
   With the MCP there is no axe-core: its accessibility snapshot and the checks below take its place,
   and the method statement says so. If its screenshot tool cannot write under the project, take the
   screenshot through its run-code tool (`page.screenshot({ path })`); results that tool returns are
   written to their files by the parent. Pass scripts to it as functions, not as strings — a string
   silently returns nothing.
2. **Sessions.** Clear the browser's cookies first, so no earlier session is reused silently. Load
   each `--storage-state` (the MCP route: `context.addCookies` from the file's cookies). A user

From wfukatsu/nexus-architect, MIT.

More from wfukatsu/nexus-architect