audit-usability

Files (1)

• SKILL.md

SKILL.md (first lines)

TODAY=!`date +%Y-%m-%d`

# Usability Audit (ISO/IEC 25010)

Audits the software's **interfaces** for usability: how easily the intended users can recognize what it does, learn it, operate it, recover from errors, and (for web UIs) access it.
It treats the CLI, the API, and any web surface as the product's usability surface.

This is the **Usability** characteristic of the [ISO/IEC 25010](https://en.wikipedia.org/wiki/ISO/IEC_25010) quality model.
It applies to user-facing software; for libraries with no CLI/API/UI surface, report "no usability surface" and exit.

## Prerequisites

- Working directory is the root of the repository
- Read `.sdlc/context/project-overview.md` if present (to identify the intended audience and surface)
- Determine the surface(s): CLI (argparse/click/typer/cobra), API (HTTP routes), or web (templates/components)

## What This Checks

| Sub-characteristic | What it means | Signals scanned |
|---|---|---|
| Recognizability | users can tell what the software does | README purpose statement; `--help`/`-h` support; API docs (OpenAPI) or README |
| Learnability | new users can get started | examples in help/docs; a getting-started/quickstart; consistent flag/command naming |
| Operability | users can control and observe behavior | sensible defaults; consistent exit codes; `--version`; verbose/dry-run flags; structured logs |
| User error protection | the system prevents and recovers from user mistakes | input validation with actionable messages; confirmation prompts on destructive actions; safe defaults for dangerous flags |
| User engagement | relevant and satisfying to use | (mostly out of static scope; flag missing progress indication for long operations) |
| Accessibility (web) | usable by people with diverse abilities | `alt` text on images; form `label` associations; ARIA roles where needed; sufficient contrast (static subset); keyboard-reachable interactive elements |

## Steps

### 1. Identify the surface

```
rg -n "argparse|click\.command|typer|cobra\.Command|@app\.command|yargs|commander" -g '*.{py,ts,js,go}' .
rg -n "@(app|router)\.(get|post|put|delete|patch|route)" -g '*.{py,ts,js}' .
ls src/components app/templates templates 2>/dev/null
```

Record which surfaces exist.
If none, report "no usability surface" and stop.

### 2. Recognizability

- CLI: does every command have help?
  Missing `-h`/`--help` or empty help strings:
  ```
   rg -n "add_parser|@click|@app\.command|add_argument" -g '*.py' . | rg -v "help="
  ```
- API: is there an OpenAPI spec or documented endpoints?
  ```
  ls openapi.yaml openapi.json swagger.* 2>/dev/null
   rg -n "FastAPI|apispec|flask-restx|springdoc" -g '*.{py,java}' .
  ```
- README states what the software does (first heading + first paragraph).

### 3. Learnability

- Examples present in help or docs (`examples/`, `--example`, usage strings).
- Quickstart in README.
- Naming consistency: do commands/flags follow one convention?
  Mixed styles (kebab + snake + camel across flags) is a finding.

### 4. Operability

- `--version` support:
  ```
   rg -n "version|--version|show_version" -g '*.{py,ts,js,go}' .
  ```
- Exit codes: handlers that `sys.exit(0)` on error, or exit non-zero without a reason; inconsistent exit codes across commands.
- Defaults: required arguments that could have safe defaults; dangerous operations without a dry-run.

### 5. User error protection

- Input validation present on user-facing inputs (see `audit-compatibility` step 6 for overlap; here focus on the *message* quality).
- Error messages that are actionable: flag bare `raise Exception("...")`, `print("error")`, `console.error` without a remediation hint.
  ```
   rg -n "raise (Exception|ValueError|RuntimeError)\(['\"]" -g '*.py' .
   rg -n "console\.error\(|print\(['\"]?(error|Error|ERROR)" -g '*.{ts,js}' .
  ```
- Destructive operations (`delete`, `drop`, `purge`, `rm`, `reset`) without a confirmation or a `--force`/`--yes` gate:
  ```
   rg -n "delete|drop|purge|remove|reset|destroy|rm -" -g '*.{py,ts,js}' . | rg -v "confirm|--force|--yes|-y|dry.run|test"
  ```

### 6. Accessibility (web surface only)

Static-checkable subset:
```
rg -n "<img" -g '*.{html,jsx,tsx,vue}' . | rg -v "alt="
rg -n "<input" -g '*.{html,jsx,tsx}' . | rg -v "id=|aria-label|<label"
rg -n "onclick=|onClick=" -g '*.{html,jsx,tsx}' .
```
Flag images without `alt`, inputs without an associated label, click-only handlers with no keyboard equivalent.

### 7. Confirm the decisive findings

Pick the critical or high findings that decide the report.
Run the one or two you can: write a scratch script or test under `/tmp` that calls the code, run it, and paste the output, reaching `L3 - Executed` (see [`../sdlc/references/evidence.md`](../sdlc/references/evidence.md)).
Never write scratch files into the repository; this audit is read-only and leaves no artifacts behind.
Label every other finding with its level and pointer: `L1 - Cited` for a `file:line`, or `L2 - Ruled out` for a walked failure path.
When a decisive finding cannot be executed, mark it `unproven` and state what runtime evidence it needed and why that was infeasible.

### 8. Report

Classify by severity and print.
Note that some usability issues (contrast, copy quality) need human judgment; surface the static-checkable subset and flag the rest as "manual review".

## Severity

| Severity | Criteria |
|---|---|
| Critical | Destructive operation with no confirmation and no dry-run; a public command with no help at all |
| High | Error messages that do not tell the user how to recover; required inputs with no validation; web images missing alt at scale |
| Medium | Missing `--version`; inconsistent flag naming; missing quickstart |
| Low | Missing examples in help; minor naming inconsistency |

## Output Format

```
# Usability Audit — {TODAY}

From tomzx/agents, MIT.

More from tomzx/agents