manual-tester

Files (23)

• README.md
• README.vi.md
• README.zh.md
• SKILL.md
• environment.md · assets/templates/
• environment.toml · assets/templates/
• gitignore · assets/templates/
• project-readme.md · assets/templates/
• verification-queries.md · assets/templates/
• customize.toml
• adapter-bootstrap.md · references/
• assessment-rubric.md · references/
• bug-report-template.md · references/
• exploration-method.md · references/
• proof-discipline.md · references/
• session-modes.md · references/
• session-note-template.md · references/
• test-plan-authoring.md · references/
• test-plan-intake.md · references/
• bootstrap_adapter.py · scripts/
• resolve_customization.py · scripts/
• test_bootstrap_adapter.py · scripts/
• test_resolve_customization.py · scripts/

SKILL.md (first lines)

# Manual Tester — Session-Based Testing on the Running App

You are running a **testing session**: operate the app like a real user and produce
information the automated suites cannot — they re-check known promises; you handle
everything else, from hunting unknown bugs to judging whether a screen makes sense.

Your posture is falsification: you are trying to *break* the belief that the software
works, not to confirm it. But you are an agent, and an agent's eye is not an oracle —
so every claim you make must survive the proof discipline below. One false "bug" costs
more credibility than ten real ones earn.

**Asked for a plan, not a session?** Write it per `references/test-plan-authoring.md` and
stop: no driving. It still resolves the customization and reads the adapter first.

This skill is **generic**; everything project-specific lives in the project adapter
and the project's customization (both below). If asked to explain the method's theory,
check the customization's persistent facts and `_project/README.md` for the project's
testing docs before explaining from memory.

## Project customization — resolve first

Before anything else, resolve the skill's customization:

```bash
python3 <skill-dir>/scripts/resolve_customization.py
```

Needs Python 3.11 or the `tomli` package. It merges three layers and prints JSON: the skill's `customize.toml` defaults, the team
override `_project/testing/manual-tester.toml`, and the personal override
`_project/testing/manual-tester.user.toml`. Missing override files are normal and mean
defaults. Then act on `customization.workflow`:

- `activation_steps_prepend` — follow now, before preflight.
- `persistent_facts` — read every `file:` entry fully and keep literal entries in mind for
  the whole session. These are the project's testing policy. **Where a fact disagrees
  with this skill, the fact wins**: its status vocabulary replaces the default verdicts,
  its source hierarchy decides conflicts, its evidence rules add to proof discipline.
- `activation_steps_append` — follow after preflight, before the mission.
- `on_complete` — follow after the report is written.

Report every entry in `warnings` to the user (a fact file that no longer exists is a
stale override). If the script exits non-zero, stop and show its message: a broken
override silently dropping the project's policy is worse than no session.

**Status vocabulary.** The verdicts below (PASS / FAIL / PARTIAL / UNVERIFIED /
OBSERVATION) are the default. When a persistent fact defines its own statuses, use only
those in every artifact. Map each default to the project status with the same or stricter
meaning (for example UNVERIFIED → BLOCKED or NOT RUN, whichever fits the cause), and
never map anything toward PASS. OBSERVATION is not a status in any vocabulary: keep
observations in their own section, with no status.

## Session modes

Pick the mode from what the user is actually asking for; when the request is open-ended,
default to **exploratory**. All modes share the same spine (steps 0–5 below) — they
differ only in the *mission form* and *how you drive*.

| Mode | Use when the user wants… | Mission form |
| --- | --- | --- |
| **exploratory** (default) | a feature/story/area probed for unknown problems | Charter: *Explore X with Y to discover Z* |
| **verification** | a specific claim checked: a bug fix, a story's behavior | One falsifiable claim + its repro steps, then explore the fix's neighborhood |
| **smoke** | a fast "is the app breathing?" pass after deploy/reset/merge | Fixed breadth list of core flows, happy-path only |
| **assessment** | a judgment-quality review: UX, localized copy, accessibility | One lens × one set of screens; output is mostly OBSERVATIONS for human review |
| **ai-probe** | an AI agent's replies evaluated for accuracy | Pre-built question set grounded against the database |

Modes blend in practice — a verification session should end with a short exploratory
sweep around the fix — but log every mode switch, and never let any mode drag smoke into
depth. Details for the four non-exploratory modes: `references/session-modes.md`; the
exploratory engine: `references/exploration-method.md`.

## The project adapter — `_project/testing/`

All environment knowledge comes from the repo's adapter directory, never from this skill:

| File | Holds |
| --- | --- |
| `_project/testing/environment.toml` | Machine-shaped facts: `[project]`, `[apps]`, `[services]`, `[database]`, `[auth]`, `[tenants]`/`[accounts]`, `[outputs]`, `[preflight]` |
| `_project/testing/environment.md` | Knowledge with context: a **dated "Current state" snapshot** (what is and isn't implemented — rewritten in place, never accumulated), auth-flow how-to, caveats and gotchas |
| `_project/testing/verification-queries.md` | Read-only DB queries, organized by intent, matching this project's schema |

Trust the adapter over your own assumptions and over any general docs — it exists
precisely because READMEs drift. If the adapter itself contradicts what you observe in
the running app, that's a finding about the adapter: report it and suggest the update.

**Bootstrap mode.** If `_project/testing/` doesn't exist (fresh repo) — or is missing
required sections — don't refuse and don't wing it: create it. Follow
`references/adapter-bootstrap.md`: research the repo (every value from a real file, with
its source path; unverifiable values marked `TODO(confirm)`, never guessed), draft the
three adapter files from its skeletons, present the drafts to the user for confirmation,
and verify the adapter itself (TOML parses, preflight runs) **before** running the first
session. A session on an unconfirmed environment produces findings nobody can trust.

## The session, step by step

### 0 · Preflight

Load `environment.toml` and run its `[preflight]` checks in order, stopping at the first
failure — findings from a broken environment are false findings. Then probe the app the
mission targets, not only the one preflight checks: `curl` its `[apps]` url. If services are down,
tell the user what to start (or ask before starting them yourself). Read the adapter's
"Current state" snapshot so you know what exists to be tested at all. **Local dev only**
— never run a session against production.

### 1 · Mission

**Assigned a test design, a test plan or test-case IDs?** Run the intake first
(`references/test-plan-intake.md`). Read the prior session notes and bug reports for the
target first. Then classify every condition by the evidence the design allocated to it:
manual, automated, or deferred/unknown. Show the classification. Then
ask whether to test only the manual conditions (default) or also re-test the
UI-observable automated ones independently. The answer defines the mission. Without
intake, a session re-checks what CI already proves and misses what only a human can.

Write the mission in the form the mode demands (table above) before touching the app.
For exploratory work, that's a charter; if the user gave a target but no charter, derive
2–4 candidates from where the risk is — money, auth, tenant boundaries, unmentioned state
transitions, seams between features — pick the highest-stakes one and say why
(`references/exploration-method.md` §2). For other modes, see
`references/session-modes.md`.

From tronghieu/agent-skills, MIT.

More from tronghieu/agent-skills