human-loop-testing

Files (2)

• SKILL.md
• openai.yaml · agents/

SKILL.md (first lines)

# Human Loop Testing

Treat the human as the observer of real product behavior. Never answer for them, infer a pass, or send multiple actions while waiting for a result.

## Select the mode

Inspect the repository, feature, acceptance criteria, and existing tests before asking discoverable questions. If the user did not already choose a mode, ask one question with these choices:

- **Human** — run guided manual checks; a confirmed failure enters the diagnose-fix-retest loop.
- **Automated** — run the repository's agent-runnable tests without human steps.
- **Hybrid** — complete Human mode first, then run relevant automation.

Selecting Human or Hybrid authorizes in-scope fixes for confirmed defects. Stop for approval if a fix expands scope, changes architecture or product policy, or needs destructive/external action not already authorized.

## Prepare the session

1. Resolve target feature, acceptance criteria, environment, build or commit, platform, device or browser, persona, account, test data, and prerequisites.
2. Discover repository facts yourself. Ask only missing human choices, one question at a time.
3. Reuse the project's test IDs. If none exist, use `TC-HITL-<FEATURE>-NNN`.
4. Define independent cases with observable expected results, including relevant happy path, denial, boundary, persistence, and recovery behavior.
5. Show only test-case names and total count. Do not reveal all action steps.

## Drive Human mode

Send exactly one action per message. Setup, navigation, interaction, recovery, and verification are separate actions. Use this format:

```text
Test: <TC-ID — case name>
Step: <current/total>
Action: <one action for the human>
Expected: <one observable result>

Reply:
Result: PASS | FAIL | BLOCKED
Note: <what happened>
Evidence: <screenshot, video, or redacted log; required for FAIL/BLOCKED>
```

Wait for the reply. Do not provide the next action in the same message.

Maintain session state: mode, environment, case, step, prerequisites, verdicts, evidence, fixes, and retests. Never skip or silently reset a failed step.

## Process the reply

### PASS

Record the note and advance exactly one step. Mark a case passed only after every required step passes.

### BLOCKED

Do not mark pass or fail. Record the blocker and evidence, then provide one recovery action. Resume the blocked step only after the prerequisite is restored.

### FAIL

1. Freeze progression and record expected versus observed behavior.
2. Require screenshot, video, or relevant redacted log. Ask one targeted evidence question if the failure is still ambiguous.
3. If product intent conflicts with docs, tests, or human expectation, ask for the product decision before changing behavior.
4. Build the smallest agent-runnable reproduction where possible. Use the repository diagnosis and TDD skills when available.
5. Diagnose, add a regression test at the correct seam, implement the in-scope fix, and run the smallest relevant automated checks.
6. Restore the human prerequisite state, then send the failed action again as the only next step.
7. Continue only after the human reports PASS. Keep the case failed or blocked if the issue remains.

Never weaken expectations merely to obtain a pass. Never expose secrets, tokens, passwords, or PII in prompts or evidence.

## Complete Automated and Hybrid modes

- In Automated mode, use the repository-prescribed framework and evidence rules. Do not invent human results.
- In Hybrid mode, start automation only after all Human cases pass. If automation finds a product defect, fix and rerun it; repeat affected Human steps when observable behavior changed.
- Follow repository defaults such as headless Playwright for web and physical-device/wireless-ADB workflows for Android when present.

## Save evidence

Use the project's existing test-results convention. If none exists and repository writes are authorized, save `docs/testing/hitl/<YYYYMMDD-HHMMSS>/summary.md` containing:

- Feature, environment, build or commit, platform, device or browser, and persona
- Test IDs and per-step PASS/FAIL/BLOCKED results
- Human notes and evidence references
- Diagnosed cause, changed files, automated checks, and human retest results
- Remaining blockers and risk

Redact secrets and PII. If repository writes are not authorized, return the same evidence summary in chat.

Declare completion only when all required Human steps pass and, for Hybrid mode, relevant automation passes.

From deepaktiwari09/team-agent-skills, MIT.

More from deepaktiwari09/team-agent-skills