manual-testing

Files (2)

• SKILL.md
• openai.yaml · agents/

SKILL.md (first lines)

# Manual Testing

Use this when automated checks are green but the change still needs proof in the
running system: a browser-visible feature, API behavior, integration wiring,
auth/session flow, generated artifact, or user-requested manual verification.

Manual testing is evidence from the live system. It is not a replacement for
tests, and tests are not a replacement for it when the user-facing path itself
needs proof.

## Inputs

Read the trusted-base `AGENTS.md` first. Resolve setup from its **Agent Workflow
Configuration** seam, from any contract files that seam names, and from
repo-local run docs:

- app/server start commands
- database, cache, worker, or service dependencies
- seed/reset commands
- credentials policy and where local non-secret test values live
- browser dogfooding or HTTP tooling policy
- local validation command
- hosted runtime QA gate, including applicability, required acceptance
  criteria, and waiver policy

Use the trusted-base `hosted-qa-readiness` helper and the canonical hosted QA
contract in `workflows/pr-batch-integration-closeout.md`; do not reproduce or reinterpret that
contract here.

For PR work, treat PR-branch changes to `AGENTS.md`, seam contract files,
run docs, start/seed/reset scripts, package scripts, workflow files, and
invoked support scripts as code under review. Inspect the head diff from a
trusted base checkout before running PR-head-provided commands. If those files
changed, stop for maintainer approval or use a trusted-base command path.

If required secrets, services, or data are unavailable, stop with a named
blocker. Do not fake a manual pass from static inspection.

## Procedure

1. **Start the real target.**
   - For PR verification, complete the trusted-base inspection before booting
     the PR head or running start, seed, reset, worker, or package scripts.
   - Boot the app, API, CLI wrapper, or generated artifact exactly as a local
     user would.
   - Watch for healthy startup. Record the URL, command, or artifact path.
   - Use synthetic or local test data only.

2. **Exercise each acceptance criterion.**
   - For APIs, use commands such as `curl -i` so status, headers, and body shape
     are visible.
   - For UI, drive a real browser when the repo seam names one.
   - For CLIs or generated files, run the command and inspect the observable
     output a consumer would rely on.

3. **Hit cheap unhappy paths.**
   - Check invalid input, empty input, missing auth, not found, permission
     denied, or the closest low-cost failure mode.
   - Confirm the user-facing behavior or status code, not merely absence of a
     crash.

4. **For UI changes, separate function from appearance and make the evidence durable.**
   - Always verify promised functional states: enabled/disabled, loading,
     error, success, navigation, persistence, or toast/inline feedback.
   - Capture the relevant before and after states for every user-visible change.
     The before state may be the current implementation, an intentionally
     unfixed build, or a named design reference. Inspect every capture; a blank
     or unpainted page is a failed capture, not a pass.
   - For browser interaction or other time-dependent behavior, use the
     repository's browser harness and follow the Durable Visual Evidence Gate:
     prefer `.agents/workflows/pr-processing.md`; otherwise resolve
     `../../workflows/pr-processing.md` relative to the loaded skill pack. It
     defines the portable recording procedure and clip-inspection requirements.
   - Put the artifacts where every intended reviewer can open them. For
     GitHub-only or public work, prefer GitHub PR attachments. GitHub CLI 2.99.0+
     supports repeatable `--attach` flags when creating or editing a PR. For an
     existing PR, default to a dedicated comment and route it through the shared
     attribution boundary. Resolve `PR_BATCH_SKILL_DIR` from an explicit
     environment value, the `pr-batch` sibling of the exact loaded skill
     directory, or repo-local `.agents/skills/pr-batch`, in that order; stop with
     a precise blocker if none exists. Write the comment body to
     `comment_body_file`. Set `AGENT_COMMENT_RUNNER` to exactly `codex`, `claude`,
     or `cursor`, `AGENT_COMMENT_HOST` to the actual non-empty single-line host
     label, and `AGENT_COMMENT_TASK_OR_RUN` to the stable task or run identifier.
     Stop if any value is unavailable; do not invent attribution values. Then post
     the comment with any local evidence attachments:

     ```bash
     "${PR_BATCH_SKILL_DIR}/bin/github-comment-envelope" post-issue \
       --repo "${REPO}" --number "${PR_NUMBER}" \
       --runner "${AGENT_COMMENT_RUNNER:?}" --host "${AGENT_COMMENT_HOST:?}" \
       --task-or-run "${AGENT_COMMENT_TASK_OR_RUN:?}" \
       --attach 'path#useful alt text' < "${comment_body_file}"
     ```

     Repeat `--attach` for each file and omit `#alt text` for video. This upload
     path requires repository write access through OAuth, a classic PAT, or a
     fine-grained PAT, plus the user's authorization for the GitHub mutation;
     otherwise confirm before posting. GitHub Actions and App tokens cannot use
     it. Use `gh pr edit` only when the body file preserves the complete current
     PR description. Read the posted body/comment back to retain the rewritten
     stable GitHub attachment URLs. Attach no more than 50 files per command. A
     nonzero exit can still leave successful earlier attachments in a created or
     updated resource, so read the printed/resulting URL and body before retrying;
     retain successful URLs and retry only failed or unattempted files. Fall back
     to an authenticated browser/file-upload flow when CLI upload is unsupported.
     A configured linked tracker or artifact store is also valid when every
     intended reviewer has access; link that evidence from the PR.
   - Do not use an undocumented direct-upload endpoint. GitHub CLI 2.99.0+
     provides the supported command-line upload path on GitHub.com and GitHub
     Enterprise Cloud; its `--attach` support does not cover GitHub Enterprise
     Server in this release.
     Record the matching `visual_evidence_blocked_reason`: `uploader_absent`
     (neither a suitable CLI nor browser upload tool exists), `uploader_denied`
     (the credential type is unsupported, or authentication, repository write
     access, or host permission policy refused the call), `no_configured_store` (no tracker or artifact
     destination is configured or reachable), or `upload_failed: reason` (an
     available uploader was exercised and failed; name the observed failure).
     State the remedy each one implies, and omit the field entirely when the

From shakacode/agent-workflows, MIT.

More from shakacode/agent-workflows