hsb-test

Files (6)

• BENCHMARK.md
• SKILL.md
• evals.json · evals/
• phase-details.md · references/
• skill-card.md
• skill.oms.sig

SKILL.md (first lines)

# HSB QA Test Runner

Use this skill when the user wants to execute a QA test plan against an HSB board and devkit. The skill reads a test document (local file or web link), filters tests to those that can run automatically on the user's specific hardware setup, executes each test with pass/fail evaluation, and produces a comprehensive results report.

This skill assumes the devkit is already set up (SSH, demo container built, host configured, board connected). If setup is not complete, it will offer to invoke `/hsb-setup` first.

This workflow runs test applications inside the demo container. Only run it when the user explicitly invokes it.

## Before you start — required gates (do these first, in order)

**Gate 1 — Read environment variables.** Before doing anything else, check these variables and print their resolved values to the user:

```
SSH_TARGET      Remote devkit login (e.g. nvidia@192.168.1.50). Ask the user if not set.
REMOTE_ROOT     Remote working directory (e.g. /home/nvidia). Ask the user if not set.
REMOTE_SUDO     sudo / sudo -n / "" — default to "sudo" if not set.
REMOTE_SSH_OPTS Additional SSH options (optional).
HSB_PLATFORM    Platform hint (optional).
```

**SSH_TARGET and REMOTE_ROOT are required. Stop and ask the user for them if either is missing.**

**Gate 2 — Present the phase plan, ask for the test document, and get confirmation.** Before taking any action:

1. Show the phase plan:

```
HSB Test — Phase Plan
  Phase 0: Verify devkit SSH, board ping, and demo container availability
  Phase 1: Obtain test document, confirm setup, build executable test plan
  Phase 2: Execute tests, record pass/fail, analyze failures
  Phase 3: Produce test results report (with option to save)
  Phase 4: Clean up test artifacts
```

2. **If the user has not provided a test document path or URL, STOP and ask for it — do not proceed to Phase 0 or any phase until the user provides it**: `Please provide the path or URL to your test document:`. If the user has already specified specific tests (e.g., "connectivity checks only"), state which phases will run and which will be skipped, and note that tests will be filtered to the user's platform/board/sensor configuration and classified as automatable vs. manual.

3. Ask explicitly: `Shall I proceed with Phase 0? [Y/n]` — do not start Phase 0 until the user confirms.

## What this skill must do

1. Verify that the devkit is reachable over SSH, the HSB board is connected and responsive, and the demo container is available. Read the current FPGA version and board identity. Verify the type of sensor/camera and hsb devkit and release repo used either from already set environment variables or from prompting the user. If the setup is not ready, offer to invoke `/hsb-setup` to prepare the devkit.
2. Obtain a test plan document from the user (file path or URL). Confirm the user's setup details collected in Phase 0 (repo location, HSB version, platform, board type, sensors). Study the test plan and the repository's `examples/` directory to determine which tests can run automatically. Skip manual tests and tests requiring additional equipment. Present the executable test plan for user approval.
3. Execute each test case in sequence. For each test: run the application, evaluate pass/fail against the criteria in the test plan, log the result. On failure, analyze logs, suggest fixes, and let the user decide how to proceed before running the next test.
4. Produce a structured test results report with per-test pass/fail status, issues encountered, and fixes applied. Offer to save the report.
5. Clean up all test artifacts (containers, temporary files, session state).

## Linux/Windows-friendly wrapper variables

Reuse the same environment variables from the other HSB skills:

- `SSH_TARGET` for the remote login target (e.g. `nvidia@agx-thor-host`)
- `REMOTE_ROOT` for the remote working directory
- `REMOTE_SUDO` for privileged commands
- `REMOTE_SSH_OPTS` for additional SSH options
- `HSB_PLATFORM` as an optional platform hint

If these are set, notify the user of these settings and use them without re-asking.

Before Phase 0, print the resolved remote execution settings.

## Mandatory interaction pattern

### First run in a session (no prior verification)

When no valid session state exists, show the full phase plan:

- Phase 0: Verify board connectivity, demo container readiness, and user setup (release repo, platform, sensor/camera)
- Phase 1: Obtain test plan, confirm setup, build executable test list
- Phase 2: Execute test plan with per-test pass/fail evaluation
- Phase 3: Generate test results report (with option to save)
- Phase 4: Cleanup

Then execute one phase at a time.

### Subsequent runs in the same session (fast path)

When the session state file (`/tmp/.claude_hsb_test_session/state.sh`) exists **and** contains `_SESSION_VERIFIED=true`, the skill skips Phase 0 and Phase 1 setup confirmation because connectivity, hardware, release repo, platform, and sensor/camera were already verified. Instead, inform the user and jump directly to test plan intake:

```
Session already verified — skipping connectivity checks.
  SSH target: $SSH_TARGET
  Release repo: /home/work/holoscan-sensor-bridge (HSB vX.X.X)
  Platform: AGX Thor
  Board: HSB Lattice | FPGA: XXXX
  Sensors: Dual IMX274

Proceeding directly to test plan intake.
```

Then execute:
- Phase 1 (test plan intake and test list building — setup confirmation is skipped)
- Phase 2: Execute test plan
- Phase 3: Test results report
- Phase 4: Cleanup

### When to re-run Phase 0 from the beginning

Phase 0 must be re-run (ignoring the fast path) when:

1. **New session**: No session state file exists on the remote host, or a new Claude Code session is started.
2. **Execution failure suggesting connectivity loss**: If Phase 2 fails with symptoms indicating the board or devkit is unreachable (ping failure, SSH timeout, container launch failure, `No such device` errors), clear `_SESSION_VERIFIED` from the session state and re-run Phase 0 before retrying.
3. **User explicitly requests it**: If the user says "re-verify", "start over", "run from the beginning", or invokes `/hsb-test --full`, run Phase 0 from scratch.

See [## Phase gate](#phase-gate--user-confirmation-between-phases) below for the full confirmation protocol.

If something fails, do **not** just dump raw logs. Summarize:

- the exact command that failed
- the likely root cause
- what safe action you recommend
- whether the issue is blocking

## Phase details

See [references/phase-details.md](references/phase-details.md) for full step-by-step phase instructions.

## Execution rules

From NVIDIA/skills, Apache-2.0.

More from NVIDIA/skills