amc-run-rtsp-calibration

Files (6)

• BENCHMARK.md
• SKILL.md
• evals.json · evals/
• run_rtsp_calibration.py · scripts/
• skill-card.md
• skill.oms.sig

SKILL.md (first lines)

# Skill: Calibrate from RTSP Streams

## When to Use This Skill

Activate this skill when the user wants to calibrate from live RTSP camera streams. Typical prompts:

- "calibrate RTSP streams" / "calibrate from live cameras"
- "run AMC on RTSP"
- The user provides one or more `rtsp://...` URLs

VIOS records fixed-duration clips from each stream, the AMC microservice ingests those clips into a project, then the workflow follows the same verification, calibration, polling, and results path as pre-recorded MP4 calibration.

Do not use this skill for local MP4 files already on disk; route those requests to `skills/amc-run-video-calibration/SKILL.md`. Do not use it for the bundled sample dataset; route that to `skills/amc-run-sample-calibration/SKILL.md`.

Never reuse files from the bundled sample dataset, extracted sample zip, `assets/`, or previous projects for RTSP calibration unless the user explicitly provides those paths for this RTSP scene. Similar camera names, stream counts, or `cam_00`/`cam_01` ordering are not evidence that sample alignment, layout, GT, or detector settings apply.

## Prerequisites

- [ ] AMC microservice and UI running (follow `skills/amc-setup-calibration-stack/SKILL.md` if needed).
- [ ] VIOS is running and reachable from the AMC microservice.
- [ ] `VIOS_BASE_URL` is configured in the AMC microservice environment before capture starts.
- [ ] RTSP URLs are reachable from the VIOS host.
- [ ] Camera streams have enough moving people/objects for calibration; record at least 2-3 minutes when possible.
- [ ] Python 3 with `requests` installed when using the bundled script.

## Data Privacy

RTSP URLs may contain usernames, passwords, hostnames, or network topology. Do not print full RTSP URLs if credentials are embedded. This skill does not handle bearer credentials; if the VIOS deployment requires authentication, stop and hand the user to a manual admin-managed workflow instead of collecting or relaying secrets in chat, scripts, or logs.

## What to Ask the User

### Required

1. RTSP URLs, one per camera.
2. Camera names, one per stream. Use `cam_00`, `cam_01`, ... if the user does not provide names.
3. Recording duration in seconds. Minimum is `60`; prefer `120`-`180` or more when the scene has sparse motion.
4. Microservice URL, for example `http://<HOST_IP>:8000` or `http://<HOST_IP>:8000/v1`.
5. Project name.
6. Calibration asset source for this RTSP scene:
   - a local directory to scan, such as `/data/my_rtsp_calib/`;
   - explicit paths to settings, alignment, layout, and optional GT files; or
   - confirmation that the user will upload/tune settings and alignment in the AMC UI.

If the user does not provide a local asset source, stop and ask whether they want to provide a path or use UI upload. Give the UI link as `http://<HOST_IP>:<AUTO_MAGIC_CALIB_UI_PORT>`; the default UI port is `5000`.

### Auto-Detected or Asked

RTSP clips are recorded by VIOS, so there is no local videos directory to anchor file discovery. Only scan a directory the user explicitly provided for this RTSP scene. If the user provides a settings file path, use that file's directory as the scan directory. If the user provides a calibration asset directory, scan only that directory. Otherwise ask this question before planning uploads or calibration:

> Do you have a local calibration asset directory or settings file for these RTSP streams, or should you upload/tune settings and alignment in the AMC UI at `http://<HOST_IP>:<AUTO_MAGIC_CALIB_UI_PORT>`?

| File | Candidate filenames | UI fallback |
|---|---|---|
| Calibration settings | Explicit user path, or `settings.json`, `config.json`, or `calibration_config.json` in the user-provided asset directory | UI Step 3: Parameters |
| Alignment JSON | Explicit user path, or `alignment_data.json` in the user-provided asset directory/settings directory | UI Step 4: Alignment |
| Layout PNG | Explicit user path, or `layout.png` in the user-provided asset directory/settings directory | UI Step 4: Alignment |
| Ground truth zip | Optional explicit user path, or `GT.zip`/`gt.zip` in the user-provided asset directory | Omit metrics |

Posting the settings file replaces UI Step 3 and may pin `detector` or `detector_type`. If it pins `resnet` or `transformer`, pass that same detector to `/calibrate`. If no settings file pins a detector, ask the user which detector to use; do not silently default to `resnet`.

### Optional

6. `sensor_id` per stream if the cameras are already registered in VIOS. Leave unset for auto-registration.
7. Ground truth zip (`GT.zip`) for evaluation metrics.
8. Focal lengths, one per camera.
10. Whether to run VGGT refinement after AMC completes, only when the project reports `vggt_state == "READY"`.

## Instructions

The bundled script in [scripts/run_rtsp_calibration.py](scripts/run_rtsp_calibration.py) implements this sequence end to end. Use the prose below for decisions, UI fallback, and troubleshooting.

### Step 0 - Verify AMC and VIOS

Confirm the AMC microservice is reachable:

```bash
curl -sf http://<HOST_IP>:<MS_PORT>/v1/ready
```

Confirm VIOS is reachable before starting capture. Probe in this order and stop at the first working URL:

```bash
: "${REPO_ROOT:?set REPO_ROOT to the auto-magic-calib checkout. Run amc-setup-calibration-stack Step 0b first.}"
grep -q "AutoMagicCalib" "$REPO_ROOT/README.md" 2>/dev/null && grep -q "auto-magic-calib-ms" "$REPO_ROOT/compose/ms/compose.yml" 2>/dev/null || { echo "ERROR: REPO_ROOT is not an auto-magic-calib checkout: $REPO_ROOT" >&2; exit 1; }
VIOS_BASE_URL=""

# Default local VIOS port.
if curl -sf http://localhost:30888/vst/api/v1/sensor/list >/dev/null 2>&1; then
  VIOS_BASE_URL="http://localhost:30888"
  echo "VIOS detected at $VIOS_BASE_URL"
fi

# Running AMC microservice container environment.
if [ -z "$VIOS_BASE_URL" ]; then
  VIOS_BASE_URL=$(docker exec auto-magic-calib-ms-1 printenv VIOS_BASE_URL 2>/dev/null)
fi

if [ -n "$VIOS_BASE_URL" ]; then
  curl -sf "${VIOS_BASE_URL}/vst/api/v1/sensor/list" >/dev/null \
    && echo "VIOS up at $VIOS_BASE_URL" \
    || { echo "VIOS_BASE_URL=$VIOS_BASE_URL is set but not responding"; VIOS_BASE_URL=""; }
fi

[ -n "$VIOS_BASE_URL" ] || {
  echo "VIOS is not reachable. Export VIOS_BASE_URL=http://<VIOS_HOST>:30888 and relaunch the AMC microservice." >&2
  exit 1
}
```

If VIOS is not reachable, ask the user to deploy VIOS and provide the base URL. Do not start RTSP capture until `${VIOS_BASE_URL}/vst/api/v1/sensor/list` returns 200.

If VIOS is reachable but the AMC microservice is missing `VIOS_BASE_URL`, do not edit checked-in compose files. Export the variable and relaunch the microservice with a temporary compose override:

```bash
cd "$REPO_ROOT/compose"
export VIOS_BASE_URL="http://<VIOS_HOST>:30888"
OVERRIDE_FILE="${TMPDIR:-/tmp}/amc-vios.override.yml"
cat > "$OVERRIDE_FILE" <<'YAML'
services:

From NVIDIA/skills, Apache-2.0.

More from NVIDIA/skills