health-preflight

Files (1)

• SKILL.md

SKILL.md (first lines)

# Health preflight (the doctor before you spend)

There is no `npa doctor`. The readiness surface is `npa workbench health`, and it
has exactly two public commands. Run them before anything that costs GPU time or
provisions cloud resources.

```bash
npa workbench health preflight --json
npa workbench health preflight --checks nebius --json
npa workbench health access --json
```

All three exit non-zero on any FAIL. That is the point: they are gates, not reports.
Use `--warn-only` only when you deliberately want the report without the gate
(for example while collecting a status snapshot), never to make a red run green.

## What each command answers

`preflight` answers *"do I hold the credentials nearly every workbench tool
needs?"* — one PASS/WARN/FAIL/SKIP row per check over Hugging Face, NGC, S3, and
Token Factory. Request the optional `nebius` check before provisioning:

```bash
npa workbench health preflight --checks all
npa workbench health preflight --checks nebius --json
npa workbench health preflight --checks s3,token_factory
npa workbench health preflight --project <alias> --checks s3,nebius
npa workbench health preflight --checks all --offline  # presence only; Nebius is SKIP
```

Valid `--checks` values are `all`, `hf`, `ngc`, `s3`, `token_factory`,
`encord`, `nebius`. The default remains `hf,ngc,s3,token_factory`, so hosted-inference
work does not require a Nebius Cloud profile. Explicit `all` includes `encord` and `nebius`.
Empty selections and unknown check names are errors, including unknown names
combined with `all`. Repeated names run once.

Select `--project <alias>` (or `-p`) when checking a configured project's S3
storage. This reads the project's bucket, endpoint, and credential pair through
the project storage resolver, without adopting host credential files or shell S3 settings, or
changing saved configuration. A complete exact-project record wins over ambient
S3 settings. Unknown projects and missing or deselected project storage produce
a JSON-compatible FAIL and a nonzero exit, including offline; `--warn-only`
preserves that FAIL while returning zero. The flag scopes only the S3 check;
other checks and Nebius auth profile selection are unchanged.

The S3 probe issues one `ListObjectsV2` request with `MaxKeys=1`. An empty bucket
or prefix is valid. It discards object names and continuation tokens instead of
enumerating checkpoint directories, so a large dataset does not increase the
number of requests. Listing permission still does not prove write access.

Online `nebius` runs the selected Nebius CLI profile through `iam whoami` and
`iam get-access-token` with browser launch and update checks disabled. It
discards both commands' output and removes ambient `NEBIUS_IAM_TOKEN` and
`NEBIUS_IAM_TOKEN_FILE` values so a stale token cannot mask profile readiness.
PASS requires both calls to succeed. In offline mode it returns SKIP because a
local profile file is not proof of usable authentication.

A failed Nebius identity probe does not establish that a token expired. Profile
configuration may be missing, or access or connectivity may have failed. For
diagnosis or authorized human login on an operator VM, see
`skills/atomic/vm-nebius-auth/SKILL.md`. A readiness-only request does not
authorize login or credential changes. Authentication PASS also leaves project
permissions, quota, and capacity unverified.

Profile selection uses `NPA_NEBIUS_PROFILE`, then `NEBIUS_PROFILE`, then the
CLI's active/default profile. The project selector does not choose an auth
profile. PASS proves authentication; permission to provision a particular
resource still depends on the selected identity's access to that project.

Online `hf` authenticates against Hugging Face `whoami-v2`; public repository
metadata is not sufficient. Online `ngc` performs a registry token exchange;
that proves the key, not entitlement to every NGC artifact. Online `encord`
authenticates and performs the cheapest read-only storage-folder listing.
`access` performs the capability-specific repository/artifact probe.

`access` answers the different and more specific question *"is my token actually
entitled to fetch bytes from the gated assets this capability pulls?"* It probes
a representative payload path at the exact catalog revision with HEAD or a
one-byte Range request. Metadata access is not proof. Holding an HF token is not
the same as having account entitlement, and `Ready` is technical fetch evidence,
not proof of legal acceptance.

```bash
npa workbench health access --capability all
npa workbench health access --capability cosmos3,groot
```

Capabilities: `all`, `cosmos`, `cosmos3`, `cosmos3-serving`, `groot`, `lerobot`,
`nurec`, `paidf`, `sim2real`, `sonic`, `token_factory`, `vlm_eval`.

For anything still gated, `access` prints the exact "Agree and access
repository" URL. **Hugging Face gated licenses must be accepted interactively on
the model page.** There is no API that accepts them for you, so no amount of
retrying or re-tokenizing will clear a gate. Open the printed URL, accept, then
re-run. `scripts/accept-model-access.sh` collects the URLs for a batch.

## Ordering: preflight before provisioning, not after

The recurring cold-start failure this prevents is a mid-run stop after you have
already paid for a cluster. Run the checks in this order:

1. `npa configure --show`: confirm the project stanza, bucket, and endpoint you
   think you are using are the ones on disk.
2. `npa workbench health preflight` — credentials exist and authenticate.
3. `npa workbench health preflight --checks nebius`: the selected Nebius CLI
   profile can resolve identity and mint an IAM token before provisioning.
4. `npa workbench health access --capability <the one you will run>`: gated
   model entitlements for that capability only; `all` is slower and reports
   failures you do not care about today.
5. Only then `npa provision-if-absent`, `npa cluster up`, or
   `npa workbench workflow submit`.

Image pullability is a *separate* gate that health does not cover; see
`npa workbench workflow preflight-images <spec.yaml>` in
`skills/atomic/submit-workflow/SKILL.md`. A green health report with an
unpullable image still hangs in `ImagePullBackOff`.

**A green `s3` row does not mean the submit is execution-ready.** The generic
`s3` check lists its configured bucket. Submit resolves one effective target,

From nebius/nebius-physical-ai, Apache-2.0.

More from nebius/nebius-physical-ai