agent-development

Files (1)

• SKILL.md

SKILL.md (first lines)

# Agent Development (cheap-token chat backend)

How to *develop* the NPA agent chat backend. For *operating* a deployed agent
(deploy/bootstrap/verify, chat UX, API grounding, Rerun) use
`skills/tools/npa-agent/SKILL.md`; for fresh deploy/teardown loops use
`skills/workflows/agent-fresh-operate/SKILL.md`.

Guiding principle: **the cheapest token is the one you never spend.** Protect and
widen the zero-token grounded path; make the unavoidable model calls small,
structured, and cheap-model-first.

## Architecture (three layers)

1. **Grounded intent router (zero tokens)** — `npa/src/npa/cli/agent_chat.py`.
   `match_chat_intent()` classifies a turn; `build_grounded_reply()` answers from
   live session state with **no model call** (`"grounded": true`). Most operator
   turns end here.
2. **Cost-tier routing (cheap model calls)** — `npa/src/npa/cli/agent_routing.py`.
   Only turns that fall through the router reach Token Factory, and this layer
   keeps them cheap.
3. **Token Factory client** — `npa/src/npa/clients/token_factory.py` (operator/SDK
   path) and the embedded `_provider_chat` / `_chat_with_resilience` in
   `npa/src/npa/cli/agent.py` (agent-VM path).

## The embedded-backend mechanism (critical)

The agent VM runs `backend.py`, which is built as one big f-string inside
`_bootstrap_agent_stack` in `npa/src/npa/cli/agent.py`. Pure helper modules are
inlined into it via placeholder substitution — the pattern to reuse when adding
logic:

- Real module (normal Python, **no** brace escaping): `agent_chat.py`,
  `agent_workflow.py`, `agent_routing.py`.
- Embed helper `_embedded_agent_<name>_source()` strips the docstring +
  `from __future__` line.
- Placeholder constant `_AGENT_<NAME>_EMBED` appears in the template and is
  replaced in the `.replace(...)` chain near the end of `_bootstrap_agent_stack`.

Rules:

- Put testable logic in a **real module** and embed it; do **not** write new
  logic directly inside the f-string unless it must touch template variables.
- Code written *inside* the f-string must escape literal braces as `{{` / `}}`
  and newlines in strings as `\\n`; substitutions use single `{var}`.
- After editing the template, **validate the rendered backend compiles** (see
  Testing). A stray brace is a `SyntaxError` at import of `agent.py`.
- **Embedded routes register before the template's own routes.** The
  `_AGENT_*_EMBED` placeholders sit earlier in the f-string than most `@app.*`
  blocks, and Starlette resolves the **first** matching route, so an embedded
  module silently wins over a same-path handler written further down in
  `agent.py`. `/artifacts/file/{filename}` and `/artifacts/download` each carried
  two definitions this way, and the shadowed copies in `agent.py` were the weaker
  ones — no `Content-Disposition`/`nosniff` headers, no run-scoped inventory
  authorization — so the file a maintainer would open described a contract the
  deployment did not serve. Before adding a route, grep the embedded modules for
  its path; `test_rendered_backend_registers_no_shadowed_routes` fails the build
  on any method+path registered twice.

## Cost-tier routing (`agent_routing.py`)

Pure, side-effect-free functions (no network) so they unit-test cheaply:

- `classify_tier(text, intent, messages)` → `cheap` (default) / `standard`
  (long/compound) / `reasoning` (analytical language) / `vision` (image content).
- `build_model_ladder(tier, configured, interactive, requested_model,
  allow_tier_defaults)` → cheapest-capable first; explicit user model wins;
  operator allowlist (`NPA_AGENT_LLM_MODELS`) respected when set.
- `flavor_variants` + `filter_available` — use only documented concrete model
  IDs and drop unavailable defaults. The current public replacements have no
  verified `-fast` variants; never invent one. Explicit selections are tried
  even when absent from the model list (dedicated endpoints can differ).
- `chat_extra` / `thinking_enabled` — disable hidden reasoning traces off the
  reasoning tier (don't pay for discarded tokens).
- `enforce_input_budget` — cap oversized pastes (head+tail preserved).
- `usage_summary` — surface per-turn token usage.

The `/chat` handler classifies the tier, enforces the input guardrail, honors an
explicit model override, and returns `tier` + `usage` + `input_budget_ok`.

## Adding a capability cheaply (decision order)

1. **Grounded intent** — can a regex intent + grounded state reply answer it?
   (0 tokens) Add to `_INTENT_RULES` / `build_grounded_reply` in `agent_chat.py`
   and cover it in `test_agent_chat.py`. Prefer this for anything high-frequency.
   Three invariants apply to every new or edited rule:
   - **Add an `INTENT_APIS` entry.** It is not only reply metadata:
     `_semantic_route` builds the semantic fallthrough's `known_intents` from its
     keys, so an intent missing from it is unreachable by any paraphrase the
     regex misses and its grounded replies report an empty `apis_used`.
     `test_every_intent_declares_its_apis` fails the build otherwise.
   - **Earlier rules win**, and `match_chat_intent` applies several hard-coded
     checks before the list at all. Adding a phrasing an earlier rule already
     claims silently changes nothing; check what currently matches first.
   - **Keep sibling rules symmetric.** The tool-capability rules are near
     copies, so a verb added to one belongs in all of them. "what can lancedb
     do" degraded to the generic component reply for exactly this reason while
     the identical Sonic/LeRobot/Genesis wording stayed tool-specific.
   A phrasing that matches nothing is not neutral — it falls through to a paid
   model call for an answer the grounded layer already had.
2. **Cheap model** — if it needs generation, let routing pick the cheap tier;
   only add reasoning/vision signals to `classify_tier` when the turn truly
   needs them.
3. **Escalate deliberately** — reserve `MiniMaxAI/MiniMax-M3` for
   analytical/physical-AI/vision turns; it is overkill for routine chat.
4. **Visual feedback** — UI **Describe this** captures the active viewer frame
   and posts multimodal `/api/chat` with `visual_context`. Helpers live in
   `agent_visual_feedback.py` (embedded). Never ground these turns; use vision
   when an image is attached. See `skills/atomic/agent-visual-feedback/SKILL.md`.

## Token Factory notes

- OpenAI-compatible: base `https://api.tokenfactory.nebius.com/v1/`, key
  `NEBIUS_TOKEN_FACTORY_KEY`. Same `chat/completions` shape everywhere.
- Public defaults follow the [August 2026 migration notice](https://docs.tokenfactory.nebius.com/august-2026-deprecation-notice):
  `nvidia/Nemotron-3_5-Lightning` for cheap/standard text,
  `MiniMaxAI/MiniMax-M3` for reasoning/vision. No default `-fast` IDs are added.
  Vision never falls back to the text-only Lightning default.
- Cost-ordered default ladder lives in `DEFAULT_LLM_MODELS` (cheap first). A bare
  `npa agent deploy` seeds this whole ladder, so per-turn routing reaches every

From nebius/nebius-physical-ai, Apache-2.0.

More from nebius/nebius-physical-ai