minions-llm-proxy-ops

Files (2)

• SKILL.md
• opencode-free-tier.md · references/

SKILL.md (first lines)

# .minions LLM proxy stack ops

## When to Use

- A model or combo through 9Router (:7352) or OmniRoute (:20128) errors,
  403s, rate-limits, or returns empty content.
- opencode.ai free tier (`oc/` models) breaks — FreeTierError, "Model is
  unavailable", or silent empty responses.
- The `auto-fastest` combo needs its model list refreshed or reordered.
- You need to change 9Router provider headers, combos, or settings so they
  survive reboots (i.e. via `lib/9router.sh`).

The stack (boot.sh) runs OmniRoute (:20128) and 9Router (:7352). 9Router routes
`auto-fastest` (a combo) to free OpenCode (`oc/`) models; OmniRoute is the
upstream LLM proxy for Hermes. `lib/9router.sh` (`ninerouter_preconfigure`)
re-applies 9Router config idempotently on every boot — edit it there so config
survives restarts, and use its API calls as the reference for ad-hoc changes.

## Always-on rules

- **Verify a proxy change with real completions, never HTTP status alone.**
  9Router's combo fallback counts an empty `finish_reason:"in_progress"` (or
  `finish_length` with 0 visible tokens) response as SUCCESS, so a combo whose
  first model is silently broken returns empty content forever without ever
  falling back. Pass signal = non-empty `choices[0].message.content`; empty
  content is a failure regardless of `finish_reason` (free models sometimes
  return correct content with `finish_reason:"length"`, so don't require
  `stop`). A 2xx with empty content is a failure.
- **Lead a combo with the known-good model.** Since fallback only fires on
  hard errors, put the model that actually produces content FIRST in the combo
  model list; put flaky/unverified models after.
- **Free-tier model lists rotate.** Query the live list before trusting any
  hardcoded model id: `curl -s https://opencode.ai/zen/v1/models` (only ids
  ending `-free` / `-contributor-free` are actually free). A stale id either
  hard-errors or silently returns empty — both degrade the combo.
- **9Router dashboard login**: default password `123456`; cookie session via
  `POST /api/auth/login`. All admin APIs need the cookie (`-b "$COOKIE"`).
  `requireLogin`/`requireApiKey` are disabled by preconfig.
- **9Router combo model list lives in `etc/9router-models.json` (single
  source). 9Router only — `lib/omniroute.sh` is NOT wired to it.**
  `lib/9router.sh` reads that file at boot — never duplicate the list inline
  in a shell script. `tests/test_9router_models.sh` (blocking CI) is an OFFLINE
  structural guard; `scripts/sync-9router-models.sh` owns upstream drift
  (`--check`/`--write`) and live probing (`--probe`, `--probe-write` need a
  running 9Router on :7352: keep 200/429, drop 400/410).

- **Config lives in 9Router's own DB, keyed by canonical provider id.**
  `PUT /api/providers/<id>/overrides` REPLACES the whole header map for that
  provider (idempotent; send the full map each time). `POST /api/combos` +
  DELETE-by-id is how combos are recreated (there is no PUT on combos).

## Standard procedure: audit/fix a provider's routing

1. Log in and save a cookie:
   `curl -s -c /tmp/nr.txt -X POST $BASE/api/auth/login -H 'Content-Type: application/json' -d '{"password":"123456"}'`
2. Inspect current state: `GET /api/combos`, `GET /api/settings`,
   `GET /api/providers/<id>/overrides` (shows `headers` + `builtinHeaders`).
3. If upstream 403s / rejects: set header overrides (see
   `references/opencode-free-tier.md` for the OpenCode-specific header set and
   session-id generator).
4. Fix the combo model list against the LIVE free list (step above), known-good
   model first.
5. Test with a real prompt at >=128 max_tokens (small budgets get eaten by
   reasoning tokens and look like empty failures):
   `POST /v1/chat/completions {"model":"auto-fastest","messages":[...],"stream":false,"max_tokens":128}`
   Assert non-empty content. To validate a combo's model list, test each
   member individually the same way — bare ids route directly
   (`model=oc/<id>`), so you can run them in parallel and prune dead ids
   before recreating the combo. If a model errors, read
   `~/.minions/var/log/9router.log` — it logs the upstream URL, status, and
   provider-side error body.
6. Any change must land in `lib/9router.sh` or it is lost: the boot
   preconfig deletes and recreates combos (and re-puts overrides) every boot,
   so live-only edits — dashboard or ad-hoc API — silently revert on next
   boot. Edit the script, then let boot apply it, or apply + edit together.

## Add an OpenAI-compatible provider (OmniRoute :20128), wire a model into `auto-fastest`

`scripts/add-tailscale-provider.sh <host[:port]>` — lives in the skill's
`scripts/` dir (this repo: `skills/minions-llm-proxy-ops/scripts/`; standalone
install: `~/.minions/skills/minions-llm-proxy-ops/scripts/`). Idempotent,
`--dry-run` supported; `TSC_PORT` sets the port for a bare host (default
`11434`); `MODEL_NAME`/`PROVIDER_NAME` env, model defaults to `qwen3.8:27b`.
The five REST steps below are the verified sequence (OmniRoute 3.8.53) and how
the script works — use them as the manual/API reference for any new upstream
that only needs a base URL:

1. **Create the NODE** — `POST /api/provider-nodes` with
   `{name, type:"openai-compatible", baseUrl:"http://<host:port>/v1",
   prefix:name, apiType:"chat"}`. `baseUrl` MUST end in `/v1` (the base for the
   OpenAI-compatible endpoints) and OMIT `chatPath`/`modelsPath` entirely —
   the server stores them `null` (its own default), which is the correct shape.
   The **server generates** the concrete id `openai-compatible-chat-<uuid>` —
   that uuid IS the provider id used by `/api/providers` and combos. **Pitfalls
   (verified against OmniRoute 3.8.53):** explicit `null` → `400 Invalid input`
   (so omit the keys, don't null them); non-null strings like
   `"/v1/chat/completions"` → `201` but **wrong** — they double the `/v1`
   `baseUrl` already carries, producing `.../v1/v1/...`. If you ever DO set
   them, use bare `"/chat/completions"` or `"/models"` (no `/v1`).
2. **Create the CONNECTION** — `POST /api/providers` with
   `{provider:"<node-id>", name, apiKey:"<any non-empty string>", priority,
   providerSpecificData:{prefix, apiType:"chat", baseUrl, nodeName:name}}`.
   **Pitfall:** a keyless connection is rejected (`"API key is required"` /
   `"does not allow a connection without a credential"`). The key value is
   ignored by the upstream; a dummy credential is fine. `omniroute providers
   add openai-compatible …` fails with `Invalid provider` because the CLI will
   not accept the abstract type — the node must exist first and the concrete
   id must be passed.
3. **Activate** — `PUT /api/providers/<cid> {"isActive":true}`. The connection
   is created `isActive:false`; activation returns `testStatus:"active"`
   immediately when the upstream is reachable.
4. **Import the model** — `POST /api/provider-models {provider:"<node-id>",
   modelId}`. The alternate `validate-and-add` endpoint requires a real
   `Authorization` header on this deployment and 401s — use the plain one.
5. **Append to `auto-fastest`** — `GET /api/combos`, find `auto-fastest`, add
   `{id:"<combo>-model-<name>-<model>", kind:"model", model:"<name>/<model>",
   providerId:"<node-id>", weight:0}` to `models[]`, then `PUT /api/combos/<id>`
   with the FULL body `{name, strategy, enabled, config, models}`. **Pitfall:**
   `PUT` rejects a body that has only `models` (`COMBO_001 Invalid JSON body`).

From gitricko/.minions, MIT.

More from gitricko/.minions