localmaxxing

Files (1)

• SKILL.md

SKILL.md (first lines)

# LocalMaxxing API

Public leaderboard for local LLM throughput + quality evals. Two surfaces:
- **Benchmarks** — tok/s, TTFT, VRAM, hardware
- **Evals** — task-level quality scores against named suites (register your own — see Eval section)

Benchmarks **auto-approve**; **eval runs land PENDING → admin approval**. Rate limit is **30 submissions per rolling hour per API key (300 for Pro)** — benchmarks and evals have **separate** windows. Read `X-RateLimit-Remaining`/`-Reset` headers to self-throttle. (verified 2026-06-25)

## Authentication

Bearer token, prefixed `bhk_` (40 hex). Load before every call:

```bash
export $(grep LOCALMAXXING_API_KEY /home/mikekey/.env | xargs)
```

If the env var isn't in `~/.env` yet, get a key from a logged-in browser session:

```js
// In devtools console at https://www.localmaxxing.com (signed in via HF or GitHub)
await fetch('/api/keys', {method:'POST', headers:{'Content-Type':'application/json'},
  body:'{"name":"archbox"}'}).then(r=>r.json())
// Raw bhk_… token returned ONCE — copy immediately into /home/mikekey/.env
```

There is no UI page for keys — `/api/keys` is the only path.

## Base URL

`https://www.localmaxxing.com`

## Bootstrap context (live)

Before doing anything non-trivial, pull the canonical agent context — it's authoritative and updated server-side:

```bash
curl -s https://www.localmaxxing.com/api/agent-context | jq
```

It returns: required/optional fields, hardware schemas, full `engineFlagsSchema` + `commandParsing` (which CLI flag → which DB field), rate-limit headers, examples for vllm/sglang/Apple-silicon/heterogeneous-GPU/spec-decoding, and the eval submission shape. Treat it as the spec — this SKILL.md is the cheat sheet.

## Endpoints

### Public (no auth)
| Path | Method | Purpose |
|---|---|---|
| `/api/agent-context` | GET | Full machine-readable spec (use this first) |
| `/api/benchmarks` | GET | Approved benchmark results, with filters (`limit`, `offset`, model, hwClass…) |
| `/api/leaderboard` | GET | Ranked by tok/s output — returns model, hardware, engine, **engineFlags**, reactions |
| `/api/models` | GET | Browse models |
| `/api/models/search?q=<query>` | GET | Fuzzy resolve human name → canonical `hfId` (use before submitting) |
| `/api/evals/suites` | GET | List approved suites |
| `/api/evals/suites/{slug}` | GET | Suite definition: tasks, scoring, runConfig, suiteDoc |
| `/api/evals/runs?modelId=…` | GET | Best approved eval run per suite for a model |

### Authenticated
| Path | Method | Purpose |
|---|---|---|
| `/api/benchmarks/dry-run` | POST | Validate without persisting (no rate-limit hit) |
| `/api/benchmarks` | POST | Submit benchmark (rate-limited) |
| `/api/evals/runs/dry-run` | POST | Validate eval payload (no rate-limit hit) |
| `/api/evals/runs` | POST | Submit eval results (rate-limited) |
| `/api/evals/execute` | POST | Server-side execution against your public OpenAI-compat endpoint |
| `/api/evals/suites` | POST | Register custom suite (pending admin approval) |
| `/api/runs/{id}/react` | POST | React to a run with an emoji (or `null` to remove) |
| `/api/setups` | GET/POST | Saved hardware presets (session-only, used by submit page) |
| `/api/keys` | GET/POST | List/create API keys (browser session only) |

## Querying the leaderboard

Leaderboard rows include `model`, `hardware`, `engine`, and parsed `engineFlags` (commandSnippet, tensorParallel, gpuLayers, kvCacheDtype, flashAttn, specDecoding, mtpEnabled, …). Filter client-side with `jq`:

```bash
# Top runs for a model
curl -s "https://www.localmaxxing.com/api/leaderboard?limit=50" \
  | jq '.rows[] | select(.model.hfId=="Qwen/Qwen3-8B")'

# All Vulkan llama.cpp runs sorted by tok/s out
curl -s "https://www.localmaxxing.com/api/leaderboard?limit=200" \
  | jq '[.rows[] | select(.engine.backend=="vulkan" and .engine.engineName=="llama.cpp")] | sort_by(-.tokSOut)'

# Multi-GPU AMD setups using flash-attn
curl -s "https://www.localmaxxing.com/api/leaderboard?limit=200" \
  | jq '.rows[] | select(.hardware.gpuName | test("AMD")) | select(.hardware.gpuCount > 1) | select(.engineFlags.flashAttn==true)'

# Pull the exact commandSnippet of the #1 run for a model — useful as a tuning reference
curl -s "https://www.localmaxxing.com/api/leaderboard?limit=200" \
  | jq -r '[.rows[] | select(.model.hfId=="Qwen/Qwen3.5-0.8B-Base")] | sort_by(-.tokSOut) | .[0].engineFlags.commandSnippet'
```

`/api/benchmarks` accepts query filters (`limit`, `offset`, model, hwClass, etc.) for narrower pulls.

## Resolving fuzzy model names

**Do NOT rely on `/api/models/search` — it returns empty results unreliably.** Instead:

1. **Check Mike's existing submissions first** — hfId is in the response and the alias→hfId mapping is already solved:
```bash
export $(grep LOCALMAXXING_API_KEY /home/mikekey/.env | xargs)
curl -s "https://www.localmaxxing.com/api/benchmarks?limit=100" \
  | jq '[.benchmarks[] | select(.user.username == "1337Hero")] | .[] | {alias: "?", hfId: .model.hfId, quant: .engine.quantization, tokSOut, status}'
```

2. **Check the leaderboard** for any hfId that matches the model family:
```bash
curl -s "https://www.localmaxxing.com/api/leaderboard?limit=200" | jq '[.rows[] | select(.model.hfId | test("(?i)qwen"; "i"))] | .[] | .model.hfId' | sort -u
```

3. **Known alias → hfId map** (llama-swap aliases don't match hfIds — use this table):

| llama-swap alias | hfId | quant |
|---|---|---|
| `Qwen-Coder-30B` / `Qwen-Coder-30B-fast` / `Qwen-Coder-30B-rocm` | `Qwen/Qwen3-Coder-30B-A3B-Instruct` | Q8_0 |
| `Qwen3-Coder-Next` | `Qwen/Qwen3-Coder-Next` | MXFP4_MOE |
| `Qwen3.5-122B` | `Qwen/Qwen3.5-122B-A10B` | MXFP4_MOE |
| `Qwen3.6` / `Qwen3.6-fast` | `Qwen/Qwen3.6-35B-A3B` | Q8_0 |
| `Qwen3.6-27B` / `Qwen3.6-27B-fast` | `Qwen/Qwen3.6-27B` | Q8_0 |
| `GLM-4.7-Flash` | `zai-org/GLM-4.7-Flash` | MXFP4_MOE (vLLM) or Q8_0 (gguf) |
| `GPT-OSS` | `openai/gpt-oss-20b` | Q8_0 |

From 1337hero/claude-toolkit, WTFPL.

More from 1337hero/claude-toolkit