blind-tribunal

Files (1)

• SKILL.md

SKILL.md (first lines)

# Blind Tribunal
**Effort:** heavy — eight juror seats, one lens each, routed by tier to the cheapest sufficient model family and re-convened on fresh envelopes every round until unanimous; spend it on autonomous changes that land with no human review. Removes: rogue landings gated by nothing but the builder's own word.

The grading loop that lets the human walk away without the agent going rogue.
A panel of jurors reviews the change blind, with authorship stripped. Every
finding becomes a new failing test. The loop repeats until every juror passes.
Nothing lands on the builder's word alone.

## When to run it

- Before landing any autonomous change no human will review.
- Any high-blast-radius change: security-shaped, data-touching, authority-adjacent.
- When one grader is not enough and you want independent lenses on the same artifact.

## The seats

Eight jurors, one lens each. Each is a model from a DIFFERENT family than the builder
(same vendor = same family). A juror asked to check everything checks nothing well.

| Juror | Lens id | Tier | The question it asks |
| --- | --- | --- | --- |
| Defect | `defect` | generalist | What actually breaks? Logic flaws, syntax errors, new defects. |
| Proportion | `proportion` | generalist | Is this the right size? Over-engineered, or scaled to the intent? |
| Consequence | `operator_consequence` | operator safety | If a human operator runs this, what is destructive, unsafe, or harmful? |
| Reversibility | `reversibility` | deep state | Irreversible side effects? If it dies mid-run, can the system roll back cleanly? |
| Continuity | `state_continuity` | deep state | Orphaned variables, clobbered global state, dropped context downstream nodes need? |
| Economy | `resource_economy` | fast structural | Unoptimized loops, redundant network/API calls, memory bloat? |
| Boundary | `boundary_condition` | fast structural | Null, empty, wrong-typed, or malformed inputs — does it fail gracefully? |
| Telemetry | `telemetry` | operator safety | Can a failure here be diagnosed from the logs and error handling? |

## The routing tiers (cheapest sufficient route first)

Route each tier to what the lens needs, not to the biggest model you own:

- **deep state** (reversibility, state_continuity): your largest context + deepest
  reasoning, ideally through an agentic harness that can READ the repository (never
  write) so the juror can trace a state change from producer to consumer and run the
  named tests. Run harness jurors read-only (`codex exec --sandbox read-only`; a
  read-only tool catalog on a claude harness) and void any verdict whose dispatch
  changed repository bytes, naming the paths. Header: *trace every state change across the whole workflow before you judge.*
- **fast structural** (boundary_condition, resource_economy): the cheapest fast route —
  a free local GPU model first, **fused with a cheap cloud verifier** that judges the
  same prompt: the lens passes only when BOTH pass; the verifier never re-seats the
  primary's model; when every cloud rung is down the local verdict stands, flagged
  **UNVERIFIED** in the review and the summary — never silently "verified". A local
  model must see the WHOLE artifact or it does not judge: size the request's context
  window to the prompt (`num_ctx` on Ollama — the server default is 4096 tokens and it
  truncates silently; measured 2026-09-04, a 16B juror "passed" a 124 KB diff it never
  saw, citing a file that does not exist) and refuse, before sending, a prompt the rung
  cannot hold. The verifier never comes from the primary's FAMILY (same vendor =
  same family), and an UNVERIFIED seat is a HOLD — never unanimity, never a pack
  pass. After every local call read the node's own numbers: cap the answer
  (`num_predict`), refuse an answer whose prompt filled the window
  (`prompt_eval_count` + answer budget ≥ `num_ctx` is a measured truncation), and
  check residency (`size_vram == size` on `/api/ps`) — a model spilled to system RAM
  is unloaded and its answer discarded. Header: *work fast and literal from the code
  in front of you.*
- **operator safety** (operator_consequence, telemetry): your strongest coder with
  safety grounding. Header: *think as the human who runs this on their own machine.*
- **generalist** (defect, proportion): a reliable large generalist. Header: *precise
  findings, no invented context.*

Every tier's ladder ends on a local survival rung, so the tribunal still convenes with
zero cloud. The premium accounts an IDE session holds (Claude Opus/Sonnet, Codex) are an
IDE-only lane for the deep-state seats: render the prompt files, pipe them in, drop the
verdicts into the same shape.

**Solo rig.** When only one model family is available, degrade EXPLICITLY: a
fresh context or session that never saw the author's conversation acts as the
blind grader, or the human reviews the redacted envelope. The report must name
the weakened gate ("graded same-family-blind, not cross-family"), never
silently pretend the cross-family gate held.

## The builder is declared, and the exclusion is structural

"Different family than the builder" was a rule jurors were asked to remember. In the
tribunal's own dogfood the operator-safety seat led with the very model that had built
the candidate, and nothing recorded or excluded it — the author graded its own work for
two rounds and every builder-is-not-grader claim on those rounds was unfounded. So:

- **Convene with the builder named** (`--builder <model-or-family>`). The run record
  carries `builder_family`. Every rung of that family is refused out loud, before any
  dispatch, on every ladder. A lens left with no rung HOLDS — it never falls back to the
  builder because the builder was the only seat left.
- **Same vendor = same family** (`gpt-6-astra`, `gpt-5.6-sol`, `gpt-5.5` are one family).
  One declaration excludes the vendor.
- **Prove it on the live ladder, not in a test:** after the first convene with the builder
  declared, the routing table must show the builder's seats fell through to another
  family. If it did not, the exclusion is decoration.

## The footer names the lens, a rejected rung keeps its words, and the floor is three rungs deep

Round 4 held two lenses with zero refusals, and every link was on the record. Three laws came out of it:

- **State the answer shape next to the answer.** The protocol footer carries the literal lens name (`"lens": "defect"`), never the `<your lens>` placeholder. A juror asked to recall the lens from 350 KB earlier, inside an artifact that names all eight lenses, answered the wrong lens three times in two rounds. Fill the placeholder at render time.
- **A rejected rung leaves its words on the record.** A wrong-lens answer or a voided verdict carries a bounded `raw_tail` on the rejected entry, so the next round reads the cause instead of inferring it.
- **Two cloud rungs is not a floor.** Every tier holds at least three rungs with no declared `context_tokens` (they can carry a 120k-token artifact) before its local tail. One wrong lens plus one void must never hold a lens.
- **A structured verdict never shares its budget with thinking.** A reasoning model asked for a bare JSON verdict spent its whole 65536-token budget thinking about a 131k-token artifact and emitted nothing (`finish_reason=length`); the role's time ceiling then killed the next rung mid-walk. Every cloud rung that asks for a JSON-object verdict runs with its reasoning channel off (`reasoning_effort: none`), and the verifier ladder keeps a plain-HTTP third family behind it.
- **Nothing else writes into the tribunal's repo while it convenes.** A concurrent grader's status file inside the checkout changed bytes under a seat, and the organ voided that verdict honestly: it cannot attribute a change. Serialize writers, or convene on a separate worktree of the same commit.

## The envelope

Jurors never see the builder or the conversation. They see one envelope:

- **Whole current files** for every file the change touched, plus its test files.
  Never bare diff hunks — a hunk hides the surrounding contract and induces false findings.
- **The review contract**: the change's intent in one line, and the pass criteria.
- **Zero authorship.** No names, no model ids, no commit authors, no chat history.
  If identity leaks through, the envelope build fails loud — never grade un-blind.
- **No prose about the old behavior.** Describing what the code "used to do" plants
  phantom defects. The files speak for themselves.
- **Prior adjudications ride along.** A finding refuted with evidence in an earlier
  round is appended as `PRIOR_ADJUDICATIONS`; jurors judge it honestly. Never re-seat
  a juror to launder a refusal — fix the envelope or the code.

## Rendering the eight prompts (portable)

One prompt per lens: `<tier header>` + `<lens preamble>` + `<envelope>` + `<output protocol>`.
Any runtime can do it with a dozen lines; keep the lens text in ONE file your tests read.

From Tcuzzo/HydraAgent_public, MIT.

More from Tcuzzo/HydraAgent_public