llm-judgment-layer

Files (1)

• SKILL.md

SKILL.md (first lines)

# LLM Judgment Layer

Seasoning, not the meal. The app is 100% deterministic and works offline;
the model answers a few narrow questions per time-bucket and code keeps
control of everything else.

## HARD-GATE

- The model **never** decides policy. It ranks, scores, or answers yes/no;
  code sorts, clamps, thresholds, and chooses.
- The UI **never** waits on the network. Render the cached/default plan
  first; update in place if the model answers.
- Every failure path lands **silently** on the deterministic fallback.
  No error copy, no retry storms, no dead UI.
- The API key **never** ships in the client. Server-side proxy adds it;
  unset proxy URL means the layer is inert by design.

## Core loop

1. **State** — aggregate statistics only, composed in the model's primary
   language. No names, dates, identifiers, or free text.
2. **One batched request per time-bucket** (usually per local day), fired
   lazily on first need, cached by bucket stamp. All questions ride in one
   call against the same state.
3. **Parse per question** — a half-good response still contributes its
   good answers; failed questions fall back individually.
4. **Gate on confidence** — apply a model answer only above a threshold
   set by the stakes of being wrong. Below threshold: keep previous value.
5. **Decide in code** — sort, clamp, threshold. Guarantee a valid result
   even when every probability is low.

## Question primitives

| Primitive | Returns | Use for |
|-----------|---------|---------|
| Choice | one winner + probabilities | exactly-one selection |
| Noul (P(yes) per item) | independent probabilities | multi-select mixes — rank in code, take top N |
| Score (ordered levels) | float over a legend | graded settings like difficulty — round, clamp, gate |

Multi-select is the classic trap: a single Choice forces one winner;
independent Nouls give a per-item probability you can rank yourself.

## Timing

Batch per time-bucket, not per event. Difficulty that shifts mid-session
feels erratic; per-event calls multiply cost for no benefit at fine
granularity. New users start at a safe default, never at a model guess.

## Failure taxonomy (all → silent fallback)

No network or timeout · non-2xx · malformed or missing answers ·
low confidence · no proxy configured. The fallback is a first-class
feature: deterministic rotation, keep-yesterday's value, safe defaults.
Test the fallback path as heavily as the happy path.

## Privacy

The state object is the entire data contract: counts, ratios, recency.
If a field could identify a person, it does not belong. The proxy adds
the key server-side and forwards verbatim — no rewriting, no logging.

## Anti-patterns

- Letting the model's raw answer reach the UI ungated
- Error copy when the model fails (the user did nothing wrong)
- Per-event requests where a daily cache suffices
- PII or locale leaking into the state object
- Ignoring `confidence` — it's a separate axis from the answer itself

## Integration

| Skill | When to chain |
|-------|---------------|
| `judgment-gate` | Apply the same discipline to planning decisions |
| `work-router` with `tech-lead` intent | Feasibility review of the proposed integration from the active profile |

From igmarin/agnostic-planning-skills, MIT.

More from igmarin/agnostic-planning-skills