web-consensus

Files (3)

• .skill-check
• SKILL.md
• evals.toml

SKILL.md (first lines)

# Web consensus

Cheap researchers gather evidence in parallel, a reviewer from another model family attacks it, the disputed points are researched again from scratch, a script checks every link and quote, and a judge on the strongest model writes the verdict from sources only.

## Who does what

| Step | Subagent role | Runs on |
|---|---|---|
| Evidence, cold verification | `explorer`, one per angle or question, in parallel | `bulk` tier: fast and cheap |
| Challenge | `reviewer` | `review` tier: another model family |
| Link and quote check | `code-jig check-sources` | no model |
| Verdict | `judge` | `deep` tier: the strongest model |

This skill asks for subagents: every role in the table runs through `spawn_agent` with that `agent_type`, even when you wait for its result before the next step, because the point is a different model and a fresh context, not speed. Each subagent gets its own brief and nothing about the others, so no role can quietly agree with another, and never a fork of this conversation. You only route: write the evidence files, run the script, pass material between roles. Run the whole flow even when the answer looks obvious or one page settles it: "it's a simple fact", "the docs already say it" and "one search is enough" are the answers from memory this skill replaces. If `spawn_agent` is not available in this session, stop and say so: a consensus from one model playing every role is not this skill's output, so do not play the roles yourself.

## Flow

1. **Evidence.** One `explorer` per angle. Each returns claim / URL / exact quote triples and nothing else. Write them to `evidence-<angle>.jsonl` in the workspace, one JSON object per line: `{"claim": "...", "url": "...", "quote": "..."}`.
   - `direct`: the question as asked.
   - `disconfirming`: the strongest evidence that the expected answer is wrong: corrections, retractions, errata, dissenting primary sources.
   - `primary-source`: official docs, specifications, source code, changelogs and papers only.
   - `academic`, when the question has a research literature: papers, reproductions, negative results.

   Vary the wording of the searches between angles. The same index returns very different results for differently worded questions.

2. **Check.** `code-jig check-sources "evidence-*.jsonl"` (the command expands the pattern itself, so the same line works in PowerShell). It fetches each URL once and reports per claim: `ok` (the page contains the quote), `quote-missing`, `dead`, `blocked` (401/403/429: usually bot-blocking, not absence), `unreadable` (PDF or binary: quote unverified). It needs the network: in a sandbox, ask for escalation on the first run, not after a refusal. When the workspace is read-only, pipe the lines in and pass `-` as the file (`printf '%s\n' "$line1" "$line2" | code-jig check-sources -`, in PowerShell `$lines | code-jig check-sources -`), and hand the evidence to the later roles inline in their briefs. One run covers every file. When it runs, the final reply quotes its summary line (`N claims: ...`) exactly as printed. If it still cannot run, nothing is checked: say so, the verdict's first line is `UNVERIFIED:` and no claim is agreed.

3. **Challenge.** A `reviewer` reads the evidence files and the check report, and is told not to search. Its brief asks for a sweep before anything else: one line per claim, in order, none skipped or merged. `MISMATCH` when the quote does not contain what the claim asserts, `DOUBT` when the quote alone cannot tell (a number, a name, a baseline or a causal link it never states), `OK` only when the quote plainly carries the whole claim, each claim judged against its own quote. The sweep comes first because a reviewer that picks its findings first stops at the few defects that caught its eye. Tell it that a wrong flag costs one rebuttal while a fabrication it lets pass reaches the user as fact, so it flags every claim its quote does not plainly establish, and never invents a mismatch. Then it lists the questions that would settle each dispute.

4. **Cold verification.** For each of those questions, a fresh `explorer` gets only the question: no evidence and no challenge. Write its triples to `evidence-cold.jsonl` and run the check on it. When the challenge lists no disputes, skip this step and say so in the report; there is then only the first check report.

5. **Verdict.** A `judge` gets the question, every evidence file, every check report (one, or two after cold verification) and the challenge. It answers in a short paragraph, then lists agreed, disputed and unsourced claims with URL and quote, its confidence, and what new evidence would change the answer. Relay the verdict; do not rewrite it, and never add that sources are independent.

## Rules

- A claim can only be agreed when `check-sources` says `ok`, or `unreadable` with the quote confirmed by a second source.
- The searcher never dictates the answer alone. Evidence files from the same search are not independent sources; say how many distinct sites agree.
- Prefer primary sources over secondary ones, and recent over old when the subject moves.
- When results look stale, ask the user to start the session with live search (`codex -c web_search="live"`).
- If web search is not available in this session, say so and stop. Do not answer from memory.

## Common mistakes

| Mistake | Instead |
|---|---|
| Answering from one search because the question looks easy | Run every step; the user asked for sources checked by roles that did not write them. |
| Forking this conversation into a researcher | A fresh brief; a fork re-reads everything and inherits your bias. |
| Judging in the main session | The `judge` role exists so the verdict runs on the strongest model whatever tier this session uses. |
| Treating `blocked` as `dead` | Say it was blocked; find a second source or quote it as unverified. |
| Paraphrasing in the `quote` field | The check needs the exact words on the page. |

From jeancarlos/code-jig, Apache-2.0.

More from jeancarlos/code-jig