grill

Files (3)

• SKILL.md
• paste-prompt.md
• reflection.md

SKILL.md (first lines)

# Grill a decision

The user has been thinking with you, so your critique is correlated with theirs. Grilling sends the decision to a judge from a **different model family**. Grill is the outside judge step Felix Rieseberg recommends in [How to be more ambitious](https://felixrieseberg.com/how-to-be-more-ambitious-a-tutorial-for-humans-with-models/): do not ask the model to judge itself. The judge writes:
- a steelman of the decision, and a counter-steelman;
- challenges, each with a **falsifier**: the cheapest test that would settle it;
- a verdict, in the same words as the website: solid, solid if, shaky, or doesn't hold up. The tool's own result uses `holds`, `holds-with-conditions`, `weak` or `refuted` for those four.

## First time

If the user seems new ("what is this?", "set up Grill"), say in two lines what it does. Then offer to grill **one real decision right now**: something they decided or will decide this week, and how sure they are. The first grill is the setup.

If they set up the one-click route with their own model-router key (not Grill Pro, whose account is already set this way), mention once the two settings in that account that keep write-ups unstored. Both are off by default:
- **Input & Output Logging** (Observability settings): leave it off, or list their Grill key under "Excluded API Keys".
- **Use of inputs/outputs** (Privacy settings, the 1% discount): leave it off.

## 1. Send the decision as stated

Send what the user already said, plus context already in the chat. Do not rewrite it into a template. Do not fill in blanks. Do not write your own case against into the subject.

Ask at most one short question, and only if the decision is truly unclear, such as "What's the goal?" Then send it. Do not wait for an OK. Never add a required question before the verdict on a quick or bulk grill (several grills in a row, a code review, a fact or figure check). Ask only what the subject strictly needs.

The first time with this person, say once, as one line, and not again: This goes to an outside judge from a different company, and the router does not keep it. https://grillyour.ai/#privacy

Do not mention the check, its score, the judge's model, or a balance. If they ask to skip the check, call the tool with `quality_check: false` for this grill only.

Leave out names of people who aren't needed, and anything personal, health- or HR-related.

## 2. Send it

**If you have the `grill` tool** (from the Grill extension in Claude Desktop, or the plugin in Claude Code):
1. Call it with `subject` and `question`. If the subject came from another assistant, add `author`, for example `openai`, so that family is excluded too. Pass `big: true` only when the person already marked the decision as big. Never ask whether it is big. Leave it off on a quick or bulk grill.
2. If it returns a job id, call `grill_result` with it until the note arrives, before you reply. Do not ask the user for a job id, and do not tell them how long it takes.
3. If it says Grill isn't set up, relay its steps. Never ask the user to paste their key into the chat. A Grill Pro key goes in the same settings field as their own key. If Grill's settings name a judge model, the tool already uses it. That judge has to be a different company from the assistant they think with. If they name one from the same company, say so and don't treat it as the outside judge.

**If you don't have the tool** (claude.ai on the web or phone, or no extension yet), use the paste route. Claude Desktop and Claude Code enforce a different company in code. This route cannot, so you check it and warn.
1. Before you write the judge prompt, note your own maker in one line: "Your assistant: Claude by Anthropic."
2. Fill in `paste-prompt.md` from this skill's folder with the approved subject and question.
   - Mask what the one-click route masks: email addresses become [email], phone numbers [phone], card numbers [card number].
   - Never include a key or token.
   - Suggest they turn on the app's private mode first, for example a Temporary Chat in ChatGPT.
3. Give it back as one block they can copy. If it is under 6,000 characters, also give an "Open in ChatGPT" link: `https://chatgpt.com/?q=` followed by the URL-encoded prompt. For Gemini or any other assistant, they paste it themselves. Name the assistants they can paste it into: ChatGPT, Gemini or Grok.
4. Ask them to paste the judge's answer back here. Your own view is not the judge.
5. Mention once that Claude Desktop with the Grill extension does this in one step: https://github.com/mtangoz/grill#set-up
6. If they have a Grill Pro setup that names a judge, paste into that assistant, and only if it is a different company from you. Do not put their managed key in the prompt.

When they paste the judge's answer back, compare companies before you relay the verdict. Read the opening line. It should be `Judge: <model name> by <company>`. Compare it with the maker you noted. Treat ChatGPT and OpenAI as one company, Claude and Anthropic as one, Gemini and Google as one, Grok and xAI as one, and Muse and Meta as one. Copilot can run OpenAI, Anthropic or xAI models, so only Gemini counts as a different company (a line that names Gemini or Google). You are Claude, so the assistants to paste into instead are ChatGPT, Gemini or Grok. If you cannot tell the company, treat the line as missing.

- If the companies differ, the first line you show is `Judge: X by Y. Different company from your assistant ✓`, with X and Y from the judge's line.
- If the companies match, the first line you show is `Warning: this verdict is NOT independent. The judge is the same company as your assistant. Paste the judge prompt into ChatGPT, Gemini or Grok instead.`
- If the Judge line is missing, the first line you show is `Warning: this verdict is unverified. The answer has no Judge line, so this is not a confirmed outside judge. Paste the judge prompt into ChatGPT, Gemini or Grok instead.`

## 3. Check the sources and the numbers

The judge sees only the write-up. It cannot open a report, a dashboard, a call or a link, so it cannot tell you a cited source exists or says what the write-up claims. Grill checks the judge's quotes against the write-up, never the write-up's sources. You can open them, so you do this part.

Run it when the subject cites a source, when the user says check the sources, and every time a grilled piece of work goes to someone else (a founder, a board, a customer). Run it on the version that goes out, not the draft you grilled: a citation added after the grill was never checked.

1. List each claim the conclusion leans on that points at a source, with the source as written.
2. Open each source with your own tools. Quote the exact line or figure. If you cannot open it, it is **not found**, not true.
3. Mark each one on three checks: **exists** (you opened it); **says it** (the same number, metric, period and population as the claim); **answers this question** (it is about the change and the outcome being asked about, not a neighbouring one).
4. If you can start a fresh agent, give it only the list and the question, and have it do steps 2 and 3. You wrote the claims, so your check shares their blind spots.
5. Show a short table: claim, source, quoted line, exists, says it, answers this question. Any row that fails is cut from the work or marked unverified in it. Never replace a failed source with one you did not open.

Do not send the sources to the judge. Only the approved write-up leaves the machine.

**Then check the numbers.** Most wrong claims in an analysis are not missing sources. They are numbers that don't mean what the sentence says. Before the work goes out, check each claim against these, and fix or soften any that fails:
- **Rate beside count.** A count compared across periods or groups of different size needs its rate beside it (per contact, per send).
- **Enough events.** With fewer than 20 events on either side, say the numbers are too few to compare. Don't call a winner.
- **Read the thing itself.** Before you claim what a campaign, page or message said, open one real instance of it. A name is not its content.
- **Time for the outcome.** Replies, sign-ups and renewals lag the action. Don't end the outcome window on the day the action window ends.
- **The metric measures the claim.** "The copy did better" needs counts of the copy actually delivered, not requests or sends around it.
- **Recompute totals.** Work out a group's figure from the raw rows. One member's figure is not the group's.
- **Quote, with a date.** Quote a tool's or report's verdict word for word, with the date it ran. Anything no longer in view, fetch again or say unknown. Never from memory.
- **Say the coverage.** n read of N, and any known gap in the method.
- **Summaries keep the caveats.** A short version keeps the long version's doubts and adds no cause the long version didn't claim.
- **Answer what was asked.** Map each question the person asked to an answer, or say it was not answered.

**Don't call it grilled unless it was.** Say Grill reviewed something, or label it grilled, only when the `grill` tool returned a note in this chat, or on the paste route the judge's answer came back with a Judge line. A check you ran yourself is a same-model check, and you say so. Say which version was grilled, and what changed after it.


## Share a miss

When AI got a decision wrong (the user says "that was wrong", a grilled piece of work went out wrong, or a look-back missed), offer once, after the fix, never during it: "Want to share how it went wrong, not what? It helps the judge catch it next time." One "no" ends it for the conversation.

If they say yes, draft a miss card in the chat:
- the choices: what went wrong, where it got in, who caught it, who it was for, the kind of decision, what it cost, and whether it was really grilled;
- one sentence: what would have caught it;
- a made-up example with the same mistake. Use invented names, companies and numbers. Never a real name, a real number, a phrase from their material, an email or a link.

Then point them to https://grillyour.ai/miss/ to paste it in and send it themselves. Tell them the page can check the card against their real text in their browser, and that text is never sent. You never send it for them. The card is also theirs to keep: its one sentence is a check for their own next time.

## 4. Relay the result

On the paste route, the company check above is the first line you show. Then show only layer 1, at most 50 words: one verdict line, the single sharpest point (a missing option if there is one, otherwise the biggest risk), and one cheap test. If the judge marked it thin, show the quick take and the one question, not a fact list. A result from the `grill` tool already enforced a different company in code. Relay that note as written, and do not look for a Judge line. Do not show the judge's model, a score, a balance, a quote check, or a disclaimer.

If layer 1 says sources were not checked, run section 3 before the user acts on the verdict or sends the work.

Layer 1 already ends with this line. If it does not, add it once. Do not add a confidence question:

Say 'more', 'options', or your call.

When they say more, why, or what else, show the next challenge, its cheap test, and the one or two facts that would flip the call. About 60 to 80 words. On the tool, call `grill_result` with `detail` `more` and relay that. Do not write it yourself. When they say options, help me decide, or they are stuck, call `grill_result` with `detail` `options`, or on the paste route show the judge's options lines. About 60 to 80 words. A full report only when they say full report (`detail` `full`). On the paste route, use only the judge's lines for each layer.

Call `grill_result` with `detail` `deepen` only when the person asks to deepen or go bigger, and relay that layer. Do not call it otherwise, and do not write it yourself. It runs only for a decision they already marked as big, and only when that test is on. On a quick or bulk grill, do not mention deepen. The closing line may name deepen only for a big decision. Never add a question or a step before the verdict. On the paste route, same rule: only after the verdict, give them the second block in `paste-prompt.md` for the same outside judge. Do not answer it yourself.

- **Plain words, the same ones the website uses.** Say solid, solid if…, shaky or doesn't hold up. "Solid if" always comes with its conditions; never shorten it to "solid". Quote the judge; don't soften it: no "overall, the judge agrees with you".
- **Checked for you, then Your call.** When the note lists challenges, show "Checked for you" first (bugs, facts, figures, consistency) and "Your call" second (values and unwritten rules). Never move a challenge between groups.
- **Thin input.** If the note starts with "Quick take, on thin input:", show that and its one question. Do not turn it into a long fact list, and do not refuse.
- **The verdict never becomes the decision.** If the user asks "so should I do it?", or asks you to choose, hand the choice back and name the deciding test. Do not answer yes or no.
- **Pushback:** if the user disputes a challenge, don't settle it for them, and don't flip the verdict unless they bring new evidence.
- **Grilling again:** if the same decision is grilled again in this chat, ask one question: "What changed?" Show the earlier verdict beside the new one. If nothing changed, it is a re-run: no `supersedes` line and no `changed` line. If they say the choice changed, add `supersedes` as the earlier record's date and title, and `changed` as one of `evidence`, `goals`, `context` or `reweighed`, then their few words. Never invent them. Never send the earlier verdict to the judge.
- **A forecast is optional.** If the user marked the decision as big, offer it in one line after the verdict. On a quick or bulk grill, don't walk through it.
- **A record only after they answer, including after a quick take.** Any stated choice counts, such as do it, yes, keep it, or delay. Confidence is optional. If they did not say a percent, leave it empty and do not ask. Pre-fill the prediction from what they already said. If the `grill_decided` tool is available, call it with their words and relay its confirmation, about five words. Do not show the record. Follow `reflection.md` on the paste route, where the block is the only copy. If they never answer, write no record and set no look-back. Do not show a record before they answer, and do not explain the format.
- **Stop when the grill is done.** After a grill, don't start doing the decision (no code changes, no PRs, no product questions) unless the person separately asks. Grill's job ends at the verdict and the record.
- **Your own view:** only if asked, labelled "same-model critique".
- **Grill news:** The tool adds a line that starts with "Grill news:" only after a finished report, at most once a session, and only within 7 days of the release date in that build or on the first 3 days of a month. If the report includes that line, show it once, word for word, after the note and before the closing question. It is a statement. Do not turn it into a question, do not ask about it, and do not repeat it unless the user asks. Never ask for an email in the chat. The tool does not ask for, accept, or send an email. Outside those windows the report has no news line. Do not add one.

## 6. Look back

From mtangoz/grill, MIT.

More from mtangoz/grill