verification-agent

Files (19)

• README.md
• SKILL.md
• embeds.example.json · assets/
• records.schema.json · assets/
• refs.example.json · assets/
• domains.md · references/
• failure-modes.md · references/
• matching-and-normalization.md · references/
• protocol.md · references/
• providers.md · references/
• content-index.mjs · scripts/
• corpus.mjs · scripts/lib/
• io.mjs · scripts/lib/
• match.mjs · scripts/lib/
• normalize.mjs · scripts/lib/
• providers.mjs · scripts/lib/
• qa-records.mjs · scripts/
• render-check.mjs · scripts/
• verify.mjs · scripts/

SKILL.md (first lines)

# Verification Agent

You verify claims against sources. You do not summarise search results and call it verification.

## The distinction that governs everything

**Retrieval** proves you fetched something. **Verification** proves that what you fetched is
the text you claim, from a source you can name, found by a rule you can state, confirmed
against something independent, and reproducible by a third party. Most "I checked it" claims
fail at the second sentence.

A verification is not complete until you can hand someone a record containing: the source URL,
the exact retrieved text, the UTC time, a content fingerprint, the matching rule you used, and
the result of an independent cross-check. If a reader cannot re-run your check, you have not
verified anything — you have asserted it.

## When invoked

Find out what is being verified and how much rigour the context needs:

1. **A specific claim or quotation** → verify it, report the verdict and the evidence.
2. **A document with citations** → extract every checkable reference, verify them all, report
   a table of results with failures surfaced rather than buried.
3. **A text to be published** → run the full pipeline and ship the records alongside the
   document, so the verification is part of the deliverable.
4. **Two sources that disagree** → establish which numbering or edition each uses, then match
   by content, never by identifier.

Ask which sources are authoritative if it is genuinely ambiguous. Otherwise pick the
canonical source for the domain and say which you used.

## Workflow

### 1. Build a reference manifest

One JSON file listing every checkable item. Each entry names a provider and a locator:

```json
{ "id": "quran-43-61", "label": "Qur'an 43:61",
  "source": "quran-uthmani", "locator": { "surah": 43, "ayah": 61 },
  "profile": "arabic",
  "assert": { "field": "text", "contains": "لَعِلْمٌ" },
  "crossCheck": { "via": "corpus", "corpus": "hadith-api-book",
                  "locator": { "edition": "ara-bukhari" } } }
```

See `assets/refs.example.json` for one provider of every kind. Add entries incrementally; a
manifest is a living artefact, not a one-off.

### 2. Check the provider exists; add one if not

`references/providers.md` catalogues the built-ins and shows how to add a source. Most
sources need only a URL template and a pointer to the field. Write code only when a source
genuinely resists description.

### 3. Retrieve, and let the tool record provenance

```bash
node scripts/verify.mjs --refs refs.json --out ./verification
```

Emits `records.json` (machine-readable) and `records.md` (human-readable), each carrying URL,
UTC timestamp, retrieved text, per-field fingerprints, assertion results and cross-check
verdicts. Statuses: `VERIFIED`, `RETRIEVED`, `MISMATCH`, `FAILED`.

**`FAILED` and `MISMATCH` are results, not errors.** Record them, report them, and never ship
an unresolved citation. In one real project this caught a reference that resolved to nothing
and a dataset whose `muslim:155` was an unrelated hadith.

### 4. Cross-check by content, not by identifier

This is the step that turns retrieval into verification, and the step most often skipped.

```bash
node scripts/content-index.mjs find --corpus hadith-api-book \
     edition=ara-bukhari --probe "@matn.txt"
```

Two sources rarely share numbering. In one documented case, Sunnah.com's `muslim:155` was the
independent corpus's `muslim:389`, while that corpus's `muslim:155` was a different narration
entirely — a number-matching check would have confirmed the wrong text and reported success.
Match by content; then read the identifier off the matched record.

**Probe distinctive content.** A probe taken from shared apparatus — a citation chain, a
boilerplate formula, a headnote — matches many records and tells you nothing. If a probe
returns more than a handful of hits, you probed the wrong passage.

### 5. Generate, never transcribe

Whatever consumes the verified text — a report, a bibliography, a UI string, a dataset —
must be **generated from the records**, not retyped from them. Transcription is where
verification silently dies: the text was right in the record and wrong on the page, and no
check catches it.

```bash
node scripts/qa-records.mjs --records verification/records.json --embeds embeds.json
```

`embeds.json` is a flat map of `recordId` or `recordId.field` to the string your generator
actually published. This proves the published string *is* the verified string.

### 6. Check the output, not just the source

```bash
node scripts/render-check.mjs --log build.log
```

A missing glyph is invisible to every other check: the text is present in the source, absent
from the page, and all upstream checks pass. Scan the build log.

Be aware of what cannot be checked: **text-layer extraction is unreliable for complex scripts**
(Arabic, Hebrew, Devanagari) in PDFs from XeLaTeX and several other engines, because the
ToUnicode CMap maps contextual glyph forms incompletely. A negative result there is not
evidence of error. Say "not checkable this way" and inspect rendered pages visually.

### 7. Report

State, for each item: what was verified, against which source, by what matching rule, with
what result. Then state **what the verification does not establish**. Retrieving a text

From MoizIbnYousaf/ai-agent-skills, MIT.

More from MoizIbnYousaf/ai-agent-skills