bibliography-audit

Files (1)

• SKILL.md

SKILL.md (first lines)

# bibliography-audit

This file is the canonical Empirical Workflow Kit implementation. Runtime views
defined in `workflow.manifest.yaml` link here; edit only the canonical tree.

Resolve `<skills_root>` from `workflow.manifest.yaml:canonical_source.skills_root`
before using a path below.

Read `runtime-profile.yaml` before using optional connectors or parallel
workers. Parallel batches are an optimization, not an evidence requirement.
When workers are unavailable or not authorized, process the same five-entry
batches sequentially and preserve one independent report per entry.

Adapted from `bibliography-audit` in `ericluo04/claude-academic-workflow` at commit
`8958cc246e65cdf7c36604f397a1c1719b7e2c14`; see `THIRD_PARTY_NOTICES.md`.

Given a `.bib`, verify batches of 5 entries against canonical metadata, then
emit a PASS/WARN/FAIL report and a `corrected.bib` the user reviews and merges.
Use parallel workers only when authorized. This skill never overwrites the
source; the source `.bib` is never modified.

One agent asked to audit 200 entries in a single pass drifts: early entries get careful treatment,
late entries get pattern-matched. Batches of 5 stay short enough that the cliff never arrives, and
the batch contract in phase 1 polices the same drift inside each batch.

Per-entry verification against canonical metadata is adapted from Scott Cunningham's
[MixtapeTools](https://github.com/scunning1975/MixtapeTools).

Hallucinated citations have a recognizable signature: plausible authors, plausible title, a DOI
that 404s, and no match in Crossref, OpenAlex, or arXiv. Surfacing that pattern is the point.

## Inputs and scoping

Take a `.bib` path as the argument. With no argument, Glob in this order and ask if ambiguous:

1. `references.bib`, `refs.bib`, `bibliography.bib`, then `*.bib` in the cwd.
2. The manuscript roots configured in `runtime-profile.yaml`.

Metadata verification and claim support are separate operations. A correct DOI and author list do
not establish that a manuscript sentence is supported by the cited work; claim support must be
audited against the version and source rung actually read.

With several `.bib` files in one project, grep the `.tex` for `\bibliography{}` / `\addbibresource{}`
to pick the one actually loaded. Skip anything under `_archive/` or `Old/` unless asked.

Scope before dispatching. Zotero-exported libraries carry far more entries than the manuscript
cites (one real project here: 345 entries, 170 cited), and auditing an uncited entry is wasted work.
Collect the cited keys and offer to restrict to them:

```bash
grep -ohE '\\[a-zA-Z]*[Cc]ite[a-zA-Z]*\*?(\[[^]]*\])*\{[^}]*\}' /abs/path/to/*.tex \
  | grep -oE '\{[^}]*\}$' | tr -d '{}' | tr ',' '\n' | tr -d ' ' | sort -u
```

Report both counts and let the user choose. Default to cited-only above 100 entries.

## Phase 0: parse

Read the `.bib` in full. Split on `^@\w+\{` boundaries and capture the entry type, citation key, and
every `field = {value}` pair (braced and quoted forms, escaped braces, multi-line values). Malformed
entries (unbalanced braces, missing key, duplicate key) go straight to FAIL with no subagent.

Triage by entry type, because the checks that apply differ:

- Indexed types (`@article`, `@inproceedings`, `@book`, `@incollection`, `@phdthesis`): full check set.
- Non-indexed types (`@misc`, `@online`, `@software`, `@dataset`, and `@unpublished` with no arXiv
  id): blog posts, docs pages, and repos legitimately have no DOI and appear in no index. Check the
  `url`/`howpublished` link resolves (`curl -sSI -o /dev/null -w '%{http_code}'`) and that `year`
  looks sane. Never assign the fabrication verdict to these.

Create `<bib_dir>/bibliography-audit_<YYYYMMDD_HHMM>/` containing `entries/` (one `.bib` fragment per entry,
subagent input) and `reports/` (one `.json` per entry, subagent output), plus `input.bib` as a
verbatim copy of the source for provenance.

## Phase 1: dispatch

Group the well-formed entries into batches of 5, preserving file order; the
last batch holds the remainder. When parallel workers are authorized, run at
most 8 batches per wave, wait for the wave, and then send the next one. When
they are not, run batches sequentially. Respect `--max-parallel N` if the user
passes it and drop to 2 when rate-limited.

Each subagent gets its batch's raw entry texts, the absolute `entries/` paths, one absolute output
path `reports/<citekey>.json` per entry, and the check list below. Subagent working directories
reset between Bash calls, so every path handed to a subagent must be absolute.

The batch contract: the subagent MUST return the full per-entry output schema for every one of its
entries, each verified independently against the lookup tools, with no verdict inferred from a
batchmate. State in the subagent prompt that the failure mode to guard against is skimping on the
later entries in the batch, and that entry 5 gets the same treatment as entry 1.

Give each subagent a 60-second lookup budget per entry. On expiry it writes
`{"verdict": "WARN", "diagnostic": "lookup_timeout"}` for that entry and moves to its next one.
Raise the budget on a cold cache; `paper.py` disk-caches responses for 30 days, so reruns are fast.

For a long parallel run, dispatch bounded waves and act on completion events,
sending the next wave as each one finishes.

## Phase 2: per-entry checks

### The two lookup tools

`paper.py` is the resolver from the `research-sources` skill. Read
`<skills_root>/research-sources/SKILL.md` if unfamiliar.

```bash
<skills_root>/research-sources/scripts/paper.py resolve "<query>" --json
```

It returns a JSON array. Interpreting it is the whole game:

- One element with `title`, `authors`, `year`, `venue`, `is_oa` populated: a confident canonical
  match.
- One element with only `{"doi": ..., "title": null}`: the identifier exists as a string but no
  index recognizes it. Exit code is still 0, so test the payload, not `$?`.
- Several elements carrying only `{doi, title, venue}`: no confident match, just relevance-ranked
  candidates. Treat as "not found" unless one clears the 0.85 title threshold below.

It does not return volume, issue, or pages. Crossref does, and Crossref is free and unmetered.

From bobbyZhong10/empirical-workflow-kit, MIT.

More from bobbyZhong10/empirical-workflow-kit