citable

Files (56)

• CHANGELOG.md
• LICENSE
• README.md
• SKILL.md
• map-format.md · references/
• build_table.py · scripts/
• LICENSE-reporters-db · scripts/data/
• reporters.json · scripts/data/
• inventory.py · scripts/
• template.html · scripts/
• LICENSE · scripts/vendor/nupunkt/
• __init__.py · scripts/vendor/nupunkt/
• _version.py · scripts/vendor/nupunkt/
• cli.py · scripts/vendor/nupunkt/
• cli_model_utils.py · scripts/vendor/nupunkt/
• __init__.py · scripts/vendor/nupunkt/core/
• base.py · scripts/vendor/nupunkt/core/
• constants.py · scripts/vendor/nupunkt/core/
• language_vars.py · scripts/vendor/nupunkt/core/
• parameters.py · scripts/vendor/nupunkt/core/
• tokens.py · scripts/vendor/nupunkt/core/
• general_abbreviations.json · scripts/vendor/nupunkt/data/
• legal_abbreviations.json · scripts/vendor/nupunkt/data/
• document.py · scripts/vendor/nupunkt/
• __init__.py · scripts/vendor/nupunkt/evaluation/
• dataset.py · scripts/vendor/nupunkt/evaluation/
• evaluator.py · scripts/vendor/nupunkt/evaluation/
• metrics.py · scripts/vendor/nupunkt/evaluation/
• __init__.py · scripts/vendor/nupunkt/hybrid/
• adaptive_tokenizer.py · scripts/vendor/nupunkt/hybrid/
• confidence_tokenizer.py · scripts/vendor/nupunkt/hybrid/
• confidence_tokenizer_fixed.py · scripts/vendor/nupunkt/hybrid/
• layout.py · scripts/vendor/nupunkt/
• __init__.py · scripts/vendor/nupunkt/models/
• default_model.json.gz · scripts/vendor/nupunkt/models/
• __init__.py · scripts/vendor/nupunkt/optimization/
• discovery.py · scripts/vendor/nupunkt/optimization/
• hyperparameter.py · scripts/vendor/nupunkt/optimization/
• py.typed · scripts/vendor/nupunkt/
• segmentation.py · scripts/vendor/nupunkt/
• __init__.py · scripts/vendor/nupunkt/tokenizers/
• paragraph_tokenizer.py · scripts/vendor/nupunkt/tokenizers/
• sentence_tokenizer.py · scripts/vendor/nupunkt/tokenizers/
• __init__.py · scripts/vendor/nupunkt/trainers/
• base_trainer.py · scripts/vendor/nupunkt/trainers/
• __init__.py · scripts/vendor/nupunkt/training/
• core.py · scripts/vendor/nupunkt/training/
• hyperparameters.py · scripts/vendor/nupunkt/training/
• optimizer.py · scripts/vendor/nupunkt/training/
• __init__.py · scripts/vendor/nupunkt/utils/
• compression.py · scripts/vendor/nupunkt/utils/
• iteration.py · scripts/vendor/nupunkt/utils/
• paths.py · scripts/vendor/nupunkt/utils/
• statistics.py · scripts/vendor/nupunkt/utils/
• __init__.py · scripts/vendor/nupunkt/workflows/
• automated_training.py · scripts/vendor/nupunkt/workflows/

SKILL.md (first lines)

# citable

Produces `citable-<document>.html` (+ `.csv`) in the case folder: a local page, opened in Chrome, that **locates** sources. This is pattern-matched review: the scripts find and link the citations; the reviewer keeps the judgment role, checking each citation and recording the result in the status and note fields. For lawyers and paralegals cite-checking their own or an opponent's US filing.

## Requirements
Run on the user's computer (device shell) in the folder holding the filings and exhibits. Check once: `python3` (3.10+) with `pdfplumber` + `Pillow`; `pdftotext`, `pdftoppm` (poppler); `tesseract` (only for exhibits whose ¶ numbers are images). Missing tesseract → those links are marked approximate (amber), not wrong.
Scripts: `scripts/inventory.py`, `scripts/build_table.py` (needs `scripts/template.html`, `scripts/data/` and `scripts/vendor/` beside it; bundled, nothing to install). Copy the whole `scripts/` folder into the case folder's parent or run it from the skill directory; never write into the user's PDFs. If the skill directory is not on the user's computer, copy the scripts as `citable-scripts.tar.gz` into the case folder (never into `citable-backups/`), unpack them to a scratch directory outside it, and tell the user the archive can be deleted. `--out <path stem>` writes the table elsewhere (links still reach the case folder). Before overwriting a table, the build copies it and its CSV to `citable-backups/` beside it (timestamped; the user can delete old copies).

## Workflow
1. **Ask** which filing(s) to process (one table per filing) and confirm the case folder (where the filing and record exhibits sit). Tell the user that documents cited by name (unlettered exhibits, locally saved authorities) are matched to files by name, so file names should resemble how the filing cites them (see step 3). If the filing is a Word file, tell the user: "citable v1.0 is designed to work with PDFs. If your draft is in Word, make a copy, accept all tracked changes, delete all comments, and save the copy as a PDF (File → Save As → PDF), and then run citable on that copy." If the filing, or a document it cites with a pinpoint, has no text layer (an image-only scan), stop and say it must be OCR'd first (the build refuses such a filing); one cited without a pinpoint may be mapped once identified (tell the user). If it is not a US court filing (e.g. an arbitral memorial), say that case-law and statute links will not work, but record documents and any authorities saved in the folder will link once mapped.
2. **Inventory:** `python3 scripts/inventory.py "<case>"`. Note each PDF's role, ECF numbers, `text_para_markers`, `margin_para_images`, `exhibit_refs` and `stamps_first_page` (other courts' ECF documents and NYSCEF numbers found in each file; the build uses these stamps to link docket and NYSCEF citations to copies in the folder). If a document cited by ¶ (e.g. an arbitral award) has no ¶ numbers in its text layer (`text_para_markers` near 0), check `margin_para_images`: roughly one or more per page suggests the ¶ numbers are images. That is a signal, not proof (logos and stamps also count), so render one page (`pdftoppm`) and look at it before listing the document in `para_image_docs`.
3. **Draft `citable-map.json`** in the case folder (format: `references/map-format.md`):
   - `exhibits`: letter/number → file, from the declaration's "attached … as Exhibit X" sentences, filenames and ECF attachment numbers. An exhibit that is only pages inside another file is mapped as `{"file": …, "pages": "20-25"}`; do not create new files. If two filings use the same exhibit letters for different documents, use one map per side.
   - `docs`: every other file the filing cites by name rather than exhibit label: other filings (`"X Dec": "X decl.pdf"`), exhibits not yet lettered (`"2026-09-10 medical report": "2026-09-10 Medical Report (Dr Y).pdf"`), and authorities saved in the folder (`"CL-12": "CL-12 Tza Yap Shum v Peru.pdf"`). Use the name exactly as the filing uses it. Each links by ¶, by page, or (no pinpoint) to its first page. If no file plausibly matches a name, or two do, ask; do not guess.
   - `record_terms`: defined terms that mean an exhibit ("Award" → A), including deposition short names ("Vavra Dep." → A, so "Vavra Dep. 124:19-21" links).
   - `authorities`: every case in the filing. Look each one up with the CourtListener connector (`analyze_citations` on the table of authorities or the citation list, which is one call). Record the opinion URL, a short name as the filing uses it, and `cl_name` only if CourtListener's name differs. Not found → leave it out (the table marks it amber). Then check page markers on the opinion CourtListener displays for the case (the `combined` opinion if the cluster has one, else the lead; take opinion ids from `cluster_opinions`, never cluster ids): `search_document` for the cited reporter's page after the first page (e.g. `*663` for 556 U.S. 662), up to 10 opinion ids per call. Found in a `star-pagination` span → `"markers": true`; otherwise `false` (`<page-number>` markers cannot be link targets), plus `"marker_reporter"` if the copy marks a parallel reporter's pages instead (e.g. Iqbal's shows `*1942`: `"129 S. Ct. 1937"`). Add `"stub": true` if there are no markers and the text is under about 5,000 characters. Pinpoints are green only where `markers` is true. CourtListener allows about 125 requests a day and 50 an hour: tell the user before starting how many cases need marker checks (unchecked ones stay amber), and check cases cited with a pinpoint first. A cite CourtListener resolves to a different case gets `no_link` (see the format note), not omission.
   - `unreported`: each Westlaw/Lexis cite that gives a docket number, court and date. Search CourtListener RECAP (`search`, type `rd`, `court`, `docket_number`, entry dates ±2 days) for the opinion, order or report and recommendation filed that day; take its `filepath_local` (recap-documents endpoint) → `recap` = `https://storage.courtlistener.com/<filepath_local>`, plus `entry` (court, docket, Dkt. number). If the match is doubtful (e.g. an order adopting a report and recommendation, which then holds the cited passage), say so in the map summary. Decision not in RECAP → `docket_url` and `note`. Where the filing quotes the decision at a star page, search the RECAP document for a short exact phrase of the quote and record the PDF page in `pages` (find it from the page stamps or page breaks around the match); a paraphrase gets no page. These links are always amber.
   - `dockets` (only if the filing cites docket entries): per case number, the CourtListener docket URL and short descriptions of the cited entries (from the docket). `nyscef`: the index number, and a file only where the stamps do not show one. `doc_series` and `numbered_paragraphs`: see the format note.
   - `treaties`: rule = copy in the record first, else the official reporter. For a UN Treaty Series link you need the registration number (search treaties.un.org); URL pattern `https://treaties.un.org/doc/Publication/UNTS/Volume%20<vol>/volume-<vol>-I-<regno>-English.pdf`. Confirm the URL resolves before recording it.
4. **Show the map to the user and get confirmation before building.** Never guess an exhibit or document mapping. Only after the user confirms, record `"confirmed": {"by": "<name>", "date": "<YYYY-MM-DD>", "sha256": "<output of python3 scripts/build_table.py --fingerprint citable-map.json>"}`. The build refuses an unconfirmed map, or one changed since it was confirmed. Any later addition or change (including for a second filing, and remapping exhibits to OCR'd copies) is shown and confirmed again, with a new fingerprint.
5. **Build:** `python3 scripts/build_table.py "<case>" "<filing>.pdf" citable-map.json`. If it stops for finding no citations, tell the user; add `--allow-no-citations` only once they confirm the filing cites nothing. If a table for that filing already exists, its saved (or previously carried) statuses and notes are carried forward automatically, and the old table is kept in `citable-backups/`. For a revised draft with a new filename, add `--previous "<old table>.html"`. Only work the reviewer has **saved** carries forward, so tell them to click Save first.
6. **Report in chat** (briefly): number of citations; links to the exact page; each amber/red link and why; any `MAP WARNING`, `COVERAGE WARNINGS`, `FORMAT WARNINGS` and `POSSIBLY MISSED` lines printed by the build (these are for the user, never put them in the page); the "not treated as citations" list; on updates, the changed, relinked, reworded, new and removed counts.
7. Suggest the reviewer check the table against the filing once for missed citations.

If a script crashes or plainly misses a kind of citation, tell the user what failed. Change the scripts only with the user's agreement, and list the changes.

## What the table does
- Each passage appears once, with its citations highlighted and numbered and the filing's formatting kept. There is one sub-row per citation with links, a cite-check status (`check pending` / `confirmed` / `partial support` / `unsupported`) and a note. A citation with several pinpoints is one citation.
- Link dot: green = exact page, amber = approximate (e.g. an ambiguous *Id.*, a docket search or listing, a record cite whose pinpoint the script could not read, a web address taken from the filing, a PDF page used because the document has no printed page numbers), red = unresolved (e.g. a ¶ that does not exist in the exhibit, with the reason shown), grey = listed for checking but not linked (*supra*; Westlaw/Lexis cites not found on CourtListener; state statutes; local rules; another case's docket entries; `no_link` cites).
- Record pinpoints: ¶, page, section ("§1.2"), transcript page:line (via the printed page numbers, incl. condensed transcripts) and Bates numbers (read exactly from the text layer). An exhibit to an exhibit ("Exhibit A to Exhibit BB") links to the outer exhibit (amber). A deposition exhibit inside an exhibit ("Ex. H, Smith Dep. Ex. 9, at 2") links to its own map entry if it has one (map it as a page range of Ex. H); if not, its page is counted from its slip sheet or exhibit sticker in Ex. H, or Ex. H opens at its first page, amber either way; only a Bates number read in Ex. H stays green. A record cite followed by numbers the script could not read is amber, never green.
- Footnotes are found by matching the numbers called in the text to the footnote that starts with that number; a continued footnote is recognised by smaller type or single spacing (in a double-spaced filing), with the divider line only as a fallback. A number called twice is read with the next footnote of that number. Two footnotes with one number on one page, called once, are both read (number small or full-size) if the line above the second ends a sentence; a wrapped line that begins with the number ("28 U.S.C. § 1331") stays in its footnote. Each pair gives a FORMAT WARNING; so does a line that begins with the number but follows a line that does not end a sentence (it is read as a continuation). A repeated number on a later page is not detected.
- *Id.* is resolved through record and case citations, skipping sources cited in explanatory parentheticals ("quoting …"). If a footnote cite sits between *Id.* and its text antecedent, the row is amber and explained. An *Id.* with a ¶ pinpoint after a case citation is red (cases are not cited by ¶) and names the last record citation. An *Id.* with a page pinpoint ("Id. at 1") whose source was cited by transcript page:line keeps its literal link, amber (the filing may mean another document).
- Page pinpoints use the document's own printed page numbers; with no printed numbers, the PDF page. Where the document also carries ECF page stamps and the two numberings differ, a second link opens the ECF page; both are green and their hover text says to check which reading the filing means. "[Bates range] at N" counts N pages from the first Bates number (amber).
- Case pinpoints open CourtListener at the page marker (Chrome text fragment); reporters and their variant abbreviations come from Free Law Project's reporters-db. Where CourtListener's copy has no page markers a link can jump to, marks another reporter's pages, or has little text, the link is amber and says so; state-reporter links remind the reviewer to check that the marker is for the cited reporter. A pinpoint before the first page is red; one more than 150 pages after it is amber. Westlaw/Lexis cites found on CourtListener open the court's PDF (amber): at the page holding the quoted passage where the map records one, otherwise at p. 1.
- Docket citations (Docket/ECF/Dkt./Doc. No., with "at P" or "¶ N" pinpoints) and NYSCEF citations link to the copy in the case folder (found from the filing stamps) where there is one, otherwise to the CourtListener docket listing (amber). Entries of another case's named docket ("(IOEngine Docket) … Doc. Nos. 1, 12") are listed (grey), not linked. Federal rules and statutes open on LII; CPLR sections on the New York Senate site; state statutes (e.g. ILCS, Mass. Gen. Laws, Cal. … Code §) are listed (grey) but not linked. `See footnote N` links to the footnote.
- A citation standing as its own sentence takes the sentence before it as its proposition.
- If a rebuild finds the map now sends a citation elsewhere, the row is flagged "relinked" and its status reset (the note is kept).
- Statuses and notes are held only in the open page until Save writes them into the HTML file (Chrome can overwrite the original; otherwise a copy goes to Downloads), so tell the reviewer to save frequently. Export CSV is also available. "Promote to proposition for checking" moves a reference-only item into the table.

## Confidentiality
The page makes no network calls and stores nothing in the browser; a clicked link tells that site which source is opened. Statuses and notes live in the saved HTML file (or a copy in Downloads); Export CSV also writes to Downloads. The page, its backups in `citable-backups/` and the CSV contain the filing's text and the review: treat them as attorney work product.

## Limits (say so if relevant)
PDFs need a text layer. OCR'd PDFs are fine. Image-only scans must be OCR'd first. OCR errors in the filing itself can cause missed citations; OCR misreads of Bates stamps or transcript page numbers make those links amber (first page). *supra* is listed but not linked. Case pinpoints jump only where CourtListener's copy marks the cited reporter's pages. State statutes and local rules are listed but not linked, and less common forms may be missed. A case cited only by name ("Masud,") or only by docket number is not read. Treatises and secondary sources are not detected. Less common state reporters may be missed. Uncited factual sentences are not flagged. Reading ends at the filing's signature line. Without one (e.g. a draft), it stops at a table of contents that follows a body heading (e.g. Introduction, Argument) or numbered paragraphs, read as an attached document (COVERAGE WARNING); otherwise the whole file is read, including any attachments. Body text before a table of contents is discarded as cover material if it has neither (COVERAGE WARNING if it is long). A notice of motion with its own signature, followed by its memorandum in the same PDF, is read only to the notice's signature (COVERAGE WARNING); split the PDF to table the memorandum. A bold heading in sentence case without an outline marker may be read as body text.

From lawve-ai/awesome-legal-skills, AGPL-3.0.

More from lawve-ai/awesome-legal-skills