fact-check

Files (1)

• SKILL.md

SKILL.md (first lines)

Fact-check a Word document and produce an annotated copy with findings as Word comments plus a markdown summary report.

Target file: $ARGUMENTS

## Parse Arguments

Extract from `$ARGUMENTS`:
- `file_path`: the .docx file to check (required)
- `--tier2`: enable web-based checks (link verification, claim cross-referencing) [Phase 2, not yet implemented]
- `--summary-only`: skip Word comments, only produce the markdown report

If `--tier2` is passed, inform the user that Tier 2 checks are planned for a future update and proceed with Tier 1 only.

## Tier 1 Checks (Always Run)

Analyze every paragraph and table cell in the document for:

### 1. Internal Consistency
- Contradictory statistics between sections (e.g., "73% of districts" in one place, "nearly 80%" in another referring to the same metric)
- Conflicting dates, names, or figures
- Inconsistent terminology for the same concept

### 2. Unsourced Claims
- Statistics, percentages, dollar amounts, or specific numbers with no inline citation
- Attributed quotes without a source
- Specific dates or timelines presented as fact without attribution
- Exception: widely known facts and general statements do not need citations

### 3. Citation Integrity
- Every inline citation (Author, Year) or numbered reference must have a matching entry in the References/Bibliography/Works Cited section
- Every reference entry should be cited at least once in the body
- Flag orphaned citations (in-text but not in references) and orphaned references (in references but never cited)
- Check for malformed citations (missing year, missing author, etc.)

### 4. Logical Coherence
- Conclusions that don't follow from the evidence presented in that section
- Non sequiturs or logical leaps
- Claims that contradict the document's own data or tables

### 5. Plausibility Flags
- Percentages over 100% or negative percentages
- Negative counts where only positive values make sense
- Absolute claims ("all", "none", "every", "never") that are likely overstated
- Numbers that seem implausible for the context (e.g., "500% increase")

### 6. Stale References
- Sources, legislation, or policies dated more than 5 years before today's date
- Flag as INFO, not ERROR (old sources may still be valid, but worth noting)
- Exception: foundational/seminal works and historical references are expected to be older

## Analysis Approach

1. Read the entire document first to understand its structure, topic, and citation style
2. Build a list of all inline citations and all reference entries
3. Go section by section, checking each paragraph against all 6 categories
4. For each finding, record:
   - `paragraph_text`: a snippet of the paragraph (first 80 chars or the relevant sentence)
   - `search_text`: the specific text to anchor the Word comment on (the problematic phrase)
   - `category`: one of `CONSISTENCY`, `UNSOURCED`, `CITATION`, `LOGIC`, `PLAUSIBILITY`, `STALE`
   - `severity`: `ERROR`, `WARNING`, or `INFO`
   - `comment_text`: clear, actionable description of the issue

### Severity Guidelines
- ERROR: Factual contradiction, orphaned citation, percentage > 100%, conclusion contradicts own data
- WARNING: Unsourced statistic, absolute claim, logical leap, malformed citation
- INFO: Stale reference, minor inconsistency in terminology, uncited reference entry

## How to Insert Word Comments

Use this Python approach with `python-docx` and `lxml`:

```python
from docx import Document
from docx.oxml.ns import qn
from docx.oxml import OxmlElement
from lxml import etree
from docx.opc.part import Part
from docx.opc.packuri import PackURI
import os
from datetime import datetime

# 1. Build the comments XML container
comments_xml = '''<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:comments xmlns:wpc="http://schemas.microsoft.com/office/word/2010/wordprocessingCanvas"
            xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
            xmlns:o="urn:schemas-microsoft-com:office:office"
            xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"
            xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math"
            xmlns:v="urn:schemas-microsoft-com:vml"
            xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"
            xmlns:w10="urn:schemas-microsoft-com:office:word"
            xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"
            xmlns:wne="http://schemas.microsoft.com/office/word/2006/wordml">
</w:comments>'''
comments_element = etree.fromstring(comments_xml.encode('utf-8'))

# 2. For each comment, add to the XML and mark the paragraph
def add_comment_to_xml(comments_element, comment_id, author, date_str, text):
    comment = etree.SubElement(comments_element, qn('w:comment'))
    comment.set(qn('w:id'), str(comment_id))
    comment.set(qn('w:author'), author)
    comment.set(qn('w:date'), date_str)
    p = etree.SubElement(comment, qn('w:p'))
    r = etree.SubElement(p, qn('w:r'))
    t = etree.SubElement(r, qn('w:t'))
    t.text = text
    t.set(qn('xml:space'), 'preserve')

def add_comment_markers(paragraph, search_text, comment_id):
    """Add commentRangeStart, commentRangeEnd, and commentReference to a paragraph."""
    found = False
    for run in paragraph.runs:
        if search_text.lower() in (run.text or "").lower():
            comment_start = OxmlElement('w:commentRangeStart')
            comment_start.set(qn('w:id'), str(comment_id))
            run.element.addprevious(comment_start)

            comment_end = OxmlElement('w:commentRangeEnd')
            comment_end.set(qn('w:id'), str(comment_id))

From jasonbhorne/claude-code-skills, MIT.

More from jasonbhorne/claude-code-skills