paperless-canonical-ingestion

Files (3)

• SKILL.md
• openai.yaml · agents/
• filing-matrix.md · references/

SKILL.md (first lines)

# Paperless Canonical Ingestion

## Outcome
Create and maintain a Paperless corpus that is canonical, deduplicated, and queryable by agent role.

## Source Discovery (read-only)
- Google Drive: `search_drive_files`, `list_drive_items`, `get_drive_file_download_url`
- Gmail: `search_gmail_messages`, `get_gmail_message_content`, `get_gmail_attachment_content`
- Microsoft: `office-mcp.files` with `search`, `get`, `download`
- Paperless: `search_documents`, `get_document`, `post_document`, `update_document`, `bulk_edit_documents`

## Workflow
1. Define scope and query set.
- Start from explicit folders, entities, and case names in the request.
- Expand with keyword variants (tax years, policy numbers, case IDs, entity aliases).

2. Collect candidate files.
- Pull metadata first: filename, path/link, modified date, sender/issuer, mime type.
- Download only shortlisted files needed for canonical comparison or ingest.

3. Canonicalize duplicates and near-duplicates.
- Use normalized filename stem + issuer + date + page count + first-page text similarity.
- Treat variants as duplicates when differences are forwarding wrappers, scan quality, or cosmetic edits.
- Keep one canonical file per logical document set.
- Record why non-canonical copies were skipped.

4. Rank canonical candidates (highest wins).
- `executed/signed/final` > draft.
- authority-issued original > forwarded attachment.
- complete packet > partial extract.
- machine-readable PDF > low-quality image scan (unless scan is the only executed copy).
- newest corrected version > older versions (unless chronology itself is legally important).

5. Upsert into Paperless.
- If a logical document already exists: use `update_document` for metadata normalization.
- If not present: ingest with `post_document`.
- Normalize `title`, `document_type`, `correspondent`, and tags using `references/filing-matrix.md`.

6. Apply ownership routing.
- Add owner tags so downstream task routing is deterministic.
- Use these role tags: `owner-estate`, `owner-hc`, `owner-io`, `owner-cos`, `owner-hd`, `owner-wellness`.
- Use multiple owner tags when the document has cross-functional impact.

7. Verify and report.
- Re-query by entity/case tags and spot-check duplicates.
- Return summary: `uploaded`, `updated`, `skipped_duplicate`, `needs_user_decision`.

## Canonical Baseline Seed Pack
When seeding family-office context, prioritize these categories:
- Estate core: will, irrevocable trust, POA, healthcare directives.
- Tax core: latest 3 years personal/entity returns, notices/orders, appeals, payment proofs.
- Entity core: formation docs, operating agreements, annual reports, key minutes/resolutions.
- Property core: deed/title, mortgage/loan docs, insurance declarations, major service contracts.
- Risk core: active policies, material liabilities, active litigation/case packets.

## Guardrails
- Do not perform Microsoft write operations.
- Do not delete source files from Drive/OneDrive/SharePoint/Gmail.
- Do not ingest both canonical and clearly duplicate variants.
- Prefer `update_document` over re-upload when Paperless already has the canonical file.

## Completion Checklist
- Canonical selected for each logical document set.
- Paperless metadata normalized (`title`, `document_type`, `correspondent`, tags).
- Owner-routing tags applied for all seeded records.
- Duplicate decisions documented in the final summary.
- Ambiguities surfaced explicitly for user decision.

## References
- Filing and owner-routing matrix: `references/filing-matrix.md`

From hvkshetry/StewardOS, MIT.

More from hvkshetry/StewardOS