scientific-literature

Files (109)

• plugin.json · .claude-plugin/
• .python-version
• .standalone-db
• SKILL.md
• USAGE.md
• cluster_synthesis.py
• atoms.tsx · dashboard/components/
• corpus-detail.tsx · dashboard/components/
• curation-diagrams.ts · dashboard/components/
• database-selector.tsx · dashboard/components/
• db.ts · dashboard/components/
• embedding-map.tsx · dashboard/components/
• faceting-note-viewer.tsx · dashboard/components/
• investigation-detail.tsx · dashboard/components/
• investigation-list.tsx · dashboard/components/
• investigation-shell.tsx · dashboard/components/
• kefed-graph.tsx · dashboard/components/
• kefed-instance.tsx · dashboard/components/
• kqed-view.tsx · dashboard/components/
• mermaid.tsx · dashboard/components/
• paper-curation.tsx · dashboard/components/
• paper-detail.tsx · dashboard/components/
• stage-view.tsx · dashboard/components/
• tokens.ts · dashboard/components/
• lib.ts · dashboard/
• page.tsx · dashboard/pages/scientific-literature/acquisition/
• page.tsx · dashboard/pages/scientific-literature/corpus/[id]/
• page.tsx · dashboard/pages/scientific-literature/faceting-note/[id]/
• page.tsx · dashboard/pages/scientific-literature/investigation/[id]/
• page.tsx · dashboard/pages/scientific-literature/kqed/[id]/
• page.tsx · dashboard/pages/scientific-literature/map/
• page.tsx · dashboard/pages/scientific-literature/ontology/
• page.tsx · dashboard/pages/scientific-literature/
• page.tsx · dashboard/pages/scientific-literature/paper/[id]/
• page.tsx · dashboard/pages/scientific-literature/template/[id]/
• route.ts · dashboard/routes/acquisition/
• route.ts · dashboard/routes/bundle/[id]/
• route.ts · dashboard/routes/corpora/
• route.ts · dashboard/routes/corpus/[id]/
• route.ts · dashboard/routes/databases/
• route.ts · dashboard/routes/faceting-note/[id]/
• route.ts · dashboard/routes/faceting-note/[id]/run/
• route.ts · dashboard/routes/faceting-notes/
• route.ts · dashboard/routes/investigation/[id]/
• route.ts · dashboard/routes/investigations/
• route.ts · dashboard/routes/kqed/[id]/
• route.ts · dashboard/routes/map/
• route.ts · dashboard/routes/ontology/
• route.ts · dashboard/routes/paper/[id]/checks/
• route.ts · dashboard/routes/paper/[id]/curation/
• route.ts · dashboard/routes/paper/[id]/
• route.ts · dashboard/routes/stage/[id]/
• route.ts · dashboard/routes/template/[id]/
• architecture-kqed.md · docs/
• example-fragments-sirt3.md · docs/
• example-kqed-sirt3.md · docs/
• grounding-text-guidelines.md · docs/
• kqed-phase1-grounding-plan.md · docs/
• kqed-phase2-sensemaking-plan.md · docs/
• kqed-phase3-analysis-plan.md · docs/
• …and 49 more

SKILL.md (first lines)

# Scientific Literature Skill

Multi-source scientific literature search, ingestion, and analysis.
Sources: Europe PMC, PubMed (NCBI), OpenAlex, bioRxiv/medRxiv.

## Quick Start

> **Path note:** Replace `<skill-path>` with your installation directory
> (e.g. `~/.claude/plugins/cache/scientific-literature/` when installed as a plugin).

```bash
# Count papers before committing (EPMC)
uv run --project <skill-path> python <skill-path>/scientific_literature.py count \
    --query "CRISPR AND gene editing"

# Search EPMC and store results in a corpus
uv run --project <skill-path> python <skill-path>/scientific_literature.py search \
    --source epmc --query "CRISPR AND gene editing" --collection "CRISPR Papers" \
    --max-results 500

# Ingest a single paper by DOI (OpenAlex + PubMed fallback)
uv run --project <skill-path> python <skill-path>/scientific_literature.py ingest \
    --doi "10.1038/s41587-020-0700-8"

# List papers in a corpus
uv run --project <skill-path> python <skill-path>/scientific_literature.py list \
    --collection "collection-abc123"
```

## Full-Text Ingestion & Per-Paper Cache — REQUIRED convention

This is the ONLY correct way to ingest a paper's full text and cache its files. It is
**mandatory and uniform across every scilit investigation** (search corpora, deep-dives,
CAIS, etc.) — never improvise an alternate layout.

### 1. Paper identity (deterministic)
Every paper is a `scilit-paper` whose id is a **pure function of its best stable
identifier**: `DOI → PMID → arXiv → content-hash(title|first-author|year)`. Compute it
with `paper_identity()` in `paper_identity.py`; create/find papers only via
`kqed.upsert_paper(driver, meta)`. Same paper → same id, always. The tier is recorded in
`scilit-identity-basis` / `scilit-identity-value`. (Full design: `docs/paper-identity-design.md`.)

### 2. One directory per paper; every file is named by ITS artifact id
```
~/.alhazen/cache/fulltext/<paper-id>/
    scilit-fulltext-<paper-hash>.pdf   # source rendition  ┐ ONE full-text artifact
    scilit-fulltext-<paper-hash>.txt   # kreuzberg text     ┘ (renditions share the id-base)
    <other-artifact-id>.<ext>          # any OTHER artifact of the paper (table/figure/supplement/data)
```
**A paper may have several artifacts; each FILE is named by the id of the artifact it
belongs to (basename = artifact id, suffix = format).** There is ONE **full-text artifact**
per paper, `scilit-fulltext-<paper-hash>` (paper-hash = the paper id's 12-hex suffix); its
`.pdf` source and `.txt` extraction are two **renditions of that single artifact**, sharing
the id-base and differing only by suffix. Any other content (extracted tables, figures,
supplements, datasets) is its OWN `alh-artifact`, file `<that-artifact-id>.<ext>`.
**EVERY such file goes in that paper's `fulltext/<paper-id>/` subdirectory — and nowhere
else** (NOT `cache/pdf/`, `cache/text/`, `cache/extracted/`, or `cache/papers/`).

### 3. Rules
- **MOVE files into place** (copy, then remove the original) — **never symlink**, and never
  leave a paper's content scattered in `cache/pdf/`, `cache/text/`, `cache/extracted/`, or
  `cache/papers/`.
- The **filename basename equals the artifact id** — a loose file is self-identifying.
- **Register every file as an `alh-artifact` with a complete file xref:**
  `cache-path = fulltext/<paper-id>/<artifact-id>.<ext>` **plus** `content-hash` (sha256),
  `file-size`, `mime-type`, and `format` — linked to the paper via `alh-representation`.
  The full-text artifact's `cache-path` points at its text rendition (`.txt`, the indexable
  one); the `.pdf` is the sibling by suffix.
- **The path is computable from the paper id** — the full-text artifact id is deterministic
  (`scilit-fulltext-<paper-hash>`), so `paper → fulltext/<paper-id>/scilit-fulltext-<paper-hash>.pdf`
  needs no graph lookup. Keep it that way.
- A paper with a local source PDF must carry `scilit-acquisition-status = held`; a
  cited-but-undownloaded reference is `needed`.

### 4. Ingestion flow (per paper)
1. `upsert_paper(meta)` → deterministic `<paper-id>`; full-text artifact id `scilit-fulltext-<paper-hash>`.
2. Download the source PDF → **move** to `fulltext/<paper-id>/scilit-fulltext-<paper-hash>.pdf`; register the artifact with the complete file xref (cache-path + content-hash + file-size + mime-type + format); set status `held`.
3. kreuzberg-extract the text → write `fulltext/<paper-id>/scilit-fulltext-<paper-hash>.txt` as the **same** artifact's `.txt` rendition.
4. Extracted tables / figures / supplements / data → each its OWN artifact, file `<artifact-id>.<ext>` in the same dir, with its own complete file xref.
5. Never write any of the above outside `fulltext/<paper-id>/`.

> `cache/pdf/` may still hold OTHER skills' documents (tech-recon, health-coach) — those are
> out of scope for this convention; do not touch them. This convention governs `scilit-paper`
> content only.

Reconciliation prototypes that retrofit existing data to this layout live in
`prototypes/reconcile_*_fulltext.py` (they MOVE sources into `fulltext/<paper-id>/`).

---

## KEfED Model Authoring — build STRUCTURED, RICH, ACCURATE, EXPRESSIVE models

A KEfED model (`kefed-model`, authored under a bundle with `add-experiment`) is a **graph of
`kefed-model-node`s** that reconstructs an experiment. Follow these principles:

1. **Reconstruct the real protocol from the paper's Methods.** Read the Experimental Procedures and
   build the actual material chain — *organism → sample preparation → assay → readout* — including where
   cells came from and how they were prepared (harvest, sort, lineage-deplete, transduce, transplant,
   culture...). The **combination of nodes in the graph carries the experimental-design semantics**, not
   any single node.

2. **Minimal ontological commitment per node; maximum reuse.** Node *definitions* are **generic and
   reusable** — `experimental mouse`, `bone-marrow harvest`, `competitive transplantation` — NOT
   `HSC (WT or SIRT3-KO mouse)`. What differentiates *this* experiment lives in the attached **variables +
   values**, never in node names. Share one OOEVV element wherever the concept genuinely recurs:
   `add-entity-node` / `add-process` find-or-create a def by name **within the model's element-set**, so
   pass `add-experiment --element-set <id>` to make a family of experiments **share one element-set** and
   thus one def per concept (verify: distinct defs ≪ node count).
   **A node name is the KIND of thing, never its specifics.** Strip every specific out of the name and
   move it into a variable: `C57BL/6 male mouse` → `mouse` (+ `strain`, `sex`, `species`);
   `aged mouse tail-tip fibroblasts` → `fibroblast` (+ `tissue source`, `donor age state`, `species`);
   `MeV-derived iPSC clones` → `iPSC clone` (+ `reprogramming vector`);
   `S. cerevisiae ancestor strain (RWY12/BY4741, tlc1-delta)` → `yeast strain` (+ `strain background`,
   `telomere genotype`). **Drop parentheticals** that encode sample size / provenance / genotype
   (`(811 single cells)`, `(all cell types)`, `(humanized line)`) — put that in the node's DEFINITION.

3. **Variable = a grounded QUALITY + a reusable VALUE-SPECIFICATION.** (`add-variable`)
   - **Quality** (`ooevv-quality`, `ensure-quality`) = *what* is measured — the semantic anchor. Ground
     **only** with a verified, definition-matching curie (PATO/EFO); otherwise leave it **ungrounded**

From sciknow-io/alhazen-skill-deep-research, MIT.

More from sciknow-io/alhazen-skill-deep-research