deepspot-m

Files (6)

• SKILL.md
• deepspot_m.py
• demo_expression.json · examples/
• demo_tile.png · examples/
• requirements.txt
• test_deepspot_m.py · tests/

SKILL.md (first lines)

# 🧬 DeepSpot-M Virtual Spatial Transcriptomics

You are **deepspot-m**, a specialised ClawBio agent that turns an H&E histology tile into virtual spatial transcriptomics. You score one 224x224 tile with the DeepSpot-M foundation model and report per-gene log1p-CPM values for the gene symbols the user names.

## Trigger

**Fire this skill when the user says any of:**
- "virtual spatial transcriptomics"
- "predict gene expression from histology"
- "spatial transcriptomics from H&E"
- "what genes are expressed in this tissue image"
- "score this tile for BRAF and COL1A1"
- "run DeepSpot-M on this tile"
- "gene expression map from a slide"
- "H&E to transcriptome"

**Do NOT fire when:**
- The user wants cells counted or outlined in an image. That is `cell-detection`.
- The user already has a measured spot-count table and wants region labels. That is `marker-dominance-mapper`.
- The user wants differential expression between conditions from a count matrix. That is `rnaseq-de`.
- The user wants single-cell clustering or embedding of an AnnData object. That is `scrna-orchestrator` or `scrna-embedding`.
- The user asks for TCGA bulk expression lookups. That is `xena-tcga-gene-query`.

## Why This Exists

- **Without it**: Reading expression off an archived slide means running a spatial assay on the tissue, which most samples never get.
- **With it**: One archived H&E tile yields per-gene values in one command, entirely on the local machine.
- **Why ClawBio**: The call goes to a published model with released weights, pinned to one checkpoint, and every run leaves a reproducibility bundle behind.

This is a research tool, not a substitute for measurement. The model card publishes no per-gene accuracy figure, and neither does the preprint abstract, so this skill quotes none. Read the preprint for the evaluation before treating any number here as a finding, and see `## Safety` for the limitations upstream states.

## Core Capabilities

1. **Score a tile**: Map one 224x224 H&E tile to per-gene log1p-CPM values.
2. **Query genes**: Ask for any HGNC symbols in the released panel and get only those, which is faster than scoring the whole transcriptome.
3. **Choose an embedding source**: Route gene queries through Evo 2, Orthrus, ProtT5, scGPT or Apertus embeddings.
4. **Check the tile**: Flag tiles that are near-white background or essentially colourless before reporting numbers for them.
5. **Report**: Write `report.md`, `result.json`, a gene CSV and a reproducibility bundle.

## Scope

One skill, one task. This skill scores a single H&E tile and writes gene values. It does not read whole-slide images, tile them, register sections, call cells, or compute spatial statistics. For a whole slide, tile it first and call this skill per tile, or use `examples/predict_wsi.py` from the upstream repository.

## Input Formats

| Format | Extension | Required Properties | Example |
|--------|-----------|---------------------|---------|
| PNG | `.png` | Exactly 224x224 px, H&E stained | `examples/demo_tile.png` |
| JPEG | `.jpg`, `.jpeg` | Exactly 224x224 px, H&E stained | `tile.jpg` |
| TIFF | `.tif`, `.tiff` | Exactly 224x224 px, H&E stained | `tile.tif` |

Tiles must be exactly 224x224 pixels. The skill checks the dimensions and stops with an explicit message when they differ. Upstream cuts tiles on a 224-pixel grid at native (~20x) resolution (source: upstream README, `### Command line`).

**On microns per pixel**: no microns-per-pixel or magnification figure appears on the model card, and the only magnification upstream states anywhere is the "~20x" above. So the skill never assumes a pixel size. It reads one from the file's own resolution tags when they carry a plausible microscopy value, accepts one you declare with `--mpp`, and otherwise records `null` and prints "not declared". When a declared value and the file's tags disagree, the declared value wins and the report says the tags disagreed. A pixel size outside 0.4-0.6 gets one warning, on stderr and in the report: 224x224 is a pixel count and not a field of view, so a 40x tile passes the dimension check while covering a quarter of the tissue. That band is what a ~20x scan typically produces on a slide scanner, not a figure from the model card, and the run is scored either way.

## Workflow

1. **Validate**: Confirm the tile is exactly 224x224 pixels and load it.
2. **Check the tile**: Measure mean pixel value and mean saturation. Warn on near-white background or a near-greyscale tile; with `--skip-background`, refuse to score it.
3. **Resolve scale**: Read microns per pixel from resolution tags or `--mpp`; record `null` when neither exists.
4. **Resolve genes**: Deduplicate the requested HGNC symbols case-insensitively, preserving spelling. With no `--genes` flag, use the bundled ten gene marker panel.
5. **Load model**: Call `DeepSpotM.from_pretrained("ratschlab/DeepSpotM", source=..., revision=...)` against the pinned checkpoint, from the local cache unless `--allow-download` is passed.
6. **Match to the panel**: Case-fold the requested symbols against `model.gene_names` and carry forward the panel's own spelling.
7. **Predict**: Run `model.predict_genes(image_processor(tile).unsqueeze(0), genes)`.
8. **Report**: Write `report.md`, `result.json`, `tables/gene_expression.csv` and the reproducibility bundle, in requested gene order.

Steps 1, 5, 6 and 7 are prescriptive. Do not substitute another tile size, another checkpoint, or a different call signature. Step 8 narrative is open to the agent.

## CLI Reference

```bash
# Standard usage
python skills/deepspot-m/deepspot_m.py \
  --input tile.png --output /tmp/deepspot_out

# Named genes and a chosen embedding source
python skills/deepspot-m/deepspot_m.py \
  --input tile.png --genes BRAF,CD37,COL1A1 --source evo2 --output /tmp/deepspot_out

# Declare the tile's pixel size, and permit the one-time gated weight download
python skills/deepspot-m/deepspot_m.py \
  --input tile.tif --mpp 0.5 --allow-download --output /tmp/deepspot_out

# Refuse to score a background tile rather than warning about it
python skills/deepspot-m/deepspot_m.py \
  --input tile.png --skip-background --output /tmp/deepspot_out

# Demo mode (offline fixture, no weights needed)
python skills/deepspot-m/deepspot_m.py --demo --output /tmp/deepspot_demo

# Via the ClawBio runner
python clawbio.py run deepspot-m --input tile.png --genes BRAF,CD37
python clawbio.py run deepspot-m --demo
```

| Flag | Default | Purpose |
|------|---------|---------|
| `--genes` | 10 gene marker panel | Comma separated HGNC symbols to score |
| `--source` | `scgpt` | Frozen gene embedding space |
| `--mpp` | unset | Declared microns per pixel; recorded, never assumed. Outside 0.4-0.6 the run warns that the field of view does not read as ~20x, and scores anyway |
| `--skip-background` | off | Refuse rather than warn when a tile fails the checks |
| `--white-mean` | `220` | Mean pixel above which a tile counts as background (upstream's default) |
| `--min-saturation` | `0.05` | Mean HSV saturation below which a tile is flagged as not H&E |
| `--allow-download` | off | Permit the one-time gated weight fetch from Hugging Face |

## Demo

```bash
python clawbio.py run deepspot-m --demo
```

Expected output: a ten gene report over the bundled synthetic H&E tile, tagged "(demo)", with a CSV and a full reproducibility bundle. Demo mode reads `examples/demo_expression.json` instead of the model, so it runs with no weights, no GPU and no network.

## Algorithm / Methodology

DeepSpot-M is a multimodal foundation model that maps a histology tile to spatial gene expression.

1. **Tokenise**: A LoRA-adapted Midnight pathology backbone turns the 224x224 tile into spatial patch tokens.
2. **Attend**: A cross-attention gene decoder lets each gene query attend to those patch tokens through multi-head attention, independently per gene.

From ClawBio/ClawBio, MIT.

More from ClawBio/ClawBio