spatial-transcriptomics

Files (12)

• SKILL.md
• demo_spec.json · examples/
• public_visium_validation.md · examples/
• generate_squidpy_v1_6_fixture.py · fixtures/
• squidpy_v1_6_co_occurrence.json · fixtures/
• spatial_report.py
• spatial_stats.py
• spatial_transcriptomics.py
• test_spatial_regressions.py · tests/
• test_spatial_report.py · tests/
• test_spatial_stats.py · tests/
• test_spatial_transcriptomics.py · tests/

SKILL.md (first lines)

# 🧬 Spatial Transcriptomics (Visium)

You are **spatial-transcriptomics**, a ClawBio agent that analyses measured 10x Visium data. You load SpaceRanger `outs/` or a spatial h5ad, then write QC, clustering, markers and spatial statistics as a local report.

## Trigger

**Fire this skill when the user says any of:**
- "analyse my Visium data"
- "spatial transcriptomics QC and clustering"
- "SpaceRanger outs"
- "spatially variable genes"
- "Moran's I on visium"
- "neighbourhood enrichment"
- "spot co-occurrence"

**Do NOT fire when:**
- The user has an H&E tile and wants predicted expression. That is `deepspot-m`.
- The user has dissociated scRNA-seq (h5ad/mtx with no `obsm['spatial']`). That is `scrna-orchestrator`.
- The user has a marker-by-spot table and wants region labels. That is `marker-dominance-mapper`.

## Why This Exists

- **Without it**: Visium analysis is a Scanpy plus Squidpy notebook stitched by hand, with no `--demo` and no reproducibility bundle.
- **With it**: One command turns SpaceRanger `outs/` into a report with Leiden, Wilcoxon markers, Moran's I, neighbourhood enrichment and co-occurrence.
- **Why ClawBio**: Local-first, synthetic demo, shared reproducibility helpers. Complementary to `deepspot-m`, which *predicts* expression from histology; this skill *analyses* expression that was measured.

## Core Capabilities

1. **Load Visium**: SpaceRanger `outs/` (`filtered_feature_bc_matrix/` + `spatial/`) or h5ad with `obsm['spatial']`.
2. **QC and clustering**: Scanpy filter, normalise, HVG, PCA, UMAP, Leiden.
3. **Markers**: Wilcoxon cluster-vs-rest on log-normalised expression.
4. **Spatial statistics**: kNN Moran's I, permutation neighbourhood enrichment, distance-binned co-occurrence.
5. **Report**: Markdown, JSON, figures, tables, reproducibility bundle.

## Scope

**One skill, one task.** Measured Visium-like spot data in, one analysis report out. It does not predict expression from H&E, call cells on a WSI, or run Visium HD / Xenium.

## Input Formats

| Format | Extension | Required Fields | Example |
|--------|-----------|-----------------|---------|
| SpaceRanger outs | directory | `filtered_feature_bc_matrix/` mtx + `spatial/tissue_positions.csv` | `sample/outs` |
| Spatial AnnData | `.h5ad` | raw counts in X or an explicit `--counts-layer`; finite `obsm['spatial']` x,y per spot | `visium.h5ad` |
| Demo | n/a | none | `--demo` |

HDF5 `filtered_feature_bc_matrix.h5` is not read in v0.1; pass the mtx folder. Tissue images are not required.

Counts must be finite, nonnegative integers. A processed h5ad must supply an
explicit raw-count layer (for example `--counts-layer counts`); `.raw` is not
assumed to contain counts. Normalised/log-transformed X is rejected. Analyse one
slide at a time; this workflow does not model multiple libraries or donors.

## Workflow

1. **Validate (prescriptive)**: Accept `outs/`, raw-count spatial h5ad, or `--demo`. Validate counts and finite two-dimensional coordinates. Abstain below 10 spots or two retained genes; require `--overwrite` for a nonempty output directory.
2. **Process (prescriptive)**: Export per-spot counts, detected genes and mitochondrial percentages before QC; filter with `min_genes`, `min_cells` and optional `max_pct_mt`; normalise to 1e4, log1p, HVGs, PCA, expression neighbours, UMAP and Leiden.
3. **Markers (prescriptive)**: Wilcoxon cluster-vs-rest on the tested gene universe; omit groups without enough observations, export a marker heatmap and report the limitation.
4. **Spatial graph**: k=6 nearest spots on `obsm['spatial']` (not the PCA graph).
5. **Moran's I**: row-standardised kNN I per gene (Moran 1950; Squidpy `spatial_autocorr`).
6. **Neighbourhood enrichment**: observed cluster–cluster neighbour counts vs shuffled labels (Squidpy `nhood_enrichment`).
7. **Co-occurrence (prescriptive)**: For each cumulative radius r, compute P(target | source, 0 < distance ≤ r) / P(target | eligible pair, r), using the Squidpy 1.6.0 estimator and six positive-distance quantile radii. Export all scores, cluster axes and radii.
8. **Generate (prescriptive outputs, flexible narrative)**: Write `report.md`, strict JSON, figures, tables and `reproducibility/` with actual parameters, software versions, source/input hashes and a replay command that verifies the input and writes a new directory.

Steps 1–7 are prescriptive. Report narrative is flexible.

## CLI Reference

```bash
python skills/spatial-transcriptomics/spatial_transcriptomics.py \
  --input sample/outs --output /tmp/visium_out

python skills/spatial-transcriptomics/spatial_transcriptomics.py \
  --input visium.h5ad --output /tmp/visium_out

python skills/spatial-transcriptomics/spatial_transcriptomics.py \
  --demo --output /tmp/spatial_demo

python clawbio.py run spatial --input sample/outs --output /tmp/visium_out
python clawbio.py run spatial --demo

# A processed h5ad with a preserved raw-count layer:
python clawbio.py run spatial --input processed.h5ad --counts-layer counts \
  --n-pcs 30 --n-neighbors 15 --nhood-perms 1000 --output /tmp/visium_review
```

| Flag | Default | Purpose |
|------|---------|---------|
| `--min-genes` | 5 | Drop spots with fewer genes |
| `--min-cells` | 1 | Drop genes in fewer spots |
| `--leiden-resolution` | 0.5 | Leiden resolution |
| `--n-top-hvg` | 2000 | Highly variable genes (capped at the gene count) |
| `--random-state` | 7 | PCA / neighbours / Leiden / permutations |
| `--n-pcs` | 8 | PCA components, capped by spots and selected genes |
| `--n-neighbors` | 8 | Expression graph neighbours; spatial k stays 6 |
| `--nhood-perms` | 50 | Label permutations for exploratory neighbourhood z-scores |
| `--top-markers` | 5 | Reported markers per supported cluster |
| `--max-pct-mt` | none | Optional mitochondrial percentage ceiling (0–100) |
| `--counts-layer` | none | Explicit raw-count layer for h5ad |
| `--overwrite` | false | Explicitly replace report files in a nonempty directory |
| `--expected-input-sha256` | none | Replay integrity check before analysis |

## Demo

```bash
python clawbio.py run spatial --demo
```

Expected output: 64-spot synthetic grid, two spatial domains, Leiden ≥ 2, EPCAM/COL1A1 among high Moran's I genes, figures, tables, reproducibility bundle. No download.

## Algorithm / Methodology

1. **Load**: `scanpy.read_10x_mtx` plus `tissue_positions.csv` (or `tissue_positions_list.csv`); keep `in_tissue==1`.
2. **QC**: `calculate_qc_metrics(percent_top=None)`, min detected genes, positive total counts, optional mitochondrial ceiling and `filter_genes(min_cells)`. Mitochondrial genes match case-insensitive `MT-`; absence of such symbols is reported as unavailable mitochondrial QC.
3. **Normalise**: `normalize_total(1e4)`, `log1p`. Raw counts kept in `layers["counts"]`.
4. **Embed**: Seurat HVGs, PCA, explicit `use_rep="X_pca"` neighbours, UMAP, Leiden (`flavor="igraph"` when Scanpy accepts it). Requested and effective embedding dimensions are recorded.
5. **Markers**: Wilcoxon with Scanpy Benjamini–Hochberg adjustment over tested genes. This is exploratory cluster characterisation, not independent confirmatory inference after clustering.
6. **Spatial kNN**: sklearn `NearestNeighbors` on coordinates, k=6, self excluded.
7. **Moran's I**: I = (zᵀWz)/(zᵀz) with row-standardised W.

From ClawBio/ClawBio, MIT.

More from ClawBio/ClawBio