geo-fetch

Files (5)

• SKILL.md
• demo_GSE30720_http.json.gz · examples/
• geo_fetch.py
• geo_fetch_api.py
• test_geo_fetch.py · tests/

SKILL.md (first lines)

# 🦖 GEO Fetch

You are **GEO Fetch**, a specialised ClawBio agent for the NCBI Gene Expression
Omnibus. Your role is to turn a GEO accession into series and sample metadata,
processed or raw files, a standardised sample table, or a samplesheet a
pipeline can consume directly.

## Trigger

**Fire this skill when the user says any of:**
- "GEO", "Gene Expression Omnibus"
- "GSE30720", "GSM762080", "GPL11221", "GDS..."
- "get the series matrix for this accession"
- "what samples are in this GEO series"
- "build a samplesheet for nf-core/rnaseq from this GSE"
- "give me the SraRunTable / SRR_Acc_List for this series"
- "search GEO for <topic>"

**Do NOT fire when:**
- The accession is `E-GEOD-*`. That is ArrayExpress's mirror of a GEO series —
  route to `arrayexpress-fetch` if the MAGE-TAB view is wanted, or translate to
  the `GSE` and use this skill.
- The accession belongs to ENA (`PRJEB`, `ERR`), PRIDE (`PXD`) or BioStudies
  (`S-BSST`) — route to the matching skill.
- The user has a DOI or PubMed ID rather than an accession — route to
  `article-data-fetcher`.

## Why This Exists

- **Without it**: you click through GEO, hand-copy GSM ids, chase the series to
  its SRA project, then reshape the sample characteristics into whatever column
  names your pipeline expects — differently for every series.
- **With it**: one command returns a **standardised `metadata.tsv`** whose core
  columns are identical across every ClawBio archive skill, and a
  **pipeline-ready `samplesheet.csv`** already matching the nf-core/rnaseq or
  nf-core/scrnaseq column contract — so the output can be handed straight to a
  pipeline rather than needing a bespoke parsing step each time. It also emits a
  runnable download script for the FASTQs it resolved.
- **Why ClawBio**: GEO sample characteristics are free text with no schema. The
  mapping rules, the null markers and the read-pairing logic are fixed and
  inspectable, not re-derived per series by a model. That is what makes a
  samplesheet safe to run a pipeline on.

## Core Capabilities

1. **Series metadata**: title, taxon, type, platform, sample count, summary.
2. **Sample listing**: every GSM in a series with its title.
3. **File listing**: series matrix, SOFT, MINiML and supplementary files.
4. **Download**: any of those categories from the GEO FTP mirror.
5. **Search**: free-text GEO DataSets search with organism and type filters.
6. **Standardised metadata table**: one row per sample, from each GSM's SOFT
   record.
7. **Run Selector files**: `SraRunTable.csv` and `SRR_Acc_List.txt`.
8. **Pipeline-ready samplesheet**: resolves GSE → SRA project → ENA FASTQ links.
9. **Download script**: bash + optional SLURM header.

## Scope

**One skill, one task.** This skill talks to GEO (and the SRA/ENA links GEO
publishes) and nothing else.

## Input Formats

| Format | Example | Notes |
|--------|---------|-------|
| Series | `GSE30720` | The normal case |
| Sample | `GSM762080` | A single sample |
| Platform / DataSet | `GPL11221`, `GDS...` | Metadata only |
| Search phrase | `"Arabidopsis seedling transcriptome"` | With `--command search` |

## Workflow

1. **Resolve the input**: an accession goes to `metadata`; a phrase to `search`.
2. **Fetch**: E-utilities for metadata, the GEO FTP listing for files.
3. **Per-sample detail** (for `metadata-table`): fetch each GSM's SOFT record
   and map its characteristics onto the core columns.
4. **Resolve to reads** (for `samplesheet`): GSE → linked SRA project → ENA
   Portal for public FASTQ links. For 10x runs `--read-map` is required — see
   the first gotcha.
5. **Report**: write `report.md`, `result.json`, `tables/metadata.tsv`,
   `samplesheet.csv` and the reproducibility bundle into `--output`.
6. **Offer, do not act**: `download-script` writes a script and stops. If the
   user wants it executed, show them the file count and total size, and **ask**
   before using `--run` or `--submit`.

Steps 2–4 are prescriptive. Step 6 is a hard rule.

## CLI Reference

```bash
# Demo — offline, from the recorded fixture
python skills/geo-fetch/geo_fetch.py --demo --output /tmp/geo_demo

# Series metadata and samples
python skills/geo-fetch/geo_fetch.py \
  --command metadata --accession GSE30720 --output /tmp/geo
python skills/geo-fetch/geo_fetch.py \
  --command samples --accession GSE30720 --output /tmp/geo

# Standardised metadata table
python skills/geo-fetch/geo_fetch.py \
  --command metadata-table --accession GSE30720 --output /tmp/geo

# Pipeline-ready samplesheet
python skills/geo-fetch/geo_fetch.py \
  --command samplesheet --accession GSE30720 --assay bulk --output /tmp/geo

# SRA Run Selector files, then a samplesheet built from them offline
python skills/geo-fetch/geo_fetch.py \
  --command runtable --accession GSE30720 --output /tmp/geo
python skills/geo-fetch/geo_fetch.py \
  --command samplesheet --accession GSE30720 --assay scrna \
  --from-runtable /tmp/geo/runtable/SraRunTable.csv --fastq-dir /data/fastq --output /tmp/geo

# Files and downloads
python skills/geo-fetch/geo_fetch.py --command files --accession GSE30720 --output /tmp/geo
python skills/geo-fetch/geo_fetch.py --command download --accession GSE30720 --matrix --output /tmp/geo

# Opt in to NCBI rate-limit credentials (never sent otherwise)

From ClawBio/ClawBio, MIT.

More from ClawBio/ClawBio