meta-analysis

Files (60)

• SKILL.md
• LICENSES.md · references/
• PROSPERO_template.md · references/
• ai_pre_screening_template.py · references/
• JBI_Case_Series.md · references/checklists/
• NOS.md · references/checklists/
• PRISMA_DTA.md · references/checklists/
• PROBAST.md · references/checklists/
• QUADAS2.md · references/checklists/
• QUADAS3.md · references/checklists/
• ROBINS_I.md · references/checklists/
• RoB2.md · references/checklists/
• data_integrity_checklist.md · references/
• empirical_lessons.md · references/
• icmje_coi_guide.md · references/
• phase10_recovery.md · references/
• phase3_screening_detail.md · references/
• phase4_extraction_detail.md · references/
• phase4_km_composite.md · references/
• phase6_statistical_synthesis.md · references/
• phase9_circulation.md · references/
• post_submission_release_ops.md · references/
• r_templates.md · references/
• review_orchestration.md · references/
• single_arm_proportion_ma.md · references/
• submission_package_drift.md · references/
• check_exclusion_code_validity.py · scripts/
• protocol_negative.md · scripts/check_exclusion_code_validity_challenge/fixture/
• protocol_positive.md · scripts/check_exclusion_code_validity_challenge/fixture/
• screening_negative.tsv · scripts/check_exclusion_code_validity_challenge/fixture/
• screening_positive.tsv · scripts/check_exclusion_code_validity_challenge/fixture/
• problem.md · scripts/check_exclusion_code_validity_challenge/
• verify.sh · scripts/check_exclusion_code_validity_challenge/
• check_pool_consistency.py · scripts/
• check_ratio_ci_symmetry.py · scripts/
• extraction_negative.csv · scripts/check_ratio_ci_symmetry_challenge/fixture/
• extraction_positive.csv · scripts/check_ratio_ci_symmetry_challenge/fixture/
• problem.md · scripts/check_ratio_ci_symmetry_challenge/
• verify.sh · scripts/check_ratio_ci_symmetry_challenge/
• cohort_overlap_check.py · scripts/
• dta_extraction_qc.py · scripts/
• extract_assist.py · scripts/
• suggestions.tsv · scripts/extract_assist_challenge/expected/
• confirmed_example.csv · scripts/extract_assist_challenge/fixture/
• paper.md · scripts/extract_assist_challenge/fixture/
• schema.yaml · scripts/extract_assist_challenge/fixture/
• problem.md · scripts/extract_assist_challenge/
• verify.sh · scripts/extract_assist_challenge/
• extraction_consensus_log_init.py · scripts/
• prisma_5way_consistency.py · scripts/
• screening_reconcile.py · scripts/
• tag_cleanup_gate.sh · scripts/
• skill.yml
• FINAL_POOL_LOCK.yaml.template · templates/
• extraction_form_v2.md · templates/
• supplementary_8file_checklist.md · templates/
• test_pool_consistency.sh · tests/
• test_prisma_5way_consistency.sh · tests/
• test_ratio_ci_symmetry.sh · tests/
• test_screening_reconcile.sh · tests/

SKILL.md (first lines)

# Meta-Analysis Skill

## Meta-Analysis Types

| Type | RoB Tool | Statistical Model | Reporting Guideline |
|------|----------|-------------------|-------------------|
| **DTA** (diagnostic test accuracy) | **QUADAS-3** (QUADAS-2 for legacy reviews) | Bivariate / HSROC | PRISMA-DTA |
| **Intervention** (treatment effect) | RoB 2 (RCT) / ROBINS-I (NRSI) | Random-effects (REML, Hartung-Knapp CI) | PRISMA 2020 |
| **Prognostic factor** (association of one factor with outcome) | QUIPS | Random-effects | PRISMA 2020 |
| **Prediction model** (model performance) | PROBAST | Random-effects | PRISMA 2020 |
| **Observational** (prevalence/association) | NOS / JBI | Random-effects | MOOSE |

If the type is ambiguous (DTA vs intervention), ask the user to clarify before proceeding.

---

## Workflow Phases

### Phase 1: Protocol Development

**Goal**: Produce a PROSPERO-ready protocol document. Write the protocol, extraction forms and
manuscript text in English whatever language the user writes in — PROSPERO records and the
target journals are English-language.

1. **Research question**: PIRD (Population, Index test, Reference standard, Diagnosis) for DTA;
   PICO (Population, Intervention, Comparator, Outcome) for intervention.

2. **DTA only — do QUADAS-3 phases 1 and 2 now, not at risk-of-bias time.** They are
   review-level and belong in the protocol: phase 1 states the **synthesis question(s)**
   (population, index test(s), target condition — a review may have more than one); phase 2
   defines the **ideal test accuracy trial** for each (objective, participants, index test(s),
   definition of the target condition, analysis). Every later risk-of-bias and applicability
   judgement is made against that trial. Write the review-specific guidance for answering each
   signalling question here too, with clinical **and** methodological input, and publish it as a
   web appendix. Defining the ideal trial after seeing the studies is a judgement fitted to the
   results, not an assessment. See `references/checklists/QUADAS3.md`.

3. **Eligibility criteria**: study design, population, index test / intervention, comparator /
   reference standard, outcomes (Se/Sp for DTA; effect size for intervention), and exclusion
   criteria with justification.

4. **Search plan**: at least 3 databases — PubMed, Embase, and Cochrane CENTRAL (add Scopus / Web
   of Science as needed); a Boolean strategy from the PIRD/PICO components; a grey-literature plan
   (conference abstracts, trial registries); language restrictions and date range stated
   explicitly, with justification.

5. **RoB plan**: tool by type (table above), at least 2 independent assessors, and the
   disagreement-resolution method (consensus, third reviewer).

6. **Synthesis plan**: bivariate random-effects (Reitsma) or HSROC (Rutter & Gatsonis) for DTA;
   random-effects for intervention (Phase 6); heterogeneity, subgroup / sensitivity, and
   publication-bias plans.

7. **PROSPERO registration document**: read `${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md`
   and follow its field guide, word limits, output format (Markdown + DOCX via pandoc), and Common
   Pitfalls Checklist. Save to the project's `7_Submission/` or equivalent directory.
   - **Submit before the formal search, and record the date.** The user submits the registration;
     record the initial submission date (not the later date the record is published) in the
     project's status file (`STATUS.md`, `project_state.json` or `project.yaml`, whichever the
     project keeps). The Phase 2 entry check looks for it. A pilot search is fine before
     submission; the formal search, screening and extraction, AI first-pass drafts included, are not.
   - **Registration-ID format gate.** A PROSPERO ID is `CRD42` + 9 digits (14 characters total),
     e.g. `CRD42024500001`. Validate any ID that appears in the manuscript or registration doc with
     `grep -oE 'CRD42[0-9]+'` and assert a 14-character length / `^CRD42\d{9}$` — a 15-character ID
     (a stray digit) is a transcription error a reviewer will check against the live record.
   - **Review-type selection.** Pick the *least-wrong* portal review type for the actual design and
     state any portal constraint in the protocol. A descriptive single-arm proportion synthesis is
     not an "Intervention review"; choosing that type only to satisfy a portal field contradicts a
     later GRADE / effect-certainty statement. Whatever certainty language the protocol commits to
     (GRADE vs "evidence statements only") must match the manuscript verbatim — a guideline-style
     "we recommend" is not licensed by a descriptive review type.

### Phase 2: Search Strategy

**Goal**: Develop and validate reproducible search strategies.

**Entry check — registration before the formal search.** Look for the registration submission
date recorded in Phase 1 step 7. If none is recorded, stop and ask the user to choose: submit the
registration now and run the search after that date, or proceed without it, in which case Phase 8
item 5 limits what the manuscript may claim. Record the choice. Screening or extraction done before
submission, AI first-pass drafts included, cannot be made prospective afterwards. Under
`/orchestrate --e2e` this stop is the `REGISTRATION_PENDING` halt; no autonomous default passes it.

1. Build search blocks from the PIRD/PICO components and execute them per database with
   `/search-lit` (PubMed: MeSH + free text; Embase: Emtree + free text; further databases as the
   protocol specifies). Searching only: selecting records and extracting results belong to
   Phases 3–4, not to `/search-lit`'s own selection and deep-read steps.
2. **Report per PRISMA-S** (Rethlefsen et al. 2021, PMID:33499930): one section per database with
   date of search, number of results, and any limits applied.
3. **Merge and deduplicate** into a single spreadsheet: by DOI first, then PMID. Save raw counts
   for the PRISMA flow.

### Phase 3: Screening & Selection

**Goal**: Systematic title/abstract and full-text screening with two independent reviewers. Read
`references/phase3_screening_detail.md` when executing a round (exclusion-code sets, AI pre-screening
template and Methods boilerplate) or when a 3f/3f.5 gate fires (set algebra, reconciliation table).

**3a. Round 1 — title/abstract (single reviewer).** Define the exclusion codes from the protocol.
Mark every record INCLUDE / EXCLUDE / MAYBE with a reason code → `round1_{date}.tsv`.

**3b. Round 2 — dual independent title/abstract.** A second independent reviewer (or AI as a
*documented* second-pass tool with human verification) re-screens all R1 records. Report Cohen's κ
in Methods. `round2_tag` = INCLUDE / EXCLUDE / MAYBE (MAYBE = disagreement **or** either reviewer
flagged uncertainty), plus `round2_reason`.

**3c. Round 3 — adjudication (first reviewer).** MAYBE records first, then INCLUDE records for a
brief confirmation pass → `round3_decision` (plus `round3_reason` only when overturning R2).
Optional AI pre-screening may compress the effort, but **AI suggestions are not decisions**: the
reviewer independently confirms or overturns every one.

**3d. Round 4 — full text** (`/fulltext-retrieval`) for `round3_decision = INCLUDE`: full-text
exclusion codes, two independent reviewers, Cohen's κ, consensus or a third reviewer. Flag
comparative studies for priority extraction.

**3e. PRISMA flow.** Track counts at every stage (R1 → R2 → R3 → R4 → final included); draw it with
`/make-figures` once final.

**3f. Post-consensus count reconciliation gate (MANDATORY before Phase 5 write-up).** Reconcile

From Aperivue/medsci-skills, MIT.

More from Aperivue/medsci-skills