self-review

Files (200)

• SKILL.md
• ai_overclaiming.md · references/domain-probes/
• case_report.md · references/domain-probes/
• clinical_prediction_model.md · references/domain-probes/
• diagnostic_accuracy.md · references/domain-probes/
• equity_fairness.md · references/domain-probes/
• health_economic_evaluation.md · references/domain-probes/
• image_synthesis.md · references/domain-probes/
• mendelian_randomization.md · references/domain-probes/
• mllm_evaluation.md · references/domain-probes/
• model_development.md · references/domain-probes/
• narrative_review.md · references/domain-probes/
• network_meta_analysis.md · references/domain-probes/
• observational_confounding.md · references/domain-probes/
• polygenic_risk_score.md · references/domain-probes/
• qualitative_research.md · references/domain-probes/
• radiomics.md · references/domain-probes/
• rct_trial.md · references/domain-probes/
• record_routinely_collected_data.md · references/domain-probes/
• scoping_review.md · references/domain-probes/
• self_improving_system.md · references/domain-probes/
• sr_ma.md · references/domain-probes/
• survey_research.md · references/domain-probes/
• survival_prognostic.md · references/domain-probes/
• evalues_schema.md · references/
• README.md · references/exemplar_findings/
• cohort_arithmetic_mismatch.md · references/exemplar_findings/
• cross_sectional_mediation.md · references/exemplar_findings/
• estimand_drift_posthoc_primary.md · references/exemplar_findings/
• over_adjustment_collider.md · references/exemplar_findings/
• prediction_two_null_conflation.md · references/exemplar_findings/
• scope_overreach_cross_sectional.md · references/exemplar_findings/
• unadjusted_confounder.md · references/exemplar_findings/
• p_tests_schema.md · references/
• panel_review_template.md · references/
• confounding_completeness.md · references/phases/
• phase2_5a2_design_power.md · references/phases/
• phase2_5a_source_fidelity.md · references/phases/
• phase2_5b_screening_counts.md · references/phases/
• phase2_5c_reference_scans.md · references/phases/
• phase2_5d_xref_qc.md · references/phases/
• phase2_5f_claim_artifact.md · references/phases/
• phase2_6_panel.md · references/phases/
• phase2_systematic_check.md · references/phases/
• phase3c_json_output.md · references/phases/
• phase4_fix_support.md · references/phases/
• _frontmatter.py · scripts/
• _prose.py · scripts/
• _qc_findings.py · scripts/
• check_analysis_definitions.py · scripts/
• defined.txt · scripts/check_analysis_definitions_challenge/expected/
• undefined.txt · scripts/check_analysis_definitions_challenge/expected/
• defined.md · scripts/check_analysis_definitions_challenge/fixture/
• undefined.md · scripts/check_analysis_definitions_challenge/fixture/
• problem.md · scripts/check_analysis_definitions_challenge/
• verify.sh · scripts/check_analysis_definitions_challenge/
• check_aphorism_density.py · scripts/
• check_artifact_coverage.py · scripts/
• check_baseline_drift.py · scripts/
• drifted.txt · scripts/check_baseline_drift_challenge/expected/
• …and 140 more

SKILL.md (first lines)

# Self-Review Skill

Check the user's own manuscript before submission and produce an actionable list of anticipated
reviewer comments, each with a specific fix — not a written review.

## Optional Flags

- `--fix`: after the report, apply fixes for every issue with `fixable_by_ai: true`, editing the manuscript in place, then report a diff summary. Never fix `fixable_by_ai: false` issues (missing data, design flaws). Maximum 2 fix-and-re-review iterations; if the score is still below threshold after the second, stop and report what remains as structural (inside `/write-paper` this routes to Phase 7.4a Audit Recovery).
- `--json`: also emit the structured JSON block (Phase 3c). Default when called from `/write-paper` Phase 7.
- `--panel`: run the multi-agent panel review (Phase 2.6) — domain-expert reviewers in parallel plus an editor synthesis — instead of the single-pass review. Opt-in and **off by default**, because it spawns N reviewer agents + 1 editor and costs several times more tokens; reserve it for a high-stakes final pass on a top-tier target. Do **not** combine with `--fix`: a panel diagnoses and prioritizes; run `--fix` as a separate pass once the author has triaged the panel's findings.

## Severity Framing

- **Fatal**: a design flaw existing data cannot fix (data leakage that invalidates all results, no reference standard, label–feature circularity). Submission would likely be rejected. Reserve Fatal for true design-level problems.
- **Fixable**: significant but addressable with existing data (missing calibration, unclear exclusion criteria, absent CIs, incomplete reporting). Most issues are Fixable.

## Two Objectives: the Floor and the Ceiling

The **floor** (categories A–K and the gates of Phases 2.5–2.5f) minimizes rejection-for-cause —
fabricated citations, numbers that do not reconcile, overclaims, missing checklist items, leakage —
and much of it works by **adding** a hedge, caveat, disclosure or audit trail. Iterated, that
over-hardens a manuscript: every finding is correct, yet the whole reads as a defensive audit
(over-hedged, audit trail in the body, Abstract buried under caveats, the strongest robustness
result hidden in Limitations). The **ceiling** pass (category L / Phase 2.5g) runs **after** the
floor gates, reads the accurate manuscript as a whole, and recommends only SUBTRACTION — REMOVE,
MOVE, or TIGHTEN. It is advisory, never blocks, and cannot relax a floor gate. Report its findings
as their own block (Phase 3), not folded into the "add this" comments. Phase 2.5i then reads the
floor + ceiling state to declare the loop done, including a zero-edit PASS.

## Workflow

### Phase 1: Intake

1. Get the manuscript — PDF, Word doc, or pasted text.
2. Ask the user: target journal (sets reporting standards and scope); manuscript type (original research / review / perspective / technical note / letter / meta-analysis / case report); anything they are already worried about. On an interactive run, offer the `--panel` review (Phase 2.6) **once**, in one line, then proceed single-pass unless the user opts in. Never offer or apply the panel under `--json` or when called from `/write-paper`.
3. Read the full manuscript.
4. **SSOT gate — confirm there is one manuscript, not several.** Self-review reads a single file,
   so drift between a legacy working copy and the live submission copy is invisible to it. Before
   a `--panel` run or any pre-submission pass, check for multiple copies:

   ```bash
   find . \( -path '*manuscript*' -o -path '*main_document*' \) -name '*.md' | grep -v node_modules
   ```

   If more than one manuscript-like file exists, confirm which is the SSOT and run
   `/sync-submission`'s divergence gate — a `STALE_COPY` (an SSOT numeric claim or heading that did
   not propagate to the other copy) is a P0 that must clear first:

   ```bash
   python3 "${CLAUDE_SKILL_DIR}/../sync-submission/scripts/detect_copy_divergence.py" \
     --ssot <ssot>.md --copy <other-copy>.md
   ```

   Review the SSOT copy, never a stale one. **Under `--panel` this is blocking:** if `find` returns
   more than one file and the SSOT is not pinned (no `SSOT.yaml` with `truth.manuscript_md`, no
   explicit `--ssot <path>`), STOP before spawning any reviewer and have the user name the SSOT
   (and clear any `STALE_COPY`), because a panel on a stale copy wastes the whole pass. Do not
   auto-pick the longest/newest file. A single-pass review may proceed on the one file it was given.

### Phase 2: Systematic Check

Work every category the Research-Type Adaptation table (below) marks as applicable, and for each
item decide whether a reviewer would raise it as a Major or Minor comment. The per-item check
tables are in `references/phases/phase2_systematic_check.md` — read it once you have the
manuscript and know its type.

| | Category | What it asks |
|---|---|---|
| **A** | Study Design & Data Integrity | patient-level splits, leakage, input-text contamination, analysis unit |
| **B** | Reference Standard & Ground Truth | definition specificity, timing, annotator independence |
| **C** | Validation & Statistical Reporting | CIs, **calibration**, comparator, effect size, power-aware nulls, equivalence margins, interaction anchoring |
| **D** | Clinical Framing & Importance | intended use, overclaiming, novelty, **endpoint↔conclusion scope** |
| **E** | Reproducibility | preprocessing, model detail, hardware/software, data & code availability |
| **F** | Reporting Completeness | abstract↔body consistency, flow diagram, ethics, missing data, word cap |
| **G** | Reporting Guideline Compliance | match the type to its checklist; `/check-reporting` does the item-level audit |
| **H** | Circularity | label–feature overlap, tautological prediction, circular validation |
| **I** | Protocol Heterogeneity | multi-site acquisition, harmonization, temporal protocol drift |
| **J** | Method Transparency | model provenance, fine-tuning, classical-style body conventions |
| **K** | Reviewer-team consistency | *SR/MA only* — dual-vs-single conjunction, LLM-as-reviewer (both fabrication-grade) |
| **L** | Editorial impression & defensiveness | *advisory, never blocking* — the ceiling category: REMOVE / MOVE / TIGHTEN |

**Run the deterministic gates at Phase 2 entry, on every path:**

```bash
# D. endpoint↔conclusion scope
python3 "${CLAUDE_SKILL_DIR}/scripts/check_scope_coherence.py" \
  --manuscript manuscript.md --out qc/scope_coherence.json --strict

# J. classical-style body conventions
python3 "${CLAUDE_SKILL_DIR}/scripts/check_classical_style.py" \
  --manuscript manuscript.md --out qc/classical_style.json --strict

# K. reviewer-team consistency (SR/MA only; pass the extraction JSON file or directory)
python "${CLAUDE_SKILL_DIR}/scripts/check_reviewer_team_consistency.py" \
    --manuscript manuscript.md --prospero prospero/record.md \
    --extraction-json extraction/ --out _audit_self/reviewer_team_consistency.md

# J/D. Perspective structure (genre-gated: silent unless article_type is a Perspective).
# Pass the known type via --type; it also self-detects from the front-matter article_type.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_perspective_structure.py" \
  --manuscript manuscript.md --type "${TYPE:-}" --out qc/perspective_structure.json

# Was every reported analysis ever defined (outcome, reference standard)?
python3 "${CLAUDE_SKILL_DIR}/scripts/check_analysis_definitions.py" \
  --manuscript manuscript.md --json --strict > qc/analysis_definitions.json
```

Verdict mapping: `CROSS_SECTIONAL_PROGNOSTIC`, `SURROGATE_CARE_DIRECTIVE`, `SECTION_SYMBOL`,
`INBODY_AI_DISCLOSURE`, any reviewer-team hit (exit 1), `MODEL_OUTCOME_UNDEFINED` (a Cox /
Fine–Gray / logistic model with no outcome named), `MODEL_NOT_IN_METHODS`, and
`REFERENCE_STANDARD_UNDEFINED` (discrimination or calibration with nothing to score against) are
Anticipated **Major** Comments. `CROSS_SECTIONAL_YIELD_LANGUAGE`, `ELIGIBILITY_PROSE`,
`DECIMAL_INCONSISTENCY`, `EM_DASH_OVERUSE`, `PERSPECTIVE_HEADING_NOT_ASSERTION`,
`PERSPECTIVE_ABSTRACT_NO_AUTHORIAL_MOVE`, and `TIER_LABEL_UNDEFINED` are **Minor**. The
per-verdict rationale and resolution paths are in the Phase 2 reference file.

`ANALYSIS_LOAD` is **informational, never a verdict**: many analyses are the cause of omitted
definitions, not the defect. Do not cut analyses to satisfy it; restore the definitions they
crowded out, and if load is genuinely high, move the defensive analyses to the supplement.

From Aperivue/medsci-skills, MIT.

More from Aperivue/medsci-skills