mat-synthesis-extraction

Files (3)

• SKILL.md
• README.md · examples/pt-cu-alloy-co-oxidation/
• parse_pdfs.py · scripts/

SKILL.md (first lines)

# mat-synthesis-extraction

## Goal

Given a folder of scientific paper PDFs, extract all synthesis procedures described in each paper and structure them according to the **GeneralSynthesisOntology** developed in LeMat-Synth [1]. Output is one JSON file per paper containing a list of per-material synthesis records.

The ontology captures: target compound, compound type, synthesis method, starting materials (with amounts/units/purity), sequential process steps (with actions, conditions, equipment), and overall equipment list.

---

## Instructions

### Step 1 — Parse PDFs to text

Run the PDF parser to extract plain text from all PDFs in the input folder.

```bash
${CLAUDE_SKILL_DIR}/../../venv/run cpu python ${CLAUDE_SKILL_DIR}/scripts/parse_pdfs.py \
    --pdf-dir /path/to/pdf_folder \
    --output-dir /path/to/output/texts
```

This produces:
- One `.txt` file per PDF (named `<paper_stem>.txt`)
- A `parse_summary.json` listing extraction status and character counts

Inspect `parse_summary.json` to confirm all PDFs extracted successfully. Papers with `"status": "empty"` are likely scanned images — skip them or obtain a text-layer PDF.

---

### Step 2 — Extract synthesized material names

For each `.txt` file produced in Step 1, identify which materials are synthesized in the paper. Read the paper text and extract a comma-separated list of synthesized compound names.

**System prompt to use:**
```
You are a materials science expert. Given the full text of a scientific paper, identify ALL distinct materials that are synthesized (not just characterized or used as reagents). Return ONLY a comma-separated list of chemical names or formulas (e.g. "NiCo2O4, CoFe2O4, Fe3O4"). If no synthesis is described, return an empty string.
```

**Input:** full paper text from Step 1
**Output:** comma-separated string of material names → split into a Python list

---

### Step 3 — Extract GeneralSynthesisOntology per material

For each (paper_text, material_name) pair from Step 2, extract the structured synthesis ontology. Use the system prompt and JSON schema below.

**System prompt:**
```
You are a helpful assistant that extracts the structured synthesis for a specific material from the paper text.

Focus ONLY on the synthesis procedure for the specified material. Search through the entire paper text to find the synthesis procedure that describes how this specific material is made.

IMPORTANT: You must output ONLY a valid JSON object with a "structured_synthesis" field. Do not include any reasoning, explanations, or markdown formatting.

If you cannot find a synthesis procedure for the specified material, return a minimal structure with the material name and an empty synthesis.
```

**User message template:**
```
Paper text:
<PAPER_TEXT>

Extract the synthesis procedure for: <MATERIAL_NAME>
```

**Required JSON output schema (GeneralSynthesisOntology):**
```json
{
  "structured_synthesis": {
    "target_compound": "string (required) — composition and description of the target",
    "target_compound_type": "one of: 'metals & alloys' | 'ceramics & glasses' | 'polymers & soft matter' | 'composites' | 'semiconductors & electronic' | 'nanomaterials' | 'two-dimensional materials' | 'framework & porous materials' | 'biomaterials & biological' | 'liquid materials' | 'hybrid & organic-inorganic' | 'functional materials & catalysts' | 'energy & sustainability' | 'smart & responsive materials' | 'emerging & quantum materials' | 'other'",
    "synthesis_method": "one of: 'PVD' | 'CVD' | 'arc discharge' | 'ball milling' | 'spray pyrolysis' | 'electrospinning' | 'sol-gel' | 'hydrothermal' | 'solvothermal' | 'precipitation' | 'coprecipitation' | 'combustion' | 'microwave-assisted' | 'sonochemical' | 'template-directed' | 'solid-state' | 'flux growth' | 'float zone & Bridgman' | 'arc melting & induction melting' | 'spark plasma sintering' | 'electrochemical deposition' | 'chemical bath deposition' | 'liquid-phase epitaxy' | 'self-assembly' | 'atomic layer deposition' | 'molecular beam epitaxy' | 'pulsed laser deposition' | 'ion implantation' | 'lithographic patterning' | 'wet impregnation' | 'incipient wetness impregnation' | 'mechanical mixing' | 'solution-based' | 'mechanochemical' | 'other'",
    "starting_materials": [
      {
        "name": "string",
        "amount": "number or null",
        "unit": "string or null — e.g. 'g', 'mL', 'mmol', 'wt%'",
        "purity": "string or null — e.g. '99.9%', 'ACS grade'",
        "vendor": "string or null"
      }
    ],
    "steps": [
      {
        "step_number": "integer",
        "action": "one of: 'add' | 'mix' | 'heat' | 'cool' | 'reflux' | 'age' | 'filter' | 'wash' | 'dry' | 'reduce' | 'calcine' | 'dissolve' | 'precipitate' | 'centrifuge' | 'sonicate' | 'anneal' | 'ion exchange' | 'impregnate'",
        "description": "string or null",
        "materials": [{"name": "string", "amount": "number or null", "unit": "string or null", "purity": "string or null", "vendor": "string or null"}],
        "equipment": [{"name": "string", "instrument_vendor": "string or null", "settings": "string or null"}],
        "conditions": {
          "temperature": "number or null",
          "temp_unit": "string or null — 'C', 'K', or 'F'",
          "duration": "number or null",
          "time_unit": "string or null — 'h', 'min', 's', 'days'",
          "pressure": "number or null",
          "pressure_unit": "string or null",
          "atmosphere": "string or null — e.g. 'air', 'N2', 'Ar'",
          "stirring": "boolean or null",
          "stirring_speed": "number or null",
          "ph": "number or null"
        }
      }
    ],
    "equipment": [{"name": "string", "instrument_vendor": "string or null", "settings": "string or null"}],
    "notes": "string or null"
  }
}
```

**Retry strategy:** If extraction fails or JSON is invalid, retry with slightly increased temperature (0.3, then 0.5). If all retries fail, write a minimal record:
```json
{"target_compound": "<material_name>", "target_compound_type": "other", "synthesis_method": "other", "starting_materials": [], "steps": [], "equipment": [], "notes": "Extraction failed."}
```

---

### Step 4 — Collect and write output JSON

From learningmatter-mit/AtomisticSkills, MIT.

More from learningmatter-mit/AtomisticSkills