ml-property-predict-scd

Files (15)

• SKILL.md
• README.md · examples/CT-SCD_QM9/
• run_ct_scd_qm9.py · examples/CT-SCD_QM9/
• README.md · examples/CT-SCD_matbench/
• run_ct_scd_matbench.py · examples/CT-SCD_matbench/
• config-recipes.md · references/
• dataset-contract.md · references/
• repo-map.md · references/
• transfer-recipes.md · references/
• dataset_template.py · templates/
• extract_embeddings.py · templates/
• finetune_config.template.yaml · templates/
• generate_conformer_rdkit.py · templates/
• pretrain_config.template.yaml · templates/
• train_lightweight_head.py · templates/

SKILL.md (first lines)

# ml-property-predict-scd

## Goal

Use `SelfConditionedDenoisingAtoms` for four related workflows:

- apply a frozen SCD checkpoint as a live atomistic encoder
- train a lightweight head on top of a frozen SCD backbone
- fine-tune an entire pretrained SCD checkpoint on a new property task
- pretrain a new SCD model or add a new dataset adapter

## First Checks

1. Commands run in the `scd` environment through `venv/run`, as shown below. It
   is created on first use, on x86_64 Linux only (PyG publishes its compiled
   extensions, which the upstream code imports, for x86_64), with a CUDA 12.6 or
   13 torch build chosen by driver.
2. Confirm the upstream repo exists at `$SCD_REPO_DIR` (default:
   `~/.cache/atomisticskills/SelfConditionedDenoisingAtoms`), or clone it at the
   verified commit:

   ```bash
   SCD_REPO_DIR="${SCD_REPO_DIR:-$HOME/.cache/atomisticskills/SelfConditionedDenoisingAtoms}"
   git clone https://github.com/TyJPerez/SelfConditionedDenoisingAtoms "$SCD_REPO_DIR"
   git -C "$SCD_REPO_DIR" checkout 528c13ead7cc2b1c2b3dfddc0d167248e670746b
   ```

   Optionally build the faster TorchMD graph kernel for non-periodic runs
   (needs the CUDA toolkit); without it, use `noise_in_loader: true`:

   ```bash
   cd "$SCD_REPO_DIR/models/ET_models" && ${CLAUDE_SKILL_DIR}/../../venv/run scd python setup.py build_ext --inplace
   ```
3. Read the upstream `README.md` and `examples.ipynb`.
4. Then read the local references in this skill:
   - `references/repo-map.md`
   - `references/transfer-recipes.md`
   - `references/config-recipes.md`
   - `references/dataset-contract.md` if a new dataset is involved

## Checkpoint Selection

- Use `ct-scd-pcq` for molecule property prediction, molecule embeddings, and molecule-side transfer learning.
- Use `ct-scd-amp` for materials property prediction, periodic materials embeddings, and materials-side transfer learning.

Do not swap these by default. The public checkpoints were pretrained on different domains.

## Instructions

### 1. Frozen backbone embeddings

Default to `out["mol_emb"]` for graph-level downstream ML.

- Use `return_atom_embs=True` only when the downstream task needs atom- or site-level features.
- Keep the checkpoint frozen and in `eval()` mode.
- Disable the denoising head for this workflow to avoid wasted compute.
- Pass `graph_batch=batch` only when `allow_periodic` or `noise_in_loader` is enabled. Do not force `graph_batch` on the fast molecular path.
- Reuse `templates/extract_embeddings.py` as the starting point. It keeps the model live and returns embeddings on demand instead of defaulting to a frozen feature dump.

### 2. Lightweight training with a frozen SCD backbone

Use `templates/train_lightweight_head.py` for three lightweight options:

1. `scalar_head`
   Appropriate for invariant scalar regression targets. This path trains only the model's native `scalar_head` using pretrained backbone weights.
2. `atom_emb_mlp`
   Pools `atom_embs` with `sum` or `mean`, then trains a 1- or 2-layer MLP head.
3. `mol_emb_mlp`
   Uses `mol_emb` directly, then trains a 1- or 2-layer MLP head.

Important details:

- `scalar_head` is usually the lightest path for standard scalar property prediction.
- `reset_head()` is a sensible default for `scalar_head` mode when switching to a new target.
- `set_head_agg` controls the native scalar-head reduction and should be set through the checkpoint-loading path, not by editing tensors after the fact.
- the upstream model exposes `finetune()` and `reset_head()`, but `finetune()` does not by itself implement scalar-head-only training; explicitly freeze non-`scalar_head` parameters in the lightweight script.
- the lightweight `scalar_head` path should reset the checkpoint's output affine buffers to identity because pretrained `mean/std` buffers are not downstream target statistics.
- for `atom_emb_mlp`, either `sum` or `mean` pooling may work better depending on whether the target behaves more like an extensive or intensive quantity.
- `atom_emb_mlp` and `mol_emb_mlp` should compute frozen-backbone features on the fly, not treat a static embedding cache as the default workflow.
- `mol_emb_mlp` is the cleanest external-head baseline on the pretrained graph representation.

### 3. Full-model finetuning

Use the native training path when you want all model weights updated:

```bash
cd "${SCD_REPO_DIR:-$HOME/.cache/atomisticskills/SelfConditionedDenoisingAtoms}"
${CLAUDE_SKILL_DIR}/../../venv/run scd python train.py --conf configs/my_finetune.yaml --load-hf ct-scd-pcq --job-id my_run
```

or

```bash
cd "${SCD_REPO_DIR:-$HOME/.cache/atomisticskills/SelfConditionedDenoisingAtoms}"
${CLAUDE_SKILL_DIR}/../../venv/run scd python train.py --conf configs/my_finetune.yaml --load-hf ct-scd-amp --job-id my_run
```

Start from:

- `templates/finetune_config.template.yaml`
- `configs/finetune_qm9.yaml` for public molecular examples

For public materials finetuning, do not rely on `configs/finetune_matbench.yaml` unless the private `StructureCloud` dependency is available. Copy the template and configure the periodic settings yourself.

Full-model finetuning usually gives better results than the lightweight frozen-backbone options, but it costs more GPU memory and more wall time.

### 4. Pretraining from scratch

Use the native training path:

```bash
cd "${SCD_REPO_DIR:-$HOME/.cache/atomisticskills/SelfConditionedDenoisingAtoms}"
${CLAUDE_SKILL_DIR}/../../venv/run scd python train.py --conf configs/my_pretrain.yaml --job-id my_pretrain
```

Start from:

- `templates/pretrain_config.template.yaml`
- `configs/pretrain_pcq.yaml` for molecules

From learningmatter-mit/AtomisticSkills, MIT.

More from learningmatter-mit/AtomisticSkills