complexa-sweep

Files (27)

• BENCHMARK.md
• SKILL.md
• skillspector-baseline.yml · config/
• evals.json · evals/
• launch.sh · evals/files/launch_review/
• sweeper.yaml · evals/files/launch_review/
• pareto_results.csv · evals/files/
• SNAPSHOT.md · evals/files/project/
• binder_generate.yaml · evals/files/project/configs/pipeline/binder/
• model_sampling.yaml · evals/files/project/configs/pipeline/
• search_binder_local_pipeline.yaml · evals/files/project/configs/
• example.yaml · evals/files/project/configs/sweeps/
• generate_inference_configs.py · evals/files/project/script_utils/
• source_manifest.json · evals/files/project/
• summary_review.csv · evals/files/
• sweep_results.csv · evals/files/
• CONFIGURATION_GUIDE.md · references/
• EVALUATION_METRICS.md · references/
• INFERENCE.md · references/
• SEARCH_METADATA.md · references/
• SWEEP.md · references/
• hardware.md · references/
• sweep_axes.md · references/
• preflight.sh · scripts/
• write_manifest.py · scripts/
• skill-card.md
• skill.oms.sig

SKILL.md (first lines)

# complexa-sweep

Run cartesian-product parameter sweeps over Proteina-Complexa design pipelines. Pick or author a sweeper YAML in `configs/sweeps/`, expand it to N inference configs with `script_utils/generate_inference_configs.py`, loop `complexa design` over those configs, then aggregate per-config success metrics into a ranked summary CSV plus a manifest.

> **Important:** The `complexa design` CLI does **NOT** accept `--sweeper` directly. Sweeps are driven by `script_utils/generate_inference_configs.py`, which writes one `configs/inference_configs/inf_{idx}_{run_name}.yaml` per sweep combination. You then loop `complexa design` over those generated configs.

## Instructions

Sweeper authoring, Cartesian enumeration, cost estimates, and analysis of supplied
CSVs need only the relevant configs and data. Complete those artifacts without
setting up inference. Read the relevant axis section in
[sweep_axes.md](references/sweep_axes.md), then check the matching keys in the
selected source config for current defaults. Summary CSVs with supplied counts
and objective columns do not require the full inference or metric guides.
For a partial snapshot,
enumerate combinations locally and provide the future generator command;
running that command requires the full project and its dependencies.

For a requested launch, check readiness and the budget below. A single existing
job or a SLURM template does not require a parameter sweep.

## Step 1: Pre-flight

Before GPU execution, or when the request conditions its workflow on hardware,
set `SKILL_DIR` to the directory containing this manifest and run preflight once.
Use a separate shell assignment; an inline environment assignment cannot set
`$SKILL_DIR`'s expansion in the same command.
Reuse an existing probe from the same request if the environment is unchanged:

```bash
bash "$SKILL_DIR"/scripts/preflight.sh
```

Read `./complexa_setup/preflight.json`. A sweep multiplies GPU time by the number
of configs and targets. Compute the budget before launch and obtain confirmation
if the user has not already authorized that scope and cost:

> "This sweep produces N configs × ~M minutes per config ≈ TOTAL GPU-hours. OK to proceed? (y / reduce / cancel)"

If `gpu.available=false`, stop before GPU execution. Sweeper authoring,
Cartesian-product enumeration, cost estimates, and analysis of supplied CSVs
can run on CPU. For those requests, finish the requested artifacts and record
the launch prerequisites. Label synthetic input data as synthetic and keep
estimated GPU time separate from measured runtime.

For a hardware-conditional request, verify the allocated hardware before
selecting the CPU fallback. Missing weights, dependencies, or sufficient VRAM
for an allocated GPU are launch blockers, not evidence of a CPU-only environment.
Report probe or execution failures without reclassifying the selected branch.
The shared preflight's `gpu.available=false` also covers failed or missing
`nvidia-smi`; check device/allocation evidence before treating it as GPU absence.
In containers, `/proc/driver/nvidia/gpus` and loaded kernel modules may describe
the host, not this task's allocation. Check container device exposure and any
available scheduler allocation information together. A CPU allocation can share
a GPU host; an allocated GPU with inaccessible devices is blocked. If allocation
remains unclear, report it as undetermined.

## Step 2: Pick the pipeline + target

Use the user's supplied pipeline, targets, and run name. Protein, ligand, and AME
runs use `search_binder_local_pipeline`, `search_ligand_binder_local_pipeline`,
and `search_ame_local_pipeline`, respectively. Ask only for missing values:

- `pipeline_config_name` — e.g. `search_binder_local_pipeline` (default), `search_ligand_binder_local_pipeline`, `search_ame_local_pipeline`.
- `task_name` — e.g. `02_PDL1`, `22_DerF21`, `39_7V11_LIGAND`. Passed as `--override generation.task_name=<task>`.
- `run_name` — short tag for output dir naming.

## Examples

### Step 3: Pick or author the sweeper YAML

Sweeper YAMLs live in `configs/sweeps/`. Each key is a dot-notation Hydra path; each value is a list. The cartesian product becomes N configs.

#### Canned sweepers

| File | Axis | Values | Configs |
|---|---|---|---|
| `configs/sweeps/beam_width.yaml` | `generation.search.beam_search.beam_width` | 1, 2, 4, 8 | 4 |
| `configs/sweeps/bb_ca_temperature.yaml` | `generation.model.bb_ca.simulation_step_params.sc_scale_noise` | 0.1, 0.4 | 2 |
| `configs/sweeps/search_replicas.yaml` | `generation.search.best_of_n.replicas` | 1, 4, 16, 64 | 4 |
| `configs/sweeps/example.yaml` | beam_width × nsteps | (2,4) × (200,400) | 4 |

Use a matching sweeper, then check that its axes affect the selected algorithm.
The binder config defaults to `best-of-n`: a beam-width sweep must also pin
`generation.search.algorithm: [beam-search]`. A replicas sweep pins
`generation.search.algorithm: [best-of-n]`. For AME, reward-guided search also
requires a configured reward model; its default is `null`.

#### Authoring a new sweeper

Minimal multi-axis example (saved to `configs/sweeps/my_sweep.yaml`):

```yaml
# 3 beam widths × 2 nsteps = 6 configs
generation.search.algorithm: [beam-search]

generation.search.beam_search.beam_width:
  - 2
  - 4
  - 8

generation.args.nsteps:
  - 200
  - 400
```

Rules (from `script_utils/generate_inference_configs.py:load_sweeper_file`):

- Top-level mapping only. Keys are dot-notation Hydra paths.
- Keep budgets, launch status, and other planning metadata in the launch plan, outside the sweeper. Every top-level entry becomes an override axis; a separate `targets` list would multiply the grid again.
- Values must be **lists**. A scalar is auto-wrapped into a single-element list (which pins a value without adding a dimension).
- For a list-valued parameter, nest each candidate list: `metric.sequence_types: [[self, mpnn]]` pins one list. The flat form sweeps two strings; `--override` parses scalars only.
- Cartesian product: total configs = product of list lengths. Multiply by the number of targets unless target names are already one of the axes; do not count them twice.
- If a key appears in both the sweeper file and an `--override`, the override wins and that axis collapses.
- Reject empty axes before launch: an axis of `[]` produces zero configurations.

See [references/sweep_axes.md](references/sweep_axes.md) for the full catalogue of swept keys (typical ranges, cost multipliers, what improves/regresses).

From NVIDIA/skills, Apache-2.0.

More from NVIDIA/skills