simulation-orchestrator

Files (25)

• CHANGELOG.md
• SKILL.md
• evals.json · evals/
• config_0000.json · evals/files/case3_init/
• config_0001.json · evals/files/case3_init/
• config_0002.json · evals/files/case3_init/
• config_0003.json · evals/files/case3_init/
• manifest.json · evals/files/case3_init/
• campaign.json · evals/files/case3_status/
• campaign.json · evals/files/case4_campaign/
• config_0000.json · evals/files/case4_campaign/
• config_0001.json · evals/files/case4_campaign/
• config_0002.json · evals/files/case4_campaign/
• result_job_0000.json · evals/files/case4_campaign/
• result_job_0001.json · evals/files/case4_campaign/
• result_job_0002.json · evals/files/case4_campaign/
• config.json · evals/files/
• simulation.json · evals/files/
• aggregation_methods.md · references/
• campaign_patterns.md · references/
• sweep_strategies.md · references/
• campaign_manager.py · scripts/
• job_tracker.py · scripts/
• result_aggregator.py · scripts/
• sweep_generator.py · scripts/

SKILL.md (first lines)

# Simulation Orchestrator

## Goal

Provide tools to manage multi-simulation campaigns: generate parameter sweeps, track job execution status, and aggregate results from completed runs.

## Requirements

- Python 3.10+
- No external dependencies (uses Python standard library only)
- Works on Linux, macOS, and Windows

## Inputs to Gather

Before running orchestration scripts, collect from the user:

| Input | Description | Example |
|-------|-------------|---------|
| Base config | Template simulation configuration | `base_config.json` |
| Parameter ranges | Parameters to sweep with bounds | `dt:[1e-4,1e-2],kappa:[0.1,1.0]` |
| Sweep method | How to sample parameter space | `grid`, `lhs`, `linspace` |
| Output directory | Where to store campaign files | `./campaign_001` |
| Simulation command | Command to run each simulation | `python sim.py --config {config}` |

## Decision Guidance

### Choosing a Sweep Method

```
Need every combination (full factorial)?
├── YES → Use grid (warning: exponential growth with parameters)
└── NO → Is space-filling coverage needed?
    ├── YES → Use lhs (Latin Hypercube Sampling)
    └── NO → Use linspace for uniform sampling per parameter
```

| Method | Best For | Sample Count |
|--------|----------|--------------|
| `grid` | Low dimensions (1-3), need exact corners | n^d (exponential) |
| `linspace` | 1D sweeps, uniform spacing | n per parameter |
| `lhs` | High dimensions, space-filling | user-specified budget |

### Campaign Size Guidelines

| Parameters | Grid Points Each | Total Runs | Recommendation |
|------------|------------------|------------|----------------|
| 1 | 10 | 10 | Grid is fine |
| 2 | 10 | 100 | Grid acceptable |
| 3 | 10 | 1,000 | Consider LHS |
| 4+ | 10 | 10,000+ | Use LHS or DOE |

## Script Outputs (JSON Fields)

| Script | Output Fields |
|--------|---------------|
| `scripts/sweep_generator.py` | `configs`, `parameter_space`, `sweep_method`, `total_runs` |
| `scripts/campaign_manager.py --action init` | `campaign_id`, `total_jobs`, `config_dir`, `command_template` |
| `scripts/campaign_manager.py --action status` | `campaign_id`, `status`, `jobs`, `progress`, `total_jobs`, `created_at` |
| `scripts/campaign_manager.py --action list` | `jobs` (array of job records) |
| `scripts/job_tracker.py` | `job_id`, `status`, `start_time`, `end_time`, `exit_code` |
| `scripts/result_aggregator.py` | `summary` (incl. `minimize`), `statistics`, `best_run`, `failed_runs` |

> **Note on swept parameter names**: `sweep_generator.py` writes each swept value into the base config by key path. A bare name (e.g. `kappa`) overwrites a top-level key; a dot-notation name (e.g. `parameters.kappa`) targets a nested key. The swept key path must match where the solver reads the value — sweeping `kappa` against a config that nests `parameters.kappa` would add an unused top-level key and silently leave the base value in place. See `references/sweep_strategies.md`.

## Workflow

### Step 1: Generate Parameter Sweep

Create configurations for all parameter combinations:

```bash
python3 scripts/sweep_generator.py \
    --base-config base_config.json \
    --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \
    --method linspace \
    --output-dir ./campaign_001 \
    --json
```

### Step 2: Initialize Campaign

Create campaign tracking structure:

```bash
python3 scripts/campaign_manager.py \
    --action init \
    --config-dir ./campaign_001 \
    --command "python sim.py --config {config}" \
    --json
```

### Step 3: Track Job Status

Monitor running jobs:

```bash
python3 scripts/job_tracker.py \
    --campaign-dir ./campaign_001 \
    --update \
    --json
```

### Step 4: Aggregate Results

Combine results from completed runs:

```bash
python3 scripts/result_aggregator.py \
    --campaign-dir ./campaign_001 \
    --metric final_energy \
    --json
```

`result_aggregator.py` **minimizes by default**: `best_run` is the run with the
**lowest** metric value (and `summary.minimize` is `true`). If higher is better
(e.g. yield, accuracy, throughput), pass `--maximize` so `best_run` becomes the
**highest** value:

```bash

From HeshamFS/materials-simulation-skills, Apache-2.0.

More from HeshamFS/materials-simulation-skills