parameter-optimization

Files (11)

• CHANGELOG.md
• SKILL.md
• evals.json · evals/
• doe_methods.md · references/
• optimizer_selection.md · references/
• sensitivity_guidelines.md · references/
• surrogate_guidelines.md · references/
• doe_generator.py · scripts/
• optimizer_selector.py · scripts/
• sensitivity_summary.py · scripts/
• surrogate_builder.py · scripts/

SKILL.md (first lines)

# Parameter Optimization

## Goal

Provide a workflow to design experiments, rank parameter influence, and select optimization strategies for materials simulation calibration.

## Requirements

- Python 3.10+
- No external dependencies (uses Python standard library only)

## Inputs to Gather

Before running any scripts, collect from the user:

| Input | Description | Example |
|-------|-------------|---------|
| Parameter bounds | Min/max for each parameter with units | `kappa: [0.1, 10.0] W/mK` |
| Evaluation budget | Max number of simulations allowed | `50 runs` |
| Noise level | Stochasticity of simulation outputs | `low`, `medium`, `high` |
| Constraints | Feasibility rules or forbidden regions | `kappa + mobility < 5` |

## Decision Guidance

### Choosing a DOE Method

```
Is dimension <= 3 AND full coverage needed?
├── YES → Use factorial
└── NO → Is sensitivity analysis the goal?
    ├── YES → Use quasi-random (preferred; "sobol" is accepted but deprecated)
    └── NO → Use lhs (Latin Hypercube)
```

| Method | Best For | Avoid When |
|--------|----------|------------|
| `lhs` | General exploration, moderate dimensions (3-20) | Need exact grid coverage |
| `quasi-random` | Sensitivity analysis, uniform coverage (preferred) | Very high dimensions (>20) |
| `sobol` | Deprecated alias of `quasi-random` (emits a warning) | New code (use `quasi-random`) |
| `factorial` | Low dimension (<4), need all corners | High dimension (exponential growth) |

> **Factorial sizing:** the factorial grid is `levels` evenly spaced values per
> parameter, producing exactly `levels ** params` samples. Set the resolution
> explicitly with `--levels` (e.g. `--params 2 --levels 4` -> 16 samples). If you
> use `--budget` instead, the script back-computes `levels = round(budget ** (1/params))`
> and **warns** whenever the realized sample count differs from the requested
> budget (e.g. `--budget 20 --params 2` realizes 16 samples). For an exact design,
> pass a perfect power (`--budget 16`) or, preferably, `--levels`.

### Choosing an Optimizer

```
Is dimension <= 10 AND budget <= 100?
├── YES → Bayesian Optimization
└── NO → Is dimension <= 20?
    ├── YES → CMA-ES
    └── NO → Random Search with screening
```

| Noise Level | Recommendation |
|-------------|----------------|
| Low | Gradient-based if derivatives available, else Bayesian Optimization |
| Medium | Bayesian Optimization with noise model |
| High | Evolutionary algorithms or robust Bayesian Optimization |

## Script Outputs (JSON Fields)

| Script | Output Fields |
|--------|---------------|
| `scripts/doe_generator.py` | `samples`, `method`, `coverage` (`count`, `dimension`; plus `levels` and a top-level `requested_budget`/`note` for factorial) |
| `scripts/optimizer_selector.py` | `recommended`, `expected_evals`, `notes` |
| `scripts/sensitivity_summary.py` | `ranking`, `notes` |
| `scripts/surrogate_builder.py` | `model_type`, `metrics` (`mse`, `cv_error`, `output_variance`), `notes` |

## Workflow

1. **Generate DOE** with `scripts/doe_generator.py`
2. **Run simulations** at DOE sample points (user's responsibility)
3. **Summarize sensitivity** with `scripts/sensitivity_summary.py`
4. **Choose optimizer** using `scripts/optimizer_selector.py`
5. **(Optional)** Fit surrogate with `scripts/surrogate_builder.py`

## CLI Examples

```bash
# Generate 20 LHS samples for 3 parameters
python3 scripts/doe_generator.py --params 3 --budget 20 --method lhs --json

# Full factorial with 4 levels per parameter (2 params -> 16 samples)
python3 scripts/doe_generator.py --params 2 --levels 4 --method factorial --json

# Rank parameters by sensitivity scores
python3 scripts/sensitivity_summary.py --scores 0.2,0.5,0.3 --names kappa,mobility,W --json

# Get optimizer recommendation for 3D problem with 50 eval budget
python3 scripts/optimizer_selector.py --dim 3 --budget 50 --noise low --json

# Build surrogate model from simulation data
python3 scripts/surrogate_builder.py --x 0,1,2 --y 10,12,15 --model rbf --json
```

## Conversational Workflow Example

**User**: I need to calibrate thermal conductivity and diffusivity for my FEM simulation. I can run about 30 simulations.

**Agent workflow**:
1. Identify 2 parameters → `--params 2`
2. Budget is 30 → `--budget 30`
3. Use LHS for general exploration:
   ```bash
   python3 scripts/doe_generator.py --params 2 --budget 30 --method lhs --json
   ```
4. After user runs simulations and provides outputs, summarize sensitivity:
   ```bash
   python3 scripts/sensitivity_summary.py --scores 0.7,0.3 --names conductivity,diffusivity --json
   ```
5. Recommend optimizer:
   ```bash
   python3 scripts/optimizer_selector.py --dim 2 --budget 30 --noise low --json

From HeshamFS/materials-simulation-skills, Apache-2.0.

More from HeshamFS/materials-simulation-skills