dspy-optimize-anything

Files (2)

• SKILL.md
• example.py

SKILL.md (first lines)

# GEPA optimize_anything

## Goal

Optimize any artifact representable as text — code, prompts, agent architectures, vector graphics, configurations — using a single declarative API powered by GEPA's reflective evolutionary search.

## When to Use

- **Beyond prompt optimization** — optimizing code, configs, SVGs, scheduling policies, etc.
- **Single hard problems** — circle packing, kernel generation, algorithm discovery
- **Batch related problems** — CUDA kernels, code generation tasks with cross-transfer
- **Generalization** — agent skills, policies, or prompts that must transfer to unseen inputs
- When you can **express quality as a score** and provide **diagnostic feedback** (ASI)

## Inputs

| Input | Type | Description |
|-------|------|-------------|
| `seed_candidate` | `str \| dict[str, str] \| None` | Starting artifact text, or `None` for seedless mode |
| `evaluator` | `Callable` | Returns score (higher=better), optionally with ASI dict |
| `dataset` | `list \| None` | Training examples (for multi-task and generalization modes) |
| `valset` | `list \| None` | Validation set (for generalization mode) |
| `objective` | `str \| None` | Natural language description of what to optimize for |
| `background` | `str \| None` | Domain knowledge and constraints |
| `config` | `GEPAConfig \| None` | Engine, reflection, and tracking settings |

## Outputs

| Output | Type | Description |
|--------|------|-------------|
| `result.best_candidate` | `str \| dict` | Best optimized artifact |

## Workflow

### Phase 1: Install

```bash
pip install -U "gepa>=0.1.1,<0.2"
```

### Phase 2: Define Evaluator with ASI

The evaluator scores a candidate and returns Actionable Side Information (ASI) — diagnostic feedback that guides the LLM proposer during reflection.

**Simple evaluator (score only):**

```python
import gepa.optimize_anything as oa
from gepa.optimize_anything import EngineConfig, GEPAConfig

config = GEPAConfig(engine=EngineConfig(max_metric_calls=100))

def evaluate(candidate: str) -> float:
    score, diagnostic = run_my_system(candidate)
    oa.log(f"Error: {diagnostic}")  # captured as ASI
    return score
```

**Rich evaluator (score + structured ASI):**

```python
def evaluate(candidate: str) -> tuple[float, dict]:
    result = execute_code(candidate)
    return result.score, {
        "Error": result.stderr,
        "Output": result.stdout,
        "Runtime": f"{result.time_ms:.1f}ms",
    }
```

ASI can include open-ended text, structured data, multi-objectives (via `scores`), or images (via `gepa.Image`) for vision-capable LLMs.

### Phase 3: Choose Optimization Mode

**Mode 1 — Single-Task Search:** Solve one hard problem. No dataset needed.

```python
result = oa.optimize_anything(
    seed_candidate="<your initial artifact>",
    evaluator=evaluate,
    config=config,
)
```

**Mode 2 — Multi-Task Search:** Solve a batch of related problems with cross-transfer.

```python
result = oa.optimize_anything(
    seed_candidate="<your initial artifact>",
    evaluator=evaluate,
    dataset=tasks,
    config=config,
)
```

**Mode 3 — Generalization:** Build a skill/prompt/policy that transfers to unseen problems.

```python
result = oa.optimize_anything(
    seed_candidate="<your initial artifact>",
    evaluator=evaluate,
    dataset=train,
    valset=val,
    config=config,
)
```

**Seedless mode:** Describe what you need instead of providing a seed.

```python
result = oa.optimize_anything(
    evaluator=evaluate,
    objective="Generate a Python function `reverse()` that reverses a string.",
    config=config,
)
```

### Phase 4: Use Results

From OmidZamani/dspy-skills, MIT.

More from OmidZamani/dspy-skills