performance-profiling

Files (7)

• SKILL.md
• optimization_strategies.md · references/
• profiling_guide.md · references/
• bottleneck_detector.py · scripts/
• memory_profiler.py · scripts/
• scaling_analyzer.py · scripts/
• timing_analyzer.py · scripts/

SKILL.md (first lines)

# Performance Profiling

## Goal

Provide tools to analyze simulation performance, identify bottlenecks, and recommend optimization strategies for computational materials science simulations.

## Requirements

- Python 3.8+
- No external dependencies (uses Python standard library only)
- Works on Linux, macOS, and Windows

## Inputs to Gather

Before running profiling scripts, collect from the user:

| Input | Description | Example |
|-------|-------------|---------|
| Simulation log | Log file with timing information | `simulation.log` |
| Scaling data | JSON with multi-run performance data | `scaling_data.json` |
| Simulation parameters | JSON with mesh, fields, solver config | `params.json` |
| Available memory | System memory in GB (optional) | `16.0` |

## Decision Guidance

### When to Use Each Script

```
Need to identify slow phases?
├── YES → Use timing_analyzer.py
│         └── Parse simulation logs for timing data
│
Need to understand parallel performance?
├── YES → Use scaling_analyzer.py
│         └── Analyze strong or weak scaling efficiency
│
Need to estimate memory requirements?
├── YES → Use memory_profiler.py
│         └── Estimate memory from problem parameters
│
Need optimization recommendations?
└── YES → Use bottleneck_detector.py
          └── Combine analyses and get actionable advice
```

### Choosing Analysis Thresholds

| Metric | Good | Acceptable | Poor |
|--------|------|------------|------|
| Phase dominance | <30% | 30-50% | >50% |
| Parallel efficiency | >0.80 | 0.70-0.80 | <0.70 |
| Memory usage | <60% | 60-80% | >80% |

## Script Outputs (JSON Fields)

| Script | Key Outputs |
|--------|-------------|
| `timing_analyzer.py` | `timing_data.phases`, `timing_data.slowest_phase`, `timing_data.total_time` |
| `scaling_analyzer.py` | `scaling_analysis.results`, `scaling_analysis.efficiency_threshold_processors` |
| `memory_profiler.py` | `memory_profile.total_memory_gb`, `memory_profile.per_process_gb`, `memory_profile.warnings` |
| `bottleneck_detector.py` | `bottlenecks`, `recommendations` |

## Workflow

### Complete Profiling Workflow

1. **Analyze timing** from simulation logs
2. **Analyze scaling** from multi-run data (if available)
3. **Profile memory** from simulation parameters
4. **Detect bottlenecks** and get recommendations
5. **Implement optimizations** based on recommendations
6. **Re-profile** to verify improvements

### Quick Profiling (Timing Only)

1. **Run timing analyzer** on simulation log
2. **Identify dominant phases** (>50% of runtime)
3. **Apply targeted optimizations** to dominant phases

## CLI Examples

### Timing Analysis

```bash
# Basic timing analysis
python3 scripts/timing_analyzer.py \
    --log simulation.log \
    --json

# Custom timing pattern
python3 scripts/timing_analyzer.py \
    --log simulation.log \
    --pattern 'Step\s+(\w+)\s+took\s+([\d.]+)s' \
    --json
```

### Scaling Analysis

```bash
# Strong scaling (fixed problem size)
python3 scripts/scaling_analyzer.py \
    --data scaling_data.json \
    --type strong \
    --json

# Weak scaling (constant work per processor)
python3 scripts/scaling_analyzer.py \
    --data scaling_data.json \
    --type weak \
    --json
```

### Memory Profiling

```bash
# Estimate memory requirements
python3 scripts/memory_profiler.py \
    --params simulation_params.json \
    --available-gb 16.0 \

From beita6969/ScienceClaw, MIT.

More from beita6969/ScienceClaw