gfxgraph-integration

Files (1)

• SKILL.md

SKILL.md (first lines)

# gfxGRAPH Integration Guide

Use gfxGRAPH to enable transparent CUDA Graph capture/replay on AMD RDNA2 GPUs (gfx1030/gfx1031)
where native HIP Graph support has parity gaps. gfxGRAPH is a drop-in replacement — existing
`torch.cuda.CUDAGraph` code works unchanged after `gfxgraph.enable()`.

## Quick Start

### Installation

```bash
# From source (editable — recommended for development)
pip install -e /path/to/gfxGRAPH/python/

# Standard install
pip install /path/to/gfxGRAPH/python/
```

### Verify Installation

```python
import gfxgraph
print(gfxgraph.__version__)       # → "0.3.4"
print(gfxgraph.health_check())    # → {'ok': True, 'gpu': 'AMD Radeon RX 6700 XT', ...}
```

### Enable (Standalone PyTorch)

```python
import gfxgraph
gfxgraph.enable()  # patches torch.cuda.CUDAGraph globally

# All existing CUDA graph code now works on RDNA2:
graph = torch.cuda.CUDAGraph()  # actually BridgedCUDAGraph
```

### Enable (SGLang)

Set environment variables before launching:

```bash
export SGLANG_RDNA2_KERNELS=1           # activates gfxGRAPH in SGLang
export HSA_OVERRIDE_GFX_VERSION=10.3.0  # required for gfx1031 → gfx1030 mapping
export PYTORCH_ROCM_ARCH=gfx1030
export AMD_SERIALIZE_KERNEL=3
export AMD_SERIALIZE_COPY=3

python3 -m sglang.launch_server --model-path <model> ...
```

### Enable (Environment Variable)

```bash
GFXGRAPH=1 python3 my_script.py         # standard mode
GFXGRAPH=debug python3 my_script.py     # verbose logging
GFXGRAPH=validate python3 my_script.py  # correctness checking mode
```

## Core API

### `gfxgraph.enable(**kwargs)`

Monkey-patches `torch.cuda.CUDAGraph` → `BridgedCUDAGraph`. Options:
- `debug=True` — verbose logging (also via `HGB_LOG_LEVEL=debug`)
- `validate=True` — compares graph output vs eager to catch silent bugs (PyTorch #155684)

### `gfxgraph.disable()`

Restores original `torch.cuda.CUDAGraph`. Safe to call even if not enabled.

### `gfxgraph.stats()`

Thread-safe performance counters:
```python
{'enabled_at': 1712..., 'capture_count': 32, 'replay_count': 1847,
 'fallback_count': 0, 'validation_failures': 0, 'avg_replay_us': 42.3}
```

### `gfxgraph.health_check()`

Quick GPU smoke test — captures and replays a trivial graph:
```python
{'ok': True, 'gpu': 'AMD Radeon RX 6700 XT', 'rocm': 'gfx1030',
 'native_bridge': False, 'vram_total_mb': 12288, 'vram_free_mb': 10240,
 'details': 'Graph capture/replay OK, output verified'}
```

## BridgedCUDAGraph

Drop-in replacement for `torch.cuda.CUDAGraph`. Supports both high-level and low-level APIs.

### Low-Level API (SGLang/vLLM compatible)

```python
from hipgraph_bridge import BridgedCUDAGraph

g = BridgedCUDAGraph()
g.capture_begin()
# ... operations ...
g.capture_end()
g.replay()
g.pool()  # returns mempool id
```

### High-Level API (shape bucketing + eager fallback)

```python
g = BridgedCUDAGraph()
with g.capture(dynamic_shapes=True, buckets=[1, 4, 8, 16, 32], model_fn=my_model):
    output = my_model(static_input)
g.replay(batch_size=12)  # auto-selects bucket 16, pads input
```

### Eager Fallback Behavior

When graph capture fails (common on RDNA2 for certain operations):
1. gfxGRAPH logs a warning
2. Sets `_eager_fallback = True`
3. Subsequent `replay()` calls execute the model eagerly via `model_fn`

From carlosfundora/gfxGRAPH, MIT.

More from carlosfundora/gfxGRAPH