debug-graph-breaks

Files (2)

• SKILL.md
• eval.yaml · evals/

SKILL.md (first lines)

# debug-graph-breaks

## Description
Diagnose and eliminate `torch.compile` graph breaks by running the script, reading the output, applying fixes directly, and re-running to verify — whether the goal is `fullgraph=True` or simply reducing the most impactful breaks.

## Instructions for Claude

### Overview
When this skill is invoked, help the user find, understand, and fix graph breaks in their `torch.compile`d code. The workflow has three phases: **Detect**, **Diagnose**, **Fix**.

The graph break documentation website at https://meta-pytorch.org/compile-graph-break-site/ has detailed pages for each known graph break type. When the user encounters a specific graph break (e.g., GB0059), fetch the corresponding page (e.g., `https://meta-pytorch.org/compile-graph-break-site/gb/gb0059.html`) to get detailed context, examples, and fix suggestions specific to that break. If the user provides a local path to a clone of the graph break website repository, read the documentation files directly from that directory instead.

### Phase 1: Detect — Find the graph breaks

If the user provides a script path and no existing logs, run it with `TORCH_LOGS="graph_breaks"`:
```bash
TORCH_LOGS="graph_breaks" python your_script.py 2>&1
```
This prints each graph break with:
- The file and line in user code that triggered it
- The graph break reason (includes GB type, explanation, hints)
- A user code traceback
- A link to documentation for that specific graph break

If the output is ambiguous or a break's origin is unclear, re-run with verbose mode:
```bash
TORCH_LOGS="graph_breaks" TORCHDYNAMO_VERBOSE=1 python your_script.py 2>&1
```
The verbose mode adds the internal Dynamo stack trace and recent bytecode instructions, which can help when the standard output is ambiguous.

If the user provides existing `TORCH_LOGS` output, parse it directly and skip to Phase 2.

Other detection methods are available if the user requests them or provides the relevant artifacts:

#### Option B: tlparse report
If the user has a tlparse report (a structured HTML report from `TORCH_TRACE`), they can provide:
- A local directory path to the tlparse output
- A URL to the report

A tlparse report contains:
- **`index.html`**: Main page with a **stack trie** showing all compilations as a tree, color-coded by status (green = success, lime = graph break, red = error). Each compilation is identified by a **compile id** like `[0/0]` (frame 0, first compile) or `[1/0]` (frame 1, first compile — often a resume after graph break).
- **`failures_and_restarts.html`**: Table of all `RestartAnalysis` events (graph breaks) and compilation failures, with the full graph break reason for each.
- **Per-compilation directories** (e.g., `-_0_0_0/`, `-_1_0_1/`): Contain build products:
  - `dynamo_graph_break_reason_N.txt` — Full graph break details: user code location, reason, explanation, hints, user stack trace, and internal Dynamo traceback.
  - `dynamo_output_graph_N.txt` — The FX graph produced by that compilation.
  - `compilation_metrics_N.html` — Compile time, graph metrics (ops, nodes, inputs), restart reasons, guard count, cache metrics.
  - `*_ORIGINAL_BYTECODE_N.txt` / `*_MODIFIED_BYTECODE_N.txt` — Bytecode before/after Dynamo modification.

When reading a tlparse report:
1. Start with `failures_and_restarts.html` to get a quick summary of all graph breaks.
2. Read each `dynamo_graph_break_reason_N.txt` for full details.
3. Use `compilation_metrics_N.html` to understand compile time impact per subgraph.
4. The compile IDs with `_1` suffix (e.g., `[0/0_1]`) indicate a restart — the `_1` means Dynamo restarted analysis after discovering a graph break on attempt `_0`.

#### Option C: `fullgraph=True` (fix one break at a time)
If the user wants to fix breaks one at a time rather than all at once, use `fullgraph=True`:
```python
torch.compile(fn, fullgraph=True)(*args)
```
This raises an error on the first graph break encountered. Fix it, re-run, fix the next.

### Phase 2: Diagnose — Understand each graph break

For each graph break found, do the following:

1. **Parse the graph break reason.** Extract the GB type string (e.g., "Failed to trace builtin operator", "Unsupported Tensor.item() call with capture_scalar_outputs=False") and the GB documentation URL from the output.

2. **Fetch the graph break documentation page.** Use the URL from the log output (e.g., `https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0059.html`) to get detailed information, examples, and suggested fixes for that specific graph break type. If a local clone of the graph break site was provided, read the corresponding file from the local directory instead.

3. **Classify the graph break** using the hint categories from the log output:
   - **USER_ERROR**: The user's code has a bug that also fails in eager mode. Verify by running the script without `torch.compile`.
   - **FUNDAMENTAL**: Dynamo fundamentally cannot trace this pattern (e.g., generators, certain dynamic control flow). The user must restructure their code.
   - **SUPPORTABLE**: Dynamo could theoretically support this but doesn't yet. The user can file a PyTorch issue or find a workaround.
   - **CAUSED_BY_EARLIER_GRAPH_BREAK**: Fix earlier graph breaks first — this one may resolve itself.
   - **DYNAMO_BUG**: Likely a Dynamo bug. Suggest filing an issue.

4. **Identify the user code location.** Use the user stack trace to pinpoint the exact line in user code causing the break. Read that code directly to understand context.

5. **Read the user's script.** Always read the relevant code in the user's script to understand the surrounding context. Compare it to examples from the graph break documentation page to identify the exact pattern causing the break and determine the most targeted fix.

6. **Prioritize.** Not all graph breaks are equal. Decide which to fix first:
   - **Easiest wins first**: `print()` calls, logging, simple `.item()` calls — these have straightforward fixes.
   - **Highest impact**: Graph breaks in hot loops or inside the core forward pass matter more than breaks in initialization or infrequent code paths.
   - **CAUSED_BY_EARLIER_GRAPH_BREAK last**: These may auto-resolve when earlier breaks are fixed.
   - **Cascading breaks**: A graph break inside a loop (GB7000) causes the entire frame to fall back to eager. Fixing the inner break eliminates both.
   - If the user does NOT need `fullgraph=True`, identify which breaks matter most for performance and which can be left alone.

### Phase 3: Fix — Eliminate graph breaks

For each graph break, apply a fix based on its type. Always read the user's code before making changes and then make changes directly.

#### Fixing graph breaks in third-party libraries (e.g., HuggingFace transformers)

When graph breaks originate in a third-party library (not the user's code), the user cannot directly modify the library. Instead:

1. **Inspect the library source.** If the user provides the path to the installed package (e.g., the `transformers` site-packages directory), read the relevant source files to understand the problematic code and its call chain.

2. **Subclass to override.** Create a subclass of the model that overrides the method causing the graph break with a compile-compatible implementation. This avoids modifying the library and is portable:
```python
from transformers import BartModel

class CompilableBartModel(BartModel):
    def _override_problematic_method(self, *args, **kwargs):
        # Compile-compatible reimplementation
        ...

model = CompilableBartModel(config)
compiled_model = torch.compile(model, fullgraph=True)
```

3. **Verify with `fullgraph=True`.** After applying the subclass fix, re-run with `fullgraph=True` to confirm all graph breaks are resolved.

This approach is preferred over monkey-patching because it creates a self-contained, reusable fix.

#### Escape hatches (compile-safe alternatives)

These are APIs that let code run inside a compiled region without causing graph breaks:

**`torch._higher_order_ops.print(format_str, *args, **kwargs)`**

From meta-pytorch/skills, BSD-3-Clause.

More from meta-pytorch/skills