quark-onnx-ptq-workflow

Files (2)

• SKILL.md
• example-xint8-yolov8n.md

SKILL.md (first lines)

# quark-onnx-ptq-workflow

> 📘 **Quick-start example — read this first.**
> A fully worked end-to-end walkthrough is available at **[`example-xint8-yolov8n.md`](example-xint8-yolov8n.md)** (XINT8 quantization of YOLOv8n for AMD NPU CNN deployment, calibrated on COCO val2017). It is the fastest way to see exactly what this workflow produces — open it alongside this SKILL.md before running anything.

## Purpose

Chain the ONNX PTQ path — model intake → quantization planning → script + manifest generation → confirmed execution — while keeping the user informed at each checkpoint. This workflow orchestrates the atomic ONNX skills so the user does not have to manually chain them. Unlike the Torch flow there is no single shipped `quantize_quark.py` for ONNX; the workflow generates a small standalone Python script in the user's working directory that imports from `quark.onnx`, then runs that script after the user confirms.

📂 **Worked example:** see [`example-xint8-yolov8n.md`](example-xint8-yolov8n.md) for the full YOLOv8n + XINT8 + NPU-CNN walkthrough referenced throughout the steps below.

## Inputs

- Input `.onnx` model path (with optional sibling `.onnx_data` external-weights file)
- Calibration data: folder of representative samples (or `CalibrationDataReader` Python class)
- Output `.onnx` path (and optional `.onnx_data` if `use_external_data_format=True`)
- User goal: scheme (XINT8 / A8W8 / A16W8 / BF16 / BFP16 / `MX*` / `MXFP*`), deployment target (CPU / CUDA / ROCm / AMD NPU CNN / AMD NPU Transformer), accuracy target
- `session_context.json` with `constraints.backend = "onnx"` for user goal and constraints
- `env_context.json` for hardware / execution-provider facts
- `workspace_context.json` for validated paths
- `onnx_install_result.json` and `quark_install_result.json` to confirm runtime is ready

## Outputs: run_manifest.yaml

Records the generated calibration/quantization script path, the exact `python3` invocation, and the resolved `QConfig`. Side artifacts: the generated script in the user's working directory, the quantized `.onnx` (and `.onnx_data` if external) in the user's output directory. The manifest is built in Step 3.

Schema: [`run_manifest.schema.json`](../../../shared/contracts/run_manifest.schema.json)

## Interaction Flow

1. **Intake** — call `quark-onnx-model-intake` to produce `model_analysis.json`
2. **Plan** — call `quark-onnx-quant-plan` to produce `quant_plan.json`
3. **Manifest** — generate a standalone quantization script in the user's working directory + `run_manifest.yaml`, then stop for user approval
4. **Execute** — run the confirmed script and report the artifact paths

## CRITICAL RULES

1. **NEVER call `ModelQuantizer.quantize_model(...)` directly from this workflow.** Always generate a standalone script the user reviews first.
2. **NEVER skip a step.** Even if the user provides all details upfront, execute each step in order.
3. **STOP at every checkpoint** and wait for user confirmation before continuing.
4. **Show concrete output** at each step — tables, JSON, the full script body, the exact command — not just prose descriptions.
5. **NEVER modify Quark's own source code, examples, or tutorials.** The Quark repo (`quark/`, `examples/`, `tutorials/`, `tools/`, `docs/`, `tests/`) is read-only from this workflow's perspective. See *Upstream Quark Code is Read-Only* below.
6. **NEVER silently fall back to CPU execution.** If the user requested CUDA/ROCm and the provider is unavailable, stop and surface the gap to `quark-onnx-install` / `quark-onnx-debug` — do not quietly degrade.

## Upstream Quark Code is Read-Only

The Quark repository is the upstream source of truth. This workflow may read it freely (config sources, example scripts, tutorial notebooks, custom-op headers) but must not write into it. That includes:

- No edits to `quark/` package source.
- No edits to anything under `examples/onnx/` — including `examples/onnx/yolo_quantization/quantize_yolo.py`. Even small "just to make the script accept my preprocessing" patches are forbidden, because they make the run irreproducible against a clean Quark install.
- No edits to anything under `tutorials/onnx/` — the Ryzen AI tutorials for YOLOv8 / ResNet-50 / image classification are reference material only.
- No edits to `tools/`, `docs/`, `tests/`, or `pyproject.toml` / `requirements.txt`.

**If a shipped example or tutorial as-is does not cover what the user needs** (different model, different preprocessing, different exclude list, different calibration source), write a fresh standalone script in the user's working directory (or `/tmp/`) that imports from `quark.onnx`. Pattern:

```python
# user_workspace/my_onnx_ptq.py — NOT inside the Quark repo
from quark.onnx import ModelQuantizer, QConfig, QLayerConfig, XInt8Spec, CLEConfig
from onnxruntime.quantization.calibrate import CalibrationDataReader
# ... user-specific data reader + QConfig the shipped example doesn't cover ...
```

Then reference *that* script in the `run_manifest.yaml` instead of a modified example. The shipped scripts stay untouched, the user's customization is local to their workspace, and the run remains reproducible against any Quark version.

If the user actually needs an upstream Quark change (a new preset, a new algorithm config), surface it as a Quark contribution — do not silently patch their local checkout.

## Required Artifact Flow

```text
Step 1 (Intake)   ──► model_analysis.json
Step 2 (Plan)     ──► quant_plan.json
Step 3 (Manifest) ──► run_manifest.yaml  +  user_workspace/<name>_ptq.py
Step 4 (Execute)  ──► quantized .onnx (+ .onnx_data if external)  (only after user says yes)
```

---

## Step 1: Model Intake

**Goal**: Analyze the `.onnx` graph and produce `model_analysis.json`. Hand off to `quark-onnx-model-intake`.

### Actions

1. **Confirm the input path.** It must end in `.onnx`. If a sibling `.onnx_data` exists, both must move together.
2. **Run `quark-onnx-model-intake`.** That skill produces:
   - opset / IR version
   - input / output names, shapes, dtypes
   - op-type histogram and quantizable-op count
   - whether QDQ nodes are already present (if yes — model is already quantized; warn and stop)
   - external-data status (model.onnx_data present? total bytes? >2 GB?)
   - deployment-target compatibility (CPU / CUDA / ROCm / NPU CNN / NPU Transformer)
3. **Surface risks:**
   - Already quantized → stop, do not re-quantize.
   - opset < 13 → recommend upgrading to opset 17+ for best support.
   - Dynamic input shapes → calibration data reader must produce matching shapes; warn if shapes vary.
   - >2 GB model → external data must be handled; set `use_external_data_format=True` later.
   - NPU CNN target → only XINT8 / A8W8 are gated as supported; warn early.

### Output to Show User

Present a summary table:

```text
Model Analysis:
  Model path:       ./models/yolov8n.onnx
  Opset / IR:       17 / 8
  Input:            images, [1, 3, 640, 640], float32
  Output:           output0, [1, 84, 8400], float32
  Total ops:        ~226 nodes (Conv, MatMul, Add, Mul, Sigmoid, Concat, …)
  Quantizable ops:  ~118 (Conv + MatMul + …)
  QDQ already?      No
  External data:    No (6.2 MB inline)
  Risks:            None
  NPU CNN target:   Compatible (Conv-heavy, no unsupported ops)
```

### >>> CHECKPOINT 1: Confirm model analysis is correct before continuing

---

From amd/Quark, MIT.

More from amd/Quark