quark-onnx-install

Files (1)

• SKILL.md

SKILL.md (first lines)

# quark-onnx-install

## Purpose

Install the correct ONNX Runtime build for the user's accelerator backend, plus the matching `onnx`
package and supporting tooling (`onnxslim`, `onnxscript`). ONNX Runtime must be installed before
Quark's ONNX-to-ONNX flow because Quark uses ORT as the calibration / inference engine and registers
custom ops (`BFPQuantizeDequantize`, `MXQuantizeDequantize`, `Extended*`) into it. Getting this wrong —
installing the CPU build on a GPU machine, or installing both `onnxruntime` and `onnxruntime-gpu`
side-by-side — causes EP-not-available errors, silent CPU fallback, or import-time DLL conflicts that
are hard to diagnose later. This skill exists separately from `quark-install` so that ONNX Runtime
setup has a clear, single-responsibility boundary, parallel to `quark-torch-install` for Torch.

## Inputs

- `env_context.json` with detected accelerator info (CPU / CUDA major+minor / ROCm major+minor)

## Outputs: onnx_install_result.json

Records the installed ONNX Runtime build, accelerator backend tag, the `onnx` package version, and
verification status.

```json
{
  "status": "ok",
  "onnxruntime_package": "onnxruntime-gpu",
  "onnxruntime_version": "1.23.2",
  "accelerator_tag": "cuda-12",
  "onnx_version": "1.18.0",
  "onnxslim_version": "0.1.84",
  "onnxscript_version": "0.1.0",
  "verification": {
    "import_onnx_ok": true,
    "import_onnxruntime_ok": true,
    "available_providers": ["CUDAExecutionProvider", "CPUExecutionProvider"],
    "expected_provider_present": true,
    "custom_ops_compile_ok": true
  }
}
```

On failure, set `status: "failed"` and include a `failure_reason` with the exact failing verification
command.

## Python Version Requirements

- **Supported**: Python 3.11, 3.12, 3.13
- **Not supported**: Python 3.14+
- **Recommended for new setups**: Python 3.13 via Miniforge/Miniconda

## Package Version Matrix

**Authoritative sources**:

- `tools/ci/install_onnxruntime.sh` — accelerator → `onnxruntime*` variant + version mapping (CI truth)
- `docs/source/install.rst` — user-facing supported version range
- `requirements.txt` — `onnx`, `onnxscript`, `onnxslim` pin

Before generating install commands, **always read these sources** to get the current verified
combinations. Do not memorize version numbers — the matrix changes with each Quark release.

### Current ranges (verify before use)

| Package | Range (verify against requirements.txt / install.rst) |
|---------|-------------------------------------------------------|
| `onnx` | `>=1.21.0, <=1.22.0` |
| `onnxruntime*` | `>=1.22.2, <=1.25.1` |
| `onnxslim` | `>=0.1.84` |
| `onnxscript` | unpinned |

### How to read the source

1. Open `tools/ci/install_onnxruntime.sh` and locate the `install_onnxruntime` function. It dispatches
   on `accelerator_version` (`cpu`, `cuda-11.*`, `cuda-12.*`, `rocm-*`) and decides:
   - which variant to install (`onnxruntime`, `onnxruntime-gpu`, `onnxruntime_rocm`),
   - whether to use pypi.org or the AMD internal Artifactory wheel.
2. Cross-check the chosen `onnxruntime` version against the range in `docs/source/install.rst`
   (search for "ONNX Runtime version").
3. Read `requirements.txt` for the `onnx` / `onnxslim` / `onnxscript` constraints.
4. Construct the install commands using the patterns below.

### Install command patterns

#### CPU

```bash
pip install "onnxruntime>=1.22.2,<=1.25.1"
pip install "onnx>=1.21.0,<=1.22.0" "onnxslim>=0.1.84" onnxscript
```

#### CUDA 12.x / 13.x

Matches the current `install.rst` recommendation:

```bash
pip install onnxruntime-gpu                # default pypi build targets recent CUDA
pip install "onnx>=1.21.0,<=1.22.0" "onnxslim>=0.1.84" onnxscript
```

#### CUDA 11.x

Per `install_onnxruntime.sh`:

```bash
pip install --no-cache-dir onnxruntime-gpu \
  --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/
pip install "onnx>=1.21.0,<=1.22.0" "onnxslim>=0.1.84" onnxscript
```

#### ROCm 6.x

Internal `onnxruntime_rocm` wheel from AMD Artifactory (no pypi build):

```bash
# Resolved via _install_onnxruntime_from_artifactory in tools/ci/install_onnxruntime.sh
# Wheel pattern: onnxruntime_rocm-<ort_ver>-cp<py_ver>-*.whl
# Base URL: https://xcoartifactory.xilinx.com/artifactory/uai-pip-local/onnxruntime/rocm-<ver>
```

From amd/Quark, MIT.

More from amd/Quark