quark-onnx-model-intake

Files (1)

• SKILL.md

SKILL.md (first lines)

# quark-onnx-model-intake

## Purpose

Validate the target `.onnx` model and extract the structural facts that an ONNX quant plan needs
to make correct quantization decisions. This step exists because different ONNX models have very
different quantization requirements — a CNN destined for AMD NPU CNN needs NHWC layout and PoF2
scales, an LLM exported to ONNX needs MatMulNBits or SmoothQuant + GPTQ, a >2 GB model needs
external-data handling, and a model that already contains QDQ nodes (or Quark's custom-op domains)
must not be silently re-quantized.

## Inputs

- Model path — a single `.onnx` file, or a directory containing `model.onnx` plus its
  `model.onnx_data` / external-weight blobs
- `env_context.json` for Python and accelerator constraints (CUDA major / ROCm major / NPU
  presence)
- `workspace_context.json` for the validated model path
- `onnx_install_result.json` (optional) — the installed `onnxruntime*` variant determines which
  execution providers are realistic deployment targets

## Outputs: model_analysis.json

Captures the model's graph metadata, op-type coverage, quantizable-op count, deployment-target
compatibility, and risks. Schema is shared with the Torch intake; ONNX-specific fields live under
`model.onnx_specific` and `quantization_targets.onnx_specific`.

Schema: [`model_analysis.schema.json`](../../shared/contracts/model_analysis.schema.json)

```json
{
  "analysis_status": "complete",
  "model": {
    "model_path": "/data/models/resnet50/model.onnx",
    "model_type": "onnx",
    "loading_class": "onnx.load",
    "estimated_size_gb": 0.10,
    "onnx_specific": {
      "ir_version": 8,
      "producer_name": "pytorch",
      "producer_version": "2.5.1",
      "opsets": [{"domain": "ai.onnx", "version": 17}],
      "uses_external_data": false,
      "external_data_files": [],
      "input_layout_guess": "NCHW",
      "inputs": [
        {"name": "input", "dtype": "float32", "shape": [1, 3, 224, 224]}
      ],
      "outputs": [
        {"name": "output", "dtype": "float32", "shape": [1, 1000]}
      ],
      "has_dynamic_dims": false,
      "has_qdq_already": false,
      "has_quark_custom_ops": false,
      "architecture_guess": "cnn"
    }
  },
  "quantization_targets": {
    "linear_layer_count": 53,
    "onnx_specific": {
      "op_counts": {"Conv": 53, "BatchNormalization": 53, "Relu": 49, "MaxPool": 1, "GlobalAveragePool": 1, "Gemm": 1},
      "quantizable_op_counts": {"Conv": 53, "Gemm": 1, "MatMul": 0, "Add": 16, "Mul": 0},
      "preset_candidates": ["XINT8", "A8W8", "INT8_CNN_DEFAULT"],
      "exclude_defaults": []
    }
  },
  "risks": [
    {
      "severity": "low",
      "message": "Inputs are NCHW. AMD NPU CNN target expects NHWC.",
      "recovery_hint": "Run quark.onnx.tools.convert_nchw_to_nhwc before quantization."
    }
  ]
}
```

## Supported Domains and Preset Families

Quark ONNX ships a large `DefaultConfigMapping` of named presets in
`quark/onnx/quantization/config/custom_config.py`. The intake should narrow the candidate set
based on what the model actually looks like:

| Category | Representative presets | When to suggest |
|----------|------------------------|-----------------|
| **AMD NPU CNN (XINT8 family)** | `XINT8`, `XINT8_ADAROUND`, `XINT8_ADAQUANT`, `XINT8_WEIGHTSONLY_ADAROUND` | Model is Conv-heavy (CNN), inputs are 4D, deployment target is `npu_cnn`; requires PoF2 scales and (usually) NHWC layout |
| **AMD NPU Transformer (INT16 transformer)** | `INT16_TRANSFORMER_DEFAULT`, `INT16_TRANSFORMER_ACCURATE` | Model is MatMul/Gemm-heavy, deployment target is `npu_transformer` |
| **General CPU/GPU INT8** | `A8W8`, `A8W8_ADAROUND`, `A8W8_ADAQUANT`, `INT8_CNN_DEFAULT`, `INT8_CNN_ACCURATE` | Generic INT8 deployment on CPU/CUDA/ROCm |
| **A16W8 / wide activations** | `A16W8`, `A16W8_ADAROUND`, `A16W8_ADAQUANT` | Activations need 16-bit headroom (outlier-heavy models) |
| **Block formats (custom-ops required)** | `BFP16`, `BFP16_ADAQUANT`, `MX4`, `MX6`, `MX9`, `MXFP4`, `MXFP6`, `MXFP8`, `BF16_MIXED_BFP16`, `BF16_MIXED_MXINT8` | Modern AMD accelerators with `quark.onnx.operators.custom_ops` available |
| **Float fallbacks** | `FP16`, `BF16`, `FP16_ADAQUANT`, `BF16_ADAQUANT` | Accuracy-first; no INT quant |

`preset_candidates` in `model_analysis.json` should list the 1–3 presets that are realistic given
architecture + deployment target; never list all of them.

## What to Extract

### From the ONNX model proto (without loading weights)

- **IR version**: `model.ir_version`
- **Producer**: `model.producer_name`, `model.producer_version`
- **Opsets**: `[(opset.domain or 'ai.onnx', opset.version) for opset in model.opset_import]` —
  for QDQ insertion, `ai.onnx` opset must be ≥ 13 (≥ 19 recommended for full INT4/FP8 support)
- **Custom domains**: presence of `com.amd.quark` or `com.microsoft` indicates the model has
  already been processed by Quark or ORT and should not be naively re-quantized
- **External data**: `os.path.exists(model_path + '_data')` or any tensor with
  `tensor.data_location == TensorProto.EXTERNAL`; total file size > 2 GB without external data is
  invalid
- **Inputs / outputs**: name, dtype (from `type.tensor_type.elem_type`), shape (dim_value or
  symbolic `dim_param`)
- **Dynamic dims**: any `dim_param` or `dim_value == 0` in inputs → flag as risk for NPU targets
- **Input layout guess**: 4D float input with channel dim == 3 or small power-of-2 → likely NCHW
  (channel-first) or NHWC (channel-last); use the convention of the producer
- **Architecture guess**: histogram of op types — Conv-dominated → CNN; MatMul/Gemm + softmax →
  Transformer; LSTM / GRU / RNN → recurrent; mix of Conv + MatMul + Attention → hybrid

### Quantization Targets

- **Quantizable op counts**: count `Conv`, `Gemm`, `MatMul`, `ConvTranspose`, `Add`, `Mul`,
  `BatchNormalization` (will be folded into Conv during pre-processing)

From amd/Quark, MIT.

More from amd/Quark