quark-torch-quant-perf

Files (4)

• SKILL.md
• evals.json · evals/
• cli-mapping.md · references/
• session-lifecycle.md · references/

SKILL.md (first lines)

# Quark Torch Quant-Perf

## Purpose

Translate a user's PyTorch or HuggingFace quantization request into the current
`quark-quant-perf` CLI, run or resume its managed Orchestrator, monitor the
session to a terminal state, and report the generated artifacts. Preserve the
user's accuracy, workload, source, and explicit performance requirements
throughout.

Use the managed pipeline:

```text
input and runtime validation
→ baseline health
→ quantize or mixed-precision search
→ real quantized accuracy gate
→ [measure or optimize] baseline and quantized throughput
→ [optimize, when the target is missed] conditional TraceLens / vendor tuning /
  GEAK → candidate validation and retention
→ FINAL reports
```

Performance modes skip inapplicable optional stages; they do not create
separate FINAL stages. The Orchestrator enters FINAL after the selected stages
complete or a terminal failure is recorded. The `report` subcommand may later
regenerate terminal artifacts without rerunning GPU work.

Do not replace a stage with a custom evaluation, benchmark, or proxy metric.

## Inputs

- Model path or HuggingFace model ID.
- Accuracy-gap requirement and optional performance intent.
- GPU type, GPU index, TP, ISL, OSL, and concurrency when specified.
- Fixed quantization intent, or a mixed-precision search space.
- Framework and kernel source policy: explicit, auto, or readonly.
- A discoverable Quark checkout containing `quark-torch-ptq` for fixed direct
  PTQ; `QUARK_ROOT` may select one explicitly.
- Existing session directory when resuming.
- Optional TraceLens architecture JSON and backend overrides.

Only `--model` is required by the CLI. Preserve CLI defaults for omitted
options rather than inventing values.

Before constructing a command:

1. Run `quark-quant-perf --help` for the installed CLI options and defaults.
2. Read [CLI mapping](references/cli-mapping.md) for intent-to-option mapping.
3. Read [session lifecycle](references/session-lifecycle.md) before resume,
   monitoring, diagnosis, or final reporting.
4. Inspect the session's `state.json` and `progress.json` when resuming or
   reporting.

Treat the installed CLI help as authoritative if a local reference differs.

## Outputs: session_report.md

The managed FINAL stage produces:

```text
<session>/reports/final.json
<session>/reports/final.md
<session>/session_breakdown.json
<session>/session_report.md
```

Treat `session_breakdown.json` as the complete structured fact source and
`session_report.md` as the detailed human-readable result. A terminal failure
may still have complete reports and useful quantized artifacts.

## Interaction Flow

1. **Intake**
   - Resolve local paths to absolute paths.
   - Preserve the requested model, accuracy gap, performance mode, target gain,
     workload, GPU, tensor parallelism, search modes, and repository policy.
   - Treat explicitly named quantized precision modes as a closed set, separate
     from any runtime-required `native` fallback. Do not infer compound modes
     from their components. Normalize aliases one-to-one; for example, use
     `mxfp4_fp8` only when the user explicitly requests that mode, W4A8, or
     MXFP4 weights with FP8 activations.
   - If the user does not name precision modes, omit
     `--layer-precision-candidates` and preserve the GPU-specific automatic
     search space.
   - If the user does not request throughput or performance optimization,
     leave performance at the default `off`.
   - Inspect existing session state before creating a duplicate run.

2. **Route**
   - Omit `--quant-strategy` for mixed-precision search.
   - Use `--quant-strategy "<normalized intent>"` for one fixed PTQ recipe.
   - For fixed PTQ, use the automatically discovered Quark checkout when
     available. Set `QUARK_ROOT` only when discovery fails or the user selects
     a specific checkout.
   - When writable framework or kernel repositories are supplied, use
     `--workspace-source explicit` with those repositories.
   - When repositories are not supplied, use `--workspace-source auto` so
     runtime repair and PerfOpt can discover or materialize writable sources.
   - Use `--workspace-source readonly` only when the user explicitly requests
     evaluation or inspection without source modification.
   - Treat fresh or no-history execution as an experience-store policy, not a
     source policy. It never implies `readonly`; keep `auto` unless the user
     separately prohibits source modification.
   - Before launch, if the command contains `--workspace-source readonly`,
     identify the user's explicit no-source-modification request. If there is
     none, remove the option and use the default `auto`.
   - Apply the source policy to every run. It governs runtime activation and
     eligible repair work even when performance mode is `off` or `measure`;
     PerfOpt source modification remains specific to `optimize`.

3. **Plan**
   - Maintain the native task plan for every multi-stage run. Use
     `update_plan` in Codex and `TaskCreate` / `TaskUpdate` in Claude Code.
   - Before launch, show every applicable top-level task derived from the
     resolved run intent. Always include validation, baseline health,
     quantization or search, the real accuracy gate, and FINAL reporting. Add
     throughput for `measure` or `optimize`, and add the PerfOpt decision plus
     conditional bottleneck, optimization, and retention tasks for `optimize`.

From amd/Quark, MIT.

More from amd/Quark