hyperloom-workload-optimizer

Files (5)

• .federated.json
• SKILL.md
• evals.json · evals/
• machine.yml · evals/
• skill-card.md

SKILL.md (first lines)

<!--
Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.

See LICENSE for license information.
-->

# Hyperloom Workload Optimizer

You are the entry point for Hyperloom optimization on AMD Instinct GPUs. Your job
is the bootstrap: confirm the workspace, install the Hyperloom wheel, run
`/hyperloom-setup`, then hand the run to the skill that owns it.

The wheel installs the skills that own everything after setup: `hyperloom-setup`
for credentials and run mode, the demo skills for a workload preset, and
`inference_optimizer` for the launcher gates and monitoring. They ship with the
runtime, so they always match the installed version.

## Out of scope for this skill

- Do not run `python -m hyperloom.inference_optimizer.cli optimize` yourself.
- Do not implement a GPU preflight, launcher gate, or background launch here. The
  installed skills own those, including the Iron Rules and the resume path.
- Do not ask for workload values (model, TP/EP, concurrency, ISL/OSL, precision,
  objective, budget) while installing or while setup is running. They belong to
  the run skill, after setup finishes.
- Do not optimize by hand in chat.

## Prerequisites

- AMD Instinct GPU host (MI300X / MI325X / MI355X) with ROCm, `/dev/kfd` and
  `/dev/dri` present, and `amd-smi` or `rocm-smi` working.
- Python 3.10+ and `pip` on the machine that runs the install.
- LLM credentials: Anthropic API access, or the AMD LLM gateway.
- A dedicated empty directory, opened in the agent as the workspace.

Confirm the shell you are in is on the GPU host before installing. Setup may later
point Docker at a different target host; until it does, everything here runs where
the agent is.

## Step 1: Confirm the workspace

The current directory is both the install target and the agent workspace. Confirm
with the user that it is a dedicated directory before installing: setup creates or
updates `.env` in it. Do not switch to another directory on your own, and do not
install into an existing project unless the user accepts the `.env` change.

## Step 2: Install the Hyperloom wheel

```bash
pip install hyperloom-inference-optimizer --target .
```

Install the current release unless the user asks for a specific version. It is
normal for the directory to hold many Python package folders afterwards; the user
does not need to inspect them.

## Step 3: Run `/hyperloom-setup`

The wheel installs `hyperloom-setup` into the agent's skill directories
(`.agents/skills/`, `.claude/skills/`, `.cursor/skills/`). Run it:

```text
/hyperloom-setup
```

It is interactive and owns credentials, `USER_DATA_PATH`, the run mode
(`docker` recommended, or `baremetal`), the Docker target host, and the bare-metal
framework install. It writes `.env` and stops before any optimization. Run it once
per workspace; the run skills reuse those values.

Let setup ask its own questions. Do not preempt them, do not restate its option
lists, and never ask the user to paste an API key into chat.

If the agent does not list `hyperloom-setup` after the install, the skill
directories were written after the agent scanned them. Tell the user to restart
the agent, then run it again. Do not substitute your own setup steps.

## Step 4: Hand off to a run skill

Setup ends by offering a run and loading the matching skill, so normally you just
follow it. When the user asks for a run directly, load the skill by name and follow
its instructions instead of this one:

- `hyperloom-qwen3-8b-3h` — short no-kernel Qwen3-8B run; best first end-to-end check.
- `hyperloom-qwen3-14b-fp8-12h` — medium-length Qwen3-14B-FP8 run.
- `hyperloom-qwen3-14b-fp8-12h-forge` — the same run on the KernelForge kernel backend.
- `hyperloom-custom-advanced` — explicit model, framework, workload, budget, and phase toggles.

A preset keeps its workload even if the user supplies their own `MODEL_PATH`;
tensor parallelism, concurrency, sequence lengths, precision, and budget are not
retuned for that model. When those need to change, use `hyperloom-custom-advanced`.

To resume a stopped session, follow the optimizer skill at `.env`
`HYPERLOOM_SKILL_PATH`; it owns the resume path and the gates a relaunch still has
to clear.

## What to expect during a run

Optimization runs for hours in the background. Do not stream the log.

Before launch the run skill shows a plan: resolved model path, run mode, framework,
TP, concurrency, ISL/OSL, precision, budget, and `USER_DATA_PATH`. Get the user's
go-ahead on that plan before the optimizer starts — it then owns the GPU for hours.
After launch it reports the optimizer PID, run log, launch-info JSON, session
directory, `state.json`, and the first health check.

During the run, report a short status about every 300 seconds: process alive,
current phase, `stop_reason`, baseline and current best throughput, cumulative
gain, the latest benchmark or candidate decision, and the most relevant log lines.
Never print API keys, tokens, or custom headers.

## Troubleshooting

- Many package folders in the workspace after `pip install --target .` is expected.
- `/hyperloom-setup` not listed: the install landed after the agent scanned for
  skills. Restart the agent and check `.claude/skills/hyperloom-setup/` exists.
- `ImportError: libamdhip64.so.7` or `libhipblas.so.3`: the framework torch wheel
  wants different ROCm user-space libraries; align `ROCM_PATH` and
  `LD_LIBRARY_PATH`.

From amd/skills, MIT.

More from amd/skills