comfyui

Files (3)

• SKILL.md
• comfy_client.py · scripts/
• txt2img_flux2_klein.json · scripts/workflows/

SKILL.md (first lines)

# ComfyUI (from zero → first image)

Local image generation via [ComfyUI](https://github.com/Comfy-Org/ComfyUI)
REST API, driven by Remedy’s built-in `comfyui` tool.

**You can start with nothing installed.** If the user asks for images and
ComfyUI is missing, follow **From scratch bootstrap** below (with their approval
for downloads / shell). Then generate with the native tool.

This skill pairs with Remedy Desktop (or CLI) **connected to any LLM provider**
(xAI, OpenAI, Ollama, …). The provider runs the agent; ComfyUI runs **on the
user’s GPU/CPU** for pixels.

## Defaults (portable — any machine)

| Item | Value |
|------|--------|
| Base URL | `http://127.0.0.1:8188` (also probes 8189, 8190, 8000, …) |
| Env | `COMFYUI_URL`, `COMFYUI_HOME`, `COMFYUI_PORT` (or `REMEDY_*`) |
| Config | `comfyui_url` / `comfyui_home` / `comfyui_port` in `~/.remedy/config.toml` |
| Side file | `~/.remedy/comfyui.json` → `{"url":"...","home":"..."}` |
| Discovery | `comfyui` action=`locate` (API ports + process + bounded home search) |
| Starter workflow | `scripts/workflows/txt2img_flux2_klein.json` (API format) |

**Do not** `list_dir` the whole disk hunting for ComfyUI. Use `action=locate` /
`status`. Discovery is built in for every OS/user.

---

## Agent decision tree (always start here)

1. Call **`comfyui` / `action=status`** (or `locate`).
2. **API up** → jump to **Generate images**.
3. **Install found, API down** → **Start ComfyUI** (use `start_hint` from the tool).
4. **Nothing found** → **From scratch bootstrap** (this is expected on a fresh PC).
5. After models are in place and the server is up → **Generate images**.

Never invent success. Never paste DSML/tool XML into chat. Prefer the `comfyui`
tool over raw `curl` (especially on Windows).

---

## From scratch bootstrap (fresh install)

Goal: machine has **no** ComfyUI → user can generate a PNG from Remedy chat.

**Ask once** before large downloads (disk: tens of GB for full Flux.2 Klein;
smaller if using 4B). Confirm GPU type: **NVIDIA** (default), AMD, Intel, or CPU-only.

### Phase A — Install ComfyUI

#### Windows (recommended: official portable)

Docs: https://docs.comfy.org/installation/comfyui_portable_windows  
Releases: https://github.com/Comfy-Org/ComfyUI/releases

| GPU | Download (latest) |
|-----|-------------------|
| NVIDIA (modern RTX) | `ComfyUI_windows_portable_nvidia.7z` |
| NVIDIA (older / CUDA 12.6) | `ComfyUI_windows_portable_nvidia_cu126.7z` |
| AMD | `ComfyUI_windows_portable_amd.7z` |
| Intel | `ComfyUI_windows_portable_intel.7z` |

Direct patterns (resolve via “latest” on GitHub Releases):

```text
https://github.com/Comfy-Org/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia.7z
https://github.com/Comfy-Org/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia_cu126.7z
https://github.com/Comfy-Org/ComfyUI/releases/latest/download/ComfyUI_windows_portable_amd.7z
https://github.com/Comfy-Org/ComfyUI/releases/latest/download/ComfyUI_windows_portable_intel.7z
```

Agent steps (with approval):

1. Pick install root — prefer a short path with space, e.g.  
   `%USERPROFILE%\ComfyUI_windows_portable` or `D:\AI\ComfyUI_windows_portable`.
2. Download the matching `.7z` (browser, `curl -L -o …`, or PowerShell  
   `Invoke-WebRequest -Uri … -OutFile …`). File is large — warn the user.
3. Extract with **7-Zip** (https://7-zip.org/). Nested archives need full extract.
4. Expected layout after extract:

```text
ComfyUI_windows_portable/
  ComfyUI/                 ← main.py lives here (this is COMFYUI_HOME)
  python_embeded/
  run_nvidia_gpu.bat       ← or run_amd_gpu.bat / run_intel_gpu.bat / run_cpu.bat
  update/
  README_VERY_IMPORTANT.txt
```

5. Persist location for Remedy:

```json
// ~/.remedy/comfyui.json
{
  "home": "C:\\Users\\<you>\\ComfyUI_windows_portable\\ComfyUI",
  "url": "http://127.0.0.1:8188"
}
```

Or set env `COMFYUI_HOME` / `COMFYUI_URL`, or `comfyui_home` / `comfyui_url` in
`~/.remedy/config.toml`.

#### Windows alternative: git + venv (advanced)

```bash
git clone https://github.com/Comfy-Org/ComfyUI.git
cd ComfyUI
python -m venv .venv
# Windows: .venv\Scripts\activate
pip install -r requirements.txt
# GPU torch: follow ComfyUI / pytorch.org for the right CUDA wheel
python main.py --listen 127.0.0.1 --port 8188
```

#### macOS / Linux (git)

```bash
git clone https://github.com/Comfy-Org/ComfyUI.git

From AhmiDarrow/RemedyAI, Apache-2.0.

More from AhmiDarrow/RemedyAI