comfyui-video-production

Files (14)

• SKILL.md
• EVAL.md · eval/
• benchmark.yaml · eval/
• .gitignore · eval/results/
• .gitkeep · eval/results/
• run-eval.sh · eval/
• test-cases.yaml · eval/
• trigger-tests.yaml · eval/
• api-reference.md · references/
• concatenation.md · references/
• evolution.md · references/
• instance-management.md · references/
• troubleshooting.md · references/
• workflows.md · references/

SKILL.md (first lines)

# ComfyUI Video Production Pipeline

> All recommendations in this skill are subordinate to `state/inventory.json`, `foundation/hardware-profile.md`, and current version constraints.

End-to-end video production orchestration for ComfyUI with explicit validation,
evidence-based fallback paths, and optional instance-management patterns.

## Quick Start: Which Pipeline?

**Creating a multi-shot narrative video?**
→ **Keyframe Pipeline** - Generate keyframes → Animate → Stitch with transitions

**Animating existing images?**
→ **I2V Batch Pipeline** - Load images → Queue I2V jobs → Auto-validate → Combine

**Need smooth transitions between scenes?**
→ **Transition Pipeline** - Crossfades, motion blur, zoom effects via FFmpeg

**ComfyUI stuck or crashed?**
→ **Instance Management Pattern** - Health evidence and recovery planning when implemented

**Debugging video issues?**
→ **Validation Suite** - Check resolution, FPS, codec, face consistency, color grading

---

## Core Pipelines

### Pipeline 1: Keyframe-to-Video (Complete Narrative)

**Use when:** Creating story-driven videos with multiple distinct shots

```
1. Keyframe Generation Phase
   - Generate consistent keyframes with IP-Adapter/LoRA
   - Validate face consistency, lighting, pose progression
   - Save to organized directory structure
   - Auto-retry failed generations

2. I2V Animation Phase
   - Queue each keyframe to I2V model (Wan 2.2, LTX-2, AnimateDiff)
   - Monitor progress via ComfyUI API
   - Validate each clip (resolution, fps, duration)
   - Auto-retry with different seeds if failed

3. Concatenation Phase
   - Pre-flight validation (ensure all clips match)
   - Apply transition effects (crossfade, motion blur)
   - FFmpeg encoding with proper codec
   - Export final video with metadata

4. Quality Assurance
   - Face consistency check across clips
   - Color grading consistency
   - Audio sync validation (if applicable)
   - Generate QA report
```

**Expected output:** Single cohesive video with smooth transitions

---

### Pipeline 2: Batch I2V Processing

**Use when:** You have multiple images to animate independently

```
1. Image Discovery
   - Scan directory for source images
   - Validate image specs (resolution, format)
   - Generate processing manifest

2. Parallel I2V Queue
   - Queue all images to ComfyUI with appropriate prompts
   - Stagger submissions to avoid overload
   - Monitor queue depth and ETA

3. Progressive Validation
   - Check each completed video immediately
   - Flag issues (wrong resolution, fps, corruption)
   - Auto-retry flagged videos

4. Export & Organize
   - Move validated videos to output directory
   - Generate index with metadata
   - Create contact sheet (thumbnail preview grid)
```

**Expected output:** Directory of validated animated clips

---

### Pipeline 3: Assembly Handoff

After clip validation, pass ordered clips, timing, audio, and delivery requirements to
`video-assembly`. This skill owns production orchestration; it does not duplicate the
FFmpeg editing workflow.

---

## Model Selection Candidates

### Image-to-Video Models

| Candidate | Select only when |
|-----------|------------------|
| LTX family | The exact installed variant supports the requested resolution and features |
| Wan 2.2 A14B/MoE | Inventory and hardware validation show the exact files and precision fit |
| Wan 2.2 TI2V 5B | The installed workflow supports the requested conditioning mode |
| AnimateDiff | Required motion modules and node classes are inventoried |
| SVD | The installed checkpoint and requested clip constraints match |

---

## Optional Instance Management

Use instance-management patterns only when session state identifies the actual endpoints
and an implementation exists to perform them. Observe queue, process, disk, and VRAM
evidence before classifying a stall. Never describe restart, failover, queue restoration,

From calebmills99/VUORSE-VORTEX, MIT.

More from calebmills99/VUORSE-VORTEX