ffmpeg-ops

Files (36)

• SKILL.md
• edl-schema.json · assets/
• encoding-presets.json · assets/
• hls-ladder.json · assets/
• analysis-validation.md · references/
• audio.md · references/
• capture-devices.md · references/
• color-grading.md · references/
• color-hdr.md · references/
• cookbook.md · references/
• edit-as-code.md · references/
• encoding.md · references/
• error-decoder.md · references/
• filtergraph.md · references/
• hardware-accel.md · references/
• images-gif.md · references/
• look-recipes.md · references/
• quality-metrics.md · references/
• restoration.md · references/
• streaming-hls.md · references/
• stt-whisper.md · references/
• subtitles.md · references/
• trim-concat.md · references/
• visualization.md · references/
• capability-scan.sh · scripts/
• cut-from-edl.py · scripts/
• detect-segments.py · scripts/
• gen-luts.py · scripts/
• loudnorm-scan.py · scripts/
• make-chapters.py · scripts/
• make-sprites.py · scripts/
• probe-media.py · scripts/
• quality-compare.py · scripts/
• smart-compress.py · scripts/
• verify-commands.sh · scripts/
• run.sh · tests/

SKILL.md (first lines)

# ffmpeg Operations

Operational expertise for ffmpeg/ffprobe: the ~30 commands that cover most real work,
the footguns that silently ruin output, EDL-driven editing (edit-as-code), and eight
scripts that replace the logic an agent would otherwise re-derive every task.

## Doctrine: probe first

**Never transcode, cut, or filter blind.** Every media task starts by probing the
input — codec, duration, frame rate (constant or variable?), pixel format, rotation,
stream layout. Half of all "ffmpeg did something weird" reports are a property of the
*input* the command never checked.

```bash
python skills/ffmpeg-ops/scripts/probe-media.py input.mp4            # human summary
python skills/ffmpeg-ops/scripts/probe-media.py --doctor input.mp4   # TRIAGE: hazards + exact fixes
python skills/ffmpeg-ops/scripts/probe-media.py --json input.mp4 | jq '.data.streams'
python skills/ffmpeg-ops/scripts/probe-media.py --keyframes-near 92.5 input.mp4
```

`--doctor` makes the doctrine self-enforcing: VFR, HDR transfer, rotation
metadata, interlacing, non-yuv420p delivery, and moov-at-EOF each come back as a
finding **with the exact fix command**, and exit 10 means "fix before processing".
The `--keyframes-near` form answers "can I stream-copy a cut at 92.5s?" — it
reports the nearest keyframes so you know whether a copy cut will snap (see
Footguns). When a command fails with a cryptic message, decode it:
[references/error-decoder.md](references/error-decoder.md).

**Before recommending an encoder, verify the build has it.** Installed ffmpeg builds
vary wildly (especially hardware encoders — *listed* ≠ *working*):

```bash
bash skills/ffmpeg-ops/scripts/capability-scan.sh           # full: proof-encodes each hw encoder
bash skills/ffmpeg-ops/scripts/capability-scan.sh --quick   # list-only, no GPU touch
```

## Cookbook

Copy-paste recipes, each with the trap it avoids in its comments: [references/cookbook.md](references/cookbook.md). Commands are bash-form (see [Windows notes](#windows-notes)); always pass `-y` or `-n` so no agent-run command goes interactive.

| Need | Cookbook section |
|---|---|
| Web H.264, HEVC, AV1, remux, VFR normalize, FFV1 archive, fit-to-size | Convert and compress |
| Copy vs frame-accurate trim, concat demuxer vs filter | Cut and join |
| Scale, crop, 9:16 blurred pad, rotate, fps, speed, slow-mo, timelapse | Resize, transform, retime |
| Watermark, drawtext timecode, burned vs soft subtitles | Overlay, text, subtitles |
| Extract, replace, mix, loudness, trim silence | Audio |
| 16 kHz mono STT extraction, piping, silence chunking | Speech-to-text prep |
| Thumbnails, contact sheets, GIFs, chapters, dataset frames, sprites | Images, GIFs, frames |
| Corruption check, frame hashes, metadata strip, ffprobe one-liners | Diagnostics and validation |
| Post-download yt-dlp patterns | yt-dlp interop |
| Synthetic test fixtures | Generative/test sources |

## Footguns

The table that pays this skill's rent. Each row is a class of silent failure.

| Footgun | The trap | The rule |
|---|---|---|
| `-ss` + `-c copy` | Cut starts seconds early or with frozen/black lead-in (snapped to prior keyframe) | Copy cuts snap. Check `probe-media.py --keyframes-near`; re-encode when exact |
| Output-side `-to` after input-side `-ss` | Timestamps reset at the seek point, so `-to` silently becomes a *duration* | Keep `-ss`/`-to` on the same side of `-i` (both input-side is fast and absolute) |
| Missing `-pix_fmt yuv420p` | Encode "works" but Safari/QuickTime/TVs show black or refuse to play (defaulted to yuv444p/yuv422p from a high-quality source) | Always set it for delivery H.264/H.265 |
| Missing `-movflags +faststart` | Browser can't start playback until the whole file downloads (moov at EOF) | Always set it for web-served MP4 |
| Default stream selection | ffmpeg picks ONE stream per type (highest-res video, most-channels audio) — extra audio tracks and all subs are silently dropped | `-map 0` to keep everything, explicit `-map` otherwise |
| `-vf` + `-c:v copy` together | Hard error — filters require decoding | Filtering implies re-encode; pick one |
| VFR source (phone/Zoom/Loom/screen-rec) | Cut math drifts, concat desyncs, players stutter | Normalize first: `-fps_mode cfr -r 30` + re-encode (cookbook) |
| `-vsync` (deprecated) | Old flag, removed direction | Use `-fps_mode` (cfr/vfr/passthrough) |
| `scale=W:-1` | Odd height → encoder error with yuv420p | Always `-2` |
| concat demuxer on mismatched inputs | "Works" then glitches/desyncs at boundaries (codec/timebase mismatch) | Demuxer = identical params only; else concat *filter* with re-encode |
| amix default | Each input's volume halved (normalize defaults on) | `amix=...:normalize=0` + explicit `volume=` |
| One-pass loudnorm | Dynamic mode pumps quiet passages; output silently 192 kHz | Two-pass linear via `loudnorm-scan.py`; add `-ar 48000` |
| `-shortest` absent on audio-replace | Output runs as long as the LONGEST input (silence or frozen frame tail) | Add `-shortest` when muxing separate A/V |
| BT.601/709 colour shift | Slightly wrong colours after scaling SD↔HD (matrix guessed from resolution) | Tag explicitly when it matters: see [references/color-hdr.md](references/color-hdr.md) |
| drawtext/subtitles path escaping | Filter args re-parse `:` and `\` — Windows paths like `C:\x` explode inside filter strings | cd to the asset's dir and use bare relative names; or escape as `C\:/path` |
| Interactive overwrite prompt | Agent-run command hangs forever on "File exists. Overwrite? [y/N]" | Always pass `-y` or `-n` explicitly |
| `%` in cmd.exe | `%06d` patterns and `%{pts}` get mangled by cmd variable expansion | Use PowerShell or bash; in .bat double to `%%` |

### Windows notes

Platform-agnostic commands, but when running on Windows:

- **PowerShell quoting is friendlier than bash here**: single quotes are fully
  literal, so `-vf 'scale=1280:-2,fps=30'` needs no escaping. Double quotes only
  interpolate `$` and backtick — filtergraphs rarely contain either.
- **`NUL` not `/dev/null`** for two-pass logs: `-passlogfile` defaults are fine, but
  `ffmpeg ... -f null NUL` (PowerShell also accepts `-f null -`, which is portable —
  prefer it).
- **Font paths in drawtext**: `fontfile='C\:/Windows/Fonts/arial.ttf'` — forward
  slashes, escaped drive colon, inside the filter string.
- **Prefer `-f null -` and relative paths** to sidestep both quoting tables at once.

## Decision trees

**Codec** — `H.264 (libx264)`: default; universal playback, fast, good per-bit at
`-crf 18..23`. → `H.265 (libx265)`: same quality ~40% smaller; slower; needs
`-tag:v hvc1` for Apple; fine for storage/modern devices. → `AV1 (libsvtav1)`: best
compression, royalty-free, web-first (YouTube/Netflix path); encode cost highest;
playback on older hardware is software-only. → `VP9`: only when a pipeline demands
webm and AV1 is unavailable. → `FFV1`: archival masters only.

**Cut method** — Need exact frames OR applying any filter → re-encode (input-side
`-ss`, `-crf 18`). Cut points happen to sit on keyframes (verify with
`--keyframes-near`) OR a ±2s slop is acceptable → stream copy with
`-avoid_negative_ts make_zero`. Many cuts from one source → EDL workflow below.

**CPU vs hardware encode** — Hardware (NVENC/QSV/AMF/VideoToolbox) is 5-20× faster
but **worse quality per bit** than libx264/x265 at slow presets. Use hardware for:
speed-critical batch work, live/streaming, drafts, "good enough" deliveries (bump
bitrate ~30% to compensate). Use CPU for: final masters, size-constrained targets,
quality comparisons. Always `capability-scan.sh` first — listed encoders fail at
runtime on driver mismatches. Details: [references/hardware-accel.md](references/hardware-accel.md).

## EDL workflow (edit-as-code)

For any multi-cut edit, do not fire ad-hoc trim commands. Write an **edit decision
list** — a JSON file naming every clip, time range, and *why* — then cut from it.
The edit becomes reviewable (rationale is written down), rerunnable (regenerate the
output any time), and diffable (versions of the edit are git history).

From 0xDarkMatter/claude-mods, MIT.

More from 0xDarkMatter/claude-mods