vox-video-director

Files (27)

• SKILL.md
• openai.yaml · agents/
• content-timing.beats.json · examples/
• beat-layer.md · references/
• local-engine.md · references/
• models-and-gotchas.md · references/
• prompt-guide.md · references/
• voices.md · references/
• aroll_assemble.py · scripts/
• aroll_clips.py · scripts/
• asr_beats.py · scripts/
• assemble.py · scripts/
• atlas_cloud.py · scripts/
• audio.py · scripts/
• clips.py · scripts/
• confetti.py · scripts/
• croll_keyframes.py · scripts/
• extract_elements.py · scripts/
• kenburns.py · scripts/
• keyframes.py · scripts/
• mg_scrapbook.py · scripts/
• motion.py · scripts/
• provider.py · scripts/
• style_bakeoff.py · scripts/
• styles.py · scripts/
• text_overlay.py · scripts/
• timing.py · scripts/

SKILL.md (first lines)

# Vox Video Director

Turn a one-line topic into a finished **Vox-style paper-collage video**: a bold, punchy,
narrated explainer/ad where each beat is a torn-paper collage poster that comes alive, with
voice-over, optional music and captions. Runs through authenticated **Bailian CLI (`bl`)** + local **ffmpeg**.

The look is the modern editorial paper-collage popularized by Vox explainers and creators
like Stav Zilber / rom1trs: hand-cut paper cut-outs, torn edges, tape, halftone dots,
newspaper clippings, bold flat color per beat, big cut-out headlines.

## The core idea (read this first)

The Vox collage look and the collage motion are **two different steps**:

1. **The look is born in the IMAGE step.** Each beat is a finished collage *poster* made by a
   text-to-image model. All the collage DNA (torn paper, cut-outs, halftone, bold color,
   headline text) lives in that image. If the image isn't a rich collage, nothing downstream
   will save it.
2. **The motion is added after.** By default an AI video model animates the whole poster (the
   "living poster" path — simple, automated). For dramatic *piece-by-piece* assembly you cut
   the poster into parts and drive them with the local keyframe engine (advanced path).

Everything hinges on the prompts. **Before writing any image or video prompt, read
`references/prompt-guide.md`** — it has the exact prompt structures that make the difference
between "a real Vox collage" and "a moving PowerPoint".

## Prerequisites (check, don't skip)

- `bl --version` — requires Bailian CLI 1.14.3 or newer.
- `bl auth status` — if unauthenticated, run `bl auth login --api-key <key>` and stop until login succeeds.
- `command -v ffmpeg ffprobe` — required for assembly (`brew install ffmpeg` on macOS).
- `python3 -c "import PIL"` — Pillow, for captions/watermark overlays.

## Standard workflow (topic → film)

This is the default, most-automated path. Every stage is one script, all driven by a single
`beats.json` per project under `out/<project>/`.

1. **Topic → beat map.** First **read `references/beat-layer.md`** (the story layer) and pick a
   narrative `arc` that fits the topic (`timeline` for history, `pas`/`bab` for ads,
   `how_it_works` for explainers, `man_in_hole` for transformations, …). Then write
   `out/<project>/beats.json` following that arc. **Beat 1 must hook within 3s.** Default to
   **one beat = one generated video segment = one shot**. Size every segment from its narration,
   not a fixed template: target **5–10s**, use `python3 scripts/timing.py out/<project>
   --estimated`, and split any narration that estimates above 10s. A 60s film is usually 7–10
   content-sized beats, not six fixed 10s blocks or twelve fixed 5s shots. Vary `camera_move`
   across adjacent beats (never repeat; use `static` on the payoff) and write rich
   `element_motion`. Each beat needs `narration`, `title_cn`/`title_en`, `scene`, `bg`, `feel`,
   `hook`, and one `shots` item whose `dur` may be omitted. This draft with estimated durations
   is the **first mandatory approval gate**. Use `examples/content-timing.beats.json` as the
   default schema example.

2. **Pick the visual style (hybrid — do this BEFORE keyframes).** Do not reuse one house style
   for every topic. Read `references/prompt-guide.md` (§5 theme presets); pick 3–4 **theme presets**
   (`styles.THEME_PRESETS`: `american-retro`, `swiss-modern`, `punk-zine`,
   `soviet-constructivist`, `wpa-propaganda`, `70s-groovy`, `chinese-ink`, `atomic-age`,
   `newsprint-editorial`) that fit
   the topic's era/culture/tone — **or compose a custom theme** by mixing the prompt-guide dimensions
   (medium/era/palette/type/finish) when none fit. Match the topic, **not** the language (an
   English film on Chinese history should look Chinese). A theme bundles the whole LOOK layer
   (idiom+palette+type+finish+mood+motion). Run a bake-off and let the user pick by eye — AI
   proposes, the library is the quality floor, the human decides. Set the pick as `"theme"`:
   `python3 scripts/style_bakeoff.py out/<project> american-retro,swiss-modern,punk-zine,atomic-age`
   Set the chosen name as `"collage_style"` in beats.json (keyframes.py reads it).

3. **Voice + measured timing.** `python3 scripts/audio.py out/<project>`
   Generates one consistent narrator with **`bl speech synthesize` + cosyvoice-v3-flash**, then
   replaces text estimates with measured audio durations plus a short edit tail. New segments
   clamp to 5–10s. If `timing_issues` is non-empty, split those narration beats and run audio
   again **before** paying for images or videos. Pick `voice_id` for the topic and language; see
   `references/voices.md`. Bailian CLI 1.14.x has no music generation command; provide a local
   `bgm_path` or assemble without music.

4. **Keyframes (the collage look).** `python3 scripts/keyframes.py out/<project>`
   Generates one collage poster per beat/shot with **`bl image generate` + qwen-image-3.0**,
   headline text baked in. Compose prompts with the 5-part structure in
   `references/prompt-guide.md`. Verify each poster looks like a *real layered collage*
   before animating — re-roll cheap ($0.08) here rather than paying to animate a weak image.

5. **Motion.** `python3 scripts/clips.py out/<project>`
   Animates each poster with **`bl video generate` + happyhorse-1.1-i2v**. Two independent axes
   (see `references/beat-layer.md` §3, tested on our stack):
   • **`camera_move`** — ONE move per shot. Safe/default: `{static, push_in, pull_out, pan, tilt,
     parallax}`. **Bold/experimental** `{orbit, dolly_zoom, roll, whip}` are **available, not
     banned** — they can warp the flat art, so pair with `constraints: loose` and **re-roll**.
     Any custom phrase also passes through.
   • **`element_motion`** — where the energy lives; **AI writes it per beat to fit that scene** (not a
     template). Make it RICH (several elements moving) — be bold. A **hero element flying across
     the frame** (paper bird/plane/coins) is a great **occasional** punch on a key beat, **not
     every shot** (a flyer in every frame reads as a formula).
   `motion_style` = amplitude `calm | punchy | max` (the theme sets a default). **`constraints`**
   = `strict` (default: defect guards on — flat-2D, one-way, no-morph; best for clean text-heavy
   explainers) or `loose` (let the model explore 3D/bold moves; re-roll the misses). **Headline
   text is hard-protected only on shots that have a title** (detail shots without a headline are
   free to go wild). The Bailian default is `happyhorse-1.1-i2v`; validate real-person
   and brand use against the selected model's current policy before a full run.
   **Aspect routing** (`styles.resolve_video_aspect`, second approval gate): `clips.py` resolves
   `doc["aspect"]` against the chosen `video_model`'s own supported ratios — exact match wins;
   Omni is 16:9/9:16 only, Kling reference-to-video adds 1:1, Kling image-to-video/video-edit and
   Seedance just follow the input/ratio param. When there's no exact match it picks the nearest
   ratio but **stops and asks you to confirm** (set `"aspect_approx_confirmed": true` once you
   have) rather than silently reframing the film — every clip in one run shares the same resolved
   aspect so the finished film is never mixed.

6. **Assemble.** `python3 scripts/assemble.py out/<project>`
   ffmpeg: normalize + concat all shots, lay the single narration ducked under the music,
   burn captions timed per beat, add the watermark. Output `out/<project>/final.mp4`.

7. **Verify.** You can't read an mp4 directly — extract frames to jpg and look:
   `ffmpeg -ss <t> -i final.mp4 -vf "scale=640:-1,format=yuvj420p" -frames:v 1 f.jpg`

### Cadence — content decides duration

- Default to one narration beat and one generated clip per segment. Keep each segment in the
  model-safe **5–10s** range.
- Estimate before generation from text length; after TTS, treat measured `narration_dur` as
  authoritative and round up to the next 0.5s with a ~0.45s edit tail.
- Video APIs receive the next whole second (for example 6.5s → request 7s); assembly trims the
  returned clip to the exact edit duration, so narration is never cut.

From modelstudioai/skills, Apache-2.0.

More from modelstudioai/skills