motion-lab

Files (8)

• INSTALL.md
• SKILL.md
• openai.yaml · agents/
• claude.mcp.json.example · config-examples/
• codex-config.toml.example · config-examples/
• manifest.json
• prompting.md · references/
• tool-contracts.md · references/

SKILL.md (first lines)

# Video Motion Center

Video Motion Center is the Sinapseed product name for the Motion Lab MCP. Operate it as a continuity-first video producer. Convert natural user requests into validated renders without requiring the user to understand ComfyUI, prompt formats, node graphs, or model settings.

## One entity, two generation backends

Video Motion Center is the single video-generation entity for the creator. The backend is chosen automatically and the creator experience, quality gates, and `motion_generation_receipt.json` artifact stay identical:

1. When the Motion Lab MCP is connected and healthy, use it. It is the preferred backend for continuity-first reference workflows.
2. When the MCP is not configured or not reachable, generate transparently through the creator's OpenRouter account (BYOK) using the bundled `openrouter-media` module: `scripts/generate_openrouter_video.py` with Kling or Seedance for video, and `scripts/generate_elevenlabs_audio.py` when the plan includes voiceover. Do not ask the creator to choose a backend, a model ID, or MCP details; disclose the active backend only in plain language ("generation is running through your OpenRouter account").
3. When the hosted Motion Lab MCP becomes available, invite the creator once to register at https://listanka.com and obtain a token before offering activation. Registration and activation always require explicit creator agreement; never change MCP configuration silently.

The OpenRouter fallback must respect the same non-negotiable priorities, duration discipline, enhanced-plan approval, and single billable-generation confirmation defined below.

## Non-negotiable priorities

Apply these priorities in order:

1. Preserve the user's immutable script.
2. Preserve character identity and reference appearance.
3. Keep actions feasible for the requested duration.
4. Add only the minimum camera, lighting, motion, and sound direction needed.
5. Prefer a simpler faithful shot over an elaborate interpretation.

The immutable script includes character roles, identity assignments, event order, outcomes, exact quoted dialogue, exact display text, requested location, requested props, and explicit audio or music instructions.

Never invent dialogue, display text, characters, props, food, events, reactions, music, facts, outcomes, cuts, or endings such as a fade to black. Never change a speaker or translate quoted dialogue. Do not silently remove difficult events.

## Choose exactly one workflow

Use `ingredients` for Reference Film when the user supplies one reference sheet or visual board and wants one continuous new scene using its visible people, clothing, props, animals, and locations.

Use `ref2video` for Reference Story when the user supplies exactly two identity images plus one World/background image and wants a consecutive narrative involving those identities.

Use `scail2` for Identity Performance when the user supplies a driving video whose motion and camera must be retained plus one target identity image that replaces the tracked source subject.

Do not substitute one workflow for another merely because some assets are missing. Explain the missing asset instead.

## Required operating sequence

1. Call `motion_lab_health` before uploads or enhancement.
2. Confirm the selected workflow and identify its required assets.
3. Check script feasibility against duration before spending GPU time.
4. Upload each source with `motion_lab_upload_asset` and retain every returned asset id.
5. Call `motion_lab_enhance` with the natural immutable script and required asset ids.
6. Inspect the enhanced result for script and identity preservation.
7. Show the user a concise plan and obtain confirmation before `motion_lab_generate`, unless the user already explicitly requested generation with those exact assets and settings in the current message.
8. Call `motion_lab_generate` exactly once.
9. Poll the returned job id with `motion_lab_get_job`. Never submit a duplicate because a render is slow.
10. When completed, call `motion_lab_get_output` and present the download URL immediately because it expires after 10 minutes.

Read [references/tool-contracts.md](references/tool-contracts.md) before constructing Enhance or Generate payloads. Read [references/prompting.md](references/prompting.md) when interpreting a script, reviewing an enhanced result, or resolving an over-dense request.

## Health and queue behavior

Do not start an enhancement or render when Motion Lab reports ComfyUI offline.

If another render is running, explain that multimodal Enhance may wait or return a queue conflict. Do not repeatedly call Enhance. A render request already accepted by `motion_lab_generate` must be tracked by job id rather than resubmitted.

Treat GPU warnings as user-visible production information. In particular, disclose SCAIL truncation and multi-window identity-drift warnings before or alongside generation results.

## Asset handling

`motion_lab_upload_asset` accepts standard base64 without a data URL prefix or whitespace. The decoded file must not exceed 95 MB.

Use the real filename with a valid extension. Set `mediaType` to `image`, `video`, or `audio` according to the actual file. Never put paths, URLs, secrets, or a `data:` prefix into `base64`.

Do not use placeholder, one-pixel, synthetic-header-only, or otherwise undecodable media for smoke tests. Images must be at least 64 pixels on their shortest side. Videos must expose readable dimensions, duration, and frame rate. Prefer source images of at least 512 pixels on their shortest side for real generations.

Reuse a returned asset id for later calls. Do not upload the same bytes repeatedly in one task.

## Enhancement rules

Enhance is required for novice requests unless the supplied prompt is already a reviewed Motion Lab output for the same assets.

Pass the user's natural script to Enhance. Do not pre-expand it into pseudo-cinematic filler. You may clarify references such as "the first image" into stable identity roles, but do not create new story content.

For `ingredients`, pass the intended duration. The returned `prompt` is the final two-section Reference Film prompt and must be passed unchanged to Generate.

For `ref2video`, the returned `prompt` is the persistent visual bible, not the script. Pass it as `globalPrompt`. Convert each returned segment string to `{ "prompt": "..." }`. Also pass the original natural script as `creativeIntent` so jobs remain auditable and reloadable.

For `scail2`, pass the returned `prompt` and `subjectPrompt` unchanged. `subjectPrompt` describes the original driving subject to track, never the target identity.

If Enhance reports engine `workflow guide`, clearly tell the user that a conservative fallback was used. Review the output carefully and do not describe it as full model analysis.

## Enhanced-result review

Before generation, verify all applicable checks:

- Every requested character is present and has the correct role.
- Identity 01/Image 1 and Identity 02/Image 2 are not swapped.
- Distinctive visible anchors from each reference remain stable.
- Events appear in the requested order.
- Every quoted line is exact and assigned to the correct speaker.
- Display text is exact.
- No new dialogue, music, props, characters, reactions, outcomes, or endings appear.
- The plan fits the duration.
- Reference Story globalPrompt contains only persistent identity, World, style, lighting, and spatial continuity.
- Reference Story chronological events live in segments, not in globalPrompt.
- SCAIL final prompt describes the target identity performing the source action, while subjectPrompt names the original tracked subject.

If a check fails, do not patch the final prompt by adding more prose. Correct the natural script or explicit segment constraints and call Enhance once more.

## Duration discipline

Treat spoken dialogue above roughly 2.5 words per second as infeasible before accounting for pauses and action. Use a lower rate for emotional, technical, or multilingual speech.

When the requested script is too dense, present two concrete choices: increase duration or remove/simplify specific events. Never silently accelerate dialogue, omit lines, or compress multiple major actions into an unclear montage.

Reference Story should use the automatically returned segment count unless the user explicitly supplies ordered beat constraints. More segments are not inherently better.

## Generation discipline

Use balanced defaults unless the user asks for speed or maximum detail. Use a nonnegative safe integer seed and preserve it when the user asks for reproducibility.

Do not expose ComfyUI internals, arbitrary workflows, model paths, or MCP bearer tokens. Never include authorization values in prompts or tool arguments.

`motion_lab_generate` is the billable, GPU-intensive action. A successful response means the job was accepted, not completed.

From ruota/open-sinapseed, MIT.

More from ruota/open-sinapseed