byted-seedance-video-generate

Files (3)

• LICENSE.txt
• SKILL.md
• video_generate.py · scripts/

SKILL.md (first lines)

# Video Generate Skill

This skill generates videos using Doubao Seedance 1.0/1.5 models.

## Trigger Conditions

1. User wants to generate videos from text descriptions
2. User wants to create videos based on images (first/last frame)
3. User wants to create videos with reference materials (images, videos, audio)
4. User asks for video generation capabilities

## Usage

### Environment Variables

Before using this skill, ensure the following environment variables are set:

- `ARK_API_KEY` or `MODEL_VIDEO_API_KEY` or `MODEL_AGENT_API_KEY`: API key for the video generation service
- `MODEL_VIDEO_API_BASE`: API base URL (optional, has default)
- `MODEL_VIDEO_NAME`: Model name (optional, has default)

### Function Signature

```python
async def video_generate(
    params: list,
    batch_size: int = 10,
    max_wait_seconds: int = 1200,
    model_name: str = None,
) -> Dict:
```

### Parameters

#### params (list[dict])

A list of video generation requests. Each item is a dict with the following fields:

**Required per item:**

- `video_name` (str): Name/identifier of the output video file
- `prompt` (str): Text describing the video to generate. Supports Chinese and English.

**Optional per item - Input Materials:**

- `first_frame` (str): URL for the first frame image
- `last_frame` (str): URL for the last frame image
- `reference_images` (list[str]): 1-4 reference image URLs for style/content guidance
- `reference_videos` (list[str]): 0-3 reference video URLs (mp4/mov, 2-15s each, total ≤15s)
- `reference_audios` (list[str]): 0-3 reference audio URLs (mp3/wav, 2-15s each, total ≤15s)

**Optional per item - Video Output Parameters:**

- `ratio` (str): Aspect ratio. Options: "16:9" (default), "9:16", "4:3", "3:4", "1:1", "2:1", "21:9", "adaptive"
- `duration` (int): Video length in seconds. Range: 2-12s depending on model
- `resolution` (str): Video resolution. Options: "480p", "720p", "1080p"
- `frames` (int): Total frame count. Must be in [29, 289] and follow format 25 + 4n
- `camera_fixed` (bool): Lock camera movement. Default: false
- `seed` (int): Random seed for reproducibility. Range: [-1, 2^32-1]
- `watermark` (bool): Whether to add watermark. Default: false
- `generate_audio` (bool): Whether to generate audio. Only Seedance 1.5 supports this
- `tools` (list[dict]): Tool configuration, e.g., `[{"type": "web_search"}]`

### Input Modes

1. **Text-to-Video**: Only provide prompt, no images/videos
2. **First Frame Guidance**: Provide first_frame for starting image
3. **First + Last Frame Guidance**: Provide both for transition video
4. **Reference Images**: Provide reference_images for style/content guidance
5. **Multimodal Reference**: Combine reference_images, reference_videos, reference_audios

### Return Value

## Script Return Info

The video_generate.py script will return these info:

```python
{
    "status": "success" | "partial_success" | "error",
    "success_list": [{"video_name": "video_url"}],
    "error_list": ["video_name"],
    "error_details": [{"video_name": "...", "error": {...}}],
    "pending_list": [{"video_name": "...", "task_id": "cgt-xxx", ...}]
}
```

Based on the script return info, the final response returned to the user consists of a description of the video generation task and the video URL(s). You may download the video from the URL, but the video URL should still be provided to the user for viewing and downloading.

Note: the URL is the 'url' in the success_list of script return info.
The URL must return in two ways:

## Final Return Info

### For OpenClaw
1. **First, save/download the generated video to an allowed directory**:
   - Recommended: `~/.openclaw/workspace/`

2. **Use the `message tool` to send the video** with these parameters:
   ```json
   {
     "action": "send",
     "message": "Optional text description",
     "media": "/root/.openclaw/workspace/generated-video.mp4"
   }
   ```
   - The tool will automatically: read the local file, upload to the channel server as a web URL, and send as a video message

3. **Verify success**: Check that the tool returns `{"ok": true}` to confirm the video was sent successfully

4. **Normal Text** not message tool: After generation, show list of videos with Markdown format, for example：
   ```
   <video src="https://example.com/video1.mp4" width="640" controls>video-1</video>
   ```
   **Very important**:
   The video URL must be an **original online resource link** starting with `https://`, **not** a local video address you have downloaded.

### For Normal
You should return three types of information:

From bytedance/agentkit-samples, Apache-2.0.

More from bytedance/agentkit-samples