byted-mediakit-videoedit

Files (15)

• LICENSE
• SKILL.md
• danmaku_format.md · references/
• cut_and_merge.py · scripts/
• parse_media_info.py · scripts/
• video_effects.py · scripts/
• package.json · template/
• render.mjs · template/
• chapter-title.tsx · template/src/
• danmaku-burst.tsx · template/src/
• fancy-text.tsx · template/src/
• index.tsx · template/src/
• lower-third.tsx · template/src/
• quote-callout.tsx · template/src/
• remotion-root.tsx · template/src/

SKILL.md (first lines)

# AI Video Intelligent Editing

## Overview

This Skill helps users understand video context by analyzing danmaku and subtitle content, automatically extracts and splices video clips based on editing requests, and uses FFmpeg to complete transition effects and final synthesis.

## Input Specifications

- **Video files** (required, supports multiple): Local video file paths, supports formats like `.mp4`, `.flv`, `.mkv`, etc.
- **Danmaku files** (optional, one per video): XML format danmaku files (supports Bilibili format), corresponding to video files in order one-to-one
- **Subtitle files** (optional, one per video): `.srt` / `.ass` / `.json` format subtitle files, corresponding to video files in order one-to-one; leave empty for videos without subtitles

> **Note:** Subtitles and danmaku are the only basis for understanding video content. If neither is provided for a video, its content cannot be understood, and only explicit time segment instructions from the user can be executed.

## Workflow

### Step 0: Dependency Verification

Before performing any operations, verify that the runtime environment meets the requirements.

**Verification Commands:**

```bash
python --version
ffmpeg -version 2>&1 | head -1
ffprobe -version 2>&1 | head -1
node --version
```

**Acceptance Criteria and Fixing Guidelines:**

| Dependency | Minimum Requirement  | Verification Method | Installation Command When Not Met                                                                                         |
| ---------- | -------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Python     | 3.9+                 | `python --version`  | See instructions below                                                                                                    |
| ffmpeg     | Any version          | `ffmpeg -version`   | macOS: `brew install ffmpeg` · Linux: `sudo apt install ffmpeg` · Windows: [ffmpeg.org](https://ffmpeg.org/download.html) |
| ffprobe    | Included with ffmpeg | `ffprobe -version`  | Installed with ffmpeg, no separate operation needed                                                                       |
| Node.js    | 18+                  | `node --version`    | macOS: `brew install node` · or [nodejs.org](https://nodejs.org/)                                                         |

**Python Installation Instructions (if version is not met):**

- macOS: `brew install python@3.11`
- Linux: `sudo apt install python3.11`
- Or via [pyenv](https://github.com/pyenv/pyenv): `pyenv install 3.11 && pyenv global 3.11`

**Text Effect Rendering Dependency (Remotion) Installation:**

Check if `byted-mediakit-videoedit/template/node_modules` exists:

```bash
ls byted-mediakit-videoedit/template/node_modules/@remotion/renderer 2>/dev/null && echo "已安装" || echo "需要安装"
```

If not installed, execute:

```bash
cd byted-mediakit-videoedit/template && npm install
```

Installation takes about 1-2 minutes, and `node_modules` will appear in the directory after completion. This step is skipped in subsequent conversations if `node_modules` already exists.

**Processing Rules:**

- All dependencies met → proceed to next step
- Missing or insufficient version → explain to user what is missing, provide installation commands for the corresponding platform, **wait for user to complete installation and re-confirm before continuing**

### Step 1: Check Understandability of Each Video

Before any analysis, confirm individually whether each video has subtitles or danmaku:

| Video       | Has Subtitles | Has Danmaku | Understandability                                                                                                                                        |
| ----------- | ------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| video_A.mp4 | ✅            | —           | **Intelligent Analysis**: Can understand video based on content semantics, supports content requests like "find highlights/find appearance introduction" |
| video_B.mp4 | —             | ✅          | **Intelligent Analysis** (downgraded): Infer content through danmaku, accuracy lower than subtitles                                                      |
| video_C.mp4 | —             | —           | **Only Explicit Commands**: Cannot understand content, can only cut according to precise time segments provided by user                                  |

**Processing Rules for "Only Explicit Commands" Videos:**

1. **User provides explicit time segments** (e.g., "cut video_C from 1:00–2:30"): Use directly, no analysis needed
2. **User provides content requests** (e.g., "find appearance introduction from video_C"): **Immediately inform** user that the video lacks subtitles/danmaku, cannot perform content analysis, please provide:
   - Specific time segments, or
   - Subtitle/danmaku files
3. **Mixed scenarios** (some videos understandable, some not): Normal analysis for understandable videos, separately explain limitations for non-understandable videos, and ask user how to handle

### Step 3: Parse Danmaku and Subtitles

Run the parsing script to convert danmaku and subtitles into timeline-based text summaries for video content analysis.

**Single Video:**

```bash
python byted-mediakit-videoedit/scripts/parse_media_info.py \
  --video ep1.mp4 \
  --danmaku ep1.xml \
  --subtitle ep1.srt \
  --output /tmp/media_timeline.json
```

**Multiple Videos (repeat a set of** **`--video`/`--danmaku`/`--subtitle`** **for each video):**

```bash
python byted-mediakit-videoedit/scripts/parse_media_info.py \
  --video ep1.mp4 --danmaku ep1.xml --subtitle ep1.srt \
  --video ep2.mp4 --danmaku ep2.xml \
  --output /tmp/media_timeline.json
```

**Important: Only run this script for videos with subtitles or danmaku.** Videos without both subtitles and danmaku are not passed to the script, and their segment times are directly provided by the user.

Optional parameters:

- `--interval 5`: Danmaku summary time interval size (seconds), default 5
- `--top-danmaku 10`: Number of most frequent danmaku per interval, default 10
- `--include-danmaku`: Force include danmaku summary in output (even with subtitles); **only add this parameter when user explicitly requests to reference danmaku**
- `--video` can be omitted (only for single video), script uses danmaku filename as placeholder when omitted
- `--subtitle` can be omitted (when the video has no subtitles)

**Script automatically determines what to write based on the following rules (no additional Claude judgment needed):**

| Input Situation                       | Content Written to Timeline     | Output Mode            |

From bytedance/agentkit-samples, Apache-2.0.

More from bytedance/agentkit-samples