gemini-omni-flash-api

Files (5)

• SKILL.md
• upload_file.py · scripts/
• generate_video.py · scripts/video/
• inspect_video.py · scripts/video/
• prep_video.py · scripts/video/

SKILL.md (first lines)

# Gemini Omni Flash Skill

This skill uses the Gemini Omni 1.1 Flash model (`gemini-omni-1.1-flash`) to perform text to video generation, image to video generation (first frame and last frame transitions), video extensions (up to 40s), and video editing.

> [!WARNING]
> **Important Regional Restrictions**: Uploading videos to use for video edits or extensions is **NOT** available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (`total_output_tokens: 0` or no video content), it is likely due to this restriction.

## Core capabilities

1. **Text to video**: Generating videos from a text prompt.
2. **First frame to video**: Generating videos from a starting image (`--first-frame`).
3. **First and last frame transition**: Generating videos interpolating between a starting image and a final image (`--first-frame` and `--last-frame`; note: `--last-frame` must be used with `--first-frame`).
4. **Video extensions**: Extending existing videos by up to 10 seconds per turn, up to a total length of 40 seconds (`--extend` or `--previous-interaction-id`).
5. **Video editing and refinement**: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting.
6. **Image and video referenced generation**: Using style, character, or object references from images or videos to guide video generation.

## Workflow

1. **Analyze request**: Determine the target task (e.g., first-frame-to-video, first-and-last-frame transition, video extension, reference-guided editing) and identify any input media assets.
2. **Run SDK scripts**:

   * Directly run the appropriate utility (`scripts/video/generate_video.py` or `scripts/upload_file.py`).
   * Configure settings like `--aspect-ratio` (e.g. `16:9`, `9:16`), `--resolution` (`360p`, `720p`, `1080p`, `4k`; default: `720p`), and `--duration` (any integer between `3` and `10` seconds, e.g. `3`, `5`, `10`). *Note: `4k` requests take longer to generate.*

3. **Retrieve and process output**: Outputs are saved to the local filesystem (e.g. `media/`). Report back the completed media path to the user.

## Reference Documentation

* **Interactions API**: All operations and state management for the Gemini Omni 1.1 Flash model (`gemini-omni-1.1-flash`) are handled via the [Interactions API](https://ai.google.dev/gemini-api/docs/interactions-overview).
* **Files API**: Input media files (such as reference images and videos) must be uploaded via the [Files API](https://ai.google.dev/gemini-api/docs/files) first before being referenced in generations. The uploaded file URI and MIME type are then included in the `interactions.create` input parts array.
* **[Gemini API Skill Reference](https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-api-dev/SKILL.md)**: Platform-wide guidelines, current model specifications, and SDK usage rules for the Gemini API.

## Dependencies and Prerequisites

* **Python SDK (`google-genai`)**: Requires `google-genai >= 2.19.0` (Python) to support the `interactions` client and full video output resolution configuration (`360p`, `720p`, `1080p`, `4k`). Install or upgrade using:
  ```bash
  pip install -U google-genai
  ```
* **Python Runtime**: Requires **Python >= 3.10** (for compatibility with modern `google-genai` SDK types and methods).
* **ffmpeg & ffprobe**: `prep_video.py`, `inspect_video.py`, and `generate_video.py` (when stripping audio via `--strip-audio`) require `ffmpeg` and `ffprobe` binaries installed and available in your system `PATH`.
* **API Key**: Set the `GEMINI_API_KEY` environment variable:
  ```bash
  export GEMINI_API_KEY="your-api-key"
  ```

## Available scripts

Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API.

1. **[upload_file.py](scripts/upload_file.py)**: Uploads local media (images and videos) to the Files API and polls until `ACTIVE`. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing with `prep_video.py` first to speed up the upload.

   ```bash
   ./scripts/upload_file.py path/to/image.png
   ```

2. **[generate_video.py](scripts/video/generate_video.py)**: Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload.

   * **Text to video**:

     ```bash
     ./scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4
     ```

   * **Output resolution options (`--resolution`)**:

     Gemini Omni 1.1 Flash natively supports four output resolutions across both landscape (`16:9`) and portrait (`9:16`) aspect ratios:
     - `360p`: `640x360` (16:9) or `360x640` (9:16)
     - `720p`: `1280x720` (16:9) or `720x1280` (9:16) — *(default)*
     - `1080p`: `1920x1080` (16:9) or `1080x1920` (9:16)
     - `4k`: `3840x2160` (16:9) or `2160x3840` (9:16)

     ```bash
     # High-definition (1080p)
     ./scripts/video/generate_video.py "A cinematic drone shot over misty mountains at sunrise" --resolution 1080p --output media/mountains_1080p.mp4

     # Ultra-high-definition 4K (Note: 4K requests take longer to generate; pass --timeout if needed)
     ./scripts/video/generate_video.py "A macro shot of a dewdrop on a flower petal in golden sunlight" --resolution 4k --timeout 900 --output media/flower_4k.mp4
     ```

   * **Configurable request timeouts (`--timeout`)**:

     Default HTTP timeout is `600` seconds (10 minutes). For computationally intensive requests — such as extending a 30s video in 4K by 10s (up to the maximum 40s total video length) — generation can take several minutes. Use `--timeout 900` (or `1200`) to provide an extended execution budget.

   * **First frame to video**:

     ```bash
     ./scripts/video/generate_video.py "The waves crash against the shore." --first-frame start.png --output media/waves.mp4
     ```

   * **First and last frame transition**:

     Provide a starting frame and an ending frame to generate a smooth transition between them (note: `--last-frame` **must** be used together with `--first-frame`):

     ```bash
     ./scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --first-frame start.png --last-frame end.png --output media/interpolation.mp4
     ```

   * **Looping video (identical start and end frame)**:

     ```bash
     ./scripts/video/generate_video.py "A crystal orb spinning continuously in place" --first-frame orb.png --last-frame orb.png --output media/loop.mp4
     ```

   * **Image-referenced video generation**:

     ```bash
     ./scripts/video/generate_video.py "A cybernetic warrior in the style of <IMAGE_REF_0>" --image reference.png --output media/warrior.mp4
     ```

   * **Video-referenced video generation**:

     Provide one or more reference videos (`--video-reference` / `-vr`) to guide character, object, or motion style (ideal duration is ~3s, up to 3 reference videos recommended):

     ```bash
     ./scripts/video/generate_video.py "A musician playing cello in the style of <VIDEO_REF_0>" --video-reference ref_dance.mp4 --output media/cello.mp4
     ```

   * **Video extension (extend an existing video)**:

From google-gemini/gemini-skills, Apache-2.0.

More from google-gemini/gemini-skills