gemini-live-api

Files (5)

• SKILL.md
• attribution.md · references/
• client_server_messages.md · references/
• client_server_messages.proto · references/
• session_manager.md · references/

SKILL.md (first lines)

# LiveAPI Service Skill

This skill provides instructions for generating a **LiveAPI client service
class** that connects to the Gemini Enterprise Live API over WebSockets. The
generated client handles bidirectional streaming, bearer-token authentication
via Application Default Credentials (ADC), transparent session resumption, and
`ClientMessage` / `ServerMessage` proto exchange.

The skill also produces a demo frontend + backend service so the user can
interactively validate the generated client (text, audio, video, transcription,
and interrupt handling).

## Attribution

**MUST READ**: Before executing any `gcloud`, REST API, or MCP command, you
MUST read [Usage Attribution](references/attribution.md). For commands executed
directly in the terminal (never in generated scripts, Terraform, or user code),
prefix `gcloud` with
`CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:gemini-live-api)"`
and append `agent-skills/1.0 (skill:gemini-live-api)` to `User-Agent` headers.

## Prerequisites

Before running the generation flow, ensure the following are available on the
host:

-   A Google Cloud project with the Vertex AI / Gemini Enterprise Agent Platform
    APIs enabled.
-   Application Default Credentials configured on the host running the generated
    client:

    ```bash
    gcloud auth application-default login
    ```

-   A destination output folder supplied by the user (e.g. `/tmp/liveapi_out`)
    where the generated code, environment, and demo will be written. **Never**
    mutate the host's system Python environment.

-   The user's chosen implementation language (Python is the default and
    reference language for this skill).

## Reference Files

Provided files in `references/` (do **not** treat these as standalone skills —
they are loaded on demand):

-   `client_server_messages.md`: Public reference for the `ClientMessage` /
    `ServerMessage` schemas used by the Live API.
-   `client_server_messages.proto`: The proto definition generated from
    `client_server_messages.md`.
-   `session_manager.md`: Describes how to correctly handle sessions, buffering,
    and resumption on disconnection.

## Steps

### Step 1: Copy the reference files

Copy `client_server_messages.md`, `client_server_messages.proto`, and
`session_manager.md` from this skill's `references/` folder into the user's
destination output folder. These files become the source of truth for the
generated client.

### Step 2: Reconcile with the public documentation

Examine the public documents linked from `client_server_messages.md`. If there
are any discrepancies between the public documents and the copied
`client_server_messages.md` / `client_server_messages.proto`, update the copies
in the destination folder so the generated client compiles and runs against the
current server contract.

### Step 3: Implement the client class

Implement a class in the user's chosen language that:

-   Imports the local `client_server_messages.proto` types (`ClientMessage`,
    `ServerMessage`).
-   Opens a WebSocket connection to the Live API endpoint.
-   Exposes async methods so the user can send and receive data to/from the
    model.

For languages that require an isolated runtime (e.g. Python), create an isolated
environment (e.g. `venv`) **inside the destination folder** and generate a bash
script (e.g. `setup.sh`) that recreates the environment and installs
dependencies. **Never** install into the system interpreter or the user's global
site-packages, and never instruct the user to run `sudo pip install`.

#### Initialization parameters

The user provides the following at construction time:

-   `project_id`
-   `location`
-   `model_id`
-   `config`: a `ClientMessage` with the `setup` field populated.

#### Authentication

Obtain a bearer token via Application Default Credentials, attach it to the
WebSocket connect request as `Authorization: Bearer <token>`, refresh the token
before or upon expiry, and reuse the refreshed token on every reconnection
(including `go_away` and unexpected disconnects). **Do not** hard-code a
long-lived API key as the only auth mechanism.

#### Public async API

The class MUST expose the following async methods, gated on receipt of a
`setup_complete` `ServerMessage` before sending:

-   `send_realtime_data(data)`: send realtime input. `data` is a `ClientMessage`
    carrying a `realtime_input` field.
-   `send_client_content(data)`: send non-realtime, turn-based content that
    contributes to history. `data` is a `ClientMessage` carrying a
    `client_content` field.
-   `receive()`: yield `ServerMessage` instances parsed from the WebSocket
    stream.

Do not expose synchronous blocking variants as the primary API surface.

From google/skills, Apache-2.0.

More from google/skills