agent-platform-inference

Files (9)

• SKILL.md
• gemini_genai_sdk.py · scripts/
• gemini_openai_sdk.py · scripts/
• gemini_vertexai_sdk.py · scripts/
• openmaas_genai_sdk.py · scripts/
• openmaas_openai_sdk.py · scripts/
• openmaas_vertexai_sdk.py · scripts/
• requirements.txt · scripts/
• verify_all.sh · scripts/

SKILL.md (first lines)

# Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google
Cloud Agent Platform to use Generative AI models. It covers:

*   **First-Party publisher models** (Gemini) — section 2.
*   **Third-Party publisher models** (OpenMaaS: Llama, DeepSeek, Qwen, etc.)
    — section 3.
*   **Custom endpoints** (any model on a numeric `projects/.../endpoints/<id>`
    resource — tuned Gemini models, OSS LLMs self-deployed from Model Garden
    via the `agent-platform-deploy` skill, and legacy custom models) —
    section 4.

## Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere
to the following safety tiers based on the action requested. (The skill is
read-only; other safety tiers are omitted):

1.  **Tier R: Read-only / Inference (`client.models.generate_content`,
    `client.chat.completions.create`, `client.completions.create`,
    `client.embeddings.create`)**
    *   Requires **interactive confirmation** with 'Yes'/ 'No' options before
        executing model inference on behalf of the user, to prevent unexpected
        cost or quota consumption.
    *   **Required Fields in Confirmation Card**: The confirmation prompt must
        clearly explain the proposed inference execution and explicitly list all
        of the following parameters:
        *   **Project ID**: The Google Cloud project ID or number (e.g.
            `123456789012`, `my-project`).
        *   **Region / Location**: The target region (e.g. `us-central1`,
            `global`).
        *   **Model ID**: The exact model ID (e.g. `gemini-2.5-flash`,
            `deepseek-ai/deepseek-v3.2-maas`).
        *   **SDK**: The SDK choice (e.g. `Google GenAI SDK (google-genai)`,
            `OpenAI SDK`).
        *   **Input Prompt** (or **Input Image** / **Input Media**): The prompt
            text or media URI.
        *   Any additional generation parameters (e.g. `max_output_tokens`,
            `response_schema`) if specified.
        Natural-language paraphrases without explicitly listing these
        parameters are NOT sufficient.
    *   **Same-turn restriction**: Do not execute the inference scripts or
        commands in the same turn as presenting the confirmation prompt. Stop
        and wait for the user's reply; only execute after explicit 'Yes' /
        approval.
    *   **Gold Standard Example**:
        > I will perform model inference with the following parameters. Please
        > confirm this information before I proceed:
        > * **Project ID**: `my-project`
        > * **Region**: `us-central1`
        > * **Model ID**: `gemini-2.5-pro`
        > * **SDK**: Google GenAI SDK (`google-genai`)
        > * **Input Prompt**: "Summarize the plot of Hamlet in 3 sentences"
        >
        > Do you confirm? [Yes/No]
    *   **Post-Execution Response Grounding (CRITICAL)**:
        After receiving explicit user approval and executing the inference call
        via the SDK, the response returning the generated text **MUST
        explicitly confirm the execution parameters** alongside the model's
        output. Never return a bare model response alone. Always include:
        *   **Model ID**: The exact model ID used (e.g. `gemini-2.5-pro`
            or `<MODEL_ID>`).
        *   **SDK**: The SDK used (e.g. `Google GenAI SDK (google-genai)`
            or `OpenAI SDK`).
        *   **Project ID**: The Google Cloud project ID/number used.
        *   **Region**: The region or endpoint location used (e.g. `global`
            or `us-central1`).
        *   **Generated Output**: The model's complete generated answer.

## Phase 0: Environment Setup

**CRITICAL**: Before running any of the Python sample scripts in the `scripts/`
directory (e.g., `scripts/openmaas_openai_sdk.py`), you MUST ensure the
environment is correctly initialized by following these steps:

1.  **Google Cloud Authentication**: Authenticate with your Google Cloud
    credentials and configure active Application Default Credentials (ADC) for
    Agent Platform access:

    ```bash
    gcloud auth login
    gcloud auth application-default login
    ```

2.  **Enable API** (if not already enabled):

    ```bash
    gcloud services enable aiplatform.googleapis.com
    ```

3.  **Python Dependencies**: The scripts import `vertexai` (from
    `google-cloud-aiplatform`), `google-genai`, and `openai`. Do **not** create
    a virtual environment — it starts empty and hides packages the environment
    already provides, forcing a redundant install. Probe, and install only what
    is missing:

    ```bash
    python3 -c "import vertexai, google.genai, openai" \
      || pip install -r scripts/requirements.txt
    ```

    `scripts/requirements.txt` is a fallback for an environment that does not
    already provide these SDKs; do not install it on top of a working
    environment.

4.  **Verify Setup (Optional)**: Run all sample scripts at once to verify the
    environment is working end-to-end:

    ```bash
    ./scripts/verify_all.sh
    ```

5.  **Execution**: Run the scripts with a plain `python3 scripts/...`. There is
    no environment to activate first.



> [!IMPORTANT] **CRITICAL: Model IDs & Availability** * **Gemini Models**: See

From google/skills, Apache-2.0.

More from google/skills