cloud-logging-finops

Files (24)

• SKILL.md
• report_template.html · assets/
• executive_summary_authoring.md · references/
• actions.py · scripts/
• aggregations.py · scripts/
• analytics.py · scripts/
• attribution.py · scripts/
• billing.py · scripts/
• build_report.py · scripts/
• collect_logging_data.py · scripts/
• constants.py · scripts/
• enable_log_analytics.py · scripts/
• exclusion_rules.py · scripts/
• inventory.py · scripts/
• messages.py · scripts/
• monitoring.py · scripts/
• output.py · scripts/
• perf.py · scripts/
• pricing.py · scripts/
• provenance.py · scripts/
• requirements.txt · scripts/
• telemetry_integrity.py · scripts/
• verify_claim.py · scripts/
• waste.py · scripts/

SKILL.md (first lines)

# Cloud Logging Finops

**Non-interactive runs.** When no user can answer (e.g. a scheduled or
automated run), ask nothing: audit the project named in the request, or the
active gcloud project; use `--billing-mode=search`; cover all five report topics
(Step 2.5); and never enable an API or upgrade a log bucket. Put anything that
needed a user (a disabled API, a missing role, the upgrade advisory) in the
report instead.

## Usage Attribution

**MUST READ**: Before executing any `gcloud`, REST API, or MCP command, you
MUST read and follow the instructions in
[Usage Attribution](references/attribution.md).

## Step 0: Locate the Cloud Billing Export

The audit covers one Google Cloud project, `{project_id}`: the project the user
names or, if they don't name one, the active gcloud project (`gcloud config get
project`), confirmed with the user.

Discover everything else (log buckets, Observability Analytics, sinks,
exclusions, metrics) yourself; never ask the user about those. The billing export is the
exception: finding it unaided can mean scanning every project on the billing
account, so ask once, before running anything, and don't start a search until
the user has chosen answer 3 (single-select, using the host's question tool if
it has one):

> **Do you know where the Cloud Billing export for `{project_id}` is?**
>
> 1.  **Yes** — here it is *(the BigQuery table ID `project.dataset.table`, or
>     just the project and dataset)*
> 2.  **There's no billing export** for this project
> 3.  **No / not sure** — search for it *(seconds on a small project; on a large
>     billing account it can take a long time)*

Answer | Flags                                                                                         | Cost figures
------ | --------------------------------------------------------------------------------------------- | ------------
1      | `--billing-mode=explicit` plus `--billing-table` (or `--billing-project`/`--billing-dataset`) | invoiced
2      | `--billing-mode=none`                                                                         | list-price estimates
3      | `--billing-mode=search` *(default)*                                                           | invoiced if found, else estimates

Answer 3's search has no time limit: it checks `{project_id}` first (usually
under 10 s, 1–2 min with many tables), then every project on its billing account
(can take many minutes; progress prints every 30 s). Tell the user this before
launching. If the wait is unacceptable, offer to switch to answer 1 (they look up
the table) or answer 2 (skip the export and use list-price estimates). Don't
suggest setting up an export to make the search faster.

If the search found an export, tell the user: *"Next time, pass
`--billing-table=<table>` and discovery drops to about two seconds."*

## Step 1: Environment Setup

Check these prerequisites before Step 2; fix only what is missing:

*   **gcloud:** if `gcloud` is not installed, follow
    https://docs.cloud.google.com/sdk/docs/install-sdk.md.txt.
*   **Project:** run `gcloud config get project`; if it isn't `{project_id}`
    (from Step 0), run `gcloud config set project {project_id}`.
*   **Credentials:** the scripts use Application Default Credentials. Run
    `gcloud auth application-default login`, or set
    `GOOGLE_APPLICATION_CREDENTIALS` to a service account key.
*   **Billing:** the project must have an active billing account attached.
*   **APIs:** don't enable APIs up front; the Logging, Monitoring, BigQuery and
    Observability APIs are on by default in new projects. Only if the collector
    reports an API as disabled (`SERVICE_DISABLED`, "has not been used in
    project"), name it, ask the user, and on approval run `gcloud services
    enable {api} --project={project_id}` (tagged as described below):
    *   `logging`, `monitoring`: required; without them there is nothing to
        audit.
    *   `bigquery`: invoiced costs from the billing export.
    *   `observability`: resolves the project's default log scope, so log
        samples match what Logs Explorer shows.
    *   `cloudasset`, `cloudbilling`: only answer 3's search across the billing
        account.

IAM roles for the identity running the audit. The skill never grants roles: the
scripts run with the user's own credentials and read only what those
credentials can. When a step fails on permissions, its advisory names the step;
tell the user which role below it needs:

Role                                                                 | Needed for
-------------------------------------------------------------------- | ----------
`roles/logging.viewer`, `roles/monitoring.viewer`                    | Every audit: log buckets, sinks, log samples, volume metrics
`roles/bigquery.jobUser`, `roles/bigquery.dataViewer` (on the export dataset) | Invoiced costs from the billing export
`roles/logging.viewAccessor` on log views in other projects        | Reading samples through a default log scope that includes them
`roles/cloudasset.viewer`, `roles/billing.viewer` (billing account)  | `--billing-mode=search` across the billing account
`roles/logging.configWriter`                                         | Only the optional Observability Analytics upgrade (Step 3B)

```bash
[ -f /tmp/finops_venv/bin/python ] || (uv venv /tmp/finops_venv && uv pip install -r scripts/requirements.txt --python /tmp/finops_venv/bin/python)
```

The bundled scripts tag their Google Cloud API requests. For commands you run
yourself, follow [Usage Attribution](references/attribution.md).

## Step 2: Data Collection

Start the collector below as a background process (it takes from seconds to
many minutes), then go to Step 2.5 while it runs:

```bash
/tmp/finops_venv/bin/python scripts/collect_logging_data.py \
  --project-id {project_id} --billing-mode={none|explicit|search}
```

Optional: `--billing-project`, `--billing-dataset`, `--billing-table` (e.g.
`billing-proj.ds.gcp_billing_export_v1_XXXX` or a curated FinOps view).

Outputs: `logging_usage_summary.json`, `monthly_details/`, `service_details/`,
and `[FINOPS ADVISORY]` lines (see Step 3). The collector's output ends with a
`[COLLECTION PROFILE]` table of per-stage timings; if the run took longer than
the times in Step 0, or the user asks why the audit took so long, read that
table before guessing.

## Step 2.5: Ask What the Report Is For

Always ask this while the collector runs (multi-select). If the user skips it,

From google/skills, Apache-2.0.

More from google/skills