clickstack-otel-collector

Files (2)

• SKILL.md
• metadata.json

SKILL.md (first lines)

# Set up an OpenTelemetry collector for Managed ClickStack

This skill wires an OpenTelemetry collector into a Managed ClickStack service running on
ClickHouse Cloud, sends rich synthetic telemetry through it, and confirms the data is actually
visible in ClickStack. It uses [`clickhousectl`](https://clickhouse.com/docs/interfaces/cli)
for all cloud and SQL operations.

**Scope.** This skill supports two paths, chosen in Step 0:

1. **Deploy a new collector** locally. You can do this **two ways: individual `docker` commands,
   or a `docker compose` file** (recommended, fewer commands and one file to start/stop). Make the
   user aware of both up front and let them pick in Step 0; do not assume plain `docker`. Either
   way runs the ClickStack distribution of the collector, preconfigured for Managed ClickStack.
2. **Configure your own existing collector** by adding the ClickHouse exporter configuration.
   We give you the exact config to drop in; you reload your collector. Use this if you already
   run a collector in a **gateway** role.

A full Kubernetes deployment (Helm, secrets in K8s Secrets) is out of scope here; the config
we generate in path 2 can be applied to a collector running anywhere.

The end state is:

- A dedicated `hyperdx_ingest` SQL user on the target service, with exactly the grants the
  collector needs (it creates the `otel.*` schema on first write).
- A collector forwarding logs, traces, and metrics into the `otel` database on the service,
  either the new local ClickStack collector or your existing one.
- Rich synthetic telemetry across several services, severities, span statuses, and metric
  types, so ClickStack's Search, Service Map, and dashboards have something real to show.
- The service confirmed **awake**, and the user walked through the ClickStack onboarding in the
  Cloud console so they can actually *see* their data.

Secrets (the OTLP auth token and the SQL password) are generated locally, written **once** to a
`0600` env file, and passed to Docker via `--env-file`. They are never pasted into the chat,
never passed with `docker run -e`, and never echoed back after creation.

Follow these steps in order. Each step depends on state established by the previous one.

---

## Step 0: Choose your path

Ask the user two short questions before doing anything else, because they determine which later
steps run.

**Question 1: Do you already have an OpenTelemetry collector running in a gateway role?**

- **No, set one up for me.** -> the **new-collector** path. Continue to Question 2.
- **Yes, I have one.** -> the **existing-collector** path. Skip Question 2 (it does not apply),
  and in Step 6 you will configure their collector rather than deploy a new one.

**Question 2 (new-collector path only): Run the collector with individual Docker commands, or a
Docker Compose file?**

- **Docker Compose (recommended).** Fewer commands, one file to start and stop, easiest to
  re-run. Best if `docker compose` is available.
- **Individual Docker commands.** Use if Compose is not installed or you prefer explicit
  commands.

Record the answers as `COLLECTOR_PATH` (`new` or `existing`) and, for the new path,
`DEPLOY_MODE` (`compose` or `run`). Refer back to them in Step 6 and Step 7.

---

## Step 1: Batch the permissions up front

Coding agents prompt for approval the first time they see each shell command. To avoid
interrupting the user every few steps, ask them once, up front, to allowlist the command
prefixes below (the "always allow for this project / session" option in their agent). There are
no destructive operations and nothing targets anything outside this project or their ClickHouse
Cloud service.

| Command prefix | Used for | Needed when |
| --- | --- | --- |
| `openssl rand …` | generate the OTLP token and SQL password | always |
| `clickhousectl cloud …` | auth, resolve the service, run SQL via the Query API | always |
| `jq …` | parse JSON from `clickhousectl` | always |
| `docker …` / `docker compose …` | run/inspect the collector and the telemetry generator | new-collector path, and the optional telemetry check |
| `curl …` | local health check against `localhost:13133` (and installing `clickhousectl` if missing) | new-collector path |

Tell the user, in your own words: *"If your agent supports it, choose 'always allow' for each
of these the first time it asks. The whole run is read-only against your machine except for the
collector container, and write operations against ClickHouse are limited to creating the ingest
user and the `otel` schema."*

If the user is on the existing-collector path and does not want to run the optional telemetry
check, you can drop `docker` and `curl` from the list.

**Two approvals are semantic, not prefix-based, so allowlisting won't pre-clear them.** Warn the
user to expect these and approve them explicitly when they appear:

- The `clickhousectl` install in Step 3 uses `curl … | sh`, which many agent sandboxes flag as
  "downloading and running untrusted code" regardless of any `curl` allowlist rule.
- The `CREATE USER` / `GRANT` in Step 5 may be flagged as "modifying shared production
  infrastructure," again independent of the `clickhousectl` prefix rule.

Neither is solved by the table above; they are one-time, intentional, and safe to approve.

Then continue.

---

## Step 2: Confirm the target service and lay down the secrets file

The user's prompt contains a service identifier, either a service ID (UUID) or a service name.
Treat that value as `SERVICE_REF`.

Create a working directory and a **`0600` env file** that will hold all configuration and
secrets for this run. The key names match exactly what the collector image reads, so this same
file is passed straight to `docker run --env-file` (or referenced by Compose) in Step 6. Write
it under a tight `umask` so the secret is never briefly world-readable:

```bash
WORKDIR="${WORKDIR:-$HOME/clickstack-otel-collector}"
mkdir -p "$WORKDIR" && chmod 700 "$WORKDIR"
ENV_FILE="$WORKDIR/collector.env"

# Generate secrets WITHOUT printing them; write straight into a private file.
( umask 177
  {

From ClickHouse/agent-skills, Apache-2.0.

More from ClickHouse/agent-skills