authoring-error-tracking-alerts

Files (3)

• SKILL.md
• block-templates.md · references/
• event-triggers.md · references/

SKILL.md (first lines)

# Authoring error tracking alerts

Authoring an error tracking alert is a _routing_ problem, not a measurement problem. The trigger events
already exist and fire on real conditions in the ingestion pipeline — your job is to pick the right
trigger for the user's intent, dedupe against what's already configured, and wire a destination they can
actually act on.

## When to use this skill

- The user asks to set up alerts / notifications for errors or exceptions in their project.
- The user wants a starter set of alerts after enabling error tracking.
- The user pastes an issue link and asks "notify me when this happens again" — usually `_reopened` with a
  per-issue property filter.

## When _not_ to use this skill

- Tuning the spike detector itself (multiplier, window, threshold). That lives behind the spike detection
  config endpoint and is not exposed via MCP today.
- Investigating an active incident — query the issue / its events directly via
  `posthog:query-error-tracking-issue` and `posthog:query-error-tracking-issue-events` instead of
  authoring more alerts mid-fire.
- Configuring volume-threshold alerts (count of `$exception` events over a window). That's a logs-style
  alert and is not in scope here — error tracking alerts ride the lifecycle events instead.

## Tools

| Tool                                           | Job                                                              | Where it fits                |
| ---------------------------------------------- | ---------------------------------------------------------------- | ---------------------------- |
| `posthog:error-tracking-alerts-list`           | List existing alerts (paginated); dedupe before creating.        | Step 2 — dedupe.             |
| `posthog:integrations-list`                    | Find the user's Slack workspace id (filter by `kind=slack`).     | Step 3 — pick channel.       |
| `posthog:integrations-channels-retrieve`       | List Slack channels for a workspace.                             | Step 3 — pick channel.       |
| `posthog:error-tracking-alerts-create`         | Create the alert (HogFunction with `type=internal_destination`). | Step 4 — ship.               |
| `posthog:error-tracking-alerts-partial-update` | Toggle, rename, or modify an existing alert.                     | When tuning, not authoring.  |
| `posthog:error-tracking-alerts-delete`         | Soft-delete an alert.                                            | When the user says "remove". |

## Trigger events — pick exactly one per alert

There are three lifecycle events. Each has a different "noise vs urgency" trade-off — picking the wrong
one is the most common cause of alert fatigue here.

| Event                            | Fires when                                            | Use when                                                                                                                            |
| -------------------------------- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `$error_tracking_issue_created`  | A brand-new issue first appears.                      | Small projects, or projects where every new error type is genuinely worth a look. Floods large/noisy projects.                      |
| `$error_tracking_issue_reopened` | A previously resolved issue starts emitting again.    | Catch regressions on issues someone already triaged. The safest "I want to know if this comes back" trigger.                        |
| `$error_tracking_issue_spiking`  | The spike detector flags abnormal volume on an issue. | Production projects with high baseline volume. Threshold/multiplier is shared across the project — check spike config before using. |

If the user is vague ("alert me on errors"), default to `_spiking`. It's the most signal-dense trigger
and the least likely to cause alert fatigue. Confirm explicitly before proceeding.

## Workflow

### 1. Confirm intent

You need three things from the user before creating anything:

- **Which trigger event.** If unspecified, recommend `_spiking` and ask for confirmation. Do not silently
  pick one.
- **Which channel.** Slack channel name, webhook URL, Linear team, etc. Never hardcode a production
  channel. If the user says "the dev channel", ask for the exact channel id or name.
- **Which scope.** All issues (most common), or scoped to a specific issue / exception type / assignee.

### 2. Dedupe against existing alerts

Scan **every** destination before you decide. The endpoint is paginated and defaults to 100 rows per
page, so a single default call misses older alerts on a project with hundreds of destinations.

Call `posthog:error-tracking-alerts-list` with `type: ["internal_destination"]` and `limit: 1000`. If the
response still carries a non-null `next`, keep paging with `offset` until `next` is null. Then filter the
collected rows client-side by `filters.events[].id` and by any per-issue scope in `filters.properties`.

The list response carries no `inputs`, so it does not tell you which channel or URL a row delivers to.
Read the destination off `name` and `description` instead — the naming convention below puts the channel
in the name. When a row matches on event and scope but its name does not say where it delivers, do not
guess: show the user the matching names and ask whether one of them is the alert they mean.

- If an alert exists for the **same event** delivering to the **same channel** with the **same scope**,
  stop. Tell the user it already exists and ask whether they want to change anything (in which case use
  `error-tracking-alerts-partial-update`) or skip.
- Multiple alerts on the same event for the same channel produce duplicate Slack messages — the user
  almost never wants this.
- Multiple alerts on the same event for **different** channels (e.g. one for `#oncall`, one for the
  oncall webhook) is fine and sometimes intentional. Confirm.
- Multiple alerts on the same event and channel but **different scopes** (e.g. two `_reopened` alerts
  each pinned to a different `$exception_issue_id`) are not duplicates. Confirm before adding. Note that
  a scoped alert and an all-issues alert to the same channel both fire on the scoped issue.

PostHog's "alerts configured" recommendation only inspects `filters.events` — adding per-issue
`filters.properties` does not affect the status the alerts banner reports.

### 3. Pick the integration

For Slack:

1. `posthog:integrations-list` with `kind=slack` → pick the integration `id` (an integer).
2. `posthog:integrations-channels-retrieve` with that id → pick the channel id (e.g. `C0123ABC`). Channel
   names like `"#oncall"` are accepted but channel ids are preferred — they survive renames.

For webhook: the user supplies a single `https://` URL. Refuse `http://` URLs.

For Linear / GitHub / GitLab: confirm the integration is connected via `posthog:integrations-list` first,
then ask the user which project / repository / team to file issues into.

### 4. Create the alert

Call `posthog:error-tracking-alerts-create` with:

```json
{
  "type": "internal_destination",
  "template_id": "template-slack",
  "name": "<short, channel-attributed name>",
  "enabled": true,
  "filters": {
    "events": [{ "id": "$error_tracking_issue_created", "type": "events" }]
  },
  "inputs": {
    "slack_workspace": { "value": <slack_integration_id_int> },
    "channel": { "value": "<channel_id>" },
    "text": { "value": "..." },

From PostHog/skills, MIT.