elasticsearch-anomaly-detection-explainer

Files (2)

• SKILL.md
• explainer-reference.md · references/

SKILL.md (first lines)

# Anomaly Detection Score Explainer

Explain anomaly scores, model behavior, and why results look the way they do. Use the **ML REST API** for job config and
the **standard `_search` API** against `.ml-anomalies-*` for results — no ES|QL, fully compatible with Elastic
Serverless. For job lifecycle (create, start, stop), use the `elasticsearch-anomaly-detection` skill.

<!-- begin-partial: preamble -->

## Environment Configuration

This skill executes Elasticsearch operations through the `elastic` CLI. If the
[`elastic` CLI](https://github.com/elastic/cli#configuration) is not installed, tell the user what it is needed for. Do
not guess credentials, call the HTTP API directly, or attempt other workarounds.

This skill references operations in HTTP-shorthand form (e.g., `GET /`, `GET /_cat/indices`, `GET /{index}/_mapping`,
`GET /{index}/_settings/index.mode`, `POST /_query`). The [Operations](#operations) table at the end of this document
maps each shorthand to the equivalent `elastic` CLI command — always use the CLI rather than calling the HTTP API
directly.

<!-- end-partial: preamble -->

> **Prerequisite:** ML anomaly detection requires a Platinum-equivalent license on self-managed clusters. Serverless
> projects include ML. The caller needs `monitor_ml` to read job config and anomaly results.
>
> **Serverless note:** The `_ml/.../results/*` REST endpoints return HTTP 410 in Elastic Serverless. Always use
> `POST /.ml-anomalies-*/_search` for result queries instead — fully supported everywhere this skill runs.

## Process

1. **Decide whether to fetch data or interpret what the user supplied.** If the user embeds an anomaly record (or job
   config) in the prompt, interpret it directly using the domain knowledge below — do not call APIs to re-fetch fields
   already present. If the job ID, time range, or record is missing, retrieve it from the cluster.

   The decision: proceed with judgment-only explanation when the record contains `record_score`, `initial_record_score`,
   `actual`, `typical`, and `function`; otherwise fetch the missing pieces before explaining.

2. **Verify connectivity when calling the cluster.** Call `GET /`. If the call fails, stop and surface the connection
   error — do not guess endpoints or credentials.

3. **Resolve the job ID and load config.** When the job ID is unknown, call `GET /_ml/anomaly_detectors` to list
   candidates. Call `GET /_ml/anomaly_detectors/{job_id}` for full `analysis_config` (bucket_span, detectors,
   custom_rules, use_null, model_plot_config) and `GET /_ml/anomaly_detectors/{job_id}/_stats` for `state`,
   `model_size_stats.memory_status`, and data counts.

   The decision: confirm detector function and direction match the user's question before interpreting scores. A
   `low_count` job legitimately fires on drops; a `high_count` job does not.

4. **Retrieve anomaly records for the time range.** Call `POST /.ml-anomalies-*/_search` with `result_type: record`, the
   job ID, a timestamp range, and optional `record_score` filter. Read `initial_record_score`, `record_score`, `actual`,
   `typical`, `function`, `multi_bucket_impact`, and `anomaly_score_explanation`.

   Always show **both** `initial_record_score` and `record_score`. The gap is the renormalization story.

5. **Classify the score pattern before speculating on causes.**
   - **`initial_record_score` >> `record_score`** — **Renormalization.** A later, more extreme anomaly rescale this
     record downward. This is expected, healthy model behavior — not a broken model or reason to distrust the detection.
     Use `initial_record_score` for alerting severity; show both scores and explain the gap explicitly.
   - **`initial_record_score` == `record_score`** — No renormalization has occurred since detection.
   - **`actual` << `typical` with `low_count`, `count`, or `low_mean`** — **Absence / drop anomaly.** A high score is
     legitimate — the job detected an outage, pipeline stall, or service failure. This is **not** a false positive.
     Recommend incident investigation, not score tuning.
   - **`actual` >> `typical` with `high_count` or `high_mean`** — Spike anomaly; confirm with `single_bucket_impact`.

   **Only cite `anomaly_score_explanation` factors present in the record.** If `high_variance_penalty` is `false`, do
   not blame variance. If a factor is absent, note that it was not returned — do not invent it.

6. **Quantify renormalization across the job (optional).** Re-query `POST /.ml-anomalies-*/_search` for records in the
   time range sorted by `timestamp` ascending. Compute `score_drift = initial_record_score − record_score` per record
   and filter to `|score_drift| ≥ 20`. Large negative drift (initial >> record) confirms renormalization after a more
   extreme anomaly appeared later.

7. **Add context when the user asks "what caused this?" or "why so low/high?"**
   - **Model bounds** — If `model_plot_config.enabled` is true, call `POST /.ml-anomalies-*/_search` with
     `result_type: model_plot` for the same job and time range. Compare `actual` to `model_lower` / `model_upper`.
   - **Influencers** — Call `POST /.ml-anomalies-*/_search` with `result_type: influencer` for the bucket time range;
     sort by `influencer_score` descending.
   - **Categorization jobs** — Call `POST /.ml-anomalies-*/_search` with `result_type: category_definition` to list
     learned log patterns (`terms`, `regex`, `examples` per `category_id`).

   For aggregations, cross-job queries, bucket-level results, or custom filters beyond score and time, see
   [references/explainer-reference.md](references/explainer-reference.md).

## Common multi-step workflows

| Task                            | Steps (in order)                                                                                     |
| ------------------------------- | ---------------------------------------------------------------------------------------------------- |
| **Explain a specific anomaly**  | job config → records (job + exact time) → show initial_record_score vs record_score + score factors. |
| **Why is my score low?**        | job config → records → renormalization check → model plot (if enabled) → explain score factors.      |
| **Why is my score high?**       | job config → records → check function direction, insufficient history, use_null, cardinality.        |
| **Renormalization drift**       | records (timestamp sort) → compute score_drift → list records where initial >> record.               |
| **Which entities contributed?** | influencers (job + time range) → sort by influencer_score.                                           |
| **Visualize model bounds**      | model plot (job + time range) → compare model_lower/model_upper vs actual.                           |
| **Categorization job patterns** | category_definition (job_id) → terms, regex, examples per category.                                  |

## Critical principles

- **Retrieve the record first** (or use the one the user supplied). Never explain scores without `initial_record_score`,
  `record_score`, `actual`, `typical`, and `function`.
- **Renormalization is healthy.** When `initial_record_score >> record_score`, a more extreme anomaly appeared later and
  lowered this score — expected behavior, not a model failure.
- **Direction matters.** `low_count` fires when values drop; `high_count` fires on spikes. A high score on a traffic
  stop with `low_count` is correct detection, not a false positive.
- **Explain factors before speculating.** Read `anomaly_score_explanation` from the record. Only address factors that
  are present and relevant.
- **Job config is essential.** `bucket_span`, detector function, `custom_rules`, `use_null`, and memory status all
  affect scores. Inspect job config when a score is surprising.
- **Model plot is the most visual explanation.** When enabled, show model bounds to illustrate where the actual value
  falls relative to the expected range.
- For job health ("missing documents", "memory limit", "datafeed not running") use the `elasticsearch-anomaly-detection`
  skill.

## Domain knowledge

### Score types

| Term                     | Meaning                                                                                  |
| ------------------------ | ---------------------------------------------------------------------------------------- |
| **record_score**         | Normalized 0–100 for a single anomaly record; updated by renormalization. >75 critical.  |
| **initial_record_score** | Score assigned at detection time, before renormalization. Use for alerting.              |

From elastic/agent-skills, Apache-2.0.

More from elastic/agent-skills