otel-collector

Files (113)

• SKILL.md
• README.md · components/attributes/
• advanced.md · components/attributes/
• configuration.md · components/attributes/
• quirks.md · components/attributes/
• verification.md · components/attributes/
• README.md · components/drain/
• advanced.md · components/drain/
• configuration.md · components/drain/
• quirks.md · components/drain/
• verification.md · components/drain/
• README.md · components/file_log/
• advanced.md · components/file_log/
• configuration.md · components/file_log/
• operators.md · components/file_log/
• quirks.md · components/file_log/
• verification.md · components/file_log/
• README.md · components/file_storage/
• advanced.md · components/file_storage/
• configuration.md · components/file_storage/
• quirks.md · components/file_storage/
• verification.md · components/file_storage/
• README.md · components/filter/
• advanced.md · components/filter/
• configuration.md · components/filter/
• quirks.md · components/filter/
• verification.md · components/filter/
• README.md · components/interval/
• advanced.md · components/interval/
• configuration.md · components/interval/
• quirks.md · components/interval/
• verification.md · components/interval/
• README.md · components/k8s_attributes/
• advanced.md · components/k8s_attributes/
• configuration.md · components/k8s_attributes/
• quirks.md · components/k8s_attributes/
• verification.md · components/k8s_attributes/
• README.md · components/load_balancing/
• advanced.md · components/load_balancing/
• configuration.md · components/load_balancing/
• quirks.md · components/load_balancing/
• verification.md · components/load_balancing/
• README.md · components/log_dedup/
• advanced.md · components/log_dedup/
• configuration.md · components/log_dedup/
• quirks.md · components/log_dedup/
• verification.md · components/log_dedup/
• README.md · components/memory_limiter/
• advanced.md · components/memory_limiter/
• configuration.md · components/memory_limiter/
• quirks.md · components/memory_limiter/
• verification.md · components/memory_limiter/
• README.md · components/otlp/
• advanced.md · components/otlp/
• configuration.md · components/otlp/
• quirks.md · components/otlp/
• verification.md · components/otlp/
• README.md · components/otlp_exporter/
• advanced.md · components/otlp_exporter/
• configuration.md · components/otlp_exporter/
• …and 53 more

SKILL.md (first lines)

# OpenTelemetry Collector

This skill covers the configuration surface of individual OpenTelemetry Collector components. It targets [opentelemetry-collector](https://github.com/open-telemetry/opentelemetry-collector) and [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib).

It does **not** cover OTTL expressions (see `otel-ottl`), declarative SDK configuration (`otel-declarative-config`), deploying the eBPF profiler or other profiles producers (`otel-profiles`), or end-to-end pipeline design choices. Reach for those skills when the question is about transformation language, SDK setup, or pipeline composition.

## Workflow

1. **Identify the component.** Find the `type` in the user's config or question (`log_dedup`, `interval`, `otlp`, …). Note that many components have been renamed to snake_case since v0.146.0 with deprecated aliases preserved — see [Recent renames](#recent-renames).
2. **Load the component page before answering.** If the component is in the [Component index](#component-index), always read `components/<type>/README.md` first — the top-level index is only a routing aid, not enough evidence for an answer. Follow its **Details** index to the smallest relevant file: read `configuration.md` for versioned keys, defaults, or validation; `quirks.md` for stability, availability, failure behavior, or migration traps; and the other detail files only as the question requires. Do not infer exact configuration or status from the summary row or memory.
3. **If the component is not indexed**, say so explicitly and fall back to the upstream README under `processor/<name>/`, `receiver/<name>/`, `exporter/<name>/`, `connector/<name>/`, or `extension/<name>/` in [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib). Don't invent config keys from memory — Collector components evolve quickly.
4. **Apply Collector-wide conventions.** Named instances (`type/name`), stability levels, and pipeline placement rules in [Collector-wide conventions](#collector-wide-conventions) apply to every component.
5. **Verify.** Run the component page's **Verification** recipe — `telemetrygen` (see the `otel-telemetrygen` skill) plus a `debug` or `file` exporter — to confirm the component behaves as the docs claim. Alpha- and Development-stability components are common here, and behavior changes between releases. See [Verification harness](#verification-harness) for how to run a recipe end-to-end.

## Component index

Each component is a directory under `components/<type>/`. The `File` column points at the lean `README.md` (metadata, description, main use-cases, and a **Details** index); the full config reference, verification recipe, advanced use-cases, and quirks live in on-demand detail files linked from that README.

Coverage is intentionally selective. If a component is not indexed here, fall back to the upstream component README for the user's Collector version.

Verified 2026-10-25: Stability cells here and in component README headers match `metadata.yaml` at core and contrib `v0.163.0`; rename and "since" versions match `metadata.yaml` at the named tag and the one before. Offline, cite that release and date, not current stability.

Verified 2026-10-31: the v0.164.0 notes in component pages (`PermissiveLabelSanitization` and `filelog.protobufCheckpointEncoding` Stable; `file_log`, `transform` and `resource_detection` in the `core` distribution) match the [contrib v0.164.1 changelog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.164.1/CHANGELOG.md), [`pkg/translator/prometheus/metadata.yaml`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.164.1/pkg/translator/prometheus/metadata.yaml), [`pkg/stanza/fileconsumer/metadata.yaml`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.164.1/pkg/stanza/fileconsumer/metadata.yaml) and the [`otelcol` manifest at v0.164.0](https://github.com/open-telemetry/opentelemetry-collector-releases/blob/v0.164.0/distributions/otelcol/manifest.yaml).

| Type | File | Kind | Signals | Stability | Summary |
|------|------|------|---------|-----------|---------|
| `log_dedup` | `components/log_dedup/README.md` | processor | logs | Alpha | Deduplicates identical log records over a time window; emits one aggregated log with a count. Renamed from `logdedup` in v0.151.0; alias preserved. |
| `interval` | `components/interval/README.md` | processor | metrics | Alpha | Buffers metrics and emits one point per series per interval: latest value for cumulative monotonic series, gauges, and summaries; the summed value for delta series. Non-monotonic sums pass through unchanged. |
| `tail_sampling` | `components/tail_sampling/README.md` | processor | traces | Beta | Buffers whole traces and makes a single keep/drop decision after a wait window via policies. Supports in-process concurrency through `num_shards`; scaling across Collector instances still requires a `load_balancing` layer. v0.160 adds tracestate-aware thresholds for rate/bytes limiting. |
| `drain` | `components/drain/README.md` | processor | logs | Alpha | Clusters log bodies and annotates each record with a template. v0.158 replaced `extract_parameters` / `params_attribute` with `masking_rules` + `parameter_key_prefix` for named values and `emit_wildcards` + `wildcards_attribute` for remaining `<*>` positions. |
| `redaction` | `components/redaction/README.md` | processor | traces, logs, metrics | Beta (traces), Alpha (logs/metrics) | Allow/block-list masking or removal of sensitive attribute keys and values, with hashing and URL/DB sanitizers. |
| `filter` | `components/filter/README.md` | processor | traces, metrics, logs, profiles | Alpha (traces/metrics/logs), Development (profiles) | Drops spans, metric data points, log records, and profiles that match OTTL conditions (or legacy metric-name / severity filters). The most direct telemetry-volume lever. |
| `transform` | `components/transform/README.md` | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Development (profiles) | Applies OTTL statements to mutate spans, metric data points, log records, and profiles in place (rename/redact/convert/aggregate attributes). The OTTL language itself lives in the `otel-ottl` skill. |
| `probabilistic_sampler` | `components/probabilistic_sampler/README.md` | processor | traces, logs | Beta (traces), Alpha (logs) | Head sampling — deterministically keeps a configured percentage of traces/logs by hashing the trace ID (or an attribute). The cheaper, stateless counterpart to `tail_sampling`. |
| `attributes` | `components/attributes/README.md` | processor | traces, metrics, logs | Beta | Modifies span/log/datapoint **attributes** via an ordered action list (insert/update/upsert/delete/hash/extract/convert), scoped by include/exclude matching. |
| `resource` | `components/resource/README.md` | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Development (profiles) | Modifies the **resource** attributes of telemetry via the same action grammar as `attributes` (e.g. set `service.name`, drop noisy resource keys). No include/exclude matching. |
| `k8s_attributes` | `components/k8s_attributes/README.md` | processor | traces, metrics, logs, profiles | Stable (traces/metrics/logs, since v0.161), Development (profiles) | Enriches telemetry with Kubernetes pod/namespace/node/workload metadata, associating each record to a pod by IP or resource attribute. v0.161 enables the v1 semconv gates by default (singular `k8s.pod.label.*`, `container.image.tags`). |
| `routing` | `components/routing/README.md` | connector | traces, metrics, logs | Alpha | Routes same-signal telemetry to different pipelines by OTTL `condition`/`statement` per context; ordered table, first match wins, `default_pipelines` fallback. Replaces the deprecated `routingprocessor`. |
| `memory_limiter` | `components/memory_limiter/README.md` | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Alpha (profiles) | Safety valve against OOM: refuses data with backpressure when Go heap exceeds a soft/hard limit. Its receiver-middleware **extension** reached Beta in v0.158 but is absent from stock distributions; the bundled processor remains the standard form. |
| `load_balancing` | `components/load_balancing/README.md` | exporter | traces, logs, metrics | Beta (traces/logs), Alpha (metrics) | Distributes telemetry across downstream Collectors via a consistent-hash ring keyed on trace ID / `service.name` / (v0.162+) OTel randomness, etc., pinning related records to one backend. The standard way to scale `tail_sampling`/`span_metrics`. Renamed from `loadbalancing` in v0.153.0; alias preserved. |
| `otlp` (receiver) | `components/otlp/README.md` | receiver | traces, metrics, logs, profiles | Stable (traces/metrics/logs), Alpha (profiles) | The canonical OTLP ingress over gRPC (`4317`) and/or HTTP (`4318`, protobuf + JSON). `protocols:` block, at least one required. Default endpoint is `localhost`, not `0.0.0.0` — must be set explicitly to receive containerized traffic. |
| `otlp_grpc` (exporter) | `components/otlp_exporter/README.md` | exporter | traces, metrics, logs, profiles | Stable (traces/metrics/logs), Alpha (profiles) | Sends OTLP over gRPC to a downstream endpoint; gzip, sending queue, and retry are enabled by default. Exporter-side `sending_queue.batch` is opt-in unless the `pkg.exporterhelper.queueBatchEnabled` gate is enabled. Renamed from `otlp` in core v1.50.0; alias preserved. |
| `file_log` (receiver) | `components/file_log/README.md` | receiver | logs | Beta | Tails log files (glob `include`), turns each line/entry into a log record, and parses it via a stanza `operators` pipeline. Fingerprint + offset tracking across rotation; `storage` for durable offsets. v0.160 adds opt-in `skip_unmodified_files`. |
| `resource_detection` | `components/resource_detection/README.md` | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Development (profiles) | Auto-detects resource attributes via an ordered `detectors` list with global retry settings. Includes Azure Container Apps, Azure App Service (v0.160+), and Azure Functions (v0.162+) detectors. Renamed from `resourcedetection` in v0.153.0; alias preserved. |
| `file_storage` | `components/file_storage/README.md` | extension | — (not pipeline-scoped) | Beta | Persists component state to local disk (bbolt) so it survives a Collector restart: receiver read offsets (`storage:`), exporter persistent send queues (`sending_queue.storage:`), and stateful-processor decision caches. One bbolt file per consumer; `directory` must exist unless `create_directory: true`. Not placed in a pipeline — listed under `service.extensions:` and referenced by other components. |
| `prometheus` (receiver) | `components/prometheus/README.md` | receiver | metrics | Beta | Scrapes Prometheus-format HTTP endpoints and converts them to OTLP metrics, embedding `prometheus/prometheus`'s scrape manager — the `config:` block is the same YAML as `prometheus.yml` (`global`, `scrape_configs`, `scrape_config_files`, service discovery, relabeling). Extra keys: `trim_metric_suffixes`, `target_allocator` (sharded targets from the OTel Operator), `api_server` (debug API). Scrape-only: rejects `remote_write`/`remote_read`/`rule_files`/alerting. Stateful, does not auto-shard across replicas. |
| `prometheus` (exporter) | `components/prometheus_exporter/README.md` | exporter | metrics | Beta | Exposes pipeline metrics for Prometheus to scrape. In-memory accumulator expires stale series after 5m. v0.160 adds `resource_constant_labels` and deprecates `resource_to_telemetry_conversion`; `add_metric_suffixes` remains deprecated in favor of `translation_strategy`. |
| `prometheus_remote_write` (exporter) | `components/prometheus_remote_write/README.md` | exporter | metrics | Beta | Pushes OTLP metrics as Prometheus Remote Write. Nests HTTP client settings under `http`; uses `remote_write_queue` (not `sending_queue`). v0.160 adds `resource_constant_labels` and deprecates `resource_to_telemetry_conversion`; RW2 remains gated and in development. |

## Collector-wide conventions

### Named instances

Every component type supports the `type/name` pattern so the same type can be configured more than once. The pipeline references the named form:

```yaml
processors:
  log_dedup/health-checks:
    interval: 30s
    conditions:
      - 'log.attributes["log.type"] == "health_check"'
  log_dedup/access-logs:
    interval: 10s

service:
  pipelines:
    logs/health:
      processors: [log_dedup/health-checks]
    logs/access:
      processors: [log_dedup/access-logs]
```

### Stability levels

Components publish a stability level per signal. Treat these as load-bearing when recommending production use:

| Level | Use in |
|-------|--------|
| Development | Tests and prototypes only — breaking changes expected. |
| Alpha | Limited, non-critical workloads — config keys can still change. |
| Beta | Production viable — breaking changes rare. |
| Stable | Production — backward compatibility guaranteed. |

Stability now varies per component — and per signal for multi-signal components (e.g. `redaction` is Beta for traces but Alpha for logs/metrics). Don't assume a single level for the indexed set: check each component's README header metadata table for the authoritative stability and surface it when the user asks about production readiness.

### Recent renames

Many components have been renamed to snake_case since v0.146.0, and renames are still landing (e.g. the `awss3` receiver → `aws_s3` in v0.162.0, `sqlserver` → `sql_server` in v0.163.0; receivers and exporters of one family rename separately, so check the component's `metadata.yaml` `type`). The legacy names remain as deprecated aliases — old configs keep working but new configs should use the new names. Check the upstream component README for the exact rename version before flagging a config as broken.

Examples: `logdedup` → `log_dedup`, `hostmetrics` → `host_metrics`, `spanmetrics` → `span_metrics`, `servicegraph` → `service_graph`, `k8sattributes` → `k8s_attributes`, plus several `_log` and `_check` receivers.

### Pipeline placement

Two rules of thumb that apply across components:

- `memory_limiter` belongs first in any processor list, before anything that allocates buffers (`log_dedup`, `transform`, `tail_sampling`, …).
- The `otlp_grpc` exporter's sending queue is enabled by default, but its `sending_queue.batch` sub-block is opt-in. Enable exporter-side batching explicitly (or with the alpha `pkg.exporterhelper.queueBatchEnabled` migration gate). The separate `batch` processor remains Beta and supported; use it when pipeline-level batching is needed and place it after data-dropping processors.

### Verification harness

Each component page's **Verification** section gives a config, a `telemetrygen` command, and the expected output. To run any of them:

1. Save the YAML to a file and start a collector that bundles the component — for components in the `contrib`/`k8s` distributions, `otelcol-contrib --config <file>.yaml`; for components not in any distribution, build a custom collector with the [OpenTelemetry Collector Builder (OCB)](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder) first.
2. Send telemetry with `telemetrygen` (see the `otel-telemetrygen` skill).
3. Watch the `debug` exporter's stdout (or the `file` exporter's output) for the expected result.

The Verification configs are **minimal repros**: they omit `memory_limiter` and other production scaffolding on purpose, to isolate the component under test. Don't copy them verbatim into production.

## Adding a new component to this skill

When extending coverage:

1. **Create the directory `components/<type>/`.** Files carry no frontmatter — only `SKILL.md` has frontmatter.
2. **Write a lean `README.md`** — always loaded, kept small:
   - **Header metadata table** — kind, `type` name, signals, per-signal stability, distributions, Go module, upstream README link, and a rename note if the component was renamed.
   - **Description** — what the component does and the mechanism, in 1–2 tight paragraphs (push detailed mechanism/reference into `configuration.md`).
   - **Main use-cases** — "Use when" / "Avoid when".
   - **Related components** — cross-links.
   - **Details index** — a bullet list where each item links a detail file followed by an em dash and a short description of its contents (e.g. `- [Configuration](configuration.md) — config keys, defaults, validation`), so the reader loads only what a question needs.

From ollygarden/opentelemetry-agent-skills, Apache-2.0.

More from ollygarden/opentelemetry-agent-skills