scaffold-observability

Files (1)

• SKILL.md

SKILL.md (first lines)

# Scaffold ArchiPy OpenTelemetry

## Read first

Read these plugin rules in full before generating files (paths relative to this skill directory; Claude Code: `${CLAUDE_SKILL_DIR}/../../rules/`):

- `../../rules/config-and-di.mdc`

## Before writing files

1. Inspect `pyproject.toml`/`uv.lock`, `AppConfig`, bootstrap order, app transports, adapters, and `.env.example`.
2. Infer required instrumentation extras from the installed stack.
3. Ask only for unresolved choices: enabled signals, metrics exporter (`otlp` / `pull` /
   `pushgateway`), OTLP endpoint/protocol or Pushgateway URL, service name, sampling ratio,
   and log level.
4. Preserve existing config/bootstrap. Never write collector credentials or OTLP headers with secret values.

## Install only matching extras

| Stack | Extra |
|-------|-------|
| Core OTLP traces, metrics, logs | `otel` |
| FastAPI | `otel` + `fastapi` (no separate extra) |
| gRPC | `otel-grpc` |
| SQLAlchemy | `otel-sqlalchemy` |
| Redis | `otel-redis` |
| Elasticsearch | `otel-elasticsearch` |
| Kafka | `otel-kafka` |
| ScyllaDB | `otel-scylladb` |
| MinIO/botocore | `otel-minio` |

Combine required extras, for example:

```bash
uv add "archipy[otel,fastapi,otel-sqlalchemy,otel-redis]"
```

Do not use removed ArchiPy 4.x extras: `prometheus`, `sentry`, or `elastic-apm`.

## Configure

Add non-secret defaults/documentation to `.env.example`:

```bash
OTEL__IS_ENABLED=true
OTEL__SERVICE_NAME=my-service
OTEL__OTLP_ENDPOINT=http://localhost:4317
OTEL__PROTOCOL=grpc
OTEL__TRACES_ENABLED=true
OTEL__METRICS_ENABLED=true
OTEL__METRICS_EXPORTER=otlp
# Pull scrape: OTEL__METRICS_EXPORTER=pull
# Pushgateway: OTEL__METRICS_EXPORTER=pushgateway
# OTEL__METRICS_PUSHGATEWAY_URL=http://pushgateway.monitoring:9091
OTEL__METRICS_PULL_HOST=0.0.0.0
OTEL__METRICS_PULL_PORT=8200
OTEL__SYSTEM_METRICS_ENABLED=true
OTEL__LOGS_ENABLED=true
OTEL__LOGS_EXPORTER=console
OTEL__TRACES_SAMPLE_RATIO=0.1
OTEL__LOGS_LEVEL=WARNING
OTEL__FASTAPI_EXCLUDED_URLS=health,docs,redoc,openapi.json
```

Use `BaseConfig.OTEL`; ArchiPy builds providers programmatically. Do not rely on OpenTelemetry SDK `OTEL_*`
autoconfiguration. For `http/protobuf`, use port 4318; ArchiPy appends `/v1/{signal}` when the base endpoint has no path.
Set `OTEL__METRICS_EXPORTER=pull` for scrape-only metrics. Set `OTEL__METRICS_EXPORTER=pushgateway`
and `OTEL__METRICS_PUSHGATEWAY_URL` (job defaults to `SERVICE_NAME`) for Prometheus Pushgateway —
no local `/metrics` server. `otlp` pushes OTLP to a collector; do not confuse it with Pushgateway.
If pull scrape cannot bind, ArchiPy logs a warning and keeps traces/logs/metrics providers.
Set `OTEL__LOGS_EXPORTER=otlp` to push logs to a collector (default `console` writes INFO/DEBUG to
stdout and WARNING+ to stderr).

## Initialize before adapters

After `BaseConfig.set_global(config)` and before constructing the DI container/adapters:

```python
from archipy.configs.base_config import BaseConfig
from archipy.helpers.utils.otel_utils import OtelUtils

config = AppConfig()
BaseConfig.set_global(config)
OtelUtils.init_otel_if_needed(config)
container = ApplicationContainer()
```

Initialization is idempotent. AppUtils calls it again safely. Early initialization matters for SQLAlchemy engines,
Kafka clients, ScyllaDB sessions, and logs created during bootstrap.

## Instrument application code

- FastAPI: `AppUtils.create_fastapi_app(config)` with `archipy[otel,fastapi]`.
- gRPC server: `create_grpc_app` / `create_async_grpc_app` with `archipy[otel-grpc]`.
  - Traces: contrib server interceptor when `OTEL.TRACES_ENABLED`.
  - Metrics: ArchiPy `GrpcServerOtelMetricsInterceptor` /
    `AsyncGrpcServerOtelMetricsInterceptor` records `rpc.server.call.duration` when
    `OTEL.METRICS_ENABLED` (works without traces).
- Tuning (archipy >= 5.16.0): `OTEL.TRACES_SAMPLER`, `PROPAGATORS` (`b3`, `b3multi`; B3 ships in `archipy[otel]`),
  `OTLP_COMPRESSION` (`none` by default), `SPAN_BATCH_*`, `RESOURCE_DETECTORS`; `USE_STANDARD_ENV` makes the standard
  `OTEL_*` variables fill unset settings.
- gRPC clients: `OtelUtils.grpc_client_interceptors()` / `async_grpc_client_interceptors()`.
- Traces: `trace_root`, `trace_span`, `async_trace_root`, `async_trace_span`, `trace_class`.
- Metrics: `measure_duration`, `async_measure_duration`, `count_calls`, `async_count_calls`.
- Exceptions: `BaseUtils.capture_exception` records on the current span.

Import decorators from `archipy.helpers.decorators`. `measure_duration`, `count_calls`, `trace_span` and `trace_root` detect `async def`; the `async_*` twins remain.
Only capture non-sensitive argument names in trace attributes.

## ArchiPy 4.x migration

- `TracingUtils` → `OtelUtils`
- `capture_transaction` / `capture_span` → `trace_root` / `trace_span` (+ async twins)
- Prometheus/Sentry/Elastic APM config → `BaseConfig.OTEL`
- Removed FastAPI rate-limit handler → `fastapi-redis-sdk`

## Do not

- Never commit OTLP credentials; no secrets in source, logs, or committed env files.

From SyntaxArc/archipy-plugin, MIT.

More from SyntaxArc/archipy-plugin