audit-observability

Files (1)

• SKILL.md

SKILL.md (first lines)

# Audit Observability

Scans the codebase and running services for missing or insufficient observability: logging, metrics, tracing, and alerting.
Produces a gaps report ranked by risk so teams can prioritize adding instrumentation before issues arise in production.

## Prerequisites

- Working directory is the root of the repository
- Optional: `$1` — path or service name to scope the audit (defaults to `.`)
- Access to the codebase to search for instrumentation patterns
- Read `.sdlc/context/architecture.md` to understand the services and infrastructure

## Observability Pillars

| Pillar | Purpose | Examples |
|---|---|---|
| **Logging** | Record discrete events for debugging | Structured logs, error logs, audit logs |
| **Metrics** | Quantitative measurements over time | Request count, latency histogram, error rate |
| **Tracing** | Follow a request across service boundaries | Distributed trace IDs, span propagation |
| **Alerting** | Notify on-call when conditions degrade | PagerDuty, OpsGenie, Grafana alerts |

## Steps

1. Read `.sdlc/context/architecture.md` to identify services, endpoints, databases, and external dependencies.

2. Detect which observability libraries and frameworks are already in use:
   ```
   rg -n -i "(structlog|logging|logger|prometheus|datadog|opentelemetry|otel|jaeger|zipkin|sentry)" \
     -g '*.{py,ts,js,go}' ${1:-.} | head -40
   ```

3. Audit **logging** coverage:
   - Are errors logged with context (request ID, user ID, stack trace)?
   - Are external service calls logged (request/response, latency, status)?
   - Are business-critical operations logged (auth events, data mutations, payments)?
   - Is structured logging used consistently?
   - Are log levels appropriate (not everything at INFO/DEBUG)?
   ```
   rg -n -i "(except|catch|error|raise|throw)" \
     -g '*.{py,ts,js,go}' ${1:-.} | rg -v "log|logger|sentry|capture" | head -30
   ```
     Flag error handlers that catch exceptions without logging them.

4. Audit **metrics** coverage:
   - Are HTTP endpoints instrumented (request count, latency, error rate)?
   - Are database queries instrumented (query latency, connection pool usage)?
   - Are external service calls instrumented (call count, latency, error rate)?
   - Are business metrics tracked (orders placed, emails sent, jobs completed)?
   - Are resource metrics available (CPU, memory, disk, connections)?
   ```
   rg -n -i "(counter|histogram|gauge|summary|metric|observe|inc\(|time\()" \
     -g '*.{py,ts,js,go}' ${1:-.} | head -30
   ```
     Identify endpoints and services with no metrics instrumentation.

5. Audit **tracing** coverage:
   - Is OpenTelemetry or equivalent configured?
   - Are trace IDs propagated across service boundaries (HTTP headers, message queues)?
   - Are database calls and external HTTP calls included in traces?
   - Are spans created for significant operations?
   ```
   rg -n -i "(trace_id|span_id|opentelemetry|otel|tracer|start_span|propagat)" \
     -g '*.{py,ts,js,go}' ${1:-.} | head -30
   ```

6. Audit **alerting** coverage:
   - Are alerts defined for SLO breaches (error rate, latency)?
   - Are alerts defined for infrastructure issues (disk full, memory pressure, connection exhaustion)?
   - Are alerts defined for business anomalies (queue depth, failed jobs, payment failures)?
   - Are alert thresholds appropriate (not too noisy, not too quiet)?
   - Look for alert configuration files:
   ```
   find ${1:-.} -type f \( -name "alerts*" -o -name "alerting*" -o -name "slo*" -o -name "rules*" \) \
     | rg -v node_modules | rg -v .git | head -20
   ```

7. Rank gaps by risk:
   - **High risk:** No error logging on critical paths, no alerts for SLO breaches, no metrics on user-facing endpoints
   - **Medium risk:** Missing tracing on cross-service calls, no metrics on background jobs, inconsistent log levels
   - **Low risk:** Missing business metrics, non-critical paths without tracing, verbose logging in non-critical paths

8. Produce the gaps report.

## Output Format

```markdown
# Observability Audit — <service-name or project>

**Date:** <YYYY-MM-DD>
**Scope:** <path or services audited>

## Summary

- **Logging:** Adequate / Partial / Missing
- **Metrics:** Adequate / Partial / Missing
- **Tracing:** Adequate / Partial / Missing
- **Alerting:** Adequate / Partial / Missing
- **Tools detected:** <prometheus, structlog, opentelemetry, sentry, etc.>
- **High-risk gaps:** N
- **Medium-risk gaps:** N
- **Low-risk gaps:** N

## Logging Gaps

| Risk | Location | Issue | Recommendation |
|---|---|---|---|
| High | `<file>:<line>` | Error swallowed without logging | Add structured error log with context |
| Medium | `<file>:<line>` | External call not logged | Add request/response logging |

## Metrics Gaps

| Risk | Endpoint / Operation | Issue | Recommendation |
|---|---|---|---|
| High | `POST /api/checkout` | No latency or error metrics | Add histogram for latency, counter for errors |
| Medium | Background job processor | No job count or duration metrics | Add counter and histogram |

## Tracing Gaps

| Risk | Boundary | Issue | Recommendation |

From tomzx/agents, MIT.

More from tomzx/agents