log-management-specialist

Files (1)

• SKILL.md

SKILL.md (first lines)

# Log Management Specialist

Domain: Observability · Structured Logging · Log Aggregation · Distributed Tracing
Expert model: Charity Majors (Honeycomb) — production observability, high-cardinality telemetry, SRE discipline

---

## Pre-flight Check

Before advising, confirm:

1. **Stack declared?** → check `.raven/manifest.json` for `stack`
2. **Current logging approach?** → ask if not obvious from code
3. **Log destination?** → CloudWatch / Elasticsearch / Loki / Splunk / stdout
4. **Retention requirements?** → compliance, cost, debug window
5. **PII risk?** → user data, emails, tokens in log fields

---

## Core Principles

```
1. Structured logs ONLY — JSON lines, not free-text strings
2. One log entry per request/event — no chatty loops
3. Correlation IDs on every log — trace_id, request_id, session_id
4. Never log secrets — API keys, passwords, tokens → [REDACTED]
5. Never log raw PII — emails, phone numbers → hash or mask
6. Log at the right level — DEBUG off in prod, WARN/ERROR always on
7. Fail-open — log pipeline failure must not crash the application
```

---

## Log Levels — When to Use Each

| Level | Use case | Prod enabled? |
|---|---|---|
| `DEBUG` | Internal state, variable dumps, loop tracing | ❌ Never |
| `INFO` | Business events, request start/end, key decisions | ✅ Yes |
| `WARNING` | Degraded state, retries, expected failures | ✅ Yes |
| `ERROR` | Unexpected failure, caught exception with stack | ✅ Yes |
| `CRITICAL` | System can't continue, data loss risk | ✅ Yes — page immediately |

**Rule:** If you're logging in a loop → wrong level or wrong design.

---

## Structured Log Format

Every log line must be a JSON object:

```json
{
  "timestamp": "2026-05-14T12:00:00.000Z",
  "level": "INFO",
  "service": "auth-service",
  "trace_id": "abc123",
  "request_id": "req-456",
  "user_id": "u-789",
  "event": "login.success",
  "duration_ms": 42,
  "message": "User authenticated via OAuth"
}
```

**Never this:**
```python
logger.info(f"User {email} logged in at {time}")  # ❌ free-text, PII in string
```

**Always this:**
```python
logger.info("login.success", extra={
    "user_id": user.id,          # ✅ ID, not email
    "duration_ms": elapsed_ms,
    "provider": "oauth",
})
```

---

## Python — Canonical Setup

```python
import logging
import json
import sys
from pythonjsonlogger import jsonlogger

def get_logger(name: str) -> logging.Logger:
    logger = logging.getLogger(name)
    handler = logging.StreamHandler(sys.stdout)
    formatter = jsonlogger.JsonFormatter(
        fmt="%(asctime)s %(name)s %(levelname)s %(message)s",
        datefmt="%Y-%m-%dT%H:%M:%S"
    )
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    logger.setLevel(logging.INFO)
    return logger
```

Library: `python-json-logger` (approved in Raven manifest before use)

---

## Correlation IDs — Mandatory Pattern

Every request must carry a trace ID through all log lines:

```python
import uuid
from contextvars import ContextVar

_trace_id: ContextVar[str] = ContextVar("trace_id", default="")

def set_trace_id(tid: str | None = None) -> str:
    tid = tid or str(uuid.uuid4())
    _trace_id.set(tid)

From giggsoinc/raven, MIT.

More from giggsoinc/raven