log-error-digest

Files (3)

• LICENSE
• SKILL.md
• analyze_logs.py · scripts/

SKILL.md (first lines)

# Log Error Digest

Automated log file analysis that produces error clustering, frequency statistics, and time distribution reports.

## Features

- **Error Clustering**: Groups similar error messages by normalizing dynamic parts (IPs, UUIDs, numbers, etc.) to identify root causes
- **Frequency Statistics**: Counts occurrences by error type, sorted by severity
- **Time Distribution**: Shows error distribution by hour and by date, helping pinpoint peak error periods

## Supported Log Formats

| Format | Description | Auto-detection |
|--------|-------------|----------------|
| JSON | One JSON object per line with `timestamp`/`level`/`message` fields | Starts with `{` |
| syslog | RFC 3164 format, e.g. `Jan  1 12:00:00 host proc[pid]: msg` | Starts with month name |
| Nginx | Access log or error log format | Starts with IP or date/path pattern |

## Usage

```bash
python scripts/analyze_logs.py <log_file_path> [options]
```

### Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `log_file` | Path to the log file (required) | - |
| `--format` | Log format: `auto`/`json`/`syslog`/`nginx` | `auto` |
| `--top` | Show Top N error clusters | `20` |
| `--output` | Export results to a JSON file | Terminal output only |
| `--level` | Filter by log level (e.g. `ERROR`, `WARN`) | All levels |
| `--since` | Only analyze logs after this time (ISO format) | No limit |
| `--until` | Only analyze logs before this time (ISO format) | No limit |

### Examples

```bash
# Auto-detect format and analyze the entire log file
python scripts/analyze_logs.py /var/log/app.log

# Specify Nginx format, show only Top 10 errors
python scripts/analyze_logs.py /var/log/nginx/error.log --format nginx --top 10

# Filter ERROR level only, export JSON report
python scripts/analyze_logs.py app.log --level ERROR --output report.json

# Analyze logs within a specific time range
python scripts/analyze_logs.py app.log --since 2024-01-01T00:00:00 --until 2024-01-02T00:00:00
```

## Output

### Terminal Output

```
=======================================================
              Log Analysis Report
=======================================================

📊 Overview
  Detected format: json
  Total lines:     15,234
  Parsed:          15,100 (parse failures: 134)
  Matched entries: 12,800
  Errors:          2,341
  Time range:      2024-01-01 00:03:12 ~ 2024-01-01 23:58:45

🔴 Top Error Clusters (47 total)
  #1   [×523  ] Connection refused to database at 10.0.1.5:5432
       First seen: 2024-01-01T00:15:30  Last seen: 2024-01-01T23:45:12
  #2   [×312  ] Timeout waiting for response from user-service after 30000ms
       First seen: 2024-01-01T02:10:00  Last seen: 2024-01-01T22:30:45
  #3   [×198  ] File not found: /data/uploads/img_99421.png
       First seen: 2024-01-01T08:00:00  Last seen: 2024-01-01T20:15:33
  ...

⏰ Time Distribution (by hour)
  00:00  █████░░░░░░░░░░░░░░░  42
  01:00  ██░░░░░░░░░░░░░░░░░░  18
  ...
  14:00  ████████████████████  523
  ...

📅 Time Distribution (by date)
  2024-01-01  ████████████████████  2,341
```

### JSON Output

Use the `--output` parameter to export a structured JSON report for further processing or integration with monitoring systems.

From zebbern/claude-code-guide, MIT.

More from zebbern/claude-code-guide