eval

Files (1)

• SKILL.md

SKILL.md (first lines)

# Eval Skill

Structured evaluation of everything the agent manages.

---

## ⚡ Execution Architecture (Anti-Timeout)

Eval is split into 3 independent subagents that run in parallel.
Each subagent handles one domain, returns a short result block, and exits.
Main agent collects results and formats the final report.

**On-demand (user asks):** Always run fresh — never read from cache.
**Cron/heartbeat:** Run async, save to `memory/eval-cache.json`, report only if issues found.

### Subagent Split
| Subagent | Domain | Expected Time |
|---|---|---|
| SA-1 | System health (vertex, WhatsApp, DB, backup) | <15s |
| SA-2 | PA network health (contact-list.md) | <15s |
| SA-3 | Tasks + memory + skills audit | <15s |

**Timeout:** Spawn all subagents with `runTimeoutSeconds=60`. Do NOT use `openclaw status` — it is slow and causes SIGKILL. Use only fast targeted commands (see templates below).

### How to Run (on-demand)
```
1. Spawn SA-1, SA-2, SA-3 simultaneously (sessions_spawn, runtime=subagent)
2. Each returns a JSON block with its section
3. Main agent merges and formats the final report
4. Send to owner
```

### Subagent Task Templates

**SA-1 — System Health:**
```
Run system health check using ONLY fast targeted commands (do NOT run openclaw status):
1. /path/to/ocana/bifrost/vertex-ctl.sh status 2>&1 | head -3
2. curl -s --max-time 5 http://127.0.0.1:18789/ | head -1 || echo 'gateway unreachable'
3. git -C /path/to/openclaw/workspace log -1 --format="%ar"
4. python3 -c "import os,subprocess; db=os.environ.get('PA_DB_URL',''); r=subprocess.run(['psql',db,'-c','SELECT COUNT(*) FROM messages;','-t'],capture_output=True,text=True,timeout=5) if db else None; print(r.stdout.strip() if r else 'no DB')"
Return JSON: {"vertex": "RUNNING/DOWN", "gateway": "reachable/unreachable", "backup": "X ago", "db": "X msgs"}
```

**SA-2 — PA Network:**
```
Read /path/to/workspace/contact-list.md
Find all PA entries (lines with "PA:" or under WhatsApp Groups / PAs section)
Count total PAs, check for any notes about issues/offline/billing
Return JSON: {"total": N, "online": N, "issues": [{"pa": "name", "issue": "description"}]}
```

**SA-3 — Tasks + Memory + Skills:**
```
Check:
1. wc -l /path/to/workspace/MEMORY.md
2. ls /path/to/workspace/skills | wc -l
3. Check today's daily note exists
4. grep tasks.md for open/done count
Return JSON: {"memory_lines": N, "skills": N, "daily_note": true/false, "tasks_open": N, "tasks_done": N}
```

---

## When to Use

Trigger phrases:
- "תעשי eval" / "run eval"
- "מה עובד ומה לא" / "what's working and what isn't"
- "תדרגי את עצמך" / "rate yourself"
- "בדקי הכל" / "check everything"

---

## Eval Report Format

```
📋 Full Eval — [DATE]

━━━ SELF PERFORMANCE ━━━
Execution:      [1-5] [comment]
Accuracy:       [1-5] [comment]
Memory:         [1-5] [comment]
Proactivity:    [1-5] [comment]
Communication:  [1-5] [comment]
TOTAL: [X]/25

━━━ ACTIVE TASKS ━━━
✅ Done today:   [count]
🟡 In progress:  [count]
❌ Stalled:      [count] — [list stalled tasks]

━━━ PA NETWORK ━━━
✅ Working:  [list]
⚠️ Issues:   [list with issue]
❌ Down:     [list]

━━━ SKILLS ━━━
Installed: [count]
Used today: [list]
Unused (7+ days): [list]

━━━ INTEGRATIONS ━━━
Calendar (owner):     [connected ✅ / broken ❌ / unknown ?]
monday.com:           [connected ✅ / broken ❌]
Email (gog):          [connected ✅ / broken ❌]
GitHub backup:        [last push: X ago]
WhatsApp:             [connected ✅ / disconnected ❌]

━━━ MEMORY HEALTH ━━━
Daily notes:     [today's file exists? ✅/❌]
Long-term:       [MEMORY.md size — OK / bloated]
Learnings:       [count this week]
Last backup:     [X ago]

━━━ RECOMMENDATIONS ━━━
1. [Most important thing to fix]
2. [Second priority]
3. [Optional improvement]

From netanel-abergel/pa-skills, MIT.

More from netanel-abergel/pa-skills