copilot-review-smart

Files (63)

• .gitignore
• README.md
• SKILL.md
• agents.mjs
• collection_failure.mjs
• decision.mjs
• decision.test.mjs
• decision_source.mjs
• harness.test.mjs
• gh · harness/bin/
• fake_openrouter.mjs · harness/
• fixture_data.mjs · harness/
• fixture_runtime.mjs · harness/
• gh.mjs · harness/
• run_fixture.mjs · harness/
• README.md · listener/
• package.json · listener/
• cli.mjs · listener/src/
• daemon.mjs · listener/src/
• gh.mjs · listener/src/
• hooks.mjs · listener/src/
• http-transport.mjs · listener/src/
• index.mjs · listener/src/
• listener.mjs · listener/src/
• notify.mjs · listener/src/
• paths.mjs · listener/src/
• signature.mjs · listener/src/
• state.mjs · listener/src/
• tick-runner.mjs · listener/src/
• tunnels.mjs · listener/src/
• pr_monitor_webhook.test.mjs · listener/test/
• README.md · observability/
• package.json · observability/
• cli.mjs · observability/src/
• paths.mjs · observability/src/
• project.mjs · observability/src/
• server.mjs · observability/src/
• stages.mjs · observability/src/
• state.mjs · observability/src/
• tui-cli.mjs · observability/src/
• tui.mjs · observability/src/
• audit-fs.mjs · observability/test/
• observability.test.mjs · observability/test/
• package.json · observability/test/
• suite.mjs · observability/test/
• tui.test.mjs · observability/test/
• pr_monitor.mjs
• pr_monitor.test.mjs
• pr_monitor_observability.mjs
• pr_monitor_tui.mjs
• pr_monitor_webhook.mjs
• Dockerfile · resources/deploy/
• README.md · resources/deploy/
• copilot-review-smart-listener@.service · resources/deploy/
• docker-compose.yml · resources/deploy/
• cloudflared.md · resources/deploy/ingress/
• ngrok.md · resources/deploy/ingress/
• reverse-proxy.md · resources/deploy/ingress/
• tailscale-funnel.md · resources/deploy/ingress/
• 0003-llm-decision-maker-behind-deterministic-gates.md · resources/docs/adr/
• …and 3 more

SKILL.md (first lines)

# Copilot Review — Smart (LLM-decided) watchdog

Webhook-first: a signed GitHub Delivery wakes one tick scoped to the affected
PR, and the fallback is an **Expectation** the tick arms (`next_check_at`) —
not a schedule. Cron is the alternate mode for Hosts that cannot expose a
public HTTPS endpoint. Design record:
`docs/adr/0005-webhook-first-expectation-fallback.md` (repo root).

Both modes drive open PRs toward a Copilot clean bill of health without
spamming: they ping `@copilot code review` only when a review is actually
useful, ask Copilot to resolve merge conflicts when needed, and tell the OWNER
when a PR is ready for human review — staying silent the rest of the time.

Triggers:

- "Stop spamming @copilot on that PR, it's already clean."
- "Watch PRs and tell ME when one is ready, not ping Copilot forever."
- "Set up the webhook PR watchdog (Listener) on this Host."
- Building a PR-monitor cron that should be quiet when there's nothing to do.

## Reference implementation

`pr_monitor.mjs` (next to this file) is the portable reference tick script:
plain ESM JavaScript, no build step, runs on Node >= 18 with an
authenticated `gh` CLI (see `docs/adr/0002` for the repo's script standard).
It is env-driven and **the repos to watch are passed as context, never
defaulted** — an installed skill must not silently watch someone else's
repos:

```bash
DRY_RUN=1 PR_MONITOR_REPOS="your-org/your-repo" node pr_monitor.mjs
```

`pr_monitor_webhook.mjs` is the Listener (webhook transport only, no decision
logic): plain ESM, `node:http`, no dependencies. Its implementation lives in
the `listener/` package next to this file (one module per concern — see
`listener/README.md`); the script itself is the CLI shim. It spawns ticks of
`pr_monitor.mjs` and arms their Expectations:

```bash
PR_MONITOR_REPOS="your-org/your-repo" \
PR_MONITOR_WEBHOOK_SECRET="$(openssl rand -hex 32)" \
PR_MONITOR_PUBLIC_URL="https://your-public-host" \
node pr_monitor_webhook.mjs --serve --setup-hooks
```

Other env: `PR_MONITOR_STATE_PATH` (default `~/.cache/pr-monitor/state.json`)
and `DRY_RUN=1` (print would-be comments, never post, never persist state).
The decision source — an explicit command, an explicit model+credential pair,
or the auto-discovered agent session, with its precedence, settings and launch
examples — is documented under
[Decision source](#decision-source-what-an-invoking-agent-defines-or-exports).

## Invocation API

Tick (shipped):

- **One-shot repo scope**: `node pr_monitor.mjs --repo owner/repo --json-report`
- **One-shot single-PR scope**: `node pr_monitor.mjs --pr owner/repo#123 --json-report`
- **Env equivalents**: `PR_MONITOR_REPOS`, `PR_MONITOR_PR`, `PR_MONITOR_REPORT=jsonl`
- **PR ref formats**: `owner/repo#123` or `https://github.com/owner/repo/pull/123`

Listener (webhook mode):

- `node pr_monitor_webhook.mjs --serve [--setup-hooks] [--keep-alive] [--tunnel ngrok|cloudflared]`
- `node pr_monitor_webhook.mjs --daemon` / `--stop` / `--status`
- `node pr_monitor_webhook.mjs --setup-hooks [--rotate-secret]` / `--list-hooks` / `--teardown`

Contract:

- One tick invocation = **exactly one tick** of the watchdog.
- Repo scope and single-PR scope are **mutually exclusive**; missing scope fails fast.
- `--json-report` / `PR_MONITOR_REPORT=jsonl` emits **one JSON line per PR** plus
  **one overall JSON line** at the end for machine-readable loop control. Each PR
  line carries `next_check_at` and `owner_notifications` for the Listener.
- `DRY_RUN=1` still prints would-be comments, but never posts to GitHub and never
  persists state.

Implementation status:

- [x] single-PR scope
- [x] agent-facing JSON tick report
- [x] transcript-based All-clear / Notify-ready
- [x] needs-human terminal reporting
- [x] fixture harness runner for one-shot and loop scenarios
- [x] webhook Listener (`pr_monitor_webhook.mjs`)
- [x] `next_check_at` + structured owner notifications in the tick report
- [x] Hook management, process management, Expectation persistence, tunnels, deploy recipes

Harness command:

```bash
npm run harness:copilot-review-smart -- all-clear-human
npm run harness:copilot-review-smart -- loop-single-pr-to-done --loop
```

Loop recipe (caller-owned; the script never sleeps):

1. Run one tick.
2. Read the final overall JSON line.
3. If `done: true`, tell the owner the scope reached either **All-clear** or
   **needs-human**, then stop.
4. Otherwise sleep the cadence (default **1 hour**, never below **15 minutes**)
   and run the next tick.

Terminal meanings in the JSON report:

- `terminal: "done"` → All-clear observed; Notify-ready path.
- `terminal: "needs-human"` → the loop cannot safely advance on its own for this
  head sha (for example conflict retries exhausted, or recurring LLM failure).
- `terminal: "skipped"` → draft/WIP PR skipped entirely and excluded from loop
  completion.

## Webhook mode (primary)

Full design record:
`docs/adr/0005-webhook-first-expectation-fallback.md` (repo root).

A delivery is a trigger, not a decision: it wakes one tick scoped to the

From dyegolara/skillbook, MIT.

More from dyegolara/skillbook