cpu-reduction

Files (2)

• SKILL.md
• clean-worktree-targets.py · scripts/

SKILL.md (first lines)

# CPU / disk reduction — clear build & test residue

The maintainer's Mac is 10 CPUs / 64 GiB. Three residue families need different treatment — diagnose by symptom before sweeping.

| Family | Symptom | Cost | Cause |
| --- | --- | --- | --- |
| **Old Rust builds** | a build hangs on `Blocking waiting for file lock`; a few `rustc`/`cargo`/`lld` pegged; disk filling | LOCK contention + CPU + tens of GB of `target/` | detached/orphaned builds that outlived their launcher; abandoned worktree `target/` dirs |
| **Orphaned node tests** | load floor sits ~20 with nothing building | CPU + swap | `node`/`tsx`/`esbuild`/`tsc -w` reparented to launchd when their harness died |
| **Orphaned load generators** | load in the **hundreds**; N identical `/bin/zsh` at 25–70%, consecutive PIDs, same start second | CPU — starves every build | a probe spawned `(while :; do :; done) &` and died before its cleanup line ran |

**A high load FLOOR with nothing building is almost never active compilation — it is orphaned node processes** (§3). A *hung build* or *filling disk* is the Rust family (§2).

> Prevention beats cleanup: `rust-build-hygiene` is how to spin builds up so they die with you.

## 1. Measure

```sh
uptime                                   # load; the 1-min figure LAGS — see §7
sysctl -n hw.ncpu vm.swapusage
memory_pressure | grep -i "free percentage"
df -h ~/.cache                            # disk — worktree target/ dirs live here
ps -Ao pid,ppid,pgid,%cpu,%mem,rss,etime,state,comm -r | head -40
```

Load well above `hw.ncpu` with nothing building → §3. A build stuck on a lock, or `~/.cache` near full → §2. Start with `make build-status`: it prints which builds hold the machine-wide compile slots, who is queued behind them and for how long, token occupancy, and any build running outside the cap — a build sitting at `Compiling` with no CPU is queued behind the governor, not hung.

## 2. Old Rust builds — locks, orphans, disk

### 2a. Orphaned builds holding a target lock (the hang)

A build that outlived its launcher still holds a target-dir lock, so a live build blocks behind it — often 30+ minutes, reading as "the build is just slow."

```sh
# builds still running, with age + which target dir
ps -Ao pid,ppid,%cpu,etime,command | grep -Ei 'rustc|cargo|ld64|lld|cc1' | grep -v grep
# is a build blocked on a lock right now? (check the build log)
grep -l 'Blocking waiting for file lock on' /tmp/*build*.log 2>/dev/null
pkill -f '<target-dir-path>'              # kill the orphaned builds for that target
```

Cause is almost always TWO builds on ONE target dir — most often a stopped agent's **detached** cargo build that `TaskStop` did not reap (TaskStop kills the agent, not its background bash jobs; setsid/nohup-detached builds reparent to PID 1). Artifacts persist and stay warm — hand the contention-free target to ONE fresh foreground build. Never point two concurrently-building trees at one target dir.

### 2b. Worktree-owned Rust targets eating disk

```sh
# Audit only. This is the default.
python3 .claude/skills/cpu-reduction/scripts/clean-worktree-targets.py

# Repeat every safety check, then delete eligible build output.
python3 .claude/skills/cpu-reduction/scripts/clean-worktree-targets.py --apply
```

It deletes no worktree, branch, source file, or shared cache. Layouts it covers:

```text
~/.cache/nub/worktrees/<slug>/target/
~/.cache/nub/worktrees/<slug>/aube-target/
~/.cache/nub/worktrees/<slug>/target-linux/
~/.cache/nub/worktrees/<slug>-target/
~/.cache/nub/worktrees/<slug>-launcher-target/
~/.cache/nub/worktrees/<slug>-target-native/
```

Spellings vary; the invariant is a top-level target-named directory inside a registered worktree, or a target-named sibling prefixed by that worktree's full basename.

Safety model: the cleaner protects an owner's entire target set when `git status --porcelain` reports any staged, unstaged, or untracked work — an uncommitted worktree is active state. It refuses symlinks, unmatched directories, targets owning the installed `nub-dev`/`nubx-dev` binary, and apply mode while any Rust build is running. Status and process state are rechecked at the deletion boundary.

**Do not remove clean worktree checkouts merely to recover disk.** Build output is regenerable; a checkout and branch are not interchangeable with cache. Remove a checkout only via the `worktree` skill after proving it abandoned and pushed.

**Shared targets stay warm** — they seed fresh worktrees and may own the installed `nub-dev` binary. Never include them in a worktree cleanup; prune only with explicit operator approval:

```text
~/.cache/nub/shared-target
~/.cache/nub/shared-target-<content-hash>
```

**Judge every reclaim by `df`, never `du`.** APFS clone/shared-block accounting means summed `du` sizes need not equal the `df` delta, and `du -sh` over the worktree root can take minutes.

```sh
df -h /System/Volumes/Data
```

### 2c. Orphaned `rustc` at high CPU

Rare — Rust builds are noisy but they finish. If `ps` shows a long-`etime` `rustc`/`cc1plus`/`lld` at high %cpu with no parent cargo, it's an orphan; `kill -9` after confirming its `ppid` chain has no live build.

### 2d. Orphaned synthetic load generators (load in the hundreds)

A probe that needs contended measurement sometimes manufactures the contention and then dies before cleanup, leaving busy-loops spinning forever under launchd.

Signature — **all four together**, never `/bin/zsh` alone:

```sh
ps -Ao pid=,ppid=,%cpu=,etime=,comm= | awk '$2==1 && $3>20 && $5=="/bin/zsh"'
ps -o command= -p <pid>          # full argv reveals the `while :; do :; done` and its dead cleanup
pgrep -P <pid>                   # NO children — it is not wrapping a real build
```

Consecutive PIDs + identical start second + `PPID=1` + no children + a spinning loop in the argv = safe to `kill -9`. **Read the full argv first** — an agent's own background `cargo` also runs as `/bin/zsh -c source <snapshot> && …`, and that one *has* children and must be left alone.

**`pgrep -f` can return 0 for everything** when it cannot read other processes' argv under the sandbox. It reports absence, not an error — cross-check with `ps aux | grep` before concluding "nothing is running."

Prevention, in the probe itself:

```sh
LOADPIDS=$(jobs -p)
trap 'kill $LOADPIDS 2>/dev/null' EXIT INT TERM
```

## 3. Orphaned node test processes (the load floor)

Orphans are `PPID=1`:

```sh
# every orphaned dev process, with what it costs
ps -Ao pid=,ppid=,%cpu=,rss=,etime=,command= | awk '$2==1' \
  | grep -Ei 'nvm/versions/node|/bin/tsx|esbuild --service|cache/nub/worktrees|tsc -b -w'
# what it totals
ps -Ao ppid=,user=,rss=,command= | awk '$1==1 && $2=="colinmcd94"' \

From nubjs/nub, MIT.

More from nubjs/nub