refresh-index

Files (8)

• LICENSE.txt
• SKILL.md
• evals.json · evals/
• classify.md · references/
• commit-and-pr.md · references/
• edge-cases.md · references/
• snapshot.md · references/
• summary-template.md · references/

SKILL.md (first lines)

# Refresh Index

Re-ingest every enabled repository in `data/skill-index-resources.json` so `data/skill-index/{owner}_{repo}.json` matches upstream. Classify each repo as **updated / unchanged / failed / skipped**, verify the catalog rebuilds, then open one data-only PR after explicit confirmation.

This is the inverse of `skill-index-updater` (that skill **adds** repos). Keep SKILL.md as the spine and load step detail from `references/` so the agent's context budget stays small.

## When to Use

- User asks to "refresh the index", "update the indexed skills", "sync the catalog", "re-ingest all repos", or "batch-maintain the skill index"
- A scheduled refresh is due, or a release needs current upstream skill metadata

Do **not** trigger for: adding a new repository (`skill-index-updater`), authoring or improving a single skill (`skill-creator`), opening an upstream PR (`skill-upstream-pr`), or installing/updating skills on the local machine (`asm install`, `asm update`).

## Repo Sync Before Edits (mandatory)

Before re-ingesting anything, pull the latest remote branch:

```bash
branch="$(git rev-parse --abbrev-ref HEAD)"
dirty=0
if [ -n "$(git status --porcelain)" ]; then
  git stash push -u -m "pre-refresh-index: ${branch}"
  dirty=1
fi
git fetch origin
git pull --rebase origin "$branch"
if [ "$dirty" -eq 1 ]; then
  git stash pop || {
    echo "✗ Stash pop failed — recover with: git stash list && git stash show -p stash@{0}"
    exit 1
  }
fi
```

If `origin` is missing or rebase conflicts occur, **stop and ask the user** before continuing. Never silently overwrite local edits to `data/skill-index/` or `data/skill-index-resources.json`.

## Prerequisites

Verify each before any ingest. Stop and tell the user if any fails.

- `node` on PATH (`command -v node`) — Node >= 22 per `package.json` engines
- `npm` on PATH (`command -v npm`) — required by `npm run preindex`; the ingester evaluates skills in-process, so no `asm` binary is needed
- `gh` on PATH and authenticated (`gh auth status`) — required for PR creation
- `git` on PATH and inside the ASM repo working tree (`git rev-parse --show-toplevel`)
- Network access to `github.com` (each ingest clones the upstream repo)

## Pipeline

Follow these steps in order. Each step has a verification check — do not proceed if the check fails. Long scripts and templates live in `references/`; read the linked file when you reach that step.

Many agent hosts start each command in a fresh shell. Start every code block with `ROOT="$(git rev-parse --show-toplevel)"; RES="$ROOT/data/skill-index-resources.json"` instead of relying on variables from an earlier step.

### Step 1: Enumerate the index

Read `data/skill-index-resources.json` and split entries, keyed on the `source` string (`github:owner/repo`) because that is the identifier `preindex` echoes:

```bash
ROOT="$(git rev-parse --show-toplevel)"
RES="$ROOT/data/skill-index-resources.json"
jq -r '.repos[] | select(.enabled == true)  | .source' "$RES"
jq -r '.repos[] | select(.enabled == false) | .source' "$RES"
jq -r '.repos[] | select(.enabled == true) | "\(.source)\t\(.owner)_\(.repo)"' "$RES"
jq empty "$ROOT"/data/skill-index/*.json   # exits non-zero if any file is invalid
git -C "$ROOT" status --porcelain -- website/ data/
```

- `enabled[]` — `"enabled": true`. These will be refreshed.
- `disabled[]` — `"enabled": false`. These land in **skipped** with reason `"disabled in skill-index-resources.json"`. The list may be empty; then `W = 0`.

Verification: `enabled[]` is non-empty, every per-repo JSON parses, and `git status --porcelain -- website/ data/` prints nothing. If any check fails, stop. A clean `website/` is what lets Step 5 restore the tracked files `build-catalog` rewrites.

### Step 2: Snapshot pre-run skill counts

Record current `skillCount` for each enabled repo so Step 7 can report deltas. Run the loop in `references/snapshot.md`. Missing index file → pre-count `0`.

### Step 3: Run `npm run preindex`

Re-ingest every enabled repo. Capture stdout and the exit code — `preindex` exits 1 if any repo fails, but **do not abort**. Partial results still classify.

`preindex` writes each ingest to `getIndexDir()` before copying it into `data/skill-index/`. Without an override that is the user's real `~/.config/agent-skill-manager/skill-index/`, a change no git diff shows. Always set `ASM_CONFIG_DIR` to a scratch directory:

```bash
ROOT="$(git rev-parse --show-toplevel)"
mkdir -p /tmp/refresh-index/asm-config
LOG="/tmp/refresh-index/preindex.log"
cd "$ROOT"
ASM_CONFIG_DIR=/tmp/refresh-index/asm-config npm run preindex > "$LOG" 2>&1
PREINDEX_EXIT=$?
cat "$LOG"
```

`preindex` clones every enabled repo in sequence and can run for many minutes. If the host's command timeout is shorter, run the command in the background and wait until the log ends with the `Done: N succeeded, M failed` line; then read the exit code.

Verification: the log exists and contains one line per enabled repo: `  {source} ... {N} skills` or `  {source} ... FAILED: {error}`. If the log is empty or no line matches, stop — that is environmental, not a per-repo failure.

### Step 4: Classify each repo

Match each `{source}` log line to `data/skill-index/{owner}_{repo}.json` in `git status --porcelain` (which also lists new, untracked index files). Every ingest rewrites timestamps, so "changed" means changed after the timestamps are stripped. Read `references/classify.md` for the comparison and the four-row signal table. Capture post-run `skillCount`; failed repos keep the pre-count.

### Step 5: Rebuild the website catalog (verification only)

Confirm the refreshed index is structurally valid. **`website/catalog.json` is gitignored — never stage it.**

```bash
cd "$ROOT"
npx tsx scripts/build-catalog.ts
jq empty website/catalog.json
git restore -- website/ data/repo-stars.json
```

Verification: the first two commands exit 0. If either fails, stop — the data files are internally inconsistent and the PR must not land. `build-catalog` also rewrites tracked outputs (`website/*-stats.json`, `website/robots.txt`, and `data/repo-stars.json` on any networked run) with fresh values; `git restore -- website/ data/repo-stars.json` reverts them. That is safe only because Step 1 confirmed `website/` and `data/` were clean. It does not touch the gitignored `website/catalog.json`.

### Step 6: Detect unexpected diff scope

Confirm the only files that changed are under `data/skill-index/` (and `data/skill-index-resources.json` only if the user explicitly bumped `updatedAt`):

```bash
UNEXPECTED=$(git status --porcelain | cut -c4- \
  | grep -v -E '^data/skill-index/' \

From luongnv89/asm, MIT.

More from luongnv89/asm