live-verify

Files (1)

• SKILL.md

SKILL.md (first lines)

# `/gflow:live-verify` — Live-verification enforcement

gflow-cli reverse-engineers a blackbox: Google Flow. Offline checks (`ruff`, `pyright`,
unit/BDD tests) verify gflow-cli's own code does what it's supposed to; they cannot verify
Flow still behaves the way it was captured, because Flow is external and changes without
notice (see [#174](https://github.com/ffroliva/gflow-cli/issues/174)). This gate enforces two
things, both evidence-based (no claim without a fresh verification artifact — see the
`superpowers:verification-before-completion` skill):

1. **Part 1 — Pre-flight**, at the *start* of feature/fix work: confirm the checkout reflects
   current `develop` before investing effort.
2. **Part 2 — Live-verify**, *before claiming done* (after `/code-review` and
   `/ponytail:ponytail-review`, before commit/PR): exercise the change against real Flow.

Full design rationale:
[`docs/superpowers/specs/2026-07-19-live-verify-design.md`](../../docs/superpowers/specs/2026-07-19-live-verify-design.md).

---

## Part 1 — Pre-flight

Run before writing any code for a new feature/fix:

```bash
git fetch origin
git rev-parse --abbrev-ref HEAD
git rev-list --count HEAD..origin/develop
git log --oneline -5
```

`git rev-list --count HEAD..origin/develop` is an asymmetric DAG set-difference — it counts
commits `develop` has that the current `HEAD` lacks, regardless of `HEAD`'s own private
unmerged history. This is what catches a genuinely diverged stale branch, not just a
behind-by-fast-forward one.

- **If the count is nonzero:** stop. `git pull` (on `develop`) or rebase/merge (on a feature
  branch) before continuing.
- **If the branch's last real commit looks old relative to recent `develop` activity** (a
  smell for stale WIP — e.g. missing a function/guard that `develop` already has): stop and
  surface it. Don't silently proceed, don't silently switch — name the divergence and ask the
  user how to proceed.
- **A separate sibling checkout is not itself a red flag** — this project's workflow
  routinely uses sibling checkouts for isolated feature branches, and a real feature branch
  is *supposed* to differ from `develop`. The actual signal is "differs from `develop` in a
  way that suggests staleness" (missing something `develop` has), not "differs by adding new
  work on top of it." When in doubt, diff the specific file(s) about to be touched against
  `origin/develop`'s version before assuming they match:

```bash
git diff origin/develop -- <path/to/file>
```

## Part 2 — Live-verify

"Live" means: **drive the real generation commands** (`t2i`, `i2i`, `i2v`, and siblings like
`t2v`/`r2v` where applicable) against a real authenticated Flow account, covering **multiple
variations** of the change — not one happy-path call. A change touching a generation code
path is default-in-scope; skipping this gate requires a named reason, not silence.

**1. Define the live matrix.** Before running anything, name explicitly:
   - Which command(s) does the change touch?
   - Which variations actually exercise it? (E.g. for a mention-resolution fix: a character
     mention, a media mention, an ambiguous-name case, an unresolvable name.)
   - **Which surface(s)?** A capability that ships as both a CLI command and an MCP tool has
     **two live paths, not one** — and the MCP queued path runs different code
     (`mcp/tools.py` → queue payload → `worker/codec.py` → request → daemon owns download and
     recording). A CLI run does not exercise it. Decide per change whether the MCP path needs
     its own live run, and if you skip it, say so with a reason. The cheap version is usually
     enough: one queued MCP call on a credit-free operation proves the payload keys round-trip,
     which is the hop that silently no-ops (#495). Offline parity tests cannot see this —
     they never build a real payload and decode it.

**2. Check the cost tier for each variation — the tier follows the operation, not the
   command family:**
   - Bare entity CRUD with no image generation (`create_entity`, `list_characters`,
     `patch_entity` at the API level; `t2i`/`i2i` themselves) are **credit-free** — run as
     needed to cover the matrix without a separate confirm each time. Still mind WAF/volume
     discipline: don't fire dozens of live calls back-to-back without surfacing it to the
     user first.
   - **Anything that generates real media is costed**, even on an otherwise-free command
     family — e.g. `gflow character create --face-prompt` generates real face/body images
     and drives real image generation despite being "character CRUD" in name (zero credits, daily-capped); `i2v` and other
     video-generation paths are always costed. Always get explicit operator go-ahead before
     running a costed variation. Batch the ask: name what will run and why, once, not one
     confirm per call.

**3. Run each variation, capture evidence per run.** Use the release skill's actual 5-layer
   ledger shape (`skills/release/SKILL.md` §4b), adapted per-field to what the change under
   test produces:

   | Layer | Evidence |
   |---|---|
   | File count | New file(s)/DB row(s) produced by the run |
   | Magic bytes / field value | The specific artifact or field the change is supposed to affect (e.g. a real image's magic bytes, or de-tagged prompt text in the catalog) |
   | Dimensions/shape | For media output: actual dimensions match the requested aspect ratio/model |
   | Structlog invariants | The expected log event fired (e.g. `mention_resolved`, not `mention_unresolved`) |
   | User-confirmable artifact | Real output a human could open and check (image/video file, `size > 1024` bytes) |

   Write this to a lightweight per-feature evidence note at `tmp/live-verify/<feature-slug>.md`
   (gitignored — this is not the full `docs/LIVE_VERIFICATION_vX.Y.Z.md` release ceremony).
   Fold it into the real `LIVE_VERIFICATION` doc when the feature ships in a release.

**4. On pass:** all matrix variations green — proceed to commit.

**5. On fail:** go to Failure-routing below.

## Failure-routing (reproducibility re-test)

**1. Check for a known match first** (costed failures only, to avoid an unnecessary
   re-spend): grep `KNOWN_ISSUES.md` and open GitHub issues for a matching error signature.

```bash
gh issue list --repo ffroliva/gflow-cli --search "<error text>" --state all
```

**2. If no match, re-test once.** Free for `t2i`/`i2i` — just re-run. For a costed failure,
   re-testing needs the same operator confirm as any costed run.

**3. Compare outcomes and route:**

From ffroliva/gflow-cli, MIT.

More from ffroliva/gflow-cli