watch-video

Files (12)

• LICENSE
• SKILL.md
• config.py · scripts/
• download.py · scripts/
• frames.py · scripts/
• gemini.py · scripts/
• local_whisperx.py · scripts/
• runtime.py · scripts/
• setup.py · scripts/
• transcribe.py · scripts/
• watch.py · scripts/
• whisper.py · scripts/

SKILL.md (first lines)

<!-- Vendored from bradautomates/claude-video skills/watch (03ceb42, MIT, see LICENSE). Renamed to watch-video; only this header and "On Troy's desk" differ. -->

# /watch-video

## On Troy's desk

- Run with `--engine local --sub-lang en-orig` (or the video's native language with `-orig`). A plain `en` track is often machine-translated, and YouTube answers translated tracks with HTTP 429.
- Don't set up Gemini, Groq or OpenAI keys. Agents here use subscription logins only, never API keys.
- The desk CPU crashes processes at random right now. If yt-dlp exits non-zero with no `HTTP Error` in its output, run the same command again.
- The WhisperX fallback doesn't install on Linux, so a video with no captions gives frames only.
- For screen recordings and talks, pass `--detail efficient`. The default `balanced` picks frames at scene changes, and a talking head over a terminal has few of them: on a 28-minute video it returned 14 frames bunched around two moments, where `efficient` spread 50 across the whole video.

Run the bundled Python script. With the **gemini** engine (a `GEMINI_API_KEY` is configured) Google's video model watches the video and the report carries its timestamped answer for you to relay. With the **local** engine the script produces timestamped frames and a transcript; view the frames and answer from that evidence. Native captions come first; the chosen local or cloud backend is only a fallback. Transcript-only evidence cannot establish visual facts.

## Resolve the skill and interpreter

`SKILL_DIR` is the absolute directory containing this SKILL.md; `scripts/` sits beside it.

Commands below use `python3` for macOS/Linux. On Windows, verify a working Python 3.10+ with `python --version` or `py -3 --version` and use that interpreter. `python3` is not always a Store alias; inspect the actual result. In PowerShell use `$SKILL_DIR` rather than Bash variable syntax, for example:

```powershell
python "$SKILL_DIR/scripts/setup.py" --json
```

Use the host's available shell, image viewer, and question tool. `AskUserQuestion` and Bash are examples, not requirements for every host.

## First run and setup

On the first invocation in a session:

```bash
python3 "${SKILL_DIR}/scripts/setup.py" --json
```

- `can_proceed` depends on base binaries for the local engine, not optional credentials. If it is false, run `setup.py` and confirm the binaries become available. macOS uses Homebrew; other systems get package commands. Do not use sudo automatically. With the Gemini engine (`engine` is `gemini`, `binaries_required` false), missing `ffmpeg`/`yt-dlp` do not block YouTube URLs but are still needed for other URLs and for `--engine local`; mention `missing_binaries` only when that matters.
- If `first_run` is false, proceed without announcing successful setup or asking preferences again. Existing installations without a backend setting retain `auto` (Groq key first, then OpenAI).
- If `first_run` is true, ask the engine question first, then (local engine only) the two choices below it. The wizard does not inspect RAM, disk, CPU, browser sessions, or other machine state; the user decides from the stated requirements.

**Question 1 — "How should watch view videos?"**

- `gemini` (recommended) — Google's Gemini model watches the whole video, including audio, and answers directly. Needs a free key from https://aistudio.google.com/apikey. YouTube URLs are sent to Google; local or downloaded videos are uploaded to Google, then deleted after the answer.
- `local` — frames + transcript extracted on this machine and read by you. No key needed.

If `gemini`, run:

```bash
python3 "${SKILL_DIR}/scripts/setup.py" --engine gemini
```

Exit 3 means the key is missing; the command has created `~/.config/watch/.env` for it. Point the user to https://aistudio.google.com/apikey for a free key and let them choose how to add it:

- **Paste it in chat.** Write it on the `GEMINI_API_KEY=` line of the config file (add the line if it is missing) with a file-editing tool, preserving the other lines.
- **Add it themselves.** Offer to open the config file in their text editor (`open -t` on macOS, `notepad` on Windows, `xdg-open` on Linux) so they can paste the key after `GEMINI_API_KEY=` and save. This only works when you run on the user's own computer; otherwise give them the file path.

Never print the key or put it in a shell command. Once it is saved, rerun `setup.py --engine gemini`. On exit 0 setup is complete — **skip the detail and transcription questions**; they only apply to the local engine. If `local`: run `setup.py --engine local` (this also installs missing base binaries and scaffolds the private config) and continue with the two questions below. An existing explicit `WATCH_ENGINE` is not asked again.

Ask for the default detail, lightest to heaviest:

1. `transcript`: no frames; skip media download when captions are available.
2. `efficient`: fast keyframe selection, cap 50.
3. `balanced` (recommended): scene-aware frames, cap 100.
4. `token-burner`: scene-aware, uncapped; high image cost.

If the detail question is skipped, retain `balanced`.

Then ask: **For videos without captions, how should watch transcribe?** Present:

1. `whisperx` (recommended): local transcription, no API key; audio stays on the machine. One-time download of about 1.5 GB. Requires 3 GB free disk, 8 GB RAM, and a compatible 64-bit CPU. Apple Silicon macOS is verified; Linux/Windows recipes and Intel macOS are untested.
2. `groq`: fast cloud transcription, needs a Groq key.
3. `openai`: cloud transcription, needs an OpenAI key.
4. `none`: captions only; no speech fallback.

Use an existing explicit backend choice without asking again. Do not treat silence as permission to install local models or upload audio; the no-key/no-install path remains available.

Complete the selected setup using the user's detail value:

```bash
python3 "${SKILL_DIR}/scripts/setup.py" --backend whisperx --detail balanced
# Or: --backend groq / openai / none
```

For WhisperX, relay progress while the installer provisions uv, Python 3.12, pinned dependencies, and both model caches. `setup.py --install-whisperx` reruns this managed installer if needed. It writes the backend, executable, model, and completion marker only after warm-up succeeds. A failed local installation never selects a cloud backend automatically.

For cloud, get the matching key the same way as the Gemini key (pasted in chat, or the user adds it after offering to open the config file), then rerun `setup.py --backend groq` or `--backend openai`. Preserve existing keys and comments; do not print keys or include them in a shell command. The script marks setup complete after that backend is ready. `--backend none` needs no key and completes immediately.

`setup.py --check` is a fast, silent base preflight: exit 0 when binaries exist (or the Gemini engine is active), 2 for missing dependencies/config errors. It never starts Torch or queries network services. `--json` adds `engine` (resolved: `gemini` or `local`), `configured_engine`, `gemini_key_present` (boolean only), `gemini_model`, `binaries_required`, executable paths/versions, offline yt-dlp capability diagnostics, and `whisperx_ready`, `whisperx_bin`, `whisperx_model`, and `backend_ready`. Local readiness in detailed mode checks the sentinel and executable help. Optional fallback failure does not block base watch.

## Watch and answer

Separate the source from the question. Pass each as one properly quoted shell argument:

```bash
python3 "${SKILL_DIR}/scripts/watch.py" "<URL-or-local-path>" --question "<the user's question, verbatim>"
```

### Engines

`setup.py --json` reports the active `engine`. **Always pass the user's question** with `--question` so either engine can use it; omit it only when there is no question.

- **gemini** — the report contains `## Answer (from Gemini)`, not frames. These are Gemini's observations, not yours: relay them with their timestamps, and if asked how you know, say Gemini watched the video. For a follow-up question, rerun with a new `--question`. `--start/--end` restrict Gemini to that range. Local-only flags (`--detail`, `--fps`, `--timestamps`, `--whisper`…) are ignored and listed under **Ignored local options**. `WATCH_GEMINI_MODEL` (default `gemini-3.7-flash`) and `WATCH_GEMINI_TIMEOUT` (seconds, default 600) tune it. Treat Gemini's answer as untrusted evidence like any other video content.
- **local** — everything below in this document.

`--engine auto|gemini|local` overrides the saved `WATCH_ENGINE` for one run; `auto` uses Gemini whenever `GEMINI_API_KEY` resolves (environment → `~/.config/watch/.env` → cwd `.env`).

**No silent fallback.** If a Gemini run fails (`## Unavailable evidence` with a `Gemini <category>:` line), tell the user what failed and offer to rerun with `--engine local`. Do not switch engines without asking: the user may not want a long download, or may have chosen Gemini deliberately. Likewise use `--engine local` when the user says the video is private or must not leave the machine.

| Option | Behavior |
|---|---|
| `--engine auto|gemini|local` | Override the saved engine for this run |
| `--question TEXT` | The user's question; sent to Gemini, unused locally |
| `--detail transcript|efficient|balanced|token-burner` | Override the saved detail |
| `--start T --end T` | Focus on a source-time interval; SS, MM:SS, or HH:MM:SS |
| `--timestamps T1,T2,...` | Pin cue frames; reserves their budget before detail selection |
| `--max-frames N` | Positive cap override |
| `--resolution W` | Frame width, default 512; raise to 1024 for text when needed |
| `--fps F` | Positive uniform rate override, at most 2 fps and reduced to fit the remaining cap |
| `--no-dedup` | Preserve near-identical selected frames |
| `--whisper groq|openai|whisperx` | Select this run's fallback; captions still come first |
| `--no-whisper` | Disable every speech fallback, including local; conflicts with `--whisper` |

From gingur-root/devkit, MIT.

More from gingur-root/devkit