elevenlabs-integration

Files (2)

• SKILL.md
• SKILL.md.lock

SKILL.md (first lines)

<!-- bodyRef: skills/sources/elevenlabs-integration.md -->

<!-- Generated by `harness skills bootstrap`. Edit to tune; re-run to regenerate. -->
<!-- Static skills point at live docs by design — do not hardwire version-pinned API here. -->

# ElevenLabs Integration

Integrate ElevenLabs **correctly and securely** for narration/TTS and webhook handling. In this project ElevenLabs powers the `media_generation_graph` `generate_narration` node: text → audio, stored in S3, run inside GitHub Actions. This skill is concept-first — when in doubt, read the live docs (links below) rather than trusting memory; the API surface and tier limits change.

## Pull current docs first

Fetch these before writing integration code (do not rely on stale snapshots):

- Authentication: https://elevenlabs.io/docs/api-reference/authentication
- Errors & status codes: https://elevenlabs.io/docs/eleven-api/resources/errors
- Webhooks (signature verification): https://elevenlabs.io/docs/eleven-api/resources/webhooks
- Post-call webhooks (Agents): https://elevenlabs.io/docs/agents-platform/workflows/post-call-webhooks
- API reference index: https://elevenlabs.io/docs/api-reference

Official SDKs (prefer these over hand-rolled HTTP — they bundle retries, signature helpers, streaming): Python `elevenlabs` (`pip install elevenlabs`), JS/TS `@elevenlabs/elevenlabs-js` (`npm i @elevenlabs/elevenlabs-js`).

## Auth & secrets

- Auth is a single header: **`xi-api-key: <KEY>`**. The key is a **secret** — never ship it to the browser, a v0/Next.js client component, a `NEXT_PUBLIC_*` var, Playwright page context, or committed config. Server-side / Action runtime only.
- This project (per spec) uses env-based secrets (GitHub Actions / Vercel env), with Secrets Manager deferred. Read the key from `process.env.ELEVENLABS_API_KEY` / `os.environ["ELEVENLABS_API_KEY"]` at call time; fail fast with a clear error if missing.
- Lock keys down at creation: ElevenLabs keys support **scope restriction** (limit endpoints), **credit quota** (cap spend), and **IP whitelisting** (Enterprise). Use a narrow, TTS-only scoped key for this engine.
- For any client-side audio need, do **not** expose the key — mint a **single-use token** server-side instead (see auth doc).

## TTS / narration usage

- Generate audio server-side in the media graph node; treat `voice_id`, `model_id`, and `output_format` as **config**, not hardcoded constants — surface them so they can change without a code edit. Look up current model IDs in the API reference rather than pasting one that may be retired.
- Stream when assembling video (lower latency, lower memory) but for ffmpeg assembly you usually want the full file written to disk/S3 first — pick per node.
- Persist generated audio to **S3 keyed by a content hash** of (text + voice + model + format). This is your idempotency + cache layer (see below).

## Idempotency (no native idempotency key)

The TTS API has **no idempotency-key header** — same request = new generation = more credits burned. You enforce idempotency yourself:

- Before calling, compute a deterministic key (hash of approved demo-script text + voice + model + format). If S3 already has that object, **reuse it** — skip the call. This makes graph retries free and deterministic.
- LangGraph nodes can re-run on resume/interrupt; never assume a node ran exactly once. Make `generate_narration` safe to re-enter.

## Webhook signature verification (if consuming ElevenLabs webhooks)

- Header: **`ElevenLabs-Signature`**, format `t=<timestamp>,v0=<hash>`. HMAC-SHA256 is computed over `timestamp + "." + raw_payload` using the **shared webhook secret** generated when the webhook is created.
- **Use the raw request body** for verification — never the re-serialized parsed JSON. In Next.js route handlers read the raw body (e.g. `await req.text()` / disable body parsing); in Express use `express.text()`/`express.raw()`. Any reparse/reformat breaks the HMAC.
- Prefer the SDK helpers over hand-rolled HMAC: Python `construct_event(raw_body, sig_header, secret)`, JS `constructEvent(payload, signature, secret)`. They verify the signature, enforce the **~30-minute timestamp tolerance** (replay protection), and return the parsed event — raising on failure.
- Verify **before** trusting/processing. Return 401 on bad signature, 200 quickly on success.
- Webhooks can be **redelivered** → dedup on the event/`request_id` before acting (idempotent handler).

## Error & retry semantics

- Errors return JSON with `type`, `code`, `message`, `request_id` (+ `param` on validation). Log `request_id` — it's what support needs.
- **429 is overloaded** — branch on the `code`, don't blanket-retry:
  - `rate_limit_exceeded` → **exponential backoff + jitter** (e.g. 1s, 2s, 4s … cap ~32s), then retry.
  - `concurrent_limit_exceeded` / `too_many_concurrent_requests` → you hit your **tier's concurrency cap** (Free 2, Starter 3, Creator 5, Pro 10, Scale/Business 15). Backoff won't help — **queue/throttle** outstanding requests instead. Critical: don't fan out N narration calls in parallel across LangGraph branches above this cap.
  - `system_busy` → transient overload; a plain retry usually succeeds.
- Retry only idempotent/safe operations; 4xx validation errors (bad voice/model/params) are **not** retryable — fix the input.

## Test / sandbox vs production

- ElevenLabs has no separate sandbox base URL — you isolate via **separate scoped, low-quota API keys** for dev/CI vs prod, plus the S3 cache so test runs don't re-burn credits.
- In CI/Playwright runs, mock or stub the ElevenLabs call by default (assert request shape) and gate real calls behind an explicit flag — keeps the demo-capture pipeline deterministic and credit-free.
- Verify behavior against a key with a tight **credit quota** so a runaway loop can't drain the subscription.

## Checklist

- [ ] Key read server-side from env only; never in client/`NEXT_PUBLIC_*`/Playwright context
- [ ] Key is scope-restricted (TTS) with a credit quota
- [ ] `voice_id`/`model_id`/`output_format` are config, model IDs confirmed against live API reference
- [ ] Audio cached/idempotent in S3 by content hash; `generate_narration` safe to re-enter
- [ ] Concurrent TTS calls bounded below the subscription tier's concurrency limit
- [ ] 429 handling branches on `code` (backoff vs queue vs retry); `request_id` logged
- [ ] (Webhooks) raw body preserved; signature verified via SDK helper before processing
- [ ] (Webhooks) timestamp tolerance enforced; handler dedups redeliveries
- [ ] Separate dev/prod keys; CI stubs ElevenLabs by default

## Gotchas

- **Raw-body trap in Next.js/Express:** the framework parses JSON by default, so `JSON.stringify(req.body)` ≠ the bytes ElevenLabs signed and verification silently fails. Must read the unparsed body (`req.text()` / `express.raw()` / disabled body parser).
- **429 means two different things** — blindly backing off on `concurrent_limit_exceeded` just keeps you at the cap. Branch on `code`: backoff for rate limits, queue for concurrency, plain retry for `system_busy`.
- **Parallel LangGraph fan-out silently exceeds tier concurrency:** generating narration for multiple features/branches at once trivially blows past Free(2)/Starter(3)/Creator(5). Add a semaphore/queue sized to the tier.
- **No native idempotency:** retried or replayed nodes re-bill credits. Without the S3 content-hash cache, LangGraph retries and interrupt/resume cycles quietly multiply spend.
- **Streaming vs file for ffmpeg:** a streamed response isn't a finished file — assembling video before the stream completes yields truncated/corrupt audio. Materialize fully (or to S3) before `assemble_video_ffmpeg`.
- **Key leakage via client components / v0:** it's easy to drop the key into a Vercel/v0 component or `NEXT_PUBLIC_` var; that publishes it. Keep all ElevenLabs calls in API routes / the Actions worker, or use single-use tokens.
- **Webhook 30-min tolerance + clock skew:** a CI runner or container with a drifted clock will fail timestamp validation even with a correct signature — ensure host time is synced.
- **Python vs JS helper arg names differ** (`construct_event(raw_body, sig_header, secret)` vs `constructEvent(payload, signature, secret)`) — copying the JS arg order into Python breaks verification.

<!-- sources fetched at generation: https://elevenlabs.io/docs/api-reference/authentication, https://elevenlabs.io/docs/eleven-api/resources/errors, https://elevenlabs.io/docs/eleven-api/resources/webhooks, https://elevenlabs.io/docs/agents-platform/workflows/post-call-webhooks, https://help.elevenlabs.io/hc/en-us/articles/19571824571921-API-Error-Code-429 -->

From ichiorca/shipsignal, Apache-2.0.

More from ichiorca/shipsignal