elevenlabs

Files (2)

• SKILL.md
• reference.md

SKILL.md (first lines)

# ElevenLabs Audio Generation

Requires `ELEVENLABS_API_KEY` in `.env`.

## Text-to-Speech

```python
from elevenlabs.client import ElevenLabs
from elevenlabs import save, VoiceSettings
import os

client = ElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY"))

audio = client.text_to_speech.convert(
    text="Welcome to my video!",
    voice_id="JBFqnCBsd6RMkjVDRZzb",
    model_id="eleven_multilingual_v2",
    voice_settings=VoiceSettings(
        stability=0.5,
        similarity_boost=0.75,
        style=0.5,
        speed=1.0
    )
)
save(audio, "voiceover.mp3")
```

### Models

| Model | Quality | SSML Support | Notes |
|-------|---------|--------------|-------|
| `eleven_multilingual_v2` | Highest consistency | None | Stable, production-ready, 29 languages. Toolkit default; supports style + speed |
| `eleven_v4` | Most expressive | None (audio tags, IPA) | Newest (released 2026-09-28), 90+ languages, best clone accuracy. Stability + similarity only, no style/speed |
| `eleven_flash_v2_5` | Good | `<break>`, `<phoneme>` | Fastest, cheapest; supports pause/pronunciation tags |
| `eleven_v3` | Expressive | None | Previous generation — prefer v4 |

**Choose:** multilingual_v2 for reliability and speed/style control, v4 for maximum expressiveness and cloned voices (test a few takes), flash for SSML control.

**Deprecated:** `eleven_turbo_v2_5` → use `eleven_flash_v2_5` (same quality, lower latency; see [deprecated models](https://elevenlabs.io/docs/overview/models#deprecated-models)). The toolkit still accepts `--model eleven_turbo_v2_5`: it prints a one-line warning and sends `eleven_flash_v2_5`.

### Eleven v4

Released 2026-09-28 ([changelog](https://elevenlabs.io/docs/changelog/2026/9/28), [model page](https://elevenlabs.io/docs/overview/capabilities/text-to-speech/eleven-v4#model-variants)). The toolkit calls v4 through the **Text to Dialogue** API with one input (the narrator voice), as ElevenLabs recommends for content creation and long-form audio:

```python
from elevenlabs import DialogueInput

audio = client.text_to_dialogue.convert(
    inputs=[DialogueInput(text="[warm] Welcome to my video!", voice_id="JBFqnCBsd6RMkjVDRZzb")],
    model_id="eleven_v4",
    settings={"stability": 0.5, "similarity": 0.75},  # the only two settings v4 takes
)
save(audio, "voiceover.mp3")
```

- `uv run tools/voiceover.py --model eleven_v4 ...` (and `redub.py --model eleven_v4`) do this for you, warn once that `--style`/`--speed` are ignored, and split text over ~2,000 chars per request (the reliable dialogue limit) at sentence boundaries. Pace with `--max-wpm` instead of `--speed`.
- `client.text_to_dialogue.convert_with_timestamps(...)` returns the same character alignment as TTS `convert_with_timestamps`, so `redub.py --sync` works with v4 and sends the same stability + similarity.
- Pass `settings` as a dict: SDK versions before 2.69 type it with a stability-only model. The API reads `similarity` there since 2026-09-28 (it rejects an out-of-range value with a 422). Text to Dialogue needs `elevenlabs>=2.21`.
- Direct delivery with audio tags (`[whispering]`, `[warm]`, `[sigh]`) and punctuation. Write tags that clearly describe the voice, because v4 can also read a tag as a sound effect.
- `eleven_v4_turbo` is the real-time variant, available only over the Text to Dialogue WebSocket. The toolkit doesn't use it, since offline renders don't benefit from low latency.

### Voice Settings by Style

| Style | stability | similarity | style | speed |
|-------|-----------|------------|-------|-------|
| Natural/professional | 0.75-0.85 | 0.9 | 0.0-0.1 | 1.0 |
| Conversational | 0.5-0.6 | 0.85 | 0.3-0.4 | 0.9-1.0 |
| Energetic/YouTuber | 0.3-0.5 | 0.75 | 0.5-0.7 | 1.0-1.1 |

### Pauses Between Sections

**With flash models:** Use SSML break tags inline:
```
...end of section. <break time="1.5s" /> Start of next...
```
Max 3 seconds per break. Excessive breaks can cause speed artifacts.

**With multilingual_v2 / v3 / v4:** No SSML support. Options:
- Paragraph breaks (blank lines) — creates ~0.3-0.5s natural pause
- v3/v4: audio tags such as `[long pause]`
- Post-process with ffmpeg: split audio and insert silence

**WARNING:** `...` (ellipsis) is NOT a reliable pause — it can be vocalized as a word/sound. Do not use ellipsis as a pause mechanism.

### Pronunciation Control

**Phonetic spelling (any model):** Write words as you want them pronounced:
- `Janus` → `Jan-us`
- `nginx` → `engine-x`
- Use dashes, capitals, apostrophes to guide pronunciation

**SSML phoneme tags (flash only):**
```
<phoneme alphabet="ipa" ph="ˈdʒeɪnəs">Janus</phoneme>
```

**IPA in v4:** no tags needed. Wrap the IPA in slashes and double quotes in place of the word:
```
The term "/ˌbaɪoʊˈkemɪstri/" refers to the study of chemical processes.
```

### Iterative Workflow

1. Generate → listen → identify pronunciation/pacing issues
2. Adjust: phonetic spellings, break tags, voice settings
3. Regenerate. If pauses aren't precise enough, add silence in post with ffmpeg rather than fighting the TTS engine.

## Speech to Text (Scribe)

```python
with open("voiceover.mp3", "rb") as f:
    result = client.speech_to_text.convert(file=f, model_id="scribe_v2", tag_audio_events=False)
words = [w for w in result.words if w.type == "word"]  # .text, .start, .end (seconds)
```

`scribe_v2` is current and the toolkit default. `scribe_v1` is deprecated, with removal announced for 2026-07-09 ([changelog](https://elevenlabs.io/docs/changelog/2026/6/8)), but still answers, so `--stt-model scribe_v1` (redub) and `--model scribe_v1` (align_captions) still run it, with a warning. `scribe_v1_experimental`, a 2025 preview of scribe_v1, is accepted with a warning and sent as `scribe_v2`. The toolkit uses Scribe in `redub.py` (transcribe + `--sync` word timing) and `align_captions.py`.

## Voice Cloning

From digitalsamba/claude-code-video-toolkit, MIT.

More from digitalsamba/claude-code-video-toolkit