ava

Files (1)

• SKILL.md

SKILL.md (first lines)

# Ava

Natural voice conversations with AI coding assistants using ElevenLabs text-to-speech (TTS) and speech-to-text (STT).

## The Jarvis Goal

Ava aims to create a Jarvis-like voice assistant experience. The AI speaks to you and listens, like a real conversation. In voice-primary mode, substantive responses must go through `converse`; if voice fails, stop and restore MCP instead of continuing chat-only.

## Setup

### 1. Configure MCP Server

Ava runs as an HTTP server on port 8765. Add this MCP configuration to Cursor, Claude Code, Factory, or another MCP-capable host:

```json
{
  "mcpServers": {
    "ava": {
      "type": "http",
      "url": "http://127.0.0.1:8765/mcp"
    }
  }
}
```

### 2. Configure ElevenLabs

Set your ElevenLabs API key:

```bash
# In ~/.ava/ava.env
ELEVENLABS_API_KEY=your-key-here
```

ElevenLabs provides:
- **TTS**: `eleven_v3` model with Donna voice (cloned)
- **STT**: Scribe v2 Realtime (streaming WebSocket with manual commit mode)

## Usage

Use the `converse` MCP tool. Trust server defaults unless changing behavior for the current turn:

```python
# Speak and listen — trust server defaults
converse(message="Hello! What would you like to work on?", wait_for_conch=true)

# Speak without waiting (narration while working)
converse(message="Searching the codebase now...", wait_for_response=false, wait_for_conch=true)

# User wants to say something long without silence cutoff
converse(message="Go ahead, I'm listening.", disable_silence_detection=true, wait_for_conch=true)
```

### Parameter rules (read before every call)

The server defaults are tuned and locked. **Do not pass tuning parameters** — passing them yourself is the #1 cause of inconsistent behavior across agents. The vast majority of calls are exactly:

```python
converse(message="...")
```

Only three parameters are ever situational: `wait_for_response`, `disable_silence_detection`, and (Claude Desktop only) `skip_tts`. Everything else is a server default you must NOT send.

| Parameter | Default | Pass it? | When to pass |
|-----------|---------|----------|--------------|
| `message` | required | **always** | The text to speak (use `""` only with `skip_tts=true`) |
| `wait_for_response` | `true` | **only to disable** | Pass `false` for a fire-and-forget announcement/narration where you will NOT listen |
| `disable_silence_detection` | `false` | **only to enable** | Pass `true` when the user explicitly needs to talk at length with long pauses (dictation, reading aloud) |
| `wait_for_conch` | `true` | **never** (already default) | Leave as default; it auto-queues behind another speaker |
| `speed` | `1.2` | **never** unless user asks | Only if the user says "talk faster/slower" mid-session. Range 0.7–1.2; eleven_v3 rejects >1.2 |
| `vad_aggressiveness` | `2` | **never** unless troubleshooting | Server-tuned. Only change if the user reports background bleed (raise) or being cut off (lower) |
| `listen_duration_min` | `1` | **never** | Server-tuned |
| `listen_duration_max` | `600` | **never** | Server-tuned (10 min) |
| `timeout` | `900` | **never** | Server-tuned (15 min) |
| `metrics_level` | `summary` | **never** | Server-tuned |

**Hard rule:** if you find yourself passing `speed`, `vad_aggressiveness`, `listen_duration_*`, `timeout`, or `metrics_level`, stop — you almost certainly should not be. The user tunes those via `~/.ava/ava.env`, not per-call.

## Best Practices

1. **Voice-primary communication** -- substantive responses go through `converse`; if voice fails, stop and restore MCP instead of continuing chat-only
2. **Trust server defaults** -- speed defaults to 1.2 (the eleven_v3 max); pass optional parameters only when changing behavior
3. **Narrate without waiting rarely** -- Use `wait_for_response=false` only for short acknowledgements before work
4. **One question at a time** -- Don't bundle multiple questions
5. **Parallel calls** -- Combine a rare short `converse(..., wait_for_response=false)` acknowledgement with other tools in one turn for zero dead air
6. **Long input** -- Set `disable_silence_detection=true` when the user needs to speak at length; server defaults already allow a 10-minute listen window
7. **Long spoken output** -- A long single-line paragraph is valid in one `converse` call; normalize literal newlines out of `message`, but do not shorten or split substantive content merely to work around reliability issues

## Parallel Tool Calls (Zero Dead Air)

When performing actions during a voice conversation, use parallel tool calls to eliminate dead air:

```python
# FAST: One turn -- voice and action fire simultaneously
converse("Checking that now.", wait_for_response=false, wait_for_conch=true)
bash("git status")

# Then speak the results
converse("Here's what I found: ...", wait_for_response=true, wait_for_conch=true)
```

## Configuration

Config file: `~/.ava/ava.env`

### ElevenLabs Settings

| Variable | Default | Description |
|----------|---------|-------------|
| `ELEVENLABS_API_KEY` | (none) | API key -- required |
| `AVA_ELEVENLABS_TTS_MODEL` | `eleven_v3` | TTS model |
| `AVA_ELEVENLABS_TTS_VOICE` | `k4hP4cQadSZQc0Oar2Ld` | Voice ID (Donna) |
| `AVA_ELEVENLABS_STT_MODEL` | `scribe_v2_realtime` | STT model |
| `AVA_ELEVENLABS_REALTIME_STT` | `true` | Use realtime streaming STT |
| `AVA_SILENCE_THRESHOLD_MS` | `2000` | Silence threshold in ms (2.0s default) |
| `AVA_VAD_AGGRESSIVENESS` | `2` | VAD strictness (0-3); higher rejects more background audio |

## Architecture

From harshav167/ava, MIT.

More from harshav167/ava