hermes-fallback-providers

Files (1)

• SKILL.md

SKILL.md (first lines)

# Hermes Fallback Providers

Configure primary model fallback chains for Hermes Agent. When the main LLM provider hits rate limits (429), server errors (5xx), auth failures (401/403), or connection drops, Hermes can auto-switch to a backup provider:model pair mid-turn.

## When to Use

- Your primary model (e.g. DeepSeek via `custom` provider) occasionally hits rate limits
- You want a free-tier model as safety net (e.g. Gemini 2.5 Flash-Lite)
- You need multi-model resilience without interrupting conversations

## How Fallback Works

- **Turn-scoped**: each user message starts with the primary model. If it fails, fallback activates for **that turn only**. Next turn tries primary again.
- **Single fallback per turn**: only the first entry in the chain is tried if primary fails. If the fallback also fails, normal error handling takes over (retries, then error message).
- **Conversation preserved**: history, tool calls, and context survive the switch.

## Configuration

### Via CLI (recommended)

```bash
# Interactive picker — same UI as `hermes model`
hermes fallback add

# View chain
hermes fallback list

# Remove an entry
hermes fallback remove

# Clear all
hermes fallback clear
```

### Via Direct Config Editing

Edit `~/.hermes/config.yaml` (or `$HERMES_HOME/config.yaml`):

```yaml
fallback_providers:
  - provider: gemini
    model: gemini-2.5-flash-lite
    base_url: https://generativelanguage.googleapis.com/v1beta
  - provider: gemini
    model: gemini-2.0-flash-lite
    base_url: https://generativelanguage.googleapis.com/v1beta
```

Each entry requires `provider` and `model`. Some provider types also need `base_url` and/or `key_env`.

### For Custom OpenAI-Compatible Endpoints

```yaml
fallback_providers:
  - provider: custom
    model: my-local-model
    base_url: http://localhost:8000/v1
    key_env: MY_LOCAL_KEY    # env var name for the API key
```

## Setting Up Gemini as Fallback

### 1. API Key

Set one of these in `~/.hermes/.env`:

```bash
echo 'GOOGLE_API_KEY=AIzaSy...your-key-here' >> "$USERPROFILE/AppData/Local/hermes/.env"
```

Hermes checks `GOOGLE_API_KEY` and `GEMINI_API_KEY` for the `gemini` provider.

### 2. Provider Type

Use `provider: gemini` (the native Gemini adapter, not `custom`). The native adapter translates OpenAI-shaped messages to Gemini's `generateContent` schema, preserves tool calling and streaming.

### 3. Base URL

```
https://generativelanguage.googleapis.com/v1beta
```

This is the native Gemini endpoint. Hermes detects it and activates the native adapter.

### 4. Model Names

Discover available models on your account:

```bash
# List all Gemini models your key can access
curl -s "https://generativelanguage.googleapis.com/v1/models?key=$GOOGLE_API_KEY" | \
  python3 -c "import sys,json; [print(m['name'], m.get('description','')) for m in json.load(sys.stdin)['models']]"
```

Common models (free-tier accessible):
- `gemini-2.5-flash-lite` — cheap, fast, released July 2025
- `gemini-2.0-flash-lite` — older fallback
- `gemini-2.5-flash` — better quality, may need billing
- `gemini-2.5-pro` — best quality, billing required

### 5. Verify Connectivity

Test the raw API before relying on fallback:

```python
import urllib.request, json

API_KEY = "AIzaSy..."  # your key
body = json.dumps({"contents": [{"parts": [{"text": "Hello"}]}]}).encode()
req = urllib.request.Request(
    f"https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash-lite:generateContent?key={API_KEY}",
    data=body, headers={"Content-Type": "application/json"}, method="POST"
)
with urllib.request.urlopen(req, timeout=30) as resp:
    result = json.loads(resp.read())
    print(result['candidates'][0]['content']['parts'][0]['text'])
```

## When Fallback Activates

From statertly-pixel/mcp-cloud-deploy, MIT.

More from statertly-pixel/mcp-cloud-deploy