web-cache

Files (3)

• SKILL.md
• web_cache.py
• web_cache.sh

SKILL.md (first lines)

# Web Cache — Local Semantic Cache for Agent Web Results

## Overview

`web_cache` is a transparent cache that sits between you and the web. It stores `web_search` and `web_extract` results in a local SQLite database with vector embeddings (via Ollama nomic-embed-text:v1.5), so you can check the cache **before** making expensive web calls.

Over time, the cache becomes a rich "local internet" — you can answer most common queries from cached knowledge alone, even when offline.

## Installation

Installed automatically by `install.sh` step 10 into `~/.hermes/web-cache/`.

Requires: `sqlite-vec`, `requests` (installed in a venv at `~/.hermes/web-cache/.venv/`)

## Usage Protocol — How to Use This in Every Session

### 1. Before `web_search(query)` — Check Cache First

```python
# Step 1: Check cache
cache_result = terminal("""
    web_cache search "your search query here"
""")

# Step 2: Parse the result
import json
cache_data = json.loads(cache_result["output"])

if cache_data.get("cached"):
    hits = cache_data["hits"]
    # Use cached results — skip the web call entirely
    # Each hit has: query, summary, urls, similarity, source
else:
    # Cache miss — proceed with web_search as normal
    web_results = web_search("your search query here")
    
    # Then store the results for next time:
    terminal(f"""
        echo '{json.dumps(web_results)}' | web_cache store "your search query here" -
    """)
```

### 2. Before `web_extract(url)` — Check URL Cache First

```python
# Step 1: Check URL cache
cache_result = terminal(f"""
    web_cache extract {url}
""")

cache_data = json.loads(cache_result["output"])

if cache_data.get("cached"):
    # Use cached content — skip the web call
    content = cache_data["content"]
else:
    # Cache miss — proceed with web_extract as normal
    content = web_extract(url)
    
    # Then store for next time:
    terminal(f"""
        echo '{json.dumps(content)}' | web_cache store-extract {url} -
    """)
```

### 3. Automated Mode

For quick lookups, use `auto` and `auto-extract`:
```bash
# Returns cached results or empty miss
web_cache auto "query"
web_cache auto-extract "https://example.com"
```

These never make web calls — they only check the cache.

## Offline Operation

When you have no internet connection:
1. The cache is your only source of truth
2. Use `web_cache search <query>` — if the answer was cached before, you get it
3. Use `web_cache extract <url>` — if the page was cached before, you get it
4. No internet? No problem — the cache just returns misses for unknown queries

**Pro tip:** Before going offline, run a broad search session to populate the cache with content you'll need. Or import a cache export from another machine.

## Cache Sync (Multi-Machine)

To share cache between machines:
```bash
# On machine A: export
web_cache export ~/web-cache-export.tar.gz

# Transfer the file (AirDrop, USB, SSH, etc.)

# On machine B: import
web_cache import ~/web-cache-export.tar.gz
```

This merges — doesn't overwrite — so both machines accumulate knowledge.

## Maintenance

```bash
# View cache stats
web_cache stats

# Prune old entries when over 200MB limit
web_cache prune

# Manual backup
web_cache backup ~/backups/

# Export portable archive
web_cache export ~/backups/web-cache-YYYY-MM-DD.tar.gz
```

Cron jobs handle nightly pruning and backup automatically.

From lukemcqueen/hermes-cortex, MIT.

More from lukemcqueen/hermes-cortex