ai-cost-optimizer

Files (2)

• SKILL.md
• metadata.json

SKILL.md (first lines)

# AI Cost Optimizer

## When to Use
- Adding prompt caching to reduce repeated system prompt costs
- Choosing the right model tier for a task (Haiku vs Sonnet vs Opus)
- Using the Batch API for offline/async workloads
- Counting tokens before sending a request
- Truncating conversation history to fit a budget
- Estimating API costs before running a pipeline

## Core Rules
1. Always cache system prompts and static context — they're re-sent every turn and caching cuts that cost by ~90%.
2. `cache_control: {type: "ephemeral"}` requires a minimum of 1024 tokens to actually cache.
3. Cached tokens cost ~10% of normal input price; first-fill costs ~25% more but is amortized quickly.
4. Use `claude-haiku-4-5` for classification, routing, extraction, and simple Q&A — it's 10-30x cheaper than Opus.
5. Use the Batch API for any workload that can wait up to 24 hours — it's 50% cheaper than the sync API.
6. Count tokens with `client.messages.count_tokens()` before running expensive pipelines.
7. Truncate conversation history from the middle, not the end — preserve system context and recent messages.
8. Set `max_tokens` to the actual expected output length — you pay for capacity, not unused tokens.
9. For RAG, retrieve only the top-K most relevant chunks; don't stuff the full document into context.
10. Cache the large/static part of the prompt at the top; dynamic content goes at the bottom (cache reads are positional from the top).

## Pricing Reference (approximate, check anthropic.com for latest)

| Model | Input (per 1M) | Output (per 1M) | Cache Write | Cache Read |
|-------|---------------|-----------------|-------------|------------|
| claude-haiku-4-5 | $0.80 | $4.00 | $1.00 | $0.08 |
| claude-sonnet-4-5 | $3.00 | $15.00 | $3.75 | $0.30 |
| claude-opus-4-5 | $15.00 | $75.00 | $18.75 | $1.50 |

## Prompt Caching (Python)

```python
import anthropic

client = anthropic.Anthropic()

LARGE_DOCUMENT = "..." * 500  # 1000+ tokens of static content

response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "You are an expert document analyst. Answer questions precisely.",
        },
        {
            "type": "text",
            "text": LARGE_DOCUMENT,
            "cache_control": {"type": "ephemeral"},  # Cache this block
        },
    ],
    messages=[{"role": "user", "content": "What are the main themes?"}],
)

# Check cache performance
usage = response.usage
print(f"Input tokens: {usage.input_tokens}")
print(f"Cache creation: {getattr(usage, 'cache_creation_input_tokens', 0)}")
print(f"Cache read: {getattr(usage, 'cache_read_input_tokens', 0)}")

# Second request — cache_read_input_tokens will be high
response2 = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    system=[
        {"type": "text", "text": "You are an expert document analyst."},
        {
            "type": "text",
            "text": LARGE_DOCUMENT,
            "cache_control": {"type": "ephemeral"},  # Same block = cache hit
        },
    ],
    messages=[{"role": "user", "content": "Who are the key stakeholders?"}],
)
```

## Prompt Caching (TypeScript)

```typescript
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const STATIC_CONTEXT = "...".repeat(300); // Must be 1024+ tokens

async function askWithCache(question: string) {
  const response = await client.messages.create({
    model: "claude-opus-4-5",
    max_tokens: 1024,
    system: [
      {
        type: "text",
        text: "You are a helpful assistant. Answer concisely.",
      },
      {
        type: "text",
        text: STATIC_CONTEXT,
        // @ts-ignore — cache_control is a beta field
        cache_control: { type: "ephemeral" },
      },
    ] as Anthropic.TextBlockParam[],
    messages: [{ role: "user", content: question }],
  });

  const usage = response.usage as Anthropic.Usage & {
    cache_creation_input_tokens?: number;
    cache_read_input_tokens?: number;
  };

  console.log(`Cache hit: ${usage.cache_read_input_tokens ?? 0} tokens`);
  return (response.content[0] as Anthropic.TextBlock).text;
}
```

## Caching Conversation History

```python

From UltronCore/claude-skill-vault, MIT.

More from UltronCore/claude-skill-vault