fine-tuning-guide

Files (1)

• SKILL.md

SKILL.md (first lines)

# Fine-Tuning Guide

Fine-tune language models efficiently with proper data prep, LoRA config, and evaluation.

---

## Workflow

```
1. Decide: fine-tune vs. alternative approaches
2. Prepare dataset
3. Select base model and method
4. Configure and train
5. Evaluate and iterate
```

---

## Step 1 -- Should You Fine-Tune?

| Approach | When | Cost |
|---|---|---|
| **Prompt engineering** | Task achievable with good prompts, sufficient quality | Free |
| **Few-shot examples** | Need consistent format/style, small improvement needed | Free |
| **RAG** | Model needs access to private/up-to-date knowledge | Low |
| **Fine-tuning** | Need specific style/behavior/domain that prompts can't achieve | Moderate |
| **Full training** | Building a new capability from scratch | High |

**Fine-tune when:**
- Prompt engineering gives 70-80% quality but you need 95%+
- You have 100+ high-quality examples of desired behavior
- You need consistent output format that prompts can't reliably enforce
- You're optimizing for cost (fine-tuned small model replaces large model prompting)
- Domain-specific terminology or reasoning patterns needed

---

## Step 2 -- Prepare Dataset

### Format: Instruction Tuning (Chat Format)

```jsonl
{"messages": [{"role": "system", "content": "You are a medical coding assistant."}, {"role": "user", "content": "Code this diagnosis: chest pain on exertion"}, {"role": "assistant", "content": "ICD-10: R07.9 - Chest pain, unspecified. Consider also I20.9 if angina suspected."}]}
{"messages": [{"role": "system", "content": "You are a medical coding assistant."}, {"role": "user", "content": "Code this procedure: knee replacement"}, {"role": "assistant", "content": "CPT: 27447 - Total knee arthroplasty."}]}
```

### Dataset Size Guidelines

| Task | Minimum | Good | Great |
|---|---|---|---|
| Style/format adaptation | 50 | 200 | 500+ |
| Domain knowledge injection | 200 | 500 | 2000+ |
| Complex reasoning | 500 | 2000 | 10000+ |
| Classification | 100/class | 500/class | 2000+/class |

### Quality Checklist

```python
# Validation script for training data
import json
from pathlib import Path

def validate_dataset(path: str):
    issues = []
    examples = [json.loads(l) for l in Path(path).read_text().splitlines() if l.strip()]

    for i, ex in enumerate(examples):
        # Check structure
        if "messages" not in ex:
            issues.append(f"Row {i}: missing 'messages' key")
            continue

        msgs = ex["messages"]
        roles = [m["role"] for m in msgs]

        # Must end with assistant
        if roles[-1] != "assistant":
            issues.append(f"Row {i}: must end with assistant message")

        # Must have at least user + assistant
        if "user" not in roles or "assistant" not in roles:
            issues.append(f"Row {i}: needs both user and assistant messages")

        # Check for empty content
        for m in msgs:
            if not m.get("content", "").strip():
                issues.append(f"Row {i}: empty {m['role']} message")

    print(f"Total examples: {len(examples)}")
    print(f"Issues found: {len(issues)}")
    for issue in issues[:20]:
        print(f"  - {issue}")

    return len(issues) == 0
```

### Train/Val Split
```python
import random
random.seed(42)
random.shuffle(examples)
split = int(len(examples) * 0.9)
train, val = examples[:split], examples[split:]
```

---

## Step 3 -- Select Base Model and Method

### Base Model Selection

| Model | Size | When | Hugging Face ID |
|---|---|---|---|
| Llama 3.1 8B | 8B | General purpose, good quality/size balance | `meta-llama/Llama-3.1-8B-Instruct` |
| Mistral 7B | 7B | Strong for size, fast inference | `mistralai/Mistral-7B-Instruct-v0.3` |
| Phi-3 Mini | 3.8B | Resource constrained, surprisingly capable | `microsoft/Phi-3-mini-4k-instruct` |
| Qwen 2.5 7B | 7B | Strong math / code capabilities | `Qwen/Qwen2.5-7B-Instruct` |
| Gemma 2 9B | 9B | Google ecosystem, strong multilingual | `google/gemma-2-9b-it` |

From tezansahu/ai-garage, GPL-3.0.

More from tezansahu/ai-garage