databricks-model-serving

Files (10)

• 1-classical-ml.md
• 2-custom-pyfunc.md
• 3-genai-agents.md
• 4-tools-integration.md
• 5-development-testing.md
• 6-logging-registration.md
• 7-deployment.md
• 8-querying-endpoints.md
• 9-package-requirements.md
• SKILL.md

SKILL.md (first lines)

# Databricks Model Serving

Deploy MLflow models and AI agents to scalable REST API endpoints.

## Quick Decision: What Are You Deploying?

| Model Type | Pattern | Reference |
|------------|---------|-----------|
| **Traditional ML** (sklearn, xgboost) | `mlflow.sklearn.autolog()` | [1-classical-ml.md](1-classical-ml.md) |
| **Custom Python model** | `mlflow.pyfunc.PythonModel` | [2-custom-pyfunc.md](2-custom-pyfunc.md) |
| **GenAI Agent** (LangGraph, tool-calling) | `ResponsesAgent` | [3-genai-agents.md](3-genai-agents.md) |

## Prerequisites

- **DBR 16.1+** recommended (pre-installed GenAI packages)
- Unity Catalog enabled workspace
- Model Serving enabled

## Foundation Model API Endpoints

ALWAYS use exact endpoint names from this table. NEVER guess or abbreviate.

### Chat / Instruct Models

| Endpoint Name | Provider | Notes |
|--------------|----------|-------|
| `databricks-gpt-5-2` | OpenAI | Latest GPT, 400K context |
| `databricks-gpt-5-1` | OpenAI | Instant + Thinking modes |
| `databricks-gpt-5-1-codex-max` | OpenAI | Code-specialized (high perf) |
| `databricks-gpt-5-1-codex-mini` | OpenAI | Code-specialized (cost-opt) |
| `databricks-gpt-5` | OpenAI | 400K context, reasoning |
| `databricks-gpt-5-mini` | OpenAI | Cost-optimized reasoning |
| `databricks-gpt-5-nano` | OpenAI | High-throughput, lightweight |
| `databricks-gpt-oss-120b` | OpenAI | Open-weight, 128K context |
| `databricks-gpt-oss-20b` | OpenAI | Lightweight open-weight |
| `databricks-claude-opus-4-6` | Anthropic | Most capable, 1M context |
| `databricks-claude-sonnet-4-6` | Anthropic | Hybrid reasoning |
| `databricks-claude-sonnet-4-5` | Anthropic | Hybrid reasoning |
| `databricks-claude-opus-4-5` | Anthropic | Deep analysis, 200K context |
| `databricks-claude-sonnet-4` | Anthropic | Hybrid reasoning |
| `databricks-claude-opus-4-1` | Anthropic | 200K context, 32K output |
| `databricks-claude-haiku-4-5` | Anthropic | Fastest, cost-effective |
| `databricks-claude-3-7-sonnet` | Anthropic | Retiring April 2026 |
| `databricks-meta-llama-3-3-70b-instruct` | Meta | 128K context, multilingual |
| `databricks-meta-llama-3-1-405b-instruct` | Meta | Retiring May 2026 (PT) |
| `databricks-meta-llama-3-1-8b-instruct` | Meta | Lightweight, 128K context |
| `databricks-llama-4-maverick` | Meta | MoE architecture |
| `databricks-gemini-3-1-pro` | Google | 1M context, hybrid reasoning |
| `databricks-gemini-3-pro` | Google | 1M context, hybrid reasoning |
| `databricks-gemini-3-flash` | Google | Fast, cost-efficient |
| `databricks-gemini-2-5-pro` | Google | 1M context, Deep Think |
| `databricks-gemini-2-5-flash` | Google | 1M context, hybrid reasoning |
| `databricks-gemma-3-12b` | Google | 128K context, multilingual |
| `databricks-qwen3-next-80b-a3b-instruct` | Alibaba | Efficient MoE |

### Embedding Models

| Endpoint Name | Dimensions | Max Tokens | Notes |
|--------------|-----------|------------|-------|
| `databricks-gte-large-en` | 1024 | 8192 | English, not normalized |
| `databricks-bge-large-en` | 1024 | 512 | English, normalized |
| `databricks-qwen3-embedding-0-6b` | up to 1024 | ~32K | 100+ languages, instruction-aware |

### Common Defaults

- **Agent LLM**: `databricks-meta-llama-3-3-70b-instruct` (good balance of quality/cost)
- **Embedding**: `databricks-gte-large-en`
- **Code tasks**: `databricks-gpt-5-1-codex-mini` or `databricks-gpt-5-1-codex-max`

> These are pay-per-token endpoints available in every workspace. For production, consider provisioned throughput mode. See [supported models](https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models).

## Reference Files

| Topic | File | When to Read |
|-------|------|--------------|
| Classical ML | [1-classical-ml.md](1-classical-ml.md) | sklearn, xgboost, autolog |
| Custom PyFunc | [2-custom-pyfunc.md](2-custom-pyfunc.md) | Custom preprocessing, signatures |
| GenAI Agents | [3-genai-agents.md](3-genai-agents.md) | ResponsesAgent, LangGraph |
| Tools Integration | [4-tools-integration.md](4-tools-integration.md) | UC Functions, Vector Search |
| Development & Testing | [5-development-testing.md](5-development-testing.md) | MCP workflow, iteration |
| Logging & Registration | [6-logging-registration.md](6-logging-registration.md) | mlflow.pyfunc.log_model |
| Deployment | [7-deployment.md](7-deployment.md) | Job-based async deployment |
| Querying Endpoints | [8-querying-endpoints.md](8-querying-endpoints.md) | SDK, REST, MCP tools |
| Package Requirements | [9-package-requirements.md](9-package-requirements.md) | DBR versions, pip |

---

## Quick Start: Deploy a GenAI Agent

### Step 1: Install Packages (in notebook or via MCP)

```python
%pip install -U mlflow==3.6.0 databricks-langchain langgraph==0.3.4 databricks-agents pydantic
dbutils.library.restartPython()
```

Or via MCP:
```
execute_code(code="%pip install -U mlflow==3.6.0 databricks-langchain langgraph==0.3.4 databricks-agents pydantic")
```

### Step 2: Create Agent File

Create `agent.py` locally with `ResponsesAgent` pattern (see [3-genai-agents.md](3-genai-agents.md)).

### Step 3: Upload to Workspace

```
manage_workspace_files(
    action="upload",
    local_path="./my_agent",
    workspace_path="/Workspace/Users/you@company.com/my_agent"
)
```

### Step 4: Test Agent

```
execute_code(

From ThomazRossito/data-agents, MIT.

More from ThomazRossito/data-agents