dk-lm-studio-pydantic-ai

Files (2)

• SKILL.md
• lmstudio_utils.py · references/

SKILL.md (first lines)

# LM Studio with Pydantic AI

Use LM Studio as a local inference backend with Pydantic AI to extract structured data from text using self-hosted LLMs.

## When to Use This Skill

Invoke this skill when:

- User wants to use a local LLM (LM Studio) with Pydantic AI
- User mentions LM Studio, local inference, offline AI, or self-hosted LLMs
- User asks to extract structured data from text (keywords, entities, sentiment)
- Building document processing pipelines
- Need structured output without API calls to OpenAI/Anthropic

Do **not** use this skill for:

- Cloud-based LLM providers (OpenAI, Anthropic, Google)
- General Pydantic AI agent patterns (use `building-pydantic-ai-agents` instead)
- Non-Pydantic AI frameworks

## Prerequisites

```bash
uv add "pydantic-ai-slim[openai]" python-dotenv
```

The `lmstudio` Python package is **optional** — by default, CLI-based helpers are used. If you prefer the Python SDK approach, add it with `uv add lmstudio`.

Ensure LM Studio is installed with a downloaded model.

## Quick-Start Patterns

### Ensure Model is Loaded

Always call this before creating any Pydantic AI agent. The helper functions are in `references/lmstudio_utils.py`:

```python
# lm_server.py (CLI-based, no extra package needed)
import sys
sys.path.insert(0, "path/to/references")
from lmstudio_utils import ensure_model_loaded

def ensure_model_loaded(model_name: str) -> str:
    """Ensure LM Studio server is running and model is loaded. Returns model identifier."""
    from lmstudio_utils import ensure_server_running, load_model
    ensure_server_running()
    return load_model(model_name)
```

### Create a Structured Output Agent

```python
import os
from pydantic import BaseModel
from typing import List

from dotenv import find_dotenv, load_dotenv
load_dotenv(find_dotenv())

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.settings import ModelSettings
from pydantic_ai.exceptions import UnexpectedModelBehavior

from lm_server import ensure_model_loaded

INFERENCE_MODEL = os.getenv("LOCAL_MODEL_INFERENCE")  # Required - must be set in .env
LMSTUDIO_BASE_URL = os.getenv("LMSTUDIO_BASE_URL", "http://localhost:1234/v1")


# Define output schema (customize for your use case)
class PersonInfo(BaseModel):
    name: str
    email: str
    skills: List[str]


# Create agent
def extract_person_info(text: str) -> PersonInfo:
    ensure_model_loaded(INFERENCE_MODEL)

    agent = Agent(
        model=OpenAIChatModel(
            INFERENCE_MODEL,
            provider=OpenAIProvider(
                base_url=LMSTUDIO_BASE_URL,
                api_key="x",
            ),
        ),
        model_settings=ModelSettings(
            thinking=False,
            extra_body={"chat_template_kwargs": {"enable_thinking": False}},
        ),
        retries=5,
        output_type=PersonInfo,
        instructions="Extract person information from the text. Return only valid JSON.",
    )

    try:
        return agent.run_sync(user_prompt=text).output
    except UnexpectedModelBehavior as exc:
        raise RuntimeError(f"Extraction failed: {exc}")
```

### Agent Factory Pattern

For reusable agents:

```python
from typing import Type
from pydantic import BaseModel


def create_extractor(output_type: Type[BaseModel], instructions: str):
    """Factory for creating extraction agents."""
    return Agent(
        model=OpenAIChatModel(
            INFERENCE_MODEL,

From dushyantkhosla/agent-skills, MIT.

More from dushyantkhosla/agent-skills