modal

Files (1)

• SKILL.md

SKILL.md (first lines)

# Modal

## Overview

Modal is a serverless Python compute platform. You define container images, functions, and resources in Python, then run them remotely with sub-second cold starts and per-second billing. There is no YAML, no Dockerfile, no Kubernetes; the container, GPUs, scaling, and storage are all configured through Python decorators.

Common reasons to reach for Modal:

- Run AI inference (open-weights or custom models) with GPUs on demand
- Fine-tune or train models without managing infrastructure
- Sandbox untrusted or LLM-generated code
- Schedule batch jobs or cron tasks
- Stand up HTTP endpoints (FastAPI, ASGI, WSGI, raw web servers) backed by GPUs
- Fan out embarrassingly parallel work via `.map()` and `.spawn()`

Modal is Python-only. The `modal` SDK runs locally, ships your code to Modal's cloud, and executes it in containers you described declaratively.

## Installation and Auth

```bash
pip install modal
modal setup           # Opens browser to create/auth a token, writes ~/.modal.toml
```

For CI or scripted environments, set `MODAL_TOKEN_ID` and `MODAL_TOKEN_SECRET` instead.

## The Mental Model

1. You write a Python file with an `App` and one or more decorated functions/classes.
2. Each function is bound to an `Image` (the container) and optional resources (GPU, volumes, secrets, schedule).
3. You either:
   - `modal run script.py` — ephemeral; runs once and tears down (good for dev).
   - `modal serve script.py` — ephemeral; hot-reloads on file changes (good for iterating on web endpoints).
   - `modal deploy script.py` — persistent; the App stays up, accessible by name from anywhere.
4. From local code, call remote functions with `.remote(args)`. From inside Modal containers, you can call other functions the same way.

## Apps and Functions

The minimum viable Modal program:

```python
import modal

app = modal.App("hello")

@app.function()
def square(x: int) -> int:
    return x * x

@app.local_entrypoint()
def main():
    print(square.remote(5))         # runs in the cloud
    print(list(square.map(range(10))))  # parallel fan-out
```

Run it: `modal run script.py`.

### Invocation methods

Called on a function object from local code (or another Modal container):

| Method | Behavior |
|---|---|
| `.remote(*a)` | Run once on Modal, return the result |
| `.local(*a)` | Run in the current process (no Modal involved) |
| `.map(iter)` | Parallel fan-out, results in input order, capped at 1,000 in flight per call |
| `.starmap(iter)` | Like `.map()` but spreads each tuple as args |
| `.for_each(iter)` | Fire-and-forget map; ignores results |
| `.spawn(*a)` | Fire-and-forget single call; returns a `FunctionCall` handle |

For batch work that may include failures, use `.map(inputs, return_exceptions=True)` and inspect each result.

### Local entrypoints

`@app.local_entrypoint()` is the function `modal run` calls by default. Type-annotated parameters become CLI flags:

```python
@app.local_entrypoint()
def main(count: int = 10, name: str = "world"):
    ...
# modal run script.py --count 5 --name Markus
```

## Images

Build images by chaining methods. Modal caches each layer; put fast-changing steps last so caching pays off.

```python
image = (
    modal.Image.debian_slim(python_version="3.12")
    .apt_install("git", "curl")
    .uv_pip_install("torch==2.8.0", "transformers==4.46.0")
    .env({"HF_HOME": "/cache/hf"})
    .add_local_python_source("my_pkg")
)

@app.function(image=image)
def run():
    import torch  # imports needed only on Modal go inside the function
    ...
```

Useful image methods:

- `debian_slim(python_version=...)` — default starting point.
- `from_registry("nvidia/cuda:12.4.0-devel-ubuntu22.04", add_python="3.12")` — start from any public image.
- `from_dockerfile("Dockerfile")` — use an existing Dockerfile.
- `pip_install(...)` / `uv_pip_install(...)` — prefer `uv_pip_install` (faster).
- `apt_install(...)`, `run_commands("...")`, `env({...})`, `workdir("/app")`.
- `add_local_dir(local, remote)`, `add_local_file(local, remote)`, `add_local_python_source("pkg")`.
- `run_function(fn, gpu="A10G", secrets=[...])` — bake state into the image at build time, e.g. download model weights.

Pin versions tightly. Loose pins make builds non-reproducible and surprise you when an upstream release breaks the image.

Pass `force_build=True` to a single layer to bust its cache, or set `MODAL_FORCE_BUILD=1` in the environment.

## GPUs

Set `gpu=` on the function decorator. Common values:

From maragudk/fabrik, MIT.

More from maragudk/fabrik