AGENTS.md · crmne/ruby_llm

Sections

1. What this is
2. Framework structure
3. Source layout
4. What we are optimizing for
5. Ground rules
6. Setup
7. Commands
8. Public API rules
9. Boundaries (enforced by Archspec.rb)
10. Rails integration rules
11. Testing
12. Code style
13. Docs
14. Releases
15. Pull requests

First lines (shown under the repo's MIT license)

# AGENTS.md

How RubyLLM is built. For coding agents first, humans welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) for what gets accepted; this file is how to write it.

## What this is

RubyLLM is a Ruby AI framework: chat, tools, agents, structured output, embeddings, reranking, images, video, audio, OCR, moderation, and Rails integration behind one API, with twenty-three providers in the box. Plain Ruby (>= 3.2), a handful of small dependencies, no heavyweight abstractions.

## Framework structure

There are two main parts: the public Ruby API and the providers and protocols that connect it to AI services.

- **The public API has two families.** Text generation and conversations use `Chat`, `Message`, `Tool`, and `Agent`, with structured output, streaming, and loop control. Individual AI operations use `paint`, `animate`, `speak`, `transcribe`, `ocr`, `moderate`, `embed`, and `rerank`, each with a typed result. These operations do not require a conversation; some have streaming or asynchronous lifecycles.
- **Providers and protocols connect both families to services.** Providers supply endpoints, authentication, catalogs, protocol selection, and service-specific settings. Protocols implement request formats, response parsing, streaming, and error normalization. Format quirks belong in protocol dialects selected by providers.
- **Shared services support the API.** Model and provider resolution, configuration, usage and cost tracking, instrumentation, batches, file storage, and prompt caching belong to the framework as a whole. Do not force an independent operation through `Chat` to reuse them.
- **Rails adds a native application integration.** The same conversation API works on application-owned records, with Active Record persistence, Active Storage attachments, Hotwire streaming, Active Job workflows, and generators. Individual operations remain available in Rails services and jobs. Rails integration builds on the Ruby API; the plain-Ruby library has no Rails dependency.
- **Conversation history belongs to the application.** Build requests from the local transcript. Do not introduce provider-stored conversation cursors or continue a remote history that can diverge from local messages. Defer provider features that require that lifecycle. Preserving native response content for replay is different from storing a conversation remotely.
- **Realtime conversations are outside 2.0's scope.** Keep speech generation and transcription, including streamed results. A WebSocket transport for one operation does not justify a separate live conversation API. Hosted asynchronous research jobs remain individual operations, not persisted chat sessions.

## Source layout

Keep public API objects directly under `lib/ruby_llm`. This includes objects applications receive and use, such as `EmbeddingRequest`, `SpeechChunk`, `TranscriptionChunk`, `VideoJob`, `UploadedFile`, and `DownloadedFile`, even when their constructors are internal. Put implementation helpers in the namespace that owns them, with matching directories: `Protocol::BinaryStreaming`, `Transport::Connection`, `Models::Registry`, and `Support::Inspectable`. Use ordinary Zeitwerk naming and update references when moving internal constants; do not use `loader.collapse` or compatibility aliases to rearrange internal code. Keep operation arguments and public result behavior unchanged. Follow the same namespaces in specs.

| Directory | Responsibility |
| --- | --- |
| `transport/` | HTTP, WebSockets, retry boundaries, and connection middleware |
| `protocol/` | Shared streaming and response assembly |
| `models/` | Model catalog schemas, reconciliation, and alias lookup |
| `accounting/` | Usage entries and operation accounting |
| `files/` | File handling internals such as MIME detection |
| `tools/` | Shared tool selection and resolution |
| `support/` | Cross-cutting primitives such as inspection, instrumentation, deprecation, and utility functions |

Keep operation behavior with its owning domain or protocol; `support/` is not a place for provider logic or unrelated features. Preserve bundled JSON and generator-template locations when moving code that uses `__dir__`. Archspec enforces the boundaries independently of these organizational folders.

Keep application generators, migration-only helpers, and provider scaffolding under `lib/generators/ruby_llm/`, outside the runtime tree. Rake tasks belong in `lib/tasks/` or the repository's `tasks/`. Load this tooling explicitly from its commands or generated migrations. `lib/ruby_llm/active_record/` contains the record behavior applications use, not upgrade tooling.

## What we are optimizing for

The pitch is the public API. Every change is judged against four things, in this order:

1. **Ruby-native.** The API reads like Ruby that a Rails developer already knows: chainable setters, one name per concept, keyword arguments, value objects with readers, predicates that end in `?`, and nothing that looks like a port of a Python SDK. If DHH would raise an eyebrow at a method name, rename it.
2. **Seamless in Rails.** `acts_as_chat` and `acts_as_message` make an application's own records behave exactly like the plain-Ruby objects. RubyLLM owns its supporting tables, the app owns chats and messages, and every plain-Ruby feature works on a persisted record without a second API.
3. **Providers and protocols in their place.** A provider is who you talk to; a protocol is how. Wire vocabulary never leaks into the domain, and the domain never names a provider. `Archspec.rb` fails the build when it does.
4. **Craftsmanship.** Small focused diffs, specs for every behavior, RDoc on every public method, docs in the Rails-guides voice, no implementation comments, no dead code. Quality is the feature.

## Ground rules

- Never edit generated files: `lib/ruby_llm/models.json`, `lib/ruby_llm/aliases.json`, `docs/_reference/available-models.md`. `rake models` regenerates them.
- Never invent model ids. Every model name in code, specs, and docs must be a real, callable model. Check with `RubyLLM.models.find`.
- Keep diffs small and focused. No drive-by refactors, no style sweeps outside the lines you touch.
- Plain commit messages that describe the change: a subject of at most 60 characters, a body wrapped at 72 that says why. No "Generated with" footers, no Co-Authored-By tags.
- Agents may push straight to `main`, but prefer a pull request for anything beyond a small fix, so CodeRabbit and GitHub code scanning review it first. They merge their own pull requests (squash, for linear history) once CI is green and those reviews are in, with every finding fixed or answered.
- Gem publication starts when a maintainer publishes a GitHub release. A branch or tag push does not publish the gem. Keep the release tag, gem version, and prerelease setting consistent; see CONTRIBUTING.md for the release commands.

## Setup

```bash
bundle install
overcommit --install   # required: installs the git hooks that gate every commit
```

## Commands

| Command | What it does |
|---|---|
| `bundle exec rspec --tag ~live` | Fast unit run. No API keys, no cassettes needed. |
| `bundle exec rspec spec/ruby_llm/chat_spec.rb:42` | One example. |
| `bundle exec rake test` | Full suite through rspec-queue, like CI. |
| `bundle exec rspec --tag generator` | The slow Rails generator specs, excluded from the commit hook. |
| `overcommit --run` | Everything the pre-commit hook runs: RuboCop, Flay, archspec, RSpec. |
| `bundle exec rubocop` | Lint (auto-corrects on commit). |
| `bundle exec archspec check` | Architecture rules from `Archspec.rb`. |
| `bundle exec rake conformance` | The official MCP client conformance suite against `RubyLLM::MCP` (needs Node). |
| `bundle exec rake benchmark` | RubyLLM's own overhead, offline; `rake "benchmark:compare[v2.0.0]"` compares against another ref. |
| `bundle exec appraisal rails-8.0 rspec` | Rails version matrix (7.1 through 8.1, see `Appraisals`). |
| `docs/bin/serve.sh` | Docs preview at localhost:4002. |

## Public API rules

- **Chain setters, read state.** Every `Chat#with_x` returns `self`, accepts `nil` to reset, and has a bare `x` reader (instructions are the one exception: they live in the transcript, so read `chat.messages`). Every `with_x` also has a bare `x` class macro on `Agent` and appears in the delegate lists in `Agent` and `ActiveRecord::ChatMethods`; the delegation specs fail when the three drift.
- **Feature switches share one shape.** `with_thinking`, `with_citations`, `with_caching`, and `with_compaction` take nothing or `true` to enable, `false` to disable, and options as keywords or a Hash. They reject `nil`.
- **One name per concept, everywhere.** `instructions` not `system_prompt`, `max_output_tokens` not `max_tokens`, `thinking` not `reasoning`, `cache_read` and `cache_write`, `provider_options` for the provider's own vocabulary, `provider:` as a keyword. The registry's capability names come from models.dev and stay as they are (`supports?(:reasoning)`).
- **One interface over every provider.** RubyLLM is the Ruby-native AI framework, and one consistent API across every provider is a big part of why it exists. When two or more providers can do the same thing, design one RubyLLM name for it and translate it in each protocol: `pages:` on `ocr`, `uri:` and `content_type:` on `upload`, `thinking` across five different reasoning APIs. `provider_options` is for the one thing only one provider does, never a shortcut past designing the shared name. Study what each provider offers, find the concept they share, and give it a Ruby name a Rails developer would guess.
- **Capabilities are one query**, `supports?(:vision)`, never a `supports_vision?` predicate.
- **RubyLLM's own enumerations are Symbols.** `finish_reason` is `:stop`, a usage entry's status is `:succeeded`, thinking effort is `:medium`, `Model#type` is `:chat`. Values that come from a provider or from models.dev (slugs, model ids, MIME types, reasoning option values) stay Strings, and protocols turn Symbols into Strings only at the wire.
- **Bang methods are for a meaningfully different pair**, like `create`/`create!`. Mutation, persistence, blocking, or network activity alone does not earn a bang. `cancel`, `approve`, `refresh` are plain.
- **Value objects, not hashes.** Results are typed (`Message`, `Tokens`, `Cost`, `Citation`, `Moderation::Result`) with readers named after the concept. `to_h` is for serialization, `inspect` is one short line through `Inspectable`.
- **One cost interface.** `cost` returns `RubyLLM::Cost`, using the provider's reported amount when available and calculated cost otherwise. Preserve unknown amounts as `nil`. The source of a stored amount is internal accounting information, not a second public cost API.
- **Token counts are for pricing.** `Tokens` buckets (`input`, `output`, `cache_read`, `cache_write`, `thinking`) exist so `Cost` matches what the provider bills. Put each count in the bucket the provider charges it at, and judge a parsing change by whether the invoice comes out right, not by whether the counts look semantically consistent. When a provider reports a field differently from its protocol, handle it in that provider's dialect and apply it to every response. Never infer the meaning of a field from its magnitude.
- **Remote approvals use a predicate.** A tool call's `remote?` distinguishes provider-executed tools from local Ruby tools. Persist that boolean; its call ID identifies the approval request. Do not use a provider's server label as the execution-mode flag.
- **Save file and media results consistently.** Complete downloadable results expose `save(path)`, returning the path, and `to_blob` for their bytes. Show `RubyLLM.speak(...).save(path)` and equivalent calls before manual file writing. Streaming is optional; when an operation also returns its complete result, that result keeps the same saving API.
- **Errors take the message first** and the response as a keyword: `Error.new("msg", response: response)`.
- **Let the provider reject what the model can't do.** Raise before a request only when the protocol cannot express it, such as an operation the provider lacks or an attachment its wire format cannot carry. Don't check model capabilities, provider limits, or option values the API validates itself: its error is current, and a copy in RubyLLM goes stale. Don't silently downgrade an explicit request either. Trust the shape of a successful response; one rescue that turns a malformed body into a RubyLLM error is enough. Do check what maps a response back onto your inputs, such as indices, IDs, and positions: a wrong mapping corrupts results silently.
- **Public means documented.** RDoc on the method, an example in `docs/`, and `:nodoc:` on everything that is internal.

## Boundaries (enforced by Archspec.rb)

`Archspec.rb` at the repo root is the architecture documentation, and `archspec check` fails the build when code violates it. Read it before moving anything across layers. The short version:

- **Domain objects** (`Chat`, `Message`, `Tool`, `Agent`, ...) never reference `RubyLLM::Providers` or `RubyLLM::Protocols`, never name a provider or a wire format in a method, and never carry a table of provider values. They delegate through the `Provider` contract.
- **Protocols** (`lib/ruby_llm/protocols`) are wire formats: Chat Completions, Responses, Anthropic, Gemini, Converse, Cohere, Files, and provider-specific storage APIs. Serialization methods are `render_*`, parsing methods are `parse_*`, never `format_*` or `extract_*`; what a wire format can carry is a bare predicate (`provider_file_references?`), never `supports_*?`. A protocol normalizes on the way in and out: it maps its finish reasons onto `stop`, `max_tokens`, `tool_calls`, and `content_filter` in its `finish_reasons` table, it decides its own request rules (OpenAI strict mode lives in the OpenAI protocols, not in `Chat`), and it turns its error bodies into RubyLLM errors. Register every operation through `protocol`; do not add operation-specific protocol registries or macros.
- **Providers** (`lib/ruby_llm/providers`) are adapters: auth, endpoints, catalogs, dialect quirks. A provider declares which protocols it speaks; it never defines a new wire format inline. A protocol that needs something only the provider knows asks its `@provider` for it.
- **Hosting does not create a new protocol.** When a cloud provider hosts an existing wire format, reuse that protocol's modules. Keep endpoint and deployment-name differences with the provider; put serialization shared with the original service in the common protocol. Compose only the operations the hosted endpoint supports.
- **Support** (`Configuration`, `Models`, `Model`, `Connection`, errors) is a leaf layer: it knows neither protocols nor concrete providers, nor Active Record. Provider-specific registry behavior, such as how a provider spells a models.dev id, goes through a `Provider` class hook (`models_dev_alias`, `models_dev_model_id`), not a branch on the slug.
- **Model metadata comes from models.dev.** Report incorrect or missing pricing, limits, release dates, knowledge cutoffs, families, or modalities upstream instead of maintaining parallel tables. Provider model parsers record facts the provider returns. Provider `capabilities.rb` files may only augment feature capabilities that neither the provider listing nor models.dev can express, based on explicit upstream fields, exact model ids, or unambiguous operation markers, never broad family matchers that guess about current or future models.
- **Registry reconciliation stays generic.** Provider-specific model-id aliases belong with that provider's catalog code, and registry generation diagnostics belong under `tasks/`, outside the runtime library. Provider gem `models.json` files are explicitly registered, read-only fallbacks: the main registry wins conflicts, global refresh never queries or rewrites catalog-backed provider gems, and only the provider gem's own `rake models` updates its packaged catalog.
- **The plain-Ruby library never touches Active Record.** Rails integration lives in `lib/ruby_llm/active_record` and `lib/ruby_llm/railtie.rb` only, converts with `to_llm`/`from_llm`, and delegates through the domain and the `Provider` contract.

When you find provider vocabulary in the wrong layer, move it and add the rule to `Archspec.rb` so it cannot come back.

## Rails integration rules

- A persisted chat behaves like a plain chat. Whatever `Chat` can do, `acts_as_chat` records do through the same names, and `Agent.find` gives back the record with the agent's tools, instructions, and options applied.
- RubyLLM owns `ruby_llm_models`, `ruby_llm_tool_calls`, `ruby_llm_usages`, `ruby_llm_batches`, `ruby_llm_mcp_credentials`, `ruby_llm_provider_files`, and `ruby_llm_queued_messages`. Applications own chats and messages. Schema changes go through the install generator for new apps and the upgrade generator for apps on the last release; both must move together in the same change. The upgrade generator carries only the changes since the last release. Replace it each release instead of accumulating upgrade templates, and point apps on older releases to the upgrade guide in that release's documentation.
- Persisted usage entries require both a provider and a model, with presence validation and `NOT NULL` constraints. The `model` column names what the provider ran and billed: a model ID, or for an operation without one, the provider's own resource identifier, such as a hosted agent or a guardrail ID and version. Never invent one, including to satisfy a migration. Existing rows without a model must be corrected from their original requests before upgrading.
- Every billable provider attempt reaches the usage ledger in Rails and `usage.ruby_llm` everywhere, including failed and blocked attempts the provider billed. Chat records write their own rows; every other attempt, from one-shot operations and chats without a record, gets a row with no chat. Both retain the owner attributed through `owner:` or `RubyLLM.with_usage_owner`, independently of their chat and message associations. A ledger write never breaks the operation, and the plain-Ruby library reaches the ledger only through the hook the railtie installs.
- Persistence must survive other processes: cancellation, approvals, and the loop verbs read and write the database, and anything polled inside a job runs outside the query cache.
- Generators write what a Rails scaffold would: omakase style, conventional paths, no starter prose, no TODO comments beyond the one place the developer has to type. An empty prompt file means no instructions.
- Rails specs run against the dummy app in `spec/dummy`. Generator specs are tagged `:generator` and excluded from the pre-commit run because they are slow.

## Testing

- Specs tagged `:live` talk to provider APIs through VCR cassettes in `spec/fixtures/vcr_cassettes`. Everything else is a plain unit test. Every bug fix ships with a unit spec that fails before the fix.
- Cassette names derive from the example's full description. A failing `:live` example deletes its cassette on purpose, so the next run hits the real API instead of replaying a recording that may be hiding the bug.
- Re-record cassettes with `rake vcr:record[openai,anthropic]` or `rake vcr:record[all]` (needs real API keys: `.env` is committed and holds only `op-cache read` 1Password lookups, never values; there is no `.env.example`). An empty key counts as unset (`spec/support/blank_keys.rb`), so a failed lookup replays cassettes instead of deleting them. Check cassettes for leaked keys before committing. `bin/gitleaks-staged` runs in the pre-commit hook and CI runs gitleaks again.
- Protocol chat modules use `module_function`. To unit-test a render or parse method, extend the module (plus `Tools` and `Media` when the method needs them) onto a bare Object, the way the existing protocol specs do.
- `spec/support/models_to_test.rb` defines the provider and model matrix the live specs iterate over.
- Select models with `model_for(:openai)` or a named purpose such as `model_for(:openai, :temperature)` from that file. Keep literal IDs only when the exact name is the behavior under test, such as aliases, catalog matching, or wire fixtures.
- Changed a wire request? Re-record the cassette. Changed a parsed value? Update the protocol spec, not the domain spec.

## Code style

- No implementation comments. A comment earns its place only by stating a constraint the code cannot show. Public API gets RDoc; see `.rdoc_options` for what is documented.
- Follow the wire-naming idioms above, and generic Ruby ones: no `get_`/`set_` accessor prefixes, no `is_` predicates.
- Flay rejects structural duplication above mass 70. If two providers or protocols share shape, extract, do not copy.
- RuboCop runs with auto-correct on commit; do not disable cops inline unless the existing code already does at that spot for the same reason.

## Docs

- The Jekyll site lives in `docs/` with four collections: `_getting_started`, `_core_features`, `_advanced`, `_reference`. Preview with `docs/bin/serve.sh`.
- Homepage sections alternate between the base background and the darker `home-band` background. When adding, removing, or moving a section, preserve the alternation through every following section and the footer. Check the rendered result in light and dark mode, on desktop and mobile.
- Keep docs focused on RubyLLM. Do not add provider listings, promotional links, or dedicated setup sections for third-party services that only change an existing provider's API base URL or API key. Their RubyLLM integration instructions belong in their own documentation. Document generic endpoint configuration once; provider-specific additions must satisfy the acceptance policy in `CONTRIBUTING.md`, including documentation-only PRs.
- Voice is Rails-guides style: second person, present tense, short sentences, code first, motivate before mechanics. No em dashes. No hype, no "simply". RDoc follows the Rails API voice: "Returns the ...", one line where one line will do.
- Let working examples show what the framework can do. Start with the shortest useful public API call, then add options, integration examples, and provider details where readers need them. Keep guide openings consistent: title, description, and "After reading this guide, you will know". Explain concepts before summarizing them in a table.
- Document the shared API once. Adding provider support usually updates an existing example or coverage entry, not a new section. Include provider-specific notes only when a difference changes what the reader must configure, call, or handle. Put setup requirements in provider configuration and link to them. Omit interchangeable examples, wire-format details RubyLLM handles, and accounts of implementation or live-test results from user guides.
- Give each standalone AI operation a discoverable guide, and feature new operations in the release overview. Group closely related operations, such as file uploads and downloads, on one page. Do not bury an operation such as `RubyLLM.tokenize` inside a guide about another feature.
- Keep the RubyLLM module overview current and inspect the rendered API method index. RDoc's coverage report does not detect methods it never discovers, including dynamic delegates and Rails macros. Link delegated methods to their shared contract instead of repeating the documentation.
- Front-matter `title` and `description` feed llms.txt and the social-card images. Keep descriptions to one compelling sentence and never use `&`, `<`, or `>` in them.
- Cross-link with `{% link _collection/page.md %}`, never hard-coded URLs. Use the `site.models.*` ids from `docs/_config.yml` in examples so model names stay current.
- A public API change is not done until its docs page changes in the same commit, and `docs/_reference/upgrading.md` records anything that breaks.

<!-- github-automation: release-notes -->
## Releases

This section is maintained account-wide by
[crmne/github-automation](https://github.com/crmne/github-automation) and is
replaced when that policy changes. Do not edit it here. If it does not fit this
repository, say so in a review or issue, and put repository-specific release
steps in a separate section, which takes precedence.