vLLM

Fast, easy-to-use open-source library for LLM inference and serving, with an OpenAI-compatible server.

Infrastructure by vLLM project

IndexedApache-2.0★ 95,320

Install

Pick your client. Read the repository before you give any tool access to your data.

pip install vllm

Get the weekly diff

The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.

About

vLLM is a library for LLM inference and serving, first developed in UC Berkeley's Sky Computing Lab and now built by a large open-source community. It supports quantization, tensor, pipeline and expert parallelism, structured outputs, tool calling and multi-LoRA serving. Its OpenAI-compatible server also offers Anthropic Messages API and gRPC support.

Use cases: model-inference

Reviews

Write a review

No reviews yet. Installed it? Reviews come from verified installers only. How reviews work

Fact sheet

Category: Inference · open source server
Self-host: Yesraw.githubusercontent.com
Open source: Yesraw.githubusercontent.com
Protocols: openai-compatible, grpcraw.githubusercontent.com

Facts read from the vendor's own pages by claude-haiku-5-5 on Oct 24, 2026; a fact is shown only with the page it came from.

Signals

Stars: 95,320
Forks: 23,894
Open issues: 2,709
Last commit: Oct 24, 2026 (3 days ago)
Last release: Oct 18, 2026 (10 days ago)

From GitHub (fetched Oct 24, 2026).

Details

Packages: pypi vllm
License: Apache-2.0
Repository: github.com/vllm-project/vllm
Website: vllm.ai
Docs: docs.vllm.ai
First seen: Oct 24, 2026
Last checked: Oct 24, 2026
Source: README Website Docs github-graphql

Cite this page

vLLM. Agent Mart. https://agentmart.dev/infrastructure/vllm (last updated 2026-10-24)