llama.cpp
Plain C/C++ LLM inference engine that runs quantized models on many hardware backends, with an OpenAI-compatible server.
Infrastructure by ggml.ai
IndexedMIT★ 132,228
Get started
Setup steps are in the repository. Website · Repository
About
llama.cpp runs LLM and vision-language inference in plain C/C++ with no dependencies, locally and in the cloud. It supports 1.5 to 8-bit integer quantization, CUDA, Metal, Vulkan and SYCL backends, and CPU plus GPU hybrid inference for models larger than VRAM. The llama serve command launches an OpenAI-compatible API server.
Use cases: model-inference
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Fact sheet
- Category
- Inference · local
- Self-host
- Yesraw.githubusercontent.com
- Open source
- Yesraw.githubusercontent.com
- Protocols
- openai-compatibleraw.githubusercontent.com
Facts read from the vendor's own pages by claude-haiku-5-5 on Oct 24, 2026; a fact is shown only with the page it came from.
Signals
- Stars
- 132,228
- Forks
- 24,765
- Open issues
- 800
- Last commit
- Oct 24, 2026 (3 days ago)
- Last release
- Oct 15, 2026 (12 days ago)
From GitHub (fetched Oct 24, 2026).
Details
- License
- MIT
- Repository
- github.com/ggml-org/llama.cpp
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- READMEgithub-graphql
Cite this page
llama.cpp. Agent Mart. https://agentmart.dev/infrastructure/llama-cpp (last updated 2026-10-24)