“Model inference”

Unified inference platform for deploying and scaling AI models on any cloud, with an open-source Python serving framework.

Inference and training platform for hosted and dedicated models, deployable in your cloud or Baseten's.

Open-source platform for serving LLM, embedding and multimodal models on your own premises or in the cloud.

Managed Ray platform for distributed training, batch inference and data processing on GPU clusters.

Plain C/C++ LLM inference engine that runs quantized models on many hardware backends, with an OpenAI-compatible server.

Containerized inference microservices for foundation models, sold as part of NVIDIA AI Enterprise and deployable on any cloud or data center.

Open-source inference framework for LLMs and multimodal models, built for agentic workloads and large-scale serving.

Runs open models locally or in the cloud, with a REST API and Python and JavaScript libraries.

Open-source AI runtime that runs text, voice, image and agent workloads on your own hardware behind OpenAI-compatible APIs.

Cloud platform for deploying and observing voice, video and physical AI agents, built on an open-source agents framework.

Open source gateway for HTTP, gRPC, LLM, MCP and agent-to-agent traffic in one data plane.

Open-source platform for search, ranking and ML inference over vectors, tensors, text and structured data.

Inference cloud for open-weight and custom models, with 99.99% uptime SLAs and dedicated GPU endpoints.

Open source control plane for AI and agent traffic, powered by Envoy, with one OpenAI-compatible API.

Inference cloud for large open models on SambaNova dataflow hardware, with per-token pricing.

Open-source library that optimizes LLM and visual generation inference with specialized kernels and a Python framework.

Inference and fine-tuning platform for open-source models, on serverless per-token or dedicated GPU deployments.

Desktop app for running open LLMs locally, with a built-in agent called Bionic.