TensorRT-LLM
Open-source library that optimizes LLM and visual generation inference with specialized kernels and a Python framework.
Infrastructure by NVIDIA
Install
Pick your client. Read the repository before you give any tool access to your data.
pip install tensorrt-llm
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Use cases: model-inference
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Fact sheet
- Category
- Inference · open source server
- Self-host
- Yesnvidia.github.io
- Open source
- Yesraw.githubusercontent.com
- Protocols
- openai-compatible, mcpnvidia.github.io
Facts read from the vendor's own pages by claude-haiku-5-5 on Oct 24, 2026; a fact is shown only with the page it came from.
Signals
- Stars
- 14,937
- Forks
- 2,862
- Open issues
- 679
- Last commit
- Oct 24, 2026 (4 days ago)
- Last release
- Apr 20, 2026 (6 months ago)
From GitHub (fetched Oct 24, 2026).
Details
- Packages
- pypi
tensorrt-llm - License
- Other
- Repository
- github.com/NVIDIA/TensorRT-LLM
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
More from NVIDIA
- 0.745score
cdf-video-data-augmentation
NVIDIA
Run the CDF Video Data Augmentation Airflow workflow on Kubernetes or OSMO to create Cosmos video-to-video variants and auto-label both source and generated…
SkillAFreshness: A★ 3.7KClaude Code +9 - 0.745score
nvidia-skill-finder
NVIDIA
Use for NVIDIA-related requests where an NVIDIA skill might help, even if the user did not ask for a skill.
SkillAFreshness: A★ 3.7KClaude Code +9 - 0.745score
cdf-image-attribute-augmentation
NVIDIA
Run the CDF Orchestration Image Attribute Augmentation DAG on Kubernetes - person-crop clothing augmentation, attribute search, and augmented dataset…
SkillAFreshness: A★ 3.7KClaude Code +9 - 0.745score
cdf-orchestration-preflight
NVIDIA
Set up missing Linux prerequisites and audit whether a Cosmos Data Factory (CDF) Orchestration checkout and its Kubernetes or OSMO environment are ready to…
SkillAFreshness: A★ 3.7KClaude Code +9 - 0.745score
ethos-write
NVIDIA
Writes an AI agent's ethos.md — a durable snapshot of its purpose, intended behavior, boundaries, and success criteria — from the answers gathered by…
SkillAFreshness: A★ 3.7KClaude Code +9 - 0.745score
cdf-autonomous-vehicle-dataset-generation
NVIDIA
Prepare payloads, run and monitor the Autonomous Vehicle Dataset Generation Airflow workflow, and inspect results for driving-clip retrieval and Cosmos3-Nano…
SkillAFreshness: A★ 3.7KClaude Code +9
Cite this page
TensorRT-LLM. Agent Mart. https://agentmart.dev/infrastructure/tensorrt-llm (last updated 2026-10-24)