TensorRT-LLM
NVIDIA’s kernel-level serving stack for maximum GPU inference throughput.
Fit.
Catalog snapshot
- Platforms
- Linux, Windows, Docker
- Operating systems
- Linux, Windows
- Ram floor
- 32 GB
- Gpu
- Required · 16 GB VRAM
- Docker
- Yes
- Setup
- Advanced
- Deploy as
- Self-hosted
- Commercial use
- Check the license
NVIDIA-only; the build pipeline is the hard part, not the API.
Project.
Fetched 1
About.
TensorRT-LLM is NVIDIA’s serving stack: compiled kernels, in-flight batching, and quantization that push their GPUs to the throughput ceiling. Build-step complexity is real. Apache-based license with NVIDIA terms — read the repo.
TensorRT kernels · in-flight batching · quantization · multi-GPU pipelines
