Skip to content

Search
Categories
Guides
Compare

Site footer

FossfitsOpen-source AI that fits your machine© 2026

Search Categories Alternatives Licenses Platforms Guides Compare Submit a project

Fossfits
/
LLM infrastructure
/
vLLM

V

vLLM

High-throughput model server for GPU inference.

Install Repository Website Compare

Fit.

Catalog snapshot

Platforms: Linux, Docker
Operating systems: Linux
Ram floor: 32 GB
Gpu: Required · 16 GB VRAM
Docker: Yes
Setup: Advanced
Deploy as: Self-hosted
Commercial use: Allowed

Apache 2.0. CUDA is the well-tested path.

Project.

Fetched Sep 12, 2026¹

License: Apache 2.0
Stars: 91.6k¹
Forks: 22.1k¹
Open issues: 2396¹
Language: Python¹
Last commit: Sep 12, 2026¹
Latest release: Sep 1, 2026
Tags: servinggpuopenai-api

About.

vLLM is how you serve a model to more than one user without writing a batcher. It wants NVIDIA GPUs. Not a chat UI.

PagedAttention · OpenAI API · LoRA · multimodal

Alternatives.

OAOpenAI APIPartial

Nearby.

All LLM infrastructure

LllamafileSingle-file LLM runtime from Mozilla. Download, chmod, run.Compare
LLiteLLMProxy that translates many model APIs into one OpenAI-shaped client.Compare
Lllama.cppC/C++ runtime that made local GGUF models practical.Compare
EexoLink everyday devices into one cluster and run models too big for any single machine.Compare
Sshell_gptCLI that runs an LLM as a shell command for scripts, refactors, and answers.Compare