SGLang

Fast model serving with KV-cache reuse for agent and reasoning workloads.

Install Repository Compare

Fit.

Catalog snapshot

Platforms: Linux, Macos, Windows, Docker
Operating systems: Linux, macOS, Windows
Ram floor: 32 GB
Gpu: Required · 16 GB VRAM
Docker: Yes
Setup: Advanced
Deploy as: Self-hosted
Commercial use: Allowed

Shines where prompt prefixes repeat; pairs well with agent traffic.

Project.

Fetched Sep 12, 2026¹

License: Apache 2.0
Stars: 35.9k¹
Forks: 8.8k¹
Open issues: 897¹
Language: Python¹
Last commit: Sep 12, 2026¹
Tags: servingkv-cachegpu

About.

SGLang serves models with RadixAttention — KV-cache reuse across requests — which makes it unusually fast for agents and reasoning loops that replay the same prefixes. Apache-2.0, OpenAI-compatible endpoints, and a very active repo.

RadixAttention KV cache · structured generation · OpenAI API · multi-LoRA serving

Alternatives.

Nearby.

All LLM infrastructure

LllamafileSingle-file LLM runtime from Mozilla. Download, chmod, run.Compare
LLiteLLMProxy that translates many model APIs into one OpenAI-shaped client.Compare
VvLLMHigh-throughput model server for GPU inference.Compare
Lllama.cppC/C++ runtime that made local GGUF models practical.Compare
EexoLink everyday devices into one cluster and run models too big for any single machine.Compare