Xinference

Serve LLMs, embeddings, and media models from one Apache-2.0 platform.

Install Repository Compare

Fit.

Catalog snapshot

Platforms: Linux, Macos, Windows, Docker
Operating systems: Linux, macOS, Windows
Ram floor: 8 GB
Gpu: Optional · 8 GB VRAM helps
Docker: Yes
Setup: Intermediate
Deploy as: Local, Self-hosted
Commercial use: Allowed

One-box alternative to juggling Ollama plus separate embedding servers.

Project.

Fetched Sep 12, 2026¹

License: Apache 2.0
Stars: 9.6k¹
Forks: 868¹
Open issues: 7¹
Language: Python¹
Last commit: Sep 12, 2026¹
Tags: servingschedulingopenai-api

About.

Xinference launches, schedules, and serves many models from one platform: LLMs, embeddings, rerankers, image and audio models behind an OpenAI-shaped API with a built-in chat UI. Apache-2.0 and very active.

multi-model serving · built-in UI · GPU scheduling · OpenAI-compatible API

Alternatives.

Nearby.

All LLM infrastructure

LllamafileSingle-file LLM runtime from Mozilla. Download, chmod, run.Compare
LLiteLLMProxy that translates many model APIs into one OpenAI-shaped client.Compare
VvLLMHigh-throughput model server for GPU inference.Compare
Lllama.cppC/C++ runtime that made local GGUF models practical.Compare
EexoLink everyday devices into one cluster and run models too big for any single machine.Compare