Buoyant Releases Kinference: Open-Source Inference Control Plane for Kubernetes and vLLM

Buoyant, the company behind the Linkerd service mesh, has open-sourced Kinference, a dedicated inference control plane designed to orchestrate vLLM workloads on Kubernetes. Kinference provides traffic routing, load balancing, and lifecycle management specifically tuned for the stateful, latency-sensitive nature of LLM inference — problems that generic Kubernetes ingress controllers handle poorly. This fills a real gap for engineering teams running self-hosted models at scale, where request batching, model replica routing, and GPU utilization need tighter coordination than standard service mesh tooling provides. Developers can deploy Kinference alongside existing vLLM setups without replacing their broader Kubernetes infrastructure. For teams moving from prototype to production on open-weight models, this is a significant operational primitive worth evaluating immediately.