Topic hub

GPU scheduling on Kubernetes.

GPU scheduling on Kubernetes is how scarce accelerators are admitted and placed: kueue vs volcano for quota versus gang scheduling, kueue gpu ClusterQueue and LocalQueue cohorts, and gpu scheduling kubernetes priority lanes for inference and training.

01

Coverage

What this hub covers.

GPU scheduling on Kubernetes: quota and admission with Kueue, gang scheduling with Volcano or KAI, and how they fit with inference workloads.

GPU admission control with Kueue quotas, cohorts and preemption is work in progress; repository and evidence will be linked when published.

Admission and quota (Kueue ClusterQueue / LocalQueue / cohorts).

Gang scheduling and placement (Volcano, KAI).

Inference vs training queues and priority lanes.

Preemption and fair sharing.

02

Guides

Related guides.

03

Related cases

Evidence nearby.

Case 07

LLM Infrastructure Runtime

  • Queue
  • vLLM
  • KServe

LLM workloads move faster than traditional platform controls and can quickly become expensive, opaque, and hard to operate

LLM serving with operability signals

Case 19 · Flagship

AI Runtime Platform

  • Python
  • vLLM
  • KServe

AI workloads often start as API calls but become production systems that need identity, policy, scaling, telemetry, and clear ownership

Every model call: policy + trace + budget

04

Related hubs

Nearby topics.

Available for meaningful infrastructure conversations

Build systems that stay calm under pressure.

Available for conversations

Talk infrastructure

Book a working session

Meet ↗

Evidence first

For deep dives

Explore the systems

All cases ↗