All Tools
K
Dev ToolsFreeOpen Source
KNATIVE
Scale-to-zero serverless compute on Kubernetes
Apache-2.0
ABOUT
Model servers and inference APIs on Kubernetes typically stay at least one replica hot, burning GPU and CPU even with no traffic, and rollouts require custom ingress and canary wiring. Knative Serving gives each service a URL, concurrency-based autoscaling including scale-to-zero, and first-class revisions so teams can split traffic across model versions without writing their own autoscaler or ingress controller.
INTEGRATION GUIDE
1. Host model inference APIs that scale to zero overnight and scale out on burst traffic
2. Canary a new model revision by splitting a percentage of requests to the new Knative revision
3. Serve embedding or RAG backends as request-driven Knative services with concurrency limits
4. Run event-driven preprocessing jobs that start from zero when a CloudEvent arrives
5. Replace custom HPA plus ingress glue with Knative routes for GPU-backed model endpoints
TAGS
kubernetesserverlessscale-to-zeroservingmlopstraffic-splittingcncf