Back to Speakers
Annie Talvasto - CNCF Ambassadir, Azure MVP & Sr. Manager at Upbound at Upbound

Connect

November 23–27, 2026

Annie Talvasto

CNCF Ambassadir, Azure MVP & Sr. Manager at Upbound

Upbound

Session Details

AI Inference Best Practices: What Actually Works in Production

Running a model is easy. Running inference well is a lot harder. What should you actually monitor? When should you batch requests? How do you choose an inference engine? Should you quantize? Scale up or scale out? How do you keep expensive GPUs busy without destroying latency? And what changes once you have more than one model, cluster, or type of accelerator? This session walks through the practical best practices for running AI inference in production, from model and engine selection to GPU utilization, batching, caching, parallelism, routing, autoscaling, observability, and cost. We will look at the patterns that work, the common mistakes that look reasonable but hurt you later, and where the right answer depends on your workload. With demos throughout, you will leave with a concrete checklist for designing, running, and improving a production inference stack.

Full conference agenda

See every session across the AI Agents Conference November 2026 line-up.

View Agenda