Recap · PyTorch Conference 2026 · San Jose
Engineers from Red Hat, IBM, NVIDIA, Mistral AI, AWS, Google, Meta and more took the stage in San Jose to show how vLLM is making AI faster and cheaper to run.
Speakers from
Keynote
vLLM's co-creator on where open-source inference is heading, and how vLLM keeps making AI cheaper and faster to run.
See the keynote listing →Watch: Simon Mo interview on vLLM (AgentGlass) →Three themes
01
Reading a prompt and writing an answer need different things from a GPU. Teams showed how running them on separate machines, and passing memory between them, serves more people faster.
02
One vLLM model, many chips. Speakers showed the same model code running on GPUs, Google TPUs, AWS Trainium, IBM Spyre and Arm CPUs, with no custom versions for each.
03
Faster kernels, quicker model loading and lighter maths. Small gains across the stack that add up to quicker answers at lower cost.
How it fits together
01
A request comes in
A long prompt arrives.
02
Prefill reads it
Every token at once. Compute-heavy.
03
Notes are handed off
The KV cache moves to a separate set of GPUs.
04
Decode writes back
One token at a time. Memory-heavy.
With disaggregated serving, vLLM can split a request in two: one set of GPUs reads the prompt, then hands its notes (the KV cache) to another set that writes the answer.
Why it matters: the first word arrives fast, and the rest keeps a steady pace, even when traffic spikes.
Sessions
Community
Contributing to Inference OSS That Won't Stand Still: A BoF on vLLM, llm-d, and the Moving Target
Maroon Ayoub, Nili Guy
Red Hat, IBM
An open discussion on how to contribute to vLLM and llm-d while both projects keep changing quickly.
View session →
Speed work
Portable Paged Attention: From Triton to Helion
Burkhard Ringlein
IBM Research
Rewriting vLLM's core memory kernel, paged attention, in Helion so one version runs well across different hardware.
View session →
Any hardware
PyTorch-Native LLM Serving on TPU: SGLang and vLLM
Colin Taylor, Qi Zhou, Angela Yi
Meta, Google, Meta
A PyTorch-native way to serve vLLM and SGLang on Google's TPU chips, so the same model code runs beyond GPUs.
View session →
Splitting the work
State-of-the-Art KV Transfer for Disaggregated LLM Serving in vLLM
Nicolò Lucchesi, Sunita Nadampalli, Zhanqiu Hu
Mistral AI, Amazon, Red Hat
How vLLM moves its working memory (the KV cache) between separate reading and writing machines. The speakers report a faster first token, and that splitting the work beat running both on one machine.
View session →
Get involved
Join the community on GitHub and Slack, ship your first pull request, or just come say hi.