PyTorch, vLLM and the brain behind agents

Why this conversation

Agents get the attention at AGNTCon, but each runs on a model, and models come from open source software. Jonathan Bryce, Executive Director of Cloud & Infrastructure at the Linux Foundation, had given a short keynote that morning on the PyTorch community and the PyTorch Foundation, and he arrived in a PyTorch t-shirt. The host recorded with him in San Jose on Friday 23 October, the second day of AGNTCon + MCPCon North America. That keynote and the rest of the Friday are in AGNTCon + MCPCon North America 2026: day two.

Where the models come from

AI may be two letters, he says, but it covers many things. Agents and harnesses consume tokens and models, and the PyTorch Foundation hosts the open source projects that create and serve those models: the brain for the agent. There are six: PyTorch, vLLM, Ray, DeepSpeed, Helion and Safetensors. Of these, PyTorch and vLLM matter most for building the brain. With PyTorch you build the pipeline, feed the data in and get a model out. vLLM is the inference engine: load a model on a GPU and it answers requests by running them through the model.

The host uses vLLM for distributed inference, when a model outgrows one GPU or server. To Jonathan, PyTorch and vLLM are the deepest technology in the AI stack. Through the Cloud Native Computing Foundation, where Kubernetes started, he sees both moving into clusters: inference must be distributed, training runs even more so because of the data, and cloud native frameworks orchestrate the work.

From chatbots to agents

PyTorch is seven or eight years old and has moved fast all along. What changed in roughly the past year is that adoption exploded, into completely new uses. First came the chatbot era: ChatGPT, then Claude from Anthropic, then many open weights models. “The sort of standard interaction mode was I talk to a chatbot, the chatbot gives me a response.” Now you give an agent a goal, it calls the LLM hundreds or thousands of times to reach it, and it sends sub-agents off to browse the web, write code or do math.

The host wondered whether that is brute force. Early agent frameworks, Jonathan says, were aimed at frontier models, which are expensive when you pay by the token. An enterprise with millions of customer interactions has to make the work efficient: fast, affordable and good. The cutting edge, long found at the model layer, now includes a whole set of tools for starting agents in sandboxes, kagent among them, and keeping them contained, while Kubernetes itself is changing at its core. Earlier that week, at the PyTorch conference, folks from Google, Red Hat and IBM showed off a vLLM deployment over both GPUs and TPUs that managed 1,000 agents inside a cloud native environment.

No room for premature optimization

Software engineers warn against premature optimization: polishing code before you have the users to benefit from it. Jonathan does not see any of that in AI. Adoption has gone vertical and the industry is catching up. The real limits are physical, power and data center space, and GPUs cannot be built, bought and brought online as fast as the demand. Software is how the industry gets more capacity out of the same footprint. “So software is playing a really critical role in that.”

Experts, and the start of a career

The host brought up an exchange with Tim O’Reilly on what experts are for once intelligence is plentiful. Jonathan points to the jobs that were predicted to disappear over the last two years: there are now more of those roles, and more demand for experienced software engineers, than ever. Parts of the employment cycle are being disrupted, as happened with the internet. Simple economics, though, says that when each person produces more, you want more people, because more value gets created.

The five- or ten-year picture is still unclear, and he expects careers to change most at their beginning. When he first did software, in the 1990s, training went down to transistors and assembler. Then the internet arrived with higher-level languages and frameworks that made building e-commerce much faster, and universities needed time to adjust. With agents writing most of the code, expertise means breaking a problem down and describing the outcome you really want, not knowing a language’s syntax. “So really that understanding, that analysis, that type of expertise I think is where the value is going to lie.”

When it comes out

The episode is edited and in the release order. Like every episode it goes out on YouTube first, then on Apple Podcasts, Spotify and Amazon Music.