After Kubernetes, what's next

0:02 had to be in Japan for CubeCon.

0:04 And I heard that Japan is booming.

0:07 Yeah, Japan. I mean, [music] it’s one of our largest um you know, CubeCons in in Asia, at least for Japan. It’s one of larger cloud events, too. So, uh we’re very excited to be here.

0:17 I just heard about 1 million cloud native developer. Yeah, 950,000 technically, but yeah, we released a state of cloud native report for Japan that kind of highlights trends in the Japanese market from number of cloud native developers um you know uh percentage developers and companies out there that are requiring uh developers to upskill for AI related stuff. So it’s a great uh set of resources and details for people trying to understand the Japanese uh market which historically at least in cloudnative has um lagged compared to maybe Europe. uh and North America in terms of actually like cloud adoption. So I think in Japan we’re seeing uh a little bit similar patterns from uh kind of AI uh inference uh adoption. So

1:02 yeah and we are at the forefront of a change of paradigma. I have this feeling like uh we are heading toward more agent AI and so on and so forth.

1:12 Yep. For sure. Like you know I always say like agents are kind of the new microservices like they’re a new type of workload. Um it it’s you know Jonathan and I on the keynote stage today tried to reiterate the point that Kubernetes was really designed to really help you know scale distributed systems right we had some innovations like a reconciliation loop with a nice simple API server and you know basically an extensibility mechanism and that like concept has served you know containers and traditional workloads extremely well uh and microservices of course uh agents you are basically a service but they have some different characteristics right they tend to need GPUs they need to go load up uh sometimes a large model so they have some different characteristics

1:58 but at the end of the day you’re dealing with a distributed system things that could go away go wrong fail and Kubernetes and CNCF are built extremely well like our projects and tools are built around managing and making distributed systems scale which essentially agents are they’re just a little bit of different type of distributed system

2:16 wow so it’s not a different animal. Um it’s I I think it’s like yeah it’s it’s a different species like you know it’s a it’s the way to look at it is like it’s you know services are all about being dynamic resilient they could go away right they’re different sizes agents are similar but they’re a little non-deterministics right and they require different resources sometimes they’re uh usually GPU bound but sometimes they also use a bit of CPUs you have to support custom A6 and FGPA sometimes for these things so it has different characteristics but you know everything you need to build a scalable fault tolerant distributed system uh and the stuff that we’ve been doing in CNCF for the last you know 10 years is still extremely relevant for for agents in in in particular. Yeah. And thank you very

3:03 much. And I know that it’s hard to predict because we have been in it’s easy to look backward 10 years what happened. How do you see 10 year in the future?

3:13 Oh man. like you know I I the way it’s hard like it’s a very generic question like I I definitely AI is going to become part of our you know lives right I think you know our personal lives the tools that we use from a productivity basis like I think everyone’s going to have their own personalized agent that probably runs locally in some way maybe it’s in your watch or local computer um so I I think AI will be per pervasive essentially inference is going to become more localized of Of course, we’re going to use stuff, you know, in the cloud, quote unquote. Uh, so, you know, I I just feel like AI will become more pervasive. It’s going to put a lot of demand on the infrastructure. You know, the way I kind of look at it is

3:56 cloud native originally was built for uh essentially human speed, right? You know, for humans working on computers, you like shopping, calling a lift uh through a mobile app. So, built for mobile human experiences and desktop experiences. agents operate significantly faster, you know, have a lot more demand on infrastructure. So, infrastructure is gonna, you know, the need for building out infrastructure continues uh to rise to kind of support this, especially as we get all get our own like personalized agents that maybe help with programming, maybe it helps with scheduling meetings, maybe it helps you, you know, with other aspects of your life, maybe a medical, you know, so all this stuff is going to continue to grow. Like there’s a lot of applications we use, you know, like I book

4:42 uh health appointments via uh an app in the US called one medical and it was pretty simple. It’s like you could talk to your doctor. They recently added, you know, an AI feature to like look at your, you know, medical thing and give you advice and that’s essentially an inference-based service. Imagine like every app we interact with is going to have some type of inference workload and to scale and make that work across everything we use is going to be a lot of of infra. So I I don’t know you know it’s hard to predict the future also outside of like I know CNCF is going to rise to the challenge to support AI and inference workloads. We’re already there. Uh but I think AI is going to become a lot more p personalized for us. I think you’re going to see more local AI. you know, people are going to want to control which data to share uh with

5:28 the AI and how it’s personalized and there’s just going to be a lot of options. It’s kind of like open source. We, you know, had very very limited options in the early days of open source, but now we have how many different open source databases, right? You have obviously you have my SQL and Postgress, but there’s

5:43 tons SQL. Yeah. And so I think with AI, we’re going to have a lot of open, you know, models available for people to choose and so on. I think the normal consumer is not going to care about it. they’re just going to want to use the functionality. But developers, we’re going to have so much choice. It’s just like the number of Linux distros out there. There’s, you know, people like Gen 2, people like Arc Linux, people like, you know, Red Hat, Fedora, whatever, right? We’re going to have a lot of choice, I think, when it comes to AI. And I think all that choice and competition is going to lead to in innovation. So, so hopefully that helps a little bit where things are going. So, I’d say more localized AI infrastructure continues to to grow as, you know, we’re still such in the early days for people to integrate AI and inference in applications that’s just going to become, you know, second nature.

6:30 Everything’s going to have that built in.

6:31 That’s so cool. It’s incredible.

6:33 Yeah.

6:33 And we also the other part of the world, at least for my mind, and I feel at home because I

6:39 Yeah. We have 191 countries contributing to CNCF. And sometimes individuals are like learning about the technology themselves. They’re just like working on a project. Other are working for a company. You could be working for Google. You could be working for Fujitsu. Sometimes you’re working for companies that compete, you know, with each other. We just had a CNCF uh governing board meeting, a board of directors meeting yesterday. And you like in the room you have like Google, Microsoft, you know, Apple, Red Hat, uh you know, Huawei, like they all, you know, both collaborate and and compete uh at the end of the day. And I think that’s what makes uh open- source so beautiful in in many ways because we allow people to share knowledge and build what’s useful but also compete which allows truly allowed like generally when you have ecosystems where you only have one one monopolistic

7:27 winner or one dominant winner. It doesn’t allow for innovation. You need to enable that competition. I think that’s what CNCF does. And we at the end of the day that competition leads to choice for end users and better solutions.

7:40 That’s great. And in this big ecosystem, if you are young talent in technology, what is your the best advice you would love to give?

7:49 So it you know there’s two things. One, um I think it’s always important to know the basics like understanding how Linux works and open source like Kubernetes. I think like learning the fundamentals is always good. It’s it’s akin to like taking a computer science degree and learning like the basics of operating systems. I think that is always going to be useful and help give you a baseline knowledge. But for kids these days and and young folks, I would say set up your own like Hermes agent or set up your own open claw. Like learn how these things work and try to make it useful for you. Uh figure out how to run models locally, right? You know, Olama makes it pretty easy to run, you know, model models locally on your on your, you know, computer, right? You can install GLM

8:34 fairly easily through Lama. So learn how to run inference locally, play with these assistants because, you know, you’re going to be able to be ahead of so many other people that don’t know how the system works. You know, there’s a lot of interesting things of like how you craft a prompt, you know, properly. Uh there’s also tricks that you learn sometimes with AI tools where um you know, sometimes you work with models, you know, models have context windows, right? You know, let’s say a million, you know, you know, tokens or something. What sometimes happens is I’ve noticed that uh if you uh have a lot of history and context for uh a model, sometimes your result uh becomes worse. It’s like sometimes you have to flush the history,

9:20 right? And so like there there’s all these things that you kind of, you know, learn by by doing and experimenting with. So for young folks I would say learn the fundamentals like Linux operating skills always useful understanding how networking works but then also just like run your own you know local model but like have your own agent. Uh this is to me akin to what happened when I was uh growing up with computers where you like basically learn how to compile and build your own Linux distribution how to like set everything up.

9:48 I remember building the kernel.

9:50 Yeah. the make config manually one parameter I’ve done that so many times and you learned a ton about how things you know work right and I think the same thing applies for agents learn how to run a model locally learn how to you know customize an agent like all this stuff is very very useful and that’s what I would recommend to to kids like always tinker always try to run locally like build like using your own agent uh off of like openclaw Hermes with maybe an open source or open open weights model and tweaking it is almost equivalent to building and customizing your own Linux distribution.

10:27 Wow. Love it.

10:29 Thank you so much for your work and really appreciate your time together

10:31 and anytime we’ll go. So, thank you.

10:33 Thank you so much. Thank you, Chris. All right.