The video focuses on the intersection of artificial intelligence, particularly reinforcement learning, and industrial applications. Jim Gao, the founder and CEO of Phaedra, discusses his journey from leading DeepMind Energy to using AI technology to optimize Google's data centers, achieving significant energy savings. The discussion explores how reinforcement learning, a type of AI, was implemented and its potential future impacts on various industrial applications.

Jim shares his insights and experiences, elaborating on how AI goes beyond mere automation to offer creative solutions and new knowledge. He discusses the pivotal role AI could play in handling complex, dynamic systems, particularly in industries such as data management and energy optimization. The conversation underscores AI's potential to revolutionize sectors through its ability to adapt and self-learn in diverse real-world environments, paving the way for transformative innovations.

Main takeaways from the video:

Please remember to turn on the CC button to view the subtitles.

Key Vocabularies and Common Phrases:

1. conflation [kənˈfleɪʃən] - (noun) - The merging or blending of two or more ideas or concepts into one. - Synonyms: (amalgamation, combination, fusion)

There's a conflation between AI and automation.

2. reinforcement learning [ˌriːɪnˈfɔːrsmənt ˈlɜːrnɪŋ] - (noun) - An area of machine learning where software agents learn to take actions in an environment so as to maximize a notion of cumulative reward. - Synonyms: (adaptive learning, autonomous learning, self-learning)

reinforcement learning could be used to control and optimize Google's data centers.

3. impetus [ˈɪmpɪtəs] - (noun) - A force or influence that makes something happen or happen more quickly. - Synonyms: (stimulus, spur, momentum)

Really the impetus was something called Alphago.

4. optimization [ˌɒptɪˈmaɪzeɪʃən] - (noun) - The process or methodology of making something as functional or effective as possible. - Synonyms: (enhancement, refinement, improvement)

Really, what we're saying is, as long as we can map the problem we're trying to solve into reinforcement learning framework, which really, from a mathematical perspective, what we're saying is we're solving a constraint optimization problem

5. constraints [kənˈstreɪnts] - (noun) - Limitations or restrictions that must be considered when making a decision or reaching a conclusion. - Synonyms: (limitations, restrictions, parameters)

They need to know what are the constraints they have to stay within.

6. pervasive [pərˈveɪsɪv] - (adjective) - Spreading widely throughout an area or a group of people. - Synonyms: (widespread, prevalent, ubiquitous)

But fundamentally, human intuition plus hard coded controls, logic is still limited when you talk about this degree of complexity.

7. capacity [kəˈpæsɪti] - (noun) - The maximum amount or number that can be contained or accommodated. - Synonyms: (capability, volume, reach)

What is the problem with that? The problem is that because we do not know how much energy we're going to generate, you now have all this wasted excess capacity in reserve

8. scalable [ˈskeɪləbl] - (adjective) - Able to be expanded or made larger. - Synonyms: (expandable, flexible, adjustable)

So there certainly weren't designed for this, but what we do instead is we ride on top of the existing control system.

9. heuristic [hjuːˈrɪstɪk] - (noun) - A practical approach to problem-solving that employs shortcuts to produce good-enough solutions given a limited timeframe or resources. - Synonyms: (shortcut, rule of thumb, guideline)

There's a hard coded layer of rules and heuristics.

10. infrastructure [ˈɪnfrəˌstrʌktʃər] - (noun) - The basic physical systems and structures needed for the operation of a society or enterprise. - Synonyms: (framework, foundation, base)

But also, no one knew at the time, is it even possible to use AI from the cloud to control big ass infrastructure? Step number one was do the pilot

Phaidras Jim Gao on Building the Fourth Industrial Revolution with Reinforcement Learning

A lot of times when we talk about AI, both in the valley and elsewhere, I think there's a conflation between AI and automation. AI can absolutely automate things. There's no doubt about that, especially routine things. But I think that honestly undersells the real promise of AI. I think the real promise of AI is what Dem is. The CEO of DeepMind calls Aihdenhe creativity. Right. It's the ability to acquire knowledge that did not exist before. Right. And I, of course, experienced this firsthand. The reason why I'm such a true believer in the technology is because, again, I was the expert who helped design the system, but this very AI agent that we created is telling me new things about the system that I didn't know about before. Right. And that's a very, very powerful feeling.

Jim Gao, founder and CEO of Phaedra, was previously the leader of DeepMind Energy, one of the first and only Alphago style reinforcement learning applications in the wild. DeepMind energy used reinforcement learning to manage Google's data centers and drove some staggering metrics, including 40% energy savings. We're excited to ask Jim about reinforcement learning in the industrial world and to learn more from him about what other real world applications are poised to be transformed next by deep reinforcement learning.

So Phaedra is an AI company. Of course, fundamentally, we are an AI automation company. So what we do is we use a type of AI known as reinforcement learning to directly control and operate our customers very large, mission critical industrial facilities. So in practice, these AI agents, they act as virtual plan operators, virtual members of the plan operations team.

Let's go back in time and talk about the journey that led to this journey. And I believe that you once sent an email with the subject line, reinforcement learning plus data centers equals awesome. Can you question mark? Yes. Awesome. Sorry, sorry. reinforcement learning plus data centers equals awesome. Can you tell us who did you send that email to? Why did you send that email? What was on your mind at the time? And then, of course, what did that lead to?

The reason why there was a question mark is because it was generally an unknown if the combination of reinforcement learning with industrial facilities would actually be awesome. So that was an email that I had sent to a person named Mustafa Suleiman, who would later become my boss at DeepMind. And really the impetus was something called Alphago. So to set the stage properly, I had been experimenting as part of my famed the 20% time at Google with machine learning technologies. And it was actually a very specific course, introduction to machine learning by Andrew Ng on Coursera, that had just come out.

This is back in 2013, Myju. I think I was like the second cohort or something, and that class had completely changed my life. I taught myself how to program and just started tinkering around with machine learning on the side. Right. It was a very interesting technology.

And your background was mechanical engineering and environmental systems type things. That's absolutely right. My responsibility at the time was to, one, help Google design and operate their very large data centers. And once these very large data centers, which consume enormous amounts of energy, were built, we, of course, shifted our focus to operating these complex industrial systems in the most energy efficient way possible because they use billions of dollars in electricity.

So that was kind of the background. I was already tinkering around with machine learning technologies on the side to analyze the enormous amounts of data that Google's data centers were generating. In 2016, Alphago came out, and I was one of hundreds of millions of people around the world watching. It was like 03:00 a.m. in the Bay Area or something, and I found it absolutely captivating and to the point where I sent an email to moose describing this idea that if DeepMind could beat the smartest, most intelligent people in the world at complex games like Go, then surely we can train these same AI agents to play a very different game that I'm familiar with called let's optimize the pv.

The power usage effectiveness of Google's data centers, that was the context for that email. And I remember internally the way I pitched it to Google's leadership, specifically Joe Caba, who leads Google's data centers and Ors, was I showed a picture of a go board on one side and a video game controller, like an Xbox controller on the other, and I'm like, look, there are objective functions that we're trying to minimize or maximize. There are concrete knobs and levers, so actions that we can control, there are constraints that we have to stay within.

And all of this happens within a very measurable environment. I think reinforcement, learning and operating large, complex industrial systems are actually one and the same thing. So that was the original kernel of insight, I guess, that inspired it all.

The reinforcement learning systems, they need KPI's to optimize for it. They need to know how good or bad an action is. They obviously need things to control, and they need to know what are the constraints they have to stay within. Really, what we're saying is, as long as we can map the problem we're trying to solve into reinforcement learning framework, which really, from a mathematical perspective, what we're saying is we're solving a constraint optimization problem. If you can map the constraint optimization problem, if you can define it and map it to the underlying data, then it should be able to be solved using reinforcement learning.

So that's very much the lens through which we look at things, at Phaedra as well. And to take it one step further, we often talk about how reinforcement learning and controls and optimization are two wildly different fields historically that have somehow independently converged to the same area. They're two very similar concepts. Well, we've been calling them by different names this whole time. So you've had almost these independent evolutions, different ways of tackling the same problem. And Phaedra is really kind of the intersection of both of these.

So you sent the email to Mustafa two weeks later. Moose had actually flown out to Mountain View, where I was working at the time, on Google campus with a team of DeepMind folks. And we actually started mapping out exactly how reinforcement learning could be used to control and optimize Google's data centers. So that actually kicked off the original partnership between Google and DeepMind around the application reinforcement learning for the data center work. It was very, very fascinating. But, you know, most importantly is actually also how I met one of my two other co founders.

Right? So Veda was one of the original engineers on the Alphago project. So he had gone to go to, you know, he went to Seoul, South Korea, right? And, you know, he actually got to meet Lee Sedo and Larry Page and all you know, all this fun stuff. And after Alphago, he came back to the cities, or rather to the UK, and he was wondering, well, what is my next big thing going to be? And I managed to convince Beta, like, hey, what if we applied self learning frameworks like Alphago to control and optimize Google's data center?

So that's actually how I started working with my co founder, Beta. Did people think he was going to work? Or was it like, this is a crazy moonshot, let's just try. But who knows? I didn't even know if it was going to work.

Conceptually, it made sense in my mind, right? I'm like, hey, this is a operating a data center is just a different game to play, right? And there's all kinds of different games in the industrial world, right? Maybe the game is maximize energy efficiency, maybe the game is minimized water consumption, maybe the game is maximize the yield of a factory, right? But there's all these games that we're constantly playing, right? So in my mind it made sense.

But to answer your question directly, no, I had no idea if it was going to work. I still vividly remember to this day when we turn on the AI system and we watch the energy just drop. And it was so surprising for two reasons. Number one, well, we had designed the system. I played a role in designing that very mechanical system that the AI was now controlling, optimizing. So in theory, I'm literally supposed to be the subject matter expert who knows everything about these systems, but the AI is teaching me things that I didn't know about the system I helped design in the first place, right?

And two, the moves that the AI was making were just very counterintuitive. When we looked at the decisions that were coming out, the plan operators and I, we were sitting in a giant cornfield in Iowa where we would like to put its data centers, and we were looking at the decisions and we thought to ourselves, there's no way this is right, this AI sucks. I learned the wrong thing, but we're here anyway, so let's try and what the AI is saying. And we tried it and it worked and we saw the energy plummet.

So I think that was kind of when I became a believer in this technology, that fundamentally this technology is creative. It helps us discover new knowledge that didn't exist before from raw data. Was there a performance trade off or is this just a straight up Pareto gain like performance held? And that's a question.

No, it was a. It respected exactly the same constraints, and the plant operators and engineers had already put in place. So this is pure gain respecting exactly the same temperature profiles, exactly the same constraints around how quickly you can turn on and off a chiller, minimum pump, VFD speeds, all that sort of stuff.

So this is pure optimization, pure gain, which I think is one of those crazy things, like, we don't really expect. Like, usually when you think about energy efficiency, for example, right? Like, in the world that I come from, people usually think about expensive capex. Like, oh, we got to rip out the chillers. We got to buy a bunch of new chillers from Johnson controls and trains or whatever, and then we have to install them. So they're like hardware efficiency gains, right? But you don't really think about, like, pure software, like, data driven efficiency gains, right? And I think that's part of what was surprising for us.

Can you walk us through the before and after? Maybe before what you all implemented was this industrial control systems. Was this manual plant operators turning knobs. Like, how did this work before and then after? Yeah, it's a great question. So let me set the stage for, you know, for. For folks who are not as familiar with, like, large industrial facilities, right? So they're very modern. Industrial facilities are very, very complex, right? There's all kinds of machines that people are operating and controlling, right? So, you know, I often, you know, tell folks to do, like, a simple thought experiment. So imagine you have just ten machines you're controlling.

So say they're like pumps, right? And each one of those machines has ten possible set point values, so ten modes associated with it. So think something like 10% pump speed, 20% pump speed, 30% pump speed, et cetera, right? Then in this very simple toy example, you have ten raised to the ten or 10 billion different permutations for how you can operate your toy system, right? So then the question becomes, well, at any given point, what is the most optimal way of operating your toy system? And by the way, these are dynamic systems, right? So the it load is changing, the weather is fluctuating, right? You know, the people operating these systems are changing, the pipes are corroding, the heat changes are fouling, right?

So the point is, these are very complex dynamic systems. Real world systems have a lot more than ten machines, and each machine has a lot more than ten set points, right? So you can start seeing why technologies like alphago, which managed to navigate MX complexity, are helpful over here. It also helps explain why there's often so much room for optimization in the first place. Because there's so much complexity, right? Like, if you think about the total action space, right? Like all the possible actions within a modern data center, for example, right? Because of risk averseness, but also because of hard coded rules and heuristics, right?

We've only ever explored, like, 0.001% of all the possible ways that you could operate that system. So then the question becomes, what is in this ninety nine point nine nine nine nine nine percent of the action space we've never explored? Surely there are more optimal ways of operating the system than what we've done historically. So it's kind of an intuitive explanation, hopefully, of why there can be such large efficiency improvements in the first place that are undiscovered.

And the way that we operate these facilities is constrained by a mixture of hard coded controls, logic, right? So don't get me wrong, these are automated systems today already, right? They're just not opto, you know, automated intelligently, I would argue. Right? And, you know, there is a healthy mixture of human intuition as well, right? Where we have people like myself or plant operators who are constantly monitoring the system, who are like, nudging the system, adjusting things, or setting, adjusting the rules for that system, the constraints that the system has to operate within. But fundamentally, human intuition plus hard coded controls, logic is still limited when you talk about this degree of complexity, right? Yeah.

Can you talk to us about the key results? So you saw the energy levels drop immediately, but what results were you able to drive for Google initially? So there's two types of results. For Google in particular, there was a results from the pilot. So in 2016, we announced the results of the pilot. Now, the pilot was done on a couple of data centers, but fundamentally it was not an autonomous control system. What I mean by this is it was the AI generating recommendations for human experts like myself to manually review and implement.

And of course, we didn't want to jump straight to taking our hands off the steering wheel because it's a new, novel technology. But also, no one knew at the time, is it even possible to use AI from the cloud to control big ass infrastructure? Step number one was do the pilot. The AI generated recommendations. That's where we saw really steep 40% energy savings.

That experience taught us, hey, we think there's something real over here. We should actually just let the AI control things directly to get the value automatically. And also, quite frankly, the plan operators were getting tired of checking their email every 15 minutes, waiting for the AI to tell them what to do manually from that things. They had better things to do. We actually decided, and rather auras and Joe decided, hey, it's time to go to a fully automated system.

This was total uncharted territory at that point. Forget about can AI control things we didn't even know? Is it possible to control machines from the cloud, like huge industrial infrastructure in the cloud? Because to our knowledge, no one had done it before, right? Is it fair to assume that a lot of the hardware, a lot of those machines are things that Google built from scratch? Or does Google use a decent amount of commercially available data sort of stuff? It's a mixture of both.

Obviously Google but does a lot of things in house, right? But it doesn't manufacture, like, chillers and that sort of hardware. So Google does buy off the shelf hardware, but there's a lot of, like, modifications on Google, specific things, you know, that. That we did, right, for example, like, you know, programming some of our own plC's or, you know, making modifications to the building management system.

So, like, the software control layer looked quite different. That was done in house. But, you know, I still remember, like, very vividly, actually, to this day. Veda and I, we were standing in a large, like, 90 megawatt data center, right? So it was a fairly large data center. And Veda is like, typing away in his MacBook, right? He submits the PR gets merged, and all of a sudden, this huge honking, huge chiller that is the size of a bus that we're standing right next to roars to life.

And as it's coming to life, right, like, the ground is shaking vigorously and we're like, oh, my God, with a few keystrokes on its MacBook, we just turned on this enormous chiller. And that was the very first data point to us. Like, yes, it is possible to control things from the cloud. So now the next question is, how do we control things intelligently from the cloud, right, where all the compute resources live?

What were your biggest takeaways from that experience? You mentioned the creativity of the machine. Any other big takeaways or learnings? Yeah. So the creativity is absolutely a big one. I think just to elaborate on that briefly, a lot of times when we talk about AI, both in the valley and elsewhere, I think there's a conflation between AI and automation. AI can absolutely automate things. There's no doubt about that, especially routine things. But I think that honestly undersells the real promise of AI. I think the real promise of AI is what Dem is the CEO of. Of DeepMind calls AI creativity. It's the ability to acquire knowledge that did not exist before, right?

And I, of course, experienced this firsthand the reason why I'm such a true believer in the technology is because, again, I was the expert who helped design the system, but this very AI agent that we created is telling me new things about the system that I didn't know about before, right? And that's a very, very powerful feeling. It's kind of like when, if you think back to Alphago, right? Like Lisa do was the best in his field at go. He was the world champion for a decade, right? He was at the top, and his Elo rating was just something outrageous. It was like 2100 or something. It was outrageously high, but it had flat, flatline, right?

So for a full decade, his Elo rating was the same, and there was no one to challenge him because he was already at the top. So once he hit the top, he just kind of plateaued. And then after Alphago happened, and he actually got to play against Alphago privately a few more times because DeepMind had let him continue interacting with the system. What happened? For the first time in a decade, his elo waiting started climbing. This is what I mean when I say that I think the real power of AI is helping us discover knowledge that we didn't necessarily know about before.

And where you're going to see the most, um, gain from that. It's not going to be in routine automation things, right? Like call centers or whatever, right? Uh, it's going to be, I think, in very, very complex areas, right? Like areas where human intuition is insufficient because of immense complexity, but that is yet underpinned by data. So that's why you're seeing such things, like protein folding, for example, maybe. Yeah, that. That's fucking extraordinary, right?

And it's those areas of just, like, massive permutational complexity underpinned by data. That's where I think we're going to see some of the most interesting companies and products. So that was a rather long tangent. So, one, creativity is something that I learned. The other one lesson that my co founders and I learned is really around. If you want real impact, you got to turn the technology into a product. And this is actually the core reason why we decided to leave DeepMind and train technologies to start Phaedra.

Right? Like, over and over again, we were seeing the technologies that we were helping to develop at DeepMind were just extraordinary, right? I mean, they were. They were achieving crazy things, like with protein folding. But the problem is, you know, in order for the technology to make the most impact, you have to get into the real world. People have to actually use it. And that fundamentally means we're talking about a product. Turning a technology into a product is like, I mean, you guys would know much better than myself. It's like 100 fold, 1000 fold more work. Right?

And, you know, and that, for us, you know, led us to the conclusion that, like, hey, it's, you know, it's time to leave, right? It's time to actually start a company that. That creates these intelligent virtual plan operators, these intelligent AI agents as a real product. Let's talk more about that. For what you're building now. How much of what you learned at Google DeepMind sort of translates directly into what you're doing now?

How much is new? Because the environments are different, the customers are different. There's something different about it. I think the most important thing that we learned from our Google DeepMind experience is that it's possible this is not a crazy. And that isn't to downplay what we learned. It's actually a huge thing that it is, in fact, possible to use closed loop learning systems, like reinforcement learning, to drive very large improvements in complex industrial facilities.

It hadn't been done before, to our knowledge, and that was a massive proof point. I think the problem, though, is that the real world is quite diverse. Every single customer is diverse. And especially when you talk about industrial facilities, every industrial facility is a snowflake. So for us, I mean, the learnings have just been, like, massive since we left Google and DeepMind, right. Because every time we onboard a new customer, we're learning something new about, like how equipment are connected or some product gap that we didn't know about before that needs to be fixed.

Right? Or new ways that data can break. At this point, I can tell you, like, 100 different ways that data associated with mission critical cooling systems can break. Probably not the most interesting party topic for most folks, but, you know, I personally find it quite interesting. But, yeah, there's certainly been quite a lot of learnings in that regard.

Are the folks you're talking to, are they ready to let the technology take over the system and, you know, let the culling system just start going? Yeah, I mean, yes and no. Right. And that actually gets back to your earlier question, Pat, as well, right, about, like, the specific learnings from Google. I mean, when I look back, I think what we helped Pioneer at Google and DeepMind could only have been done at a company like Google. The reason why I say that is because Google is a very forward leaning company.

But also, one of the things I've learned is that Google is absolutely an anomaly when it comes to how much data it has, and the pristine quality of the data and the ease of access of the data. Google is fundamentally a data analytics company, and as such, it invested all this infrastructure in high quality, high availability data on which you can do things like real time intelligence applications, like what we were doing. And there are many other examples of this within Google and DeepMind. Having left the nest, one of our rude awakenings was Google is definitely anomaly.

And I mean, gosh, like, everyone is in various stages of their AI journey. Google is certainly on one extreme. We have customers who've encountered where, like, forget about real time intelligence, they're like, they're not capturing the data in the first place, right? Or they may be sensorizing. In the industries we work with, like pharmaceuticals and district cooling and especially data centers, almost always the customer is sensorized, right, because these are billion dollar facilities.

So of course it makes sense to throw a million dollars worth of sensors on it. But that doesn't, just because you sensorite, doesn't mean that you're storing the data. A lot of customers of ours aren't necessarily storing the data beyond, like, 90 days or six months or a year or whatever. And they might cite some reasons like, well, it's costly to store the data, or the more commonly, we're not using the data for anything, which is a true statement.

A lot of our industrial customers, they aren't using the data. Right. It's more like a forensics thing, where if something goes wrong, then we go back and we look at the logs to see what happened. Right? Yeah. And then, you know, so if we think about, like, Maslow's hierarchy of data needs or something, right, you got your sensorization, you got your storage. Then you have to invest in making sure that the data is cleaned. Right.

There's a lot of effort, as we all know here, around making sure the data is actually cleaned and usable. And that requires you to know what bad data looks like, what good data looks like, and how to convert bad data into good data, so it's actually useful. Then once you have clean data, you also need to make it accessible in a streaming and batch historical manner. There's different gradients, I guess, is what I'm trying to say of AI readiness.

The customers whom we work with are all over the spectrum, but Phaedra today is at the point where we are autonomously controlling data centers for our customers. So I was going to ask you if the basic workflow, or the basic loop is data goes in, which is a lot of what you just talked about, getting the data into the system. Step one. Step two, decision is made. Step three, action is taken as a result of a decision that was made. Step four, action is evaluated against the objective function of the system.

Yeah. And then the loop continues. So the front end of that process, which is data, goes in. Sounds like there's a lot of work to get some real world data ready to go. We call it the AI readiness journey. Right? So, like, if you think about our work with customers, like, there is a chunk of upfront work where it's just like, hey, we're going to get your facility. We're going to get you and your facility AI ready.

How about on the action is taken? Piece of that. Are the systems ready to be controlled by some sort of autonomous system, or is there work that needs to happen there, too? Yeah, it's a really good question. Yes and no. Right. And I'll elaborate on what I mean by that.

Right. Control systems today were, like, designed in, like, the 1980s. So was I. Well, me too, for that matter. There we go. But, you know, like, what I mean by that is, you know, that was the. The seventies and eighties was the third industrial revolution, right? So with that was the shift from analog to digital and the advent of the first automation systems.

In order to automate, you fundamentally first have to sensorize. But these are simple automation systems. The fourth industrial revolution is. We're biased. But, Phaedra, we think the fourth industrial revolution means intelligent infrastructure, infrastructure that can operate itself and fundamentally get better over time at doing self improving infrastructure, right? But right now, we're shoehorning intelligence into systems from the third industrial revolution, right?

So they certainly weren't designed for this. But what we do instead is, most importantly, we ride on top of the existing control system. So there is a hard coded layer of rules and heuristics. So millions of lines of if then statements programmed into what we would typically call the BMS, the building management system, or a SCADA system, right, that defines how the facility should operate. The problem with hard coded systems is that because they're hard coded, they operate the same way today as they did yesterday or a year ago or five years ago, more like ten years ago, because people don't very frequently go into the backend programming, right, to update that controls logic.

Now, what Phaedra does is we insert a new cloud intelligence layer at the very top of the control stack. So we're nothing. We don't introduce any hardware. We don't introduce any new sensorization. Right. We actually ride on top of the existing control stack. That's really really critical.

Right. You can think of it as a general in the battlefield. The general, right. Has a global view of everything that's happening across the system, and it's issuing command signals to the troops on the ground for actual execution. So the AI is looking, in our case, at 10,000 trends a minute in real time, and it's issuing decisions like which pumps to turn on or what their pump speeds should be to the local BMS system and or the PLC's for automatic implementation and execution. So that's why I said it's a mixture of yes and no. Were they designed for this in the first place? No. There is a lot of work that we have to do with our customers to be able to accept this type of external intelligence.

There's a lot of work that we do in defining the safety nets and guardrails to ensure that the AI can't do bad things to the customer system. But fundamentally, we are still riding on top of the existing controls architecture. And to be clear, we always want to do that. You don't want AI controlling things like how fast evalve opens and shuts. That's a terrible application of AIh. Hard coded rows and heuristics will do great there. So if you were to look at the, you know, like, the overall system, like, 90% of it is fine with just, you know, hard coded rows and heuristics, because it's like granular controls logic that doesn't need non deterministic, crazy powered intelligence behind it. Right.

But it's the higher level thinking and reasoning that's where you want the AI. It's the global optimization. Have you seen any of your customers at Phaedra kind of get the DeepMind order of magnitude results? So I'm glad you asked. So, one of the things we're really excited about is actually just actually, literally this week, earlier this week, Merck Pharmaceuticals became our first public customer.

So we're pretty proud about that. We've been actually been working with them for two years now. They've been using Phaedra for over two years, the full autonomous AI system, to control a massive 500 acre vaccine manufacturing facility in Pennsylvania. This is the definition of mission critical complex. They've got 62,000 tons of cooling, so they've got four very large shale plants interconnected with each other across 500 miles of manufacturing space.

Hundreds of machines interacting with each other. This is where the AI really shines. And, yeah, the results that we saw with them were quite strong. Right? Like, you know, I think Merck actually just shared some data at a conference we were at which showed 16% energy savings when we first trialed the system at one of their chiller plants. But what I always tell our customers, right, is don't over index on the magnitude of the energy savings initially.

Like, we honestly have no idea what the energy savings are going to be or the reliability improvements are going to be ahead of time, because these are non deterministic six by definition. If I could tell you what things you're not doing in order to get energy savings, why do you need the AI in the first place? But what we do know is that the unique thing about this technology, about Fedra and about reinforcement learning in particular, is that it is a closed loop system. It is a self learning system.

It can learn because it's able to take actions and it can measure the impact of its actions against its predictions, right? And that means it gets better over time. So maybe we start off at 1% energy savings, maybe we start off at 5%, maybe we start off at 10%, right? But fundamentally, it will learn and it will get better over time, right? Not infinitely, because there are so are laws of physics, right? But it will get better over time.

And once it reaches optimal, it will stay at optimal. That's super important, because with hard coded rules and heuristics, when you tuned a system as you were commissioning it, so when you're turning it on for the first time, that system today no longer performs the same way that it did ten years ago when you first commissioned that system, because the pipes have corroded and the. The heating changes are fouled and the cooling towers are scaled, whatever, or you ripped out equipment. But the promise of adaptive self learning system is that it will change with you as your customers are, for example, now putting in a bunch of H 100 and soon H 200 gpu's, well, the system will learn and adapt on the fly with you, so it can stay optimal.

I'd love to transition for a minute beyond industrial controls. Totally, and get your opinion on. I mean, you were one of the first and maybe one of the only real world applications of reinforcement learning. Yeah, we're definitely not the only. Not the only. I'd love to get your thoughts on the not the only. So, I mean, what else is. What else are people doing with reinforcement learning in the wild today? Yeah, absolutely.

So, you know, unfortunately, my knowledge is very heavily indexed on the Google and DeepMind space because that's where we spent so much time. Right. But even within Google and DeepMind, there were other very cool reinforcement learning applications. For example, the team that sat right next to us, they used RL systems to help prolong battery life. For example, you may notice that your Android phone, the battery life has been increasing.

And yes, there are hardware changes associated with that, but there are also intelligent software changes behind the scenes that proactively manage your battery life. There were reinforced alarming systems for, for YouTube video recommendations, for example, and a whole host of other things. Absolutely. There are reinforcement learning applications in the wild. To your point, though, I wouldn't say that there are a whole lot of them.

And I think that it's not a coincidence that you tend to see them at more of the big tech companies where they already invested in the data infrastructure, right? So that the underlying infrastructure, so that they can benefit from this technology, right? Like, outside of the big tech companies, there are very few applications of, like, real world reinforcement learning, like, in production at least. Yeah.

And do you think that's because of kind of low applicability? You know, you started this podcast by talking about the necessary ingredients for RL to be a good solution. Do you think it's just, there's not that many applications where RL is a good solution, or do you think it's just tech readiness? I think the applications for reinforcement learning are freaking massive. And Phaedra is one of many examples that we're just scratching the surface as an industry of what we can do with this technology. Right? Like, fundamentally, the power of the technology is that it is a self learning system.

Alphago and its successor, Alphazero, taught itself to become the best in the world at go chess and shogi. Three vastly different games, same learning framework, and it taught itself. So I think there's a lot of very interesting application areas. I think the data infrastructure is missing in a lot of them. But just to list off a few, obviously, we've already talked about the protein folding, but there's an entire untapped field around logistics that is such a gnarly computational challenge when you start looking at operations research.

Operations research underlies trillions of dollars worth of industrial activities. Not just industrial, but other source activities like shipping airplanes, FedEx, like driving routes. These are all applications of operations research. Grid balancing. I think grid balancing is probably the single most important way that AI can fight climate change. I generally believe that is where AI will have the most impact on climate change.

If you had to guess, the first time you deployed this into a data center at Google, you saw 40% energy savings. If we had just killer AI doing load balancing on the grid, what sort of energy savings. Do you think we could see, oh, my gosh, I mean, that would be wild. I think it's not so much about the magnitude of the energy savings per se, but rather about the potential cost savings. Because then you could start shifting your loads around to when it's most cost effective to do compute. Or if you had CO2 signals, you could start scheduling loads around when it's the least carbon intensive to do your non latency sensitive workloads, which I think Google has already been experimenting a bit with.

But honestly, I think it's really more around the global system level optimization. We have to keep in mind that data centers already are, but increasingly are just massive, massive load banks. Data centers, they were one and a half, 2% of us energy consumption. That's about to increase to 4% this year. I think by the end of the decade, it's projected to get up to 9% of the US in Ireland. Right now, Ireland is 22% of Ireland's national energy electricity consumption goes to data centers alone. The International Energy Agency predicts that that's going to increase to 37% by the end of the decade. Just mind boggling numbers.

But the reason why I mentioned this is because these are massive load banks on the grid, right? There is an actual opportunity if you could somehow coordinate the data centers together to help balance the grid. That is such a gnarly challenge, and it is what is holding the energy transition back, because as more and more renewable energy starts coming onto the grid, the supply side becomes increasingly stochastic. We used to have this perfectly deterministic system, at least on the supply side, where a grid operator can call someone who operates a coal fired power plant and say, hey, ramp up or down your power production, it's deterministic.

Ramp up or down the sun? Yeah, totally. Now, as you get more and more renewable penetration coming onto the grid, you have a somewhat non deterministic demand side. It's somewhat predictable, but there are definitely spikes and a massively non deterministic supply side. What is the problem with that? The problem is that because we do not know how much energy we're going to generate, you now have all this wasted excess capacity in reserve. There is a concept of spinning reserves on the grid, where there are peaker plants like giant natural gas turbines that, as we speak, are just sitting there idling, just like your car idles at a stoplight, in case we need that power as a buffer against the uncertainty.

And as renewable penetration increases, ironically, the amount of buffering you need also increases. If you look at Germany's failed energy transition, they decommissioned their nuclear baseload while ramping up their renewable energy penetration. Good motivation on the surface, although I personally think we need a lot of more nuclear on the grid, but that's another topic. But it ended up backfiring because Germany actually ended up needing to build more fossil fuel power plants to buffer against all the renewable energy that was coming onto the grid.

Now, that's why I think AI, for grid balancing, we need it. And this probably is the single most impactful thing that AI can do to solve climate change. Let's talk a bit about some of the limitations of reinforcement learning and also where you see it intersecting with transformers. Yeah, so I should state that, first of all, my co founder, Veda, is by far the expert on this topic. He knows way, way more than me. You know, I'm just.

I'm just a simple mechanical engineer, right, who happened to learn a bit about AI. I think the intersection is really interesting. Like, very, very, very potentially complementary strengths and weaknesses is how I would describe it, right. It's certainly not mutually exclusive, right? Like, what I mean by that is, and I was just talking with beta about this earlier, right? So Veda will tell you that, like, all intelligence systems have certain hallmarks of intelligence, so that we can say they are intelligence. They need to deeply understand the world, the environment that they're modeling.

There needs to be some element of memory, remembering things, and very importantly, there needs to be the ability to plan and reason. Very interlinked. Transformers are clearly quite good at the first one, right? In the sense that, like, they can take in huge amounts of structured and unstructured data, right, to learn quite good models of the world. But it is limited in the sense that these models are primarily through correlation and not causation, right. That makes it challenging for at least for what Phaedra does, right? Because.

Because we work with real world systems, we have to have causality. We have to understand, why is the AI doing certain things? Why is it not doing other things? How do we force a certain behavior that we know has to exist in our system? These are mission critical systems. What I'm trying to say, there has to be causality. That's where the limitation is with reinforcement learning systems. The power of RL based systems is very much in the planning and reasoning part, where you're able to plan long trajectories of actions and learn really intricate policies.

I think where it gets really interesting is the intersection where potentially transformer architectures can learn models like value functions or models of the world that the AI can learn policies against. But without that causality piece, it's going to be quite tricky to cut it over into at least industrial control applications like what Phaedra does.

Should we move into a rapid fire round? What are you most excited about in the world of AI in the next five or ten years in the very near future? I'm excited about just the absolute explosion of AI applications. It feels like precambrian explosion of sorts, where they're like there's like a primordial soup and like all these AI startups and services are all of a sudden springing up, right. So it's quite exciting. But when I look at where that, where that activity is happening, where that research and that entrepreneurial activity is happening, it's very clearly focused on like around LLMs and even more specifically around like natural language interactions, text based interactions, and that certainly is a large part of the economy.

It is very exciting. But in the five to ten year frame, to answer your question, im most excited about when we can start getting some of this technology into real world physical applications. Its the intersection of this technology with the real world infrastructure that we live in. Big industrial systems, cars, homes, like, you know, physical things. I think that's where we're going to see some really interesting things in the future.

Who do you admire most in the field of AI? Gosh, a tricky question. I admire a lot of people you've worked with, some of the greats and so it's going to be hard. Yeah, I mean, of course my mind jumps immediately to a lot of the people whom I've worked with. Right. I admire very much the dmind researchers whom we worked very closely with.

I often tell people working at DMind, it's kind of like being a kid in a candy shop if you're a technologist like myself, it's like you get to see years in the future all this cool technology on the forefront and it just makes your head spin as to all the possible applications of that technology. I admire Moose a lot, my old boss, who has of course, since moved over to Microsoft. I was saying earlier, one of the biggest lessons I learned and my co founders learned at DeepMind is that making a technology like what we did for Google's data centers versus making a product like what we're doing at Freedraw. Totally different things. Wildly, wildly different things.

And there are few people as good in the world as moose taking technologies and turning them into real products. I remember my co founder Katie and I, we were sitting down, we were grabbing drinks with moose at some random dive bar in Seattle. He happened to be up there. And this is before OpenAI released chat GPT-2 and just ushered in a world of craziness. And he was raving to myself and Katie about the applications of LLMs and like, you know, the, you know, and how powerful these systems are.

And we were like, okay, moose, but, you know, let us tell us, let's tell you about Phaedra. Like, we had no idea what he was talking about, right. But I mean, he was prescient. Like, he saw this ages in advance, right? You know, of what the technology, the technology that was being developed and the capabilities, right. That it would usher in. And then, of course, he went off and he started inflection. So I admire him a lot for the ability to turn technology into actual products.

You are building a very ambitious business, very hard business to build, and you've been at it for a while. In the context of the new wave of AI startups, what advice do you have for other founders or would be founders who are trying to build companies here? I mean, I'm not sure I'm even qualified because, one, I hope you ask again in one or two years when hopefully Phaedra is wildly successful. We certainly didn't choose the easy path by focusing on real world infrastructure.

Honestly, my mind gravitates towards more would be founders, people like my co founders and I, who were thinking about leaving to start something new. And my advice there is twofold. One, make sure you have co founders like, my God, it's so stressful. There's so many things that can go wrong, and you're constantly on this emotional roller coaster of up and downs. Having co founders to lean on both for the workload, but also just for the emotional support and mental sanity.

So important, right. Advice number two would be the risk is less than you think it is. I'm biased, but I think people should take the jump. Right? A lot of times when I talk with my former colleagues and other people who are thinking about making the jump, they'll say things like, well, but I've got a nice job over here. They pay me well. I'm on a rising trajectory.

But my point to these folks is always like, no matter how valuable and successful you are today in the organization, you will only be more valuable and successful for that organization or other organizations or to society in general. If you learn new skill sets, take the plunge, go out, start a company, learn what it's like to turn technologies into products, and if that fails for whatever reason. Hopefully it doesn't. Right? But if you fail, then the Googles, the Microsoft, whatever, the world, they will only want to hire you back at an even higher premium.

So why not take the plunge, right? It's the, you know, the biggest, best investment, and obviously much smarter people than me have said this for a really long time, but the best investment you can make in yourself, right, is you, right? Up leveling yourself, learning new skill sets. That's always the best thing you could do. Thank you, Jim. This is a fascinating conversation. Yeah. Thank you very much for having me. You guys really enjoyed it. Thank you.

Artificial Intelligence, Business, Innovation, Reinforcement Learning, Efficiency, Deepmind, Sequoia Capital