The video is part of the Value Chain Innovation Speaker Series hosted by Stanford's Graduate School of Business. It features an examination of the evolving landscape of AI in the enterprise with insights from Professor Yoav Shoam, co-founder and co-CEO of AI21 Labs, who discusses his journey from academia to entrepreneurship, and outlines the beginnings and evolution of his company, along with the creation and impact of AI21 Labs' models.

Please remember to turn on the CC button to view the subtitles.

Key Vocabularies and Common Phrases:

1. foresight [ˈfɔːrˌsaɪt] - (noun) - The ability to predict or the action of predicting what will happen or be needed in the future. - Synonyms: (anticipation, forecast, prescience)

One example to illustrate his foresight and impact on the field is that while everyone talks about agents today, Professor Shoham defined the framework for Agent Oriented Programming as early as 1993.

2. trajectory [trəˈdʒɛktəri] - (noun) - The path followed by a projectile flying or an object moving under the action of given forces. - Synonyms: (path, course, route)

In today's event, we will explore the evolving landscape of AI in the enterprise, examining its current state and future trajectory.

3. arbitrage [ˈɑrbɪˌtrɑʒ] - (noun) - The simultaneous buying and selling of securities, currency, or commodities in different markets to take advantage of differing prices for the same asset. - Synonyms: (exploitation, leverage, advantage)

Yeah, I mean, I guess the common denominator is where I saw an arbitrage between ideas that in AI that either I was involved in directly or, you know, indirectly and what I saw in the world and trying to capitalize on that and really kind of moving the needle that way.

4. ossified [ˈɑːsɪfaɪd] - (adjective) - Ceased developing; become stagnant or rigid. - Synonyms: (rigid, inflexible, stagnant)

And I was amazed at how backward the software was and using close standards, very ossified.

5. nimble [ˈnɪmbəl] - (adjective) - Quick and light in movement or action; agile. - Synonyms: (agile, alert, quick)

And it was obvious that we could build a much kind of more nimble, flexible, quick to deploy kind of system at a fraction of the cost.

6. infusion [ɪnˈfjuːʒən] - (noun) - The introduction of a new element or quality into something. - Synonyms: (introduction, addition, incorporation)

The first one, efficiency and cost involves the infusion of much more efficient models.

7. latency [ˈleɪtənsi] - (noun) - The delay before a transfer of data begins following an instruction for its transfer. - Synonyms: (delay, lag, wait time)

The latency is tiny, the memory footprint.

8. bifurcation [ˌbaɪfərˈkeɪʃən] - (noun) - The division of something into two branches or parts. - Synonyms: (splitting, dividing, branching)

And so what you see here are a bifurcation of.

9. contractual [kənˈtræktʃuəl] - (adjective) - Agreed in a contract. - Synonyms: (binding, agreed, promised)

Can you unpack that and give us more details about that? Well, I mean, you see that the most successful gen AI company, terrible term, by the way, generative AI.

10. visceral [ˈvɪsərəl] - (adjective) - Relating to deep inward feelings rather than to the intellect. - Synonyms: (deep, gut-level, instinctual)

I mean my first advice is always absolutely experiment. If you and your people don't have a visceral field for technology experiment.

AI in the Enterprise - What Works, What Doesn’t, and What’s Next

Foreign Good morning everyone and good afternoon, good evening or good night to those of you joining us from other parts of the world. Welcome to our Value Chain Innovation Speaker Series. My name is Barry Gilay and I'm the Associate Director of the Value Chain Innovation Initiative at Stanford Graduate School of Business.

We're excited to have with us today Professor Yoav Shoam, who is the co-founder and co-CEO at AI21 Labs. Professor Shoham is a leading AI expert who received multiple awards for his significant contributions to the field. One example to illustrate his foresight and impact on the field is that while everyone talks about agents today, Professor Shoham defined the framework for Agent Oriented Programming as early as 1993. Professor Shaum is the co-founder of several other successful AI companies. In addition to AI21 Labs, he's also a Professor Emeritus of Computer Science at Stanford University.

In today's event, we will explore the evolving landscape of AI in the enterprise, examining its current state and future trajectory. Moderating the discussion is Hai Mendelssohn, who is a professor at the Stanford Graduate School of Business and the Faculty Director of the Value Chain Innovation Initiative.

Before we begin the discussion, I'd like to quickly go over a few housekeeping items. First, the event is recorded. The recording will be available in a few days on the Stanford GSB YouTube channel as well as on the website of the Value Chain Innovation Initiative. If you have any questions to Professor Schwam, you are welcome to submit them. At any point during the moderated discussion, we ask that you please use the Q and A box to share your questions with us. Toward the end of the webinar we will have a Q and A session and during that time Professor Shahm will address as many of your questions as time permits. If you would like your name to be mentioned when we read your question, please include it at the end of your message.

As for closed captioning, if you would like to activate this feature, please click on the show captions or CC icon. Finally, we kindly ask that you take a moment at the end of the webinar to complete a brief survey. We value your feedback and we'll use it to improve future events.

And now I'm delighted to hand things over to Professor Heim and thank you very much Barghit and thank you all for joining us. It's a real treat to have you with us. Let's get started. I remember many years ago you were a superstar faculty member at the Computer Science Department and you got the entrepreneurship bug and started the company. What gave you that bug I don't know where I went wrong.

I have to confess, honestly, I came in almost, I would say, despising practical matters and. And somehow the. Somehow something in the air. You know, at the end of the day though, I think, I think there is something to. At some point you. You can prove just that many theorems. At some point you want to see something move in the world. And that's very addictive. I would have to say that. Yeah.

Here, tell us a little bit about some of the companies that you started or participated in before AI21. Lens. Yeah, I mean, I guess the common denominator is where I saw an arbitrage between ideas that in AI that either I was involved in directly or, you know, indirectly and what I saw in the world and trying to capitalize on that and really kind of moving the needle that way. And you know, some people are really good at execution and that's what makes them successful.

And to the extent that I was successful, it's not because of that. Because the ideas had enough force that they could withstand whatever poor execution I brought. Brought to the table. We had the first company called Trading Dynamics back in the days of B2B trading. And you know, I was involved. I've been involved in game theory for many years. I have a corporate course on game theory that's been on Coursera and other places. Over a million people have seen it, which is weird because not really useful, but I was involved in.

So I got exposed to your colleagues in the business school and Econ, to the energy auction in California and also the Spectrum auctions. And that was already real world, real money. And I was amazed at how backward the software was and using close standards, very ossified. And it was obvious that we could build a much kind of more nimble, flexible, quick to deploy kind of system at a fraction of the cost. So that was an example. The first company that I was involved with, Trading Dynamics, it was called and we move forward.

I remember in, I think it was 2017, we were talking and you were starting, you were involved in a secretive effort which I think became AI21 Labs. Can you tell us about that period and what was the prompt that led you to think about those new ideas? Well, it's interesting. Yeah. So that is the company that I'm currently involved with and I helped start it with two other colleagues.

Ori Ghoshin, who's kind of, you know, has no degree in anything, but is smarter than anybody else in the room. And Amnon Chashuo has a degree, but probably Better known for his having started and running mobileye and a few other companies. But really what prompted the company was, was seeing where the field was with the whole emphasis on deep learning. So, you know, LLMs weren't a thing. Well, they were happening just as we, coincidentally as we started the company. But deep learning was clearly dominating the field.

And I felt that deep learning was necessary but not sufficient that you wouldn't get the robust reasoning that we expect and that AI used to deal with before statistics took over. We started the company with a goal of marrying the two, which is maybe an odd reason to start the company, but that's why it happens. And you know, seven years later, here we are.

And was that before or after the Transformers paper? Same year. Same year as it happens, 2017. We started the company after it, but it hadn't yet made the headlines. But as we were working, clearly language was where the action was going to be. And as we were working on it, this end to end training enabled by Transformers was kind of know, creating a buzz. So it was kind of fortuitous that we started around the same time.

And can you tell us a little bit about the evolution of the company over those years? Yeah. So first maybe I'll start with like basics, where like I said, we're about 7 years old, we're about 250 people, about 100 of them are sort of technical, kind of, you know, algo Eng type stuff type people. And for three years we worked nothing on, we did nothing but work on technology. But we really wanted to be a business, not a research lab.

Only we say kind of somewhat, you know, tongue in cheek, we don't want to be a deep mind. And we of course say this with, you know, utmost admiration for the guys, but there was not a big business in solving Atari games. So the question is, what business would we be in? It was clear to us we wanted to be a B2B company. But it was also clear that there wasn't a business yet to be had there. We're talking about 2020 or so.

And so we created our own market with a writing assistant called wordtune, which at the time, today you have no end of, you know, writing assistance. Some of them built into the big platform. But at the time it was really transformative and very successful. We passed 10 million users. It was kind of a freemium kind of model. So we crossed 20 million ARR very quickly and so it's doing well. But then what happened is then enterprise suddenly happened. So we still have that business and it's a it's a good little business, but our focus is on the enterprise.

And so you know, I can tell you more about our technology, but basically if I speak, just zoom out and speak about how we see the industry, we think we're about to move from the second to the third phase of the modern AI revolution. I'll tell you what I mean by that.

So until roughly GPT3 time frame, Enterprise couldn't care less. There was sporadic experimentation. They just finished transition to the cloud and so maybe yeah, go ahead and play with it. And then we transitioned so that there's no CEO in the world that doesn't say I'm an AI first company. I want to be an AI first company. And a ton of experimentation. But a dramatic drop from experimentation to mass deployment. There's one study by AWS that takes it at a 94% drop. So 6% of the projects actually go to and we see it firsthand.

We think we're about to cross into the third phase where you'll see mass deployment and the two biggest changes will be much more efficient models in terms of the cost of. Because the economics of large language models is very different from the economics of traditional software, as you know. And so you need to do something about that. But second, as I think is well known, these models are wonderful, sometimes just brilliant and sometimes dumb as nails. And in the enterprise, for most use cases, if you're brilliant 95% of the time and ridiculous 5% of the time, you're dead in the water. We think that the next transition would be to really change that. It's not just a matter of so called more guardrails and so on.

It's really moving from models to complete AI systems. And we speak about that, but that's kind of the landscape and what we do as a company, we have our own language models and we're extremely good. Our latest family called Jamba kind of broke the transformer mold and gained dramatic efficiency, especially with so called long context. And it's doing well and we're very active in the AI systems there. I can't elaborate a lot, but it's an area that's really the reason we started the company. And so anyway, that's maybe a little too much about us as a company.

So let's kind of go into some of the details. So first of all, can you tell us a little bit more about Jamba, what it is and how does it improve over other models? Yeah, sure. So if we take a step back and think about that, really almost all the models you know of, whether it's GPT, three GPT or Claude or you know, Mistral or you name it, they're all transformer based. And transformer is what enables suddenly these models to do well. Not just on vision, but which is in a sense an easy problem.

Is very local to know that this here is a phone. I don't care what the pixel over here on the other side is. That's roughly true in language. There's nothing local. I changed a word here. The whole sentence. You can't get away from semantics. And transformers suddenly enable us to deal with that. Up until then there were convolutional neural nets and recurrent neural nets. And very good for kind of object recognition, but not for language. Transformers changed that. And the basic mechanism there is the attention mechanism which allows you to relate to attend, as it's called, the attention to attend to different parts of the input that can be quite disparate, quite far apart.

So that's good. And suddenly you saw the needle move on language task. But the price you pay is complexity. It's quadratic complexity in the input size, the so-called the context length. And if you have a contact length of say 1,000 tokens, 2,000 tokens, that's okay. You got a couple of million squared is okay. But we're now pushing a million. A million squared is not okay. So that's transformers.

What do you do about that? Turns out that there's an alternative architecture without getting too geeky, called state space models, SSMs. And recently about a year ago, there was a special version of that called Mamba. Came out of academia, I want to say Princeton and cmu. I think that made it more efficient, more interesting. So our guys, what I guys did, did something very innovative. I take no credit for it. They did a hybrid architecture that mostly Mamba, but a little bit of transformer.

So you get the best of both worlds. You get the quality of the answer is very competitive with the transformer models. And complexity that's not quite linear, but almost linear. The latency is tiny, the memory footprint. For example. We have two models now. The small model is like, I'm forgetting small model, I want to say is I think it's 52 billion total available parameters of which 12 billion are active at a given point in time. What's called the mixture of expert kind of architecture that recently became like, you know, discussed because of deep seq. But it's an old.

So it's an old technique. And so that's our small model. And which is funny to call it small, but our big model is almost 400 billion, 392 billion parameters and 94 active. The small one fits on the single gigabyte GPU, which is mind blowing. Now the model can come close and the large one on a single eight GPU pod, which is also. So, you know, we've had like very large companies switch to Jamba because suddenly the unit economics and the latency makes sense.

So when we talked about some of the factors that slowed down adoption of the enterprise, the first one, efficiency and cost. Can you unpack that and give us more details about that? Well, I mean, you see that the most successful gen AI company, terrible term, by the way, generative AI. But it is what it is. So OpenAI making, I don't know, they claim four or five billion dollars a year and losing much more than that. It's very expensive little. I'm not speaking about even training these models, just serving these models.

And so the unit economics are very different and so you need to somehow do something about it. So one way of tackling the problem is making the architecture such that it's much cheaper. And that's what Jamba does. The other is to realize that the world doesn't start and end with LLMs. Not everything you want to give an LLM.

So LLMs do arithmetic, for example, which is in some sense very impressive. On the other hand, they do it slowly and poorly and they'll never be as efficient and as Precise as an HP calculator from 1970. So you switch to AI systems which have LLM, but they have tools like the Encapsulator and API call and custom code. And some are orchestrating all of that. That is doing two things. First of all, it's making the system more efficient where possible. And second, they're making them much more precise. And that's the other key thing in the Enterprise, this precision, that without which it'll forever remain experimentation.

So can you tell us a bit more about how to solve these problems? I can, but I won't. But I'll say a few things about it. We'll be coming out before too long with a product in the space which I think would be interesting, and we'll give many of the details. But let me, I can say a few things. In fact, just yesterday a piece of mine came out in Fortune that spoke a little about these things.

So let's think a little about what's a little under the hood, what's happening with these models. And so when you run a model so basically you have this probability distribution and you're sampling from it, doing some kind of posterior based on the input, which means you have something stochastic. And so you have to sample multiple times to iron out the variance here. Sometimes like you said, you don't want to appeal to this probability distribution, but another one, and maybe it's not a distribution, it's a point distribution, namely a deterministic tool.

And so what you see here are a bifurcation of. There are some people who are trying to shoehorn all the smarts into an LLM. There's an approach called React which some of the audience here will be familiar with. It tries to teach this mini system, this LLM to take an action, get the feedback, do some kind of analysis with it and repeat. There's cherry picked examples where you get kind of wonderful things, but by and large it's uncontrollable, it doesn't work.

What you see in the Enterprise is the polar opposite of handcrafted sort of static chains. So if somebody wrote code and say I'm going to call this LLM with this kind of input and I don't trust its output so I'm going to try it multiple times with different inputs and then I'll look at the output and I'll decide do I like it or not and depending on that. So some kind of. And now I'll call it calculator, I want to know some proprietary information and propriety database. I'll call the APIs. Somebody wrote a script to do that.

AI didn't play a role here, but except when some of the tools were LLMs. What you're going to see is coming out are approaches that do the best of both worlds, that do let AI do the smart scripting and execution when it makes sense and still give the user control and observability. So you can trust these systems. That's the direction we're going in the world.

These are these AI systems that I believe that. So one of my colleagues, very well known Jan Lecun, likes to dis LLM and says that within n years and for some small N nobody will speak about LLMs. I don't think that's true, but I do think that there won't be the end all and be all. And I actually think that by the end of 25 people speak about AI systems and the core central concept that enable us to kind of move forward in the enterprise.

So before we talk some more about the implications of that for enterprise decision makers, I want to take A stab at another question. Lots of people talk about agents today and you've put together that frameworks many years ago. What do you think about, you know, what people are saying about agent? Let me start with a positive.

I think there is a there there. I don't think people should ignore the concept, I think it's irrelevant. But AI, especially in recent years has had this bad habit of taking ill defined terms and using them as if they're well defined and based on that, raising money or doing PR or you know, promising products to customers and it's kind of quicksand. So my favorite example has always been AGI. It's not a thing and, but so I would never raise money by appealing. Oh, I'm going to create AGI.

I just don't think it's a well defined thing. Agents are a little like it agents really under that age. If you went to Davos this year, which I didn't, but my co founder did, like everybody's speaking about agents. It was clearly they're talking across each other. There's several different things that are taking place there and they all make sense in isolation. They don't necessarily come together to make a complete whole and we can list them, but it has to do with doing AI systems have to do with using tools and not letting just the language model do the whole thing. It has to do with having some amount of reflection so that because to compensate for the inaccuracies of the LLMs, it has to do with running long process.

LLM typically is very transactional. You know, you put in the prompt and seconds or fraction of second later you get the response. Whereas these agents can run for a long time, you know, minutes or hours or sometimes months, and they can be proactive. It's not necessarily a stimulus response they can initiate. So they sort of become more of assistance than more of an active assistance than a passive tool. So there's a whole collection of concepts here that it's a mistake to try and call everything, anything an agent because it'll mean everything and nothing. But having said that, like I said at the beginning, there are exciting things that are happening in the space.

So, so let's kind of think, I imagine a decision maker in the enterprise is facing a terrible dilemma. On the one hand, everybody's asking them, what are you doing about AI? What are you doing about AI? So they go with today's technology, which may not fit the architecture of the future. So, so can you help people think about that question? How do you trade off the desire to get good results today or at least experiment with the desire to architect everything so that it works better in the enterprise. So yeah, so we're engaged with lots of, you know, companies, typically very large corporations and I mean my first advice is always absolutely experiment. If you and your people don't have a visceral field for technology experiment, experiment with something that's meaningful, not a throwaway thing but perhaps don't bet the farm on quite that application right now that's one thing.

The other thing is we've had companies very thoughtfully put together like 200, sometimes 400 potential use cases for gen AI and they speak to us about so which of those and you want to take something where the technology risk is fairly low. Nonetheless it's not trivial anybody. And if you're successful you'll actually move, move the needle. I'll give you an example of that.

It's not abstract. So we have one very large, large retailer both they have an offline and a very robust online market. And I forget if I'm allowed to say their names, I want this to be on safe side and, and they have millions of products on their website, thousands come up every day and they need to provide product descriptions and that's a very labor intensive, costly and costly in money and time process to do over and over again.

And so we actually had a system which is not just a language model. It's language model at the core but you start with, I don't want to call it rag people familiar probably with the retrieval augmented generation but there is input to the system that wasn't available at training time and based on that and with highly customized post training that we did is very good at product description so much so that they've slowly start to wean themselves off checking every product description that comes online and that's also a good trade off.

I speak about, people speak about the product market fit, I speak about product algo fit. What I mean by that is that technology is error prone certainly AI technology. So as a, if you're creating a product or a service you need to be aware of its strengths and weaknesses and craft the offering to leverage the strength and compensate for the weaknesses. And so human in the loop is part of it and and also the cost of it depends on the cost of error.

If one in every thousand products distributors is off the world will not crash and you can fix it. That's a good kind of trade off to do here. So when we think about a product algo fit one parameter is the, the cost of an Error the relay that requires reliability. What are some other guidelines you would have for people who say, well, I want to experiment with today's LLMs and I want to get as much productivity out of it. So what would be some of the parameters of the best fits?

I think that product health algo is more of a how question, not a whether or which question. I think your question to me was more about so where do you focus? Not how you and I go back to what I said before. It's got to be important enough for you to care if it's successful or not. And, but bite size if that's your first project.

And we've typically have people working on two, three projects simultaneously, maybe two, three different teams sometimes experiment with different technologies. I think that routinely people are speaking with language model, trying several. That's always a good thing to do and you learn from that this day and age. Related to what I said before, you will never just take a language model and use it. You will always craft an AI system and you want a partner to help, you know, to help you architect that system in a way that won't be, you know, that won't crumble under its own weight. And. And yeah, so that's what I can say right now.

So AI system is more of a mindset than a product. Right now you don't have a packaged offerings that says, here's a way to create an AI system. Right now it's mostly description of what people do in IT departments. It will become a product category. This year you already have company again. You have to be careful because people latch onto concepts and turn them into marketing slogans.

So you can go to all the cloud providers and you'll have agentic offerings and they're very thin layer that does very little for you. I mean, it's not that it's not useful today. You'll still need to do most of the work. And what you'll see, I think this year is more and more people will offer you to offload a lot of this burden from your own IT customer development effort. And we'll be coming out with our own product before too long here. And we probably won't be the only ones.

So when we think about the architecture of an AI system, is there a clear migration path from today's architecture to the future architecture or is it more like a greenfield sum? You have to rip apart what you have and rebuild? It's a good question, and I'm not smart enough to answer it. I think that it's. I don't really know. I mean there are elements that are really new and that will upend traditional.

In fact, it's open, open questions about how transformative AI will be to the, to the, to the enterprise. And it's not just these, the IT sort of infrastructure is also that. You know, I, I, I forget if it was Satya Jensen that said that ID departments are going to transform. They're going to be HR departments for agents. I think that's too cute to be a, a correct description, especially since agents are well defined but certainly they'll change.

In fact, when you think about this Android organization where more and more functions are done by machines, I don't think by the way that all functions will be taken over by AI. I really don't believe that. But more are and will and really there'll be a collaborative relationship between these sort of AI workers and human workers. I think it will have an impact but on the enterprise and honestly I'm not smart enough to really say much, much more intelligent things about it.

We have quite a number of great questions from our participants so we're going to go to some of these questions. So one of the participants is asking today you've seen AD for deployments that augment or replace. Have you seen AI deployments that augment to replace workers? And where will we be on that scale between augment and replacing? 5 years given what's happening. So I'm in the non alarmist camp, but not the complacent camp. I'll try to say what I mean by that.

So there are lots of studies about the future of work because of AI and I think, I mean they're well intentioned but I think they're, they're much more speculation than based on real fact. Historically technology created more job than it replaced. That's historically true. Some people saying this time is different. I think the onus of the kind of argument is on them because I don't see a reason to think that. I think we're just not smart enough to anticipate what jobs our kids and our grandkids will have. But there's no question there's some jobs that are going away.

I mean just let's take writing, copy editing. We used to have copy editors that's gone. I mean software does this extremely well. Editors as in a sort of magazine editor or a book editor, they haven't gone away and I don't think they'll go away. I think their job, they'll have better tools to work with. But I don't Think they'll go away. And so I think it'll be a mixed bag and I know their predictions by percentage of white collar job that'll go away. All I can say is I, I don't know the answer, but what I've seen I found highly speculative.

For example, the movement for UBI Universal Basic Income because you know, all work will be, will go away. And so what we need is to give people money and give them a different meaning to their lives. I think it's all good question, what's the meaning for our lives and you know, why sometimes work is so central part of it. That's always a good question to ask, but I don't think that jobs will go away at such a scale to make humans work less.

We have a question from Nalan Sipar who is a journalist from Germany and he's obviously interested in what will happen to journalism. So what are some of the big opportunities for AI and journalism? You know, so my favorite answer to this question is enjoy your job while you have it. But, but I don't really believe that. I think there are. Okay, I think journalists or, you know, or you know, writers are probably the most important people in the world today because the information layer that underlines the liberal democracy is under relentless, relentless attack. And if we could rely on good editors, good journalists to get at the fact and present to us in a compelling way and to sift the important from the unimportant.

Now technology has made that very challenging, by the way. It's not just AI. I know people like to speak about fake news and how AI is the main culprit here. AI has a role to play here, both positive and negative. I can speak about that. But I think the dissemination technology, social networks in general, the flat world is even a stronger sort of force. And it has to do with social and psychological and economical factors. So it's very complicated. But I think journalist's job will be not just to get the facts, but to persuade us to listen to the facts. And I think it's super important and super hard.

Great. Danny Wong is asking, in addition to the kind of factors that block adoption that we've discussed so far, what is the other most important factor that is holding back companies from using more AI systems? I honestly think all the rest are sort of second, if not third order. But sometimes it has to do with lack of expertise in the area, which is not different from adopting new technology in general. But maybe this is new in a different way. So that's sometimes part of it.

It's it's understanding the use cases well and their mapping to the strength and weakness technology. I think those are the main factors. I mean sometimes it's lack of access to compute, but I think that's rarely the real factor. So I'll interject my own follow up. What should we be doing in universities to help alleviate that problem? And by university I mean you probably mean specifically, you know, in the business school or people working with industry. So I'm, I'm asking about universities, but really it's about education. So you know, what should the education system do at any level?

Okay, if you widen the value we can get out of these systems? Well, if we widen the question sufficiently, I think that being AI educated is something we need to inculcate at all levels. You know, you, you, there's no reason you shouldn't have already in elementary school. My youngest son is in middle school and I gave a lecture to the kids there and they asked me as intelligent questions I get from, you know, Stanford students and athletes I got while still teaching. And so I think we should have it at all levels. Obviously if you kind of go to the other extreme and think about reskilling because even if I'm right and the overall trend is positive, the workforce, there's a transition period that some people can find painful and so we need to help with that too.

So I think there is a role everywhere. I think every. And I think it's happening. I don't know if in the optimal way, in the right capacity. But the universities I know all have executive education on these topics and certification for people who want to transition from and so on. I think it's across the board. Great.

We have another question about the impact of Deep seq and some of it is just about Deep SEQ itself and some of it is what do you think about it? What do you think is going to be the impact of that? Okay, so maybe 30 seconds of just facts. Deepseek is a company. They have a couple of hundred quite smart people in China. They're, you know, part of this hedge fund which was managed a lot of money, then manage less money but. And they've had models come out for the last several years.

We've known them and we've always felt, you know, known them to be a legit shop. Their Deep SEQ and particular model R1 is a fast follow on OpenAI's 01. And the reason they made so much noise is not so much because of their innovations. They had a few. They have latent attention multi mixture of experts and by the way they didn't invest. So OpenAI didn't invent anything with O1 but they in typical fashion they did it robustly for the first time R1 also their innovation over O1 are not things they invented but you know, that's the nature of the community.

But the main reason they got attention was for two reasons. One is they open sourced the model unlike the closed source of OpenAI ironically and by the way open source is a misnomer because it's really they share the weights but not the code, not the training data which limits the usability. They're also fairly open, not completely, but fairly open in description, describing how they train and so on. So that's one reason this community now could just take it and run with it. The second reason is the claim that did it very cheaply on on on weak hardware and that we should take with a big grain of salt.

There's no way the model cost, total cost was $6 million. It was at least one and probably two orders of magnitude larger than that. And I can explain why and what made really made it explode was their application, their chat application which became the number one application in the app store. So those are the reasons. There's, there's.

I didn't understand the sell off on. On Wall street didn't make any sense to me. The first of all like I said they didn't do it as cheaply as some people thought. Number two, most of compute spend is going to be on using the model on inference, not on training. As expensive as training is, especially as AI gets widely adopted inference is going to dominate the spend. And so I think it was, I think the hoopla around 01 was overblown and the counter hoopla around Deep SEQ was equally overblown But a nice model by the way people have tried deep seek against Jambo Special certain problems of question answering and jumbo was just better, much more factual and so on.

So you know, it's a good model but yet another model. Some people are saying, you know, outsource is winning. I'm sorry, open source is winning against closed sources. What's your view on that kind of debate between open source and semi open closed? Yeah, certainly there's a strong pressure on closed source and I'm not sure that's tenable. I think for example Deep seq may force OpenAI's hand. In fact Sam on record saying something along the lines of we've been on the wrong side of history.

So I think you see people Share more, not everything, but more. And you know, we, you know, we shared the Jamba one version of the Jamba open sourced. It served us well and so I think you can see more of it. Are the open or semi open source models as strong as the strongest closed source ones? Not yet, but they're getting stronger.

We have a question that asks you to summarize but really project. So I'll kind of rephrase it as what do you think enterprises should be thinking about and focused on in 2025 given what we've seen to date and given the trends that we are observing already? A broad question. I'm not sure there's a pat answer across all enterprise. I think this varies, but I would say by the end of the year, try to commit to one real deployment, keep yourself honest. It's fine to experiment, but the time now is to be part. And I really think that if you don't, you'll be left behind because I think we are moving to more of a mass deployment.

Some large fraction of Fortune 500 companies. We're going to have mass deployments, which isn't the case now. And so you want to get to mass deployments whatever it takes. And which means you need to handle the issues of reliability and, and you know, you know, controllability. But just make your CIO or cio, make them make it happen. Right. And we have another question. It's a broad question, but let's try to compress it to four minutes about kind of what will happen between China and the US and do you see kind of the industry becoming, you know, there's going to be a Chinese branch of the industry and the US branch and they'll compete with each other. What, what do you foresee at least on some of these dimensions?

I think the short answer is yes, we've already seen it happen for a long time now. China, China has not been just the, the, the quick implementer of the big innovations of the West. So I'm not sure it'll be quite bipolar. I think they're more and more pockets of strength in the world.

Obviously now Europe is making a big push after the recent action summit in Paris. A lot of fanfare. So you see more action of Europe, obviously the, the, the Emirates and Saudi Arabia are spending a lot here. So I think it's going to be a more interesting dynamic than the simple Chinese US but those are certainly going to be major polls. I think that it is going to remain fairly bifurcated for geopolitical reasons, not for technological reasons.

So basically, so you see increasing competition between the US and China, you know, learning from each other, but developing as a duopoly essentially, or not. Again, I'm pushing back a little bit on the duopoly. I think it's going to be a little more multipolar than that, but certainly these are very strong poles and you're going to see more and more competition across the stack, not only at the model level, but all the way down to the chips. And, and clearly especially now, you know, with current geopolitical sort of configurations, there's going to be even more of a, you know, you know, a so the division lines and constraints and so on restrictions. So.

And so I think you're going to see a lot of develop independent development in China and you'll see resistance to adopting the Chinese technology in the West. So I think that's likely in the next few years. Likely dynamics. And what do you see foresee for Europe? Interesting. I think they're spending or claiming to spend, not a little money. It, it varies.

So we have our German friend on the call. Germany has been a little hard to do business in. On the other hand, France has been quite innovative and nimble here. So. And doesn't end there. I mean, you know, as you know, we're based in Israel and very small country, but we've always punched above our weight and you'll probably see this here also.

So I, I think, yeah, I mean us is continue, it will continue to be the main center of gravity here. That, that won't change, but I think you'll see others centers that are of meaning as well. That's great, Yoav, thank you so much for giving us a glimpse into the future of AI in the enterprise. And we look forward to seeing more announcements later this year. And thank everybody for participating in the webinar.

Before we wrap up today's session, I'd like to give you a preview for our next webinar. In the Speaker Series on March 19, we'll be hosting Michael Marks, who is the founding and managing partner of Celesta Capital and is the former CEO and chairman of Flex, which is a company which has redefined the past and is redefining the future of global manufacturing. Michael will share his insights on how AI is impacting the business world and the opportunities that he sees for AI to drive value creation. Thank you very much.

And Bar has a few additional words to share with us. Thank you, Chaim, and many thanks to you for a very engaging and insightful discussion. As I mentioned in my opening remarks, we kindly ask that you take a moment to complete our brief survey. You can access it easily using the QR code displayed on the screen screen. We also invite you to use the link shown on the right to access the Value Chain Innovation Initiative website and take a look at our upcoming webinars. This website is also the place where we will post the recording of today's event and with that we conclude today's webinar. Thank you all for joining us. We hope you have a wonderful rest of your day and we look forward to seeing you in future events.

ARTIFICIAL INTELLIGENCE, ENTREPRENEURSHIP, INNOVATION, ENTERPRISE AI, AI SYSTEMS, AI21 LABS, STANFORD GRADUATE SCHOOL OF BUSINESS