The video features an interview with Mark Sermon, president of Mozilla, discussing his insights on open source AI and the concept of trustworthy AI. Sermon lays out his perspective on these topics, detailing the importance of accountability and agency in AI systems, and highlighting the role of Mozilla in fostering these principles. As part of their efforts, Mozilla aims to bring the same ethos from their work on Firefox into the AI space by promoting transparency, privacy, and user control over AI technologies.

The conversation further delves into the open source versus closed source debate in the context of AI, with Sermon outlining Mozilla's definition of open source and the benefits it offers to society. He emphasizes that real open source AI allows for unrestricted use, study, modification, and sharing, stressing a need for regulation that encourages competition, privacy, and safety in AI development. Mozilla's investments into open-source AI projects like Flower AI reflect their commitment to these values, supporting technology that can be universally deployed, rather than reserved for expensive proprietary systems.

Main takeaways from the video:

💡
Trustworthy AI combines accountability and agency, ensuring systems operate transparently and are under user control.
💡
Open source AI should adhere to "four freedoms": use, study, modify, and share, ensuring technology is not monopolized.
💡
The sector needs regulations focused on competition, privacy, and safety, advocating a balance between closed and open commercial and public AI solutions.
Please remember to turn on the CC button to view the subtitles.

Key Vocabularies and Common Phrases:

1. accountability [əˌkaʊntəˈbɪləti] - (noun) - The obligation to explain, justify, and take responsibility for one's actions. - Synonyms: (responsibility, liability, answerability)

And there's two things that for us make up trustworthy AI. One, the idea of accountability, which is something often when people talk about ethical AI, responsible AI means if something bad happens, that the people who made that system are held accountable.

2. agency [ˈeɪdʒənsi] - (noun) - The capacity of individuals to act independently and make their own choices. - Synonyms: (autonomy, self-direction, freedom)

But the other half, in addition to accountability, is agency. Can we control how the AI works? Can we see how the AI works? Is it working on our side of the table?

3. granular [ˈɡrænjələr] - (adjective) - Characterized by a high level of detail. - Synonyms: (detailed, specific, precise)

And how do you really, really measure that on, you know, on an not broad level, on a more granular level, what benchmarks should we use?

4. pervasive [pərˈveɪsɪv] - (adjective) - Something that is prevalent throughout an area, environment, or system. - Synonyms: (widespread, ubiquitous, all-encompassing)

How do, as we make AI pervasive in everything we do, do we build the expectation that it's being used in a trustworthy way?

5. transparency [trænsˈpærənsi] - (noun) - The quality of being easily seen through or understood; openness, communication, and accountability. - Synonyms: (openness, clarity, clearness)

It's also, you know, transparency, being able to see how things were built, what data is inside...

6. unencumbered [ˌʌnɪnˈkʌmbərd] - (adjective) - Not burdened with; free from any encumbrance. - Synonyms: (unburdened, free, liberated)

Well, the definition of open source that we've had for almost 25 years now is any software that allows you to use it in an unencumbered way.

7. proponent [prəˈpəʊnənt] - (noun) - A person who advocates for something in public; a supporter. - Synonyms: (advocate, supporter, champion)

One of the things Mozilla is doing is working with people who are trying to really popularize that level of privacy inside of an AI system.

8. monopoly [məˈnɑːpəli] - (noun) - Exclusive possession or control of the supply or trade in a commodity or service. - Synonyms: (domination, cartel, control)

And you know, the idea that one or two or three companies are going to have monopoly on safe AI, to me, that feels like a really risky thing for society to do.

9. nonprofit [nɑːnˈprɒfɪt] - (adjective) - Not operated for the purpose of making a profit; often refers to organizations that serve the public interest. - Synonyms: (charitable, not-for-profit, altruistic)

But those are nonprofit open source AI labs that are building, you know, models that are from end to end open

10. harms-based [hɑrmz-beɪst] - (adjective) - An approach focusing on identifying and mitigating specific risks and harm potential. - Synonyms: (risk-based, precautionary, protective)

And, you know, that makes sense to us, but it should be based on evidence and should be based on a kind of harms based approach.

Mozilla’s President Urges Consumers To Learn The Risks Of Different AI Systems

Hey there. We're here with Mark Sermon, president of Mozilla, and we're going to be talking about open source AI. Mark, thank you so much for joining us. Great to meet you. I'm happy to be here. Thanks for having me on, Rich.

Sure. So in 2020, long before ChatGPT Mania, you wrote a paper about trustworthy AI. Tell us, what does that mean to you? Well, you know, Mozilla, as you might know, makes Firefox. But we didn't make Firefox for its own sake. We made it because we really want to make sure the Internet was something in the hands of people, right, that people are protected, but people also can make choices. And in the Firefox era, that meant privacy. It meant being able to make a webpage, meant all those things. And trustworthy AI was really us trying to say, what does it look like to have that Firefox effect in the AI era? And there's two things that for us make up trustworthy AI. One, the idea of accountability, which is something often when people talk about ethical AI, responsible AI means if something bad happens, that the people who made that system are held accountable. And that's something we're working through. How do we build that into AI systems? How do we build that into our laws? But the other half, in addition to accountability, is agency. Can we control how the AI works? Can we see how the AI works? Is it working on our side of the table? At a broad philosophical level, those are the two things that make up trustworthy AI for us, agency and accountability.

And how do you really, really measure that on, you know, on an not broad level, on a more granular level, what benchmarks should we use? Well, that's a big thing between a concept like trustworthy AI and like day to day of building stuff is really tricky. And so that's the process that society is going through right now. How do, as we make AI pervasive in everything we do, do we build the expectation that it's being used in a trustworthy way? And so we started to look for just indicators of that. So one might be privacy, which is both actually something that helps with agency. I can choose whether this system is private and, you know, helps with accountability. If you break my privacy, there are privacy laws in a lot of countries. And so like thinking about that in the design of an AI system.

And so if you really dig down and you think about something like Apple Intelligence, it's being advertised as, you know, more private than say, you know, what you would get from OpenAI or Google that is like in the direction of trustworthy. One of the things Mozilla is doing is working with people who are trying to really popularize that level of privacy inside of an AI system. So not only if you got a fancy new iPhone, could you expect that what you're typing to that chatbot stays on your device or with you. But that that would be true of, you know, more affordable systems or, you know, all kinds of systems. So with Mozilla Ventures, one of the companies I'm most proud that we invested in is called Flower AI. And they're basically taking that same technology that's in Apple Intelligence that's coming into iPhone that keeps your AI data on the phone. They're doing that as open source and they're making it something that any developer could build with. So that's one example. Privacy in an AI system is one element you would look for in crossworthy AI, but it's also, you know, transparency, being able to see how things were built, what data is inside. It's also, you know, the ability to kind of audit and check how did the system work. And so there's like lots of things we can kind of go down the line of elements of what make a trustworthy AI system.

And I, you know, I think we're looking at the next few decades of trying to have this dance between moving fast with innovation and also doing it in a way that is trustworthy. Yeah, and you mentioned Apple and Apple Intelligence. You know, they're a company that really sort of markets their privacy prowess. Do you think they are holding their end of the bargain on that? When it comes to AI, I think Apple is better than most when it comes to privacy. And certainly Mozilla has long stood for privacy. And we have great products and we have great products that are privacy protecting in Firefox and things like our vpn. The flip side is Apple, while being good at privacy, and I do trust them, is incredibly closed, incredibly expensive. And so another value that we look for in terms of, you know, getting trustworthy AI out there is doing trustworthy things, but in an open source way. So that it's not only, you know, the most expensive technology that is private or trustworthy, or that you can kind of check for bias and so on, but that it really is everywhere. And so if you look at a contrast to Apple, you might look at Meta, who's moving in the open source direction, but are people going to trust them? And so for us, it's actually, you know, openness and trustworthiness, openness and privacy. Both need to be out there in the system. And we stand for both.

Right. And you, you mentioned open source. There's, there's been a very fiery debate in the AI community over open source and closed source. But just, just so we have it, like, what, what is your definition of open source? Well, the definition of open source that we've had for almost 25 years now is any software that allows you to use it in an unencumbered way. So you can use it for anything, you can use it freely, that you can study it, or you can look inside it, you can see the code, you can see how it works, that you can modify it, so you can take it and build it into something else, combine it with other things and that you can share it. So, like, after I've modified it, I can give it out again to somebody else. And those are what are called the four freedoms of kind of free software or open source. And that's, you know, Linux is built on that, Wikipedia is built on that, Firefox is built that. And $8 trillion in value has been created from open source because it's what the tech industry is built on. When we come to AI, it's still those same four things, but it's a little bit different than software because it's not just that the code needs to meet those four criteria, but you also need the models and the data sets to be things that you can use and share and modify and so on. So we're in a process really, of kind of debating that, figuring out as an industry, what does open source lead in AI? And I think the answer is that it's those same four things. Use, study, modify, share, applied to the software parts of AI, to the models of AI, and actually to the data sets, or at least the kind of the dataset recipe that goes into an AI model.

And the Open Source Initiative, which originally kind of provided the definition of Open Source, is about to release its final Open Source AI definition. And it basically is what I just said, right? And the Open Source Initiative actually this week criticized Meta for quote, unquote, polluting the term. What is your take on that? Well, we're with the Open Source Initiative on that. I mean, llama was absolutely a boon. I mean, you had OpenAI and others really locking up a lot of what had happened in innovation in the AI space on top of a lot of open research from the last couple of decades. And, you know, LLAMA did come along and say, here's a performant LLM that is open, but it is Actually not open source in a full sense. So we use Llama. We have something called Llama file that we've released as software so you can run Llama on a local machine. We're fans, but we're also critics of the license because the license has a bunch of things that aren't Open Source. At 700 million users, which sounds like a lot of people, the license stops being in effect. So on that basic idea that you can use it for anything, which is a rule of open source, that's not true. At 700 million users, it kind of just stops being open source. And there's a couple of other clauses in the license like that. If you just imagine Mark Zuckerberg had built Facebook on top of Linux and then when he got to 700 million users, Linus Torvald, who invented Linux, would like walk up to his door in Palo Alto and knock on it and hold open a bag for money. Like, you know, that wouldn't have been open source. And actually nobody would have built their company if that were the rules. Nobody would have built their company on Linux. And so I do think it's not so much that they're polluting it, but we really do think they need a different license, a truly open source license, if they're going to call llama open source. And we'd welcome that.

So what do you think the fix here for them is? You started to talk about this, but is it, do they change their framing? Do they stop using the term open source, or do they have a different license? I think they just need to change their license. I mean, Mark Zuckerberg got wealthy and built a lot of great stuff on top of real open source software. If they're going to contribute back, that's great, but they need to do it on the same terms.

As we continue to have this debate in the industry, what do you think the biggest misconceptions are of open source? Well, one of the misconceptions, although it's starting to turn around, is that open source AI is more dangerous than closed source AI, that there might be more risks with open source AI than closed source AI. And of course, anytime we have a new technology, especially a new powerful technology like AI, there are risks. I mean, deep bakes or biased technology or kind of misuse of the technology down the line. And you know, the idea that open source is more dangerous has been used to sort of convince regulators at times. I mean, it almost happened in California to write laws that would really shut down open source or would be biased against open Source. And what's happened is a lot of great research has come out, including research that fed into some decisions and guidance by the Department of Commerce federally that said, actually, there's no difference, there's no real marginal risk, no extra risk between open source AI and closed source AI at this stage in time. And so it's really important to look at the evidence, look at the research and see that we do have to look at what are the risks around AI as society, but not to kind of scapegoat or try to shut down open source as being particularly the problem. That's a misconception.

And where do you think that misconception comes from? Well, if I'm cynical, I think the misconception comes from people whose companies are closed source going and lobbying and saying trust us with safety. And you know, the idea that one or two or three companies are going to have monopoly on safe AI, to me, that feels like a really risky thing for society to do. The idea that you would actually, you know, use what's good about open source, that it's transparent and that it's collectively owned and everybody contributing to the safety of the core infrastructure of our society. To me, historically, that's actually what works out.

And you mentioned regulation, and I think you've said in the past that you're for regulation that puts control in the hands of users. What do you think smart regulation would look like in AI? So when we think about AI regulation, we think about three things, competition, privacy, and safety. And so think about competition. That's something that really matters to Forbes readers because it's about business. And we want to make sure that markets are open in the AI era in a way that they kind of haven't been in the social media era in many regards. And it's a chance to get it right. We were really happy to see in the executive order that came from the White House earlier this year. They were calling out competition in AI from early on. So that's one area that is really important. Another is privacy. Like, you don't actually have federal privacy rules in the US but this is technology that we're very intimate with, where we're sharing all of our information. And so having privacy rules that talk about our relationship between ourselves and tech companies, ourselves and an AI, that's actually really important. And the third is around safety. A lot of people like the EU AI act or the California with SB 1047 have been trying to build regulations on AI for safety. And, you know, that makes sense to us, but it should be based on evidence and should be based on a kind of harms based approach. Should be based on a harms based approach. So the EU AI act looks at specific high risk areas that AI might use, like looks at specific high risk areas where AI might be used, like in healthcare or immigration or policing. And so, you know, in those kind of settings, you know, you have to take extra precautions as a company. Whereas the California law really was a blanket sort of if something bad happens that is unspecified, the company will be responsible and that's not tenable. Like we need to be regulating for particular risks and harms and not just on kind of some speculative basis about AI safety.

What companies do you think are doing the best job in upholding open source standards? So, you know, when we talk about trustworthy AI and we think about, you know, what Mozilla is trying to do in the world, there are people who are doing well in open source and there are people who are doing well in some other aspects of trustworthy AI. And we kind of look at the whole ecosystem both as people who are starting to build AI into our own products, people who are investors and in the AI space. And so I say some of the interesting folks on the open source side of it are clearly hugging face, which is a place where you can go and get all the open source AI stuff. And also companies like Mistral, who were some of the first folks to put out commercial open source AI models, we actually use Mistral in some Mozilla products as a kind of LLM inside. And then I think many of the people who are most interesting on the open source side are not companies at all. And so the Allen Institute for AI in Seattle, or there's now one in France called Couillati or Cutie, I never get the pronunciation right. But those are nonprofit open source AI labs that are building, you know, models that are from end to end open. And much like Mozilla foundation or Linux foundation or the Wikimedia foundation, those are players who are trying to build something open source that everybody can kind of own in some ways by it being held in a public interest organization. And I think those are the people who are doing really the most exciting, fully end to end, fully transparent AI work. And then there's lots on the responsibility and trustworthy part. Some that you see with things like Apple Intelligence, you see a lot of really cool AI governance companies coming up like Credo AI and Fiddler AI. So there's, there's just a lot. I mean, if you go and read the Airstreet Capital, let's say this deck every year of like, here's all the, it's like a 250 page deck. And they just put it out recently. All that's going on in AI, there's just so many interesting companies coming up.

And on the flip side, what companies are doing the worst job upholding those standards? Well, the trick of the last few years or that, you know, sort of dastardly plan, if you wanted to be cynical of the last decade, was, you know, OpenAI took the word open. And it's hard to kind of say open source AI and not sometimes skip over the source and say open AI, but they're incredibly closed. And they not only themselves are a closed source company that, you know, everything is only available behind an API in a black box, which there's nothing inherently wrong with that, but they did start out with the promise of being this nonprofit research lab that was going to do open source. And you know, back in 2018, 2019, Sam Altman said, no, that's not the way it's going to work. And they took what was a spirit of open research, like let's say the Google Transformer paper, which really inspired, you know, the whole GPT series of models that OpenAI has built on. They took all of that spirit of open science and kind of locked it all up. And so, you know, certainly if what we need is, which I believe we do, an open public ecosystem of AI, they're going the opposite direction.

And just to set the stakes, like, what in your eyes is the danger of a closed AI system? Well, it's not that there's a danger of a closed AI system. We've got closed software, closed technology in all aspects of the world, and I use some of it too. It's not that open is the only thing we should use. The danger is having an exclusively closed and also exclusively commercial ecosystem for technology. So we talk about public AI as well as open source AI. We just put out a paper on this and public AI is this idea that there should be a counterbalance to the closed and commercial AI in our society. Much like you have cars and you have buses, or you have NBC and you have pbs. In most areas of society, we have both commercial and public options, closed and open options. And in the previous era of the Internet, you had Linux, you had Firefox, you had Wikipedia as public options, nonprofit run, really scaled reliable technology or Internet services. And for us, the danger is going into this AI era where we only have closed and commercial options and no public options. And that's why we are investing in trustworthy AI ourselves as a public interest player. But it's also why we think that folks like the Allen Institute for AI are really exciting because they're building that public lane in this AI era.

And you mentioned open AI. The launch of ChatGPT is kind of what really spurred this current AI frenzy. How has Mozilla changed since then? Well, we've been trying to change Mozilla since well before ChatGPT. And it takes a long time to go from being, you know, a web organization and public interest and commercial organization. We're a hybrid focused on the web. And you know, we really were focused on the web as something we want to make sure worked for all of humanity to also focus on how do we do that same thing in the AI era. But I would say, you know, we started that in about 2018, 2019, put out our trustworthy AI paper, started building open source data sets and working with people on things like auditing. But what's sped things up is as there's been more attention and more resources, we've started to build trustworthy AI thinking and product and investment into everything we're doing. And so we've gone much further towards being a company standing for an organization standing for AI and humanity service. So you're starting to see trustworthy AI, open source AI showing up in Firefox. We've set up a separate AI R&D company. We've got Mozilla Ventures, which has got about 30 or 35 AI companies it's invested in. So I guess the big thing is, you know, we've known we want to bend AI in a better direction for a while now. I would say as there's more frenzy, it's just sped us up because it feels even more urgent.

Well, Mark, thanks so much for joining us. Thank you so much. Really appreciate it.

Artificial Intelligence, Technology, Innovation, Open Source, Trustworthy Ai, Mozilla, Forbes