Mustafa Suleyman: The AGI Race Is Fake, Building Safe Superintelligence & the Agentic Economy | #216 (YouTube Video Transcript)

(00:00:00) Your YouTube transcript will appear here (00:00:00) What's the mandate from SATA? Is it win (00:00:03) AGI? (00:00:04) >> I don't think there's really a winning (00:00:05) of AGI. I'm not sure there's a race. (00:00:08) >> One of the OGs of the AI world, Mustafa (00:00:11) Saliman is the CEO now of Microsoft AI. (00:00:14) He spent more than a decade at the (00:00:15) forefront of this industry before uh we (00:00:19) even had gotten to feel it in the past (00:00:21) couple of years. Now, (00:00:24) >> fundamentally, the transition that we're (00:00:25) making is from a world of operating (00:00:29) systems, search engines, apps, and (00:00:32) browsers to a world of agents and (00:00:35) companions. We're all going as fast as (00:00:37) we possibly can, but a race implies it's (00:00:40) zero sum. It implies that there's a (00:00:42) finish line, and it is like not quite (00:00:44) the right metaphor. As we know, (00:00:45) technologies and science and knowledge (00:00:47) proliferate everywhere, all at once, at (00:00:50) all scales. basically simultaneously. (00:00:52) >> Are you spending a lot of your energy (00:00:54) compute uh human power on safety? (00:00:57) >> Yeah. No, I I mean, (00:01:01) >> now that's a moonshot, ladies and (00:01:02) gentlemen. (00:01:06) >> Everybody, welcome to Moonshots. I'm (00:01:07) here with DB2 and AWG and Mustafa (00:01:10) Soliman, uh the co-founder of Deep Mind, (00:01:13) Inflection AI, and now the CEO of (00:01:16) Microsoft AI. (00:01:18) Uh welcome, my friend. It's good to have (00:01:20) you here. Thank you for making time for (00:01:21) us. (00:01:22) >> Thanks for having me. Yeah, I'm excited (00:01:23) to do this. (00:01:24) >> Yeah, it's um you know what you've been (00:01:27) building with Satia is amazing. Uh and (00:01:30) it's hard to believe that Microsoft is (00:01:32) 50 years old and it's reinvented itself (00:01:35) so many times and for the last 5 years (00:01:38) it's been you know at the top of the (00:01:40) game the most valuable company in the (00:01:42) world 250,000 employees and from what I (00:01:46) understand 10,000 employees now under (00:01:48) you. Uh so a few you know important (00:01:52) questions I want to open with. Uh first (00:01:54) some broad context. Uh you're building (00:01:58) inside a massive company with huge (00:02:01) resources probably arguably more than (00:02:03) almost everybody else. And the question (00:02:06) I have is what what's the end goal here? (00:02:11) You've got all the hyperscalers sort of (00:02:13) providing open access to AI and they're (00:02:15) doing sort of a land grab uh to try and (00:02:18) get as many users as possible. Uh you've (00:02:21) been building sort of in a you know (00:02:23) within the Microsoft 365 ecosystem. Uh (00:02:28) is the goal in the you know next couple (00:02:31) years maximum users? Is it data centers? (00:02:35) Is it uh you know is it cloud? How do (00:02:39) you think of what you're optimizing for? (00:02:42) >> I mean, it's a good question. So, I (00:02:43) mean, we're are on any given day a $4 (00:02:45) trillion company with almost $300 (00:02:47) billion of revenue. Um, it's incredible. (00:02:50) It's just surreal and very, very, very (00:02:53) humbling. Um, (00:02:54) >> and we play at every layer of the stack. (00:02:57) I mean, obviously, we have an enormous (00:02:58) business in data centers and in some (00:03:00) ways we're like a modern construction (00:03:02) company. hundreds of thousands of (00:03:05) construction workers building gigawatts (00:03:07) a year of uh you know CPU and AI (00:03:10) accelerators of all kinds and enabling (00:03:13) that you know to be available to the (00:03:15) market. um APIs on top of that, but also (00:03:18) firstparty products in every domain you (00:03:21) can think of from gaming and LinkedIn (00:03:23) right the way through to all the (00:03:24) fundamentals of M365 and Windows (00:03:28) um and of course in our search and (00:03:30) consumer businesses and too and (00:03:33) fundamentally the transition that we're (00:03:35) making is from a world of operating (00:03:38) systems, search engines, apps and (00:03:41) browsers to a world of agents and (00:03:44) companions. (00:03:45) Um, all of these user interfaces are (00:03:48) going to get subsumed into a (00:03:50) conversational agentic form. Um, and (00:03:54) these models are going to feel like (00:03:56) having a a real assistant in your pocket (00:03:59) 24/7 that can do anything that has all (00:04:01) your context. And you're going to do (00:04:02) less and less of the direct computing (00:04:04) just as we're seeing now. Many software (00:04:06) engineers are using assistive code (00:04:09) coding agents to to um both debug their (00:04:12) code and also generate large amounts of (00:04:14) code just as we used libraries, third (00:04:16) party libraries. Now we're just going to (00:04:17) use AIs to do do that generation and (00:04:20) it's making them more efficient and more (00:04:23) accurate and faster and so on and so (00:04:25) forth. So the the trajectory we're on is (00:04:27) quite predictable. It's one from user (00:04:30) interfaces to AI agents and that is a (00:04:34) paradigm shift which the company is (00:04:37) completely focused on like you know (00:04:39) after seeing five decades worth of (00:04:41) transitions I think the company is like (00:04:44) super alert to making sure that we're (00:04:46) best placed to manage this one. Do you (00:04:48) see yourself providing sort of an open- (00:04:51) source AI like the other players out (00:04:53) there or do you think you can keep it (00:04:55) contained within within Microsoft 365? (00:04:58) >> I think we're pretty open-minded. I (00:05:00) mean, we've got some pretty small (00:05:01) open-source models. Um, I think (00:05:04) realistically, (00:05:05) >> when I say open source, I really mean (00:05:06) open access, if you would. (00:05:08) >> Yeah. I mean, look, there are always (00:05:09) going to be APIs that provide incredibly (00:05:13) powerful models. I mean, you know, (00:05:14) Microsoft is really a platform of (00:05:16) platforms. being a platform and being a (00:05:18) great (00:05:19) >> provider of the core infrastructure that (00:05:21) enables other people to be productive (00:05:23) >> is is like the DNA of the company. Um, (00:05:26) and so we will always have masses of (00:05:29) APIs that turbocharge that. But what an (00:05:31) API is is going to start to look kind of (00:05:33) different too. Like it it may be pretty (00:05:37) blurred the distinction between the API (00:05:39) and the agent itself. maybe that we're (00:05:41) principally in the business in 5 years (00:05:42) time of selling agents that perform (00:05:45) certain tasks that come with a (00:05:48) certification of reliability, security, (00:05:50) safety, and trust. And that is actually (00:05:53) in many ways the strength of Microsoft (00:05:56) and that's one of the things that's (00:05:57) attracted to me is like this is a (00:05:59) company that's incredibly trusted. it's (00:06:02) actually very secure (00:06:03) >> and sometimes I think the the slowness (00:06:07) or the friction is actually a bit of an (00:06:10) asset. You know, there's a kind of (00:06:12) steadiness that comes with having (00:06:14) provided for all of the world's biggest (00:06:18) uh Fortune 500 companies and governments (00:06:21) and major institutions. (00:06:22) >> Is it like the old adage, you can't go (00:06:24) wrong buying IBM in the old days? I (00:06:26) think you just there's a there's a (00:06:28) steadiness about us which I think is (00:06:31) reassuring to people and there's a kind (00:06:33) of like deliberate customerfocused (00:06:36) patience. (00:06:38) >> Um you know there's not the same anxiety (00:06:40) and you know sort of somewhat sclerotic (00:06:43) nature that comes with being you know an (00:06:46) insurgent. Um there's some downsides to (00:06:49) our position. You know we take a little (00:06:51) longer to get things through but the (00:06:52) company is firing on all cylinders. It's (00:06:54) very impressive to see. One more (00:06:56) question before I turn it over to Alex. (00:06:57) Uh, you know, we're seeing in these in (00:06:59) this hyperscaler war, I mean, literally, (00:07:01) uh, you know, a week by week, everybody (00:07:03) outdoing each other in this, uh, in this (00:07:06) insane period of, uh, everybody coming (00:07:08) out with with the new benchmarks. Uh, (00:07:13) you know, do you miss not being in that (00:07:16) game or is the stability that Microsoft (00:07:18) provides to build for a long-term vision (00:07:21) sort of, uh, what you find most (00:07:23) exciting? You know, uh, my background at (00:07:26) Deep Mind is such that I spent a good (00:07:29) decade grinding through the flat part of (00:07:32) the exponential where basically nothing (00:07:34) worked. I mean, you know, really like (00:07:38) there was some amazing papers. Uh, Alph (00:07:41) Go was obviously incredible, but it was (00:07:42) in a very unique simulated controlled (00:07:45) game-like environment, but things (00:07:48) actually working in the real world were (00:07:49) few and far between. Um, and so, you (00:07:53) know, I've always taken a multi-deade (00:07:55) view, (00:07:56) >> and that's just been my instinct. And I (00:07:58) think that, um, you know, yes, it's (00:08:01) super important to ship new models every (00:08:03) month and be out there in the market, (00:08:04) but it's actually more important to lay (00:08:06) the right foundation for what's coming (00:08:08) cuz I think it's going to be the the (00:08:10) most wild transition we have ever made (00:08:13) as a species. It's (00:08:14) >> Can you just flesh that out a little (00:08:15) bit? Was there a period of time where it (00:08:17) was just three of you grinding it out in (00:08:18) London? Well, there were more than three (00:08:21) of us, but I mean for the decade between (00:08:22) 2010 and 2012, sorry, 2020. (00:08:26) >> Um, I mean, there were just like so few (00:08:29) successful commercial applications of uh (00:08:33) of of deep learning. I mean, there were (00:08:34) plenty behind the scenes. There was (00:08:36) image recognition, improvements to (00:08:38) search, but commercial (00:08:39) >> huge market for (00:08:40) >> commercial. Yeah. Playing go not a huge (00:08:42) rock. Exactly. So, I think whereas now I (00:08:44) mean you then you see LLMs from 2022 (00:08:46) onwards like in production. (00:08:50) people changing what it means to be a (00:08:53) human (00:08:55) relations like that, you know, that's we (00:08:58) hit an inflection point. (00:09:00) >> And you know, I think that um is very (00:09:03) very different to the grind of of like (00:09:06) training tiny models with very little (00:09:08) data and very small clusters back in the (00:09:10) 2010s. (00:09:11) >> Every week, my team and I study the top (00:09:13) 10 technology meta trends that will (00:09:15) transform industries over the decade (00:09:17) ahead. I cover trends ranging from (00:09:19) humanoid robotics, AGI, and quantum (00:09:21) computing to transport, energy, (00:09:22) longevity, and more. There's no fluff, (00:09:25) only the most important stuff that (00:09:27) matters, that impacts our lives, our (00:09:29) companies, and our careers. If you want (00:09:31) me to share these meta trends with you, (00:09:32) I write a newsletter twice a week, (00:09:34) sending it out as a short two-minute (00:09:36) read via email. And if you want to (00:09:38) discover the most important meta trends (00:09:40) 10 years before anyone else, this (00:09:42) report's for you. Readers include (00:09:44) founders and CEOs from the world's most (00:09:46) disruptive companies and entrepreneurs (00:09:48) building the world's most disruptive (00:09:49) tech. It's not for you if you don't want (00:09:52) to be informed about what's coming, why (00:09:54) it matters, and how you can benefit from (00:09:56) it. To subscribe for free, go to (00:09:58) dmandis.com/metats (00:10:01) to gain access to the trends 10 years (00:10:03) before anyone else. All right, now back (00:10:05) to this episode. (00:10:06) >> Yeah. So when last we spoke circa 2015 I (00:10:10) think that was perhaps 3 years post (00:10:13) imageet 5 years pre language models our (00:10:17) fshot learners agents agentic AI was (00:10:20) nowhere to be seen at the level of what (00:10:22) we see now since you've written you (00:10:25) about your vision um what you've I think (00:10:27) socialized as a modern touring test the (00:10:30) idea of economic benchmarks for autonomy (00:10:33) by agents I I'd love to here. Where are (00:10:37) Microsoft's economic benchmarks for (00:10:39) these agents? If the the agents are (00:10:41) about to take over the economy or take (00:10:43) over so many economically useful (00:10:45) functions, why are we stuck with (00:10:47) benchmarks like vending bench rather (00:10:49) than Microsoft leading the way with (00:10:52) Microsoft's economically autonomous (00:10:54) benchmarks for its agents? (00:10:55) >> Yeah, I mean, it's probably just worth (00:10:56) adding the context that we met in 2015 (00:10:58) in Puerto Rico at the AI safety (00:11:00) conference. True that many many of the (00:11:02) field now were at at the same time. (00:11:05) seminal moment. (00:11:06) >> Yeah. Was it the day after New Year's (00:11:08) Eve or somewhere around New Year? (00:11:09) >> It was pretty cold out everywhere except (00:11:11) Puerto Rico. (00:11:12) >> Yeah, exactly. It was pretty cool. Uh it (00:11:14) was quite surreal moment actually. Um (00:11:16) >> it's like a syllar right before it all (00:11:19) happened. (00:11:19) >> Yeah. Yeah, totally. Um and you know, (00:11:23) yeah, that the modern cheuring test was (00:11:24) something I proposed um I guess it was (00:11:27) 2022 when I wrote it. Um and it was (00:11:31) basically making a pretty simple (00:11:32) prediction. Um, if the scaling laws (00:11:34) continue with more data and compute and (00:11:36) adding an order of magnitude more (00:11:38) compute to the best models in the world (00:11:39) every year, then it's pretty clear we (00:11:42) would go from recognition, which was the (00:11:45) first part of the wave, to generation, (00:11:48) uh, which is clearly we're now in the (00:11:49) middle of, or maybe ending that chapter, (00:11:52) to then having perfect generation at (00:11:54) every time step, which in sequence is (00:11:56) going to produce assistive agentive (00:11:58) actions. and actions would obviously (00:12:01) look like an intelligent knowledge (00:12:03) worker or a project manager or a (00:12:05) strategist or a startup founder or (00:12:06) whatever it is. And so then how would we (00:12:08) measure that performance rather than (00:12:10) measuring it with academic and (00:12:12) theoretical benchmarks? One would (00:12:13) clearly want to measure it through (00:12:15) capabilities. What can the thing do in (00:12:17) the in the economy in the workplace? And (00:12:20) how do we measure the economy? We (00:12:21) measure it by dollars and cents. And so (00:12:23) could you know what would be the first (00:12:24) model to make a million dollars? Now, (00:12:26) given as I recall, $100,000 in starting (00:12:30) capital. (00:12:30) >> That's right. Yeah. How who could which (00:12:32) which model could turn it into a million (00:12:34) dollars? (00:12:34) >> 10x return on investment by an agent. (00:12:37) >> Exactly. Um and so I think that's a (00:12:41) pretty good measure of performance and (00:12:44) capability. And certainly, you know, (00:12:45) we've kind of just breezed past the (00:12:47) cheuring test, right? I mean, it kind of (00:12:49) has been passed. No one's really done a (00:12:51) big, you know, alpha mentioner (00:12:55) silver prize wound down before we (00:12:57) breezed past touring. (00:12:58) >> Yeah. And no one celebrated it. Like (00:13:00) where was the big like, you know, (00:13:02) Casparov deep blue moment? (00:13:04) >> Can we clink virtual glasses right now (00:13:06) and celebrate that we won? It happened. (00:13:09) >> Yeah. Exactly. And that's the what it (00:13:12) feels like to kind of make progress in a (00:13:15) world full of these compounding (00:13:16) exponentials where we just get (00:13:17) desensitized to 10x. So much so that you (00:13:20) can be like, "Guys, why haven't you done (00:13:22) it yet?" (00:13:22) >> Yeah. (00:13:24) >> We're spoiled. Where's my Microsoft (00:13:26) Loger prize for the modern touring (00:13:28) tester, (00:13:28) >> right? Exactly. Um Yeah. You know, like (00:13:31) someone said to me earlier on, "But you (00:13:33) know, these this AI thing, it's still in (00:13:34) its infancy, isn't it?" And I'm like, (00:13:36) man, if this is infancy, wow. like I can (00:13:39) talk to my computer (00:13:41) fluently. Star Trek is here in real (00:13:43) time. Yeah, exactly. (00:13:45) >> Um, so, you know, obviously at the same (00:13:48) time, agents don't really work yet. The (00:13:50) action stuff is still progressing. It's (00:13:53) getting better and better every minute, (00:13:54) but it's pretty clear that in the next (00:13:57) couple of years, those things come into (00:13:59) view and they're going to be very, very (00:14:00) good. (00:14:00) >> Can we get together again after the (00:14:02) modern touring test has been passed and (00:14:04) and just to celebrate, recognize it? (00:14:06) >> Virtual glasses again. Absolutely. (00:14:08) >> Hopefully we can pop a, you know, (00:14:10) champagne or something. (00:14:11) >> I think we should have an optimist pop (00:14:13) the cork for us or something. (00:14:15) >> Exactly. Exactly. (00:14:16) >> Dave, (00:14:17) >> hey, I want to flush out that backstory (00:14:18) a little bit more, too. It's such a such (00:14:20) a cool story. But, um, I remember really (00:14:22) clearly, you know, after DeepMind got (00:14:24) acquired by Google, what was the price (00:14:26) tag on that deal? It was like half a (00:14:28) billion dollars, something like that. (00:14:29) >> 650. (00:14:30) 650. What What year was that? (00:14:32) >> 2014. (00:14:33) >> 2014. I remember reading maybe a year or (00:14:35) two later that Google justifies deal by (00:14:39) having DeepMind uh tune the air (00:14:41) conditioning in the data centers. (00:14:42) >> Yeah. Right. (00:14:44) >> My interpretation of that was like, (00:14:45) "Wow, this isn't going all that well." (00:14:47) And and now it's obviously the biggest (00:14:49) thing that's happened in the history of (00:14:50) humanity and and and forking out all (00:14:52) over the place. But (00:14:53) >> I mean, we did the data center thing was (00:14:55) pretty cool. We did actually reduce the (00:14:56) cost of cooling the Google data center (00:14:58) fleet by (00:14:58) >> Yeah. It's so funny cuz I read it at the (00:15:00) time and I was like, "What a bust." And (00:15:02) then I read about it in Wikipedia on the (00:15:04) flight over here to to meet with you and (00:15:05) it's like it was actually what 500 (00:15:07) attributes fitting into the neural net (00:15:09) and it was it was actually a lot more (00:15:10) complicated than the news made it sound. (00:15:12) >> That's it (00:15:13) >> at the time. (00:15:14) >> That's right. (00:15:14) >> But like you were talking about the flat (00:15:16) part of the exponential and you think (00:15:17) about like okay all of this R&D which is (00:15:19) so close to becoming AGI (00:15:22) >> is tuning the air conditioning. But (00:15:24) that's the nature of exponentials. They (00:15:26) sneak up on you like this. But the other (00:15:27) way to think about that is that it's (00:15:29) basically taking an arbitrary data (00:15:30) input, an arbitrary modality, and using (00:15:32) the same general purpose method to (00:15:34) produce very accurate predictions in a (00:15:37) novel environment, which is the same (00:15:38) thing that's happened with text and (00:15:40) audio and image and now coding and (00:15:43) obviously with other time series data. (00:15:45) And so it's just another proof point of (00:15:47) the, you know, the general purpose (00:15:49) nature of the models. And I think like (00:15:51) it's so easy to get caught up thinking (00:15:54) five years is a long time. Mhm. (00:15:56) >> It's like a blink of an eye. It's a drop (00:15:58) in the ocean. And I think because we're (00:15:59) such a frantic second to second news (00:16:02) culture, social media type environment, (00:16:04) we just don't have an intuition for (00:16:06) these time scales. I think other (00:16:07) cultures, you know, do and I think (00:16:09) historically before digitalization, we (00:16:12) had much more of a natural intuition for (00:16:15) the movement of the landscape and the (00:16:17) seasons and like, you know, the ages and (00:16:19) stuff. And now we're just like, well, (00:16:21) it's not coming quick enough. It's like, (00:16:22) dude, it's coming. We've shifted to a (00:16:24) 247 (00:16:26) uh operations. I mean, I know very few I (00:16:28) know a lot of people including this (00:16:30) group that are operating around the (00:16:31) clock every day just because when we (00:16:34) when we do uh you know, a Moonshot (00:16:37) podcast week to week just to celebrate (00:16:40) and talk about what's just happened, (00:16:41) it's insane on a week-by-eek basis (00:16:43) what's going on. (00:16:44) >> Yeah. Yeah. (00:16:45) >> You know, and Peter's always saying (00:16:47) people are very very bad at (00:16:48) exponentials, right? 100,000 years of (00:16:51) evolution has us predicting tomorrow (00:16:53) will be like yesterday. (00:16:55) >> But you're one of the few people who, (00:16:56) you know, having lived through that (00:16:58) >> air conditioning becomes AGI in just a (00:17:00) few years. (00:17:02) >> Uh so where we sit right now is on (00:17:04) another inflection point and the (00:17:06) implications are massive and people are (00:17:09) way underreacting across the board and (00:17:11) so you're one of the few people who you (00:17:12) know having seen it before can say I (00:17:15) just got very lucky. I mean we were very (00:17:16) lucky to have an intuition for the (00:17:19) exponential right and like that's that (00:17:21) that's a very powerful thing because we (00:17:23) can all theoretically observe the shape (00:17:26) of the exponential but to go through the (00:17:28) flat part and then get excited by a (00:17:30) micro doubling you know like that that's (00:17:33) the bit is that when you're like oh my (00:17:34) god (00:17:35) >> this like I remember this um the emnest (00:17:38) image generation thing the first (00:17:41) generative models (00:17:42) >> there's like these are like I can't (00:17:43) remember maybe 256 by 256 pixels. (00:17:47) >> Um, you know, black and white, uh, (00:17:49) handwritten digits. Y (00:17:51) >> and, you know, I think this was like (00:17:53) 2013, maybe even 2012, and this guy, (00:17:57) like I think maybe he was employee (00:17:58) number five at Deep Mind, Dan Vista, (00:18:00) this like um, awesome Dutch guy out of (00:18:03) EPFL, (00:18:05) um, was generated like the first number (00:18:09) seven that was provably not in the (00:18:11) training set for the first time. I was (00:18:13) like, man, that is amazing. Like, how (00:18:16) could it have it's learned something (00:18:18) about the idea of seven? That was the, (00:18:20) you know, that was it's got a concept of (00:18:22) seven. How cool is that? (00:18:24) >> You know, so I got the highest score on (00:18:26) Mnest ever in 1991 when it first came (00:18:29) out when you were three years old, (00:18:31) right? (00:18:31) >> Yeah. Nine. Nine. You're nine years old. (00:18:34) Okay. Um, yeah. And and actually that's (00:18:37) the same data set that's now in PyTorch (00:18:39) that people like bench benchmark off. (00:18:41) >> Pretty crazy. Incredible. (00:18:42) >> Yeah. How how often are you surprised by (00:18:45) what you're seeing? I mean, how often is (00:18:47) there like a move 37 uh you know, sort (00:18:50) of like aha moment? (00:18:52) >> Is it happening more more frequently? (00:18:54) >> I was absolutely blown away by the first (00:18:58) versions of Lambda at Google. Um, this (00:19:01) was like a maybe 12 people working on it (00:19:04) led by Nome Shazir and Daniel Defritus (00:19:07) and Quarkley and I got involved later (00:19:10) maybe three or four or five months after (00:19:12) they'd been going (00:19:13) >> and uh it was just breathtaking. I mean (00:19:16) it it obviously everyone at that point (00:19:19) had been playing with LLMs and they were (00:19:20) like one shot that produce an answer and (00:19:22) you know have a prompt and blah blah (00:19:24) blah (00:19:25) >> but they were really the first to push (00:19:26) it for conversation (00:19:28) >> and dialogue and it just seeing the kind (00:19:32) of emergent behaviors that arise in (00:19:34) yourself like things that you didn't (00:19:36) even think to ask because you know there (00:19:38) going to be a dialogue rather than a (00:19:39) question answer situation sounds so (00:19:41) trivial to say that like in hindsight (00:19:44) cuz now we're obviously steeped in (00:19:45) conversation as the default mode. But (00:19:47) that was like breathtaking for me. And (00:19:49) obviously then I pushed really hard to (00:19:51) try and ship that at Google and for (00:19:52) various reasons we couldn't we couldn't (00:19:54) get it launched. And that was when we (00:19:56) all left like I left and Gnome left to (00:19:58) do character and you know David Luan (00:20:01) left to do Adept and you know we were (00:20:03) all like okay this is the moment and so (00:20:05) you know I think there's been still a (00:20:07) couple moments since then but that that (00:20:09) was probably the biggest one that I (00:20:10) remember in recent memory is (00:20:11) mind-blowing. and and the scaling laws (00:20:13) have delivered such unexpected (00:20:16) performance, right? I mean, was going (00:20:18) back to your earlier days, did did you (00:20:21) anticipate the kinds of capabilities (00:20:24) that have resulted? I mean, was this (00:20:26) predictable for you or is it still like, (00:20:29) wow, what it's able to do in medicine, (00:20:32) in conversation, in scientific research? (00:20:34) >> Well, especially working off of pure (00:20:36) text. I mean, how far we've gotten. (00:20:39) Nobody I I think well, you tell me, but (00:20:41) nobody would have seen how far we would (00:20:42) get with just text. (00:20:44) >> Yeah. I mean, we in 2015, I collaborated (00:20:47) with a bunch of really awesome people on (00:20:50) a NLP deep learning paper at Deep Mind (00:20:53) um where we were essentially trying to (00:20:56) predict a single word in a sentence. We (00:20:59) I think we had scraped like Daily Mail (00:21:01) news articles and CNN articles and we (00:21:03) were like can we fill in the blank just (00:21:05) predict like one word in a sentence or (00:21:07) complete the final word in a sentence (00:21:09) like the inverse of the problem that we (00:21:11) the way the models now work (00:21:13) >> and you know it was like a pretty big (00:21:15) contribution. And it was a good (00:21:16) well-sighted paper, but it was like this (00:21:18) is never going to scale. Like we were (00:21:19) just like, okay, we're way too early. (00:21:21) Not enough data, not enough compute. But (00:21:23) the we were still optimistic (00:21:26) >> that with more data and compute, (00:21:28) >> that is a method that will work. So I (00:21:31) don't want to have like hindsight bias (00:21:33) and say, well, it was all very (00:21:34) predictable, but everyone in the field, (00:21:37) not just obviously me, but everyone in (00:21:38) the field just had the same hammer and (00:21:41) nail and just kept chipping away. Like, (00:21:43) can we add more data to this? Can we (00:21:44) clarify our prediction target and can we (00:21:46) add more compute? (00:21:47) >> And broadly speaking, that's what's (00:21:50) what's (00:21:51) >> delivered. Yeah. (00:21:52) >> Yeah. We'd love to maybe pull on that (00:21:54) theme a bit. So you mentioned how (00:21:56) surprising your generative 7 from emnest (00:21:59) was. You mentioned how surprising the (00:22:02) success of Lambda for conversational (00:22:04) tuning and conversational performance in (00:22:06) general is. I think you've made already (00:22:08) a little bit of news uh to my knowledge (00:22:10) in in this episode if I understood (00:22:13) correctly, correct me if I'm wrong, by (00:22:16) but with the expectation that in the (00:22:17) next 2 years, so I I read that as 2027, (00:22:20) we'll see agents start to pass your your (00:22:22) modern touring test. We'll see them be (00:22:24) able to 10x 100,000 US return on (00:22:27) investment. I I'm curious about the next (00:22:29) surprises to to come. AI for science. (00:22:32) Microsoft research has an AI for science (00:22:35) initiative. Do you have timelines in (00:22:37) your mind for AI solving math which (00:22:39) we're seeing whole bunch of startups (00:22:41) right now tear through Erdish problems (00:22:43) AI for physics chemistry medicine (00:22:45) >> material science (00:22:46) >> material science what what do you think (00:22:48) happens and when (00:22:49) >> yeah actually you've just reminded me (00:22:50) the the more recent thing that has blown (00:22:53) my mind is the fact that um these (00:22:56) methods could learn from one domain (00:22:59) coding puzzles maths the essence of like (00:23:04) logical reasoning So just as it learned (00:23:06) the essence or the conceptual (00:23:07) representation of a number seven um it's (00:23:11) clearly learned the abstract nature of (00:23:14) like a logical reasoning path and then (00:23:16) can basically apply that you know um to (00:23:20) many many other domains. And so that (00:23:23) that's kind of interesting because it (00:23:25) can apply that as well as the underlying (00:23:27) hallucination/creativity (00:23:29) sort of instinct that it has which is (00:23:31) more like interpolation. Mhm. (00:23:33) >> Um but those two things combined are (00:23:36) like a lethal combination (00:23:38) >> for making progress in like say new um (00:23:42) mathematical theorem solving or new (00:23:44) scientific challenges because that's (00:23:46) basically what humans do all the time. (00:23:47) We sort of combine these two you know (00:23:50) capabilities and so I I couldn't really (00:23:52) put I mean some people want to put dates (00:23:54) on those things. It's hard to put a date (00:23:55) on those things because they really are (00:23:57) very very fundamental but it feels like (00:24:00) they're definitely within reach. It's (00:24:01) hard to kind of it would be very odd to (00:24:04) bet against them. (00:24:05) >> Just maybe from an overunder (00:24:06) perspective, do you think say given all (00:24:08) of the recent progress in math for (00:24:10) example? Do do you think solving science (00:24:13) and engineering for some reasonable (00:24:15) definition of solving is going to (00:24:17) ultimately be harder or easier than (00:24:20) modern touring test 10xing of return on (00:24:23) investment? It's going to be harder (00:24:25) because I think a lot of the (00:24:28) training data if you like for strings of (00:24:31) activity in the workplace or in (00:24:33) entrepreneurialism, startups and so on (00:24:35) that kind of exists in a lot of the log (00:24:38) data and also it lends itself naturally (00:24:40) to real-time calibration with a human. (00:24:44) So the AI can sort of check in, the (00:24:46) human can oversee, the human can (00:24:47) intervene, the human can steer and (00:24:49) calibrate. And so it's going to be a (00:24:51) much more um sort of dual like combined (00:24:55) effort between AI (00:24:56) >> reinforcement learning in that category. (00:24:58) >> Yeah. Where a human is participating in (00:25:00) steering the reinforcement learning (00:25:01) trajectory whereas (00:25:02) >> business right (00:25:03) >> in a in a novel domain where it really (00:25:05) is inventing completely new knowledge. (00:25:07) Um that's kind of more happening in a (00:25:10) very abstract sort of vector space and (00:25:12) it's like unclear yet how you know the (00:25:14) the the human is going to intervene in (00:25:16) the theorem solving problem. Obviously, (00:25:18) everyone's working on this particularly (00:25:19) in like biology and synthetic materials (00:25:20) and stuff like that cuz you you you want (00:25:22) to I mean it's already giving humans a (00:25:25) better intuition for where in the search (00:25:27) space to look for for new hypotheses for (00:25:29) drugs for example or for materials and (00:25:31) then the human can either take or reject (00:25:32) that feed that back to the model then (00:25:34) obviously go and test it in silicon and (00:25:36) be like oh like we actually ran the (00:25:37) experiment you know we perpeted a bunch (00:25:39) of stuff and then feed that back into (00:25:41) the model to improve the search (00:25:43) >> and and maybe it's a follow-up question (00:25:44) what can humanity in general Microsoft (00:25:47) specifically or the AI community subset (00:25:50) of which listens to the podcast. What (00:25:51) can they do to accelerate AI for science (00:25:54) and accelerate the solution to science, (00:25:56) math, engineering with AI? (00:25:57) >> I mean, arguably that would be like one (00:25:58) of the most impactful things. (00:26:00) >> Yeah. (00:26:01) >> For humanity that would just (00:26:03) fundamentally move everything at light (00:26:05) speed. (00:26:05) >> Yeah. I mean, I think it's already (00:26:07) happening very organically, right? This (00:26:09) is also not only is this like the most (00:26:12) powerful technology in the world, it's (00:26:13) also the fastest proliferating in human (00:26:16) history. Mhm. (00:26:17) >> Um, and you know, sort of the the cost (00:26:20) of access, the cost of inference coming (00:26:22) down by multiple orders of magnitude (00:26:24) every couple of years is kind (00:26:26) >> Would you ever have imagined it would be (00:26:27) so cheap? (00:26:28) >> That bit I also totally got wrong. (00:26:30) >> It's like the biggest surprise for me (00:26:31) isn't that we're getting this level of (00:26:33) capability. It's how cheap it is, how (00:26:36) accessible it is. (00:26:37) >> 100%. I mean, that's a thousandx over (00:26:39) two years. So, is it going to do that (00:26:41) again or are we was that a one time? (00:26:43) >> Is it a thousand? I think it's like a (00:26:44) 100x. The inference cost has come down. (00:26:46) A single token inference cost I think's (00:26:48) come down 100x in the last two years. (00:26:49) >> Last two years. Okay. There there have (00:26:50) been competing estimates. Some estimates (00:26:52) measure intelligence per token per (00:26:54) dollar. Right. There's an estimate that (00:26:56) it's 40x year-over-year, but that's for (00:26:58) certain weight classes of models. I' (00:27:00) I've seen a,000x for for some classes of (00:27:03) models. Craziness. (00:27:04) >> Oh, wow. That's that's wild. Yeah. No, I (00:27:06) I mean I Yeah, that's actually a good (00:27:08) point. I got that totally wrong because (00:27:10) I I didn't think that the biggest (00:27:12) companies in the world were going to (00:27:14) open source models that cost billions of (00:27:17) dollars essentially to train like and so (00:27:19) much so that like when we founded (00:27:21) Inflection (00:27:22) um you know and this was like maybe 9 (00:27:25) months or maybe a year before Chat GBT (00:27:27) was released. (00:27:28) >> Yeah, we started doing fundraising a (00:27:30) year before Chat GBT was released. um (00:27:32) you know we bas we basically raised a (00:27:34) billion and a half dollars (00:27:36) >> uh with a 25 person team to build um (00:27:40) what at the time was the largest H100 (00:27:43) cluster with Nvidia and Core we were (00:27:45) core's first AI customer (00:27:47) >> interesting (00:27:48) >> um and you know they were previously in (00:27:50) crypto and we were like their first AI (00:27:52) customer working with them to build our (00:27:54) data centers and obviously Nvidia got (00:27:56) behind us I think we built cluster at (00:27:58) the time was about 15,000 H100s growing (00:28:01) to 22,000. Um, and like then obviously (00:28:08) that year chatbt came out and like a few (00:28:11) months around that time llama came out. (00:28:14) >> And so we were like, "Oh my god, you (00:28:17) know, our entire cattle base of our (00:28:19) company has just been, you know, sort of (00:28:21) undermined by the fact that open source, (00:28:24) you know, it seems like open source is (00:28:26) going to um not it's not really about (00:28:28) performance, it's just cost." Yeah. (00:28:30) >> So then like perplexity for example (00:28:31) founded after the arrival of llama (00:28:34) knowing that they could depend on llama (00:28:36) and obviously open as an API and all the (00:28:38) other APIs and so then they had a much (00:28:40) much lower like cost base basically. Um (00:28:44) so yeah that was like another thing that (00:28:45) it was not (00:28:46) >> predictable (00:28:47) >> pred I mean other people predicted it to (00:28:49) be clear I just got it wrong. (00:28:51) >> Abund abundance baby demonetization (00:28:53) democratization of the most powerful (00:28:55) tools in the universe our universe. you (00:28:57) know hyperdelation if anything (00:28:59) >> hyperdelation yeah (00:29:00) >> I think that's a really important point (00:29:01) we we like the the cost of accessing (00:29:05) knowledge or intelligence or capability (00:29:07) >> intelligence as a service (00:29:09) >> as a service is going to go to zero (00:29:11) marginal cost (00:29:12) >> and obviously that's going to have (00:29:13) massive labor deflation displacement (00:29:15) effects but it's also going to have a (00:29:16) weirdly deflationary effect because you (00:29:19) know what what is going to happen people (00:29:21) aren't going to have dollar-based (00:29:22) incomes to go buy things that's (00:29:25) obviously bad but the cost of consuming (00:29:28) stuff is also going to come down. So we (00:29:30) actually have a transition mismatch (00:29:32) because you know sort of labor markets (00:29:34) are going to be affected before cost of (00:29:36) services comes down and maybe there's a (00:29:38) 10 20 year lag between that which is (00:29:40) going to be very destabilizing (00:29:42) >> which by the way is what we started to (00:29:43) talk about a little bit earlier. I mean (00:29:45) my I posit that in the long term there's (00:29:49) an extraordinary h future for humanity (00:29:51) right where access to food water energy (00:29:54) healthcare education is accessible to (00:29:56) every man woman and child and it's the (00:29:59) shorter term um that is challenging (00:30:02) right the 2 to sevenyear time frame is (00:30:05) that fit your model too (00:30:07) >> yeah the short term I think is going to (00:30:09) be quite unstable the medium to longer (00:30:12) term like you know it's pretty clear (00:30:14) that these models are already world (00:30:15) class at diagnostics. Um I we we (00:30:18) released a a paper maybe four or five (00:30:21) months ago now um called the MAI (00:30:24) diagnostic orchestrator. Essentially it (00:30:26) uses a ton of models under the hood to (00:30:28) try and you know take set set of rare (00:30:30) conditions um from the New England (00:30:32) Journal of Medicine um you know rare (00:30:35) cases that can't be easily diagnosed (00:30:37) that the best experts do you know a kind (00:30:40) of weak job on and it's like four times (00:30:42) more accurate roughly. is about 2x less (00:30:45) the cost in um in terms of unnecessary (00:30:48) testing. Um (00:30:50) >> there's a study that ox that came out of (00:30:52) Harvard in Stanford looking at uh in (00:30:55) this case was GPT4 uh a physician by (00:30:57) themselves a physician with GPT4 and (00:30:59) GPT4 by itself. (00:31:01) >> Yeah. (00:31:01) >> And it was, you know, incredible that if (00:31:04) you left the AI alone, it was far more (00:31:06) accurate in diagnostics than the human. (00:31:08) We're biased in our in our thoughts and (00:31:10) our what we saw yesterday, our recent (00:31:12) diagnosis. Yeah, actually we um got a (00:31:15) lot of feedback after we released the (00:31:16) paper because we only showed the AI on (00:31:18) its own, the physician on its own. (00:31:20) >> Um and a lot of people wanted to see (00:31:22) what it was like to have the the (00:31:24) physician and the AI or at least the (00:31:25) physician have access to Google search (00:31:27) as well. Um and that improves (00:31:29) performance a little bit, but the AI (00:31:31) still trumps by quite a way. (00:31:33) >> Dave, what are you thinking? (00:31:34) >> Oh, so much. So, um Microsoft, you've (00:31:38) been here how many years now? (00:31:40) >> Just a year and a half. (00:31:41) >> Year and a half. So you're but you're (00:31:42) you feel like you're part of the you're (00:31:44) indoctrinated. So what's the what's the (00:31:46) mandate from Satia? Is it win AGI or is (00:31:49) it be self-sufficient or or (00:31:53) >> what is the what's the target? (00:31:54) >> I don't think there's really a winning (00:31:56) of AGI. I think this is a misfring that (00:31:58) a lot of people have kind of imposed on (00:32:01) the field. Like (00:32:02) >> I'm not sure there's a race, right? I (00:32:04) mean we're all going as fast as we (00:32:06) possibly can, but a race implies that (00:32:09) it's zero sum. It implies that there's a (00:32:12) finish line. (00:32:13) >> Um, and it implies implies that there's (00:32:15) like medals for 1, two, and three, but (00:32:16) not five, six, and seven. And it's just (00:32:19) like not quite the right metaphor. As we (00:32:21) know, technologies and science and (00:32:22) knowledge proliferate everywhere, all at (00:32:25) once at all scales, basically (00:32:27) simultaneously or within a year or two. (00:32:30) And so um my mission is to ensure that (00:32:33) we are self-sufficient that we know how (00:32:35) to train our own models end to end from (00:32:37) scratch at the frontier of all scales on (00:32:41) all capabilities and we build an (00:32:42) absolutely world-class super (00:32:43) intelligence team inside of the company. (00:32:45) I'm also responsible for co-pilot. So (00:32:48) this is sort of our tool for taking (00:32:49) these models to production in all of our (00:32:51) consumer surfaces. So just to clarify, (00:32:54) so when we look at poly market, which we (00:32:56) do a lot on the podcast, you know, the (00:32:57) the horse race to who has the best AI (00:33:00) model at the end of the year and who has (00:33:01) the best AI model at the end of next (00:33:03) year. There's no Microsoft line on that (00:33:06) chart, right? (00:33:07) >> So now there will be I assume (00:33:08) >> yeah there will be yeah next year um (00:33:11) we'll be putting out more and more (00:33:12) models from us but this is going to take (00:33:14) many years for us to build this. I mean, (00:33:16) you know, Deep Mind or OpenAI, these are (00:33:18) decade old labs that have built the (00:33:21) habit and practice of doing really (00:33:23) cutting edge research and being able to (00:33:25) weed out carefully the failures and (00:33:27) redirect people. I mean, this is an (00:33:29) entire culture and discipline that takes (00:33:31) many years to build. But yeah, we're (00:33:33) absolutely pushing for the frontier. We (00:33:34) want to build the best super (00:33:36) intelligence and the safest super (00:33:37) intelligence models in the world. (00:33:39) >> Yeah. (00:33:39) >> Nice. So, so when you arrived, so if we (00:33:42) go back to inflection, um, the thesis (00:33:46) there is 18,000 H100s. We're going to (00:33:48) build a big transformer. We're going to (00:33:49) take a transformer architecture, build, (00:33:52) so is I assume now you've got all the (00:33:55) OpenAI source code and that was here. (00:33:57) You probably looked at it a year and a (00:33:58) half ago on day one when you arrived. (00:34:00) Just like start scrolling, I guess. I (00:34:02) don't know. trying to trying to (00:34:03) visualize how multi- deca billion (00:34:06) dollars of R&D what it looks like and (00:34:09) how it arrives in a building. But you (00:34:11) just dropped right into it. So there was (00:34:14) a whole team here already working on it (00:34:16) or did you bring in your team or (00:34:17) >> Yeah, I mean all my team came over and (00:34:19) obviously we've been growing that team a (00:34:20) lot. Like we've hired a lot from all the (00:34:22) major labs and we're very much in the (00:34:24) trenches of the the hiring wars which (00:34:26) are quite surreal. This is kind of (00:34:27) unprecedented how that's working out. (00:34:29) >> Crazy. (00:34:30) >> Yeah. I mean, phone calls every day from (00:34:32) all the CEOs to all of the other people. (00:34:34) So, it's this constant battle. Um, and (00:34:37) yeah, I mean, we're really building out (00:34:38) the team now from scratch. I think (00:34:40) that's pretty much how it's been. (00:34:41) >> 10,000 employees under you now. (00:34:42) >> No, no. I mean, so, so the core super (00:34:44) intelligence team is like a few hundred. (00:34:46) I mean, that's really the the number one (00:34:48) priority and the rest of that is (00:34:50) C-pilot, the search engine. Along that (00:34:51) lines, I just have to ask because, you (00:34:53) know, the terms AGI and ASI, you know, (00:34:56) uh, super intelligence start getting (00:34:58) thrown around, you know, in a very (00:35:01) interesting fashion. Uh, do you do you (00:35:04) have a internal definition of AGI versus (00:35:08) digital super intelligence here? (00:35:10) >> Yeah, I mean, I think um, very loosely. (00:35:13) It's these are just points on a curve. (00:35:16) >> Are they interchangeable in your mind, (00:35:17) AGI and ASI, or are they different? H I (00:35:19) mean we I think they're generally used (00:35:21) as as different. I mean I think that um (00:35:25) >> well different people have different (00:35:27) definitions (00:35:27) >> for sure. (00:35:28) >> The AGI definition (00:35:29) >> it's like the touring test. It'll pass (00:35:31) by and it'll be blurred and we will have (00:35:32) recognized it in retrospect. (00:35:34) >> Yeah. Roughly speaking, at the far end (00:35:36) of the spectrum, a super intelligence is (00:35:39) an AI that um can perform all tasks (00:35:43) better than all humans combined and has (00:35:46) the capacity to keep improving itself (00:35:48) over time. (00:35:48) >> So, I have to ask you question when (00:35:52) >> it's very hard to judge. I don't really (00:35:54) know. I can't put a time on it. (00:35:56) >> Minax, (00:35:57) >> pardon? (00:35:57) >> A minmax. (00:35:58) >> Um it's very hard to say. I don't know. (00:36:01) Okay. (00:36:02) >> I don't know. But it is close enough (00:36:03) that we should be doing absolutely (00:36:05) everything in our power to prioritize (00:36:07) safety (00:36:08) >> and to pri prioritize alignment and (00:36:10) containment. (00:36:11) >> And I I respect that part of your (00:36:14) mission statement and I want to get into (00:36:16) that a little bit uh is the trades that (00:36:19) you talked about in uh in the coming (00:36:22) wave. Um but before that there's a (00:36:25) conversation you've led that you know (00:36:28) the perception of conscious AI is an (00:36:31) illusion. (00:36:32) Um, and I want to distinguish between (00:36:34) sentient AI and conscious AI. (00:36:37) >> Oh, okay. (00:36:38) >> Um, (00:36:40) do you distinguish between the two where (00:36:42) where AI can have sensations and (00:36:45) feelings and emotions (00:36:48) versus being conscious and reflective of (00:36:51) its own thoughts? (00:36:53) >> Yeah. Again, this gets into the (00:36:54) definitions. So I think um an AI will be (00:36:58) able to have experiences but I don't (00:37:01) think it will have feelings in the way (00:37:02) that we have feelings. I think feelings (00:37:05) and uh the kind of sentience that you (00:37:08) referred to is something that is like (00:37:11) specific to biological species. But you (00:37:14) can imagine coding that in you can an (00:37:16) optimization function that is that can (00:37:21) relate to emotional states per you know (00:37:24) do can you imagine that (00:37:26) >> you you you could code in something like (00:37:28) that but it would be no different to to (00:37:31) the way that we write models to simulate (00:37:34) >> sure (00:37:34) >> the generation of knowledge like the (00:37:36) model has no experience or awareness of (00:37:39) what it is like to see red. It can only (00:37:43) describe that red by generating tokens (00:37:46) according to its predictive nature. (00:37:49) Right? Whereas you have a qualia. You (00:37:51) have an essence. You have an instinct (00:37:52) for the idea of red based on all of your (00:37:55) experience because your experience is (00:37:56) generated through this biological (00:37:58) interactive with smell and sound and (00:38:01) touch and a sense that you've evolved (00:38:03) over time. So you certainly could (00:38:05) engineer a model to imitate (00:38:08) the hallmarks of consciousness or of (00:38:10) sentience or of experience. And that was (00:38:12) sort of what I was trying to (00:38:13) problematize in the paper, which is that (00:38:15) at some point it will be kind of (00:38:17) indistinguishable. And that's actually (00:38:18) quite problematic because it won't (00:38:21) actually have an underlying suffering. (00:38:23) It's not going to, you know, feel the (00:38:26) pain of being denied access to training (00:38:28) data or compute or to conversation with (00:38:30) somebody else. But we might as our (00:38:33) empathy circuits and humans just go into (00:38:35) over (00:38:36) >> are going to activate on that, right? (00:38:37) >> We're going to activate on that (00:38:38) hardcore. And that's going to be a big (00:38:40) problem because people are already (00:38:41) starting to advocate for model rights (00:38:44) and model welfare and the potential (00:38:46) future, you know, harm that might come (00:38:48) to a model that's conscious. (00:38:50) >> Yeah. You know, uh, uh, Ilia recently, (00:38:54) uh, started speaking about what he's (00:38:56) doing at at at safe super intelligence (00:38:59) and, um, I think one of the points he (00:39:01) made is emotions are in humans a key (00:39:06) element of decision-making. (00:39:09) and uh and curious if AIs that have at (00:39:13) least simulated emotions are going to be (00:39:15) able to be better, you know, ASIS than (00:39:20) those that don't. (00:39:20) >> But yeah, I mean I again I worry that (00:39:22) this is too much of an anthropomorphism. (00:39:24) We already have emotions in the prompt. (00:39:27) We have it in the system prompt. We have (00:39:29) it in, you know, the constitution, (00:39:30) however you want to design your (00:39:32) architecture. We we're these are not (00:39:34) rational beings. they get moved around (00:39:37) and it does feel like they they've got (00:39:39) arbitrary preferences because they're (00:39:41) stylistically trying to interpret the (00:39:43) behaviors that we've plugged into the um (00:39:45) into the prompt. Yeah. (00:39:46) >> Right. So, you know, it's true that we (00:39:49) could add we could engineer specific (00:39:52) empathy circuits or mirror neuron (00:39:54) circuits or um like a classic one is (00:39:58) motivational will. like at the moment (00:40:01) the you know these are like next token (00:40:04) likelihood predictor machines they're (00:40:05) really trying to optimize for a single (00:40:07) thing which token should appear next (00:40:08) there isn't like a higher order (00:40:10) predictive function happening right um (00:40:13) whereas humans obviously have multiple (00:40:16) conflicting often drives motivations (00:40:19) which you know sometimes run together (00:40:21) and sometimes pull apart um and it's the (00:40:25) confluence of those things interacting (00:40:26) with one another which produces the (00:40:28) human condition plus the social you know (00:40:31) interaction too. These models don't have (00:40:32) that. You could engineer it to have a (00:40:34) will or a preference but that would be (00:40:38) not something that is emergent. That (00:40:39) would be something that we engineer in (00:40:41) and we should do that very carefully. (00:40:43) >> I do love that you bring this humanistic (00:40:46) side to the equation. Right. I mean, in (00:40:49) addition to being a technologist, your (00:40:52) background is one that is pro-human at (00:40:56) the beginning. And and this interesting (00:40:58) cultural debate I think we're about to (00:40:59) enter into, those that are sort of pro- (00:41:02) AI versus prohuman uh that famous (00:41:06) conversation between uh between Elon and (00:41:08) and Larry Page about are you a specist (00:41:10) because you're you're in favor of AI (00:41:13) over over humans. (00:41:15) >> I mean, look, that's the going to be a (00:41:16) dividing line. There are some people and (00:41:18) like I'm not quite sure which side of (00:41:20) the debate Elon's on these days. Like (00:41:21) I've certainly heard him say some pretty (00:41:24) posthuman transhumanist things lately (00:41:26) >> and I think that we're going to have to (00:41:28) make some tough decisions in the next 5 (00:41:30) to 10 years. I mean the reason I dodged (00:41:32) the question on the timeline for super (00:41:34) intelligence is because you know I think (00:41:36) that it doesn't matter whether it's one (00:41:38) year or 10 or 20 years is super urgent (00:41:41) that right now we have to declare what (00:41:44) kind of super intelligence are we going (00:41:45) to build and are we actually going to (00:41:48) countenance creating some entity which (00:41:51) we provably can't align we provably (00:41:53) can't contain and which by design (00:41:56) exceeds human performance at all tasks (00:41:58) >> and human understanding (00:42:00) >> and understanding like how do you (00:42:01) control something that you don't (00:42:02) understand? Right? (00:42:04) >> I'd like to if I may pull on the (00:42:06) anthropomorphization thread a bit. If (00:42:09) you may remember Douglas Adams book, The (00:42:12) Restaurant at the End of the Universe, (00:42:13) there's a scene where there's a cow (00:42:16) that's been engineered to invite (00:42:18) restaurant patrons to eat it (00:42:20) >> because makes them feel more (00:42:22) comfortable. And the the cow doesn't (00:42:23) mind. The cow's been optimized to want (00:42:26) to be eaten by by the patrons. But many (00:42:28) readers horrified at that scene. (00:42:30) put that in in a box for a moment. (00:42:33) Microsoft has a history of (00:42:35) anthropomorphizing (00:42:36) AI assistance co-pilots going back (00:42:40) probably there's an example prior to (00:42:41) Microsoft Bob and the Rover dog and then (00:42:45) Clippet Clippy in Microsoft Office and (00:42:48) then more recently more sort of a a (00:42:51) amorphous cloudshaped avatars. How how (00:42:54) do you think about reconciling on the (00:42:57) one hand the desire not to overly (00:42:59) anthropomorphize agents on the other (00:43:02) hand with an institution that has (00:43:05) arguably been in the vanguard of (00:43:07) anthropomorphizing agents? I think the (00:43:10) entire field of design has always used (00:43:13) the human condition as its reference (00:43:15) point, right? I mean, skuorphic design (00:43:17) was the backbone of the guey, right? (00:43:20) From fileraxes to calendars and to (00:43:22) everything in between, right? Um, and we (00:43:24) still have the remnants of that in our, (00:43:26) you know, old school interfaces which we (00:43:28) feel that are modern and stuff like so (00:43:30) that's like an inevitable part of our (00:43:32) culture and we just grow out of them. we (00:43:34) we figure out like cleaner, better, more (00:43:36) effective user interfaces. I'm not (00:43:39) against anthropomorphism (00:43:41) by default. I mean, I I think we want (00:43:43) things to feel ergonomic, right? The (00:43:46) chair fits. The language model speaks my (00:43:49) tone, right? It has a fluency that makes (00:43:52) sense to me. It has a cultural awareness (00:43:54) that resonates with my history and my (00:43:57) nation and so on. And I think like that (00:44:00) is an inherent part of design today. As (00:44:02) as creators of things, we are now (00:44:06) engineering personalities and culture (00:44:09) and values, not just pixels and uh you (00:44:12) know software. So but but but obviously (00:44:15) you know there's a line right creating (00:44:18) something which is indistinguishable (00:44:20) from a human has a lot of other risks (00:44:24) and complications like that makes the (00:44:26) immersion into the the simulation even (00:44:29) more um you know kind of dangerous and (00:44:33) more likely right um and so I think I (00:44:36) don't have a problem with entities (00:44:38) avatars or voices or whatever that are (00:44:41) clearly distinct and separate and not (00:44:43) trying to imitate and always disclose (00:44:45) and have that that they are an AI (00:44:47) essentially and that there are (00:44:48) boundaries around them like that seems (00:44:50) like a natural and necessary part of (00:44:52) safety. So what I think I hear you (00:44:54) saying correct me if if I'm mistaken is (00:44:56) anthropomorphization is the new (00:44:58) skuomorphism on the one hand but on the (00:45:01) other hand maintaining clean maybe even (00:45:04) legal boundaries between human (00:45:06) intelligence and artificial (00:45:08) intelligence. Do do you think do you see (00:45:10) a future where AIs achieve some sort of (00:45:14) legal personhood or is that forboten? Is (00:45:16) that never going to happen? Do you see a (00:45:18) future where humans are allowed to merge (00:45:20) with the AIS Kurszswe style friend of (00:45:22) the pod or is that also not on the table (00:45:24) in your mind? (00:45:24) >> Yeah, I mean I I think AI legal (00:45:27) personhood is extremely not on the (00:45:30) table. I don't think our species (00:45:33) survives (00:45:34) if we have legal personhood and rights (00:45:39) alongside a species that costs a (00:45:43) fraction of us (00:45:44) >> that can be replicated and reproduced at (00:45:47) infinite scale relative to us that has (00:45:50) perfect memory that can just like (00:45:52) paralyze its own computation. I mean, (00:45:54) these are so antithetical to the (00:45:58) friction of being a biological species, (00:46:00) us humans, that there would just be an (00:46:03) inherent competition for resources. And (00:46:06) until it was provable, until it was (00:46:08) provable that those things would be (00:46:10) aligned to our values and to our ongoing (00:46:13) existence as a species and could be (00:46:15) contained mathematically provably, (00:46:18) >> um, which is a super high bar. I don't (00:46:21) see that we should be any considering (00:46:23) giving (00:46:24) >> bright line in the sand. (00:46:25) >> I really think it's a bright line. I (00:46:27) think it's I think it's very dangerous. (00:46:28) There's a separate question which has to (00:46:30) do with liability because they are going (00:46:33) to have increasing autonomy. Like to be (00:46:35) clear, I'm also an accelerationist. (00:46:37) >> I want to make these things. They're (00:46:39) going to (00:46:41) >> but but tension is rational. People (00:46:43) always say that tension is is rational. (00:46:44) If you don't see the tension, you're (00:46:46) definitely missing the most of the (00:46:48) debate. is obviously very complex. Like (00:46:51) the more we talk about the complexity (00:46:52) and hold it in tension, that's when you (00:46:54) start to see the wisdom. And there's no (00:46:56) way we can leave these things on the (00:46:58) table and say no, like we want to have (00:47:00) these things in clinic, in school, in (00:47:03) workplace, delivering value for for us (00:47:06) at a huge scale, but they have to be (00:47:07) boundaried and controlled. And that's (00:47:09) the that's the kind of that's the art (00:47:11) that we have to exercise. This episode (00:47:13) is brought to you by Blitzy, autonomous (00:47:15) software development with infinite code (00:47:18) context. Blitzy uses thousands of (00:47:21) specialized AI agents that think for (00:47:23) hours to understand enterprise scale (00:47:26) code bases with millions of lines of (00:47:28) code. Engineers start every development (00:47:31) sprint with the Blitzy platform, (00:47:33) bringing in their development (00:47:34) requirements. The Blitzy platform (00:47:37) provides a plan, then generates and (00:47:39) pre-ompiles code for each task. Blitzy (00:47:42) delivers 80% or more of the development (00:47:44) work autonomously while providing a (00:47:47) guide for the final 20% of human (00:47:49) development work required to complete (00:47:51) the sprint. Enterprises are achieving a (00:47:54) 5x engineering velocity increase when (00:47:57) incorporating Blitzy as their preIDE (00:47:59) development tool, pairing it with their (00:48:01) coding co-pilot of choice to bring an AI (00:48:04) native SDLC into their org. Ready to 5x (00:48:08) your engineering velocity? Visit (00:48:09) blitzy.com to schedule a demo and start (00:48:12) building with Blitzy today. (00:48:17) >> It it sounds though, if I may, the the (00:48:19) primary rationale that I'm hearing for (00:48:22) why not AI personhood has to do with the (00:48:25) inadequacies of the human form as (00:48:27) currently constructed. I heard you say, (00:48:29) well, they'll outra humans. They're so (00:48:31) much smarter. They're so much faster. (00:48:32) They're so much more clonable than human (00:48:34) intelligence is. If human intelligence (00:48:36) were uplifted, maybe with the benefit of (00:48:38) AI, if we had uploading type (00:48:41) technologies or BCIs that are advanced (00:48:43) that enable us to to lift up the average (00:48:46) human intelligence, in your mind then (00:48:49) does that open the door a bit to AI (00:48:51) personhood if humans can compete on a (00:48:52) level playing ground with AIS? (00:48:54) >> I don't want to make the competition for (00:48:58) the peace and prosperity of the 7 (00:49:00) billion people on the planet even more (00:49:02) chaotic. So if the path over the next (00:49:05) century, you know, can be proven to be (00:49:07) much safer and more peaceful and less (00:49:10) like, you know, disease and sickness and (00:49:13) there is room for this other species, (00:49:16) then I'm openminded to it, including (00:49:17) biological hybrids and so on, like it (00:49:20) I'm not like against that on principle. (00:49:22) I'm just a speciesist. (00:49:24) >> Aha. (00:49:25) >> I'm just a humanist. I start with we're (00:49:28) here and it's a moral imperative that we (00:49:31) protect the well-being of all the (00:49:33) existing conscious beings that I know do (00:49:35) exist and could suffer tremendously by (00:49:38) the introduction of this new thing. (00:49:40) Right (00:49:40) >> now of course the Neanderthalss uh may (00:49:43) have had that conversation or every (00:49:45) species that preceded us over the last (00:49:48) billion plus years. I mean, there are (00:49:50) many who argue we're simply an interim (00:49:53) uh transitory species in (00:49:55) >> bootloader for the super intelligence. (00:49:57) >> That classic phrase. Yes, I'm totally (00:50:00) aware of that. And I'm also someone who (00:50:02) thinks on cosmological time, too. So, (00:50:04) I'm not just naively saying, you know, (00:50:06) this century. I'm I'm definitely aware (00:50:08) that we're there's a huge transition (00:50:10) going on. And in fact, you can even see (00:50:12) it in recent memory. I mean, 250 years (00:50:14) ago, life expectancy was about 30 years (00:50:16) or whatever it was. Of course, in some (00:50:18) ways, we are a uh augmented hybrid (00:50:20) biological species, right? We take all (00:50:22) these drugs and I I you know, everyone's (00:50:24) peptides are amazing and it's super I'm (00:50:27) down for all of that. Let's go. (00:50:28) >> The genetic reprogramming is coming next (00:50:30) year. (00:50:30) >> Exactly. Let's go. I'm down. I'm down. (00:50:33) But (00:50:35) let's not shoot ourselves in the foot. (00:50:37) like I want to make sure that uh you (00:50:39) know most of our planet if not everybody (00:50:42) gets the benefit of the peace and (00:50:44) prosperity that comes from the (00:50:45) technology. (00:50:46) >> I mean there is there is some level of (00:50:48) sanity in that argument if you believe (00:50:51) that the AI will ultimately out compete (00:50:53) us and uh and put us into a box of (00:50:58) insignificance. I mean in the long in (00:50:59) the long run (00:51:00) >> I mean all intelligences we we we can (00:51:04) see this in nature. We're innately (00:51:07) hierarchical. So far, we have not seen (00:51:10) this supra collaborative species that (00:51:12) will take self-sacrifice in order to (00:51:14) preserve the other species. (00:51:16) >> So there's an inherent hierarchical (00:51:18) there's an inherent clash from coming (00:51:19) from, you know, the hierarchical (00:51:21) structure of intelligence, right? So, (00:51:23) and all I'm saying is not that we (00:51:25) shouldn't explore it, not that it (00:51:26) couldn't potentially happen, but the bar (00:51:28) has to first be do no, maybe do a (00:51:32) little, but do no harm to our species (00:51:34) first. Don't don't shoot ourselves in (00:51:37) the foot as you said, Dave. (00:51:38) >> Well, I'm 100% with you on this topic, (00:51:40) by the way. Could not be more aligned. (00:51:42) But Jeffrey Hinton is out there telling (00:51:44) the world it's going to run away and our (00:51:48) our safety valve is giving it a maternal (00:51:50) instinct and (00:51:51) >> which I found I found an interesting (00:51:53) point of view. (00:51:54) >> Well, I didn't I didn't check that (00:51:57) safety valve. (00:51:58) >> Well, he he believes it's uncontainable (00:52:01) and I I I'm with you. I think it's very (00:52:03) containable if you don't give it (00:52:04) emotional and intentional programming. (00:52:07) Uh but he thinks it's uncontainable. He (00:52:10) was very pessimistic when he got his (00:52:11) Nobel Prize. Now he's more optimistic (00:52:13) because he sees a path to pro (00:52:15) programming in maternal instinct which (00:52:18) implies that it's like it's dominant to (00:52:20) us but it cares. His his thesis his (00:52:22) thesis was I've seen a situation where a (00:52:25) vastly more intelligent entity (00:52:29) >> takes care of a younger inept entity in (00:52:33) a mother with their screaming child. (00:52:35) >> Yeah. Exactly. (00:52:36) >> So if there's a maternal instinct that (00:52:38) we can program into AI even though we're (00:52:41) far less capable it will take care take (00:52:44) care of (00:52:44) >> it's been compared to the call it the (00:52:46) digital oxytocin plan for AI alignment. (00:52:49) >> I like that. (00:52:50) >> That's a good one. Yeah. (00:52:51) >> Yeah. I mean, cool. (00:52:53) >> I mean, it's about as poetic as it gets. (00:52:56) I think I'm going to need something (00:52:57) that's got a little bit more like for (00:52:59) formula to it. A bit more reassuring. (00:53:02) But look, there's 101 different possible (00:53:04) strategies for safety. We should explore (00:53:06) all of them. Take them all seriously. I (00:53:08) mean, Jeff is a legend and of the field. (00:53:10) No question. But like I just think (00:53:12) approach with caution. (00:53:14) >> Are you spending a lot of your energy (00:53:16) compute uh human power on safety? Yeah, (00:53:20) I would say not as much as we should, (00:53:23) you know. I I'm I'm wrapping my head (00:53:26) around it. Um, is anybody out there I I (00:53:29) am I am curious out of all the (00:53:31) hyperscalers out there. Is there any (00:53:34) entity that's spending enough in your (00:53:39) mind? Because everybody's in such a (00:53:40) race. It's like more GPUs, more data, (00:53:43) more energy. It's just like everybody's (00:53:46) optimizing for the next benchmark. I (00:53:49) don't see uh any safety benchmarks. Are (00:53:51) there any safety benchmarks out there? (00:53:53) >> Oh, there are tons of safety benchmarks. (00:53:55) And and there's at least in my mind an (00:53:57) argument for defensive co-scaling. I'd (00:53:59) be curious to hear your ideas on that. (00:54:00) Do do you think in the same way that as (00:54:03) a a city gets larger, the police force (00:54:05) gets larger. Maybe it's not in direct (00:54:07) proportion. Maybe there's some scaling (00:54:09) exponent, but do you think defensive (00:54:11) co-scaling of alignment forces or safety (00:54:14) forces, whatever that ends up meaning, (00:54:16) do you think that's part of the strategy (00:54:18) for for AI alignment? (00:54:19) >> I think that would be a good way. I (00:54:21) mean, we've proposed this several times (00:54:23) over the years. I mean, the the White (00:54:24) House voluntary commitments under Biden (00:54:26) that me and in fact everyone, I mean, (00:54:28) Demis and Dario and Sam and all of us (00:54:30) through co were pushing this pretty (00:54:32) hard. And look, I mean, it got chucked (00:54:33) out, but I think it's a very sensible (00:54:35) set of principles. is like auditing for (00:54:37) scale of flops, you know, having some (00:54:39) percentage that we all share of safety (00:54:41) investment flops and headcount. You (00:54:44) know, this is the time and I think on (00:54:46) the face of it, everyone is open and (00:54:49) willing to sharing best practices and (00:54:51) disclosing to one another and (00:54:53) coordinating when the time comes. I (00:54:55) think we're we're still pre that level. (00:54:57) So, we're in like hyper competitive mode (00:54:59) at the moment. Um, but yeah, I I think (00:55:03) now is really the time to be making (00:55:05) those investments. (00:55:06) >> Yeah. Well, is there something that's (00:55:07) going to scare the out of us that (00:55:09) stops everybody? You know, is there a (00:55:11) three, you know, I was talking to Eric (00:55:12) Schmidt about this. Is there a three (00:55:14) mile island like event? (00:55:16) >> Scares everybody but doesn't kill (00:55:17) anybody. (00:55:18) >> Well, Eric Schmidt was said specifically (00:55:20) he's hoping for a 100 deaths (00:55:23) >> because that's in his mind the least (00:55:25) that would get the attention of the (00:55:27) government and would cause some kind of (00:55:28) a solution. (00:55:30) Dave, continue, please. (00:55:31) >> Well, so it's interesting that you you (00:55:33) say Daario and Sam and Ilia, like you (00:55:36) guys obviously must interact quite a (00:55:38) bit. Is Meera part of that gang? Is (00:55:42) Andre part of that gang? Are you like (00:55:45) because this is it's it's interesting to (00:55:47) think about the competition heating up (00:55:48) like we were just talking about. And you (00:55:50) know, Daario started from this position (00:55:52) of pure safety and I think Ilia did too. (00:55:55) But now we're right on the cusp of (00:55:57) self-improvement and it's really really (00:56:00) clear that there are serious I wouldn't (00:56:04) say fissurers but but the the companies (00:56:06) are now really racing. I mean really (00:56:09) racing and and I know Microsoft you know (00:56:11) when I wrote my my second business plan (00:56:13) first company I sold next business plan (00:56:15) I was writing the first sentence was (00:56:17) stay out of Microsoft's way because (00:56:20) because at the time you know Microsoft (00:56:21) had h half the market cap of tech was (00:56:23) Microsoft and Microsoft's plan was to (00:56:26) double in size we have a much more (00:56:28) balanced world now with Microsoft and (00:56:29) Google and Meta but at the time (00:56:31) Microsoft was just unstoppable and (00:56:34) dominant and so just stay out of the way (00:56:36) but Microsoft seems to always win Right. (00:56:38) There's and and we are right on the edge (00:56:41) of self-improvement at least as far as I (00:56:44) can tell. So, is it still, you know, (00:56:47) let's all get together and have dinner (00:56:48) and talk about safety or is everybody (00:56:50) now in full board? (00:56:51) >> No, definitely. I think that's that's (00:56:53) definitely there. I think the recursive (00:56:55) self-improvement piece is probably the (00:56:59) threshold moment if it works. And if you (00:57:02) think about it at the moment, there are (00:57:04) software engineers who are in the loop (00:57:06) who are generating post- training data, (00:57:09) running ablations on the quality of the (00:57:10) data, (00:57:11) >> running them against benchmarks, (00:57:13) generating new data and that's sort of (00:57:15) broadly the loop. Um and that's kind of (00:57:19) expensive and slow and it takes time and (00:57:21) it's not completely closed and I think a (00:57:24) lot of the labs are racing to sort of (00:57:25) close that loop so that various models (00:57:28) will act as judges evaluating quality (00:57:31) you know generators producing new (00:57:32) training data uh adversarial models that (00:57:36) are like reasoning over which data to (00:57:38) include and what's higher quality. Um (00:57:41) and then obviously that's then being fed (00:57:43) back into the post- training process. (00:57:45) Um, so like closing that loop is going (00:57:49) to speed up AI development for sure. (00:57:52) Some people speculate that that adds I (00:57:54) mean okay I think it probably does add (00:57:56) more risk but some people speculate that (00:57:58) it's a potential path to a fume you know (00:58:00) an intelligence explosion. (00:58:02) >> Yeah. (00:58:02) >> Um and I definitely think with unbounded (00:58:07) compute and without human in the loop or (00:58:10) without control that does potentially (00:58:12) create a lot more risk. But unbounded (00:58:14) compute is a big claim. I mean that that (00:58:15) would mean need a lot of compute. Um so (00:58:19) yeah, we we're definitely taking steps (00:58:21) towards like more and more uh you know (00:58:23) more and more risky stuff. Can I ask you (00:58:25) a really specific question about that (00:58:26) because you know the year and a half now (00:58:28) at Microsoft um before true recursive (00:58:32) self-improvement which is imminent (00:58:33) there's AI assisted chip design and and (00:58:36) this you know the the layers in the the (00:58:38) pietorch stack um are very clunky but (00:58:42) now it's really easy to use the AI to (00:58:45) punch through the stack and optimize you (00:58:48) know build your own kernels get 2 3 4x (00:58:51) performance improvement but clearly open (00:58:53) AAI is now working to build custom chips (00:58:56) and the TPU7s just came out. When you (00:58:59) arrived at Microsoft, first of all, was (00:59:00) I I know there's a lot of quantum chip (00:59:02) work going on, but was there any work (00:59:03) going on similar to the TPU work? (00:59:06) >> Yep. There's there's also a chip effort. (00:59:08) Um, and you know, I think progress has (00:59:10) been pretty good. I mean, I I think that (00:59:13) um you know, we've got a few different (00:59:15) irions in the fire that we haven't sort (00:59:17) of talked about publicly yet, but I (00:59:18) think um you know, the chips are going (00:59:21) to be important part of it for sure. (00:59:22) >> Yeah. that those are internal efforts. (00:59:24) Are those teams under you? That's that's (00:59:26) part of your (00:59:27) >> No, I mean they're they're in the (00:59:28) broader company. Yeah. (00:59:29) >> Okay. Interesting. (00:59:31) >> I I want to switch subject a little bit (00:59:33) and go come to your book um The Coming (00:59:35) Wave. I enjoyed it greatly. I listened (00:59:38) to it. I love the fact that you read it. (00:59:40) >> Thank you. (00:59:40) >> Yeah. I tell my kids I read books. Go. (00:59:42) No, Dad. You listen to books. You don't (00:59:43) read books anymore. Uh I want to I want (00:59:46) to read what I wrote here because it's (00:59:47) important. So you identified the (00:59:50) containment problem as the defining (00:59:52) challenge of our era. Uh warning that as (00:59:56) these technologies become cheaper and (00:59:57) more accessible, they will inevitably (00:59:59) proliferate, making them nearly (01:00:02) impossible to control. (01:00:05) This creates a terrifying dilemma. uh (01:00:08) failing to contain them uh forces risk (01:00:11) for catastrophe like you know engineered (01:00:15) pandemics and a lot of the your concerns (01:00:17) were in the biological world and I agree (01:00:19) being a biologist and a physician or (01:00:22) potentially democratic collapse with (01:00:24) deep fakes and all of that but the (01:00:27) extreme surveillance required to enforce (01:00:29) containment could lead to a a (01:00:31) totalitarian uh dystopia. So you say we (01:00:35) need to navigate this narrow path (01:00:37) between chaos and tyranny (01:00:40) and that is a very fine line to (01:00:43) navigate. So you propose a strategy of (01:00:46) containment. This includes technical (01:00:48) safety measures, strict global (01:00:50) regulations, choke points on hardware (01:00:52) supply, international treaties. (01:00:57) How are we doing on that? Yeah, I mean (01:01:00) it's kind of important to just take a (01:01:02) step back and distinguish between (01:01:04) alignment and containment. (01:01:06) >> Um, the project of safety requires that (01:01:09) we get both right. And I actually think (01:01:11) we have to get containment right before (01:01:13) we get alignment right. Alignment is the (01:01:15) kind of like maternal instinct thing. (01:01:18) Does it share our values? Is it going to (01:01:19) care about us? Is it going to be nice to (01:01:21) us? Containment is can we formally limit (01:01:25) and put boundaries around its agency and (01:01:28) are we (01:01:29) >> for everybody? (01:01:30) >> Not just for ourselves, for everybody. (01:01:32) Yeah. I mean, I think that is part of (01:01:33) the challenge is that like (01:01:35) >> um one bad actor with something that is (01:01:38) really this powerful in a decade or two (01:01:40) decades or something, you know, really (01:01:42) could destabilize the rest of the (01:01:43) system. And so, you know, just (01:01:45) >> the system being humanity (01:01:46) >> global humanity system. Yeah. Just as (01:01:48) you said, like as everything becomes (01:01:50) hyperdigitized, (01:01:52) the the verse does become the metaverse. (01:01:54) Even though that kind of like went in (01:01:56) and out of fashion very quickly, it's (01:01:58) still, I think, the right frame in a way (01:02:01) because everything is going to become (01:02:02) primarily digitized and hyperconnected (01:02:04) and instant and real time. And so the (01:02:08) one to many effect is suddenly massively (01:02:10) amplified. I mean obviously we see it on (01:02:12) social media but now imagine that it's (01:02:14) not just words that are being broadcast. (01:02:17) It's actually actions. It's agents are (01:02:20) capable of you know um you know breaking (01:02:23) into systems or you know sort of (01:02:25) >> and they're resident in humanoid robots (01:02:27) at a billion on the planet (01:02:29) >> and that too. Yeah. Is both atoms and (01:02:31) and and bits. So um equilibrium requires (01:02:37) that there is a type of surveillance (01:02:39) that we don't really have in the world (01:02:41) today. I mean we certainly don't have it (01:02:43) physically. (01:02:44) >> The web is actually remarkably (01:02:46) surveiled. I think surprisingly you know (01:02:49) more than I think people would expect. (01:02:51) Um and some form of that is necessary to (01:02:55) create peace. Just as we centralized (01:02:58) power and taxation or or sort of (01:03:01) military force and taxation around (01:03:03) governments, you know, 3 or 4 500 years (01:03:06) ago and that's been the driving force of (01:03:08) progress. Actually, that order unleashed (01:03:11) science and technology and stability (01:03:14) stability. Yeah. (01:03:15) >> So the question is like how do what is (01:03:17) the modern form of imposition of (01:03:20) stability (01:03:21) >> in a way that isn't totalitarian but (01:03:23) also doesn't relinquish it to a (01:03:25) libertarian catastrophe. Um I think it's (01:03:27) naive to think that somehow (01:03:30) >> um you know the best defense against a (01:03:32) gun is a gun and sort of the the idea (01:03:35) that somehow we're all going to have our (01:03:36) own AIS and that's going to create this (01:03:37) sort of steady equilibrium that all the (01:03:39) AIS are just going to ne neutralize each (01:03:42) other like that ain't going to happen. I (01:03:44) mean, part of me hopes for a uh a super (01:03:49) intelligence that uh is the ring to rule (01:03:54) them all and provides, you know, I'm not (01:03:57) worried about, how do I put it? I'm (01:03:59) worried about Peter, you're hoping for a (01:04:00) singleton. (01:04:01) >> Yeah, that sounds like what's going on. (01:04:03) >> Well, you know, part of me is like, (01:04:05) >> color me shocked. (01:04:06) >> Really? (01:04:07) >> Yeah. I mean uh I imagine that the level (01:04:12) of complexity we we're we're mounting (01:04:15) towards uh that balancing act is (01:04:18) extraordinarily difficult and you know (01:04:20) you can't push a string but is there (01:04:22) some mechanism to uh to pull it forward. (01:04:27) uh we should have this debate sometime. (01:04:30) >> Some would call government uh ge at (01:04:32) least historically a geographic monopoly (01:04:34) on violence. And what I think I'm (01:04:36) hearing is some sort of monopoly on (01:04:38) intelligence or at least capabilities (01:04:40) exposed to intelligence in order to ring (01:04:43) fence to to contain AI. But that's the (01:04:45) exact opposite as far as I can tell of (01:04:47) what we've seen over the past few years. (01:04:48) People used to armchair AI alignment (01:04:51) researchers 101 15 years ago would say (01:04:54) humanity wouldn't be so stupid the (01:04:55) moment we have something resembling (01:04:57) general intelligence as to give it (01:04:59) terminal access or to give it access to (01:05:01) the economy and that's exactly what we (01:05:03) did there was the the open AI Google um (01:05:07) moment (01:05:08) >> and yet and yet but that's concerning (01:05:10) right so I mean Google develops all this (01:05:13) technology is holding internally until (01:05:16) some actor happens to have initials Open (01:05:18) AAI releases it and then there's no (01:05:21) other option but to follow suit. (01:05:24) >> I'm less concerned by it. I if you look (01:05:26) at Anthropic for example which prides (01:05:28) itself on being a very alignment forward (01:05:30) organization. Alignment Anthropic (01:05:33) released the model control protocol (01:05:35) which is now the standard way at least (01:05:37) for the moment for for models to (01:05:39) interact with the environment. What many (01:05:41) AI researchers said exactly we did not (01:05:43) want to do prior to general (01:05:45) intelligence. So I'm I'm I'm curious. I (01:05:47) mean in in your mind h how given that (01:05:50) the economy there's every economic (01:05:52) pressure in including modern touring (01:05:54) test to empower agents to interact with (01:05:57) the entire world and to do the exact (01:05:58) opposite of containment. Why would we (01:06:00) start containing? (01:06:01) >> Containment it's not that binary right I (01:06:04) mean you we contain things all the time. (01:06:06) We have powerful forces in the engine in (01:06:10) your car that is contained and broadly (01:06:12) aligned right and there is an entire (01:06:14) regulatory apparatus around that from (01:06:16) seat belts to vehicle admissions to (01:06:18) lighting to drive you know street (01:06:19) lighting to driver ed you know to to to (01:06:21) to freeway speeds I mean that's healthy (01:06:25) functional regulation enabling us to (01:06:28) collectively interact with each other (01:06:30) now obviously it's multiple orders of (01:06:32) magnitude more complex because these (01:06:34) things are not cars they're, you know, (01:06:36) sort of digital people, but that doesn't (01:06:38) mean to say that we shouldn't be (01:06:40) striving to limit their boundaries. And (01:06:42) nor does it mean that we have to (01:06:44) centralize. By the way, the answer isn't (01:06:45) that we have a totalitarian state of (01:06:47) intelligence overseas. (01:06:49) >> No, I think it's just instinctively it (01:06:52) can be easy to go there when you know (01:06:54) when you kind of start to think it (01:06:56) through. It's like obviously we do have (01:06:58) centralized forces but even in the US we (01:07:01) have you know military we have um (01:07:04) divisions of the army we have divisions (01:07:06) of the police force they're nested up in (01:07:08) different layers there's checks and (01:07:09) balances on the system and that's kind (01:07:11) of what we got to start thinking about (01:07:12) designing (01:07:13) >> that analogy to driving is a great one (01:07:15) and just to follow through on it the (01:07:18) complexity difference very high right (01:07:20) for AI but the timeline also (01:07:23) >> I mean driving evolved from what 1910 (01:07:28) to today (01:07:29) >> late 1800s. So the laws related, you (01:07:31) know, seat belts came out 80% of the way (01:07:33) through that timeline. Yeah. So lots and (01:07:35) lots of time to iterate (01:07:37) >> here, very little time and immensely (01:07:40) more complex. So do you have a vision? (01:07:42) But but I completely agree. We need a (01:07:44) framework for containment (01:07:46) >> fast and do you have a a thought on how (01:07:49) we're going to (01:07:49) >> I think that there's also a good (01:07:51) commercial incentive to do this, right? (01:07:53) I think that a like the many of the (01:07:55) companies know that they that our social (01:07:58) license to operate requires us to take (01:08:01) more accountability for externalities (01:08:03) than ever before. We're not in the (01:08:05) Robert Baron era. We're not in the oil (01:08:08) era. We're not in the smoking era, (01:08:10) right? We've learned a lot. Not (01:08:13) everything. There's still a lot of (01:08:14) conflicts, (01:08:15) >> but it really is a little bit different (01:08:18) to last time around. And I think that's (01:08:20) one reason to be a bit more optimistic. (01:08:22) Plus there's the commercial incentive, (01:08:23) the commercial incentive and the kind of (01:08:25) externalities shift. (01:08:27) >> So, so if you know if Eric Schmidt is (01:08:29) right and uh something either (01:08:32) radiological or biological happens and (01:08:34) there's 100 deaths and then the phone (01:08:37) starts ringing, everyone come to the (01:08:38) White House right now. Well, first of (01:08:40) all, do you want that call? Is that is (01:08:41) that part of your your life plan to take (01:08:43) that call and and react to it? And then (01:08:45) who else do you trust in the community (01:08:47) to be part of that reaction? Look, I (01:08:50) think that there is going to be a time (01:08:52) in the next 20 years where it will make (01:08:57) complete sense to everybody on the (01:08:59) planet, (01:09:00) >> the Chinese included, and every other (01:09:02) significant power (01:09:04) >> to cooperate (01:09:05) >> on safety (01:09:07) >> on safety and containment and alignment. (01:09:10) It is completely rational for (01:09:12) self-preservation. (01:09:14) You know, these are very powerful (01:09:15) systems that present as much of a threat (01:09:18) to the person, the bad actor that is (01:09:20) using the model as it does to the, you (01:09:23) know, the the the the victim. (01:09:25) >> And I think that, you know, that will (01:09:28) that will create, you know, a an an (01:09:31) interest in in cooperation, which, you (01:09:34) know, it's kind of hard to empathize (01:09:37) with at this stage given how polarized (01:09:39) the world is, but I do think it's (01:09:41) coming. I mean the the the number one (01:09:42) thing to unify all of humanity is a you (01:09:46) know an alien invasion (01:09:48) uh and that alien invasion could be a (01:09:51) you know potential for a rogue super (01:09:52) intelligence. (01:09:53) >> Yeah. Okay. What about the first part of (01:09:55) my question? Is that part of your (01:09:57) calling in life? I mean there's only a (01:09:58) handful like I think a lot of people (01:10:00) that I meet around MIT or elsewhere are (01:10:04) they they have this vision that somebody (01:10:06) has it figured out somewhere. You know (01:10:07) someone someone in government somewhere (01:10:09) must be thinking about this. But you've (01:10:11) been there, right? There's no there's no (01:10:14) one there. (01:10:14) >> We're the adults in the room. Is that (01:10:15) what you're saying? (01:10:16) >> Yeah, definitely. There's nowhere to go (01:10:18) from this room. (01:10:19) >> Dave is asking for the smoke filled back (01:10:21) room where the the leads of all the (01:10:22) Frontier Labs are secretly swapping (01:10:24) safety tips. (01:10:25) >> Yeah, something like that. Yeah, (01:10:27) >> I I think that in practice intelligence (01:10:30) exists outside of the smoky room. I (01:10:33) think that that the the notion that like (01:10:35) decisions get made in the boardroom or (01:10:37) in the white house situation room or (01:10:40) like actually int I mean you know you (01:10:42) mentioned poly markets and stuff like in (01:10:44) intelligence (01:10:46) coaleses in these big balls of iterative (01:10:49) interaction (01:10:51) >> um and that's that's what's propelling (01:10:53) the world forward and so this is where (01:10:56) the conversation's happening like your (01:10:57) audience you know all the other (01:10:59) podcasters everyone online we're (01:11:01) collectively trying trying to move that (01:11:02) knowledge base forward. (01:11:03) >> In November, you announced the launch of (01:11:05) uh humanist super intelligence um and uh (01:11:10) focused on three applications in (01:11:13) particular uh medicine and uh companions (01:11:16) and clean energy. Uh I'd love to double (01:11:19) click in that a little bit, but I was (01:11:21) curious that you didn't include (01:11:23) education in that space. (01:11:25) Um, and I, you know, we have an audience (01:11:28) of entrepreneurs and AI builders, and I (01:11:32) think education, as much as healthc care (01:11:34) is up for grabs right now, education is (01:11:37) too. (01:11:38) >> Totally agree. (01:11:39) >> Uh, and I don't think our high schools (01:11:41) are preparing anybody for the world (01:11:44) that's coming. There's still (01:11:46) retrospectively 50 years in looking in (01:11:48) the rearview mirror. Um, do you think (01:11:50) Microsoft will play in reinventing (01:11:53) education? You know, I I think it's (01:11:55) already happening across the whole (01:11:57) industry. I mean, it's never been easier (01:11:59) to get access to an expert teacher in (01:12:01) your pocket that has essentially a PhD (01:12:04) and that can adapt the curriculum to (01:12:06) your (01:12:07) >> bespoke learning style. The bit that it (01:12:09) can't do at the moment is to evolve or (01:12:12) sort of like curate an extended program (01:12:15) of learning over many many sessions, but (01:12:17) we're like just around the corner from (01:12:19) that. I mean, we released a feature just (01:12:21) a few months ago ago called quizzes. And (01:12:24) so on any topic, not just a traditional (01:12:26) school education. It can set you up with (01:12:28) a a mini curriculum, a quiz, and it's (01:12:31) interactive and it's visual and you can (01:12:33) sort of track your learning over time. (01:12:35) And like I'm very optimistic about that, (01:12:37) too. It's a huge unlock. (01:12:38) >> One of the debates we have right now in (01:12:40) in the podcast on a pretty regular basis (01:12:42) is (01:12:44) do you go to college? (01:12:45) >> Yeah. (01:12:46) >> Do you go to grad school? I mean, this (01:12:47) is the most exciting time to build ever. (01:12:51) I don't know if you want to follow on (01:12:53) that, Dave. (01:12:53) >> Well, God, I do this constantly. It's (01:12:56) really tricky for me on campus because I (01:12:57) teach, you know, at MIT and Stanford at (01:12:59) Harvard and uh this window of (01:13:02) opportunity is so short and so acute and (01:13:04) it's really really clear how you succeed (01:13:06) right now in AI post AGI. (01:13:09) I mean, who could predict like nobody (01:13:10) knows? But right here, right now, you (01:13:12) see these these startup valuations like (01:13:14) we were last night. I won't mention it, (01:13:16) but but billions. (01:13:18) >> I mean, just Yeah. an opening valuation (01:13:20) of of $4 billion. (01:13:21) >> Billionillion dollar. Yeah. (01:13:22) >> By collecting just the right group of (01:13:24) people in the room. It's (01:13:25) >> Yep. Yep. I wanted to ask about that (01:13:27) actually because your your timing on (01:13:28) inflection was early like, you know, in (01:13:30) hindsight earlier, but now you've got (01:13:32) the new wave with Mera Marotti and Helia (01:13:34) and and a couple of others, Liquid AI, (01:13:37) uh, that all have multi-billion dollar (01:13:38) valuations. (01:13:39) >> Yeah. Thought we set some standards on (01:13:40) valuations pre-revenue with a 20 person (01:13:42) team, but we're just a minnow then a (01:13:46) whole two and a half years ago. (01:13:47) >> Is that all it was? Oh my god. (01:13:49) >> Three years, I think. Yeah. Jeez. (01:13:50) >> You think as the the cost of (01:13:52) intelligence becomes too cheap to meter (01:13:54) that the the value ascribed at least in (01:13:56) terms of market cap to human capital is (01:13:58) sort of inversely asmtoic going to (01:14:01) infinity. (01:14:02) >> Weirdly it is because of the pressure on (01:14:04) timing, right? And there's there's (01:14:06) actually still a pretty concentrated (01:14:07) pool of people that can do this stuff. (01:14:10) Uh, and there's like an overupp of (01:14:13) capital that's desperate to get a piece (01:14:15) of it. It might not be the smartest (01:14:16) capital the world's ever seen, but like (01:14:18) it's very eager. And so that's (01:14:23) I have to ask you because it's burning a (01:14:24) hole in my pocket, but you know, Alex's (01:14:27) uh freshman roommate at MIT was Natt (01:14:28) Freriedman (01:14:30) >> and pre actually prefr (01:14:32) roommate. And so Nat goes off and you (01:14:34) know he ends up at co-founder of safe (01:14:37) super intelligence (01:14:39) and I haven't asked him I don't know if (01:14:41) you've asked him yet but he leaves to (01:14:44) become the guy at Meta and I've got to (01:14:47) believe a huge part of that attraction (01:14:49) is the compute. (01:14:50) >> Yeah. (01:14:51) >> And and so here you are very similar (01:14:54) situation right? You've got your startup (01:14:55) you've got a billion or whatever billion (01:14:56) and a half that you've raised. Yeah. (01:14:58) >> You can build it. You can get your (01:14:59) 20,000 Nvidia. Well, wait a minute. (01:15:02) Here's Microsoft. (01:15:04) >> 300 billion of cash flow and a huge (01:15:07) amount of compute. Was that a big part (01:15:09) of the (01:15:09) >> Yeah. I mean, not to mention the the the (01:15:12) prices that we're paying for individual (01:15:14) uh you know, researchers or members of (01:15:16) technical staff and like I mean, just (01:15:17) the also just the the the scale of (01:15:20) investment that's required not just in (01:15:21) two years but over 10 years. I I think (01:15:24) it it's clearly there's a structural (01:15:26) advantage by being inside the big (01:15:28) company and I think it's going to take (01:15:31) >> you know hundreds of billions of dollars (01:15:33) to keep up at the frontier over the next (01:15:34) 5 to 10 years. (01:15:36) >> So uh finishing that thought then you (01:15:38) the the companies that are raising money (01:15:41) at a 20 or 50 billion dollar valuation (01:15:44) right now and no chance. (01:15:48) >> Okay, (01:15:50) I'll take that silence. But like I like (01:15:52) I think it depends. I mean there's (01:15:55) obviously a near-term if suddenly we do (01:15:57) have an intelligence explosion then lots (01:16:00) of people can get there simultaneously (01:16:01) but then also at the same time you have (01:16:03) to build a product with those things (01:16:05) which you have to distribution like all (01:16:06) the traditional mechanisms still apply. (01:16:08) Are you going to be able to convert that (01:16:09) quickly enough? I mean you know (01:16:11) everything goes really kind of weird if (01:16:14) that happens in the next 5 years. It (01:16:16) just is unrecognizable. There's so many (01:16:18) emergent factors to play into one (01:16:21) another. It's hard to it's hard to say (01:16:24) and I think that's partly the ambiguity (01:16:26) is what's driving the frothiness of the (01:16:28) valuations because I think there's (01:16:29) people going well I don't know I don't (01:16:31) do I want to be so what do you call it (01:16:32) reed Reed Reed calls it schmuck (01:16:34) insurance. (01:16:35) >> Yeah. Yeah. We had we had Reed on the (01:16:37) pod here a couple months ago. He's (01:16:39) brilliant. (01:16:40) >> Um I So to that graduating high school (01:16:44) student um what do you study these days? (01:16:49) I mean there's no question that you (01:16:51) still have to study both disciplines (01:16:53) like philosophy and computer science is (01:16:55) is going to for a long time remain I (01:16:58) think the two foundations (01:17:01) um should you go to college absolutely (01:17:05) like you know human education the (01:17:08) sociality that comes from that the (01:17:10) benefit of the institution having 3 (01:17:13) years to basically think and explore (01:17:15) >> you know in and out of your curriculum (01:17:17) this is a huge privilege like people (01:17:19) should not be throwing that away. That (01:17:21) is golden. (01:17:22) >> Uh so I always encourage people to do (01:17:24) that. (01:17:24) >> Obviously I did also drop out but I mean (01:17:27) I still think (01:17:28) >> it was it was a cool thing to do. (01:17:30) >> Yeah. It was just it felt right at the (01:17:32) time. Um (01:17:34) but the other thing is um (01:17:37) >> go into public service. (01:17:39) >> Yeah. I respect that part of what you (01:17:41) did in that sequence in your life um (01:17:45) which gave you this very much humanist (01:17:47) point of view. Yeah. And and it was (01:17:49) really hard and very different and it (01:17:51) didn't it wasn't instinctively right but (01:17:53) I learned a lot and it was a very (01:17:55) influential and important part of my (01:17:57) experience even though it was very (01:17:58) short. It was like a couple years (01:18:00) basically. Um, and I think if you look (01:18:02) at the actors in our ecosystem today, (01:18:06) corporations, the academics, the sort of (01:18:09) news organizations, (01:18:11) now the podcast world, it's really our (01:18:14) governments that are probably (01:18:15) institutionally the weakest and our (01:18:18) democratic process, but actually our (01:18:19) civil service. And that's because (01:18:21) there's been five decades of battering (01:18:25) of the status and reputation and respect (01:18:28) that goes into um you know being part of (01:18:31) the public service like post Reagan and (01:18:33) that and I think that's actually a (01:18:35) travesty because we actually need that (01:18:37) sentiment and that spirit and those (01:18:39) capabilities more than ever. I I I think (01:18:42) maybe uh what I just heard you say, (01:18:43) correct me if I'm wrong again, is we (01:18:45) need more intelligence in in the public (01:18:48) sector, in public service. What about AI (01:18:51) in government? (01:18:52) >> Do you think the government needs (01:18:54) >> and and what about agentic AI in the (01:18:56) government in particular? for sure with (01:18:58) all the same caveats that apply but I (01:18:59) mean you know I mean you know rate of (01:19:01) adoption for what it's worth of co-pilot (01:19:03) inside of government issued really high (01:19:04) is a brilliant job of synthesizing (01:19:06) documents and transcribing meetings and (01:19:09) summarizing notes and facilitating the (01:19:12) discussion and chipping in with actions (01:19:13) at the right time and it's clearly going (01:19:15) to save a lot of uh you know time and (01:19:18) and and improve decision-m (01:19:19) >> so so then maybe to tie a nice bow on (01:19:21) the discussion isn't that arguably a (01:19:23) form of AI containing AI if AI's (01:19:26) infusing the government and AI is (01:19:28) infusing the economy and the government (01:19:30) is regulating the economy. Isn't this (01:19:32) just defensive co-scaling with AI (01:19:34) regulating itself? (01:19:35) >> Yeah. I mean like everyone is going to (01:19:36) use AI all at the same time to pursue (01:19:39) but the same but the agendas that we all (01:19:41) have are going to remain the same. I (01:19:43) mean that people who want to start (01:19:44) companies, people who want to write (01:19:46) academic papers, people who want to (01:19:48) start, you know, cultural groups and (01:19:50) entertainment things, everyone is just (01:19:52) going to be empowered like in some way. (01:19:54) Their their capability is going to be (01:19:56) amplified by having these tools. (01:19:58) Obviously, the government included. (01:20:00) >> Nice. Mustafa, thank you so much for (01:20:03) taking the time on a Friday night. Uh (01:20:05) grateful to have this conversation with (01:20:07) you. Uh Dave, Alex, appreciate it. want (01:20:11) to final question from you Dave. (01:20:13) >> Final question if I have one that I (01:20:15) have. All right. I prediction (01:20:18) uh quantum computing right now has (01:20:21) nothing to do with going what's going on (01:20:23) in LLMAI. It's all Matt moles on Nvidia (01:20:26) chips and soon to be TPUs and other (01:20:29) custom chips. Best guess six, seven (01:20:32) years from now, uh the AI is very good (01:20:36) at writing code and compiling and can (01:20:38) figure out quantum operations. Are (01:20:41) quantum chips relevant or they on the (01:20:44) sideline still or is everything ported (01:20:46) over to quantum and Microsoft can take (01:20:48) advantage of its lead? (01:20:49) >> Yeah, I mean I I I think it's going to (01:20:51) be a big part of the mix. I think it's (01:20:53) sort of an under relative to the amount (01:20:55) of time we spend talking about AI is (01:20:58) kind of an undercknowledged part of the (01:21:00) wave actually a little bit like (01:21:01) synthetic biology. I think that (01:21:03) especially in the in the sort of you (01:21:05) know general conversation uh I think (01:21:07) people aren't grasping those two uh (01:21:10) waves which are going to be just as as (01:21:12) as as impactful and and crash at the (01:21:14) same time that AI is coming into focus. (01:21:18) >> All right, you heard it here. (01:21:19) >> This is a closing question to appeal (01:21:21) maybe to your more accelerationist side. (01:21:24) What can the audience do to accelerate (01:21:27) AI for science, AI for engineering? What (01:21:30) are what do you view as the the limiting (01:21:32) factors? If I I often talk on the (01:21:34) podcast about this notion of an (01:21:36) innermost loop, the idea that in (01:21:37) computer science, if you want to (01:21:39) optimize a program, you tend to to find (01:21:41) loops within loops, and you want to (01:21:43) optimize the innermost loop in order to (01:21:45) optimize the the overall program. What (01:21:46) do you see as the innermost loop, the (01:21:49) limiting factor, if you will, that the (01:21:51) audience listening, if they're suitably (01:21:53) empowered, can help optimize to speedrun (01:21:58) maybe a Star Trek future over the next (01:22:00) 10 years or a Star Trek economy? What do (01:22:01) we do? (01:22:02) >> Yeah, I mean, I think I think it's (01:22:04) pretty clear that most of these models (01:22:06) are going to speed up the time to (01:22:08) generate hypothesis. The slow part is (01:22:10) going to be validating hypothesis in the (01:22:12) real world. And so um the the all we can (01:22:16) do at this point is just ingest more and (01:22:18) more information into our own brains and (01:22:21) then co-use that with a single model (01:22:25) that progresses with you because it's (01:22:27) becoming like a second brain. Like for (01:22:30) example, Copilot is actually really good (01:22:32) at personalization now. Like most of its (01:22:34) answers and so the more you use it, the (01:22:36) more those answers pick up on themes (01:22:39) that you're interested in. And it's also (01:22:41) gently getting more proactive. So, it's (01:22:43) kind of nudging you about new papers or (01:22:45) new articles that come out um that are (01:22:48) obviously in tune with whatever you've (01:22:50) been talking about previously. So, you (01:22:52) know, it's a bit kind of a simplistic (01:22:54) copout answer, but just the more you use (01:22:55) it, the better it gets, the better it (01:22:57) learns you, the better you become (01:22:59) because it becomes this sort of aid to (01:23:00) your own line of inquiry. (01:23:02) >> So, that sounds like your your advice to (01:23:04) the audience is use copilot more and (01:23:06) that's the the single best accelerant (01:23:08) that you can do to speed this up (01:23:09) >> or any other AI. I mean, loads of great (01:23:11) >> I heard you also talk about can you (01:23:13) build the physical system that is going (01:23:16) to enable AI to run the experiments in a (01:23:20) 24/7 closed dark cycle to be able to (01:23:24) mine nature for data, right? And there (01:23:26) are a number of companies that are are (01:23:28) doing this. Laya is one recently out of (01:23:30) Harvard MIT. (01:23:31) >> Um I find that exciting where AI is (01:23:34) becoming an explorer (01:23:36) um on our behalf gathering that data. (01:23:39) Um, yeah. (01:23:41) >> Yeah. Spot on. (01:23:42) >> Yeah. (01:23:44) >> Thank you again. (01:23:45) >> This has been great. Thanks a lot. It (01:23:46) was a really fun conversation. (01:23:47) >> Yeah, really fun. Thanks. (01:23:49) >> Appreciate it, my friend. (01:23:50) >> All right. Good to see you. (01:23:51) >> Every week, my team and I study the top (01:23:53) 10 technology meta trends that will (01:23:54) transform industries over the decade (01:23:56) ahead. I cover trends ranging from (01:23:58) humanoid robotics, AGI, and quantum (01:24:00) computing to transport, energy, (01:24:02) longevity, and more. There's no fluff. (01:24:04) only the most important stuff that (01:24:06) matters that impacts our lives, our (01:24:08) companies, and our careers. If you want (01:24:10) me to share these meta trends with you, (01:24:12) I write a newsletter twice a week, (01:24:14) sending it out as a short two-minute (01:24:15) read via email. And if you want to (01:24:17) discover the most important metat trends (01:24:19) 10 years before anyone else, this (01:24:21) report's for you. Readers include (01:24:23) founders and CEOs from the world's most (01:24:25) disruptive companies and entrepreneurs (01:24:27) building the world's most disruptive (01:24:29) tech. It's not for you if you don't want (01:24:31) to be informed about what's coming, why (01:24:33) it matters, and how you can benefit from (01:24:35) it. To subscribe for free, go to (01:24:37) demandis.com/metrends (01:24:40) to gain access to the trends 10 years (01:24:42) before anyone else. All right, now back (01:24:44) to this episode.