The Day After AGI (YouTube Video Transcript)

(00:00:00) Your YouTube transcript will appear here (00:00:11) Heat. Heat. (00:00:51) Welcome everybody and welcome to those (00:00:52) of you joining us on live stream um to (00:00:55) this conversation that I have to say I (00:00:57) have been looking forward to for months. (00:00:59) Uh, I had was lucky enough to ch to (00:01:02) moderate a conversation between Dar (00:01:04) Amade and Demis Hassabis last year in (00:01:06) Paris. Um, which I'm afraid got most (00:01:09) attention for the fact that you two were (00:01:10) squashed on a very small love seat while (00:01:13) I sat on an enormous sofa which was (00:01:15) probably my screw- up. But I said at (00:01:17) that point that this was for me like, (00:01:19) you know, chairing a conversation (00:01:20) between the Beatles and the Rolling (00:01:21) Stones. And you have not had a (00:01:23) conversation on stage since. So this is, (00:01:26) you know, the sequel. the the the you (00:01:28) know the bands get together again. I'm (00:01:29) delighted. You need no introduction. Uh (00:01:32) the title of our conversation is the day (00:01:34) after AGI which I think is perhaps (00:01:36) slightly getting ahead of ourselves (00:01:38) because we should probably talk about (00:01:39) how quickly and easily we will get there (00:01:42) and I want to do a bit of a sort of (00:01:43) update on that and then talk about the (00:01:44) consequences. So firstly on the timeline (00:01:46) Dario you last year in Paris said we'll (00:01:49) have a model that can do everything a (00:01:51) human could do at the level of a Nobel (00:01:53) laureate across many fields by 26 27. (00:01:56) we're in 26. Uh do you still stand by (00:01:59) that timeline? (00:01:59) >> So, you know, it's always hard to know (00:02:01) exactly when something will happen, but (00:02:03) but I don't I don't think that's going (00:02:05) to turn out to be that far off. So, um (00:02:07) you know, the the the mechanism whereby (00:02:10) I imagined it would happen is that we (00:02:12) would make models that were good at (00:02:14) coding and good at AI research and we (00:02:16) would use that to produce the next (00:02:19) generation of model and speed it up to (00:02:21) create a loop that would that would uh (00:02:23) increase the speed of model development. (00:02:25) we are now in terms of you know the (00:02:28) models that write code I have engineers (00:02:31) within enthropic who say I don't write (00:02:33) any code anymore I just I just let the (00:02:35) model write the code I edit it I do the (00:02:38) things around it I think I don't know we (00:02:40) might be 6 to 12 months away from when (00:02:44) the model is doing most maybe all of (00:02:46) what sues do end to end and then it's a (00:02:49) question of how fast does that loop (00:02:51) close not every part of that loop is (00:02:54) something that can be sped up by AI, (00:02:56) right? There's like chips, there's (00:02:57) manufacturer of chips, there's training (00:03:00) time for the model. So, it's, you know, (00:03:02) I I think there's a lot of uncertainty. (00:03:03) It's easy to see how this could take a (00:03:06) few years. I don't I I it's very hard (00:03:09) for me to see how it could take longer (00:03:10) than that. Um, but if if I had to guess, (00:03:13) I would guess that this goes faster than (00:03:15) people imagine. that that key element of (00:03:18) code and increasingly research going (00:03:21) faster than we imagine. That's going to (00:03:24) be the key driver. It's it's really hard (00:03:26) to predict again how much that (00:03:28) exponential is going to speed us up, but (00:03:29) but something fast is going to happen. (00:03:32) >> So you Demis were a little more cautious (00:03:34) last year. You said a 50% chance of a (00:03:36) system that can exhibit all the (00:03:37) cognitive capabilities humans can by the (00:03:40) end of the decade. Um clearly in coding, (00:03:43) as Dario says, it's been remarkable. (00:03:45) What is your sense of do you stand by (00:03:47) your prediction and what's changed in (00:03:49) the past year? (00:03:50) >> Yeah, look, I I I I think I'm still on (00:03:52) the same kind of timeline. I think there (00:03:53) has been remarkable progress, but I (00:03:55) think some areas of uh uh um kind of (00:03:59) engineering work, coding or so you could (00:04:01) say mathematics are a little bit easier (00:04:04) to see how they would be automated (00:04:05) partly because they're verifiable what (00:04:07) the output is. Um some areas of natural (00:04:10) science are much harder to do than that. (00:04:11) you won't necessarily know if the (00:04:13) chemical compound you've built or this (00:04:15) prediction about physics is correct. It (00:04:17) may be you may have to test it (00:04:18) experimentally and that will all take (00:04:20) longer. So uh I also think there are (00:04:22) some missing capabilities at the moment (00:04:25) uh in terms of like not just solving (00:04:27) existing conjectures uh or existing (00:04:30) problems but actually coming up with the (00:04:32) question in the first place or coming up (00:04:33) with the theory or the hypothesis. I (00:04:35) think that's much much harder and I (00:04:37) think that's the highest level of (00:04:38) scientific creativity and it's not (00:04:40) clear. I think we will have those (00:04:41) systems so I don't think it's impossible (00:04:43) but I think there may be one or two (00:04:44) missing ingredients. Um it remains to be (00:04:47) seen how you know first of all can this (00:04:48) self-improvement loop that we're all (00:04:50) working on actually close without human (00:04:52) in the loop. I think there also risks to (00:04:54) that with to that kind of system by the (00:04:56) way which we should discuss and I'm sure (00:04:58) we will but the the but but that could (00:05:00) speed things up if that kind of system (00:05:02) does work. We'll get to the risks in a (00:05:03) minute, but one other change I think of (00:05:05) the past year has been a kind of change (00:05:06) in the pecking order of the race, if you (00:05:08) will. This time a year ago, we just had (00:05:11) the Deep Seek moment and everyone was (00:05:13) incredibly excited about what happened (00:05:14) there and there was still a sense, you (00:05:17) know, that Google Deep Mind was kind of (00:05:19) lagging open AI. I would say that now uh (00:05:22) it's looking quite different. I mean, (00:05:23) they've declared code red, right? Um (00:05:25) it's been quite a quite a year. So talk (00:05:27) me through what specifically you've been (00:05:30) surprised by and how well you've done (00:05:32) this year and whether you think and then (00:05:34) I'm going to ask you about the lineup. (00:05:35) Well, look, I I think we were I was (00:05:37) always very confident we uh would get (00:05:40) back to sort of the top of the the (00:05:42) leaderboards and and the soda type of (00:05:44) models across the board because I think (00:05:46) we've always had like the deepest and (00:05:48) broadest research bench and it was about (00:05:50) kind of marshalling that all together (00:05:51) and um getting the intensity and focus (00:05:54) and the kind of startup mentality back (00:05:56) to the whole organization and it's been (00:05:58) a a lot of work and um but I think we're (00:06:01) and we're still a lot of work to do um (00:06:03) but I think you can start seeing the the (00:06:04) the the the you know the the kind of um (00:06:07) the progress that's been made in both (00:06:08) the models with Gemini 3 but also uh on (00:06:11) the product side with Gemini app getting (00:06:13) increasing uh market share. So I feel (00:06:16) like uh we're making great progress um (00:06:18) but there's a ton more work to do um and (00:06:21) you know we're bringing to bear Google (00:06:23) deep mind's kind of like the engine room (00:06:24) of Google where we're getting used to um (00:06:26) shipping our models more quickly into (00:06:29) the product surfaces. One question for (00:06:30) you Daria on on this aspect of it (00:06:32) because you've just or you're in the (00:06:33) process of you know a new round at an (00:06:36) extraordinary valuation too. Um but you (00:06:38) are unlike Demis a let's call it an (00:06:40) independent model maker and there is I (00:06:43) think an increasing concern that the (00:06:44) independent model makers will not be (00:06:46) able to continue for long enough until (00:06:48) you get to where the revenues come in. (00:06:50) Um it's made very openly about open AI (00:06:52) but talk me through how you think about (00:06:54) that and then we'll get to the AGI (00:06:55) itself. Yeah, I mean, you know, I think (00:06:57) I think I think how we think about that (00:06:59) is, you know, as we've built better and (00:07:02) better models, there's been a kind of (00:07:04) exponential relationship, not only (00:07:06) between how much compute you put into (00:07:08) the model and how cognitively capable it (00:07:10) is, but between how cognitively capable (00:07:12) it is and how much revenue it's able to (00:07:14) generate. So, our revenues grown 10x in (00:07:16) the last three years from 0 to 100 (00:07:18) million in 2023, 100 million to a (00:07:21) billion in 2024, and 1 billion to 10 (00:07:23) billion in 2025. And so th those revenue (00:07:26) numbers, you know, I don't know if that (00:07:27) curve will literally continue. It would (00:07:29) be crazy if it did. Um, but those (00:07:31) numbers are starting to get not too far (00:07:34) from, you know, the sca the scale of the (00:07:36) largest companies in the world. So (00:07:37) there's there's there's always (00:07:38) uncertainty. You know, we're trying to (00:07:40) bootstrap this from nothing. It's it's a (00:07:42) crazy thing, but but I have confidence (00:07:44) that if we're able to produce the best (00:07:46) models in the things that we focus on, (00:07:49) um, uh, then I think then I think things (00:07:51) will go well. And you know I I will I (00:07:53) will generally say you know I think I (00:07:54) think it's been a good year for both (00:07:56) both Google and Anthropic and I think (00:07:58) the thing we actually have in common is (00:08:01) that they're you know they're both kind (00:08:02) of kind of kind of companies that are (00:08:04) you know or the research part of the (00:08:06) company that are kind of led by (00:08:08) researchers who focus on the models who (00:08:10) focus on solving important problems in (00:08:13) the world right who have these kind of (00:08:14) hard scientific problems as a as a north (00:08:17) star and and and I think those are the (00:08:20) kind of companies companies that are (00:08:21) going to succeed going forward and you (00:08:24) know I think I think we share that (00:08:25) between us (00:08:26) >> very much. Uh I'm I'm going to resist (00:08:27) the temptation to ask you what will (00:08:29) happen to the companies that are not led (00:08:30) by researchers (00:08:32) uh because I know you won't answer it. (00:08:33) But let's then go on to uh the (00:08:37) predictions area now and this we are (00:08:39) supposed to be talking about the day (00:08:40) after AI. But let's talk about closing (00:08:42) the loop. This the odds that you will (00:08:44) get models that will close the loop and (00:08:46) be able to you know power themselves if (00:08:48) you will because that's the really the (00:08:50) crux for the the winner takes all (00:08:51) threshold approach. Do you still believe (00:08:54) that we are likely to see that or is (00:08:56) this going to be much more of a normal (00:08:58) technology where followers and catchup (00:09:01) can can compete? (00:09:03) >> Well, look, I definitely don't think (00:09:04) it's going to be a normal technology. (00:09:05) So, I mean, there are aspects already (00:09:07) that as Dario mentioned that it's (00:09:09) already helping with our coding and and (00:09:11) some aspects of research. The full (00:09:14) closing of the loop though, I think is (00:09:16) an unknown. I mean, I think it's (00:09:17) possible to do. you may need AGI itself (00:09:20) to be able to do that in some domains (00:09:22) again where there these domains you know (00:09:24) where there's there's more messiness (00:09:26) around them it's not so easy to verify (00:09:28) your answer very quickly um there's kind (00:09:30) of MP hard domains so as soon as you (00:09:33) start getting more and you know I also (00:09:35) include by the way for AGI physical AI (00:09:37) robotics working all of these kind of (00:09:39) things and then you've got you know (00:09:40) hardware in the loop uh that may uh (00:09:43) limit how fast the self-improvement (00:09:45) systems can work but I think in coding (00:09:47) and mathematics and these kind areas. I (00:09:49) can definitely see that working. And (00:09:50) then the question is more theoretical (00:09:52) one is what is the limit of engineering (00:09:54) and maths uh to solve uh the natural (00:09:57) sciences. Dario, you um last year, I (00:10:00) think it was last year that you (00:10:01) published Machines of Love and Grace um (00:10:03) which was a very I would say upbeat (00:10:06) essay about the potential that that you (00:10:09) were going to see unfold and you were (00:10:11) talking about you know a a what was it a (00:10:13) genius of data at country (00:10:16) data center I'm told that you are (00:10:18) working on an update to this a new essay (00:10:21) so you know wait for it guys it's not (00:10:23) out yet but it is coming out but perhaps (00:10:25) you can give us a sort of a sneak (00:10:28) preview of what a year later your big (00:10:30) take is going to be. (00:10:31) >> Yes. So, you know, my take my take has (00:10:34) not changed. It has always been my view (00:10:36) that, you know, AI is going to be (00:10:37) incredibly powerful. I think Demis and (00:10:39) I, you know, kind of agree on that. It's (00:10:41) just a question of exactly when um uh (00:10:43) and because it's incredibly powerful, it (00:10:45) will do all these wonderful things like (00:10:46) the ones I talked about in Machines of (00:10:48) Love and Grace. It, you know, will help (00:10:50) us cure cancer. It may help us to (00:10:52) eradicate tropical diseases. It will (00:10:54) help us understand understand the (00:10:56) universe. but that there are these, you (00:10:58) know, immense and grave risks that, you (00:11:01) know, not that we can't address them. (00:11:02) I'm not a doomer, but but that, you (00:11:04) know, we we we we we need to think about (00:11:06) them and we need to address them. And I (00:11:08) wrote Machines of Loving Grace first. I' (00:11:10) I'd love to give some uh a sophisticated (00:11:12) reason why I wrote that first, but it (00:11:14) was just that the the positive essay was (00:11:16) easier and more fun to write than than (00:11:18) the negative essay. Um, so, you know, I (00:11:20) finally spent some time on vacation and (00:11:22) I was able to write an essay about the (00:11:24) risks. And even when I'm writing about (00:11:26) the risks, um, I I I try, you know, I I (00:11:30) I'm like an optimistic person, right? So (00:11:32) even as I'm writing about these risks, I (00:11:34) I I wrote about it in a way that was (00:11:36) like, how do we overcome these risks? (00:11:38) How do we have a battle plan to fight (00:11:40) them? And and and the way I the way I (00:11:42) framed it was, you know, there's this (00:11:44) scene from Carl Sean's Contact, the (00:11:47) movie version of it, where, you know, (00:11:48) they they kind of discover alien life (00:11:50) and this international panel that's like (00:11:53) interviewing um uh you know, people to, (00:11:55) you know, to be humanity's (00:11:56) representative to meet the alien. Um uh (00:11:59) and uh one one of the questions they (00:12:01) asked one of the candidates is, you (00:12:03) know, if you could ask the aliens anyone (00:12:05) question, what it would what what what (00:12:06) would it be? And one of one of the (00:12:09) characters says, "I would ask, how did (00:12:11) you do it? How did you manage to get (00:12:13) through this technological adolescence (00:12:16) without destroying yourselves? How did (00:12:17) you make it through?" And and and ever (00:12:20) since I saw it, it was like 20 years (00:12:21) ago, I think I saw that movie, it's kind (00:12:22) of stuck with me. And that that's the (00:12:25) frame that I used, which is which is (00:12:26) that, you know, we we're we're we are (00:12:29) knocking on the door of these incredible (00:12:32) capabilities, right? the the ability to (00:12:34) build basically machines out of sand, (00:12:37) right? I think I think it was inevitable (00:12:39) that the instant we started working with (00:12:41) fire. Um uh but but how we handle it is (00:12:44) is not inevitable. And so I think the (00:12:47) next few years we're going to be dealing (00:12:49) with, you know, how do we keep these (00:12:52) systems under control that are highly (00:12:53) autonomous and smarter than any human? (00:12:56) How do we make sure that individuals (00:12:59) don't misuse them? Right? I have worries (00:13:01) about things like bioteterrorism. How do (00:13:03) we make sure that nation states don't (00:13:06) misuse them? That's why I've been so (00:13:07) concerned about, you know, the CCP, (00:13:09) other authoritarian authoritarian (00:13:12) governments. What are the economic (00:13:13) impacts? Right? I've talked about labor (00:13:15) displacement a lot. And and you know, (00:13:17) what what haven't we thought of which (00:13:18) which in many cases, you know, maybe may (00:13:20) be the the hardest thing to deal with at (00:13:22) all. Um so, you know, I I'm I'm thinking (00:13:25) through how to address those risks. And (00:13:28) you know, for for each of these, it's a (00:13:30) mixture of things that we individually (00:13:32) need to do as as leaders of the of of of (00:13:35) the companies and that we can do working (00:13:37) together. And then there there's going (00:13:38) to need to be some role for wider (00:13:40) societal institutions like the like the (00:13:42) government in in in addressing all of (00:13:44) these. But, you know, I I I just feel (00:13:46) this urgency that, you know, every day, (00:13:48) you know, there's there's all kinds of (00:13:50) crazy stuff going on in the outside (00:13:51) world, outside AI, right? Um but but you (00:13:54) know my my my view is this is happening (00:13:57) so fast and is such a crisis we should (00:13:59) be devoting almost all of our effort to (00:14:02) thinking about how to get through this. (00:14:04) >> So I can't decide whether I'm more (00:14:05) surprised that you a take a vacation b (00:14:07) when you take a vacation you think about (00:14:09) the risks of AI and c that your essay is (00:14:11) framed in terms of are we going to get (00:14:14) through the technological adolescence of (00:14:16) this technology without destroying (00:14:17) ourselves. So, I'm my head is slightly (00:14:19) spinning, but you then and I can't wait (00:14:20) to read it, but you you you mentioned (00:14:22) several areas that can guide the rest of (00:14:24) our conversation. Let's start with jobs (00:14:26) um because you actually have been very (00:14:27) outspoken about that and I think you (00:14:29) said that half of entry- level white (00:14:30) collar jobs could be gone within the (00:14:32) next one to five years. But I'm going to (00:14:34) turn to you Demis because so far we (00:14:37) haven't actually seen any discernable (00:14:40) impact on the labor market. Um, yes, (00:14:42) unemployment has ticked up in the US, (00:14:43) but all of the kind of economic studies (00:14:45) I've looked at and that we've written (00:14:47) about suggest that this is overhiring (00:14:50) post pandemic that it's really not (00:14:51) AIdriven. If anything, people are hiring (00:14:54) to build out AI capability. (00:14:57) Do you think that this will be as you (00:15:00) know economists have always argued that (00:15:02) it's not a lump of labor fallacy that (00:15:04) actually there will be new jobs created (00:15:06) because so far the evidence seems to (00:15:07) suggest that? Yeah, I mean I I think in (00:15:10) um the near term that is what will (00:15:12) happen. The kind of normal evolution (00:15:13) when a breakthrough technology arrives. (00:15:15) So some jobs will get disrupted but I (00:15:18) think new even more valuable perhaps (00:15:20) more meaningful jobs will get created. (00:15:22) Um I think we're going to see this year (00:15:23) the beginnings of maybe impacting the (00:15:26) junior level entry level kind of jobs (00:15:28) internships this type of thing. I think (00:15:30) there is some evidence I can feel that (00:15:32) ourselves maybe like a slowdown in (00:15:34) hiring in that but I think that can be (00:15:36) more than compensated by the fact there (00:15:38) are these amazing creative tools out (00:15:39) there pretty much available for everyone (00:15:42) uh almost for free that if you know I (00:15:44) was to talk to a a class of undergrads (00:15:47) right now I would be telling them to get (00:15:50) really unbelievably proficient with (00:15:52) these tools I think to the extent that (00:15:54) even those of us building it we're so (00:15:55) busy building it it's hard to have also (00:15:57) time to really explore the almost the (00:15:59) capability overhang even today's models (00:16:01) and products have let alone tomorrow's (00:16:04) and I think that uh can be maybe better (00:16:06) than a traditional internship would have (00:16:08) been in terms of you sort of leaprogging (00:16:10) uh yourself to be a useful uh in a (00:16:13) useful in a profession so I think (00:16:15) there's that's what I see happening (00:16:16) probably in the next 5 years um maybe we (00:16:19) again slightly differ on time scales on (00:16:21) that but I think what happens after AGI (00:16:23) arrives that's a different question (00:16:25) because I think really we would be in (00:16:26) uncharted territory at that point (00:16:28) >> do you think it's going to take longer (00:16:29) than you thought last year when you said (00:16:31) half of all white color jobs. (00:16:33) >> I have about the same view. I I actually (00:16:34) agree with you and with Demis that at (00:16:36) the time I made the comment there was no (00:16:39) impact on the labor market. I wasn't (00:16:40) saying there was an impact on the labor (00:16:41) market at that moment. Um you know now I (00:16:45) think maybe we're starting to see just (00:16:47) just the little beginnings of it you (00:16:48) know in software and coding. I even see (00:16:50) it within within anthropic where you (00:16:53) know I you know I can look forward I can (00:16:56) kind of look forward to a time where on (00:16:58) the more junior end and then on the more (00:17:00) on the more on the more on the more (00:17:02) intermediate end we actually need less (00:17:04) and not more people and you know we're (00:17:05) thinking about how to deal with that (00:17:07) within anthropic in a in a in a you know (00:17:10) sense in a sensible way. Um I you know (00:17:14) one to five years as of six months ago I (00:17:17) would stick with that. you know, if you (00:17:18) kind of, you know, connect this to what (00:17:20) I said before, which is, you know, we we (00:17:23) might have AI that's better than humans (00:17:25) at at everything in, you know, maybe one (00:17:27) to two years, maybe a little longer than (00:17:30) that. The those don't seem to line up. (00:17:32) The reason is that there's this there's (00:17:34) this lag and there's this replacement (00:17:37) thing, right? I I know that the labor (00:17:39) market is adaptable, right? Just like (00:17:41) you know 80% of people used to do (00:17:43) farming you know farming got automated (00:17:45) and then they became factory workers and (00:17:47) then knowledge workers. So you know (00:17:49) there is some level of adaptability here (00:17:52) as well right we should be economically (00:17:53) sophisticated about how the labor market (00:17:55) works but my worry is as this (00:17:57) exponential keeps compounding and I (00:18:00) don't think it's going to take that long (00:18:02) again somewhere between between a year (00:18:04) and five years it will overwhelm our (00:18:06) ability to adapt. I think I may be (00:18:08) saying the same thing Demis is just (00:18:11) factored out of that that difference we (00:18:13) have about timelines which I think (00:18:14) ultimately comes down to how how fast (00:18:16) you close the loop on CO and (00:18:17) >> how much confidence do you have that (00:18:19) governments get the scale of this and (00:18:22) have are beginning to think about what (00:18:25) policy responses they need to have (00:18:27) >> I don't think that that that it's (00:18:29) anywhere near enough work going on about (00:18:31) this I'm I'm constantly surprised even (00:18:33) when I meet economists at places like (00:18:34) this that they're not more of uh (00:18:36) professional economist, professors (00:18:38) thinking about what happens um and not (00:18:41) just sort of on the way to AGI but um (00:18:44) even if we get all the technical things (00:18:46) right that Dario is talking about and (00:18:47) the job displacement is one question (00:18:49) we're worried about the economics of (00:18:50) that but maybe there are ways to (00:18:51) distribute this new productivity this (00:18:54) new wealth more fairly I don't know if (00:18:56) we have the right institutions to do (00:18:57) that but that's what should happen at (00:18:59) that point there should be you know we (00:19:00) may be in a post scarcity world but then (00:19:02) there are even the things that keep me (00:19:03) up right now there are even bigger (00:19:05) questions than that at that to do with (00:19:07) meaning and um purpose and a lot of the (00:19:11) things that we get from our jobs not (00:19:12) just economically that's one question (00:19:14) but I think that may be easier to solve (00:19:16) strangely than uh what happens to the (00:19:18) human condition and humanity as a whole (00:19:21) and I think I'm also optimistic we'll (00:19:22) come up with new answers there we do a (00:19:24) lot of things today um from extreme (00:19:26) sports to art that aren't necessarily (00:19:29) directly to do with economic gain so I (00:19:32) think we will find uh meaning and maybe (00:19:34) there'll be even more sort sophisticated (00:19:36) versions of those activities. Um, plus I (00:19:39) think we'll be exploring the stars. So, (00:19:41) there'll be all of that to to factor in (00:19:43) as well for in terms of purpose. But I (00:19:46) think it's really worth thinking now (00:19:47) even on my timelines of like 5 to 10 (00:19:50) years away. That isn't a lot of time uh (00:19:52) before this comes. (00:19:53) >> How big do you think is the risk of a (00:19:55) popular backlash against AI that will (00:19:58) somehow kind of cause governments to do (00:20:02) what from your perspective might be (00:20:03) stupid things? Because I'm just thinking (00:20:05) back to the era of you know (00:20:07) globalization in the 1990s when when (00:20:10) there was indeed some displacement of (00:20:12) jobs governments didn't do enough the (00:20:14) public backlash was such that we've (00:20:16) ended up sort of where we are now. Uh do (00:20:19) you think that there is a risk that (00:20:20) there will be a growing antipathy (00:20:23) towards what you are doing and your (00:20:25) companies in the kind of body politic? (00:20:28) >> Um I think there's definitely a risk. I (00:20:29) think um I think that's kind of (00:20:31) reasonable. there's fear and there's (00:20:33) worries about these things like jobs and (00:20:35) livelihoods. Um I think there's a couple (00:20:38) of things that I mean it's going to be (00:20:39) very complicated the next few years I (00:20:41) think geopolitically but also the (00:20:43) various factors here like we want to and (00:20:45) we're trying to do this with AlphaFold (00:20:46) and our science work and isomorphic our (00:20:48) spinout company solve all disease cure (00:20:51) diseases come up with new energy sources (00:20:53) I think as a society it's clear we'd (00:20:55) want that I think maybe the balance of (00:20:57) what the industry is doing is not enough (00:20:59) balance towards those types of (00:21:00) activities I think we should have a lot (00:21:02) more examples I know Dario agrees with (00:21:03) me of like alpha fold like things that (00:21:06) help sort of unequivocal good in the (00:21:08) world. And I think actually it's (00:21:09) incumbent on the industry and and all of (00:21:11) us leading players to show that more, (00:21:13) demonstrate that, not just talk about (00:21:14) it, but demonstrate that. Um and but (00:21:17) then it's going to come with these other (00:21:18) intendent disruptions and um but I don't (00:21:21) I think the other issue is the (00:21:23) geopolitical competition. There's (00:21:24) obviously competition between the (00:21:25) companies but also US and China (00:21:27) primarily. So unless there's an (00:21:29) international cooperation or or (00:21:31) understanding around this um uh which I (00:21:33) think would be good actually in terms of (00:21:35) things like minimum safety standards for (00:21:37) deployment I think Dario would agree on (00:21:38) that as well. I think it's vitally (00:21:40) needed. This technology is going to be (00:21:41) crossber border. It's going to affect (00:21:42) everyone. It's going to affect all of (00:21:44) humanity. Um actually contact is one of (00:21:46) my favorite films as well. So funny (00:21:48) enough, I didn't realize it was yours (00:21:50) too, Dario. But I I think um um you know (00:21:53) those kind of things need to be worked (00:21:55) through. Um and and if we can maybe it (00:21:57) would be good to have a bit of slow a (00:21:59) slightly slower pace than we're (00:22:01) currently predicting even my timelines (00:22:03) so that we can get this right society. (00:22:05) But that would require some coordination (00:22:07) that is I I prefer your timelines. (00:22:10) >> Yes, I will concede. (00:22:13) >> But but Dario, let's turn to this now (00:22:14) because one thing since we last spoke uh (00:22:17) in Paris, the geopolitical environment (00:22:19) has, if anything, I don't know, (00:22:21) complicated, mad, crazy, whatever, (00:22:23) whatever phrase you want to use. (00:22:25) Secondly, the US has a very different (00:22:27) approach now towards China. It's a much (00:22:30) more it's a kind of no holds barred, go (00:22:32) as fast as we can, but then sell chips (00:22:33) to China. Um and that is it. So you've (00:22:37) got a different attitude towards the (00:22:38) United States. You've got a a very um (00:22:42) strange relationship between the United (00:22:44) States and and Europe right now (00:22:45) geopolitically against that. I mean I (00:22:48) hear you talk about it would be nice to (00:22:49) have a CERN like organization. I mean (00:22:51) it's a million years from where we are (00:22:53) from the real world. So in the real (00:22:55) world have the geopolitical risks (00:22:56) increased and what if anything do you (00:23:00) think should be done about that? And and (00:23:01) the administration seems to be doing the (00:23:02) opposite of what you were suggesting? (00:23:03) Yeah, I mean, look, you know, we're (00:23:05) we're we're just trying to do the best (00:23:06) we can to, you know, we're just we're (00:23:08) just one company and we're we're trying (00:23:09) to operate in, you know, the the (00:23:10) environment that exists, no matter how (00:23:12) no matter how crazy it is. But, you (00:23:14) know, I think I think at least my policy (00:23:16) recommendations haven't changed that, (00:23:19) you know, not selling chips is one of (00:23:22) the, you know, one of the one of the (00:23:24) biggest things we can do um to, you (00:23:27) know, make sure that we have the time to (00:23:29) handle this. Um, you know, you know, I (00:23:31) said I said before, you know, I I I (00:23:34) prefer Demis' timeline. I wish we had (00:23:36) five to 10 years, you know, so it's it's (00:23:39) possible he's just right and I'm just (00:23:40) wrong, but but assume I'm right and it (00:23:42) can be done in one to two years. Why (00:23:44) can't we slow down to to Demis' (00:23:45) timeline? (00:23:47) Well, no. The but but but the reason the (00:23:50) reason we the reason we can't do that is (00:23:52) is you know because we have geopolitical (00:23:55) adversaries building the same technology (00:23:58) at a similar pace. It's very hard to (00:24:01) have an enforcable agreement where they (00:24:03) slow down and we slow down and and so if (00:24:05) we can just if we can just not sell the (00:24:08) chips then this isn't a question of (00:24:11) competition between the US and China. (00:24:13) This is a question of competition (00:24:14) between me and Demis which I'm very (00:24:16) confident that we can work out. (00:24:18) >> And what do you make of the logic of the (00:24:20) administration which as I understand it (00:24:21) is we need to sell them chips because we (00:24:23) need to bind them into US supply chains. (00:24:27) >> So you know it's it's I I think it's I (00:24:31) think it's a question not just of time (00:24:34) scale but of the significance of the (00:24:35) technology. Right? If this was telecom (00:24:39) or something, then all this stuff about (00:24:41) proliferating the US stack and you know (00:24:44) wanting to build our you know chips (00:24:46) around the world to make sure that you (00:24:47) know you know this c you know the you (00:24:51) know these random countries in different (00:24:53) parts of the world you know build data (00:24:55) centers that have Nvidia chips instead (00:24:57) of Huawei chips you know I think of this (00:25:00) more as like you know it's a decision (00:25:02) are we going to you know sell nuclear (00:25:05) weapons to North Korea Uh and you know (00:25:08) because that produces some profit for (00:25:10) Boeing. Um you know where where we can (00:25:12) say okay yeah these cases were made by (00:25:14) Boeing like the US is winning like this (00:25:16) is great like I I I just you know that (00:25:18) that analogy should just make clear how (00:25:21) I see this trade-off that I just don't (00:25:23) think it makes sense. Um and and we've (00:25:26) done a lot of more aggressive stuff to (00:25:29) you know toward towards towards China (00:25:31) and other players that that I think is (00:25:32) much less effective than this this one (00:25:34) this one measure. One more area from me (00:25:37) and then I hope we'll have time for a (00:25:38) question or two. The other area of (00:25:41) potential risk that doomers worry about (00:25:43) is a kind of all powerful malign AI. Um (00:25:46) and I think you've both been somewhat (00:25:48) skeptical of the doomer approach but in (00:25:50) the last year we have seen you know (00:25:52) these models showing themselves to be (00:25:54) capable of deception duplicity. Uh do (00:25:58) you think that do you think differently (00:26:00) about that risk now than you did a year (00:26:02) ago? And is there something about the (00:26:05) way the models are evolving that we (00:26:06) should put a little bit more concern on (00:26:08) that? (00:26:08) >> Yeah, I mean you know since since the (00:26:10) beginning of anthropic we've kind of (00:26:11) thought about this risk. I mean you know (00:26:14) our our our research at the beginning of (00:26:15) it was very theoretical right? You know (00:26:17) we pioneered this idea of mechanistic (00:26:19) interpretability which is looking inside (00:26:21) the model and and trying to understand (00:26:23) looking inside its brain trying to (00:26:25) understand why it does what it does as (00:26:27) it you know as as human neuroscientists (00:26:30) which we actually both have background (00:26:31) in. um try try to understand try to (00:26:34) understand the brain and I think as time (00:26:36) has gone on we've we've increasingly (00:26:38) documented the you know bad behaviors of (00:26:40) the models when they emerge and are now (00:26:42) working on trying to address them with (00:26:44) mechanistic interpretability. So I you (00:26:47) know I think uh you know I I've always (00:26:49) been concerned about these these risks. (00:26:50) I've talked to Demis many times. I think (00:26:52) he has also been um concerned about (00:26:54) these risks. I think I have definitely (00:26:57) been and I I I would guess Demis as well (00:26:59) although I'll let him speak for himself (00:27:01) skeptical of of doomerism which is you (00:27:04) know we're doomed there's nothing we can (00:27:06) do or this is the most likely outcome. I (00:27:08) think this is a risk this is a risk that (00:27:11) if we work all work together we can (00:27:14) address we can learn through science to (00:27:16) properly you know control and and direct (00:27:19) these creations that we're building. But (00:27:21) if we build them poorly, if we go, you (00:27:25) know, if if if we're all racing and we (00:27:28) go so fast that there's no guard rails, (00:27:30) then I think there is risk of something (00:27:31) going wrong. (00:27:32) >> So, I'm going to give you a chance to (00:27:33) answer that in the context of of a (00:27:34) slightly broader question, which is over (00:27:36) the past year, have you grown more (00:27:38) confident of the upside potential of the (00:27:42) technology, science, all of the areas (00:27:44) that you have talked about a lot, or are (00:27:46) you more worried about the risks that (00:27:47) we've been discussing? Look as I've been (00:27:50) working on this for 20 plus years. So we (00:27:52) we already knew the reason I've spent my (00:27:54) whole career on AI is is the upsides of (00:27:57) solving basically the ultimate tool for (00:27:59) science and understanding the universe (00:28:01) around us. I've I've sort of been (00:28:03) obsessed with that since a kid and and (00:28:04) and building AI is the you know should (00:28:07) be the ultimate tool for that if we do (00:28:08) it in the right way. The risks also (00:28:10) we've been thinking about since the (00:28:11) start at least the start of deep mine 15 (00:28:13) years ago and um we kind of sort of (00:28:16) foraw that if you got the upsides it's a (00:28:18) dual purpose technology so it could be (00:28:20) repurposed by say bad actors for harmful (00:28:22) ends so we've needed to think about that (00:28:24) all the way through but I'm a big (00:28:25) believer in human ingenuity um but the (00:28:28) question is having the time and the (00:28:31) focus and all the best minds (00:28:33) collaborating on it to solve these (00:28:35) problems. I'm sure if we had that we (00:28:37) would solve the technical risk problem. (00:28:39) It may be we don't have that and then (00:28:41) that will introduce risk because we'll (00:28:42) be sort of it'll be fragmented. There'll (00:28:45) be different projects and people be (00:28:46) racing each other. Then it's much harder (00:28:48) to make sure you know these systems that (00:28:49) we produce will be technically safe. But (00:28:52) I I feel like that's a very tractable uh (00:28:54) problem. (00:28:56) >> If you if you have the time I want to (00:28:58) make sure there's one question gentlemen (00:29:00) keep it very short because we've got (00:29:02) literally two minutes. (00:29:04) Thanks for Hello. (00:29:05) >> Yeah. No speak. (00:29:07) >> Thanks very much. I'm Philip, co-founder (00:29:08) of Star Cloud Building Data Centers in (00:29:10) Space. Um, I wanted to ask a very (00:29:12) slightly philosophical core question. (00:29:14) The sort of strongest argument for (00:29:15) doomerism to me is the Fermy paradox, (00:29:17) the idea that we don't see intelligent (00:29:18) life in our galaxy. I was wondering if (00:29:20) you guys have any thoughts. (00:29:21) >> Yeah, I've thought a lot about that. (00:29:22) That can't be the reason because we we (00:29:23) we should see all the AIs that have So, (00:29:26) just for everyone know the idea is well, (00:29:29) it's sort of unclear why that would (00:29:30) happen, right? So if if the reason (00:29:32) there's a Fmy paradox there are no (00:29:34) aliens because they get taken out by (00:29:36) their own technology we should be seeing (00:29:38) paper clips coming towards us from some (00:29:40) part of the galaxy and apparently we (00:29:42) don't we don't see any structures Dyson (00:29:44) sphere is nothing whether they're AI or (00:29:46) natur or sort of biological so to me um (00:29:49) there has to be a different answer to (00:29:50) fmy patterns I have my own theories (00:29:51) about that but it's out of scope for the (00:29:53) next minute but um you know I I just (00:29:55) feel like uh that that I my prediction (00:29:58) my feeling is that we're past the great (00:30:00) filter it probably multisellular life if (00:30:03) I would have to guess was incredibly (00:30:04) hard for for biology to evolve that. Um (00:30:07) so we're on you know there isn't a (00:30:09) comfort of like what's going to happen (00:30:10) next. I think it's for us to write as (00:30:12) humanity what's going to happen next. (00:30:14) >> This this could be a great discussion (00:30:15) but is out of scope for the next 36 (00:30:17) seconds. But what isn't 15 seconds each (00:30:19) what when we meet again I hope next year (00:30:22) uh the three of us which I would love uh (00:30:23) what will have changed by then? I well I (00:30:27) think the biggest thing to watch is this (00:30:30) issue of AI systems building AI systems (00:30:34) how that goes whe that whether that goes (00:30:36) one way or another that that will (00:30:39) determine you know whether it's a few (00:30:41) more years until we get there or or if (00:30:44) we have you know you know if if we have (00:30:47) wonders and and a great emergency in (00:30:50) front of us that we have to face (00:30:51) >> AI systems building (00:30:53) >> I agree on that so we're we're keeping (00:30:54) close touch about that um but also I (00:30:56) think um outside of that I think there (00:30:58) are other interesting uh uh uh ideas (00:31:01) being researched like world models (00:31:02) continual learning these are the things (00:31:04) I think that will need to be cracked if (00:31:05) self-improvement doesn't sort of deliver (00:31:07) the goods on its own then we'll need (00:31:09) these other things to work and then I (00:31:11) think things like robotics may have its (00:31:13) sort of breakout moment (00:31:14) >> but maybe on the basis of what you've (00:31:16) just said we should all be hoping that (00:31:17) it does take you a little bit longer and (00:31:18) indeed everybody else to give us (00:31:20) >> I would prefer that I think that would (00:31:21) be better for the world (00:31:22) >> but you guys could do something about (00:31:24) that thank you So, it's very MUCH