Nick Bostrom: What Happens When AI Evolves Faster Than Humans? (YouTube Video Transcript)

(00:00:00) Your YouTube transcript will appear here (00:00:02) In the face of tremendous technological (00:00:04) development at unprecedented and (00:00:07) exponentially increasing speeds, we (00:00:10) stand at a pivotal moment to decide (00:00:12) whether or not we actively accelerate (00:00:15) humanity's natural evolution. And if we (00:00:18) do, then how? Welcome to Superhum. (00:00:26) >> Hi Nick. Nice to see you. Hey, Brian. I (00:00:29) think this is the first time we've ever (00:00:31) uh uh talked. (00:00:33) >> No, we actually met a couple years ago. (00:00:36) >> Uh where was that? Remind me. (00:00:38) >> At TED conference. (00:00:40) >> Oh, right. Oh, yes. Yes. (00:00:43) >> Good to see you again. (00:00:44) >> Yeah. Yeah. (00:00:44) >> I'm just I'm just 20 years younger, so I (00:00:46) look different. (00:00:48) >> That must be me. Unfortunately, it's (00:00:50) gone in the opposite direction. (00:00:52) >> Yeah. So Nick, we have a a great (00:00:53) community here uh all about health and (00:00:56) longevity and the future of the human (00:00:58) race. Uh without question, AI is the (00:01:02) most important thing happening on planet (00:01:05) Earth. And as I survey the world of (00:01:08) opinions, I mean you've been writing (00:01:09) about this. Um I read your book Super (00:01:11) Intelligence when it came out. Uh (00:01:13) there's a really wide range of opinions. (00:01:15) Some people are very excited about AI. (00:01:18) Some people think it's the end of (00:01:19) humanity. And these are really smart (00:01:21) people all with very divergent uh views. (00:01:24) How can you help us understand what do (00:01:27) you think the situation is with AI and (00:01:29) how do we think about it? (00:01:32) >> Well, so I'm both um frightened and (00:01:35) excited about it at the same time. U I (00:01:38) say I'm a fretful optimist with respect (00:01:40) to AI. I think it is not just one more (00:01:43) cool technology like you know the mobile (00:01:45) or cloud or web 2.0 or one of these (00:01:48) things people get excited about from (00:01:50) time to time but really fundamentally (00:01:52) different from everything else um (00:01:55) because it is the last invention we will (00:01:57) ever need to make. If you have machines (00:01:59) that can achieve the same general (00:02:02) intelligence that we humans have and (00:02:04) then super intelligence uh it really (00:02:05) means that from that point onwards (00:02:07) further developments will be um driven (00:02:10) by these uh machine intelligences. (00:02:14) Um, and I think what that will mean is (00:02:16) that we get a kind of telescoping of the (00:02:18) future. So, think of all those sci-fi (00:02:21) technologies that maybe the human (00:02:23) species could develop if we had 20,000 (00:02:26) years to work on it. You know, we'd have (00:02:27) space colonies and perfect VR and cures (00:02:30) for aging and uploading and all all of (00:02:33) those things, right? But I think what (00:02:35) might have taken the human species (00:02:37) 20,000 years to do if you have super (00:02:39) intelligence is driving the research at (00:02:40) digital time scales might happen within (00:02:43) you know a very brief period of time you (00:02:45) know months or a few years or something (00:02:46) like that. Um and it looks like we are (00:02:49) now actually rapidly advancing towards (00:02:51) this point where we might be able to (00:02:53) create um super intelligence. (00:02:56) >> So there's a lot of positive things that (00:02:58) can happen with AI. Obviously we all (00:03:01) imagine the amazing things we can (00:03:02) accomplish. A lot of people think that (00:03:04) uh so I guess when I talk to people in (00:03:06) AI it seems to me that they bifurcate on (00:03:10) on two paths based upon the following (00:03:12) model. The first group will say things (00:03:15) have been okay things are okay therefore (00:03:19) things will be okay. So for example (00:03:21) they'll say don't worry about job loss. (00:03:23) This is what happened during the (00:03:24) industrial revolution. Humans find new (00:03:26) things to do for example. So they'll (00:03:27) take any argument that basically is (00:03:29) negative or would be fearinducing and (00:03:32) say we're going to sort things out and (00:03:34) then the other group of people will say (00:03:36) actually this moment is different than (00:03:39) the other moments and then it branches (00:03:42) off into their own reasoning of of of (00:03:44) why there should be real concern about (00:03:46) existential risk about how we're not (00:03:48) paying enough attention to AI safety. Do (00:03:51) you identify with either one of these (00:03:53) paths that you think things will be okay (00:03:55) because things have been okay or do you (00:03:57) say that this moment is different or (00:03:58) would you classify yourself as a (00:04:00) different way of thinking about this? (00:04:03) >> I think this moment is different and we (00:04:05) can't really infer too much from the (00:04:08) fact that on balance so far it looks (00:04:10) like technology has been good um at (00:04:13) least for humans. I mean if if you look (00:04:15) at sort of um animals in in our animal (00:04:18) factories and stuff maybe they would (00:04:20) have a different opinion but um yeah I I (00:04:23) think it is fundamentally different um (00:04:26) um I think you could if you zoom out and (00:04:28) look at human history you can sort of (00:04:29) see two big transitions. So first from (00:04:32) hunter gatherer existence to (00:04:35) agricultural existence you know 90% plus (00:04:38) of humanity we were just running around (00:04:40) these small tribes um and then with the (00:04:43) agricultural revolution we started to (00:04:45) settle down and that led to the (00:04:46) development of states um specialization (00:04:50) and the rate of development picked up u (00:04:53) by orders of magnitude and the second (00:04:55) big transition is the industrial (00:04:57) revolution where for the first time uh (00:05:00) economic growth became so rapid that it (00:05:02) could outstrip population growth which (00:05:06) is the criterion for having average (00:05:09) income rise above subsistence. So for (00:05:12) all of human history before that uh the (00:05:14) bulk of humanity always lived at (00:05:17) subsistence level. You know the economy (00:05:18) grew but you know the economy grew by (00:05:20) 10% the population grew by 10% and you (00:05:23) still were just barely able to survive. (00:05:25) But for the last few hundred years, some (00:05:26) parts of the world have kind of and now (00:05:28) the striking thing is we've taken this (00:05:30) to be the normal way for things to be (00:05:32) where if if you look at it in any like (00:05:35) plausible way, it's like a huge anomaly. (00:05:37) This this normal human condition where (00:05:38) you sort of, you know, maybe you have an (00:05:40) alarm clock in the in the morning, uh (00:05:42) you commute into your office, you (00:05:44) struggle not to overeat, uh you sit in (00:05:47) front of a screen all day like that. (00:05:48) That's just not the way the human (00:05:51) condition was meant to be. And but but (00:05:53) now people think that anything that (00:05:55) projects into the future, some radical (00:05:57) departure of that is like some (00:05:58) extraordinary hypothesis that seems (00:06:00) crazy and would require some like very (00:06:03) special evidence to believe. Whereas to (00:06:05) me what seems crazy is to believe that (00:06:06) this current moment will just be the way (00:06:09) that things are for hundreds of years or (00:06:11) thousands of years indefinite into the (00:06:13) future. (00:06:13) >> Yeah. Um but yeah um so I think it is (00:06:17) it's very different and and one really (00:06:19) needs to assess it more from first (00:06:20) principles uh rather than just kind of (00:06:22) extrapolate uh previous uh trend lines. (00:06:25) So Nick, if you think this moment is (00:06:28) fundamentally different, that we can't (00:06:30) look back and match the patterns that (00:06:33) have happened to the future, do you look (00:06:37) to the future with hope or do you look (00:06:39) to the future with despair? And what is (00:06:41) your pdoom? (00:06:44) >> Yeah. Uh well, as I said, both uh hope (00:06:48) and not despair. I think um as as as (00:06:52) long as there is ignorance, there is (00:06:53) hope. uh because (00:06:56) um now I would say a kind of a moderate (00:06:58) fatalist. So I think a lot of this might (00:07:01) be baked in to there are various (00:07:03) challenges that will arise when we (00:07:05) develop machine intelligence. One of (00:07:06) these is the alignment problem. So how (00:07:09) to develop scalable methods for AI (00:07:11) alignment that will allow us to steer AI (00:07:14) systems even as they eventually become (00:07:16) far smarter and more capable than us. (00:07:19) And that is kind of an open research (00:07:21) challenge right now. Finally like all (00:07:24) the leading frontier AI labs not all of (00:07:25) them but many of them have research (00:07:27) groups working on this and it's (00:07:28) generally recognized that we need to (00:07:30) solve this. Um then there are other (00:07:33) challenges with governance etc. But um (00:07:36) whether we will solve those or not (00:07:38) probably mostly depends on how hard (00:07:40) these challenges turn out to be which we (00:07:43) don't really know. H so that's the (00:07:45) fatalism part. It's kind of baked in. (00:07:47) And then the moderate part of being a (00:07:49) moderate fatalist is I think on the (00:07:51) margin we can certainly improve the odds (00:07:52) a bit like the more we get our act (00:07:54) together you know maybe we can make some (00:07:57) percentage points difference in in the (00:07:59) odds. Um I've not really uh expressed a (00:08:02) precise pdoom. I I think there is a (00:08:05) significant existential risk um (00:08:08) connected to this transition to the (00:08:09) machine intelligence era. (00:08:12) Nevertheless, I see it as a sort of (00:08:14) portal through which we will have to (00:08:16) passage at some point. Like all the (00:08:18) possible path to really great futures (00:08:20) for humanity uh leads I think through (00:08:22) this this portal of the development of (00:08:25) machine super intelligence. Um it it it (00:08:29) might be nice at the right time if (00:08:31) whoever is developing this has the (00:08:33) opportunity to to pause or slow down for (00:08:36) you know 6 months or a year to sort of (00:08:39) work out all the safeguards to you know (00:08:41) rather than instantly cranking up all (00:08:43) the knobs to 11. Um but ultimately I (00:08:46) think it would itself be a kind of (00:08:47) existential catastrophe if we forever (00:08:50) failed to to take this next step. It's (00:08:52) like you're playing a computer game like (00:08:54) you you might want to explore a certain (00:08:56) level but at at a certain point it's (00:08:58) maybe time to sort of uh you know uh (00:09:01) level up and and check check the next (00:09:03) level. Um and I think we we are kind of (00:09:07) in that situation. (00:09:08) >> Yeah. All right, Nick. As the last (00:09:10) question uh a bunch of people here we (00:09:13) are interested in the well-being of the (00:09:16) human race. We see great potential for (00:09:18) the future. What can those who are (00:09:22) interested in doing that do about the (00:09:24) situation? I mean the the race dynamics (00:09:26) between nation states and companies to (00:09:30) build the best AI are almost on a (00:09:33) runaway train. It doesn't appear absent (00:09:35) some kind of catastrophic event in the (00:09:37) world. It seems like race dynamics are (00:09:40) going to take AI development at the (00:09:42) fastest possible speed of discovery and (00:09:45) capital. what can uh those not in the AI (00:09:48) specifically trenches uh building it, (00:09:50) what can anyone else do? (00:09:54) >> Yeah, so this uh is is a big and (00:09:56) difficult question. Um I I mean I think (00:10:00) um it depends a little bit on where (00:10:01) you're located in the world and sort of (00:10:03) your unique skills and um opportunities (00:10:06) to make a difference. Um I think broadly (00:10:10) speaking we should be aiming towards a a (00:10:12) future uh that is good for all where I (00:10:17) take all in a very wide sense here. So (00:10:20) including all humans (00:10:22) um but not just that also animals but (00:10:25) also importantly the digital minds that (00:10:28) we will be creating. So I think some of (00:10:30) these digital minds uh will have moral (00:10:33) status and and in the future most minds (00:10:35) I believe will be digital. So it is (00:10:37) important that the future we steer to is (00:10:40) is one that is also good for what will (00:10:42) be the majority population and and in (00:10:44) fact I mean we ourselves might be either (00:10:46) already or that's a separate (00:10:48) conversation but might become digital I (00:10:49) mean this this the previous talk with a (00:10:52) u crown uh option and then if you sort (00:10:56) of come out of that my guess is most (00:10:59) likely it would be a sort of upload (00:11:02) um so so I think there is a big ethic (00:11:06) ical challenge in sort of expanding the (00:11:10) circle of empathy to encompass concern (00:11:13) for these silicon implemented uh AIs and (00:11:18) uploads that we are creating. This is (00:11:21) going to be a big challenge because like (00:11:22) even with animals as I said earlier we (00:11:24) struggle but they have like at least (00:11:26) eyes that can squeak but if it's like an (00:11:27) invisible process running in a huge data (00:11:29) center it it might you know be even more (00:11:32) challenging for us. So, one of the ways (00:11:34) that I guess people could contribute is (00:11:36) to sort of um cultivate that sense of (00:11:38) expansive compassion and and empathy and (00:11:42) and then generally try to sort of steer (00:11:44) towards a generous um future. I think (00:11:47) the good news is that if this works out (00:11:49) that the pie would be extremely large, (00:11:51) there would be this unprecedented (00:11:52) economic boom. Ultimately, we would (00:11:55) unlock the resources of space. we would (00:11:57) have super advanced technology that that (00:11:59) would be more than enough to to like (00:12:01) give generous chunks to to everybody and (00:12:04) um so that that should be our first (00:12:06) instinct. First make sure everybody has (00:12:07) like plenty and then then we can sort of (00:12:09) squabble over uh the remainder. Um so so (00:12:13) those are some general directional uh uh (00:12:18) sort of uh efforts that that that people (00:12:21) can contribute to in different ways. Um (00:12:24) um so I would say there's the alignment (00:12:26) problem that's mostly I think for (00:12:28) technical people in in AI labs and other (00:12:30) places. Then the governance challenges (00:12:32) that's not a thing you solve once and (00:12:34) for all but it will be sort of managing (00:12:35) this transition as it unfolds and (00:12:37) figuring out how to nudge thing in a (00:12:38) desirable direction. Then expanding this (00:12:41) empathy to encompass digital minds (00:12:43) although it's very hard right now to (00:12:45) know exactly concretely what you would (00:12:46) do there but I think starting with the (00:12:49) principle at least is good. And then I (00:12:51) think there's like a fourth area of (00:12:52) challenge but that's even less clearly (00:12:54) visible right now. But ultimately this (00:12:56) super intelligence if we create one will (00:12:59) maybe enter a world where there exists (00:13:00) other super intelligences whether built (00:13:02) by alien civilizations or if we are in a (00:13:04) simulation our simulators or if there (00:13:06) are other branches of the Everett (00:13:07) universe (00:13:09) >> traditional theological beings they (00:13:11) would all be in this bracket of super (00:13:13) beings if they exist. And so ultimately (00:13:14) I think it's also really important that (00:13:16) our super intelligence gets along with (00:13:18) these other uh super beings. Um, and and (00:13:21) it's a kind of cooperative in that (00:13:22) space. Um, but this, yeah, it I I don't (00:13:27) have a sort of itemized to-do list for (00:13:29) each person. It's like I'm I've been (00:13:31) thinking about these things for decades (00:13:32) and I'm still kind of groping in the (00:13:34) dark here. It's just going to be a hard (00:13:36) challenge. (00:13:36) >> So, would it be fair to say we don't (00:13:39) know what to do? (00:13:42) >> Um, yeah. Yeah, that that would be fair (00:13:44) to do, but we still got to do. (00:13:45) >> Yeah. Cool. Nick, we're out of time. Uh, (00:13:48) really appreciate you hanging out with (00:13:49) us today. Great to see you. (00:13:50) >> Good to see you.