Dylan Patel: NVIDIA’s New Moat & Why China is “Semiconductor Pilled” (YouTube Video Transcript)

(00:00:00) Your YouTube transcript will appear here (00:00:00) This is the biggest change in human (00:00:02) history maybe ever. What's about to (00:00:04) happen with AI? This is the biggest (00:00:06) revolution bigger than industrial (00:00:07) revolution. Jensen is very paranoid (00:00:09) about losing. If he just kept making his (00:00:10) mainline chip, people crush him on cost (00:00:12) and performance. Acquiring Grock is how (00:00:14) you get those resources to make more (00:00:15) solutions for different parts of the (00:00:17) market to stay king. At the end of the (00:00:18) day, this is an economic war. If the US (00:00:20) and the West win in AI, China will not (00:00:24) rise to be the global hedgeimony. But (00:00:25) without AI, China definitely will rise. (00:00:28) They're just going to outrun America. (00:00:29) Hi, I'm Matt Turk. Welcome back to the (00:00:31) Matt podcast. Today I'm joined by the (00:00:33) one person Wall Street and Silicon (00:00:35) Valley turn to when they need to cut (00:00:37) through the hardware hype, Dylan Patel (00:00:39) of Semi analysis. We dove into many of (00:00:41) the most important topics [music] of (00:00:43) today. Nvidia's massive move to acquire (00:00:45) Grock, the truth about the capex bubble, (00:00:47) whether the US power grid can actually (00:00:49) handle the AI boom, and the geopolitical (00:00:52) chess match [music] playing out between (00:00:53) the US and China. But I have to warn (00:00:55) you, this conversation went off the (00:00:57) rails in the best possible way. And we (00:00:59) ended up going into all sorts of fun (00:01:00) tangents like the strange phenomenon of (00:01:03) Chinese romance dramas set inside (00:01:05) semiconductor factories and what's (00:01:07) really like when three AI famous (00:01:08) roommates live together in SF. Please (00:01:10) enjoy this fantastic conversation with (00:01:12) Dylan. (00:01:15) >> Hey Dylan, welcome. (00:01:16) >> Hello. How are you? (00:01:17) >> I'm great. I'd love to start with Grock (00:01:19) and Nvidia since it's still fresh. So, (00:01:22) not so long ago, Nvidia was saying that (00:01:24) uh one GPU could do it all, and now (00:01:26) they're doing this acquisition (00:01:28) non-exclusive deal with Grock. What does (00:01:31) that mean from your perspective? (00:01:33) >> It's very clear. We're not sure where AI (00:01:36) models are headed in terms of, you know, (00:01:38) over the next few years, what happens to (00:01:40) the architecture, but you know, the (00:01:42) thing that I think everyone is sort of (00:01:43) like agreed on is models are pretty auto (00:01:45) reggressive, right? Next token (00:01:46) generation is like the thing but beyond (00:01:48) that right attention mechanisms changed (00:01:49) the how how it works everything changes (00:01:51) right could could change and so what's (00:01:52) interesting is the reason Nvidia one is (00:01:55) because they just took like the widest (00:01:56) surface area bet and then people kept (00:01:57) developing models on that and that kind (00:02:00) of shape worked but now the workload is (00:02:02) so large that there is room for (00:02:04) specialization that will give you 10x (00:02:06) increases in certain domains right in a (00:02:08) general purpose workload grock doesn't (00:02:10) work right you know it can't train it (00:02:12) can't you know it can't inference really (00:02:13) really large models um cost efficiently, (00:02:16) right? You can't serve many many many (00:02:17) users, but what it can do is it can go (00:02:19) bl screamingly fast, right? Same with (00:02:21) the cerebrous open AI deal, but that's (00:02:23) like one workload, right? Uh very decode (00:02:26) focused, right? Gener doing auto (00:02:27) reggressive tokens in a in a single (00:02:29) stream super fast. Another direction AI (00:02:31) models could head, right? We don't know (00:02:33) are models going to think in one token (00:02:35) stream or is it actually they're (00:02:38) constantly context switching, right? and (00:02:40) they're going from they have this (00:02:41) humongous humongous context and they're (00:02:44) generating in multiple parallel streams (00:02:47) right and so Google and openi have both (00:02:49) released mechanisms of this with their (00:02:51) pro models where the model actually (00:02:53) doesn't just have one single chain of (00:02:55) thought for reasoning it has multiple (00:02:57) right and then I don't exactly like you (00:02:59) know and and and how they choose which (00:03:00) one and what the final answer to you (00:03:02) delivers is is an area of research um (00:03:05) but there there is room for that kind of (00:03:07) chip right something that works on very (00:03:08) parallel a lot lot of streams of chain (00:03:11) of thought and maybe the latency (00:03:12) requirements are not as crazy, right? (00:03:13) Maybe you don't want to go blindingly (00:03:15) fast, right? Maybe you're okay with it (00:03:17) being, you know, because I can spin up (00:03:19) 100 parallel, you know, streams of (00:03:21) thought or agents or whatever you want (00:03:22) to call them. Maybe I I care a lot about (00:03:24) cost there. And because it's 100 in (00:03:26) parallel instead of one going super (00:03:27) super fast, it's not as deep, right? The (00:03:29) tree search or the depth of the (00:03:31) inference is not as deep, but it is much (00:03:32) wider. You know, there's other parts of (00:03:34) inference. Hey, process do creating the (00:03:35) KV cache. So, Nvidia has a chip for (00:03:37) that, right? That's the CPX. So they (00:03:39) they've made the CPX, they bought Grock (00:03:41) for decode, and then they still have (00:03:42) their general purpose GPU. So they've (00:03:43) they're kind of trying to cover their (00:03:45) bases because unlike the first wave of (00:03:47) AI chip companies where they sort of (00:03:49) just made chips and then tried to figure (00:03:50) out where it would work, right? They had (00:03:51) a thesis, Grock and Cerebrus, both as (00:03:53) well as Samanova, right, which was put a (00:03:55) lot of memory on the chip and not (00:03:57) necessarily in the case of Cerebrus and (00:03:59) Grock, no memory off chip. And in the (00:04:01) case of Samanova, less memory offchip or (00:04:03) slower memory offchip with higher (00:04:04) capacity. You know, they they sort of (00:04:06) all made similar bets in that direction. (00:04:08) And it didn't work for a while until it (00:04:11) kind of did, right? Um because there's a (00:04:13) workload that now necessitates it. (00:04:14) Nvidia recognizes they're they're the (00:04:16) leader. They're at the tent pole. Hey, (00:04:18) in one respect they can just run faster (00:04:21) than everyone, but it's kind of hard to (00:04:23) be 2x better than Google or or OpenAI or (00:04:27) whoever else's internal chip, right? To (00:04:29) justify their, you know, 75% plus (00:04:31) margins, right? And then they have to be (00:04:33) 2x to 4x better to justify 4x better to (00:04:36) justify their margins because that's (00:04:37) what they're charging above COGS. You (00:04:38) know, the question is what what (00:04:40) architecture will deliver that? Well, (00:04:42) yes, keep the programmability of their (00:04:43) GPUs is great for training and for a lot (00:04:47) of workloads, but you know, guess what? (00:04:48) I think I think a lot of people will (00:04:50) just be downloading an open source (00:04:52) model, downloading an inference (00:04:53) framework and pressing go, right? A (00:04:55) little bit more complicated than that, (00:04:56) but that's that's going to be the (00:04:58) consumption method for a lot of (00:04:59) enterprises, a lot of uh startups, a lot (00:05:02) of tech companies is they're just going (00:05:03) to do that or they're going to rent the (00:05:05) G GPUs or or rent the chips and then (00:05:07) download an open source framework and (00:05:08) model and go, right? And Nvidia (00:05:10) recognizes this and hey, there is room (00:05:11) for products that aren't general (00:05:13) purpose, right? The general purpose GPU (00:05:15) will still probably be the main line for (00:05:17) training and for a lot of inference and (00:05:19) for costefficient inference, but maybe (00:05:21) blindingly fast or workloads that have a (00:05:23) ton of prefill, i.e. creating the the KV (00:05:25) cache. Maybe that those workloads could (00:05:28) be different chips, right? And the CPX (00:05:29) chip they announced, right? They say (00:05:30) it's for the context processing, (00:05:32) creating the KV cache. It's also really (00:05:33) useful for video models because video (00:05:35) models don't care about memory bandwidth (00:05:36) and so you know why pay for the (00:05:37) expensive memory that the general (00:05:39) purpose chip has or why do what Grock is (00:05:40) doing which is tying hundreds or (00:05:42) thousands of chips together and not (00:05:44) having memory but keeping the entire (00:05:46) model on chip. The trade-off for that of (00:05:48) course is you need thousands of chips (00:05:49) and you have less compute per chip and (00:05:51) so like Nvidia's trying to capture the (00:05:53) whole surface area because again you (00:05:54) don't know where models are headed and (00:05:56) it's hard to say where the research is (00:05:57) headed. (00:05:57) >> And do you think it's a good thing for (00:05:58) the market? Yet another one of those (00:06:00) deals that's structured as a as a (00:06:02) license but really an acquisition. (00:06:04) >> I certainly think it's not good from an (00:06:08) anti-competitive sense, right? I don't (00:06:10) think people should just be able to buy (00:06:12) companies without like any antitrust (00:06:15) like process at all. Now, in the case of (00:06:17) like a large company buying a startup, (00:06:19) I'm completely fine with it. The flip (00:06:20) side is like, hey, we know the deal is (00:06:22) happening, right? Uh this happened for a (00:06:24) company I was an adviser for Nvidia (00:06:26) acquired in fabrica just maybe a few (00:06:27) months before they did Grock and similar (00:06:30) style of deal right if someone wanted to (00:06:32) strike it down that's the biggest limbo (00:06:34) right we've seen this happen in venture (00:06:35) and you probably know more stories of (00:06:37) this but like a company trying to get (00:06:38) acquired they get stuck in limbo for (00:06:40) like a year (00:06:41) >> and then it falls apart (00:06:43) >> stories (00:06:43) >> yeah it falls apart the deal did because (00:06:46) some regulatory BS and now the company (00:06:49) was and the founders were focused on (00:06:50) getting the deal done instead of like (00:06:52) making the product better for a year and (00:06:53) now they're like behind or you know they (00:06:54) they they weren't focused on growth as (00:06:56) much right you know you only have so (00:06:57) much time as a founder so in that sense (00:06:59) I like the license deals right (00:07:00) >> so now is uh Nvidia also dominating the (00:07:04) the inference market is there any world (00:07:06) where Nvidia is no longer the king or (00:07:08) they seem to be getting stronger (00:07:10) >> I think the thing about Nvidia is they (00:07:13) take the Andy Grove mentality like more (00:07:16) serious than anyone else right like okay (00:07:18) fine Google like implemented OKRs (00:07:20) because Intel did it but that's like you (00:07:21) know management stuff, right? Only the (00:07:23) paranoid survive, right? This is like (00:07:25) core to the Bay Area, um core to Nvidia. (00:07:28) Um Jensen is very paranoid about losing, (00:07:31) right? These specializations, if he just (00:07:32) kept making his mainline chip, would (00:07:34) mean people could, you know, point point (00:07:36) solutions for specific parts of the (00:07:38) market would crush him on cost and (00:07:39) performance. Then he can't justify his (00:07:40) margin. That's a threat to Nvidia's (00:07:42) business model as a whole, especially if (00:07:44) the best model only changes every 3 (00:07:46) months or the model you want to roll (00:07:47) out. Okay, well then you're going to (00:07:48) have three months to figure out how to (00:07:49) make a model work on one chip (00:07:51) architecture for that point solution and (00:07:53) you know it's fine. Software software (00:07:54) advantage of Nvidia is not that (00:07:55) important. Then Jensen's super paranoid (00:07:57) about losing and frankly it's really (00:07:59) hard to hire enough talented chip (00:08:01) people. When you look across the market, (00:08:03) there is only a few companies who have (00:08:05) successfully created a chip architecture (00:08:08) software to run the models accurately, (00:08:11) run the run the models accurately, (00:08:12) right? Like cuz you can look at random (00:08:14) APIs of say an Alibaba Quen model and (00:08:17) different people are doing all sorts of (00:08:18) tricks like quantizing it, but also many (00:08:20) other tricks which then end up like (00:08:22) making the model quality lower. You (00:08:23) know, building a rack scale solution, (00:08:25) networking thousands of chips together (00:08:26) and then deploying an API and Grock did (00:08:28) the whole thing with frankly not that (00:08:30) many people. So now it's like okay well (00:08:32) I'm Nvidia I want to make four different (00:08:33) chip architectures and actually four (00:08:35) different point solutions maybe the (00:08:36) general purpose and then one here one (00:08:37) here one here and in addition my general (00:08:40) purpose thing is actually not just like (00:08:41) a GPU chip it's like GPU chips CPU chips (00:08:44) networking chips NV switch nicks like (00:08:47) you know there's many many chips and (00:08:48) each of those chips has many chiplets (00:08:50) you don't have enough engineering (00:08:51) resources right and so like acquiring (00:08:53) gro is like how you get those resources (00:08:54) to make more solutions for different (00:08:56) parts of the market as far as like are (00:08:58) they threatened like I think I think (00:08:59) like obviously There's some cool (00:09:01) startups out there, right, that are (00:09:02) raising a lot, right, currently or have (00:09:05) raised such as Etched, Maddx, uh, (00:09:07) positron, these new age of AI companies. (00:09:09) There's also the prior age of like (00:09:10) Cerebrris is is out there still, right? (00:09:13) You know, Tenstor, etc. And there's so (00:09:14) there's a lot of AI chip companies on (00:09:15) the startup side, but then there's also, (00:09:16) you know, Google's TPU, AMD GPUs, uh, (00:09:20) Amazon Tranium, uh, who are all really (00:09:22) credible competitors. And then, you (00:09:24) know, Meta's MTIA is somewhat credible. (00:09:26) and then you know Microsoft Somaya is (00:09:28) not credible but like you know maybe it (00:09:30) will be one day right so you sort of (00:09:31) have like a lot of competition they've (00:09:33) got to hold the gates back and so I (00:09:34) think (00:09:35) >> is there a risk to them being I mean (00:09:37) like there's there's risk from all of (00:09:38) those companies that I mentioned and and (00:09:40) you know effectively California/ Seattle (00:09:43) right only two two places there's (00:09:45) there's also chips from other parts of (00:09:46) the world right obviously China has a (00:09:48) number of different AI chip companies (00:09:49) that are doing cool things anyone would (00:09:51) have told you Grock was you know their (00:09:53) business revenue their revenue was not (00:09:55) like stellar right in fact they missed (00:09:57) revenue last year significantly and yet (00:09:58) they got bought right because the value (00:10:00) of the IP was there and the value of the (00:10:01) team anyone else would have been like (00:10:02) well why the heck would I buy this right (00:10:04) uh makes no sense there's definitely a (00:10:06) credible threat (00:10:06) >> yeah and do you think uh CUDA is going (00:10:09) to remain that mode I guess a (00:10:11) combination of CUDA and whatever came (00:10:13) out of the Melanox acquisition like do (00:10:15) do those persist as long-lasting (00:10:17) advantages (00:10:18) >> I think they do I think networking is (00:10:20) super important I think uh the CUDA (00:10:22) software mode is very important but it's (00:10:24) also like changing rapidly right It's an (00:10:26) incredible amount of the software that (00:10:28) Nvidia GPUs run on is not from Nvidia. (00:10:30) It's it's the developer ecosystem that's (00:10:32) open sourcing it. When you look at, for (00:10:34) example, VLM and SGLANG, right? These (00:10:37) support AMD GPUs almost as first class (00:10:39) citizens now. And VLM is getting (00:10:42) significant support for TPUs for tranium (00:10:46) and there will be other chips coming out (00:10:48) from startups that also support VLMs (00:10:50) SGLang. Now like how difficult is it? (00:10:52) You know the the the reason why CUDA is (00:10:54) so important is like okay I can do (00:10:55) whatever I need to do right programming (00:10:57) a GPU. (00:10:58) >> I think most AI chips will not be (00:11:01) consumed by people programming anything (00:11:03) for it. (00:11:04) >> They will download an open source (00:11:06) inference engine. and they will download (00:11:07) an open source model and then they will (00:11:09) put it on the and it's really simple to (00:11:11) download VLM and like make it work like (00:11:13) it's not that hard to set up uh you know (00:11:15) a server and Nvidia's putting out a lot (00:11:16) of open source software like Triton (00:11:18) inference server and and uh Dynamo and (00:11:21) all these things to to make it easy (00:11:22) because that is the consumption model (00:11:24) ultimately for the majority of AI right (00:11:27) is and it might be like oh it's my own (00:11:29) inference engine but most servers will (00:11:31) not run code besides the inference (00:11:33) engine and the model it's like not like (00:11:35) people are actually like researchers are (00:11:37) like writing code for GPUs to see ideas (00:11:39) if they'll work and train models and all (00:11:40) these things or just mess around with (00:11:42) them to figure out you know infra (00:11:44) performance or whatever it is but most (00:11:45) of it won't be there and so CUDA as a (00:11:47) mode CUDA language is like you know like (00:11:49) it's like fine right like you know no (00:11:50) one actually writes CUDA right like most (00:11:52) people write PyTorch and then like torch (00:11:54) compile and then they just run it on the (00:11:55) GPU they don't write CUDA but a lot of (00:11:57) this CUDA mode is like how does PyTorch (00:11:59) translate into high performance GPUs and (00:12:01) that surface area from when people were (00:12:04) just writing like hardcore when people (00:12:05) are hardcore writing CUDA kernels to (00:12:07) like hey they're writing PyTorch and (00:12:09) then it's compiling down to GPUs versus (00:12:11) oh I'm just downloading VLM is it is a (00:12:14) it is a curve of like not a ton of (00:12:15) people that can do CUDA kernels a whole (00:12:17) lot more people can do PyTorch right (00:12:19) random you know PhDs and random people (00:12:21) it's very simple right a crapload of (00:12:23) people can do VLM download it run it on (00:12:26) a server well if it now supports other (00:12:28) chips what is the CUDA mode's recognized (00:12:30) this and they've been building software (00:12:32) that is not necessarily the CUDA remote (00:12:34) and I I can give some examples All (00:12:35) right. So the name of the game is fast (00:12:37) tokens and lowest cost tokens, right? (00:12:40) And lowest cost tokens happens by your (00:12:42) chip being fast. But there's also (00:12:43) tricks, right? One example, right? Like (00:12:45) I mentioned with, you know, the CPX (00:12:46) versus Grock, right? Is processing your (00:12:48) prefill contacts, right? Super cheap (00:12:51) CPX, right? If I'm if I'm care a lot (00:12:53) about speed, then Grock. These are (00:12:54) optimizations on the hardware side. (00:12:56) There's optimizations on the software (00:12:57) side as well, right? And so one example (00:12:59) is when I'm doing for example if I look (00:13:02) at a cloud code or a cursor type (00:13:05) application right the workload is like (00:13:08) it takes your repo takes the relevant (00:13:10) parts of your repo puts it in the (00:13:12) context of the LLM it prompts it (00:13:14) generates right and if it's an agent (00:13:15) mode it it it circulates the context a (00:13:18) couple times it'll collapse put things (00:13:19) off to the side access different (00:13:20) contexts but what's you know especially (00:13:22) when you think about an agent for (00:13:23) software and you can see this in codeex (00:13:25) you know Codex Codex actually not as (00:13:27) good as cloud code, but it can do work (00:13:29) on time horizons of like 9 10 hours. Um, (00:13:32) and do like a big refactor better than (00:13:34) cloud code can, even though most of the (00:13:35) times cloud code is better. And and (00:13:37) what's interesting about Codeex does is (00:13:39) it'll like take your repo, it'll (00:13:41) identify parts if you're asking it to (00:13:42) refactor it, identify parts, write (00:13:44) stuff, you know, make like these notes (00:13:46) for itself everywhere, collapse the (00:13:47) context, switch from this part of the (00:13:49) repo to that part of the repo to this (00:13:50) part of the repo. But when you think (00:13:51) about it, it's like, oh, if this thing (00:13:52) is just generating tokens all the time, (00:13:54) plus it's switching what my context is (00:13:57) constantly, that's really expensive, (00:14:00) right? If you think about like what's (00:14:01) the cost of inference, um, I want to say (00:14:04) it's like it's it's $10 per million (00:14:06) tokens of output and or and $3 for (00:14:10) decode or 10 for decode and three for (00:14:12) prefill. Uh, and so if you think about, (00:14:14) oh, it just worked for nine hours on one (00:14:16) task, one refactor, huge value. But if (00:14:18) it changed context a ton of times and (00:14:20) your context is like 30k usually or 50k (00:14:23) or you know heading to hundreds of (00:14:24) thousands you know how long your how big (00:14:26) your repository is and how much context (00:14:28) switch now you're spending all this (00:14:29) money on on prefill right not the decode (00:14:31) tokens but actually why am I like (00:14:33) regenerating the KV cache I can actually (00:14:36) just like store the KV cache elsewhere (00:14:38) and then when I need it again I can pull (00:14:39) it and and plop it into CPU memory or (00:14:41) into GPU memory. And so Nvidia's got (00:14:43) this like KV cache manager and they've (00:14:45) been working really hard on like making (00:14:47) it so they can interface SSDs and stick (00:14:50) the KV cache on there and pull it out (00:14:51) whenever they want. So for this kind of (00:14:53) workload and then if you do this and you (00:14:55) look at like coding as an application (00:14:57) and you like look at these coding (00:14:58) companies and how much they're paying (00:14:59) for prefill versus decode actually (00:15:01) majority of their cost is pre-fill (00:15:02) tokens not decode tokens because their (00:15:04) context is just so large and it's (00:15:06) switching all the time even in agent (00:15:08) modes. You know, if you can now not have (00:15:10) to do the pre-fill, your costs go down (00:15:11) dramatically. But that's a very (00:15:13) complicated thing to do from a software (00:15:15) perspective. You know, companies like (00:15:16) Enthropic, Google, OpenAI have already (00:15:18) done it. But what about the wide world, (00:15:20) right? And so Nvidia is trying to make (00:15:21) the open source software for this. And (00:15:23) that's like CUDA mode, but it's like (00:15:24) actually no, none of this is CUDA, (00:15:26) right? like it's like memory management (00:15:27) and like you know storage management and (00:15:30) when do you call what and how do you (00:15:31) transfer it and how do you like spread (00:15:32) the KV cache across a bunch of different (00:15:34) storage nodes and what happens when you (00:15:36) read it and the network congestion just (00:15:37) like all these things yeah it's like (00:15:39) Nvidia's wheelhouse but it's not CUDA (00:15:41) and I think like the easy way to say it (00:15:42) is it is the CUDA mode right and so (00:15:44) things like this KV cache manager and (00:15:46) many other things they're trying to do (00:15:48) to reduce the cost of inference like is (00:15:50) how they build the new CUDA mode because (00:15:52) again today it's it's you know it is (00:15:55) quite I mean AMD is like not fully there (00:15:57) yet and TPU is being added right now and (00:15:59) tranium is being added soon as well to (00:16:01) VLM but all of them will have a very (00:16:03) good UX for download model run model on (00:16:06) VLM by the middle of the year I think (00:16:09) right certainly AMD is already there by (00:16:11) the end of this quarter we have (00:16:12) something that like tests this right (00:16:13) it's called inferencemaxa it's open (00:16:15) source all the code is and the results (00:16:16) are uh but we run across I think $60 (00:16:19) million of GPUs which are donated to us (00:16:21) by companies like Nvidia AMD openai (00:16:24) Microsoft Amazon on Crusoe, Core Weave, (00:16:27) Together AI, uh all these companies are (00:16:30) sponsoring GPUs for us to run this. (00:16:31) We're running VLM and SDG Lang every (00:16:33) night on, you know, nine different kinds (00:16:35) of GPUs on a variety of different models (00:16:37) and different work uh context lens and (00:16:39) all these things, right? To see the (00:16:40) performance and you can see the (00:16:41) performance moving every day or pretty (00:16:42) often because the software changes all (00:16:44) the time. And so like the fact that this (00:16:46) exists is the cuda boat, right? It's not (00:16:48) that like AMD you can do this on their (00:16:50) chips, Nvidia can do this on their (00:16:51) chips. It's oh when the new model comes (00:16:52) out, how fast does it get to peak (00:16:54) performance? because you know it's it's (00:16:55) a moving target or hey can I implement (00:16:57) this KV cache management thing how hard (00:16:59) is it how many engineers do I need oh (00:17:01) just one great like or 10 great if I (00:17:03) need a hundred people to develop it like (00:17:05) Google and you know so on and so forth (00:17:06) did then that's much harder (00:17:08) >> do you think AMD can uh catch up (00:17:09) >> I think AMD will be caught up at times (00:17:12) and very behind at other times like (00:17:14) currently they're super far behind right (00:17:16) because Blackwell is just way better (00:17:17) than MI355 um and then you know Rubin (00:17:20) comes out and they'll be way way behind (00:17:21) but then AMD's new chip comes out and (00:17:22) AMD will be caught up or evenlight ly (00:17:24) ahead on a hardware perspective. (00:17:26) Software's behind, right? And you have (00:17:27) this like leaprogging and and AMD is a (00:17:29) very credible second competitor. I don't (00:17:31) think they'll go beyond like I think (00:17:33) they'll stay in single digits market (00:17:34) share. Single digit percentage market (00:17:36) share. (00:17:37) >> Single digit percentage market share is (00:17:38) >> still [laughter] pretty good. (00:17:40) >> Yeah. I mean, Nvidia's revenue this year (00:17:41) is going to be like (00:17:43) >> it's a lot. (00:17:44) >> The three gajillion (00:17:45) dollars. (00:17:46) >> I think it's actually four gajillion. (00:17:48) [laughter] (00:17:49) [gasps] (00:17:49) >> What about all the startups? You (00:17:51) mentioned a few. So there's a cerebrus (00:17:54) on the one end of the spectrum and then (00:17:56) newer ones edged and and others if if (00:17:59) AMD has a you know uphill battle in (00:18:02) front of them like do you think those (00:18:03) guys can take significant market share? (00:18:06) you sort of the whole specialization (00:18:07) game, right? You you have to specialize (00:18:09) because you're never going to beat (00:18:10) Nvidia at their own game, right? They're (00:18:12) going to have the supply chain unlock. (00:18:14) They're going to get to the newest (00:18:15) memory technology or process technology (00:18:17) or whatever packaging technology, (00:18:18) whatever it is, sooner than you and (00:18:20) they're just going to crush you, right? (00:18:21) If you play their game, you have to AMD (00:18:23) is trying to play Nvidia's game, but AMD (00:18:26) is like extremely good at engineering (00:18:28) silicon, right? Everyone else has to has (00:18:31) to has to try something weird or (00:18:33) different, right? And so when you look (00:18:34) at Etched or Maddx or Posatron or (00:18:37) Cerebrris or Tenstor, you go to look at (00:18:39) all these companies, right? There are (00:18:41) unique things about what they're doing (00:18:44) and it's not clear if AI models will (00:18:47) still be within that realm when that (00:18:49) comes out, right? Uh does oh now people (00:18:52) use like engrams and other sparse (00:18:54) attention techniques. Is that like is (00:18:56) does that change like some of the (00:18:58) specializations people are doing or hey (00:19:00) people are now doing like you know (00:19:02) models are now sparse instead of being (00:19:04) dense models does that change things (00:19:06) there's so many optimizations and (00:19:08) changes on the model side and you can't (00:19:10) predict what's going to happen with the (00:19:12) ML research easily at least you can't (00:19:14) the thing you're optimizing for today (00:19:16) has to be a vision of where AI will be (00:19:18) in 2 years and Nvidia's fully accepted (00:19:20) they don't know where that's going to be (00:19:21) that's why they have a portfolio of (00:19:24) chips now not just one GPU line, right? (00:19:26) It's not just Hopper, Blackwell, Reuben. (00:19:29) Now, it's going to be, you know, it's (00:19:30) not Ampure, Hopper, you know, you know, (00:19:31) it's not that line. It's like there's a (00:19:33) variety of chips to serve the different (00:19:34) markets um and different possible (00:19:36) scenarios. They think each of them has (00:19:38) this vision today, but oh, it might turn (00:19:39) out the general purpose one sucks and (00:19:41) and actually AI models have developed in (00:19:42) a way where CPX or Grock style chips are (00:19:45) the best, right? Well, okay, now we have (00:19:46) a solution for that market. And so, I (00:19:48) think that's the challenge with the (00:19:49) startups. With that said, I think (00:19:51) they're all taking very interesting (00:19:52) bets. I think it's I think it's much (00:19:54) more exciting than the first wave of AI (00:19:57) hardware uh bets graph course rebringing (00:20:05) the memory on the chip they sort of just (00:20:07) made a bet and they optimized for a (00:20:08) certain kind of model all similar kinds (00:20:10) of model and it didn't end up working (00:20:11) out for a long time right they had to (00:20:13) pivot and they had to work on a lot of (00:20:14) things and it took a long time I think (00:20:16) these companies have like a really clear (00:20:18) vision of what they think models will (00:20:20) look like right like Etch does Maddx (00:20:22) does, Posatron does, and that's what's (00:20:24) really cool about it between the three (00:20:25) of them, uh, these new age. So, I mean, (00:20:27) I'm I'm excited for them. I'm very very (00:20:29) skeptical. I don't know what uh what a (00:20:31) venture capitalist views as likely (00:20:33) chances of succeeding, but I think all (00:20:35) of them are less than 1%. Right? (00:20:38) >> But, you know, that's that's that's a (00:20:40) >> but the world where they win is a (00:20:42) multi-silicon kind of world where any (00:20:44) given customer uses a range of different (00:20:48) GPUs. It could it could or it could be (00:20:50) any given customer has like one workload (00:20:52) they care a lot about. Anthropic clearly (00:20:55) does not give a crap about videogen (00:20:56) image gen right they just don't care. Um (00:20:59) on the flip side, company like (00:21:00) midjourney cares a lot about image and (00:21:02) videogen, right? Image and videogen is (00:21:05) very very like like I mentioned like (00:21:07) it's a very like it's not very memory (00:21:08) bandwidth heavy. It loves loves loves (00:21:11) compute, right? Whereas inference of (00:21:13) large language models in the style of (00:21:15) like you know this these you know say (00:21:17) for example coding agents cares a lot (00:21:19) about decoding for long streams of time. (00:21:21) Um and that's very memory bandwidth (00:21:23) heavy right? And so there's like that's (00:21:24) like a simple example, but there's a lot (00:21:26) more nuance there in terms of like even (00:21:28) like the size of like the matrix (00:21:30) multiply, you know, the tensor cores (00:21:31) that you you know the systolic arrays (00:21:32) that you use or the ratios of networking (00:21:34) and memory memory and like what's that (00:21:36) memory hierarchy look like and you know (00:21:37) what are you doing for different kinds (00:21:38) of attention and like oh like all these (00:21:40) sorts of things like there's a lot of (00:21:42) specialization here and so some people (00:21:44) are betting big on on different types of (00:21:45) specialization and I I think like you (00:21:47) could clearly see a world where (00:21:49) companies do care about different stuff (00:21:51) right like like if for example a chip (00:21:53) optimized for video and image generation (00:21:56) existed today and it was better than (00:21:58) Nvidia or Nvidia made it. I think (00:22:00) Midjourney would absolutely only use (00:22:02) that for inference. I think for training (00:22:03) they'd still use the general purpose (00:22:04) thing and as would like Meta and Google (00:22:06) would like they should do that, right? (00:22:08) And hey, Meta actually has two lines of (00:22:10) AI chips there. MTIA there's a line (00:22:13) that's focused on recommendation systems (00:22:15) and then there's a line that's focused (00:22:16) on Gen AI. The GI one is a new line, but (00:22:19) that recommendation systems ch line is (00:22:20) still continuing, right? It's not sexy. (00:22:22) No one cares because there's no and bite (00:22:24) dance also has a recommendation system (00:22:26) line of chips and it's not really (00:22:27) focused on Jedi which is fine because (00:22:30) you know this is a $200 billion business (00:22:31) or something which is just deciding what (00:22:33) ad to serve me right and what order to (00:22:35) put my friends stories and you know (00:22:36) things like this so so I think like it's (00:22:38) perfectly fine for there to be (00:22:39) specialized AI chips given the target (00:22:41) market is big enough and you have to (00:22:42) have vision to know what that target (00:22:44) market is unless you're hyperscaler then (00:22:46) you can like just like you can just use (00:22:47) general purpose until you've like it's (00:22:49) clearly there and then you can make your (00:22:50) asich right (00:22:51) >> fascinating turning to the geopolitical (00:22:53) aspect of of uh all of this which is (00:22:56) always fun. Huawei and Nvidia in China (00:23:00) last year that was like 10 or 12% of (00:23:02) their overall revenue and this year they (00:23:04) they were saying that their market share (00:23:06) but has basically dropped to not very (00:23:08) much. Is that Huawei chips? Is that (00:23:10) restrictions? Is that tariffs? Uh what's (00:23:12) happening? (00:23:12) >> It's a variety of things actually in in (00:23:14) some in some quarters last year. uh it (00:23:16) was even north of 20 I think but I don't (00:23:18) remember exactly but anyways you know if (00:23:20) you look at 2022 China was almost the (00:23:23) size of the US in terms of buying server (00:23:24) hardware right almost not quite but (00:23:26) getting there um and it looked like they (00:23:28) were going to be the same size as (00:23:30) America in like a year or two after that (00:23:31) right and if you look at like global (00:23:32) data center capacity global cloud (00:23:35) capacity etc etc etc it's American (00:23:37) companies and Chinese companies right (00:23:38) that dominate the world American (00:23:39) companies obviously doing a lot better (00:23:40) here but both of those dominate the (00:23:42) world and if you look at like every (00:23:44) industry right you know it's It's it's (00:23:46) very clear that like China wants to (00:23:48) insource stuff, right? So in 2015, they (00:23:50) made these 5-year plans for two 2020 and (00:23:52) 2020 uh five where they set the (00:23:55) percentage of semiconductors they wanted (00:23:57) uh domestically produced and they've (00:23:59) missed the goal both times which is (00:24:01) fine, right? They set really aggressive (00:24:02) goals and even you know shoot for the uh (00:24:04) moon even if you miss you hit the stars, (00:24:06) right? And that's sort of what's (00:24:07) happened, right? Like look, China is not (00:24:09) caught up on, you know, leading edge (00:24:11) semiconductors, but microcontrollers (00:24:13) from China are almost as good as the (00:24:15) microcontrollers are as good and cheaper (00:24:17) than the ones from Texas Instruments or (00:24:19) ST Micro or, you know, etc., right? Or (00:24:21) like this power random power chip is (00:24:22) better than or the same as the one from (00:24:24) like another company, right? And so (00:24:25) they've really built up a semiconductor (00:24:27) industry and started insourcing a lot (00:24:28) more. I don't see why China wouldn't be (00:24:30) buying you know 30 40% of the world's AI (00:24:33) chips and the US like 50 60% and then (00:24:35) the rest of the world like you know and (00:24:37) when I say US I mean US origin companies (00:24:39) that seems like a more natural state for (00:24:41) the world but there are restrictions and (00:24:43) and hey this is the biggest change in (00:24:46) human history maybe ever knowledge work (00:24:48) and you know everything that's going to (00:24:50) happen there and and then eventually (00:24:51) like robotics and all these things like (00:24:53) you know obviously there's there's a lot (00:24:54) of geopolitical stuff and so there are (00:24:55) restrictions Nvidia's been handcapped (00:24:58) hand handicapped from selling their best (00:24:59) chips to China. And so that's obviously (00:25:01) impacted the sales a lot because like (00:25:03) why would you do that? And so when you (00:25:05) look at who rents the most GPUs in the (00:25:07) world, it's three companies, right? So (00:25:09) one of them is obviously OpenAI. Second (00:25:10) one, actually they were bigger than (00:25:12) OpenAI. They are bigger than OpenAI (00:25:13) today or no, they were bigger than (00:25:15) OpenAI than OpenI. Eclipsed them (00:25:16) recently is Bite Dance. Bite Dance runs (00:25:18) rents tons of chips from Oracle and (00:25:21) Google and and you know many other cloud (00:25:23) companies because they couldn't get the (00:25:25) chips they need in in China. They're (00:25:27) mostly just serving Tik Tok, right? (00:25:29) Okay. Well, they they're not allowed to (00:25:31) buy them and that sucks, but you know, (00:25:32) they're they're allowed to rent them. (00:25:33) And so, okay, if I'm not allowed to get (00:25:34) the best ones, I'm going to rent (00:25:35) externally. And if Bite Dance is the (00:25:37) second biggest renter of GPUs in the (00:25:38) world, that's substituting demand that (00:25:39) would have been built in China in many (00:25:41) cases. It's instead being built in (00:25:42) Malaysia. And Oracle has over a gigawatt (00:25:44) of capacity in Malaysia that Bite Dance (00:25:46) is going to take, right? So, things like (00:25:48) this are, you know, you know, hundreds (00:25:50) of thousands, if not millions of chips, (00:25:51) tens of billions of dollars of cap (00:25:53) capacity that would go to China, but (00:25:54) it's not. that it's going to Malaysia (00:25:55) instead as an example. Another sort of (00:25:57) point around this is China's like you (00:25:59) know they've had these 5-year plans. So (00:26:01) and and you know the way these (00:26:02) initiatives work from China is there is (00:26:04) like some top down ordering but then (00:26:06) they just kind of whip the whole like (00:26:07) everyone just kind of gets into it and (00:26:08) it's really cool like I don't think it's (00:26:10) as top down as many people think. Like I (00:26:12) think the entire country is like (00:26:13) semiconductor pill right there are (00:26:16) dramas where people fall in love in the (00:26:18) fab or dramas where people fall in love (00:26:22) and they're photovoltaic like solar cell (00:26:24) researchers and engineers and it's like (00:26:26) it's like this is just the backdrop and (00:26:28) it's like actually this is it's like (00:26:29) super cool for your like significant (00:26:32) other to be that semiconductor engineer (00:26:34) or to be that photovoltaic you know uh (00:26:37) solar panel researcher (00:26:38) >> as opposed to an influencer (00:26:40) >> as opposed to an influencer. Right. Like (00:26:41) I'm sorry. Love Island is I I I watched (00:26:44) like for 10 minutes cuz I was forced to. (00:26:45) I was like this is freaking terrible. (00:26:47) [laughter] Um but you know like um (00:26:50) >> we are so cooked. (00:26:51) >> No, you know [laughter] seriously we're (00:26:52) cooked. We're cooked. And I think I (00:26:53) think like when you think about like (00:26:55) this happens it's like it's diffused (00:26:56) into drama even people like like there's (00:26:59) multiple dramas like taking place about (00:27:01) semiconductor industry and and they're (00:27:03) like romance comedy like the entire (00:27:06) spectrum, right? Drama like it's like (00:27:08) it's like what the heck is going on? (00:27:09) Anyways, you have all these provinces, (00:27:11) you have all these local cities studying (00:27:14) out ordinances and giving out subsidies (00:27:17) and all sorts of stuff, right? It's (00:27:19) truly like crazy. Like there's some (00:27:21) national level stuff like, "Oh, no taxes (00:27:23) on uh this. Oh, we're going to ban a few (00:27:25) things." But as far as I understand, the (00:27:27) national government has not banned (00:27:29) Nvidia's H20 or H200. But the local ones (00:27:33) have, right? A lot of local ones have (00:27:35) said, "No, you know, you must use China (00:27:37) manufactured chips." And it's like, who (00:27:38) told you that, you know, you're here to (00:27:40) uphold this? It's like, does it matter, (00:27:41) right? I mean, like, it's it's it's cool (00:27:43) because then you have this like survival (00:27:44) of the fittest, all these all these (00:27:46) provinces and cities are trying to (00:27:48) attract different companies with (00:27:49) different types of subsidies and grants (00:27:52) and industrial parks and like all these (00:27:54) different things (00:27:55) >> and then like the ones who succeed (00:27:57) actually develop an industry and they (00:27:58) take over. (00:27:59) >> This how one thinks of of China, right? (00:28:01) It almost sounds like more like the US (00:28:02) or like with the federal government and (00:28:04) states where the provinces have (00:28:05) authority over their purchasing. It's (00:28:07) It's actually like uh great. There's (00:28:09) this one um Tik Tok or not Tik Tok, Tik (00:28:12) Tok and Instagram like uh person and (00:28:14) they're like they they like sing it. (00:28:15) They're like if you want to if you want (00:28:16) to buy things in China, make sure you go (00:28:18) to the right place. And then they just (00:28:19) say the most random [ __ ] and name the (00:28:21) city. And then you look into it and (00:28:22) you're like wow this city has the entire (00:28:24) supply chain for this. Um and it's like (00:28:26) lampshades and then it names the city. (00:28:27) It's like what the [ __ ] There's a city (00:28:29) that specializes in lampshades. Like (00:28:31) it's like and it's like microphone arms (00:28:33) like microphones. It's like it's like (00:28:34) literally there's a city in China that (00:28:36) specializes in (00:28:36) >> guitars as well, right? This one one (00:28:38) city that became the guitar capital of (00:28:39) the world. (00:28:39) >> It's literally everything. (00:28:41) >> Literally everything. There's a city and (00:28:43) it's not like hey specifically for uh (00:28:46) camera arms for example, there's ball (00:28:48) bearings in this and the ball bearings (00:28:49) are like there's ball bearings. There's (00:28:50) multiple manufacturers of ball bearings (00:28:52) for camera arms (00:28:53) >> and then like most of the camera arms in (00:28:55) the world come from that one city. It's (00:28:56) like what the hell is going on? Um and (00:28:58) and so like the semiconductor industry I (00:29:00) think people don't realize is absurdly (00:29:01) specialized. I'm not answering your (00:29:03) question. I'm just going a little bit of (00:29:04) a rant because I think people don't (00:29:05) understand China semiconductors. It's (00:29:07) really sick or semiconductors in (00:29:08) general. But like you know like in Japan (00:29:11) they like focus on a few different types (00:29:13) of chemicals and they're the best at it (00:29:15) and it's like almost a cultural thing, (00:29:16) right? Japanese people were so precise (00:29:18) like with sushi and like it's all about (00:29:20) the trade and the craft and like you (00:29:21) know the French food in Japan is better (00:29:23) than the French food in France because (00:29:24) the f the Japanese chefs went there and (00:29:26) then come back and they perfected it in (00:29:27) Japan and like cuz they're so precise (00:29:29) and and there's so many different like (00:29:30) things that like Japan is so good at (00:29:32) because they're so precise and like (00:29:34) dedicated to the craft and it comes out (00:29:35) of like I don't know like samurai (00:29:37) culture or something I don't know right (00:29:38) like I don't exactly know how that (00:29:40) culture came up and so when you look at (00:29:41) like and it's like across the world (00:29:43) there's different places where things (00:29:44) like this happen right like Oh, like the (00:29:46) Netherlands makes EUV tools. Cool. I (00:29:49) guess so. And you look across the (00:29:50) semiconductor industry. There's a famous (00:29:51) economic essay called I pencil or (00:29:54) something like that. Or talking about (00:29:55) how the pencil like a simple pencil (00:29:57) comes from like oh the rubber comes from (00:29:59) like Indonesia for the eraser and the (00:30:02) graphite comes from this mine here and (00:30:03) and the wood comes from these aspen (00:30:05) trees in Canada and like you actually (00:30:06) can't make a pencil without aggregating (00:30:08) this entire supply chain. semiconductor (00:30:10) industry is like way crazier because (00:30:11) like I would say there's like 15 or 20 (00:30:13) countries that could shut down the (00:30:14) entire semiconductor industry, right? (00:30:16) Even like Austria could, right? And and (00:30:17) it's like what? And it's like well yeah, (00:30:18) there's two different companies there (00:30:19) who have like 90% share in like some (00:30:22) random niche stuff. (00:30:23) >> And it's like okay, cool. I guess (00:30:25) Austria can and oh yeah, those two (00:30:27) companies only like have less than a (00:30:28) billion of revenue, but they just happen (00:30:30) to have lynchpin critical things. And (00:30:32) there's lynch pin critical things (00:30:33) everywhere because the process is so (00:30:34) complicated. And so China's been trying (00:30:36) to replicate this. Um, (00:30:37) >> is there one thing they're missing that (00:30:38) they don't have yet? (00:30:39) >> I think there's a lot of things. I think (00:30:41) if you were to close your eyes and say (00:30:42) or if you were to cut off every country (00:30:43) and say there's no more globalism, China (00:30:45) has the most vertical stack in (00:30:47) semiconductors today and they're the (00:30:48) best at semiconductors in the world (00:30:50) because their fabs could still run (00:30:51) somewhat on a lot of things because they (00:30:53) have built some of these chemical supply (00:30:54) chains, right? Like TSMC for certain (00:30:57) kinds of chemicals 100% share from (00:30:58) Japan, right? Or Intel same thing, (00:31:00) right? or you know for certain kinds of (00:31:02) tools 100% share from Netherlands or (00:31:03) 100% share from you know this American (00:31:05) company or that you know Austrian (00:31:06) company or this or that right like (00:31:07) there's just all these like you know (00:31:09) this Swiss company like there's just all (00:31:10) these different places have 100% share (00:31:12) it might be one company might be three (00:31:14) companies but geographically or in the (00:31:16) same area and China's built that up (00:31:18) right because they've created this made (00:31:19) in China initiatives which just plowed (00:31:22) money into it and they've got this (00:31:23) culture of like the diffused like you (00:31:25) know these provinces like yeah I just (00:31:27) decided I'm going to [ __ ] focus on or (00:31:29) might not even be might not even be the (00:31:30) Right. It may be the like, you know, (00:31:32) someone brought it there and decided and (00:31:34) then people were like, "Oh, wow. You're (00:31:35) doing that?" Me, too. Like, I'm a Patel (00:31:37) and I grew up in a motel and guess what? (00:31:38) We like almost all the Patels I know (00:31:40) grew up in a motel and it's because some (00:31:43) random Patel immigrated to America and (00:31:46) like worked at a hotel motel and then (00:31:48) bought a motel and then like it just (00:31:49) started happening, right? Like you sort (00:31:50) of like these things are serendipitous (00:31:52) of sorts and like I don't know like and (00:31:53) it's like I I view it as the same kind (00:31:55) of specialization, right? Chinese cities (00:31:57) are like starting to do the these (00:31:58) things. China's missing a lot of things, (00:32:00) right? I would say like if you say minus (00:32:01) 10 years tech, China's complete and no (00:32:04) one else is complete, right? Taiwan is (00:32:06) not complete. Their the fabs would shut (00:32:07) down without foreign supply, you know, (00:32:09) and you go down or you go across the (00:32:10) stack. Uh but if you go to 10ear tech, (00:32:12) maybe maybe more like 20-year tech, you (00:32:14) could get a fully vertical supply chain (00:32:15) in China, which I do not think any (00:32:17) country could do. Like America could not (00:32:19) build a fully vertical fab without stuff (00:32:20) from elsewhere, even if it's 20-y old (00:32:22) tech. (00:32:23) >> Um probably not even 40-y old tech. And (00:32:25) so, so that's interesting. But then when (00:32:27) the flip side is like well like you kind (00:32:28) of do need specialization. That's how (00:32:31) that chemical gets the purest best, you (00:32:33) know, most engineered, you know, or that (00:32:35) that slurry of chemicals or that, you (00:32:38) know, that gas or like that tool because (00:32:40) every smart person or a lot of them in (00:32:43) that country grew up around that culture (00:32:45) and like every the supply chain is there (00:32:47) and like everyone kind of knows and like (00:32:48) it's like a a driveaway and like sort of (00:32:51) like this is what makes supply chains (00:32:53) work is that there is this (00:32:54) specialization and the best of the best (00:32:56) only comes when you have that hyper (00:32:57) specialization. So, China doesn't have (00:32:58) lithography. their lithography is like (00:33:00) 10 years behind and I think it'll be 5 (00:33:02) years behind in a couple years, right? (00:33:03) They're catching up fast. I don't think (00:33:05) they'll be as good as ASML for a long (00:33:08) time. You know, maybe I don't know, (00:33:10) maybe they will be, you know, China. You (00:33:11) shouldn't ever underestimate China, but (00:33:13) like and Chinese engineers or, you know, (00:33:14) but like for a while, right? Or like, (00:33:17) you know, I don't think they'll be able (00:33:18) to make leading edge chemicals like many (00:33:20) chi uh Japanese companies or many (00:33:21) American companies and their tools and (00:33:23) like you just go across the supply (00:33:24) chain. They're not hey forefront on (00:33:28) really anything in the manufacturing (00:33:30) supply chain on the design supply chain. (00:33:32) There's some things that they're (00:33:34) starting to be similar par but like (00:33:36) cheaper or like a year or two behind but (00:33:39) cheaper and that's like fine for a lot (00:33:40) of stuff. An example of that is Huawei, (00:33:42) right? Huawei in mobile phones was on (00:33:45) par with Apple like entirely. Yeah. And (00:33:47) they had become Apple TSMC's biggest (00:33:49) customer and they were designing the (00:33:50) best thing and they are number one in (00:33:52) telecom and their tech is just literally (00:33:53) better. And so when you think what (00:33:56) happens, you know, is is is China (00:33:59) missing anything? It's like they're they (00:34:01) don't they don't they don't have the (00:34:02) best of much, you know, today in the AI (00:34:05) supply chain. they have a complete (00:34:06) package and a couple years behind and (00:34:08) they'll figure out how to make it (00:34:09) cheaper slash do more slashcatch up and (00:34:11) and create a robust industry. But (00:34:14) there's a reason like I don't think that (00:34:16) like Jensen is scared of AMD really. (00:34:18) He's paranoid. I mentioned he's (00:34:19) paranoid. I'm sure he's a little bit (00:34:20) scared of them, right? Like I think some (00:34:22) of the things that they've done are (00:34:23) reactions and competitive dynamics with (00:34:25) AMD or Google's TPUs or whatever. Right. (00:34:27) There was a Core Weave deal today and I (00:34:29) think that's directly the result of what (00:34:30) Google's been doing. (00:34:31) >> Yeah. the two billion pipe that Nvidia (00:34:34) announced into (00:34:35) >> Nvidia invested two billion in core but (00:34:37) what's more important is that that's (00:34:38) like sort of just like the sticker (00:34:39) what's really relevant is Nvidia is (00:34:41) going to work with core reef to uh (00:34:43) acquire um and and back stop and all (00:34:46) these things the the land the power the (00:34:49) energy the transmission that help build (00:34:51) the data center all this capital side (00:34:53) stuff that because Nvidia has so much (00:34:54) money they can backs stop corore weave (00:34:56) doing it because corore reweave then can (00:34:58) be the one who generates demand anyways (00:35:00) there's like because Google was doing (00:35:01) And they did that with like a couple (00:35:02) companies such as Fluid Stack and (00:35:04) Terowolf and Cipher. These are some (00:35:06) public deals that have been announced. (00:35:07) And so Google is doing that with TPUs (00:35:08) and Nvidia reacted, right? Um, and so in (00:35:11) the same way, I think Nvidia's reacted (00:35:12) to AMD. And in the same way, I think the (00:35:15) thing is Nvidia is like deathly (00:35:16) terrified of Huawei (00:35:18) >> because Huawei has caught up to Apple (00:35:20) and actually surpassed them as TSMC's (00:35:22) biggest customer before they got banned, (00:35:24) right? They did just crush Nokia, Sony, (00:35:26) Sony, Ericson, etc., right? Like the (00:35:28) entire telecom supply chain. they just (00:35:30) like completely destroyed them. And (00:35:32) there's so many other areas like they (00:35:33) straight up made a folding phone, right? (00:35:35) You know, I have a Samsung folding (00:35:37) phone. They have a folding phone that's (00:35:38) better than Samsung's folding phone. (00:35:40) >> And it's like, bro, what? Like, you (00:35:42) know, you know, Huawei's really really (00:35:44) cracked. And so, of course, they're (00:35:46) terrified of uh and and Huawei is the (00:35:48) most vertical company in the world. No (00:35:49) company is more verticalized than (00:35:51) Huawei, which then leads to huge (00:35:53) innovations. It's something that we (00:35:55) don't fully appreciate in the US, but (00:35:56) like when you travel in Europe, you see (00:35:57) everybody who's like honors honor phones (00:36:00) and it's like the the footprint of (00:36:01) Huawei is huge in in phones in a way (00:36:03) that people (00:36:05) >> not just phones um you know security (00:36:07) cameras actually they think they have (00:36:08) like you know (00:36:09) >> a lot of training on the [laughter] (00:36:12) that a captive group of testers. (00:36:15) >> Exactly. Exactly. Um I think I think (00:36:17) Huawei is terrifying right and and so (00:36:19) like yes their chips are not as good (00:36:20) today (00:36:20) >> and is that is is that already (00:36:22) happening? I mean obviously the US and (00:36:24) China are the two biggest markets but (00:36:25) like for other markets I don't know UAE, (00:36:27) Middle East, Europe are Nvidia and (00:36:30) Huawei already uh headto-head in (00:36:32) >> well they shipped a little bit but like (00:36:35) mostly just like sticker capacity like (00:36:37) there's nothing like no no like I would (00:36:38) say like a little bit as in like a few (00:36:39) servers not like a billion dollars worth (00:36:41) of stuff right the thing is China's (00:36:44) supply chain has to ramp up right um (00:36:46) China China's express goal is to have (00:36:48) all inter internalized but then like a (00:36:50) company like Alibaba's like I I don't (00:36:52) want to use Huawei, right? Like I want (00:36:54) to make I want to use Nvidia and just (00:36:56) make the best freaking models, right? (00:36:57) Because that's my business. My business (00:36:58) is not, you know, using a Huawei thing, (00:37:00) but it's like, okay, it's being pushed (00:37:01) upon me. There's other companies too, (00:37:02) like Cameron Con and so on and so forth. (00:37:04) And so the sort of like supply chain, (00:37:06) you know, companies in China don't want (00:37:07) to use they're kind of encouraged (00:37:09) obviously and pushed, you know, you must (00:37:11) some local provincial government be (00:37:13) like, well, you're doing this much (00:37:14) business here. You got to do this, (00:37:15) right? Like there's all sorts of like (00:37:16) crazy stuff that, you know, pushing of (00:37:18) of companies to use Huawei. Um the (00:37:20) challenge is probably can't manufacture (00:37:22) enough, right? We've like done a lot of (00:37:24) work on this. Um and we've just put it (00:37:26) for free, you know, instead of like to (00:37:28) our customers because it's like (00:37:29) something that's like national security, (00:37:30) which is how was Huawei actually (00:37:32) building chips? Well, actually they were (00:37:34) uh using shell companies to get chips (00:37:36) from TSMC and using different methods of (00:37:38) like sneaking HBM, which is memory, from (00:37:41) you know, Korea through Taiwan to China, (00:37:43) right? Like all sorts of crazy stuff (00:37:44) we've reported on and and people it's (00:37:46) like a whack-a-ole, right? they shut it (00:37:48) down or like tools that get shipped to (00:37:50) China and they shouldn't be for you know (00:37:51) making leading edge chips but they (00:37:53) actually are um and all these sorts of (00:37:55) things are happening because they can't (00:37:56) make everything and if they want to make (00:37:58) the leading edge stuff they do need to (00:37:59) rely on the foreign supply chain quite a (00:38:00) bit in terms of the upstream supply (00:38:02) chain right uh memory logic chips uh (00:38:05) tools for fabs chemicals for fabs etc (00:38:07) Huawei cannot satisfy the market um (00:38:09) because there's not enough advanced (00:38:11) leading edge capacity in memory logic (00:38:14) you know and all all these other things (00:38:16) uh domestically in and they're trying to (00:38:18) build it as fast as they can, but that (00:38:19) means there's just not enough to satisfy (00:38:20) the market. So, Nvidia has a market. I (00:38:22) think they'll figure out how to sell (00:38:23) chips to China. And Jensen's in China, I (00:38:25) think, like right now or was yesterday. (00:38:26) And so, like he's clearly like wheeling (00:38:28) and dealing to try and get his chips (00:38:30) into China because, you know, I think (00:38:32) Nvidia's argument is if we sell them (00:38:34) chips, then they won't, you know, there (00:38:35) won't be enough of as much of a domestic (00:38:37) market. The feedback loop for software (00:38:38) and everything else won't be there. That (00:38:40) will sort of like really challenge it, (00:38:42) right? Like most of the open source (00:38:44) software for AI has a lot of Chinese (00:38:47) contributors, right? VLM and PyTorch, SG (00:38:51) Lang and like all of these other like (00:38:54) libraries and things that are just like (00:38:55) you know and and and it goes to (00:38:56) low-level software especially right like (00:38:58) a lot of the best open source stuff is (00:39:00) actually just from like a Chinese (00:39:01) company who decided to open source it (00:39:02) and same with models right and so like (00:39:04) it's like okay well if they can't use (00:39:05) Nvidia chips anymore then this open (00:39:07) source stuff won't be designed for (00:39:08) Nvidia chips it'll be designed for (00:39:09) Huawei chips and now does that like (00:39:10) weaken the CUDA mode and now like not (00:39:12) only is China domestic now they have (00:39:14) like a feedback loop internally and then (00:39:16) they can externalize across the rest of (00:39:17) the world right so This is the like (00:39:18) argument Nvidia makes. I'm not sure if I (00:39:21) am like I'm like you know I think I (00:39:23) think my AI timelines are so fast. I'm (00:39:26) not that fast like not in terms of like (00:39:27) AGI but like hey AI is hundred billion (00:39:29) dollars of revenue uh across the (00:39:32) industry. I think the industry could hit (00:39:34) 100 billion ARR by the end of this year (00:39:36) like 4550 for open AI like 3540 for (00:39:40) anthropic and then you know vertex deep (00:39:42) minds uh models at Google Gemini right (00:39:45) um and then vertex API for anthropic (00:39:47) models and uh bedrock APIs and Azure (00:39:51) foundry APIs like I think hundred (00:39:53) billion dollars like end of this year (00:39:54) >> that's a lot and then what's the (00:39:56) economic value of that hundred billion (00:39:57) dollars now how much of that is in China (00:39:59) right like China's number is probably (00:40:00) 10x lower right? Because they just (00:40:04) haven't been able to pervasively push (00:40:05) AI, right? Chat GPT has a billion users (00:40:08) roughly and you know, then you add on (00:40:10) Gemini and Meta claims they have 500 (00:40:12) million users. I don't know. I think (00:40:13) people just accidentally click like (00:40:14) generative sticker or something. Um, (00:40:16) [laughter] but like anyways, like (00:40:18) there's like there's like a lot of usage (00:40:19) of AI in the west already and it's going (00:40:22) to climb. It's going to keep climbing (00:40:23) and like you kind of have to get used to (00:40:24) it and so like the question is like do (00:40:26) you you know what's what's the economic (00:40:28) benefit to the world, right? And at the (00:40:30) end of the day, this is an economic war, (00:40:31) right? If the US and the West win in AI (00:40:35) and control, you know, more powerful AI (00:40:37) systems that have this feedback loop (00:40:39) that improved economic growth and (00:40:41) weapons systems and whatever else, (00:40:42) right? Engineering of grids and cyber (00:40:45) attacks and all these sorts of things. (00:40:46) They have this like advantage over (00:40:48) China, then China will not rise to be (00:40:50) the global hedgeimony. But without AI, (00:40:52) China definitely will rise to be the (00:40:54) global hedgeimony. They're just going to (00:40:56) outrun America. And so the question is (00:40:57) like you know that's I think like the (00:40:58) other view right and how fast are super (00:41:00) powerful AI systems versus you know (00:41:02) China building a domestic ecosystem for (00:41:05) chips and models and everything that is (00:41:06) a few years behind like what's what's (00:41:09) actually the value right like that's (00:41:10) sort of like (00:41:12) >> around restrictions and regulations (00:41:13) >> where where do the uh US onshoring (00:41:16) efforts fall in that category what do (00:41:18) you make of them from the chipsack to (00:41:20) like all the thing that is being built (00:41:22) everything looks like it's massively (00:41:23) delayed by the way which perhaps is not (00:41:25) surprising (00:41:26) >> I think TSM MC's manufacturing wafers (00:41:27) and they're like building real wafers (00:41:29) and there's real fabs and like you know (00:41:31) there's some other fabs that have been (00:41:32) announced and like they're doing well (00:41:33) and there's like a bunch of like (00:41:34) different kinds of plants like a Korean (00:41:35) company making a random gas plant in (00:41:37) Texas for you know their chips right (00:41:39) like uh for chips and all these like (00:41:40) sort of things are happening. Um I think (00:41:42) the chips act did really well with its (00:41:44) $50 billion. It's just I don't think (00:41:46) people understand the scale of the (00:41:48) semiconductor industry. is the most (00:41:50) complicated supply chain in the world, (00:41:52) right? It's much bigger than, you know, (00:41:55) say manufacturing airplanes. It's much (00:41:57) bigger than like, you know, really (00:41:58) anything else, right? If you look at the (00:42:00) top 10 companies like of the world, I (00:42:02) think eight of them designed (00:42:03) semiconductors, right? Now, obviously (00:42:05) like Google designed semiconductors, but (00:42:06) it's like, oh wait, no, but their cost (00:42:08) of search would be like 10x higher if (00:42:10) they didn't have TPUs and TPUs were (00:42:11) super optimized for search, right? Or (00:42:13) like, you know, you you you go down the (00:42:15) list, right? Like Meta serves (00:42:16) recommendation systems with their chips, (00:42:18) right? Like you go down the list, it's (00:42:19) everyone is making their own chips. (00:42:21) Apple devices would be materially worse (00:42:23) if they didn't have their own chips, (00:42:24) right? Um and you just go down the list, (00:42:26) it's like it's the most complicated (00:42:28) supply chain and they they're spending (00:42:29) something on the order of like $150 (00:42:32) billion roughly in subsidies a year to (00:42:34) the chip industry. (00:42:36) >> We are doing 50 over like a decade. (00:42:38) >> Yeah, (00:42:39) >> there's a difference in scale here, (00:42:40) right? The collective total amount of (00:42:42) like capex that has been spent in Taiwan (00:42:45) is like 500 billion plus, right? across (00:42:48) the industry, across all the companies (00:42:49) that are making semiconductors in Taiwan (00:42:51) and Taiwan doesn't have a domestic (00:42:52) industry. How is $50 billion of (00:42:54) subsidies going to change America's (00:42:56) needle? Right? It does move it a little (00:42:58) bit, right? I I want to be clear like (00:43:00) the chips act is awesome. I don't (00:43:02) understand why like EVs or like solar (00:43:05) was given this massive massive like (00:43:08) trillion dollar package. Semiconductors (00:43:10) were only given 50. Like semiconductors (00:43:12) need a lot bigger package to actually (00:43:13) incentivize onoring. I think what's (00:43:15) happened so far has proven that it's (00:43:17) working well. TSMC is literally making (00:43:19) chips for Nvidia and Apple and AMD and (00:43:22) others in Arizona today, right? And I (00:43:24) think that's really great. (00:43:26) >> Is is your sense that the broad American (00:43:28) government is just uh aware of of all of (00:43:31) this that it's uh I wouldn't say only (00:43:33) passed because the automotive like (00:43:35) prices went up because car manufacturers (00:43:37) are like the worst because they do just (00:43:39) in time inventory, right? Or not worse, (00:43:40) but like this is just like a thing, (00:43:41) right? Just in time inventory systems. (00:43:44) COVID happens, sales plummet fabs that (00:43:46) were making, you know, random power IC's (00:43:48) or random microcontrollers for engines (00:43:49) got repurposed to the boom from COVID, (00:43:52) which is which was data centers and PCs (00:43:54) and smartphones. So, that stuff was (00:43:55) booming. And then when people were like, (00:43:56) "Oh, wait. Actually, like, you know, I (00:43:58) have some money. I stayed at home. I (00:43:59) didn't go out. I didn't drink. I have a (00:44:00) lot of I have some cash, right? Let me (00:44:02) buy a car." They went out and bought (00:44:03) cars and cars started skyrocketing in (00:44:04) prices. Oh, let's restart and let's (00:44:06) let's Oh, yeah. Can I can you sell me (00:44:07) that microcontroller for the engine (00:44:08) again? It's like, "No, I I'm making a (00:44:11) slightly different microcontroller that (00:44:12) works for, you know, uh, let's say a (00:44:15) keyboard or a mouse, right, or (00:44:17) whatever." And it's like, and and and (00:44:18) they actually didn't just leave me (00:44:21) flatfooted and they were like a partner (00:44:22) through co, right? You know, versus you (00:44:24) just left me. Screw you Ford or whoever, (00:44:27) Toyota, um, or automotive OEM, you know, (00:44:29) you up that supply chain. And so, Chips (00:44:31) Act did not get passed, only got passed (00:44:33) because that happened. And people are (00:44:35) like, "Oh my god, the semiconductors are (00:44:36) why cars can't be made." If that didn't (00:44:38) happen, we wouldn't even have the chips (00:44:40) act. It's like it's like silly. So like (00:44:41) I don't know like I think you know (00:44:43) whereas like and and even though that's (00:44:44) what was pitched to all the senators (00:44:46) like I know people who were running (00:44:47) around Capitol Hill just pushing that (00:44:49) narrative and story and that's why it (00:44:50) finally got passed in reality it was all (00:44:52) for advanced leading edge chips, right? (00:44:54) Nothing that goes in a car, right? And (00:44:56) so it's like this like funny thing. So, (00:44:57) in other words, do you think my words my (00:44:59) words not yours, but is it is it (00:45:01) hopeless that the US is going to I'm (00:45:03) very optimistic. Okay. I mean, do you (00:45:04) think there's a world where the US just (00:45:06) decides to invest in semiconductor at (00:45:09) the scale that (00:45:10) >> you know, I thought we just needed a (00:45:11) bigger chips act, but (00:45:14) >> look, Trump's kind of gotten TSMC to (00:45:16) promise to invest a fuckload more (00:45:18) [laughter] and they're moving on it, (00:45:20) right? They're like actually like just (00:45:22) building it. It's like, I'm going to (00:45:23) tariff the [ __ ] out of you unless you (00:45:24) build a fab. But it's like we'll build a (00:45:26) fab [laughter] and they're building it (00:45:28) right now. The timelines for fabs just (00:45:30) takes forever cuz again it's the most (00:45:31) complicated thing in the world. The (00:45:33) cleanest space in the place in the world (00:45:34) is not like a hospital or a biotech lab (00:45:36) or whatever. It's a semiconductor fab. (00:45:37) And the most expensive tools in the (00:45:38) world are not you know any of these (00:45:40) medical tools or whatever. It's it's (00:45:41) semiconductor tools or it's not a (00:45:43) rocket. It's a semiconductor tool, (00:45:44) right? Like everything you know I (00:45:46) describe it as um I remember when I was (00:45:47) a kid I was like I want to be a rocket (00:45:49) scientist and then I was like oh I want (00:45:50) to be a surgeon. And I'm like wait chips (00:45:52) are like rocket surgery but even cooler (00:45:54) right? Like I think anyways like sort of (00:45:56) like there there are fabs being built in (00:45:58) America. (00:45:59) >> They won't take America to (00:46:00) self-sufficiency. I don't think that's a (00:46:01) relevant. I don't think that's a goal (00:46:03) relevant like that's relevant, right? (00:46:04) Like globalism is generally just good. (00:46:07) Hot take [laughter] (00:46:10) like in terms of economics. (00:46:11) >> We'll turn this into a short a YouTube (00:46:12) short. (00:46:13) >> Globalism. (00:46:13) >> Globalism is good. [laughter] (00:46:14) >> Dude, you're gonna get me like (00:46:15) cancelled. (00:46:18) >> [gasps] (00:46:19) >> I tweeted about ice and it was a (00:46:20) complete joke, but so many people got (00:46:22) mad at me because I can't be, you know, (00:46:23) I'm too I'm too much of a joker. You (00:46:25) know, these are serious things. (00:46:26) >> Yeah. Yeah. No, I know the I know the (00:46:28) feeling. Yes. [laughter] (00:46:29) >> Anyways, um I think I think you know I (00:46:33) think we are building fabs and I think (00:46:34) it's like going to move and now even (00:46:36) Elon's talking about building fabs now (00:46:37) because he sees the shortages in the (00:46:38) world, right? Uh there's a lot of (00:46:40) semiconductor related shortages for (00:46:41) building out AI and and so I don't think (00:46:44) it's hopeless. I think I'm like very (00:46:45) optimistic that we're going to do more (00:46:46) and more and more. And maybe this (00:46:48) administration threatens tariffs and (00:46:49) they get the deals and the next (00:46:50) administration comes back with the (00:46:51) carrot. If it is the Democrats, whatever (00:46:53) happens, I don't know. Like I was at a (00:46:55) comedy club on Sunday night and like (00:46:57) he's like, "Oh, I'm I use Chad GPT." And (00:46:59) then like there were a couple people who (00:47:00) booed and he's like, "Yeah, I'm one of (00:47:01) those guys. I know." And like it's like, (00:47:03) "Wow, people hate AI." (00:47:05) >> And that has has not even started, (00:47:06) right? Like the actual impact of AI (00:47:08) >> or like New Jersey power prices are up, (00:47:10) right? Uh is it because of a data (00:47:12) center? New Jersey, the governor's (00:47:14) election like I think literally fl like (00:47:16) there's like an election that changed (00:47:17) recently in New Jersey because power (00:47:19) prices were up and people blamed a (00:47:22) Microsoft Nebius data center in New (00:47:24) Jersey for that reason. But in reality (00:47:27) that data center has nothing to do with (00:47:29) power prices going up. It's super storm (00:47:31) standy like five years ago knocking or (00:47:34) whatever how many years ago knocking (00:47:35) down the state's electrical (00:47:36) infrastructure and then the then (00:47:37) improving all these improvements and (00:47:39) then those improvements have to be paid (00:47:41) by someone and it turns out the consumer (00:47:42) has to pay for them with higher power (00:47:44) prices, right? And so like you know like (00:47:45) there's like there's a lot like going on (00:47:47) in that regard, right? Um that kind of (00:47:50) is uh (00:47:51) >> sad. Um, and and people hate AI and (00:47:54) they're blaming AI on it and artists (00:47:55) hate AI and like you know you see all (00:47:57) this deep fake stuff and like I think I (00:47:59) think it'll be the hottest button issue (00:48:00) especially as like we're really getting (00:48:02) into like I think last year Google spent (00:48:04) $3 billion on Whimo and we're waiting (00:48:06) for their guide for this year $3 billion (00:48:08) on Whimo taxis but their t their Whimos (00:48:11) went from like 300k to like 100k or 90k (00:48:14) the new Whimo car and they're going to (00:48:16) spend more than three because they've (00:48:17) just launched in like four cities now (00:48:19) right or five cities and and they're (00:48:20) testing it a lot and the same a robo (00:48:22) taxi like people are going to hate AI (00:48:23) for that reason people are going to hate (00:48:24) AI because the slop on the internet (00:48:26) people are going to hate AI because you (00:48:27) know the perceived job replacement (00:48:29) people are going to hate AI for all (00:48:30) these reasons and so yeah it's going to (00:48:32) be a hot button political issue don't (00:48:33) you think (00:48:33) >> yeah talking about that so um capex is (00:48:37) there a capex bubble are we u investing (00:48:41) too much or actually are we investing (00:48:44) not enough given what you were saying (00:48:46) earlier about uh the the the rate of (00:48:49) revenue increase and and therefore (00:48:50) implied demand (00:48:51) that you expect for this year. (00:48:53) >> I'm obviously a maxi. I think we're (00:48:55) going to need a lot of infra and I think (00:48:57) I'm literally paid to like analyze the (00:48:59) supply chain and do consulting. Like (00:49:01) that's what my company does. So like (00:49:03) obviously I'm very [laughter] biased. (00:49:05) I think I think we're pretty good at (00:49:07) calling when when things go down though, (00:49:08) right? Before like a part of the supply (00:49:10) chain reb. Anyways, you know, again, (00:49:12) going back to the economics of it, it's (00:49:13) north of hundred billion dollars of (00:49:14) revenue exiting this year for AI from a (00:49:16) base of, you know, sub1 billion gen AI (00:49:20) from a base because ads and stuff is (00:49:22) like already a multiundred billion (00:49:23) dollar AI industry, right? You know, go (00:49:25) back to 2023 it was like less than a (00:49:26) billion, right? And 2024, I don't know (00:49:28) exactly what number, maybe let's call it (00:49:31) 10 and 25 was maybe like 30 40. It'll be (00:49:34) north of 100 easily. If you're talking (00:49:36) about hundred billion of revenue, let's (00:49:39) say at a 50% gross margin. So that's $50 (00:49:42) billion of gross profit um and $50 (00:49:45) billion of COGS. That $50 billion of (00:49:46) COGS needs to run on infra, which cost (00:49:49) roughly if a five, if you're talking (00:49:51) about fiveyear depreciation, call it (00:49:53) $250 billion, right, of infra (00:49:55) >> for hundred billion of revenue. (00:49:57) >> Mhm. (00:49:58) >> Okay. What is what is the actual spend (00:49:59) on AI infra this year? It's going to be (00:50:00) like it's I mean it depends on what (00:50:02) layer. If you're talking about energy, (00:50:04) those are longer lived assets and all (00:50:05) these other things, right? Um data (00:50:06) centers are longer lived assets. The (00:50:08) chips are not as much. People are (00:50:09) putting capex down. Um and the (00:50:11) hyperscalers capex is going to be like (00:50:13) $500 billion this year or something like (00:50:14) this. And then besides them, there's (00:50:16) also a lot more hyp uh capex elsewhere. (00:50:19) Um and so, you know, is it a bubble? I (00:50:21) mean theoretically like you know it's (00:50:23) twice as much as it should be but it's (00:50:24) also like well no there's an R&D (00:50:26) component to this and the excess spent (00:50:30) that wasn't revenue generating last year (00:50:31) is what led to models being so good this (00:50:34) year um and led to like everyone who can (00:50:37) using cloud code and like that changing (00:50:39) their life. This is like it's not a (00:50:41) bubble, right? I don't think it's a (00:50:42) bubble yet. Um I think if AI model (00:50:44) progress stops and that's the main (00:50:45) thing, right? The moment model progress (00:50:47) stops all the spending is for not. But (00:50:49) so far we've had consistent improvement. (00:50:52) As you put in more compute, you get more (00:50:54) performance and better models. (00:50:56) >> Yeah. Model performance being the (00:50:57) lagging indicator of hardware progress (00:51:00) or data center. (00:51:01) >> Yeah. of of capex, right? Yeah. (00:51:03) >> Ultimately, the capex that Microsoft (00:51:05) spent in 2024 for OpenAI is what results (00:51:08) in in 2025 for OpenAI Cory or whoever is (00:51:10) what results in their models being so (00:51:12) good this year. Same with Enthropic and (00:51:14) Amazon Google and their models now being (00:51:16) so good now is is that capex and (00:51:18) actually they still haven't paid for (00:51:19) those chips yet because those chips are (00:51:21) still have a useful life for another few (00:51:22) years right I think model progress is (00:51:24) very clear um the moment that stops (00:51:27) happening right if we hit a wall there's (00:51:29) no new research directions um then then (00:51:32) it's cooked yeah right (00:51:34) >> and that assumes that better model leads (00:51:36) to more demand which is a reasonable (00:51:39) assumption (00:51:40) >> yeah for sure (00:51:41) >> but um yeah I mean there scale the (00:51:43) adoption curve regardless of how good (00:51:45) the model is uh in the enterprise (00:51:47) >> like 2% of GitHub commits today are (00:51:48) cloud code (00:51:50) >> as in committed by cloud code you can (00:51:51) disable that where it's not (00:51:52) automatically committed but 2% of GitHub (00:51:54) commits today are cloud code $2 trillion (00:51:56) of software wages paid in the world (00:51:58) >> if it was 2% then you like you're like (00:52:01) wait a second (00:52:02) >> this is this is an insane amount um AI (00:52:05) is under earning the value that it's (00:52:07) producing in the world right by a (00:52:09) significant margin already today (00:52:11) >> Bor's journey from Cloud code who had (00:52:13) who we had on the pod was saying that (00:52:16) what he's written all of Claude what is (00:52:19) it called co-work like the new product (00:52:20) entirely with cloud code yet so we're (00:52:23) very much in that world. Yes. (00:52:24) >> Yeah. My uh one of my roommates I was (00:52:26) asking him because he's like always been (00:52:28) a really low-level good programmer and (00:52:30) he started you know I was like he's like (00:52:32) he had this um holiday obsession right I (00:52:36) mean he was using cloud code for work (00:52:37) already right like whatever. Um but he (00:52:39) had this holiday obsession. We got into (00:52:41) playing Age of Empires 2. Myself, you (00:52:44) know, my roommate, a handful of people (00:52:45) from like Open Eye, GDM, Anthropic. We (00:52:48) just would do land parties of AoE 2 over (00:52:51) the holidays a bit. Not not like (00:52:52) Christmas, but like a little bit before, (00:52:53) a little bit after, you know, cuz most (00:52:54) of us went home for Christmas. Um, but (00:52:56) like we'd do these lands. My roommate (00:52:58) got so obsessed with like the game that (00:53:01) during Christmas week, cuz he didn't go (00:53:03) home, he just stayed in San Francisco. (00:53:05) Um, he just worked on an RTS game and he (00:53:07) built an entire RTS game. And I think I (00:53:10) kid you not, I think he he used like (00:53:12) $10,000 of Claude in one week and built (00:53:15) an entire RTS from scratch uh about a (00:53:18) like but instead of like being a (00:53:19) standard RTS where it's like oh Age of (00:53:21) Empires for advance through ages or (00:53:22) Starcraft, it is it is an RTS where it's (00:53:24) China versus the US and you're in the AI (00:53:26) race and you go from the start of the (00:53:28) information age all the way through to (00:53:31) you know AGI and like robots and (00:53:33) humanoids and and and like all like (00:53:35) space fairing civil like it's crazy. He (00:53:37) built it in a week (00:53:39) >> and he didn't type a single line of (00:53:40) code, right? He can only dictate it to (00:53:42) the model. And he told me, yeah, like we (00:53:44) have an indicator internally at (00:53:45) Enthropic where you see how many people (00:53:47) actually write code now. There's only a (00:53:49) few hold outs left. (00:53:50) >> But I guess the question to the bubble (00:53:52) is is really a question of uh timing as (00:53:54) well, right? Uh it's it's whether the (00:53:57) build which is supply side and the (00:54:00) demand side are going to land sort of at (00:54:03) the same time. Is that is that fair? (00:54:04) >> Yeah. But also the economics of like say (00:54:06) you you spend let's say you spend you (00:54:08) build a gigawatt you put down roughly (00:54:10) $50 billion across you know the data (00:54:13) center the chips the networking blah (00:54:14) blah blah blah blah right let's say it (00:54:16) has a 5year useful life so it's $10 (00:54:18) billion a year is it a bubble if the (00:54:20) first year you have you didn't make any (00:54:22) money it's zero the second year it's (00:54:23) zero and then third fourth fifth year (00:54:25) you're at 50% gross margins and so you (00:54:27) make 20 2020 now you've made $60 billion (00:54:30) off of this $50 billion investment it's (00:54:33) not the best return on invested capital, (00:54:34) but it did pay for itself. (00:54:36) >> Yeah. (00:54:36) >> Um, is that is that a bubble? Well, (00:54:38) that's what's happening today is that (00:54:39) people are spending all this infra money (00:54:41) on infra and there's no return for a lot (00:54:42) of it, right? A lot of it is just doing (00:54:44) research and like trying to get adoption (00:54:45) and is free users and like what does (00:54:47) that mean? (00:54:48) >> Yeah. (00:54:48) >> Um, (00:54:49) >> depends a bit on (00:54:50) >> the timing. That's the timing though. (00:54:51) Yeah. But oh, that $50 billion capex was (00:54:53) spent in year one. (00:54:54) >> What about energy? In the in the data (00:54:56) center world, you had this fun post (00:54:58) about the gas replacement for for (00:55:00) energy. So, is uh is AI basically uh uh (00:55:04) destroying the grid? (00:55:05) >> It would if the utilities were willing (00:55:08) to let it, but I think the utilities are (00:55:10) so slow and dumb that they don't want (00:55:12) to. Not destroy, but like expanding the (00:55:14) grid. Yeah. (00:55:15) >> Um I think the US could have a way (00:55:16) better grid, but we just don't want to. (00:55:18) Like, no one's made the effort or (00:55:20) initiative. You know, there's not enough (00:55:21) power. America's not built power for 50 (00:55:23) years really, right? It's like converted (00:55:24) from coal to gas and like things like (00:55:26) this but like really just have not built (00:55:28) wholesale new power on a large scale and (00:55:31) there have been a lot of times where the (00:55:32) industry blew up right independent power (00:55:34) producers IPs have blown up multiple (00:55:36) times in the 2010s when uh Korean and (00:55:39) Japanese investors like flooded the (00:55:40) market with because they saw such a good (00:55:42) return there or before in the early (00:55:45) 2000s power was growing a little bit for (00:55:47) a little bit and so people overbuilt on (00:55:48) power so power industry has been burned (00:55:50) a couple times but no one really built (00:55:51) power and then you've got data centers (00:55:52) now all of a sudden coming online and (00:55:54) going from 2% to 10% of the US grid in (00:55:56) just a handful of years. And so you've (00:55:58) got this humongous humongous change in (00:55:59) the industry. We don't have the labor, (00:56:01) right? I think ultimately that's the (00:56:02) biggest problem is the equipment and the (00:56:03) labor and equipment is basically you (00:56:05) know again labor and time takes time to (00:56:07) build a factory so you can build the (00:56:09) things. I think the equipment side of (00:56:10) things will be solved like more (00:56:12) reasonably. And one one example was like (00:56:13) gas, right? People initially thought, (00:56:15) oh, you can only use like the two (00:56:17) vendors, right? Uh Seammens or G Vernova (00:56:20) for gas turbines, but they have the they (00:56:21) have the best ones, the most efficient (00:56:22) ones. It's like, okay, well, like, okay, (00:56:24) also Mitsubishi exists and they're (00:56:26) ramping up production fast. Oh, Ducson (00:56:28) and Korea exist and they're ramping up (00:56:29) production fast. Oh, actually, I can (00:56:31) just take Cumins engines, right? Like, (00:56:33) you know, if you've ever like ridden a (00:56:34) pickup truck or like you know, like (00:56:35) diesel trucks, like everyone loves (00:56:36) Cumins, right? You know, you see the Ram (00:56:38) on the street and has the Cumins like (00:56:39) badge. It's like it's like a that's like (00:56:40) an aura symbol for a certain kind of (00:56:42) redneck from South Georgia, which I have (00:56:44) a little bit of. Anyways, I I don't have (00:56:45) a I don't have a truck. [laughter] (00:56:47) I have though. Um but anyways, like the (00:56:49) there's like all these engines like (00:56:51) people are figuring out how to make the (00:56:52) equipment. You know, solar sucks. It's (00:56:53) too intermittent. Wind sucks. It's too (00:56:55) intermittent. Nuclear sucks. It takes (00:56:56) forever to build. Coal sucks. It's way (00:56:58) too dirty. How do you make power for (00:57:00) data centers besides gas? And like, (00:57:02) okay, the grid's not willing to put the (00:57:04) gas on your site, right? That's what (00:57:05) Elon did. Now everyone's doing it, (00:57:06) right? (00:57:07) >> This other cool post just uh last week (00:57:09) or two weeks ago that was about water (00:57:11) consumption. Uh did you want to talk to (00:57:13) that? (00:57:14) >> Yeah. Yeah. So there's this annoying (00:57:15) thing where everyone's like, "Oh, AI is (00:57:17) using all the water. Oh wow, AI and data (00:57:21) centers are going to like use up all the (00:57:23) water and now we don't have any water." (00:57:24) And it's like that's so silly. Uh water (00:57:26) is a distribution problem, not a like we (00:57:29) don't have enough problem, right? Like (00:57:30) you look at California. So California (00:57:31) has shitloads of water. But people (00:57:33) decide to make oat milk which consumes (00:57:36) like 1,000x the water of like anything (00:57:40) else like regular milk even and and cows (00:57:43) obviously eat a you know consume a lot (00:57:44) of water. Um but anyways like you know (00:57:46) data centers consume very little water (00:57:48) actually right. So the US grid will get (00:57:50) to like 10% of power by like 28 27 is (00:57:52) data centers. For water consumption it's (00:57:54) not even going to crack 1%. (00:57:56) >> Yeah. (00:57:56) >> By the end of the decade. (00:57:57) >> And what was the metric? Um and so so (00:57:59) the the comparison we made is because (00:58:01) like you know it was a bit of a [ __ ] (00:58:02) post but it was like serious research. (00:58:04) Yeah. Basically like we were doing (00:58:05) serious research because we keep getting (00:58:07) this like question and debunking it and (00:58:09) we would do it seriously but then I was (00:58:11) like no no no this is like too like (00:58:12) complicated like let's make it very (00:58:14) simple. So I was like, "Guys, why don't (00:58:15) we just compare it to like hamburgers, (00:58:17) right? Cuz cuz you know, I've heard that (00:58:19) argument from some like vegetarian (00:58:20) people before or some Hindus or like I'm (00:58:23) Hindu myself, although you know, and I I (00:58:25) I do eat beef sometimes, but you know, (00:58:28) like I I'm Hindu, but like you know, so (00:58:30) so we made this comparison to (00:58:31) hamburgers, right? Hamburgers require a (00:58:34) shitload of water cuz cows, you know, (00:58:36) when to for them they require a ton of (00:58:39) water and when a cow's taking a lot of (00:58:40) water, it's not the cow itself, it's all (00:58:41) the feed you're feeding them, right? (00:58:42) Because no one grass feeds their cows, (00:58:45) you know, and just lets the rain take (00:58:46) care of the grass. They like either rain (00:58:48) the the grass or most likely they do (00:58:51) mass industrial farming of corn, (00:58:53) soybean, alalfa, etc., which uses (00:58:56) shitloads of water, right? Like, you (00:58:59) know, or like almond milk like uses tons (00:59:01) and tons of water. Like produce is like (00:59:03) the main user of water. I think the uh (00:59:05) metric was the entirety of Elon Musk's (00:59:09) Colossus data center, right? Uses as (00:59:11) much water as two and a half in-n-outs. (00:59:14) Um because that's, you know, you do the (00:59:16) calculation on how many how many b (00:59:17) what's the average revenue per in-n-out (00:59:19) and how many hamburgers does that (00:59:20) translate to, right? If everyone's (00:59:21) ordering like a combo, right? Okay, (00:59:23) let's ignore the drink, let's ignore the (00:59:25) fries, let's just talk about the (00:59:26) hamburger, let's ignore the bread, which (00:59:28) does use have grain, let's just do the (00:59:30) meat (00:59:31) >> and the cheese. And all of a sudden all (00:59:33) this water is there's so much water, (00:59:36) right? Like a single query like all of (00:59:39) your AI usage from chat GBT of the (00:59:41) average user is like a hamburger, right? (00:59:43) Like it's like okay, this is nothing, (00:59:46) right? You know, because these things (00:59:47) are the data centers actually are like (00:59:49) they're mostly closed loops and like (00:59:50) sure they evaporate some water for like (00:59:52) cooling reasons, but like by doing (00:59:54) evaporative cooling, they're using less (00:59:55) power, right? And that's actually better (00:59:56) for the environment than uh than not (00:59:59) using evaporative cool. There's all all (01:00:00) these reasons why this myth or hoax of (01:00:03) AI of AI using all the water is just (01:00:05) nonsense, right? Like Meta's data center (01:00:07) in Louisiana is getting protested (01:00:08) because the water it's it's going to be (01:00:10) the largest data center in the world. (01:00:11) It's going to be like four or five (01:00:12) gigawatts at least announced so far. (01:00:13) We're tracking some other ones that are (01:00:15) that may be as big or bigger. Uh but (01:00:17) Meta is getting protested because the (01:00:20) local population around that area is (01:00:21) like, "Oh, the water's dirty. It's (01:00:22) because of this meta data center." And (01:00:24) like there's these trucks on these big (01:00:26) trucks on these back roads that used to (01:00:27) be empty completely. They're just like (01:00:29) mad and annoyed about that, right? But (01:00:30) at the end of the day, what actually (01:00:32) made the water dirty is that that's an (01:00:34) area where you go fracking. Like (01:00:37) >> fracking is absurdly worse and almost (01:00:39) all of that gas is being shipped to an (01:00:41) LG terminal and being shipped to Asia. (01:00:44) Like you know, you know, like Japan or (01:00:46) Taiwan or China or Korea and some Europe (01:00:48) as well, right? Like like actually all (01:00:50) of this water is dirty because of (01:00:52) regulation fracking. Like I support (01:00:54) fracking by the way, but you know that's (01:00:55) that's an insane take too maybe. Um but (01:00:58) like water usage is is is like not a (01:01:00) relevant argument. (01:01:02) >> Are you bullish on the sort of energy uh (01:01:05) companies I'm thinking constellation for (01:01:08) nuclear or Vistra I guess is an (01:01:11) independent power producer. (01:01:14) >> I think IPS will do well. I think IPS (01:01:17) can secure contracts at premiums to what (01:01:21) they've previously been able to for new (01:01:22) power plants that are either uh (01:01:24) dedicated or grid connected but come (01:01:26) with a pairing of a grid load right for (01:01:29) example utilities won't let you just do (01:01:30) data centers now but if you come with a (01:01:32) a pair right you're like hey I'm going (01:01:33) to build this massive data center but (01:01:34) we're also going to have this massive uh (01:01:36) power generating asset right say you (01:01:38) know whatever it is right some IP (01:01:40) they're going to partner with and (01:01:41) they'll build the load and the uh (01:01:43) consumption even if it's connected (01:01:44) through the grid for better stability (01:01:46) and more reliability. Um or it's not (01:01:48) it's behind the meter i.e. not connected (01:01:50) to the grid at all. Um like some part (01:01:52) some data centers like partially like (01:01:54) Colossus from Elon uh the original one (01:01:56) or part of Abene's Texas OpenAI right (01:01:59) like Cruso there's a lot of room for (01:02:01) power producers to get outsized returns. (01:02:04) I'm not necessarily bullish nuclear. Um (01:02:06) existing nuclear fine yeah it'll it'll (01:02:08) it can find a higher buyer higher priced (01:02:10) buyer but majority of it will be gas but (01:02:13) like you can do like renewables backed (01:02:14) by gas and then just turn off the gas (01:02:16) and like it's cost more but whatever (01:02:17) right or you can do wind backed by gas (01:02:19) >> and why not nuclear (01:02:20) >> takes too long (01:02:21) >> takes too long (01:02:21) >> no one can build nuclear fast (01:02:24) >> even China takes like 5 years to build (01:02:25) nuclear right like it's it's complicated (01:02:28) it's unsafe right you know I love (01:02:31) nuclear I wish it would work it's just (01:02:32) not relevant in the time scale that like (01:02:34) AI's power is going crazy. Um, but yeah, (01:02:37) there's a lot of interesting stuff like (01:02:39) have clients would like had a client buy (01:02:41) a coal plant and we were advising them (01:02:43) on the transaction based on they just (01:02:44) like showed up and they're like, "Yeah, (01:02:45) we want to buy we want to buy power (01:02:46) assets. We believe in this power story." (01:02:48) It's like, "Okay, great." So, yeah. So, (01:02:49) here's all of the like power plants that (01:02:51) we know of like you can get some of it (01:02:52) from EIA blah blah blah. um which are (01:02:54) these like and then we like worked (01:02:56) through the economics and we looked at (01:02:57) the new data centers being built in the (01:02:58) region and all this and then they (01:02:59) decided to buy a coal plant and they (01:03:01) restarted it and they're like making (01:03:02) tons of money now because now someone a (01:03:05) certain hyperscaler wants to buy the (01:03:06) entire pipeline of power and put a load (01:03:08) load near it right instead of just being (01:03:10) a grid connected asset. So it's like a (01:03:12) super awesome investment. So like you (01:03:13) know power is power is going to do (01:03:15) great. (01:03:15) >> Yeah. I was going to talk about peace (01:03:17) dividends of the whole AI boom. Uh (01:03:19) generally yes right like hyperscalers (01:03:21) are paying for uh transmission grid (01:03:23) upgrades which people will benefit from (01:03:26) right or like you know investors are (01:03:27) obviously going to benefit people who (01:03:29) work in the industry electricians wages (01:03:31) are skyrocketing you know etc right like (01:03:33) plumbers wages are skyrocketing so (01:03:35) there's like a lot of trades that are (01:03:36) doing really well too I think that's (01:03:38) definitely also um part of it yeah (01:03:40) >> I wanted to come back quickly to uh that (01:03:44) um Nvidian core wave deal that you (01:03:45) mentioned as we sort of close the (01:03:47) discussion on uh on capex and a and a (01:03:50) bubble. It seems like there is circular (01:03:52) deals but also a lot of debt kind of (01:03:54) like flushing around. So I don't know (01:03:56) the specifics of of that deal but like I (01:03:58) did hear variations of this where (01:04:01) effectively you have a large player (01:04:02) guaranteeing the debt being the last (01:04:04) recourse uh for a lot of infrastructure (01:04:08) build is sort of uh this plus the whole (01:04:11) like oracle commitment. (01:04:14) there there is a fragility into this (01:04:16) whole thing that can be a little (01:04:18) unnerving. What do you make of it? (01:04:20) >> I think it's like completely fine and I (01:04:22) think like people are like freaking out (01:04:23) and making narratives where there really (01:04:25) is shouldn't be one. It's like well okay (01:04:27) Google doesn't have enough data center (01:04:28) capacity. They need people to build data (01:04:29) centers, but no one can build a data (01:04:32) center because they don't have the (01:04:33) capital. Like don't have, you know, many (01:04:34) cases capital is not the, you know, they (01:04:36) don't have capital, right? Or like no (01:04:38) one will give them a loan because they (01:04:39) don't trust some random [ __ ] company. (01:04:40) And it's like, but then Google's like, (01:04:41) well, no, we've due diligence to them. (01:04:42) We think they can build it here. We'll (01:04:44) like even guarantee we'll buy the thing (01:04:46) or start using it once they build it. (01:04:47) You know, just having a customer alone (01:04:50) spoken for it was enough, right? Um, in (01:04:52) the case of Cororewave, they were (01:04:53) actually able to no backs stop, right? (01:04:54) Right? They were able to just say, "Hey, (01:04:55) hey look, here's our Microsoft contract (01:04:57) for this many GPUs. I want to put in (01:04:59) that data center, that data center, that (01:05:00) data center. Here's the contract for (01:05:01) renting those GPUs. I want to hire these (01:05:02) people. I want to do this." No one will (01:05:04) like they don't have any money, but then (01:05:05) they were able to like have it work out (01:05:06) because they were able to get people to (01:05:07) lend to them. I think like Cororeweave (01:05:09) did that and there was no circular (01:05:10) financing. But that was when there was (01:05:11) like the scale of investment was like (01:05:13) single digit billions or less than a (01:05:14) billion. Right? Now the scale of (01:05:16) investment is hundreds of billions. (01:05:18) >> Yeah. (01:05:18) >> Um and so the question is like, oh well, (01:05:20) if I want data center capacity, how do I (01:05:22) how do I get data center capacity? I (01:05:23) just go to everyone who's going to build (01:05:25) it looks smart is smart enough to do it (01:05:27) but can't afford to do it and tell them (01:05:29) I'll I'll take it and in fact I won't (01:05:30) just take it. I'll go to your debtor and (01:05:32) be like, I'll guarantee you. Yeah. (01:05:33) >> Because, you know, obviously you're a (01:05:34) new company. I've vetted you, but the (01:05:36) debtor hasn't. And so, you know, like, (01:05:38) you know, you know, they don't want me (01:05:39) to just be able to walk away because (01:05:41) like in the Microsoft Corwave deals, (01:05:42) Microsoft could have walked away if (01:05:43) Corwe [ __ ] it up, (01:05:45) >> right? (01:05:45) >> Yeah. (01:05:46) >> There's no I mean, yeah, there's there's (01:05:47) always like uh sort of like cancellation (01:05:48) or whatever possibilities. And so, this (01:05:50) is just a further form of guarantee um (01:05:52) as far as on like a lot of these back (01:05:54) stops as far as on like Oracle getting (01:05:56) the money and then OpenAI getting money (01:05:57) and Nvidia, you know, paying and it's a (01:05:59) whole circular. It's kind of nonsense (01:06:01) because it's like Nvidia's getting (01:06:03) equity in OpenAI. They're basically (01:06:04) saying, "Hey, every gigawatt you buy, (01:06:06) we'll also buy some equity." (01:06:07) >> Yeah. (01:06:08) >> Right. Okay. Well, cool. Now, Nvidia (01:06:09) owns an asset which they think is (01:06:11) valuable. OpenAI, right? Open AAI is (01:06:13) turning around and is like trying to (01:06:14) rent those uh use the equity they buy. (01:06:17) What do they what was their use of (01:06:18) equity? People's cash pay isn't that (01:06:20) great, right? It's mostly just 99 plus% (01:06:23) of their spend at the company is (01:06:24) probably just compute. (01:06:25) >> Yeah. (01:06:26) >> Uh so so sort of like it's like, okay, (01:06:27) well then I I raise this money. I'm (01:06:29) going to do the the whole thing I (01:06:30) explained earlier, right? Year one and (01:06:31) two I lose money. Year three, four, (01:06:32) five, I hope to make money on it, right? (01:06:34) Um, and open has been doing that, right? (01:06:35) So, I'm going to Okay, I'm going to go (01:06:36) out there. I've raised $50 billion. I've (01:06:39) raised $10 billion. I'm going to raise (01:06:40) it. I'm going to rent a cluster for five (01:06:43) years for $65 billion. And I've rented (01:06:48) that contract and now I only have enough (01:06:50) to pay for the first year to be clear. (01:06:51) But I think, you know, you trust me, (01:06:52) Oracle, you think I'm going to grow and (01:06:54) you think I'll be able to pay for it. (01:06:55) Oracle's like, "Yeah, or if you're not, (01:06:56) I think I'll be able to sell it to (01:06:57) someone else." So like, okay, cool. I'm (01:06:59) going to spend $50 billion this year. (01:07:00) >> Yep. (01:07:01) >> To build that data center. And and and (01:07:03) this these this is like for a gigawatt. (01:07:05) Um and so is it like circular that (01:07:07) OpenAI is every amount of GPUs they (01:07:08) consume and gives an investment that (01:07:11) investment is turned around to pay for (01:07:12) the first year of the rent to the (01:07:13) cluster. Um or second year then first (01:07:15) two years go, you know, it's sort of (01:07:17) like it's fine. (01:07:18) >> Yeah. Yeah. (01:07:18) >> Like it's like it's like it is a little (01:07:21) bit funky, but like I don't think it's a (01:07:23) big deal. (01:07:23) >> Yeah. Love it. Contrary intake. Maybe (01:07:25) let's finish with the models and the (01:07:28) software side of things. We talked (01:07:30) extensively about hardware and supply (01:07:31) chain and all the things. I get a sense (01:07:33) that you are super super bullish on uh (01:07:35) what's happening next in in AI. Your (01:07:38) roommate Schulto I assume was the (01:07:40) roommate that you were talking about (01:07:41) earlier on this pod effectively making (01:07:43) the point that we're just starting to (01:07:45) scratch the surface and there was so (01:07:47) much low hanging fruit around you know (01:07:49) RL and all the things you were in (01:07:51) Silicon Valley circles. Is that is that (01:07:53) your sense as well and what are you (01:07:55) tracking on the model side? (01:07:57) >> One thing is like you know simple stuff (01:07:58) like uh GitHub commits other things are (01:08:01) like what's the amount of usage how much (01:08:03) are people using like all these sorts of (01:08:05) things. I think there's so many (01:08:06) different alternative data sources for (01:08:08) tracking AI model progress area (01:08:10) tokconomics uh token economics (01:08:12) tokconomics and so that's like an entire (01:08:13) practice for us. (01:08:14) >> Are you rebranding the term from crypto? (01:08:17) >> I yeah I don't believe in crypto people (01:08:19) like I've always hated them. [laughter] (01:08:21) Um, (01:08:21) >> so now you're taking the term. (01:08:22) >> Yeah. Yeah. And Jensen's used it now. So (01:08:24) I've like I've convinced him to use the (01:08:26) word. He's used it as sovereigns and so (01:08:27) I think I think we've won. (01:08:28) >> That's awesome. Congratulations. (01:08:30) >> I've said it to him. We've written it in (01:08:31) articles. It's an entire practice of (01:08:33) consulting that I just I started in like (01:08:34) 23 2023 uh was token economics and we've (01:08:38) been trying to build out these like you (01:08:39) know but basically I think the main (01:08:40) things are like people who don't code (01:08:41) can use cloud code now right? I think (01:08:43) people don't understand that like even (01:08:44) if you don't code you've never had any (01:08:46) training in software development, you've (01:08:47) never take had a job as a software (01:08:49) developer you can code. Let's take an an (01:08:51) example of what one of the one of the (01:08:52) analysts at my company did right comes (01:08:54) from a engineering background but on (01:08:56) like semiconductor systems right uh like (01:08:59) worked on mechanical systems worked on (01:09:02) these sorts of things and they coded (01:09:03) this thing which was they wanted to do (01:09:05) an analysis of area of clean rooms right (01:09:08) clean rooms are the building that you (01:09:10) the fab has all the tools in the most (01:09:11) complicated kind of building in the (01:09:12) world has every all sorts of chemical (01:09:14) systems and all this area of that a (01:09:16) company who builds systems builds these (01:09:19) systems (01:09:20) and revenue of that company, right? And (01:09:22) so it was like, okay, uh we have this (01:09:24) fab data set. Pointed it at it was like, (01:09:26) hey, here's this fab data set. What's (01:09:27) the square footage of all of them? And (01:09:29) we have this like thing that we built (01:09:30) which uh just pulls with cloud code (01:09:32) separately which for data centers and (01:09:34) and and fabs and everything else just (01:09:36) calculates the area of something from a (01:09:37) from a satellite image, right? Very (01:09:39) simple. So we have the square footage of (01:09:40) all these things. Points at that. Here's (01:09:42) the company name. Okay, go find the (01:09:43) filings. So it dig dug through all these (01:09:45) filings. It it pulled the data, right? (01:09:47) Okay, great. now told it to um compare (01:09:50) these two. Make a chart. Great. Oh, (01:09:52) wait. There's this like weird (01:09:52) inflection. Oh, that's because they (01:09:53) bought a company five years ago. Can you (01:09:55) do a proform of this analysis without (01:09:57) those financials of that of that company (01:09:58) they acquired? Okay, great. And then (01:10:00) like we were able to like like figure (01:10:02) out an investment case for our clients (01:10:05) as well as like you know some other (01:10:07) interesting details from someone who's (01:10:08) never really coded just using clawed (01:10:11) code and it like doing this all and this (01:10:13) is like not even their and it wrote the (01:10:14) note and they just like they didn't even (01:10:16) like work on this full-time for like 3 (01:10:18) hours right they just told the model and (01:10:20) would go work on other things and told (01:10:21) the model and worked on other things (01:10:22) they just did this people don't (01:10:24) understand that like the skill sets that (01:10:25) like I think like if you go talk to an (01:10:27) analyst right a very junior analyst at (01:10:29) any right? Whether it's venture or (01:10:31) especially growth venture or public (01:10:33) markets or private equity, their their (01:10:35) job is like finding data, cleaning it, (01:10:37) making charts. It's like this is cloud (01:10:38) code now. You don't need junior (01:10:40) analysts. Just like a lot of companies (01:10:42) have stopped hiring L4 engineers because (01:10:43) it's useless. Why would I hire an L4 (01:10:46) engineer? I just tell Claude to do it. (01:10:49) You you sort of like have this has (01:10:50) happened and this is a really big like (01:10:53) shift I guess like is that like (01:10:55) low-level knowledge work just doesn't (01:10:57) matter, right? Why would I why would I (01:10:59) use Excel when I can just tell Claude to (01:11:01) manipulate CSVs? Why would I use Word (01:11:03) when Claude will just generate the (01:11:05) markdown and I can copy and paste the (01:11:06) markdown directly into our WordPress and (01:11:08) then you know and that WordPress is (01:11:09) fully formatted now and it's like oh my (01:11:10) god like what's the point of Word, (01:11:12) right? Um and what's the point of doing (01:11:13) all sorts of stuff? I think when we look (01:11:15) at model progress that's just for Opus (01:11:17) 4.5. Open's new model I think will be (01:11:19) better than Opus 4.5 and it's coming (01:11:20) like somewhat soon in Marchish um time (01:11:23) frame. I maybe February, Marchish, but (01:11:25) yeah. Um because OpenAI has a better RL (01:11:27) stack than Enthropic today. It's just (01:11:29) their pre-trained models suck compared (01:11:31) to Enthropic's pre-training, right? And (01:11:33) so like if they catch up a lot on (01:11:34) pre-training and keep their better RL (01:11:36) stack, they would actually have a model (01:11:38) that's much better, right? Flip side, (01:11:39) Google has a better pre-trained model (01:11:41) than Anthropic or OpenAI, but their RL (01:11:43) stack sucks. So if they catch up on RL, (01:11:45) like these models are going to get (01:11:47) ridiculously and then Anthropic is (01:11:48) obviously advancing as well, right? And (01:11:49) so and then and then you look across the (01:11:51) ecosystem, everyone's advancing really (01:11:53) fast progress. These moments are (01:11:55) happening, right? You know, chat GPT was (01:11:57) a moment. Gibbly was a moment. Those (01:11:58) were more consumer. Those were less like (01:11:59) I mean there chat GBT is everyone using (01:12:01) it for work too. But like I think cloud (01:12:02) code is like a new moment right 4.5 on (01:12:05) cloud code is a new moment where the way (01:12:07) you work has forever changed. And so now (01:12:09) we're trying to force everyone in my (01:12:10) company. There's 54 people here. I think (01:12:12) like half of them have coded. The other (01:12:14) half we're trying to force them to use (01:12:15) like cloud code. And it could be like oh (01:12:17) well actually you come from a consult a (01:12:19) semiconductor consulting background. Oh, (01:12:20) you come from like a semiconductor like (01:12:22) engineering of like package. Oh, you (01:12:24) worked in a fab, right? Like these kind (01:12:25) of people, they're using cloud code now, (01:12:27) right? And and their productivity is (01:12:29) being boosted. (01:12:30) >> And it's like, (01:12:31) >> you know, workspace, cloud workspace is (01:12:33) new. It sucks compared to cloud code, (01:12:34) but it'll get there, right? He he he (01:12:36) said he coded it entirely in cloud code. (01:12:37) You know that, right? Or that was on (01:12:38) your pod, right? Yeah. Yeah. So, like um (01:12:41) you I've heard that and I think maybe (01:12:42) that might have been from your pod uh (01:12:44) original uh disclosure. (01:12:46) >> My pod was before that, but yes. Oh, (01:12:48) okay. Okay. It was (01:12:49) >> I had as the guy on my pod subsequently (01:12:50) said that. (01:12:51) >> Okay. I think it's like a brand new age (01:12:53) and and like there's so much low hanging (01:12:54) fruit as Shto said on the episode when (01:12:56) he was here. There's so much low hanging (01:12:57) fruit. Yeah. I mean for for the models (01:12:59) progressing and I think model progress (01:13:00) will translate to revenue. Adoption is (01:13:03) difficult but like actually the UX of (01:13:05) cloud code sucks but like give it 6 (01:13:07) months the models will be good enough (01:13:08) that the UX can be like talking to it. (01:13:10) Yep. (01:13:10) >> And you don't even have to have like you (01:13:12) know CLI integration, right? It's (01:13:14) something even easier. or like cloud for (01:13:16) XL was released recently and it's like (01:13:18) not bad you know building models and (01:13:19) like all these sorts of things are just (01:13:20) going to be like tell someone right like (01:13:22) why tell a junior analyst right when you (01:13:24) can just do it yourself I think it's a (01:13:25) whole new world and it's a $2 trillion (01:13:27) of software work but also of wages but (01:13:29) it's also we have more north of 2% 2% is (01:13:32) claw and then you know there's codeex (01:13:33) and cursor and all these other guys so (01:13:35) probably like 5% of code committed today (01:13:36) is AI generated if not higher marked as (01:13:39) AI generated what's going to happen when (01:13:41) normal workers who do spreadsheets and (01:13:44) office processing start automating their (01:13:45) workflows. I think it's a whole new (01:13:47) world. (01:13:47) >> And speaking of Schultoe, we both agreed (01:13:49) that he was a a perfect specimen. (01:13:52) >> Dude, [laughter] I' I've been I'm (01:13:54) straight, but I've been accused of being (01:13:56) uh homosexual, which is perfectly fine (01:13:58) for for how much I like praise this man (01:14:01) because like, think about it, right? (01:14:02) He's like 6'4. He's like really (01:14:05) good-looking. He's like Australian (01:14:07) accent. Sounds amazing. Like you've (01:14:09) heard his I I have like a annoying voice (01:14:11) probably. His voice sounds amazing. He's (01:14:13) absurdly good at coding. He was an (01:14:15) Olympian level fencer. Like like he (01:14:18) picks up any sport, he's really good at (01:14:20) it, right? Because he's athletic. It's (01:14:21) like, "Holy crap, you're a specimen." (01:14:23) >> Yeah. Yeah. (01:14:24) >> This clip and sent him [laughter] for (01:14:26) sure. (01:14:28) >> Yeah. It must be uh you know I guess uh (01:14:31) may maybe some people don't follow the (01:14:33) playbyplay on on Twitter and like don't (01:14:36) haven't haven't heard of like the fact (01:14:37) that all of you guys are roommates or (01:14:39) you roommate with Scholto and then with (01:14:41) Dwarish and Darkish is like the (01:14:43) podcasters podcaster. So it must be (01:14:46) absolutely (01:14:47) >> What's a podcasters podcaster mean? (01:14:49) >> Uh the podcaster that other podcasters (01:14:52) uh aspire to to to become or learn from. (01:14:56) >> Yeah. Yeah. his his when he's preparing, (01:14:58) you know, it's like he's he's he's he's (01:15:00) so locked in and he prepares so hard for (01:15:02) interviews. It's great. (01:15:03) >> No, he's he's just uh incredible. (01:15:05) >> And then and then he might only say like (01:15:07) a hundred words on the episode, (01:15:09) >> but he's prepared so hard and then like (01:15:10) I think people just realized, oh wow, (01:15:12) he's not just like, you know, it's like, (01:15:13) oh, he just has good guests. No, no, no. (01:15:15) Like he's preparing really hard, but you (01:15:16) can't tell if you're not like realizing (01:15:18) that. And then once he started writing (01:15:20) more and he started writing more, people (01:15:21) like, oh wow, he's actually really (01:15:22) really smart. It's like, yeah, cuz he's (01:15:24) studying like crazy. Like it's like, (01:15:26) "Oh, I'm interviewing an AI researcher (01:15:27) who worked on this. I'm gonna try and (01:15:28) train a freaking model." Yeah. (01:15:29) >> Right. It's like that's the level of (01:15:31) like commitment he goes to when he (01:15:32) records this stuff. (01:15:33) >> What do you guys talk about when you (01:15:34) bump into each other? Is that is that AI (01:15:36) non-stop or you talk about everything (01:15:37) but AI (01:15:38) >> with Shoto? It's like the Age of Empires (01:15:40) game, you know, because we we got super (01:15:41) into it for a bit. We talked only about (01:15:43) that in his RTS that he made. Uh with (01:15:45) with with Dwarash, it's I mean, it's all (01:15:47) sorts. It's like normal roommate stuff. (01:15:49) It's like, [laughter] "How's your dating (01:15:50) life?" "Oh, okay. You went on a date. It (01:15:52) wasn't well. It didn't go well." "Okay, (01:15:53) well, okay." Yeah. you know, like, oh, (01:15:55) you know, like that's me. That's me. You (01:15:56) know, my days don't go [laughter] well. (01:15:58) No, I'm just kidding. Um, or like it's (01:16:00) like, oh, you want to like have dinner? (01:16:02) We can invite a few friends. Like, yeah, (01:16:03) great. Or like, you know, it's like all (01:16:04) sorts of like normal stuff, too. Um, al (01:16:06) obviously we also do talk about a lot (01:16:08) about tech, right? Like we are like this (01:16:10) is our lives. Um, and tech is the most (01:16:12) fun thing. (01:16:13) >> Awesome. Well, great. Great San (01:16:14) Francisco lore. Uh, Dylan, thank you so (01:16:17) much. Uh, that was absolutely fabulous. (01:16:18) Really enjoyed it. Learned a lot. So, (01:16:20) really appreciate uh your coming on the (01:16:22) pub. (01:16:22) >> Thank you so much. (01:16:25) Hi, it's Matt Turk again. Thanks for (01:16:26) listening to this episode of the Mad (01:16:28) Podcast. If you enjoyed it, we'd be very (01:16:30) grateful if you would consider (01:16:31) subscribing if you haven't already or (01:16:33) leaving a positive review or comment on (01:16:35) whichever platform you're watching this (01:16:37) or listening to this episode from. This (01:16:39) really helps us build a podcast and get (01:16:41) great guests. Thanks, and see you at the (01:16:44) next episode.