Dwarkesh Patel on AI Progress

Any AI developer tool can create a brand new project from scratch. That part's easy The hard part is working with the code your business already runs on IBM Bob is a new AI development partner that helps you do the hard job Moving your technology into the AI age without losing the legacy code your business was built on Let's create smarter business IBM This episode of Hard Fork is brought to you by our Hard Fork live twenty twenty six sponsors preremiere sponsor, IBM Associate sponsors Everpure, Pure Leaf and the University of Notre Dame and supporting sponsor at Lassian Well, Casey, we are still on our annual summer vacation and can you believe there is yet more amazing stuff from Hard Fork Live that we have not shared with our podcast listeners. There is, in particular, we had a really fun discussion at the event between Daniel Cocatello and Sah Kapur who have somewhat different views of how fast the AI conversation is going to go. We've heard them debate before. We wanted to sort of have an updated discussion with them now that it's beenm like you getting close to a year since the last time they had it. So I think you'll really enjoy hearing What they have to say about that. We also had the great podcaster Dwares Patel stop by and hang out with a bit, tell us a little bit about what is on his mind. And just to round it up, we took some live Q and A and heard what was on the minds of our audience after a spectacular hard fork live two. So these are all conversations that I would classify in sort of the same bucket of like insider sense making. peopleeople who are deeply enmeshed in the AI scene in San Francisco Ting to understand and explain what is going on, the pace of progress, the trajectory of these models to the outside world. And Syashh, Daniel, and Dwarash are among the three most gifted people I have ever heard try to explain this stuff to an outside world that doesn't always know exactly what's going on. It's a great set of conversations. We think you'll really enjoy it. This is our final installment of our episodes from Hardfk Live two We will be back in two weeks to our regularly scheduled hard fork programming. In the meantime, enjoy your summer. Where' sunscreen So this next segment I am so excited for because we're gonna have a conversation with two people who have very different views about how AI iss going right now Yes, we have Daniel Cocatello with us tonight. He is the co author of AI twenty twenty seven, a report that many of you, I'm sure have read. This came out in twenty twenty five and laid out a vivid scenario or account of how AI could fundamentally upend the world achieving. tasks like autonomous coding and R and D. He's since updated that prediction a few times. We'll ask him about that. And he'll be joined by Sash Kapur, who is an AI researcher at Princeton with a very different view of the future. He's the co author of AI as normal teechnology, which looks at evidence that AI is much like previous technologies that have upended the economy that take a long time to diffuse through society We've invited them both here tonight because we saw last year a very interesting debate that the two of them had at an AI conference called The Curve. We thought it was so interesting that we decided to bring them back tonight and hear how their re views have evolved since then and where they continue to disagree and where they might agree now. So please give a warm welcome to Daniel Coatello and Sayash Kapur What this? Heyaniel He Sash All right, so Daniel, Kevin mentioned us up top. You have updated your timelines a few times since you first published AI twenty twenty seven. Give us the most up to date view of your thinking. What's your best estimate for when we will achieve AI models that can do their own AI R and D Probably fifty percent by late twenty twenty eight That's soon I'm thinking about the calendar. That's two years Yeah, that's like a little bit later than Frobk expects, I think Things take longer than you planned for, you know Which is a point that Sayashh makes sometimes. Sayashh, can you summarize where your views are today? My sense is that you do not believe in the sort of sudden takeoff scenario that some other observers believe in That's exactly right. I think the main reason for that is Basically this disagreement boils down to the whether The bottle next to this intelligence explosion, the bottle next to automating RMD are all computational or whether they rely on real world bottlenecks that will be really hard to automate away. So I guess this is one place where we disagree. I think that in a lot of domains, making these advances won't be as easy as it has brain coding And to really get to sort of artificial super intelligence, you need to cover all of these different domains. you need sample efficiency across the board which is much easier to do in a field leg programming where you have these simulators, these virtual environments, but much harder to do in the real world. And some evidence bears this out addoption of AI systems has been indeed far slower in other domains as opposed to coding So give us an example of what these bottlencks are. I talk to a lot of AI researcher folks, and to them, the way at least the way they make it sound to me is, look, eventually the model just gets good enough and then it's came over. You're saying that there's something that exists called a real world, and I'd like to hear more about it. I mean, look, I mean, To be honest, I think these are just two independent, self consistent worldviews about the future of AI. And the reason that Dannel and I have had such productive conversations is that We're basically trying to figure out where these worldviews differ. Now, speaking of, I think like Daniel's actions and Daniel's predictions are entirely self consistent with the worldview that we'll get to AI systems at this point Unfortunately, in order to get evidence we're going one way or the other, we need to actually carry out lots of evaluations. We need evaluations to be of a much higher standard than we have today. So to give you one example of a bottleneck, The the other day I was talking to a lawyer friend of mine And you know, he uses these tools. he's very bullish about them. What has turned out to be the case is as he started using these tools for bigger and bigger tasks The rate of hallucinations, the rate of unreliable outputs has sort of remained the same, right It's not because the EI systems haven't gotten better. they indeed have. they are so much better today than they were just a year ago. But the fact is that the tasks that you can do with these systems actually are bounded by the rate of hallucinations or the reliability. and that's one place where AI systems continue to struggle And in a domain like software engineering where you have this instant feedback loop where you can actually run the code and see what the output would be It's a much easier botteneck to address as opposed to something like the law where even the right answer is not obvious to a domain expert. Domain experts can reasonably differ in the approach that they take. So this is just one example of a bottleneck in a domain where the right answer can be a bit more subjective than encoding. Daniel, I think when AI twenty twenty seven first came out, there were some people who dismissed it as sort of speculation or scary science fiction was a term that Some people were throwing around a lot. I reported on this. I talked to you and your co authors then. I know that you grounded this in like real like forecasting work, like months of trying to figure out what would happen as the technology got better. And I will say like a lot of that has come true already So you predicted in your AI twenty twenty seven that we would start to see large parts of coding become automated. That much has come true. I was reading today, someone was copying and pasting something that you hadd written about frrontier labs sort of restricting the use of their models for frrontier LLM development, something that has happened this week with Claude Fable What are the things that you think will happen if your scenario continues to mostly hold for let's call it the rest of twenty twenty six. What are we going to see this year So we're not going to see an intelligence explosion this year in the scenario that happens next year. That's So that's nice. I think Um Intelligence explosion being recursive self improvement leading to sort of out of control runaway superhuman AI Yeah, or to put it another way, just fully automating theI research process, causing AI research to happen even faster than it currently happens. And it's currently happening at a very fast rate compared to many other technologies But yeah, I would say the coding agents are just going to get better and better And that maybe a year from now, maybe two years from now they will be good enough that you can sort of say they've automated codating fully. They haven't fully automated coodating yet, but like maybe in a year or two, they'll have fully automated codating. att which point the bottleneck will be research taste and management and all the other aspects of the AI research process besides the actual coding. And then the companies are going to turn towards resolving those bottlenecks and teaching their AIs to do those skills as well And that's going to take some time, but it's going go by faster than you might think when all the coding has been automated U Once they've finished doing those things, they won't have superintelligence immediately the first AI system that can do the complete AI research process probably won't be able to do various other things But once they've fully automated the research process, things will probably go faster and faster. And then the type of system that can do absolutely everything is probably not far off Sash, do you believe this sort of recursive self improvement is possible I mean, in some sense, I think the process of recursive self improvement started like six decades ago In fact, the entire history of computing has been one where we develop tools that then aid us in the development of better tools. We've developed compilers that have allowed us to be two orders of magnitude better at programming. We've developed frameworks on top of that, We've developed entire systems and libraries that allow us to do things that would frankly take like an experienced software engineer years or decades of time if they were using assembly language So I think in some sense, this loop has been kickstarted already. This loop is something that the entire history of computing bears out What I disagree with in terms of like Daniels Predions is whether this process will naturally lead us to a point where we develop the automated AI R and D researcher, or whether humans will continue to have this edge and teams of humans with AI will continue to sort of outperform AI alone, and whether this process will lead to artificial super intelligence. I actually think that it's a very plausible scenario for me that we get the sort of recursive self improvement that AI systems do indeed continue performing better and better at AI research tasks But the end process of that need not be ASI. The end process of it could just be far more capable models than we have today today, perhaps following the trend of previous technologies, and yet not the point where we have these systems that outperform humans, the top human experts on everything, which is I believe, the definition of ASI. Perhaps you should talk about the point of agreement? Yeah,. the point of agreement. Yeah. so we wrote this blog post together, the authors of Azn Normal Technology and twenty twenty seven, where we talked about the things that we agree on And correct me if I'm misstating it, but roughly speaking We talk about what we might call strong AGI or like humans in the cloud, like AIs that can sort of Do all the cognitive tasks or the tasks you can do at your computer as well as professional humans or as well as the best professional humans and I guess the headline is, I agree that AIs that aren't that powerful are still normal technologies And they agree that AIs that are that powerful are not normal technologies Exactly. Or like the nomal technology thesis sort of stops being accurate or helpful in a world where we have like humans in the cloud, let's say. Yeah The reason that we spend this time talking about recursive self improvement is that you RSI is kind of the moment that observers believe is like kind of the scariest moment in the development of AI, right? It's becomes ever harder to control. And so how far away are we from it? and is it possible? I think or probably two of the most important questions that we will ever ask on the podcast. having heard your, what sounded to be like very sensible objections to why it might not be possible anytime soon. and understanding Daniel, you know, why you do think it's possible. I'm curious like if at the very least, like you hope Syos is right, you know, like would br a sigh of relief? Yeah. I would love it if you're right. Okay. Okay. Thank you. But what do you see that makes you think that he's not right? So I think that like I've tried to spend some time thinking about Like what are the barriers? What are the bottlenecks that could block? and traffic from succeeding in their stated plans And none of them really seem that strong to me basically. Yeah, so we can go through them bit by bit like data efficiency you mentioned. It does seem like AIs currently are less data efficient than humans but that also seems like something that companies could probably make rapid progress on if they tried. And also separately, it may not actually be that important for automating the AI research process. It might be that you can sort of like ninety nine percent automate the A research process without getting that data efficiency to human level. And then even though that's not like quite there, Even a ninety nine percent automation would speed things up quite a lot, which would then allow you to do decade or two decades worth of research in a year, perhaps, you know. U Th those are, I think my two arguments for why it seems like We're bringing for you close. Another argument, a sort of meta argument that I would make is that I feel like there's been a long history of AI scientists and other commenters making claims about what AIs can't do. various walls that deep learning is going to hit, and they just keep getting smashed through almost as soon as people are making the claims. And I just feel like that's probably what's going to happen with data efficiency, for example Let's pause there because that actually seems really important to me because that's been my observation as well. And it's why I am more inclined to believe the labs when they make grand pronouncements, right? So Sash, I'm curious like what is your relationship to that? Be you've also seen these models come along and blow away the benchmarks and see the eValves get saturated and we have to make new. In fact, you've been making your own eValves because the old ones got saturated Yeah, I mean, we've worked on several wells that for example, anthropic has used and for saturated with the release of Ous four hundred five, we were the first ones to say that, look, this is like solved now And I think this progress will continue I think as long as we can specify things well enough, we'll continue to build AI systems that can solve those tasks. Where I differ perhaps is whether the natural endpoint of this process is something like we solve data efficiency. I'm skeptical about that for a couple of reasons. First, sample efficiency or data efficiency is not the only bottleneck to getting what we called humans in the cloud earlier And the past sort of, if you look at past progress in AI We've continued to develop these more general systems But at any given level of generality We've been really bad at predicting what the bottle next to the next level are. We've been really bad at knowing when we solve those bottlenecks and what underlying transformative breakthroughs are needed to solve them. And know like as evidence of that, perhaps we can take the Transformer moment. and before that, we can take all of the skepticism about neural networks that pervaded the research community in AI. And know it took a matter of like a few years until the community pivoted and now everyone is all in on transransformers. But perhaps that's not the right architecturaloice either. Perhaps you're sort of yet to discover these new architectures that would allow us to make these data efficient AI systems, and perhaps those will still not be enough to get us to the point where we have the sample efficiency of humans in the cloud. So that's sort of the broad stroke of things. I think the AI community in general has been really accurate about near term predictions, about things that are Within the event horizon, so to say. And has been really bad at predicting transformative shifts that sort of change the entire research paradigm. And maybe like credit by credit is due, I think Daniel was one of the few people who got some things right in his report from twenty twenty one. Was it about what twenty twenty five looks like? But in general, I would say the community has a very poor track record on it. Well'll say more of but like, what's the prediction that they made that just wasn't true at Dog in. Like, what is a prediction that the AI industry made that just was not true at all H I guess like the entire skepticism about neural networks. So from the nineteen nineties to the twenty thousand ten s, the entire AI community has dismissed neural networks as a joke. Basically, you could count the number of researchers who took you seriously if you worked on neural networks on like two hands. And it was only through the persistence of a few people like Fif Lee who released this big data set that led to the Deep Learning Revolution. And Yoshua and Jan and Jeffrey Hinton, who later went on to win the Turing Award for their work on deep leararning, that this sort of subfield persisted and eventually was able to disprove claims of skeptics U And you know, in the same way, I think the AI community might be hurting too much around let's say transformer based models right now. and perhaps at the expense of other transformative improvements that are breakthrough improvements that are sort of being sidelined because of this community' single minded focus on it I think an experience that you both have in common and that Casey and I also share is writing things that we think are very measured and careful and precise, and then just having people interpret them in the wildest possible ways. You both published your sort of breakout essays, scenarios, and it was immediately, both of them were sort of seized on by these polarized camps you know, David Sachs, the former White House advisor was was, you know, posting things about AI being a normal technology and sort of agreeing with you and taking issue with you for changing your forecast. And oh my God, the doomers are you know are backed into a corner now. Gary Marcus and JD Vance and all Bernie Sanders and all kinds of people have used your arguments in support of kind of whatever they already believed How has that been to watch your work ripple out in maybe these ways that Arenn't way you expected Well, I guess I'll go first. It's been a sort of u a leap of fa, faith in humanity. Uh You know, at openen AI, I was doing scenario forecasts like this too, much smaller, you know, low effort versions, but they were just for internal use only. like I wouldn't be allowed to publish them. and It seemed to me that the world really needs to wake up to AI and what's coming and start thinking more seriously about it. you the discourse is not necessarily so great. and there's lots of terrible people and lots of terrible takes. And it's very chaotic and confusing. but We at Air Futures proroject are sort of making a bet that like Well We should say what we think is coming, we should be clear, we should be articulate. We should explain our reasoning. The discourse will get rolling. lotots of people will say lots of things. Hopefully in the end, it will converge towards the truth. Hopefully in the end, it'll converge towards better decision making on average. And we'll see what happens. I have faith Sash I guess the biggest surprise for me was How few people read things in depth? I mean It was honestly shocking. in the first line of the essay we compare AI to the internet or perhaps the electricity, like electrical revolution We talk about AI' impacts as sort of being at par with perhaps the first indndustrial revolution And people put us in the same camp as Gary Marcus sometimes which is just honestly shocking But you know, like one level deeper, I think it has been really nice to see sort of these intellectual communities use these essays to advance their intellectual thinking. I think Perhaps the biggest surprise to me was the fact that like our essay and perhaps both of our essayss were sort of taken so seriously by people who are thinking deeply about the future of A. and that was really heartwelming Looking back, have you ever like had second thoughts about using the adjective normal to describe AI? Because I read your writing and I think it's beautifully argued and I share it widely with folks to sort of help them explore reasons why AI may diffuse more slowly than other folks think. And yet I have never really thought that AI was all that normal You know what I mean? I do understand that. I mean, I guess part of it is the fact that We have been In these cycles of discourse where At least the people who are thinking seriously about AI Take it for granted that AI is transformative, and we do too Now within that discourse as well, there's this huge spectrum of opinions, right? Like even just between the two of us, I think ye will be as impactful as the internet. Daniel perhaps thinks this is the most important invention in the history of humanity And you know, do how do you put yourselves on that spectrum So this was the debate that we felt was really worth having. L we're not interested in the takes of people who think there's nothing to see here. L we actively sort of distance ourselves from that, let's say in the first paragraph of the essay in a lot of our writing And I think this is the debate that's worth having. So within the context of this debate, I don't know. I feel like it's a fair description of where we lie on the spectrum I don't know if you agree, Danieel, but I think it's also been helpful between us to clarify where we stand under this technology. And to just say that, today's AI is normal technology, I think is like a really powerful statement. And of course, this doesn't discount the importance of the technology. It does not discount the importance of taking its societal impact seriously. but it does s put things into perspective compared to the view that Daniel perhaps has about the future of AI. So AI twenty twenty seven, because it warns us that these sort of very disruptive changes are coming very soon, has a sort of natural set of policy responses that we might want to see in response to that. right policy response to AIs a normal technology, and it's going to take longer than Daniel says I mean, one thing that I don't know if you'll find surprising, but maybe many people here will find surprising is that Daniel and I share a lot of common ground when it comes to policy responses I think both of us value transparency immensely Both of us value the ability of external third parties to be able to see what's going on inside companies In fact I mean, we were just talking backstage about Anthropics's release of Fable five and the fact that the model Purposefully is decreated for tasks involving AI R and D And I think I speak for both of us when I say that this is a very dangerous president. We shouldn't be fine tuning our models in such a way that they lie to the customers. compompanies shouldn't be sort of allowed to do this. They should act in good faith. And so that's the sort of thing where we have a lot of policy agreements I do think there are areas where we diverge, for example, there might be sort of in these more aggressive scenarios, you might want a conditional slowdown. you might want companies to pause. Whereas when you consider AI as normal technology, the benefits of diffusion of AI and the development of more capable AI systems perhaps outweigh the risks a little bit more At least in the near term, and it was funny when we sort of I spoke to Thomas, who's another one of the couthors of Airt trarain Rin seven, we spent hours trying to figure out whereere it is on the timelines that we actually disagree And it was funny because we couldn't find any near term disagreements I mean, we wrote this blog post together where we say that you know, I agree completely with the events of AI twenty twenty seven or like at least find them plausible until the end of twenty twenty six, which is a long time he wr this last year. And so in some sense, I think There is much more common ground in terms of policy than you might think.. You guys are being much too agreeable. Daniel, what is something you are worried about more than SIOSh is? And then I'll ask the same question of SIS. What is an AI risk concerns you more than you think it concerns Syash. Well, in general, you know, strong AGI or you know, super intelligence, that sort of thing U Main one would be loss of control Number two would be concentration of power. There's a whole bunch of other ones besides that, but I'll stop there. I can elaborate if you like. Those seem pretty bad. Yeah. So I asked what about you Actually, this is another thing we were just talking about backstage. I mean I was surprised to hear that we disagree far more or like I'm far more concerned about military users of AI than Daniel is. I mean it's on the list. It's justh Yeah It's a couple notches down. But I mean like as you both know, in the essay, we explicitly carved out military A because we felt like we weren't the right people to comment on it. And you know, people who are experts on this like Michael Harwz have used our frame to argue that Military a, at least today is a normal technology in this view as well But frankly, the actions that are being taken by countries worldwide by nation states are pretty d alarming. I mean, I think we shouldn't take it for granted that companies or countries can use skillbots That is not something that requires further technological investment either. It's not something where we have any technical bottleninks we can use like off the shelf computer vision libraries. to basically build killer robots today It is actually something where we need to exercise a lot of agency I'm not really positive about where things are going right now on that fr. Well, I truly believe that whatever is about to happen to us lies somewhere in between the views of these two people. So we will continue to pay very close attention to your work. Thank you so much, Daniel and Sj. Thank you for joining us guys, That wass fun. Thank here. Thank you.. Thank you. He We'll be back with more Hartfk Live After these messages. Any AI developer tool can create a brand new project from scratch Part's easy The hard part is working with the code your business already runs on IBM Bob is a new AI development partner that helps you do the hard job Moving your technology into the AI age without losing the legacy code your business was built on Let's create smarter business IBM I'm Paul Tonorio, I cover soccer for the athletic. And I'm Amy Lawrence. I cover football for the athletic. Whatever you call it, the biggest competition in the sport is happening right now, and the athletics World Cup coverage has everything you need to follow the tournament. There's forty eight countries taking part from the tiny island of Curacaut to the five time Champions Brazil Even if you don't know you your offside from your on side, if you're eager to know more about the teams, the matches, all the stories on and off the pitch, we've got you sorted. Maybe you're the kind of person who's already up early every weekend waking the neighbors when your favorite club scores We'll make sure you get equipped with more information, more insight than anyone you know. We've got more than seventy obsessive reporters on the ground, covering the ins and outs from every game. I almost forgot to mention the best part, Amy. free access to the Athletics World Cup coverage in our app Download the athletic app and see you there One thing we know for sure is that no matter what happens with the future of AI, it will be extremely fun to talk about robots Yes. So we have already shown you I think more than ten robots tonight, including members of our robot choir. But we have one more very special robot guest tonight. We are about to bring on George Ekas. He is the director of Engineering at Tobber Life AI, a robotics company in Silicon Valley that is one of the leading distributors of humanoid robots spepecifically these unitree robots from China. And we are going to be joined by George and Toby the root. George and Toby, come on out for having me You see have George You're a very convincing humanoid. Oh no, wait, that's Toby Do do we shake hands? Okay.'s try it there. Hi, What? Short King Great I appreciate the weak grip strength. It gives me comfort. Yeah, it's sort of like a dead fish handshake. Yeah. Now he is advancing on me. All right. Oh o.. Wow. Now, we're gonna to talk about all the things that Toby and his brethren can do, but we heard that Toby can actually dance Is that true? That is the case. Okay. Can we see that? Toby, can you dance for us?, will you help us out? Hit it DJ Listen, we've all been there. Sometimes you just dance till you drop. This robot left it all on the dance floor, ladies and gentlem. G have an operator. Thank you. Thank you, Toby for your sacrifice. You will not be forgotten. We'll add you to the N meemorium next year. Now is Toby capable of st? Is okay? Yeah, probably just mis cllick on the controller out. Oh he's not autonomous right so that.' absolutely fine. They're quite durable. Oh my God, that was not in the script. Yeah. No U Yeah I' sorry, we've traumatized our audience here tonight. I'm so sorry Now George, are the choreographer on that orope, o. Well, it was great choreography. So George, what is the use case for these other than doing dance demos and sometimes falling over? Who is buying and renting these humanoid robots from your company? and what are they doing with them Well, right now, the early market for the humanoids is the research market. People want to collect a lot of data. You guys had the Nofolks on specifically Burnt right. And they're deploying the humanoids into households to try to collect a lot of data in the households. People on with the unitry robots are also targeting different use cases, different companies are pursuing different verticals with them and trying to get big data sets and train models on these humanoids There are also a set of robots that we also sell which are more reliable, more industrial right now. called quadrupeds and probably easier just to remember them as the dog robots You can put Lidar on them as goes, Mark Zuckberger or Elon Musk on them. We saw that earlier tonight, Yes. I forgot about that.. Somehow, somehow I forgot about that. But they are practical for like inspection use cases or security patrols. So those are kind of being pushed out into industry and applications more. and these are on the edge of research and requiring data to build policies. How much does one of these cost They range in cost if you want one to just dance around. I don't remember the exact figure on the low level dancing ones, but they're less than the ones that you could put dexterous hands on and then go and collect manipulation data. with on tasks. so you collect data from doing tasks with them So like more less than ten thousand dollars More M. Okay. That's a great question. The ones I was getting to are like in the fifty to seventy range. the ones with the hands. So like like a mid range sports car Yes, ye All right. I have to say, it did not inspire a lot of confidence in me to learn that the primary use case for these robots is data collection I mean, I think the vision is that these things, as we saw when we talked with Burnt from One x about their robot, as we're hearing about these unitry robots, the dream is that these things will just be in your house and will be doing chores for you, folding laundry, doing the dishes, cleaning the house. What is the timeline for that? Do you think that is realistic? Should people be preordering now in hopes of automating their chores forever Where are we on the chore spectrum? I think Bern's very optimistic. I'd put it a few more years out than he would in terms of being in your house, but in terms of maybe operating in an industrial setting where they can maybe load up a fabricator or something with a material or a part, I think that's in the next couple of years. And there's actually early implementations of that by like figure in unitry and unitry in their factory, figure in the BMW factory. So people are doing that with these But the widespread adoption, I believe in the next couple of years will happen in those settings. Let me ask one question about the data collection. Some security researchers have claimed that Unitree robots might have a backdoor that could allow remote users to control or monitor what they're seeing. is C Toby send the data to China? So they do send logging data to China just like every other Chinese thing that you can own like a computer or Um An other computer chip based thing that connects to the internet that sends logging data, they send that, but they don't actually like there hasn't been an established thing that sends camera data or telemetry data of the joints to China. So There are things that people will be like, oh, it sends data to China. It's like, yeah, and your computer sends data to Microsoft. and it's because your computer crashed and it needs to send data to Microsoft I think the difference is in this case, that Unitry is a Chinese company and some members of Congress have become very worried about the fact that these are now being sold in the United States. Some of even propose banning the importation of these specific Unitry robots. How likely do you think that is? and would that be a big hit to your business? What's your plan if they ban these? I would certainly be problematic U So' not a lot of Americ I wouldn't allowed that. Yeah ye. Yeah, if they're going to ban all Chinese humanoid robots, like I wouldn't be too stoked on that. So I don't have much more to say.. Well, much to consider. before we let you go, does Toby maybe have, you know one more C cool routine he could show us Yes, he does. Take it easy. All right, DJ Dan, will you help us out again Great This like what happened the last time Casey had a Long Island ice tea at the club. All right. fascin. j us. Thank you. A So good, I believe anything. Any AI developer tool can create a brand new project from scratch. That part's easy The hard part is working with the code your business already runs on IBM Bob is a new AI development partner that helps you do the hard job Moving your technology into the AI age without losing the legacy code your business was built on Let's create smarter business IBM I gave my brother a New York Times subscription. She sent me a year long subscription so I have access to all the games. We'll do wordle, mini, spelling be. It has given us a personal connection. We exchange articles. and so having read the same article, we can discuss it, the coverage, the options, it's not just news. Such a diversified gift. I was really excited to give him a New York Times cooking subscription so that we could share recipes. And we even just shared a recipe the other day The New York Times contributes to our quality time together. You have all of that information at your fingertips? It enriches our relationship, broadening our horizons. It was such a cool and thoughtful gift. We're reading the same stuff, we're making the same food. We're on the same page Connect even more with someone you care about Learn more about giving a New York Times subscription as a gift at nYimes d. com slash gift All right, gang, we are in the home stretch, but we had one more friend of the pot who we just wanted to bring on and have a little bit of fun with before the end of the show Yes, our next guest is friend of the pod and YouTuber and podcast sensation, Duwarash Patel, Dwarash Come on out What's up guys Good to see you. H how you. Allright How am I supposed to follow a robot dancing? You could fall over. Yeah. You could just faceplant. That would be great Marash, it's been a hell of a year for you. You are firing on all cylinders, doing interviews with Jensen Huang and other tech luminaries. You've got a new blackboard series that teaches people extremely dense and esoteric concepts in AI. You also got profiled in the New York Times in April And they made a big deal of you and your media empire that you are building here I don't really have a question about that. just I'm just kind of like in awe of what you have managed to build. I'm curious like what What you hear when you hear the conversation about AI twenty twenty seven versus AI and normal technology, whereere are you on the spectrum of like everything is changing. The scaling laws are holding to Maybe things are slowing down and we don't quite have the breakthrough ideas yet to get to AGI I think fundamentally the scary thing is we realize just how Far we are from human intelligence, yet these models are so powerful And so that raises the obvious question is when they not only have the current advantages that they do, that they can think You know, thousand times faster, they um have greater ability to absorb knowledge across a wide variety of domains. If anybody's used these models that like coding work or any sort of like computer use work, you must have experienced this. And then you think, well There's this huge overhang where Humans are able to learn about new things Literally a million times faster. If you think about how much information you see from birth to adulthood versus what these models see, we're capable of retaining information across sessions. We're learning on the job. We're not just like first dayay on the job, the way these models are experiencing things. And so I think that the really scary thing really is that like We know that there's a big difference between where these models are currently and where human intelligence lies. We're making really fast progress towards human intelligence. Already, these things are so capable. What happens when they not only have their inherent advantages because they theirre digital minds, but also have all our advantages. written and spoken before about how you've tried and failed to automate parts of your own production process with your podcast and your YouTube show, and how hard it's been to sort of get rid of some of the sort of sticky human processes there. Are you having better luck with newer models? Like is your operation more AI than it was six months ago U so most of the tokens I see in a given day are produced by AI And I can't really come here and say like, no, yeah, he's not making me more productive or I'm not using it in a significant way. I do I think people underrate how hard it is to automate jobs. L people underrate how much It takes to do every single thing a human, why even white collor worker might be doing Um At the same time, you you guys must be finding this as well Just the ability to tree out juice a amunch of information, which is a large part of my job I just gotten way better Yeah How are you guys been finding these models? I mean, sort of the same. I do feel like with each of the big leaps and model capability, it becomes better at tasks that are quite useful in for example, like the preparing for a podcast, right? If we're sitting down with a guest that I'm not that familiar with saying, hey, go a outad and you know prepare a briefing document for me about this person and give me some interesting directions to maybe take the conversation based on things they've said in public in the last three months. I mean, that's absolutely a job that I could have hired for. and now, you know I can get in about like four minutes on my computer So that's really useful. Does it make me more productive? Yes, but do I like work less or use the computer less? No I'm finding something similar. I want to use these models to automate a lot of my life and I've been very successful at doing some pieces of it. But there are just things that now the primary feeling I had, like I got access to Claude Fable yesterday. and the primary feeling I had was like, I am too dumb to use this thing. Like I actually don't know what I would prompt it to do that a previous model would not have been able to do. But I'm not building RL environments, I'm not overseeing training runs. So like what is the use for you as a media figure and podcaster? Like what is the thing that you wish the models could do that they can't currently? I think u becausecause we're so First of all Every time emr say something embarrass about the models put it in into the context that we're living in like an absurd timeline And I am reacting to My you know, close friends who are just like, well, you just had some of them on 'm talking about like the singularity in two years U'm I feel like we're so used to what these models are capable of currently, that we ask these questions like, well, what is it that they can do? are And they clearly are already AGI.'s like No, we all have jobs that wouldn't happen in a world with AGI, right? Like just get them to do something U pretty Okay. so for example I'm negotiating with a sponsor for next season or something. and like they ask for you do like the back and forth there with the relevant context about how we think about our business and stuff. It's like probably a one hour horizon passasks from me or my general manager. The models couldn't do it at all Or like let's say book a show in another city, like book an event like this, right? There's a lot of people who are involved in this. What part of it could the models do reliably? It's like Anyways All this to say, I think people really underrate what the range of even white collar work is. I mean, it seems to me like it might be very helpful in a negotiation though. particularly, I mean, you know, you're not in this position, but maybe you're just, starting a new podcast and you have some interest in response and you say, go tell me something about this market. and what's the sort of the best place to get started? Like I could see it compressing that into a much smaller problem to your point Somebody still has to do the rest of the job Yeah, that's right. I mean, they can't like do something on a computer you might want them to do, right? And it's actually quite interesting. whyy are they so bad at computer use given that it's an extremely veriable domain And I think that that actually goes to show you that it's not just about verifiability. it's about like the ability to the environment has to be one which allows you to deterministically run many parallel rollouts at the same time and like if you tryed to do that on Amazon Andy Jassie will just shut your ass down. And so you know, they have to build clones every single website because it's It takes a ton of data in the relevant domain in order for these models to become competent, like learning how Amazon works or Slack works. you got to build clones with those things. that's very labor intensive And Yeah, so I think we'll make progress on that as well. But yeah. One of the issues that you really brought to the forefront of the industry's conversation, I would say, over the past year, has been the failure of these models when it comes to continuous learning, right? So it's often observed that a good LLM might be better on day one than an intern, but the intern is almost always better after two weeks because they've been able to learn Are you still as convinced that like this is going to be a major hiccup to getting us all the way to AGI? or is have recent developments? mean, maybe any new models changed the way you think about that? So there's a big crux in how people think about how these models will evolve And one side of the discussion says, you need some way in which between sessions for a given user, the weights themselves are updating. Because if you think about the way humans learn, There's not like You know, you're way better at your job than you were the first day you were on your job. Like people often say an employee iss not net productive until six months on the job. What is happening at that time? It's not like you're building up this intntensely accurate episodic recall of every single thing that has happened to over the six months, which is what in context learning is like. that just grows linearly in size as you spend more time on the job. It's like there's some distillation back in like a higher level abstraction that's happening over time. And so does there need to be an updating that happens back in the way is the real question. Do some people say, well, no, you just Basically, you'll get to a point where these models are spending six months on the job and that six months is happening in context and we're going to train them in such a big variety of RL environments that they'll learn how to adapt to any given situation you put them in. My question with something like this is, I think that might be enough to get these laps to like a trillion dollars in revenue or like truly ludicrous outcomes. I'm concerned about, or also interested in well, do we get to superintelligence or something like that And you know, one question you ask is how would you build something that is as good as Henry Kissinger politics the relevant there's no relevant training environment for that you can run into a data center And so you do need something that can learn that on the fly. And maybe just by doing enough RLBR, you build something that can just pick up whatever Picissinger picked up through his life that through interacting with the world. Maybe not. You know, the headline coming out of this talk is going to be Dwaris says Henry Kissinger is good at politics. So I'm just preparing you for that. LBJ and whatever, the example doesn matter. You know what I'm saying? Interesting You have a very old soul.. All your references are to mid twentieth century. You live in San Francisco with Sholto Douglas, a researcher anthropic and Dylan Patel of semi analysis, very influential semiconductor newsletter. You guys Have you seen the Retman I gotta split. Well, that's my question. Semi analysis is reportedly making something like one hundred million dollars a year in revenue. Anthropic is obviously very valuable. At what point are you guys rich enough to not need roommates The problem is everybody else in SF is also getting so rich. so And The housing is increasing at the same rate that our net worth is increasing. We're never escaping this. One knock that I sometimes hear on the sort of San Francisco AI scene is that it's all very clubby and insular, that there aren't a lot of peopleles who are sort of doing the work of holding people to account or being appropriately skeptical. One detail in the New York Times profile of you was that you sometimes invest in companies, you CEO's or leaders you interview Do you think that journalists and other sort of more conventional media people have the wrong sort of framework for thinking about conflicts of interest? or do you just think you're doing something different I totally see the rationale for journalistic policies that say you're not allowed to have any sort of financial entanglement with the company that you're covering or whatever Um I think at the end of the day I hope the product speaks for itself And that if you watch an interview I do with a CEO or an executive You hopefully feel like I ask the relevant questions that at least I'm not look, I also don't try to steelman some objection that I don't have. Um, But when I do think that they're not making sense, I try to say so. and I hope that that in and of itself speaks for Who's your white whale? Who's the guest that you wish you book that is not Robert Car on. Can you make this happen? Robert Carro, Okay, Robert, if you're out there, go on to our cest. I first. Yeah, I will say that Robert Carro was also famously Conan O'Brien's white whale and Conan O'Brien never got him on the show. No, he got him on. He did he? Yeah, and Conan O'Brien needs a friend. All right, He just fact checked my ass Yeah Well, Darkest, the podcast and the show is amazing. I learned so much from it. I listen every episode and I understand about eighty percent of it now, which is up from twenty twenty five, about twenty percent. So I'm learning along with your audience and we thank you for all the work you do. It's a great show. Thank Dest Good to see you guys. Good to see you. go. All right. Okay, well, friends, we are almost there at the finish line, but before we go, we wanted to take some questions. If any of you have questions for us, we will spend a few minutes answering them. We have mic runners upstairs and downstairs. So raise your hand. someomeone will approach you with a mic, anything where an open book, you can ask us about it all It's like a YouTube comment section, but in real life 's here. Hi, my name Son, can hear me? OkayK. Hi, My name is Dallan, where I'm brother from Utah What happened to the Fedaverse Great the Forkverse, I should say, what' The Forkverse was of course, our effort to build a social network in a federated way, sort of show people what it would be like to be part of a social network that wasn't owned by a giant corporation. And I think it just ran into the challenge that any social product does, which is that if you're not constantly bringing in new users, it's like default state is to just kind of shrink And so you know, we've been in discussions recently about like what is the future of it? I think it was a fun experiment, but you know we didn't really have that strong of an idea of what was going to happen after we started it. And so we're now sort of living with the consequences of that Do we have anyone in the balcony? Yes, yes. Hi, Kevin and Kasey. I was wondering why we're not hearing more from executives like Sata and other know tech leaders who are restructuring their companies around the premise of AI. They just don't seem to want to engage with that premise when you ask them What do you think that's about? I mean, I think there's a lot of conflicting incentives here, right? There are some companies that really want you to know how much they are using AI and how much more productive they are getting and how many workers they are laying off. And sometimes that's real and sometimes it might just be covering for some overhiring they did a couple years ago I think that's going to flip at some point where companies will not want to advertise the fact that they are restructuring around AI. Right now, there is still sort of this weird market premium for that. And so I think that will continue for as long as the market premium lasts. And then it'll be like we're just gonna sort of sweep it under the rug and hide it. And if we're going to lay people off to replace them with AI, we're going call it something else becausecause we don't want to deal with the backlash, but I think that really hasn't happened yet, which has been a surprise to me. What about you? Now, I agree with that. And in the interest of answering as many questions possible, I think we should move on to the next one right here. Hi, my name is Ea. I work at Quizlet. If you've gone to school in the last twenty years you've heard of Quizlet if you haven' What? Iways U Education is being obviously like radically changed, but like people need to learn and kind of the fact that you need to learn doesn't really change. So I'm curious if Quizlet were to just like start everything from the ground up tomorrow, what do you think we should build I mean, that is really challenging. I mean know Kevin and I get a chance to go speak in schools from time to time. And I think what we find is people who are doing their absolute best to introduce fairly incremental change and kind of see what happens. There's just tremendous uncertainty right now. know school is typically trying to educate you for like a fixed target. know, Like when I went to journalism school, it was like, well, if I get these skills, then you know I can have this kind of job I think, you know, like we're not able to ask any guests on this stage about anything longer than a two year timeline because just none of them have credibly anything to say about that. So you know, how do you like educate a five year old so they'll be prepared for the world when they're eighteen? Like, you know, good luck What an inspiring message All right, let's take a couple more. Yes, up there in the balcony. Okay Can you hear me? Okay, great. Please introduce yourselves. Oh, hi, I'm Liz. Hi, Liz. Okay, so two real legitimate questions. Number one, what are we wearing now that all birds is under Okay. And two so I work as a regulator. I work for the state of California. I do privacy regulation. And so my question is on so if you were to take a stab at what would be in the AI in the newew worldor for privacy, like how are you going to protect your digital cellves, either your sons or your friends? what are we going to do when it's all owned in one walled universe Yeah, I mean, my hope is just that that is not the case. You know, we sort of asked Cindy about that tonight. Like I think there is a lot of logic in having some kind of privilege like system that protects certain kinds of conversations that you would have with a chapbot the same way, you know that a conversation with a lawyer might be protected. But also think there's a lot of wisdom about what she said is, you what systems can we build that would ensure that that sort of data never makes it into the hands of a big corporation? And I think we should outlaw dat brokers. Next question. yeah Outlaw data brokers. That's a good one was that? Oh yeah, and where do you getar your shoes, Kev U These are from Quintince. That was not sponsored content, they just are. Yeah. Th are better though. I got these from like online unspecified. I honestly don't remember, but I can look into it. Yeah. I' figure it out by the reception. How's that? All right, just a couple more. So I'm a software engineer, so take this for what it's worth There's been some talk about, you know, like lots of people are afraid of jobs going away and then you hear other people saying, oh, there's tons of hiring going on That's what I see. I said a lot of hiring going on It's all for senior engineers for people know how to who know how to fact check the models or how to architect and combine the things that they can do really fast What's happening with the entry level folks, seems like that is a real problem. Yeah, so I've talked to a couple of labor economists about this within the past couple of weeks and they have sort of said like Believe it or not, things were actually just like much worse during the great financial crisis and that like the circumstances that we're seeing today like don't approach that at all. Now, maybe they will eventually. The one labor economist I talked to Katherine Anne Edwards was telling me like Some people sometimes forget that like your first job just sucks and has nothing to do with what thing you actually want to do. And so she's sort of like encouraging younger folks to manage their expectations, which is also not a very inspiring message I think we could do one more question. So let's have the last question Yes. Hey there. My name is Kevin. Oh Great name. goodood. Yes, my name's Kevin. and What is your optimistic view over here in the middle if you're looking out? What is your optimistic view on AI for about three years out, two to three years out, justust curious to get y'alls take My optimism is around The acceleration of science and medicine This is really a place I care a lot about. I really I don't know if any of you saw the like the cheering at the conference the other week where they announced that they had created a new breakthrough therapy for pancreatic cancer. I want there to be like many, many more of those very soon. And I want the yeah, thank you. Um