Dispelling Myths About AGI

Support for the show comes from Odu Running a business takes everything you've got, and a lot of the tools out there that are supposed to make your life easier just aren't great at talking to each other And that means you end up having to toggle between a dozen different apps and services just to keep the lights on Enough of that. Now there is Odu the all in one fully integrated platform that might actually help you get it all done Thousands of businesses have made the switch, so why not you? tryry Odoo for free at odoo d. com That's odoo dot com The for the show comes from Plaud If you're an executive, small business owner, project manager, journalist, or anyone responsible for important decisions, details matter Conversations move fast and it's easy for key contacts, follow ups, and action items to slip through the cracks once the meeting ends. ClD is an AI powered note taking system built around dedicated recording hardware It captures conversations, transcribes them, and turns them into searchable transcript, summaries and action items So you can focus on the discussion instead of worrying about capturing every detail yourself Is applod. Ai slash markets to learn more and use that markets code to get up to fifteen percent on Support for the show comes from Odu Running a business is hard enough, so why make it harder with a dozen different apps that don't talk to each other Introducing Odo It's the only business software you'll ever need. It's an all in one fully integrated platform It makes your work easier CRM, accounting, inventory, e commerce and more And the best part, OdDu replaces multiple expensive platforms for a fraction of the cost That's why over thousands of businesses have made the switch. So why not you Try Odoo for free at odoo. com That's O d oo d. com Welcome to Property Markets We've spent a lot of time talking about AI lately, from the Trump administration's export restrictions on anthropics models to the ongoing questions surrounding the economics of companies like Open AI and Anthropic Taken together, these stories point to two fundamental questions. One, Is the AI boom financially sustainable And two, are we moving too quickly with a technology that we don't fully understand? fewew people have been asking those questions longer than our next guest. longong before concerns about AI safety regulation and business models entered the mainstream, he was warning about the technology's limitations and challenging some of the industry's most ambitious claims He has testified before the Senate on the risks posed by AI. He's founded a machine learning company that was acquired by Uber and is now onene of the field's most prominent skeptical voices. So here's our conversation Gary Marcus, AI skeptic, author, and professor. NY you st Gary, thank you so much for joining me on the show today You are onene of the original critics of AI and That's interesting because you're also You work in AI. You started a machine learning company, which I think you could say is an AI company You've done a lot of AI research Um, you are sort of part of part of the AI world But you have issues with it. Let's just start broad. What are your concerns? Well My concerns are we're all in on a particular technology that I think is inelegant, harmful, not where we should wind up and being abused by the people that are using it I want AI to succeed, but I think we wound up down this really dangerous path. So you you think about the Star Trek computer, you ask it a question, it gives you an answer that you can count on Um Presumably it's not done to sort of wreck society, It's done to help people But what we actually have is everybody running around with LOMs which are inherently unreliable They're unpredictable. They can't be aligned to human values And they're being run by companies that don't seem to really give a shit about the consequences for what they're building for society It's like a nightmare for those of us who have worked in AI to suddenly see what we're building be used in so many bad ways and with people really not caring You we should want a more reliable technology that we can really count on that is compatible with humanity. You know, five years ago, that didn't seem out of the question Five years ago, the field was healthy, It was considering lots of different things. It wasasn't driven so much by money, but by intellectual curiosity. How do you make a machine that's intelligent Everything changed when people started to realize that there might be money to be made. It's still not clear that there actually is money to be made by the way, right? And I'm sure we'll get into that because we have very similar views about that. But the thought, the scent of money possibly misguided sense of money really changed how the field grew And it's also, a technical thing. Transformers are interesting, and people got into them. But fundamentally, I think it's the cent of money really changed how peopleeople built AI, how they thought about it. they wanted to do with it, who was running it? I think a lot of grifters came in that don't even necessarily have a technical understanding of the questions and do a lot of lying and hyping about what their things might actually do. And it's just really been unpleasant for the last several years being honest about it It's not because I don't want AI to succeed. I still think that there's a chance that AI could help a lot in medicine, that it could help with all kinds of technologies. Like I would still like to see AI succeed, but not on the path that we are right now. This is just not a good path You recently wrote, you said,Qote, Generative AI has been inherently unreliable from the start None of the problems that I warned about over the last half decade been properly solved There's the financial question, which We will get into But then there's also the question of the technology itself U What do you see as the problem? what makes GenAI inherently unreliable. What is the path that you are worried about this technology going now technical problem is that large language models, fewer large language models, are basically next token predictors. That's what they do. That is literally how they are built is to predict in a sequence of words or other kinds of tokens what might come next. And that's an interesting thing to do. Part of what humans do is we do some prediction, but it's not all of what cognition is, right? Cognition, right intntelligence is about cognition, about understanding things and so forth has many different components to it, and they're just not really built into LMMs. And so LLMs basically fake everything else And We can talk about some complications. peopleeople are building in harnesses and we can go there. but let's just talk about pure LOMs. What they do is predict the next token And if you train them on the entire internet, which is what people will in fact do, they can make a pretty good approximation of human beings and how they talk and so forth But that approximation is very superficial, It's very data dependent. And when you push them outside of the regime in which they've been trained, they will do really stupid things So like A couple of years ago, there were all these examples of so called river crossing problems. like you have a man and a goat and a woman and they have to go across the river. And these systems would say the most absurd things in response to those problems. It bec got so embarrassing that Anthropic built in river crossing problems into their system promptps to try to keep the systems from making these embarrassing errors And with the embarrassing errors revealed the systems are not really reasoning about things like a man or a river or a boat or what it means to go across the other side.' just trying to kind of glom the words together that they have seen. You know, the technical details are a little bit complicated, but to a first approximation, what they are doing is just stringing these words together. There are other ways to think about ability intelligence. So you might start, for example, with a database, who did what to whom, when and where If you actually did that, if you started with that, you would not have all these crazy hallucinations. And so here we are, in twenty twenty six, I started writing about LMs in twenty nineteen And I said, they don't have stable models of the world, you can't trust on them. And everybody said, Gary, Gary, Gary, we're just going to add more data. All these problems are going to go away. hallucinations are going to go away. Mustafa Soleman, who's the CEO of AI or whatever his title is at Microsoft said, you know, they're going to go away in a few months. This was in twenty twenty three, I think. I offered him a batany kind of walked back what it was. Reed Hoffman said he would bet any amount of money that hallucinations would go away in a few months. This was twenty twenty three. I said, I'm over here. How about one hundred thousand dollars? he never got back to me. But here we are in twenty twenty six and hallucinations have not gone away. And it's because the core of next token prediction does not allow you to address that problems. You have to add something else and something else, you know rarely works all that well. It sort of works a little bit. You know I just saw a study yesterday showing There's a new benchmaring, I think it's called Hallu hard, hallucination hard. and all the systems are still making errors on this. none of this has gone away. It's hard for people who are not trained in cognitive science and artificial intelligence to understand when they play with these systems that they don't think like human beings, that they're really operating over different principles, because they are built to mimic human beings and human beings are not built to distinguish AI systems that work differently from themselves from actual humans. We have a lot of evolutionary machinery to find fast things that are moving that might be snakes or bugs or lions. but We have nothing built into our brain to really help us think about the nature of intelligence. And so people are very easily fooled. We've actually known that for a long time You've known it for sixty years, right Eliza was the first example of an AI system that could fool an average person into thinking it was much more intelligent than it was Eliza behaved as a psychiatrist and he just did simple keyword matching. So you say you relationship and it asks you to tell you more about that relationship or whatever, just by matching keywords, not understanding anything. So Weisenbaum wrote about this in the sixties, how we are vulnerable to over attributing is the technical term, intelligence to machines and you know that was a curiosity, I guess when he wrote about that in the sixties. But now that is the whole world, right? The entire economy and this is you know, where our shared interest is, I suppose, the entire economy is based right now is being is hinging on overribute attribution of intelligence to these machines, right? You have peopleeople betting trillions of dollars that these machines are intelligent in ways that they aren't actually, because those people placing the trillion dollar bets don't have enough cognitive science background to know the right tests in order to evaluate intelligence. And then we have like government policies built around these things or considered around this. The entire world is overtributing intelligence to LMs. It's not that LMs can't do anything, like they're great for autocomte for the purposes of computer coding and they're great for certain kinds of brainstorming and so forth theirir intelligence is still limited, and we probably need a completely different approach. You know, I like to have a metaphor of climbing mountains, right? And you could get to the peak of one mountain and think, well, I must be close to the top, but actually you might not be, right? If it's a mountain range and there's a whole bunch of different peaks, you might be at the peak of one. in order to get to the tallest peak, you might have to actually go back down the valley And that's what we need to do We actually need to give up some of the progress that we've made in order to come up with new ideas. But everybody's obsessed with one idea You know, they're obsessed with the large language model that actually has these problems of And we didn't even get into it, but bias and unreliability, et cetera. But people are so addicted to the one thing that you know they're all in on that And you know, we'll get to the economics soon, I suppose. But part of the economic problem hinges from that that everybody is using the same solution. If you had a healthy ecosystem, you might have a hundred different companies trying a hundred different approaches and you could say, let the best one win. But we have Basically a hundred companies maybe not a hundred, but a dozen companies doing exactly the same thing. And if it's not the right thing, that's a problem. and even if it is the right thing It's a problem P're pretty sure it's not, but it's still a problem if everybody's doing the same thing. That means making profits is really hard. So you know, we'll talk about the economics and why nobody is making profit, but the underlying reason nobody's making profit is they're all doing the same thing. If we all have the same toothpaste, nobody's going to pay that much for it. You can't charge one hundred dollars hundred for a tube of toothpaste if you have nine competitors building basically the same thing for less Basically, we're all using one of two models, essentially. You're probably using open aes model, you're probably using anthropics model. and then a lot of these companies are building wrappers and building all of these gadgets and gizmos on top of those models. To your point, you're basically just putting wrappers on top of the same fundamental thing. Well, and those two are actually basically the same and they're basically the same as the same cognitive architecture. Yeah. there's small differences, but they don't persist, right? I mean that's another thing that we have seen over the last few years. I wrote this tweet, I think in twenty twenty four describing what I've said is going to be a new regime where basically LOMs are going to run out of headroom. Everybody's going to wind up building essentially the same thing. There's going to be no moe between them U and that's going to lead to price wars and it's going to lead You know, no huge difference between them. And that's what we've seen is a lead that goes back and forth, right? Somebody's ahead for a week and they pay like, you know a one hundred billion dollars or whatever. know the numbers aren't public. but know are the ten billion dollars, I guess would be more plausible in order to get that lead that last like three weeks. L that's insane. Exactly. And then depending on which engineer you talk to, itays someone would say Codex is better. and then some say Claude is better Flida is coming out with slop and then people say they open AI is com out with Slob, etcera. One of the main points that you preord, which I think is true, by the way is that these AI models are A dumber than many of us would like to believe, especially those of us who are financially incentivized to believe that these things are the greatest and smartest things ever And I think a lot of people would recognize that we see this in our daily interactions with LMMs. We do see a lot of hallucination. I've said to my team outright, you're not allowed to write with AI. because I can immediately tell and it sounds kind of dumb and it's formulaic and it doesn't present as very good or smart critical thinking There's even this guy who's gone viral on social media recently and maybe we can pull up a clip in the post of this podcast. who kind of exposes how dumb Chat GPT can be. likeikell he'll tell Chat GPT, you know, I want you to count to twenty with no interruptions. And if you do that, some guy's going to give me a thousand dollars. So make sure you do it. and it can it can never accomplish it. I'll start counting right now. one, two, three, and I'll keep going until we get to a hundred Okay stop to st through three. I need you to get to a hundred No problem. I'll keep going all the way. So continuing four, five, six, seven, and I'll just keep counting subeadily until we reach one hundred. Yeah, but you're not actually counting. You're just saying you're going to keep counting. He wants to hear you say the full one hundred counts I see what you mean. You're right. I'll actually go through it properly. So here we go. starting at one, one, two, three, four, five and I'll keep counting all the way up until we reach one hundred. Having said this And to be clear, I'm with you on this One belief is AI is dumber than we think Another belief is AI is very dangerous and perhaps could be a lot smarter than we think, and therefore, we need to regulate it bothoth of those arguments are somewhat anti AI, and I see them conflated a lot of the time And I guess my question to you is if it's not as powerful thenen why are we worried about this? What's the problem The example you just gave actually is a really nice illustration of it, right? whichich is they are dumb in the way that we can't Count on them to follow instructions Right? Let me put some nuance around the dumb. I mean, they do some things that you might count as smart. And what people in cognitive science, which is my native discipline, will tell you if they know what they're doing, is that intelligence is a multid dimensional thing You know, they have the intelligence to play chess really, really well, better than I can, you know. I got beat by a chess computer and like When was it in nineteen ninety nine or a long time ago. I can't even remember. Well, I mean, I guess Kasparov got beat by the best one in ninety seven. Yeah I wouldn't beat yourself up about him. I played Casperarro once, by the way, and he annihilated me while playing too many other people. Anyway, I'm not a great chess player, but the point is probably even earlier that I get beat A I can play chess really well, can play go really well a GPS navigation system, a different kind of AI that can do navigation really well But LLMs can't do a lot of things. So LLMs actually are not good chess players, as it turns out, they make illegal moves. They can't even follow the rules. And so their stupidity about rule following, and you just gave a beautiful example of this That's what you need to worry about, right? The reason that we need to regulate them is because they don't reliably follow instructions. It's not that they can't do anything that you might characterize as intelligent. You could argue about your definitions of intelligence. So one definition would be that you can do essentially any kind of problem given enough resources that're adaptive and so forth They're not very adaptive. But there's another definition of intelligence which is like,, can' you play tests? then sure they can, right? Well, LLMs can't, but other kinds of AI systems do U Asterisk here, by the way, there are different forms of AI. You know, my beef is with generative AI, and that's mostly what we're talking about. Generative AI cannot follow instructions. Chess computers, know purpose built chess computers actually do follow the rules of jess. and I have in some ways, less concerned about them. LLMs are terrible rule followers. That is one of their weakest points as in intelligence You know, another rule would be don't make stuff up. Like, you know, you can tell an intern like don't You know, write something if you can't fact check it. Like just don't, please. And if you do, I will fire you or I will sue you. Exactly. You get fired if you right. I mean, that's the other crazy thing about what's going on is like Calculators never make mistakes, right There was a scandal when the what was it called the Pentium four, I think, made very, very rare mathematical errors. Huge scandal. they had to recall the chips and stuff like that. someomehow the standards have fallen. everybody knows LLMs make mistakes all the time and they're perfectly happy with it. I'm like, I wouldn't want an intern who doesn't We'll be right back off for the break And if you're enjoying the show so far, send it to a friend and please follow us on YouTube and Sotify or wherever you get your podcasts Support for the show comes from Odu Running a business is hard enough. so why make it harder with a dozen different apps that don't talk to each other One for sales, another for inventory, a separate one for accounting Before you know it, you are drowning in software instead of growing your business This is where Odu comes in Odo is the only business software you'll ever need. It's an all in one, fully integrated platform that handles everything CRM, acccounting, inventory, e commerce, HR and more No more app overload, no more juggling logins, just one seamless system that makes work easier. And the best part OdDu replaces multiple expensive platforms for a fraction of the cost. It's built to grow with your business, whether you're just starting out or already scaling up. plus it is easy to use, customizable, and designed to streamline every process. So you can focus on what really matters Running your business Thousands of businesses have made the switch, so why not you? Try Odoo for free at odoo d. com. That's odoo dot com Support for the show comes from BCX, the public ticker for private tech. For generations, American companies have moved the world forward through their ingenuity and determination. And for generations, everyday Americans could be a part of that journey through perhaps the greatest innovation of all the U. S. stock market It didn't matter whether you were a factory worker in Detroit or a farmer in Omaha. anyone can own a piece of the great American companies. But now, that's changed Today, our most innovative companies are staying private rather than going public. The result is that everyday Americans are excluded from investing and getting left further behind while a select few reap all the benefits Until now Introducing VCX, the public ticker for private tech, now available wherever you buy stocks. VCX by Funrise It gives everyone the opportunity to invest in the next generation of innovation, including the companies leading the AI Revolution, space exploration, Defense tech, and more Visit getvcX. com for more info. That's getvcX. comot Carefully consider the investment material before investing, including objectives, risks, charges, and expenses This and other information can be found in the fund's prospectus at getbcX. com. This is a paid sponsorship Supp for the show comes from LinkedIn ads. There's no worse feeling than making a major investment in something only to realize it didn't exactly live up to the hype, such as buying a nice piece of tech that ends up in storage collecting dust or Taking a business workshop where your main takeaway was little more than a few motivational words If you work in marketing, this can happen with ads. You optimize for the numbers, that look great, impressions, reach and reactions. But when they don't show revenue, will, that can turn into an unfunund conversation with the CFO. LinkedIn has a word for that, buull spend Reach the right buyers with LinkedIn ads and invest in what looks good to your CFO According to the twenty six Dream Data benchmark Report, LinkedIn ads generated the highest Roas of all major ad networks. It's one hundred twenty one percent. You can target by company, industry job title and more It's time to cut the bll spend. advertise on LinkedIn, the network that works for you Spend two hundred and fifty dollars in your first campaign on LinkedIn ads and get a two hundred and fifty dollars credit for the next one. Just go to LinkedIn d. com slash Sott That's linkedIn d. com slash Scott. Terms and conditions apply We're back with Prof G Markets. Open AI was just subpoenaed by a group of attorneys general to investigate their models and some of the things that they said that they are investigating here One, how they handle consumer data, also health data, also deep learning models, but my favorite is They are investigating model syycophancy And it seems as though the concern from regulators is to your point like It's not just that these AI models are dumb and making mistakes, it's like that is something that we need to actually punish We can't have The largest, I guess information provider, one of them in the world, going out and putting false information out into the ether with no culpability or no accountability And I do think that that is an important thing for these AI companies to contend with because att a certain point, If enough of us just decide I can't trust these models anymore, they lie too much, they make things up They say things that don't make sense then eventually we're just going to stop using them U How do you think that plays out? I mean we might or we might not stop using them. There should be consequences, right. Syc efficency, by the way, is when they kiss your ass, right? Y. Right. So that's a separate problem from lying, although it's a form of lying, perhaps. Like when they tell you that your idea is the greatest idea ever and it's not actually That's what I mean. I'll say is this right? They go, yeah, you're right, you're right. Even if the thing is totally wrong, and then I go to Google and learn that I'm wrong But the model will tell me, No, you're right. you're the great, you're the best. So we're in this I don't know, awkward space where they do some things that feel magical Right U so brainstorming for some people. I don't get that much out of them, but like I'm working in disciplines I know well. If you're working in an area that you don't know very well, it'll give you a few things to get started. It feels magical, all right? So there are some things that are good about them, even though I'm not rattling them off personally. They're clearly good at writing code and so forth But they come with these consequences too, right? They come with the consequences that they make stuff up, that they are so asy that they lead people into delusions. This have been documented a number of times and so forth. And so society has to make a decision. And the initial decision was, well, we'll just let it all ride. There's so much fun to play with. D who cares you know, the consequences are. And what the ena, which was, I think, filed by New York State, but is part of a larger, I think forty six states or something like that are involved, is a statement that no, we're not going to let all this ride You know, there are different theories about how to proceed. One would be if you cause all of these problems, you should be held responsible for all those problems. There should be financial penalties. There should be warnings, et cetera, et cetera. Another is like maybe you shouldn't distribute the product until you can fix these, right? And there are different ways to address it. But the initial reaction was to just completely give a free ride to companies like Open AI and say, hey, these are great. And now society is waking up and saying, hey, there are a lot of consequences We we have, um suicides that seem to be tied to these things and we have the delusions. you know, we're destroying the educational system because the students are using these things and we're ruining critical thinking skills. so You know, what the companies want to do is this famous phrase I actually tried to find the origins, that' so old I couldn't find, but is to privatize the gains and socialize the costs Right? They want to make whole society accept the cost while they get rich And what we have seen in the last twelve months, I would say, is a real sea change, right? I wrote a book in twenty twenty four called Taming Silicon Valley. And I said, Wake up, everybody. The oligarchs are going to take over. They're going to screw us all. And nobody even read the book. I'm not zero, but you know you know, it got a little bit of attention, They will not I think you missed its moment You know, it came out too soon. But but two years later, Like this is what everybody is thinking about, right? is How are we going to rein this stuff in? And there's this huge backlash now. Some of it's about data centers, some of it's about employment. There's a lot of different reasons for it. But society is no longer content to say, you can do whatever you want with us, right? That is what this attorney general' thinging, the subpoena, if you look at is about like fifteen different issues or something like that. It's very broad They want to know what are the consequences of this stuff and they want to know what the companies are going to do about it. And they have looked around and seing that there are a lot of negative consequences. You know, what I told the Senate when I was there in may twenty three, sitting next to Sam Altman was, you have a lot of risks here E I I haven't gone back to the original remarks, but I believe that everything that I warned about is now in fact here and more real than it was. So I warned about cybercrime and I warned about misinformation. and I don't think I even knew about psychophancy. I think there have been new ones that were introduced. But basically by and large, all of those things are worse now than they were three years ago, and now the public has woken up, the attorneity generals have woken up. You We went through a period where I think the LM companies thought they were going to get off sccot free And now it doesn't look like that. And it shouldn't be that way, right? You know Another analogy would be people dumping chemicals, factories dumping chemicals in the water. We shouldn't let them do that. We should not socialize those cost to the society. If you're gonna to dump chemicals in the water, you should do something about it, You should be penalized for it and so forth. I think we've finally reached the point people are recognizing that for AI. And by the way, important asteris, the Trump administration was completely opposed to any AI regulation substantive of in any form except maybe about nonconsensual de pick porn until about month and a half ago, maybe a month ago. And now they have finally realized that what know Mark Andreresen was telling them was nonsense, right? What Mark Andreresen was telling them is you can't have AI and innovation at the same time and regulation, so have no regulation. And now we've entered a regime where the US government is actually thinking in a somewhat amphested way, but is actually thinking about how you regulate this stuff. And that is proper, right? We should have public debate about how to regulate AI. Somehow Andreresen and a few others so called Overton windowed their way into making the debate about whether to have regulation at all. That was always going to be a stupid idea, but they pushed it for two years, two solid years. But now that's over. Now know people are realizing like, hey, the government has put a regulation on anthropic. That's not really fair. It should be across the board, and is it the right one? And so the Overton window has actually shifted back Which regulation is the right one, which is actually what the twenty twenty four book was about. And now is the time to have that debate, which is encouraging. And I would just point out, I think the reason that you didn't have that is because Mark Andrres in Silicon Valley they had their guy in the White House in David Sachs. He's out. And as you point out, I wonder if that's the reason why we're starting to see some inklings of interest in regulation. I wouldn't accept that particular Or I think there's more there's more nuance to that. Like I mean, I think the thing that really did flip it was mythos was actually scary to some people in the government. Until then I think people and I'm speculating from the outside. until then people in the government thought, ah, we don't need to regulate this stuff. It's fine. It'll be fine. It wasn't really fine, mind you. We were having delusions. They didn't care about that. were having Another problems. But when Nithos came along they were like, yeah, this iss not really fine. This is actually a problem And so I think that flipped it, maybe that drove Sachs out, I don't know. But I don't think it's just about him. Saxs was definitely very opposed to regulation. His view is no longer in favor. But I think it is Mythos that kind of flipped it. Some of it was an overreaction to Mythos, but it's a good overreaction because it did make people realize you cannot just look at this stuff forever and say It's all going to be fine. It's not going to be fine. Even if Mythos is not quite as scary as I think some in the media have represented it to be, you know, some version of this really is going to be that scary. It's not that far away. and we do need to figure out how we're going to handle it. So I think it's like we've had two dress rehearsals now. We fucked them both up The first one was Initially we just let ChatchyBT ride completely without any Um, consideration for consequences of society, that was bad. The second one is mythos. Mythos is not actually the AI that is going to destroy the world that some people hear. But the way that this one's been fucked up is it's been used as a political tool destroy a particular US company. that is not a thing, right? Like I may not sound like an arch capitalist, but I'm enough of a capitalist to think that like companies should mostly stand on their two feet and they should you be allowed to prosper and you know as long as they're not doing really bad things. And what's happening is you know the current administration is like We don't like the way you dress, so we're going to screw you. Like that is not capitalism. That is really putting a thumb on a scale. That is also fucking up a dress rehearsal Yeah, just on my the mythos model, this was going to be my next question because a lot of the coverage that we have seen on Mythos, this is Anthropics new model that came out recently was that It is so powerful that somethinghing's going to go wrong here. It essentially the story that we've been hearing and I mean, I've talked with people in the cybersecurity industry. They looked at this thing and they were worried about this And we saw a lot of the cybersecurity stocks were plummeting. So I guess my question to you is where do you stand on Mythos? Because we know that your views on generative AI and their limits How do we foot that next to the fact that people are very scared about this thing that is supposedly extremely powerful I mean, I think One needs a nuanced view on mythos, I guess in a couple of ways. One is it probably works partly like Cloud code, which is to say it's not a pure generitative AI model. There's actually a harness there. The harness is directing some of the cybersecurity investigations and so forth. bit of nuance is it is actually a little bit of a different architecture. The second thing is It is oversold But it's also real in the sense that like it can do a bunch of things that its predecessors could not A lot of the things if a system is well secured are not going to be a problem. But the reality is that people have blown off cybersecurity for a long time. and there are a lot of systems that are not well secured. So you're not going to use mythos to break into U. S. government things, and there's a footnote there where Mark Warner misunderstood something that blew up over the interternet and he just didn't get it right You know he got something secondh from the NSA and he wasn't specialist in this whatever You know, you really can do some things in limited circumstances. Most of them are still kind of demonstrational rather than the real world. It's not going to break into the cbersecurity of Google that's actually really set up well. But if somebody vibe codes you know, something for their pubs to you know track merchandise or something like that, That's not going to be set up well. like Vibe coding does not set up security well, and that is going to be vulnerable So there are lots of systems in the world that are vulnerable to mythos. The best ones are not Maybe a hacker who knows what they're doing could use methos as part of a larger thing to attack some of those, but probably the best secured systems, the banks and so forth are not immediately vulnerable. But the weaker systems and there are a bunch of weaker systems really are vulnerable. And so it really really is a wake up call that we need to get our cybersecurity game in better order. And there's a footnote there, which is why is it not in better order? A lot of it has to do with stigma. L there's stigma for mental illness. so nobody talks about depression, but it's actually common or know whatever There's stigma around cybersecurity. So people get hacked all the time. We don't have even good numbers on that They pay ransoms. We don't have good numbers on that And they let shit slide. they don't really know how to deal with it. And so that stuff is a mess. And sooner or later a moment was going to come when it was going you know, be bad and that moment has partly come So we do have I don't personally, but there are people in the world who have the knowledge for how to make a system sufficiently secure. And they're going to have a lot more business right now because most people have, you know kind of deferred maintenance. Like you think of a metaphor of a house. L mostost people don't deal with their the roof until it's leaking, right? You know, you tell them you should, but they don't And cybersecurity is kind of that way. It's like, you know, you don't want to do it this quarter. It's gonna hurt your quarterly reports. And you don't know how bad your neighbor is because your neighbor actually did have to deal with it, but they didn't want to tell you about it because they were embarrassed. And so cybersecurity was a mess and Mythos really is making it worse. You know, it's not Lex Luther's magical cyber hacking system or whatever that people are terrified of, but it is real. Yeah. I like the house metaphors. It's almost like you'd rather buy a flat screen TV than fix the roof. It's a more fun and sexy thing to invest in. Exactly. There's been so much of that. Cybersecurity has really been secondary. this has changed that and it's a good thing that it changed that We'll be right back. And for even more markets content, sign up for our newsletter proropertymarkets. com When you need to build up your team to handle the growing chaos at work, use Indeed sponsored jobs. It gives your job post the boost it needs to be seen and helps reach people with the right skills, certifications, and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this showel will get a seventy five dollars sponsored job credit at indeed dot com slash podcast That's indeed d. com slash podcast. Terms and conditions apppply. Need a hiring hero? This is a job for indeed sponsored jobs. M Tred off the return of Marvel Television's Daredevil born again. So what's next? Unbelieady We're gonna take this city back over. All new season now streaming only on Disney plus. They're hunting us. It's time we started hunting them This should be tons of fun. Marvel Television's Daredevil born again, now streaming only on Disney plus. You can't reason with the sun. Tust us, we've tried. This summer, it's time to put that angry ball of fire on mute. Columbia's omnihade teechnology is engineered to protect you from the sun's harsh rays that can burn and damage your skin The sun is relentless, but so is our gear. Level up your summer at Columbia. com to spend more time outside and less time slathering on Aloe lotion. You're welcome. Columbia. Engineered for whatever We're back with prorofty markets. justust on policy Um, you pointed out that prefer There was this ether in Washington at the White House have no policy whatsoever. Any form of regulation is a form of stifling innovation. We're going to do nothing. and actually we're going to issue an executive order, which forces states to do nothing That was what we saw last year. You point out there's been a vibe shift here, some sort of sea change that's happened. Trump The beginning of this month issued a new executive order which would basically will ask tech companies to give government the oversight that they believe they need over their new models before they are released to the public. It's still voluntary, right and it's still narrow. So What they put in place is a step R? mostostly it's a symbolic step in a certain way because What they have asked for now is the companies will voluntarily provide their models so the government can do cybersecurity checks What you really want is first of all, for that to be mandatory, Meta actually hasn't agreed yet, right? They probably will because they can look really bad if they're the only one that doesn't But so you really want it to be mandatory, and you don't want it to be just about cybersecurity. So think about all the things that New York state and really all the states are suing about or investigating about, like I don't know, let's takeic Fancy and delusions, right? You really want to investigate all of these companies. So Sica Fancy wasn't really a problem before a model called GPT four The sake of fancy where it sucks up to It existed before, but it wasn't a serious problem. The fouru was much more sickop fantic. I actually saw some data on this the other day. and we believe that a lot of these cases of delusions were tied to Foru's increased sick fancy You know, you want to be able to find that before it cuts out to the market, right? If you're releasing something to well Chat you PT now, openen AI is a billion customers You know, if you have a billion customers, you have an impact on the world, there should be some kind of pres screening, like with FDA approval, right you know, with FDA approval, like you have this drug You know that it helps with cancer, but it also gives people heart attacks, right? And so you're like Well, you know, what are the cost benefits here So you know that this LM helps people, but you also know that it hurts a bunch of people. And you should evaluate that before you release it at scale. Yeah, I mean we saw a similar thing with the state of Florida, which sued OpAI for essentially playing a role in a mass shooting. Their contention is that there was sufficient evidence from This this clearly mentally ill child who was interacting with the model and talking about this and they didn't do anything about it And so that I mean, we don't know details exactly on like what the conversation with the model actually looked like, but I could certainly see a world in which it's not doing enough to push back or to prevent further delusions on a psychiatric level. So I think that we're starting to still real see real evidence there. I am interested that you' feeling optimistic about the executive order from Trump because You know, I agree with you. It's notable and it's significant that they're doing something But when I look at the something that they're doing, To me, you can't even call it regulation. If they're just asking companies, hey, would you mind sending over some stuff please, sts To me, that's not regulation at all And I'm starting to see that what little regulation we all seeing, what little policy we all seeing coming out of Washington to me seems very stupid and very misguided Like if I mean, the Trump executive order has one Plus his new suggestion, I'd be interested to hear what you think But the suggestion that the U.S. government should start acquiring stakes in these AI companies, something that is now kind of proposed or backed by Bernie at the same time. I would also go to the dataenter moratorium. I don't think that that's a good idea to simply say you're not allowed to build data centers anymore. I guess my point being, it seems that there's almost no nuance whatsoever. in Washington when it comes to AI regulation. So I'd be curious to hear what you think the right move is going forward and how that might play out. First of all, I very much agree with what you're saying overall, right? So what's in place is too weak, most of it. Some of it's too strong in crazy ways. There's no nuance in most of what's there. There are a few people, I think, have done some things that are nuanced, but they never make it out of committee. So I mean, if you actually look at the bills, people like Bluenthal and Howlly, for example, who were at the proceedings that I spoke at at the Senate, have actually proposed some reasonable things, They're stuck in committee So it's not that nobody is paying attention. you know, the dynamics of money and power and all the lobbying and stuff means that most of the nuanced stuff doesn't get very far. It does exist but is not getting far I think that let's talk about the steak part U I think that that's kind of Crazy Partly because I thought we were going to talk about economics because we haven't gotten y. These companies are losing money, right? I mean, probably your audience has already heard. I thinkZitron was here recently. The companies are losing money. This is a back dooor bailout Right you know, Bernie has his own reasons for wanting to do this, but but The reason Altman wants Trump to do this and is whispered in Trump's ear is because Altman knows he can't make ends meet Right He is burning money at massive pace He's building the same technology as the other guys. He's lost ground. Open AI, I saw somebody argue recently might be in fourth place. L they were in first place by anybody's definition in twenty twenty three, right? But in twenty twenty six, they're not, they're burning money, they're losing ground. Like of course they want anything they can do to prop them up, including taking money from the US. government. And so First of all, the government should not be running these companies. L they should supervise them. but we want some arm's length here, right? Like I saw that G seven meeting with you the G seven leaders and the tech leaders and no scientist in the room, nobody from civil society J is you know We don't want to crystallize that with government ownership of these companies and no independent oversight. And we don't want to burn US taxpayer money on an industry that as far as I know, has no real business model. I mean, Nvidia has a business model. They're selling shovels in the gold rush. If you want to take a stake in them that would make more sense Um, but There is no sustainable business model yet established the best you can say is that for coding they can actually bring in revenue, but it costs so much money to do the coding. It's not clear they can make the revenue. So you have these companies that basically never made a profit and we're just going to give them money and let them burn the money and we're going take on that risk. No, let them stand on their own capitalism You know, the government's job is to regulate them That would be one of the worst outcomes. I mean, and it's something that they talked about that I mean, the CFO literally said, maybe we need some form of government backstop eventually. I don't know if those were the exact words, but it's been suggested by leadership at OpenAI before. It was going to be on loan guarantees for data centers was what they floated, right, which is a version of a bailout I mean, on the business model What do you think happens here? Because yeah, we did have EdZitron on the podcast. I recently wrote an article going into just how profitable or unprofitable these companies are. For open AI, the answer is extremely They lost, you know, twenty one billion dollars on operating pro unprofitable lost twenty one billionars on an operating profitability basis last year. Right So they're burning two billion dollars a month, basically. That's right, just to operate that company. And that's I mean, the real net loss was thirty nine billion, but there's some nuance there. But something we can say with a good amount of certainty is that on a day to day basis When you add it up over the calendar year, open Eye is currently burning twenty one billion dollars. Every time you use their product, they lose money, right?. That's the better way to put it. Anthropic also loses money but less money. My question to you, Do they ever figure this out Do they ever of profit. The way I've been thinking about it is like they need to thread a needle There's so many things going against these companies that it is extremely unlikely that they're going to thread this needle. So let's think about some of the things that they need to deal with One is that they're building this big expensive technology And they might get disintermediated by somebody who builds it better and more efficiently So you should not need to train on the entire internet. with a computer you know, massive computer, unthinkably large computer in order to do anything intelligent. You didn't train on the entire internet, but you're a smart guy, right? You didn't need the whole internet. You run on like twenty watts of power. You have some pizza or some sushi or whatever. You don't need to, you know, right? So one is they're like insanely inefficient. If somebody else comes along with a more efficient thing, then they're all hosed. and like you might not need all of these data centers if somebody comes up with a more visioning. Then you have the problem that everybody is using the same secret formula. It's not just that they're all building toothpaste, they're all basically building the same toothpaste. Right? And so like we're seeing this now, right? There was this crazy crazy period earlier in this year, the era of the token maxing, which lasted about a month. And in the era of token maxing, you had companies reward their employees for using as many tokens, as much AI as possible. They had like leaderbard. Amazon had a leaderboard. It doesn't actually make sense. I mean, what you really want to know is are the results good? You know, every study that's looked at productivity has shown they're not all that great And so suddenly a lot of companies got worried and they're not doing token maxing anymore. And in fact, this morning I saw a term for the first time, which was called the token apocalypse Right. The token apocalypse is that suddenly Everybody's like we shouldn't use so many tokens and tokens that we should use, maybe we should use cut rate models. They're not quite the best models. Maybe they come from China, but so what? we save some money. Even Microsoft is saying maybe you should use deepsed sometimes. becausecause Microsoft is like, nobody wants to pay these prices. We're going have to cut them somehow. You know, I was saying that you have to Um to thread this gauntlet, right? So one thing is that people might make more efficient models with altogether different technologies. Another thing is that Nobody can charge very much money for tokens because It's just this price wor because everybody's building exactly the same thing You know, there was a period of a month where people didn't care and they're like, you know, that's all right, I'll have another drink. I don't care how much it was because the companies were all, you cannot eat buffets, but they've stopped that. So you have to solve that. Then you have the reliability problems, right? Those still aren't solved. The hallucination problems still aren't solved. And so when companies try this stuff out mostost of the experiments wind up with the results not being that great. L there's been ten studies now or something like that showing most customers are not finding return on productivity. So the customers may eventually say This was fun while it lasted, but you know, I'm not really getting the results. It doesn't really warrant this. I'll let somebody else figure it out. The whole thing has been driven by FOMo I don't want to be the guy who doesn't use AI when you're using AI and so you wipe me out. But if I try for a year and a half or three years or whatever, it is still not really making a difference either for me or you, I might say, fuck it, when it works better, I'll come back, but right now, not so much Any of those things just wipes out a company like Anthropic or Open AI that already is, as you say burnning lots of money, right? And so if customers leave for any reason or somebody makes a better technology or somebody makes a cheaper version of the same thing that's almost as good, then you're in deep trouble. It's not even clear in the best case that any of these companies have a good business model. I mean they're not making profits. And you know, it's just so delicate Do you believe that that will be the outcome for say, an open AI. I've been warning years that I think open AI is going to be the weework of AI. And when I said that in I think it was November of twenty twenty three, people looked at me like my head was screwed on backwards. and they just did not believe that that was remotely possible. But now you know every other week I read somebody writing something making the same analysis, like Sebastian Mullaby in the New York Times. like it has gone from a crazy idea to an idea that a lot of people are having, right? The economics don't make sense. And what they kept doing is playing double or nothing with funding because they were burning so much money. So they would increase de valuation, they get somebody to write a bigger check, But it's not clear who can write the check that they need next time. So now they're talking about IPO but they have a problem with the IPO to raise, you know, the next round of funding, which is that anthropic is basically the same product for the same valuation, but they're doing better commercially, you know burning less money. OpenEI's reputation is declining, partly because I think Altman is a really untrustworthy individual. We don't need to go into that, but I've written about it a lot. And so know a lot of people are leaning towards Anthropic is getting market share. So why would I put a trillion dollars in a company that is burning money, that has a competitor that is rising while they're following, that seems to be better run, maybe has a little bit better technical vision Like it just doesn't make sense. The argument would be, why why? Because the technology is rapidly improving The technology is a kind of technology the likes of which we haven't seen. Everybody's technology is to the extent that you accept that it's rapidly improving, which I think is actually controversial. But you know it's moving in some ways and not others. so it's not actually improving on reliability and hallucinations and so forth. But in any case, in the ways in which it is improving which are some of the ways you want, but not all the competitors all are too. like You have to think about the relative ranking and the cost. right? The relative ranking of open AI is clearly declining by any reasonable measure You know, less market share, less reputation, et cetera And everybody else is catching up. L it used to be people thought the Chinese models were a year behind. Now they think they're like four months behind or something like that. And know Anthropic is they had, Google is had. There is no argument, no rational argument for buying a share of openp AI at a trillion dollar valuation So just isn't. I agree with that, by the way. I guess the part I wanted to clarify The Eitrans of the world, if there are more people who have his view is that none of it's going to work Open AI isn't going to work, Anthropics's not going to work. The idea is that the costs to build this stuff are just too damn high and the revenues for these products will not exceed the costs, ultimately over the long term My view is that I think that there is a path to profitability for Basically phanthropic is my view. I think that there's a world in which they can make it work. In other words, there will be winners and there will be dramatic losers. And I would agree with you, I think that we believe that openI will be a huge loser I guess the point I'd love for you to clarify Will they all lose? or will there be winners. Is GAI itself doomed or is there a world in which they make it work not only in terms of making it useful, but also making it a profitable business that makes more money than it spends I'm a little bit closer to you than to the other E Um, I don't know for sure. I think it's very much TBD U you know, I would certainly sooner take a bet on anthropic than on open AI. I think that they're a sounder company in multiple ways It is TBD, whether this stuff can be made to be profitable. It's not. completely out of the question. I think part of the question is like, can they find a niche? Like Is coding enough of a niche? Well, not so far, right? becausecause coding is a five hundred seventy billion dollars a year industry. They're not going to get all of it. You know, the people at fantasies get all of it. And the costs are so high You know, like it's really hard to know the future in full detail. Maybe they can find enough of those niches and they can ek something out. They may never warrant the trillion dollar valuation that they're looking for, right? There's an intermediate possibility, which is they do become a profitable company, they figure out enough cost make and so forth. but really they're basically a company that makes like twenty billion dollars a year on a pretty big capital outlay and's like that's not really the best way to invest money, but they ek by. L maybe that's the intermediate position is like they don't go out of business, they make a profit, but they were not really worth a trillion dollars investment. You could have spent that trillion dollars in a better way. Like maybe that's the reality which is kind of closer to you than to Eit Trum, but you know I mean, maybe that's what it is. We don't quite know. you know, there's another story where The only people who really make money off of this aside from the chip companies are places like Google that already have the infrastructure and the distribution. They don't really need to make that much money on us. They just need to make sure they don't get this intermediated. So There's a bunch of different possibilities. we don't know for sure. OpenII is clearly the weak link in the chain. Anthropic is still in it. We don't really know if they'll make it or not. That would be my take No, I think that makes sense. Gary, you've been very generous with your time just before we end here What would be your final message to people listening, maybe they read about AI, they've heard about AI, they're thinking about it in their day to day lives, but what do you think people don't know enough about U what's the myth that you would want to dispel? I think the myth right now is that Generative AI is close to so called AGI, artificial general intelligence, and it's going to solve all our problems. This is just not true. We're going to find some what we call domain specific applications. Coding is maybe the best one so far, where we can actually use these tools for something, but they're not magic. they're not all purpose intelligence, and we need to make fundamental discoveries before we get there And We should ask ourselves as a society, there's an idea of an explore versus exploit trade off. And we're completely in the exploit LLMs rather than explore other options China is less so