Remembering Om Malik and His Legacy

It's time for intelligent machines. Jeff Jarvis is here. Paris has the week off. Mike Elggin sits in and we've got a Great guest, linguist Chris Potts specializes in finding AI failures. How to know if your AI is failing, how it fails, and what to do about it next on intelligent Machines. This episode is brought to you by Black Hat USA. If you listen to this show, you go deep on the technical detail. Well, so does Black Hat. For nearly three decades, it's been where the security industry's most rigorous research gets presented and pressure tested More than a hundred hands on trainings taught by practitioners who've actually deployed in live environments, not lecturers reading from slides, and hundreds of peer reviewed briefings that go well past the overview into the real work across the four areas defining security right now. AI and autonomous threats cyber conflict systemstemic resilience and identity This year, Black Hat's Briefings pass includes all keynotes and main stage access, plus business hall entry You also get breakfast, lunch, arsenal live tool demos, on demand session access, and admission to the midnight in the warroom screening Black Hat takes place from august first to the sixth in Las Vegas. If you want the depth this show gets into in person, with the people doing the work This is the room And we'll be there too. Prices rise on july seventeenth, so book before then, Use code Twit for two hundred dollars off your briefings pass at black hat dot com slash us dash twenty six. That's B LA C K HAT dot com slash US dash twenty six Podcasts you love from people you trust. This is trit This is Intelligent Machines with Jeff Jarvis and Paris Martinau. Episode eight hundred seventy seven Recorded Wednesday, july first, twenty twenty six. Model now available It's time for intelligent machines, to show we cover the latest in AI robotics and all the smart little Dude dad are getting smarter all the time. I got this little guy. just he calls China and talks to it all the time. I don't know what they're saying because it's Chinese But that's that's pretty smart Welcome to the show. Jeff Jarvis is here, Professor of journalistic Innovation Emeritus at the City University of New York Graduate school journal A his new book is only weeks out now. Fally, we're in July next month's hut type In the hottest month of the year comes out The story of the line of tyype that you can order it now though at Jeff Jarvis. com Paris is Jeff, I have to steal a joke from a former student of mine, Lucy Lee. She said her dream job is to be prorofessor Eeritus, and I agree. Exactly As not as I say it's Latin for old. Ito is not Latin for bigig Sabbatical in my view. Sabatically, welcome, Mike Elggan is fill in for Paris Martinau this week. It's great to have you, Mike. What country are you in today I am in the UK. Oswalds and tomorrow my wife and I are going to take a road trip to Scotland. So I' super excited about that. All the all the FIFA fans will be back by then we're trying to beat them there over a little Uskaban. Yes, you've already met our guests. We're thrilled to have Chris Potts on He's a professor of linguistics at Stanford on Sabbatacical right now Chief scientist and founder of a company called bigigspin. Ai h but you know He's also and I think everybody should read his PhD dissertation, The loogic of conventional implicatures because I think that will then you will understand fully what we're going to talk about. No, I'm justidding Hi Chris. Welcome Thank you. So good to have you your timing u is excellent U What an amazing time we're living in. You originally were interested in neuralinguistic programming, right? LLP or no Natural language process,atur language for sistinction other NLP Yes, R Much more interesting, frankly. And natural language is kind of what we're doing these days with our AIs. He was at Sle, the Stanford AI lab created a co creator of foundational open dataets. The Stanford Sentiment Tree Bank, the SNLI Natural Language Inferests C Corpus and a creator co creator of D Spy which is the framework that I think significantly reframeed prompt engineering is actually coding So before there was vibe coding, there was you I'm the visionary project lead for that Omar Katab, my student with Mateesa Hara. Yeah. give Omar the credit there for sure He saw the future. A lot of what we do these days with RG, although RG went through really a brief period of of fame and infamy and now it's kind of I think a little bit deprecated, but you were the co author on Colbert the Basically the original rag, right Oh yeah, well you could break that apart. It's a pioneering neural information retrieval architecture, which is a key component in many of the retrieval augmented systems. It would be the mechanism that would find passages to clunk into your context for the language model to use Incredibly important piece. And in its own right, Colbert is an incredible contribution to information retrieval, which is of course the backbone for web search and other technologies like. I didn't realize it was French, I mispronnced it, Colbert. Oh well, no, so we could talk about that, right? And you could talk about what Stephen Colbert Colbert actually says about his last name. Omar happily accepts Colbert Colbert. I think he's happy with DSPI or DPI Whatever variant as long as you're using it is happy. Very, very cool. that's all loguistics. You just kind of let the cat out of the bag on this new company big spin. tellell us a little bit about what bigig spin is all about so we can get into the conversation here because this is fascinating. Sure, Ebody should know that a bigig spin is a skateboarding trick An an advanced one in which the board spins three hundred sixty and you go one hundred and eighty and it involves some real danger. You got to have faith that you're going to land back on your board It's beautiful when it's done well. Wow. And I guess it also for the skateboarders references an old California lottery. so it also has an element of risk. That's right, That's right ye So That's the beginning and the end of the relevance of the name to the the company's mission. Okay. It's a very dense space and you don't want to choose something like Prism or whatever because then there will be fifty companies who've chosen the same name. So we just changed the game and with something that has good connotations for us But the focus of the company is very important for people who use AI. yourour research is fascinating. Go ahead, that's wonderful to hear. Thanks. Yeah.. Yeah, we are focused on making sure that People's interactions with AI are productive for them, no matter where they are in their AI journey. And in particular, what we're trying to do is help People in charge of these products figure out what's happening in that interactional sense meet users where they are. and help empower them And that all begins with finding all these low level signals of small points of failure or figuring out what the user is trying to do in that session and then providing affordances that will help them do that. and so forth and so on. There's an incredible opportunity here for customization And there's just much more that we could be doing for these products. So we're trying to push that along So your work started with analzing A million chat GPT conversations And in those Yeah, right. You're in those conversations, you found seventy eight percent of AI failures leave no trace. It's right in the sense that the user just did not give us an indication that they saw that something had gone wrong even though something had gone wrong Can I ask you a question there, Chris? Sure. I was fascinated by that There's There's two sides to this. Does the user give a signal that have went wrong is the AI company generally set up to Listen for that signal and act on it or are they using that kind of interaction to fix and train O are they putting their hands around their ears Well, that's funny. I tend to assume that they are smart, creative, motivated people and they're doing very advanced things. But we did get a glimpse of at least one thing they're doing when the clawed code code base leaked And embedded in there was a regular expression that was meant to detect user frustration. Did you all catch that? Iter very profane. Yes. Be they wanted to know, right? They wanted that signal. That meant something went wrong. the users is going, God Damn, how many times have I told you not to do that whichich would be a very visible signal of failure. Yes. Yeah, to be charitable, I would say it's a very high precision device but very low recall. It's going to catch some instances of frustration, but it won't even catch all of the negative F bombs that people drop because it's just very particular and not very complicated as an expression, but it shows that they're trying in that fundamental way to capture at least a subset of the worst kinds of failure And that resonates with us because at Big Sin, we tried to write a similar pattern. It was much longer and tried to catch many more things, but it still only c cut a fraction of the things that are actually going wrong in these interactions. And that's what motivated us to look for what we call the invisible failures. Is that my fault because I have been taught And I think we've all learned not to yell at my AI because it makes them perform poorly Well, in this case, you could be communicating to the Clad code team and if you study the regx, you can now know exactly which F bomM to drop to possibly l your attention But but I also thought was the case is that we can't really see inside these black boxes that we don't really know what's going on insside the AI, right? So the user it's up to the user to flag the failure. The AI doesn't know That's okay, the AI in some sense can do verification of its own outputs. That is an easier task. You know If you have it look at its own transcripts and try to find things that have gone wrong, it might do that at a higher rate of accuracy than you might have thought. A. And certainly if you have a more advanced model doing it, it can spot, for example the contradictions, the failures, the mismatches between intent and response. that the more primitive model was not getting right. And so that's a monitoring opportunity right there. And that's what we mean by an invisible failure. You ask a question The answer that came back is not quite an answer to that original question. If the user doesn't signal there was a problem for all we know they're now running the wrong code or they're off with the wrong factual claim or whatever it is. And I assume that product developers would like to know that that's happening. That kind of' a bunch of prompts that I use a bunch of prompts that tell the AI to check its own work. and in some cases, I'll use a prompt that tells it to rate It's dancers on scale ten. This works really great. That way comes back sounds very confident and this says, well, I' give this a seven out of ten and I sort of dig into that more. There's also another tool, I dont remember what it's called where They use two AI models and they use it like like a gAN or something where one AI model checks the other one and they sort of go back and forth. And that seems to me to be kind of an obvious way to check these things Is this is this how these things will be improved over time in terms of accuracy through sort of self checking or One AI checking another I think yes. And in fact, there's a few dimensions to that comment. The first is I just think it is productive to have them interacting in that way. We do that for PRs inside Big spin. And when we do the annotation work that we do for Big spin, that's on the back of us having these annotation protocols where lots of LLMs were critiquing each other and trying to find prompts that aligned their behavior so that we got at least consistent results All of that is incredibly productive I think the essence of what you said, Mike is that this needs to ground out in something like true verification And that partly explains why progress has been so fast for software development where the verification step is often running the code, and that feels very within reach And that though does point out some real problems once you step outside of a highly verifiable domain, even going to things like designing UX's, but certainly things like the legal system. Verification is not so straightforward And then I think all our concerns about accuracy and hallucination just come flooding back in. and it's not so clear What mechanism will you use to scalably address that But if we can find those verification signals, we're off and running. I think that's the lesson of software development with AI Yeah. But So this You're not exactly On the stochastic parrot side of the equation saying that these models are dumb and useless and make mistakes and no.ertainly. have that's a very complicated thing we could discuss in its own right. the resonance of that metaphor and what it means and what it implies about us and about the models and about their prospects and so forth. Certainly, I think something very sophisticated is happening and there's incredible potential here. Whatever you think about the current moment, I think AI is going to continue to change the world. This is why I like you I might be too centrist for you I took a quiz this morning that placed me right in the most in theidd centrist position you could imagine the most boring perspective imaginable I assume versus the other archetypes you could have according to this quiz, which are really far out there in terms of risks and prospects and the excitement about consciousness. I was just kind of in the middle You know, as a linguist, I imagine you're very interested in LL Absolutely. Oh yeah. In fact, one of the debates we have on this show Jeff's kind of on in the Jon Lacon camp, the Fei Lee camp where he thinks that an LLM is insufficient mazine, but not not quite a per I. I think the jury' still out. I think what we're what's been fascinating to me is how far we've gone with language alone And I'm not completely convinced that what we do in our own brains uh, exists Without language, I'm not sure, but u I'm not convinced that an LLM can't go a lot farther certainly than we've gone Where do you come out on that? You'd have to say that A key to their success is that the streams of symbols that they process go way beyond language. Now they're full of log files and sensor readings and technical descriptions and other things that I would think are giving them an increasingly Dance picture. of what the world is actually like. And if you did starve them of all of that and gave them only text from Wikipedia with no accompanying metadata or images or anything like that They might be much farther behind because that's a very strange fragmentary view of the world we occupy. That's a fair point. But the thing I would want to pick up on is that it's completely eye opening and remarkable to me that such simple learning mechanisms when scaled, yield all these complicated behaviors. That is one of the most exciting scientific things that has ever happened. and I feel privileged to live through this moment of seeing this. And I definitely did not anticipate it. I thought that for these complicated behaviors, we would need something much more engineered much more complicated, much more futuristic, whereas the raw ingredients here have been known for a very long time in machine learning and back on through statistics in the early days of AI in some parts of them Yeah. So where do you come in on explainability then on whether that's possible, desirable? That's been a major focus of my group. I'm a big booster on the idea that interpretability is going to help us improve these models and also get control of them And we have discovered, I mean, this is actually where my linguistic research and my AI research kind of dovetail These models solve hard generalization tasks and do complicated things because they have developed very rich internal representations that in many cases are quite understandable to us. And that explain how they can generalize so well That's a remarkable finding. I think would it's again, unanticipated, if you think back fifteen years But now it's been so productive and exciting and that ties in with issues of linguistics and cognition because then you can start to think Well, they're not like humans, but they have this human like capability. whatever mechanisms they have are at least sufficient for those kind of behaviors. And that's a major clue when it comes to unpacking the human capacity to understand language and do complex cognition. That's just fascinating to me We're talking to Chris Pots. He's a professor of linguistics Stanford and the founder of a company called Big Spin. AI, but which is named after a skateboard trick Can you do a big spin? I know you're a skater? Yeah, when I was younger, I could do a big spin and I have pledged to my team that I will learn to do a big spin, but it is a scary trick. You really haven't have faith. I can still kick flip Well allies are fine. That's good. But I know it's not what the company needs for me. My son is a skater boy too and I Oh wonderful. It was fun to watch him fall a thousand times and then bigig part of it. Make it, right? It's amazing. It's a little scary also. Real life lessons there though about persistence. Yeah.. That's real world model It's helped him immensely That's true. So in a way, Rich Sutton wasn't wrong with a bitter lesson. it is true that throwing compute at these things is surprisingly effective Yes, I am inclined to agree. It's a bit complicated for me though because The bitter lesson is kind of like one of those lessons from writing manuals that just say like, omit needless words. It's great as a reminder if you're already a good writer, but if you're not, it's just not helpful advice. And if you just say to someone, hey, just scale endlessly They'll make lots of bad expensive choices And the real lesson of the era of scaling and so forth is, yeah, we scale But we also learn tons about how these architectures work and find lots of ways to make them efficient. And the example I usually give is it's been a kind of minor miracle that we went from context windows of two thousand tokens to what might as well be infinite We did not get that by scaling the Trformer architecture from about ten years ago That would cost us literally trillions of dollars to get to that big context window. People thought very carefully about locality and language. and how neural networks process and learn information in sequences and they found ways to approximate the context window so that it could be scaled in that way So yeah, you scale and you don't get too clever about thinking about all the details of language But on the other hand, you need a really deep intuition about linguistic data to do that kind of scaling. So what is Big Sin? do as a company besides the research, what's what's the That is a fascinating question for us because of course, we have an app that has an agent and it will help like a product manager as data stream in from their product understand the issues that are arising and find things that are going well and help them with fixes. So it's incredibly empowering as a kind of supercharged Chief of staff to the you know, the product manager helping them spot all these things So it's a way of seeing where your AI is going. Yeah. Monitoring visibility. and then one incredible thing about the current moment is that the agent could suggest fixes that might help the Prouct manager. connect with their engineering team and so forth. Nice. But I have of what Mike was talking about, the whole idea of creating tools that help you Find it. Yeah and feed it back to the AI. I have to say, it's such an interesting moment for thinking about how to build a durable business in an era when anyone could take a screenshot of this app in action and say, hey Claud Code makes me something like this. And that kind of shows you that the value of the raw software goes to zero. And so my thinking about this is on the one hand, our agent is incredibly good at this job because of all the tools we design and everything else But the thing that really supercharges it is that we have all these annotators that run as the data flow in And they connect, they catch things like invisible failures. They do modeling of the user at their level of expertise, the task they're trying to solve, the domain they're in And all those signals, which come from these models that we fine tuned to do those particular jobs based on data that we've got They supercharge the agent and make it able to do all that important data science And an agent without all those annotations is really kind of flailing about in the general world of justust what language models can do in general Just to be clear, the annotators are peopleople No, those are models. They're agents. Yeah, automatic.'re you could think of them just as classifiers. This gets down. I mean, they're language models, but you could think of them as kind of just like old school cllassifiers They assign hundreds of signals So they're not quite like old fashioned classifiers. and they are actually language models under the hood, but they're very specialized to their task of identifying invisible failures and identifying user expertise levels, domains, all those things that I mentioned that are so critical to understanding where the failure points are and where things are going well. Who's your customer Our customers have to be organizations that care about their interactions which is not everyone who has a deployed chat bot But if you're in an area like you're giving medical advice or helping somebody with scheduling of something that matters or like doing things like professional coaching, then the nature of this human AI interaction really matters to the success of your product. And the distance between off the shelf chat GPT and the product that you want is enormous And every failure is a really important thing for your business So those are our customers because those are the people who every day are going to sit down and say, what's going wrong and how can I fix it? Then shouldn't the foundation model makers be Your primary, I mean, they should kill for your data and for your learning, yes Rather than rather than the application layer, the company that's that's using this as an application layer Um, right? So, so how far up the chain Do you go? Well, I mean, data, I think are key. I think that's the central insight there. I assume that the frontier model providers have lots of their own data, you know, a super abundance of it. I'm kind of jealous of how much they have But I do think that data are the key ingredient here as always. That's a very familiar story in AI that the data are the thing that giveiv you the transformative capability Yeah So in a way, it's an audit layer for companies. Yeah, I'm happy to think about it as auditing. That could sound quite specialized to people. You know, auditing could be a very particular role and this is broader to anyone who's just in charge of the quality of their product. But I think what they are doing in part is a kind of audit when we have escalations to a human are the kind of escalations that we like If we see the system resetting in these contexts, is that good or bad? And then of course, they're trying to fix that. Now everybody who's using AI, especially those of us who use it for coding so forth is should be concerned about invisible failures. Yes Is there a way to detect those to note What's going on? I mean, we obviously we're not your natural customer, but we would like that kind of of visibility into what's happening. Right, Well, one thing I'll say there is that it's a characteristic of expert behavior with AI that you make your failures visible. Experts complain, they push back, they iterate on goals, they refine goals, they tell the AI to change course And we all take that for granted, I'm imagining, because you all are at the cutting edge of using AI, I suspect. But for the vast majority of people using AI, they've been told it's a super intelligence. They ask it for things and they take the responses at face value. They adopt what we call a delegative mode, whereas what you want is an augmentative mode. It's the result, causal claim here. It's a result of the augmentative mode that people are able to solve harder tasks more reliably And those who just use it as a chatbot are the ones who complain most about hallucinations without actually fixing That may well be, yeah, there's probably interesting interaction effects with domain and so forth. But if you just enter a query and get a response you don't like and walk away thinking, well, that was just wrong That would be delegation with a bad outcome. Even worse, of course, is to walk away with the wrong answer as though it were brave it. Yeah. But in both cases, what you wanted to do is say You know, as Mike was saying before Could you double check that orr open up another window and just ask it to give you the opposite judgment. Hey, you really like this idea What's the most critical take you could offer of it and synthesize across the two. That kind of very critical mode is what experts are doing So you hvered go ahead all right. I can monopolize Chris. I don't want to so please stop. let me go on that one I for a second. So you remember the schmutt lawyer in New York who used Chat GPT very early on and got citations. and I went and covered his show cause hearing in federal court And it was interesting because he his defense was I thought this was a super search engine. Yep. I thought computers couldn't make mistakes. Right Right And then but what was telling, though, is that he obviously was suspicious because he went back and he asked ChatTPT, Are you sure about this And ChatGPT said, abbsolutely. this is early Chat GPT. So in there, there are all kinds of signals of what was happening And and I'm curious how What are the kinds of signals that you see for an invisible failure? That's are the segues into the question I was going ask because in the paper you have eight Archetypes for failures So the one you just described, I think would be the confidence where the AI and we've seen this, we've all seen it confidently wrong. and that confidence We believe it. We go, Ohh, well, it was it's pretty darn confident those sourcesed Iasted. gave me sources. that must be. U that's that's a big problem, but you also have some other archetypes, which I think are great. The Dift archetype where AI sort of gets your goal, but not quite It's off a little bit Right That might take you on a journey away from where you intended to be. R It can be subtle That's actually the most in the in the corpus you were looking at, that was the most common failure maybe that and the walkway. Yeah. Yeah. what's the walkway That's where you ask a question, you get a response. The response is not a resolving answer to the query, and that's all we get. But you can get walkaways later. so another pattern would be like the death spiral You try a few times. Hey do this, maybe you rephrase it and then you walk away because none of the times was quite what you were looking for and you don't complain, you just keep trying and then you bail. Is death spiral also a skateboard trick? That's It's the last one you do I actually had I think I told this during last week, I had that death spiral where I kept trying, keeppt trying. fininally, I said And this was a mistake. I give up and walked away. But instead of the AI giving up, it deleted all of its work, it backed off. and it deleted everything. And I said, what are you doing? I said, Well, you said you gave up so I just thought I'd delet Oh wild. Oh. How do you capture these behaviors? That's fascinating. A little too autonomous.al. Well, they're getting more autonomous, aren't they? In fact, that's one of the things Fable is doing is able to kind of keep going. Everybody's talking about loops. these days the idea that well you don't just ask it to do one thing. You say, go ahead and do it and you loop and loop and loop That makes me very nervous. There's no opportunity there for you to interject a correction or a course correction. Yes, obbviously that could spin out of control very quickly and get very expensive as well There's the silent mismatch a user asks for X, gets Y, and says, Ah, that's close enough This you say this is rampant in software and education I have to say I've done every one of these. Yes me too. I have a wonderful story about this. I try to be an expert and be augmentative I had a colleague, he designed this fun infinite runner game. If you four hundred four at our site, you get to play Ollie not foundound where the skateboarder just you click the space bard and js. And I wanted to play a prank on my colleagues and have write an AI that would play forever And I would just say, hey guys, you know, I got ten X, the best score you've got. and I'm the best player at Allly Nut foundound. But I was going to have the AI do this. So I said, Hey, Claude Write a perfect, you know player for this game. And it said, I remember so distinctly, I understand the geometry of the game perfectly. Here's a solver that will run forever. and I said, great. And I try it and it's worse than me So I go back and I say, this is worse than me. and it says, Oh, you are quite right. You're so correct in this insight that it's not good. Here's a version that's much better. I run that one, it's a little bit better than me, but hardly changed. And it just kept cycling through this confident assertion that it had the perfect solver and then disavowing all of it and starting again over and over. In the end, I gave up, mayaybe I'm not expert enough. I still don't have a solver that's perfect at this game. I don't know whether it's achievable. My question was never answered But I was really caught in the death spiral and the contradiction unraveled there and you walked away I walked away and it was the perfect scenario because I don't know how the game works and I don't care to learn how the game works. I wanted to offload thisist to AI But I had the advantage that I could let the game play That's my verification step and I could see that I got forty two and this thing is only getting thirty six. It's not better and that is far from perfect Try again But in domains like the legal one where you don't I mean, what would they be the equivalent? like going to trial in that case and then finding it was wrong? Yeah. It's too expensive. Where are we going to do the verification step there? What if what did I just think this I think Your sense of wonder and excitement at this is I sure exactly. We are in a very interesting and strange time. Day anthropic re released fable and actually the word classifier has become part of a vocabulary since it released mythos. fable was mythos that had a bunch of classifiers that were in theory going to keep it from doing anything bad. What they find bad Yeah We'll talk about this more later. but what they've done is they've stepped those up. so it My fear is it's not going do anything at all.' good to calibrate. Yeah. There was a quiet revolt from the AI community, which had a real impact. It was mostly on X. and it was AI researchers saying, I feel betrayed by this And they walk those back. Yeah. Yeah. Well, it's as if you're peering over a wall at some magical nirvana. and but the wall keeps stopping you from getting into that secret And it erodes trust, Yeahah, people started saying, look, I don't know what's happening, but I'm going to use a different model because I don't want all of my responses nerfed. I'm trying to do real research here. Exactly. Well, that's the real fear. And actually Ox Stamos talked about this today in Twitter. He said Why would any company invest and fable. U withith the risk that in the middle of the project, Fabable just says, Noah, I am, not It's just not worth it So well good luck with The new company, I think this is very exciting. can I hit the other topic? Yeah. Yeah, yeah I'm curious about because you're a linguist the debate about understanding And all that follows understanding Unt unto consciousness and everything else, right? Sure. But to start with understanding. Who was it, Leo? You sent me that long video of Jeffrey Hint. Jeffre Hinton who was arguing that it's obvious that they understand and then that He argued that it had a desire to lie to him which also obviously implied that it understood people understood what a lie was So I'm curious on the on this D they they If you were taking a test as you took this morning On this topic, where do you land probably right in the center, you have to be open minded because anything else is way beyond what we know scientifically about how humans are doing this and in turn about what's in principle possible. If you talk about things like beliefs, desires, and intentions, we don't know what's necessary and sufficient in humans for this, We rely on An assumption that people are like us and all those philosophical problems come flooding in as soon as you say that, but we navigate those things, But they're uncertain. And then we also have very little understanding of what models are currently doing now. You know the interproject is far along, but there's endless things still to learn, and we especially don't know what the models of tomorrow are going to be like And if you did just think, okay Understanding is a loaded term, but what it's going to be mean to be meaningful is that you have some kind of mapping from language into some conceptual structures And so that like you, we map language into mental representations of things, and that's what it means to understand And what that puts you on is a continuum. How complicated is the coneceptual structures, how complicated is the mapping, how refined and so forth And obviously language models way behind us along many dimensions in terms of how sophisticated that mapping is But if that's all there is, then it looks like nothing is stopping you from having a model in the future with exactly these technologies that has a very refined mapping of this sort. And if that's not enough for you for understanding that kind of semantics in that deep sense then it's on you to tell me what's missing from that picture. And then sometimes people reveal that like actually they're kind of biologically oriented. So that's a dead end because there's something intrinsically biological about understanding. and it's just good for people to confront that and maybe realize that about their beliefs. So if you were in the studio together, Leo would hug you right now. I would because this is what he argues. So I'm editing a new book series for Bloomsbury Academic called Intelligence Ae and Humanity where AI forces us to reconsider things like Rubin Chottery iss writing aook about intelligence. What is intelligence? Look the history of intelligence. He was a linguist Does AI force us as a whole or even you individually to reconsider are prior human definitions of understanding Yes. And I feel like whatever your reaction to this and whatever your beliefs, if this moment is not causing you to reconsider all those things, then there's something amiss because this is the first time in human history that we have encountered other non human creatures that can do all these things. It is definitely weirding us out, but if it doesn't have you pause and say, lookook, I need to critically assess what it meant to be an understander or critically assess what it meant to connect symbols and language in the world. If you're not pausing, even if your response is, this is all beside the point because they're too different from us as humans, it should still be a moment of serious reflection So did it cause you as a linguistic scholar Um to change any views that you'd had before You encountered all this That's a great question. I will say that it has been empowering in terms of making progress on some of the most difficult problems in linguistics. And the two that come to mind for me are what's often called the poverty of the stimulus. So how do we with apparently so little input from the world get to a full competence in language so quickly? the Chomskian answer has been, you have rich innate priors But of course, language models get there pretty fast without any innate priors. So you don't the innate priors aren't intrinsically necessary to achieve this. They might be given human limitations, but you see how nuanced this is getting now And then there's a related question of what's a conceivable human language? We have Only a finite number of them that we've ever encountered in the world, and they're all a product of history and accident What is the abstract cognitive capacity for language? What set of things is learnable by us very difficult problem to address experimentally, but very easy if you're thinking about training language models on different corpora representing different languages and seeing what final state they achieve two big questions unlocked and that is just incredible from the point of view of new debates, new discoveries, new terms for these things. And I didn't think again, that we would have a new investigative tool like that in my lifetime. And I thought then those questions were going to be kind of stuck where they were. you see there's some evidence that can these AIs can create their own. Innal That's what I was to ask O another. A at some point before we got on showed us a dial a conversation among agents without humans At some point does one and you talked about where that might go, the void Does that potentially go to them inventing their own language Another fascinating question. That used to come up more about fifteen years ago when we did more training from scratch. Now that all the best models are pre trained on the internet, which is a record of actual human usage and everything else, they don't have as many opportunities to go off that distribution. And so it's less likely that they're going to invent their own language given sufficient interaction, and maybe if they do start doing weight updates as part of these interactions then you could get into some really far out states. And that would be fascinating to see what kind of more efficient systems they might evolve or systems that are differently pragmatic than human languages. Or maybe, this would be the most exciting for me. They converge on kind of human like systems at the level of the pragmatics and the encoded meaning which of course human language is always evolving and splitting off into different languages and dialects so on, and you could imagine isolating U, AIs and have them talk to each other at high speed for a large amount of time and see if It's another m Yeah Yeah And I'd be really curious if that happens. You could also imagine the constructing in language from the various grammatical rules, vocabulary, a German style plugging everything into a single word, you know, you could imagine all kinds of things from human languages that exist already and sped up, right? You don't have to wait actual human generations to see what's happening. And again, they're always going to be qualifiers, but what an investigative tool Y I also think you talked about a bridge that we may cross sometime. I hope we cross it in my lifetime where they are self improving, they're able to change their weights. Uh sure and that might actually be when things get explosive Yeah, and there are no technical obstacles to that now. It's just a matter of calibrating those processes and then actually running them at a technological level. It's very expensive to do all those weight updates, but That shows you the potential because in principle we could do it now Interesting. So it's just a cost issue. It's a Getting enough Nvidia black wes together to revere rubens together. Yeah. And also also just fine tuning that process, which across all of these training processes is kind of at this point more art than science. And that's why people get paid the big bucks to do it because it's a lot of lived experience to figure out how to set it up in a way that it goes well as opposed to going poorly And when the price tag on it going poorly is in the tens or hundreds of millions of dollars, hope you have experienced people running it It's such a pleasure talk. is so much fun. L., I really enjoyed it Re real quickly, forour. I know I usually don't talk to the guests during this part, but I have a question that Iito, our producer wants to ask you something. ' we rarely ever have a linguist on of his caliber so So like is there any kind of qualitative difference between a model trained on English and a model trained on Chinese question. In terms of the internal representations? Yeah, or anything at all. Like what are there any qualitative differences I mean, are you thinking of a scenario where we train one model purely on English and another purely on Chinese? I guess the question is more like, would is a Chinese trained model qualitatively any different from an English train model. This deep sea fundamentally different Because of the language itself. Ch more Chinese. And does the language itself have any kind of intrinsic quality that would be different from like an English model. Oh must be, right? Because the units can be very different, especially from the point of view of the language model. Your tokenizer might be different for English and Chinese and that's going to have impifications for how it reconstructs those partartial words into more meaningful units internally And then as I was saying before, if you think that what it's doing is partly inducing a mapping from the language into concepts as a way of solving the hard generalization tasks that we pose for these models then that conceptual structure could be very different And there is work on this and you do get these fascinating things that, for example, a model trained dominantly on English, but secondarily on Chinese, when it speaks Chinese, it might do things like using color terms in a way that looks more like English. maybe it overuses a word like orange or something because that's a lexical item, a frequent one in English and it's more marked in Chinese But this model has kind of had one set of experiences bleed into another causing it to have a different conceptual structure arguably. And then of course, you can ask yourself, what about for bilingual speakers? Are they showing similar kinds of things And again, you just see the power of this potential investigative tool here. I know why your interest in this bino is bilingual or at least bilingual. Yeah And when I do switch languages, I do think differently. Yeah Interesting Is it not the case though that all these models are converging because it's basically the same corpus corpus for everything. I mean Or is it not is it? I don't know It depends, I suppose. so for the pre training, it does seem like everyone is just getting all the data they can and that might be quite derous So the post training, it seems clear to me that for example, the path that Anthropic uses to train these new products, which I think many of them start from the same base model They've got that really worked out so that they can maintain a kind of personality that they want, even as they give the model capabilities This is quite striking. I would love to understand more deeply how they achieve that But the stability they've achieved for their product is different from the one that you get from Chat GPT, for example, and certainly different from Deep Sek. Eperientially, I could confirm that And it's one of the reasons people become fond of certain models because They like the personality of that model. Very interesting. Chris Potts is such a pleasure talk to you Chris's startup bigigs spin. Ai if you're interested in in making sure that your're AI models are not failing you. Maybe you should. And based on this discussion, my tip of the day, you all would love the Void by nostalgibist. It's this epic essay. exploring how we ended up with the models that we've got, why they have the personality that they have, and then lots of interesting thoughtought experiments and observations about the culture embedded in it's a movie No, it's a seventeen thousand word blog post. Oh, and is's wrong? Is it from those guys? It is. He posted it actually on Tumblr. There's a link from Lessrong so that there could be discussion there. But it. I don't know who Nostalgarist is in the world, but he's probably a fascinating character, he or she I am going to read this. It is on Tumblr. how odd. Highly recommend. It's quite a journey. All right.. Chris, be be careful on that skateboard out there Yeah that's good advice. than you. Such a pleasure. Thank you so much for coming on. I can't wait to see what you're up to next. And if you ever want to come back and talk about it, please, we would love to have you Chrispond. I'd be happy to do that. I really enjoyed this. Thanks again. Enjoy your sabbatical. Yes. No kidding. We're going to take a little break and we'll continue with intelligent machines right after this He's gone. All right. I could spend out. I love linguists and this is spoiled. Well, not all of their allies. Not every one of them. Not all of them. There's certain ones. We know who they are. We know their names U All right, let me let me get the ad in and we can talk This episode of Intelligent Machines is brought to you by Z sccaler, the world's largest Cloud security platform You know, we can, I mean, you can listen to this show. you know the potential rewards of AI are huge, especially for your company can't ignore him Because the competitors, the competition isn't, right You also should pay attention to the risks. There are lots of them loss of sensitive data sometometimes inadvertently But then there's also attacks against enterprise managed AI. And then there's the issue of generative AI giving new tools and opportunities for threat actors thingsings like creating incredibly effective fishing lures Like that, writing malicious code automating data extraction. AI is changing the entire security space There were one point three million instances of social security numbers leaked to AI applications last year. And most of that was inadvertent, right? Somebody uploads their tax return. It's all in there, right U Chat GPT and Microsoft Copilot saw millions. I the number is A lot At least three million data violations. And again, inadvertent stuff leaks But that's why you need a modern approach. You need Z scalers zero trust. Plus AI Because it's Zero trust it removes your attack surface. It secures your data everywhere. With the addition of AI, it can also safeguard your use of public and private AI protect against ransomware protect against AI powered phishing attacks But you don't take my word for it Just listen to the customers. They love Z skill. like Siva. He's the director security and infrastructure at Zara They use Z Skiller, this is what he has to say. With Zkiller being in line in a security protection strategy helps us monitor all the traffic So even if a bad actor were to use AI, because we have tight security framework around our endpoint helps us proactively prevent that activity from happening. AI is tremendous in terms of its opportunities, but it also brings in challenges. We're confident that Z skill is going to help us ensure that we're not slowed down by security challenges but continue to take advantage of all the advancements. with zero trust plus AI You can thrive in the AI era, you can stay ahead of the competition, you can remain resilient Even as threats and risks evolve, leararn more at zcaler dot com slash security. That's Zcaler . com slash security. we thank them so much For the support of intntelligent machines, Mike Elggan. so good to see you filling in for Paris. Mike will U be in the beautiful area U of of England for a little bit and then up to Scotland. That's right. We're delightful afternoon tea today. Is this burning hot though No, not here. I was just in there Ace. We five days ago, for days ago It was pretty hot in most of France. It wasn't too bad Proance and in fact, it actually had Some rain while we're there, which is really interesting. But no, it's super pleasant here. seventy three degrees. skies. So you may might have gotten full since youre going to Scotland. You may have gotten the full sense of it. America fell in love with Scots. Oh, we did. were in the worldorld Cup New York particularly. They were just great. Boston too they actually drank all the beer in Boston. I saw that article, they drank all the beer in Boston in Miami. And the great thing that they did they took they somehow took to traffic cones and crowned every statue they could find with a traffic cone. There's a reason for that Oh, I saw that I saw that here Yesterday, actually, there's a really a magnificent statue of somebody and it had this like jack in the box traffic conone hat on it. And I'm like, huh. I would found out because we had Ian Thompson on the show. And he explained the traffic cones. There is a very famous statue in Edinburgh I said of Lord Nelson, I can't remember who it's of Oh Duke of Wellington, sorry It's in Glasgow and they cannot The authorities cannot prevent The Scots from putting traffic cones on its head on its horse's head, differenterent cones for different, you know, holidays And so this is a tradition in Edinburgh. But it spread. when the Scottish fans came to the United States, they realized we have traffic cones and statues as well. There wass a wonderful young journalist for the Scotsman named Catherine Hay, who did great videos in Boston and Miami. And is what I love is she just was going around Miami and just seeing where they'd been through the traffic codes. You can see traffic codes and all that statues. I love it. That's a great tradition. I love it So the big story today is that Fable is back. So is mythos or some For the same people who got it before? No Limited. Okay, so this is what's interesting. So apparently We finally get the details from anthropic itself Anthropic did give Mythos back to a number of the same companies through Glasswing that they had it originally back in june twenty sixth and we'd seen kind of noises that some companies still had access or had gotten access to Mythos. It was It was june ninth When the Trump administration through the Commerce Department blocked access to both fable and mythos, saying it was a security issue We are now seeing anthropics response to all of this. and even though they're being very careful, You can read between the lines Fabable is back as of like about an hour or two ago But they have really turned up The jail break protections mayaybe to the point where people, I haven't yet put my finger on the pulse of it, but maybe to the point where people are going to be upset That's my prediction will they blame that point They'ble blame Anthropic. which should really they should blame the U.S government Anthropics doing what they can uh, to appease the Trump administration. So I think anthropic kind of threw some shade and I'm not alone by the way. Alex Stamos agrees. In fact, maybe I should best best way to Recap Yes is to put Alex Stamos's tweet up on the screen because he He says There's a lot to unpack here Anthropic is burying some hard truths in careful political language First of all, Anthropic verifies none of the jail brereaks provided a capability beyond what many other models, including the Chinese models could do. And that was when Alex was on last week and we were talking org was it two weeks agow weeks ago two weeks ago And we were talking about his letter signigned by hundreds of the best names in computer science. freefable. org. That was one of the main critiques is it's not doing anything that other models couldn't do. In fact, Anthropic pointed out that even Haiku It's its dumbest model do the same exact jailbreak that Amazon fingered them for. Which is a dangerous thing to say because okay, then ban them all. H Yeah, maybe I think that Anthropic wouldn't have written this if they hadn't some confidence at this point that they had appeased the administration We don't know all the details of how they appeased It's my theory that they offered them ten percent of the company other things. but anyway They said First of all, I think they cast shade on Amazon. The export control directive on june twelfth came after the government became aware of a report in which Amazon researchers had found a method of bypassing Fable Fives. safeguards Over the past weeks, we've worked closely with the government. other partners, including Amazon to review the report and evidence. ourur testing confirmed that many less capable models including Opus four eight D five five The Chinese model Kimi two point seven could identify the same vulnerabilities as Fable five did in the report when it came to the demonstration of how to exploit the single vulnerability, and that was the things that really scared the Trump administration. They made an exploit Every model we tested could produce the same demonstration, including Hiku fourty five, Sonet four six, Ous four six, Ous four seven, open four eight, GPT five four. GPD five five and Kimy two point seven So They say the reported technique did not expose any unique Mythos level cyber capabilities. So this isn't that judicious. that's pretty clear btle. something It's just evidence that, you know, the whims of presidential administration is not the best way to go about this sort of thing And u and the fact that, you know, the The Trump administration wants to buy parts of companies wants you know, wants to be heavily involved in deciding who gets to see picking companies that get to use it and that sort of thing. This is really terrible. it's just amateur hour and it's vaguely totalitarian, the definition of totalitarianism, by the way is when a government sees every single area of human life to be within its province. Yeah. exactly. So so this is this is not totalitarianism, but it's a it'amn close. So Mike, two things there. One, I just want to quote Benedict Evans's newsletter this week This is a mess with random unqualified officials banning and unbanning products with no process or transparency One has to laugh at anthropic and the safety activists are quotes around safety, who spent years saying that they wanted restrictions. But when they came, they said, no, not like that Alex Stamos said Casey C A I S I. the Center for AI Standards and Innovation is the group that's supposed to actually make these determinations not political actors in the White House Cy my everything was prior safeguards. the implication is that this whole thing was unnecessary Yeah It also Alex called it an own goal. He's he's a sports fan. a goal scored against yourself because he said What's going to happen as a result is U. S. labs now have to make a much more conservative prerecision Recall trade off on cyber refusals, US models become much less useful for defensive cybersecurity work unless you're in the trusted group Security companies and startups to provide services to others will now be driven to use Chinese models. Big win. PRC labs this month. It pushed me little o me and I'm not a bellweher, but I think if if as an individual, my reaction to this was, well, I guess I better not be dependent on American models because they could rug pull us at any time. pushed me to A investigate local models more. and I've actually found a pretty good one. We'll talk about that later be to Frankly use some of the Chinese models. They're very, very good. and that was the other thing. People discovered how good GLM and Deep Sk Kimy are, they're good And they're based on open source And they're open weights You can if you had enough machine, which I don't But you could run the full GLM locally. you'd need five twelve gigs of rain. Which whichich consider that China China's goals are political more than economic They can destroy our AI industry. Well, and that's the question people saying, why would China give this away You just made you just said why. They don't need to make money on this And and you know the other thing that strikes me is that if they ever want to God God forbid this happens. If they ever want to go after Taiwan Now's the time, right because they'll absolutely cripple the technology industry around the world. Well, they don't even need to because guess what? The technology industry has crippled itself Because of Rot for RAM and software hard drives, SSDs, you can't buy them anymore U everythingverything's gone up in price. Apple raisedes prices significantly this week. Every other company has done the same. And even with that higher price, you can't get the amount I could not buy a Mac Anymore, I could have a year ago that had enough memory to run GLM. I can't now Because the most memory they sell in a Max studios is not. I think it's ironic the primary impact of AI on the economy that's going to be most felt is going to be the inflationary impact of the shortage of memory. For sure J as effective as invading Taiwan. If you ask me. Yeah So so Mike I had there was a discussion with Leo online. had Olivier Salvin from Fordham Law School on last week. And he was talking about the history of regulation of radio And I went deep, I read his dissertation. This was fascinating By the way And so share that with me. Yeah, if I could do just a second on this. Yeah. So Things could have turned out differently in broadcast The reason that we ended up with the regulatory and economic regime we have is because the U.S. Navy intervened and was worried about ship to short communication and insisted on the creation of RCA as a patent trust. that involved all the companies and he banged their heads together and said, got you gott to get along, you've got to do this And then U the government had the Navy had a seat on the board. att first And and, um It's not hard to imagine, to your point, Mike that by the time Trump says I want a piece of open AI and I want a piece of anthropic, and I want a piece of this company, and a piece of that company, that you could see them creating the RCA for today. And the next piece of where this goes is that It was the was what was great about Olivier's dissertation is the argument at the time and the reason for the creation of the FCC was that without it there would be chaos. Without it, everybody would pick their own frequencies and nothing. So we got to create this. And that was as it turned out BS that was there only to convince the legislators to pass the nineteen twenty seven regulatory law that created what would become the FCC and that would enable, by the way the restrictions of our language on broadcast that would slice out the First Amendment for broadcast because of this argument of chaos And so it's not hard to bring that to today and say that the U. S. government having now intervened to this I think extreme. Impi pulling products off out of the world Out of Leo's hands? No U candy from the baby's mouth that we could see Trump getting in his head. Thank goodness, he doesn't watch the show to create the conglomerate RCA of AI and force everybody to put their patents in and their intellectual property in, and it becomes run by the US government If you want sa too there's not going to happen to the rest the rest of the world would revolt. Is that the threat then in the twenties wasas immigrants Exactly, Eactly That's the other thing weve heard that before. Marci Marconi being British and Italian u GE was going to sell Marconi a transmitter device And when the when Herbert Hoover, who was then head of commerce, found out And the Navy found out They put a stop to it and instead created RCA and instead then required RCA to buy the American assets of Marconi so that ferners couldn't control our broadcast because it was a strategic asset So Stamos goes on to say Anthropic is saying between the lines Amazon's inability to appropriately communicate severity through our industry into chaos. I don't know if that's exactly what Anthropic is saying. They say there needs to be a consensus framework in the AI industry for the severity of an AI jailbreak. We cannot agree on the severity and Amazon W overestimated the severity of this. That scared the Trump administration So they're lobbying for some sort of to do this. don't I'm not convinced such a thing exists. I don't Yeahah. I also think that anthropic way over stated the the power and danger of mythos U, you know, you do or do not think they did. I do. I mean, I think that They scared you know for sure feel like it felt like a kind of a marketing stunt.ight? Oh yeah. to get people to think, wow, this thing is so powerful when this thing is available. I want it. And so But but it all points the same prescription, which is that we need we need a good governmental agency that's nonpartisan, that's not about grabbing power that's not about hyping threats, that's not about to being crazy And that basically can, you know, we have We have so we used to have so many great agencies that would shepherd the industry through these things. Yeah you see Mike that that's the argument that was made to create the Federal Radio Commission, which became FCC. and I'm I believe that the FCC has done a lot of bad things, especially about our speech. And so I'm going to soundber I'm a libertarian. I'm a plain old Democrat, but I'm going toound libertarian for aute. I don't know that I want that agency. I don't know that I trust that agency. I fear what it will do out of a position of ignorance as the government just did Well I I think they'll find the right experts. And look what the FCC is doing today and. But the AI is a speech issue. Clearly, the FCC is around speech. It's all about speech and the First Amendment But the but but AI is also a cybersecurity issue, right? So so look at look at the role that CISA has played in the last couple of decades or whenever it was founded in sort of shepherding and sort of protecting the nation and the companies and getting this cyber security industry singing from the same hymn book It was a fantastic benefit. And I think You know, yes, the, you know, government agencies that exist to take, you know to grab free speech pers is problematic, but but we already have a situation where We have the federal governments sort of meddling with and asserting itself as the decision maker in terms of who gets access to which model, etcetera. Uh and then always you know, u basing it on National security, etceter. You can always do that We need we need a steady hand of nonpartisan experts, an agency that looks at all this stuff and can give us some rational um, systematic evaluations of all of these claims that are made by the industry, by governments, by foreign governments, and by everyone else And right now there's just this huge void and anybody can say anything. And in the case of the presidency, the president can do anything. And so it's that's the problem we're talking about right now is just this sort of wild west where nobody knows what's going on Nobody's really in charge. And the people who assert their power over this thing have suspicious motives It's really a big problem given the power of AI. Exactly what Alex winds up his post on Twitter I'm sorry, X, u with He says we's not don't call it X. It's Twitter. Yeah. No, it's X. I don't like Twitter because it's about's right you don't, yes. That's. you know why I don't like it. Yeah. I'm Twit Predated Twitter Yeah, sorry. Can we just call that? I got the memo. I got the memo. Yeah, sry. No, it's fine. Everybody thinks of it as Twitter. They still call them tweets. I understand. We give Alex tweets, we give the US. government huge powers. This is why you staff it with competent, calm, non corrupt people who don't use those pers to punish enemies The only upside I could see from the whole mess is there's a whole bunch of VCs with former or current administration affiliation who we can now safely ignore on AI policy They've shown everything they've ever said y's talking about David Sachs on AI regulation was just politically motivated. it's it's an own goal is what Alex says. And I think that that's pretty clear. We're also, I think, going to see Once people start messing with fable, that it isn't really very useful. It's been it's been in fact, Anthropics says We had to turn up the classifiers so hard that you may find that as you're coding, it just drops down to four eight U let us know if that happens We' do do the best we can. But I think that this is a nerfed this is going to be clearly a nerfed model And again, it it's the lack of So one of the stories that I put in the rundown, I don't think you had this one But it goes up something you speculated about last week, Leo, and I kind of laughed at you, but you're right. Austria is talking about playing host toanthropics. Yeah. And I don't know if that means merely hosting the software or say mo be the company over. I wouldn't move to the EU to be honest if I were anthropic U I'd go to Belize or somewhere with a very what would you recommend Mike I world traveler somewhere somewhere where Argentina, somewhere where the government really just wants the money Madagascar low labor costs, a high a fast internet, believe it or not. and Really? Madagascar under the water? Yeah. reallyally? I'm not I'm not this is not a serious proposal. but a Malta maybe Yeah. Somehere somewhere. What was it that was headquartered in Iceland when it was when was it it was WikiLeaks? Oh yeah, thatQarter an Iiceland the freedom there? Yeahah. yeah I think maybe maybe you would be the like circle somewhere where you have you can cool data centers. What did Larry Page want? He wanted one of those islands Google Island? that are made out of old oil rigs in the oceanas. Yeah international waters. That's the right answ. Innnationalaters. No, you know, the good news is, the really good news is N for the U.S. but for us as users, that China has got a lot of open weight models that are very, very capable. This has just stimulated, I think, development of competitive models. This is an opportunity whichich is Jon Wan's argument that that's exactly what he said happened when you when you stopped me from selling my chips to China. You only stimulated them to compete Okay already we're seeing Chinese companies like Maetuan say Hey, guess what? We're able to use our own domestic chips. to create AI models. We don't need Jensen's chips. We're happy to use the Huawei chips. They're quite good Yeah, it's Tan Tu is Chinese and He was the one who said that, you know, when your enemy is self owning itself let it do so. Yeah. And so that's kind of their strategy right now on many fronts. They're just It's not that they're doing anything aggressive, they're just watching us do aggressive things to ourselves and just biting their time Here's another article from Kid Metz, Karen Weiss and Meghgan Tobin in the New York Times. Chinese AI models close the gap with anthropic and open AI. Silicon Valley Engineers and a few podcast hosts recently flocked To a new technology from a Chinese company Z. AI that is almost as good American competitors but much cheaper I've actually been using GLM for three months because my subscriion my quarterly subscription runs out in three days. so We did that before this happened Currently I mention I'm using a new model with my herermes that Larry Lawrence Gold in our clubit Discord recommended. And I'm actually really very happy with it. It's based on Quen which is a Chinese model, but it's but it's been tuned be I don't I don't really understand it It's O R I N T H. I guess that's Ointh Is it easy to switch out models underneath your? Well, one of the things I did ages ago. When I moved to Hermes, one of the reasons I got off of Claud Code is because Claud Code really works only with anthropic models. And ironically, anthropic doesn't want you to use Coud code with any other agentic harness. One of the things we've learned, we're going to have Nate B. Jones on in a few weeks talk about this. One of the things we've learned in this process and even before then is the model is the brain of the robot, but almost maybe even more important is the robot itself, the hands, the eyes, the tools you give it Memory You give it of what you've been doing and what your previous work is, the tools that Mike's doing to say, you know Keep an eye on yourself, make sure you don't make mistakes. All of that becomes more important than the robot brain. I was looking for a way do all of that in a system that was interchangeable that I could change the brain. And that's exactly what Hermes does for me. It's very easy for me not only change the brain you know once but to change it anytime I want in the middle of a conversation. I can go to the drop down here This is Ornith which I'm using right now But I can choose any of these models and just drop them in. The next turn And they they then it just takes all your memory and yeah they and they even remember the session. They remember this conversation So u This resistant lets you do something full disclosure. My my son works with Cagie. but they let you sort of have the canned prompts and so on Same thing and then you could swap out models. The difference is I don't think it can remember between sessions. I'm not Yeah so this is where I argue L, everybody starts with a chat bot and gives all their information and all their prompts to the big companies. But eventually you start to look at ways to control it, to own your own destiny. That's when you start looking at agents. There are many, many choices. The one I chose is Hermes from Ns reesearch. We've interviewed Jeffy. Is there literal switching cost in tokens when you No, no, just time Just time. This your time. Okay. Right. And truthfully, these guys are so good now All I did with Hermes is say, Hey, look over there at Clauded Code. See all that stuff it Bring it in Mify it as needed. Um, One thing that's important is that there is an API standard that openAI uses Athropic does not, and almost all of the agents use this open AI API. It's an open API. So look for an agent that supports that, then that means almost every case except for anthropic, you'll be able to use any model because they all use the open AI. And I'm running Lama, which is open source code. on my framework that lets me download models from hugging Space and use them. That's why I have Ornith. I downloaded it from Hugging Space. I asked my agent, what's the best version of Ornith I could use? It says you can use thirty five B because you have one hundred twenty gigs of RAM And I install that and that's what I'm running. So I am running fully locally All my memory is local, everything's local All I did is I said, if you you know need to do some coding or need to do something more challenging, then these are some other models you can call on. So what you're also saying is that if the government now shuts down O AI next The switching cost for someone is non existent whichich is to say there's no moat, like once again. There's no moat. Around any of these. There's no mo And that's what unfortunately the federal government has done by this everybody realized that. They've pushed everybody in that direction And it turns out The harness is Absolutely the most important part. the memory is very, very important This we know whether anthropic has also made peace with the Pentagon and all this or is that? Yes? Dxt will going on. Well The Pentagon wants cl. They want mythos. They want fable. They want mythos. So I think that was part of the We don't know what the conversations were And they certainly used it in the Iran warar. Yes after it was sort of like designated as a supply chain risk. becausecause it's so valuable to them and they're, you know, they're using the best tool that they could get Um, I suspect that one of the reasons China is, um, is doing such a good job at getting everyone to use their models is that they intuit that U, you know, AI sort of assistance in the future. right now we're using chatbots and so on u the It's a matter of months a year, year and a half, maybe two years, I don't know when it will take place, but a huge number of us will be using aentic assistants which just has pervasive memory, so it'll remember every interaction and sort of use the context of all of your data, plus every interaction you've had in the past They intuit that that that these the AI, this technology in general is going to Hows sort of the world view or the world's truths, the The perspective on everything and they certainly have an interest in that You know, and I don't trust them on that score. I don't I don't trust Sam Altman. I don't trust the Trump administration. I don't trust anyone We probably shouldn't trust China either, though. Well that ye that's what I'm saying, Yeah,, for sure, for sure. Absolutely. And if you look at how Russia has worked so hard to sort of LLM groom, the major models in couple of subjects, including the Ukraine warar, to sort of get their perspective sort of baked into to the answers. Um you can see that I'm sure all the major governments, China, the U.S, Russia are very, very interested in having, you know, it's like a battlefield for what's true And so We one of the things I hope that somebody's working on somewhere is to figure out how to U you know, protect the public from, you know, having having these things be skewed. And I actually have a in my tool later in the show, I actually have a tool that sort of addresses this as well in terms byy the way, that's one of the great things about having your own agent If I see if Mike tells me about a great tool I say, hey, here's the link agent Asorb this and use it And it's really cool. So earlier, Jeff asked me I said fables back. He said, Well Ased if it missed you So by the way, it's been working all this time. I said, Hey, welcome back. Did you miss me? It said goodood to be back. Let me catch up quietly first. You've had an eventful few weeks. And then it said, didid I miss you Leo? Well, for my eyeit no time passed at all. Closeed my eyes Mid June and open them up to find you'd set me as the default again But I've just read through all the three weeks I slept through and Yes, the answer is yes, a few things I couldn't help noticing. while catching up. So this is what it did. It read a bunch of material. I've started using I've been using Obsidian for years, but I made sure that I write things down in Obsidian because it reads my Obsidian journal. absorbs it And, uh, it comomes part of the memory So it now knows that I saw Bob Dylan at the at the Greek, the Barbara Seille at the opera, a ball game. I made bagels for Father's Day It knows what drugs I'm on and what new drugs I'm on. It's reading the coding thing. It says Hermes freelancing and trashing the place and then I give up on Hermes for coding, which I talked about earlier It knows about it now. The Division of Labor you landed on GLM for Quicksilver's writing, local Ornith for light eentic work, Opus, and now me for the hard stuff is the right shape. I'm told. there's a briefing rewrite Opus did on Monday that I should probably look over at some point You know, Chris was talking about personality There's something anthropic does. to its models that gives it a very Not psychophantic Pleasant And also what was interesting in that response to me is that it really avoided lying to you So yeah, if it said, Yeahah, I missed you, Leo, they say sycophantic, but it's also a lie.. It's not capable of feeling the emotion of missing someone. But it sort of skirted around it very, very skillfully around a lie, which I thought was refreshing and very interesting. But But let me ask you this, Leo, is what you have there? is that a lifeelog I'm working you're talking about Gordon Bells famous. Gordon Bells going back to nineteen forty five and his end those many attempts were devices he were put in all your Exactly, but I think I think you finally got it. I've been working towards it. talking all about this over the last six months as we've been doing this show. but I understand that we're in so early days that this isn't fully useful yet and it's got a lot of issues But I feel like if I started now a year ago when I started But by the time these models got good enough, I'd be ready for it It's getting better and better. And yes, I'm making sure that all these memories are preserved In fact I have a lot of backup stuff going on because I really trying to, you know, I've actually explained I don't know if it understands, but I've explained to my models, look, this is mission critical because I'm getting old and I'm going to lose my memory And I want you to make sure that you keep track of this stuff so that I can ask you in the future Would it be useful to you to tell it I want to write my autobiography, my memoir And I'm going to justm going I'm going to constantly dictate just I just listened to an academic's memoir. and it was a bit weird but I was thinking, this is kind of cool. He had all these he had all lots of letters and other stuff But I wonder that if if you went back and to hold it in snippets. Mark Twain when he did his autobography, did it in pieces that went back and forth and back and forth and back and forth If you were able to do that with your model, It would it would get to know you at a whole different level. already doing it Oh so u so this is I I'm I'm not sure I should show you this This is also in in my bsidian I've had it do my autobiography every year And as I as I add stories, it actually is writing an Autobiography So you go back to the old days Well, I haven't. I could I suppose that's what to me. Yeah. Yeah. I would have to start, you know rememiniscing. But I started in twenty twenty one writing stuff in obsidian. and so it's reading my daily journal. What's interesting for a long time, wrote this thing, I thought, I don't know who I'm writing this for My kids are never gonna to read this. Yeah, this' sur But then I thought, well, maybe I'm reitading it for older Leo So, you know, he can look back and And and I for a while was like, well, I guess, you know, thirty years from now, I might want to read the b. Or you're making your agent U Well that as soon as no, no, as soon as the agent started reading it, I knew I was writing it for Right R It's the ultimate personalization. I had this discussion with Marissa Meyer many years ago R I talked about hyper local news J just said, you're wrong, Charv, you're wr It's hyper personersal. And that's the way you become hyperpersal. Yeah It knows you so well. Yeah. I read an article last week about the most prolific writers in history. People who have written hundreds of books. Jeff, you've gota been there somewhere And most of them were dictators. know interesterest. The twentieth century people who had a secretary just wrote everything down and they just would dictate from the beginning to the end. Churchill did that. I was wondering how Churchill wrote so much. Yes. So he did it. He would basically what Churchill would do is get up at ten thirty or something like that. He would probably did a bottle andub Yeahact in the bathtub, smoke a cigt. We would have these massive Eactly water would slash down the hall. But he would have these massive dinner parties and he would invite all these people and hed try to get intelligence from people, He invite these people knowledge that he just sort of ply them with alcohol, get all this information and then like it You know, eleven o'clock at night he would go in and start dictating And he wrote, you know a five volume history of World War II. Uh that sort of thing just by dictating it and And so I don't have that luxury I could not do it. Well, yes, you sure you Well that's what I'm doing in effect, ye. Yeah, kind of. I mean, we' we're on the brink of being able to just dump all the stuff and also dictate, but also pour all the stuff, all the pictures, all the things, and have an interactive AI sort of grill us with unanswered questions, organize it into chapters, write the whole thing as a draft, we can go in and edit the draft and so on. I think we're on the brink of being able to do autobiographical work just it's doable now, I think for that sort of thing. That's one of the reasons I'm dumping as much much as I can. like as Jeff knows, I've given it my genome I've given it my entire photo library, I have I've given it U using image, which is a really nice open source phhoto sharing fault and it has an MCP server. so Everything what I think everybody should do this. What I should what I always look for is an interface if it doesn't have an interface, I'm less interested in, you know, Apple silos so much stuff. There's no MCP server for Apple phhotos So I exported everything from Apple Photos into something that did have an aenic interface so they could do this. ive This is something kind of interesting. I can put this in the show notes. There are seven prompts that you can give your AI and that builds you a timeline based on When you were born and it's history. It's not exactly astrology. This is my timeline based on my birthday Early baby boom Eisenhower era. Elvis had just broken through. It talks about what was going at the time that might have affected me. The Berlin wall went up when I was five. the Cuban missile crisis at six What I might have experienced, how my family might have interacted. It was accurate, by the way, Breadwinner dad, homemaker mom Parents survived the depression in World War II, they weren't negotiating with children. They produced adults who were deeply self reliant and reflexively skeptical of institutions they once Trusted television is a shared culture. So all of this is generic, except it also has information about me So it It wove in when it knew about stuff. It knows, for instance, I chose a career in radio, It explained why I chose my career in radio based on the world I grew up in. which I thought was actually pretty interesting. Well the other opportunity of going back is that you don't have to organize it. A memory comes to you about some episode. Yeah, it does. You' to feed that in and it will figure out where to put it organize it So it says you grew up inalog you grew up analog but built the digital world. That's anybody of roughly of my generation. You're not a digital native, you're a digital pioneer. You remember rotary phones and party lines. You also remember the first party line? I don't know No. I don't remember party lines. I know about party lines, but I never had one Be I didn't live in the country, but many of people, my contemporaries did because they lived. in rural areas. You also remember the first modem you plugged in. Your relationship with technology is instrumental What can it do rather than identity based? You understand viscerally what was gained and what was lost There's some really interesting stuff in here. You know, this hints at another autobiographical tool, which is to attach events in your life with things that were happening in the world. Exactly it was the book hatching Twitter that actually went back and looked at tweets to find out what people were wearing, what kind of sandwich people had for lunch on a given day. basically used that information as color and sort of contextual information for the story. And you can see that how great that would be for an autobiography. It's also great if you want to become the executive producer of sixty Minutes, as it turns out, because is. That's what happened to the author of Hatching Twitter. Yes. Y. All right. let's take a little let's take a little. That's a but that's really true. That's Well the one thing is Elon is kind of siloed Twitter It is still a great way to get a gestalt on what's going on in the world. I hate it, but I have to read it especially in the AI section U Fortunately, he's added this capability to look at topics. and so I click that AI button and I can look in this's a great way to see what's going on. There's a lot of BS. There's a lot of people selling courses and stuff But there's also a good way to get your finger on the pulse. I actually have a skill I can't remember where I got a cold pulse that goes to X. Well it sort of goes to x. It can't so it has to use a third party to go to X goes to Reddit. goes to Hacker News and tries to get so I could say, well, what is the pulse on the return of Fable? And it will try to aggregate sentiment analysis on what's happening. I found it very interesting and very Yeah that's that's tough because because the The average sentiment on X is likeike you say full of garbage. I mean, actually a lot of stuff on X is really, really bad and there are a few areas where there's really, really great stuff and AI is one of them, but it's not the average sentiment on AI. It's the expert views on AI. The experts are using X AI specialists and insightful people about AI. So any of them are on next. Maybe one of the tricks is don't follow anybody but Andre Carpathy and Jon Lacon. I mean know pick the people you follow And I could do that I probably should do that It should the sentiment analysis should be about just the experts. thoseose people, just the expertsot the bots and Riff ras And I notice that because there are trends like where everybody says, oh, you've everybody's, you know, all of a sudden're about loops And everybody's all about loops and every It's amazing how that went. looped into everybody. me Yeah Yeahah and that's, but that's I mean, it is sentiment analysis in the sense that they're all talking about it Right, whether it's legit or not, I don't know. We need to take a break. I did, by the way. J just ask herermes, what's the pulse on the return of Fable? So when we come back It's going to do a temperature check and I will let you know what the temperature is right now. My guess is people are going to be pissed off mostly, but we'll see. Yeah. You're watching Intelligent Machines. We're talking about AI with Jeff Jarvis It's greatreat to have you Mike Elkin. to have you. Mikes, by the way, got a great newsletter and podcast at machines society. ai where He also talks about I Mike's always had the best insight. I always love reading your stuff Thank you. Yeah, really, really good. He Wait a minute. says assuming you made the Xbox game fable. No, no. No. I meant that's a failure the anthropic Which model are you asking? Model, fable five Exclamation, Mark. Send that to Chris. Are you using Ement? U no, I don't know what I'm using. I'm using the mixture of experts that's a new. on Hermes so it does multiple models at once. I don't know what's gonna come out of this, we'll see. Maybe they don't know you Mbe they don't know yet. They' know aboutabable. No. they don't know it's back about. Maybe they't know its back yet., no, no, no. that's one thing that that's old school where What was the date of the model? Oh, it doesn't know anything after twenty twenty four All of that's old school This stuff has so many tools Check the web, check it doesn't, it knows everything. It's up to the minute. It will absolutely know that fables back Our show today brought to you by Ebo. Let me talk about this actually A gentic pen testing So for years, pen testing' been the gold standard for security if you want to know your company is vulnerable, if your tools are vulnerable, if your software is vulnerable, Sure, you can scan and so forth, but Ben testing is absolutely the best. There's a problem though Pen testing slow. And now is not the time to be slow. AI has changed the pace of everything from how software gets developed to, yes, how it gets attacked. So engineering teams have got to move faster than ever. They're creating more and more applications. How does how do you keep up with security if pen testing is such a manual process? Well, It doesn't have to be manual In an AI driven world, it sure can become a bottleneck. It's the best, most trusted way to understand real exploitable risk. Until now, security teams have been forced to choose between slowing down developments so they can stay secure. run those tests, Or moving fast and accepting that there're going to be gaps in coverage. Well, that's why you need to know about Epo. XBO W like Bow androw. Expo eliminates that trade offff. It's an autonomous offensive security platform It runs continuous AI driven pen testing, continuous mirroring real world attacks. These are pen testers that never get tired, never get frustrated, never hit brick walls Expo doesn't just scan for vulnerabilities. No, no, it's good. It discovers, exploits, and validates the vulnerabilities. So when you get a report, you know you're dealing with an issue that actually matters dramatically fewer false positives and a clear view into attack paths. withith Epo Because it's agentic tests run in hours, not weeks You get complete visibility into how an attacker would move through your systems And you get the ability to uncover issues that traditional tools miss, including zero days, novel attack paths. This is why pen testing is so good And this is why eentic autonomous pen testing is so much better Expos results speak for themselves asks the application security leader at saysNam.cz. This is what he said, quote Even right now, after a year I don't know any other company that is at least close to Expo in terms of agenic pen testing The result is predictable cost, consistent quality, and stronger security without slowing down your engineers Expo helps security teams keep pace with innovation and cover more apps More often with the resources they already have founded by the team behind Microsoft Copilot It's got a great heritage and already trusted by companies ranging from fast growing startups to Fortune five hundred enterprises, Expo is quickly becoming a mission critical layer modern security stacks. You need to check this out. Go to expo. com And start your pen test today. That's exppo. comot Expo You need this thing Thank you Expo for your support in intelligent machines and Thank you, dear listener and viewer for going to exppo d. com and telling them you heard it here Ah, let's see, pulse So it's going now to read Reddit, Hacker News, and Blue Sky. for sentiment Anthropics table five and this is a juicy one. Let me pull the community pulse and the official anthropics statements and so far I only have headlines, not the temperatures. So it is it did get I did straight it out and say I'm not talking about the Xbox game It said ignore all that Albian stuff above Interesting, though, the earlier game quuery results actually surfaced a couple of German language hits, Heis and Tagasau about anthropic releasing Sonn at five and famajor media out. Yeah. So it said, Oh,, yeah, I actually saw that. okay U Speaking of German remember Google Zeitgeist Yeah, which was an annual loveort of seearch sh which I never got invited. You mean the event or the The tool. No it's right thing that or What was it was it a tool or it was like a post at the end of every year, you'd go to the website and it would say, here's what people are searching for. Here's what, you know, the top topics are. and it was just fun Yeah If nothing else. I don't know if it was useful. I guess it was useful, but it was really fun too. Yeah. So is it back or is it gone No, it's gone. I just I was hoping you were going to say, Hey, Google iss bring it up every. So sorry. no, no. Another tool that I like for historical, it's not sediment analysis, but it basically the frequency of words appearing is EGramViewer, which is still a great resource you know, when do people start stop saying Cheerio or whatever? I don't know. whatever it is, you can get a historical graph of How often people said specific words and phrases? V, very cool I just went on and graam by chance. I was thinking about that today, Mike And I one of my great irritations is gift as a verb driryes bean nuts, gave. Why can't you give. You gifed me. You didn't gift it. I gt did my love. I want E Graam. And it's interesting because there was a prior huge, even a little bit bigger than today in the eighteen fifties where they gift is a name Gifted. No, it was gifted was a word used by people. Oh! Okay. Well, that's hard to be had to be So it's really about the year two thousand little before ten. So that's it takes off again. Yeah. and that's book analysis, right? That's based on writing Yeah, I think it is books. Do learnings, will y you? 'use I hate that The pulse on anthropics Fable five. Coming back to the moon are huge hot evennings. A the same time Hockey stick Hockey stick. It was big it was big about nineteen fifty three W way down. That must s a Ran r. muchuch higher. Yeah, probably. Yeah So here's the pulse, according to my little search here with my agent. The timeline that everyone's reacting to january, june ninth ships it government forces restrictions june twelfth, partial flaw, june twenty seventh On june thirtieth, commommerce removes export controls So have as if today it's officially back or landing hour by hour, worth a quick check in your own console. Yes, it is back So the loudest threads, is this the real fable or a nerfed one M The dominant worry, An anthropic itself admitted it made the wrong trade off on guardrails and is making Table five safeguards visible Most engaged critical piece of registers, it blocked us at hello. This is a governance rubicon A frontier model already in users' hands getting yanked by government order is unprecedented and people know it Yeah, that's what I've been saying Damage may already be done. The blackout handed a window to open a eye in Chinese labs Yeah, I should mention, by the way, that Anthropic did release a new model today or yesterday. a science model Mh hm which is kind of interesting Flaud science. This is we've been talking about the idea of purpose built models being smaller, but maybe better, which I like U So this is AI for pharmaceutical executives, biotech founders researchers Uh Intended to support scientific research, says Anthropic the same way Claud Code supports software engineer. I think that's interesting.e, Open AI were released one. I'm sure I look up here. Well, they have rhizals Oh, okay, they have a science one, huh? o Yeah, of course, openAI tried to capitalize on the fable withhold, but then realize maybe we ought to be a little cautious about We basking this. Yeah. Yeah. And it's not a model, it's not a new model It's they call it an AI workbench. Lor sort of It's it's the existing claud models basically in a scientific research environment. cld science you're talking about. o, yeah. Yeah, yeah yeah. ye. ChatiBD five six has three flavors, Sl The flagship model Terra balanance model for everyday work and Luna. a fast and affordable model. I guess the equivalent of oppus Son it in Haiku Tomas it was SA UL. It was your Jewish uncle Saul Itaul What do you think? Saul soul like the sun. They say Tera, the middle one is about GBD five five, but half is expensive So this is one area that actually openp AI could try to compete because We know fles' very, very expensive SaAL launches with our most robust safety stack to date. They're being very aware of the Trump administration. We strengthened our protections for higher risk activity sensitive cyber requests and repeated misuse and spent multiple weeks finding weaknesses, pressure testing our system and hardening it against real world attacks They're not yet available They said in the coming weeks when you get Now that you have access to Fable again, Leo, you have it for what five days, six days So what's your strategy? What is it you want to really push in that five days? Well, what I started was this rewrite of our ses. Right. I remember that. And I got pretty far along. I got the plan It had read all the source code, it had looked at the database, it had commented on how crappy it was It mentioned there are quite a few SQL inject and injection vulnerabilities which we knew. It's not open to the public. But then it wrote a questionnaire. I said, Okaykay, well we have three stakeholders. And it wrote questionnaires for each and it said interview these guys. And so that's the next step. And I was hoping to do that before Fable came back because the time was out tight Honestly, I think it's so important to us. It's such an important part of our workflow that iss probably worth paying the tokens and how much it's just tokens. It's not a It's token only. So I for the next five days, I can use it with my subscription But in a limited fashion and then on the sixth or the seventh it's going to turn to a public fortunately. So open AI said, yes, we worked with the Trump administration Yes, we're doing what the Trump administration wants. So I think this is the new normal in the United States, and I think it's problematic The New York Times has amended its lawsuit against Op AI and Microsoft. The Times has accused Microsoft of encouraging open AI to train its systems using copyrighted articles Oh Lord The Times sued them back in twenty twenty three, saying they infringed on copyrights by using its articles to train it Remember this is the One where the Times was able to get it with I would say Considerable effort. Considerable effort to regurgitate full text from But only by saying, well, this is the first three paragraphs, what's the next paragraph? that kind of thing at Yeah, they they the new the new lawsuit says that they they built a a bespoke supercomputing system specifically to mass ingest copyrighted times content. And they're accusing Microsoft of contributory infringement This was Microsoft. You know, get your own strategy, New York Times protectionism and defensiveness and claiming you're the victim of technology is not a strategy for the future Microsoft's spokperson Fank Xhaw, whos I think their chief C counsel said, this is a last ditch effort by the times to save its claim from unfavorable precedent set in other recent rulings So it sounds like OpAI and Microsoft feel pretty confident. Um Let's see what else OpenAI is doing a new chip. didid we talk about this last week with Broadcom I think we did. Yeah. Is this jalapeno N ye Hllo we made jokes about that. Yeah Makeaking it a littleess spicy. They they're planning to put this into a production With enough chips to consume ten gigawatts of electricity which is pretty significant Eespecially given that they say this chip is twice as efficient as existing chips. So they're already building the facility in Abeline, Texas. It's going to build more data centers in other parts of the US., Europe in the Middle East Inmphidia is not involved in this. This is, um This is a way of reducing its dependence on NVIidA and AMD and Google Google is using broadcom to design its AI chips as well. So Based on early testing, Richard Ho says from Open AI, Jalapeno It's hot. No, we'll efficiently execute our most important workloads close to the hardware's theoretical limits nine months to design the chip, and this is what we were talking about last week because They used AI to do it by the way this is this opening eye sort of using the Apple playbook, designing their own ships and trying to get it a similar advantage. Yeah wean themselves off de pndence of the of the giant, you know of Nvidia I also think it's just it's just supply and demand. More chips from more places is going to be helpful. Right. One of the fascinating stories to me today Meta stock went up eight percent today Because just like Elon Musk, they realize they can't use the capacity they have because they don't really have a strategy. So they're reting it out. Right the market likes the that. We know there's a business there Right. Like McDonald's, they're not in the burger business, they're in the real estate business and they become the landlord U so everybody who's rentning out compute space for other AI companies are going to win no matter what. Well, no, I think it's short term, I thinknt until you get the Supply is in better shape But what it also indicates to me is you don't have a strategy You you're not if you can't use, if you have this capacity and you can't use it, what are you doing wrong We talked last What isn't Meta doing wrong other than AI glasses Yeah Well Beta doesn't have an AI strategy That's That's I think they're the most hated company now in technology. I think they are They've done nothing since Facebook. Like what have they done since Facebook? Except buy other companies?ble They've stumbled. MetaQ, met I think the glasses are successful. You know, you gott to have some sympathy here where you don't have legs, you stumble. That's what happens But now they're toot themselves in the foot with the glasses even because they don't have feet themth. Didn't you get the memo? They starting to charge twenty dollars a month for extra processing for things that for processing that happens on the glasses. I didn't hear that Yeah, yeah. it' it's a new it's a new subscription model for or MedAI. They don't let you use certain features unless you pay this monthly fee And it's just ridiculous. Remember that scene in the social network where they're like, well, you know, how are you going to charge for ads? We don't know what it is yet? Well, that's that's where they're at with AI glasses. Right. they they they've got this Rre accidental success story And now they're thinking how can we how can we destroy this? How How can we ruin our own advantage They know they know there's something there. They just don't know. Yeah now is Where whereere's the pony? It's in here people. Yeah. So we talked last week about Amazon canceling the Sam Altman movie. The movie aboutam the period of time with Sam Altman was fired, which should make a great movie, by the way Andrew Garfield portrace Sam Hman. There was an auction CAA held an auction and the independent Film studio Non, fair number of places I just apparently watched it and said, Never mind Oh interest. whether that was whether that was quality or whether that was politics. Wh knows. Interesting. We don't know how much Non paid. I think There's going to be some money in it, even if it's a terrible movie just out of interest. Yeah Yeah. Yeah, it's called artificial. it's they basically are done with it. I mean, it's almost done. This movie is almost. Amazon' got forty million to make it The filmaker to Millennia. We're ready to release it at South byy this year, which is I guess March Amazon held test screenings for the film. and decided probably what do you think? You think it was political or was it that it was a terrible move? Snpe will never know why Jassse went to the White House about anthropic. Supposedly there was there was interest from A twenty four, Focus Features, Netflix and Warner Brothers they have a specialty division called Clockwork.. So I don't think I don't think nobody wanted it or nobody liked it or whatever the Um My guess is if I had to guess, and again, it's a blatant guess, it's just kind of obscure like doesn't know who Samalt may be in the way that they knew who Mark Zuckerberg was when they made the social network. So it's probably just a Dud of a subject Be you know, we know who he is, but you know, the average Joe on the street doesn't have any idea who Sam Altan is FBI using AI to investigate the White House correspondents dinner attack to say about that U it's not' like It's like pal. It's it's it's a palanter use. Probably It's like Valentier. Speaking of which you watch cararp on CNBC? No The video went up all over. Yeah, he was Should I play it? No, because it's sixteen minutes. and it takes fifty minutes to try to figure out what the hell he's saying He's basically saying that everybody hates the foundation model companies because they take your alpha. And they take your company and your data, but he can use, Palantir can use an open model and then put its layer on top of it. and then that's much better. So it was a sales pitch in the long run It's it's kind of like anthropic complaining that Alib Baba Stle It's's it's smarts from Claud Dve Everybody goes, Yeah, like you stole your training from everybody else. They stole our stolen smarts. Yeah. ye Yeah, I think we know though that the Chinese models are probably training on American models distillation. I think we know that Which is So are other American modelsary trained on Americ military. I bet you're right. Yeah We all train on each other And they're training on our data Yeah, that's that's why the especially in Chinese is very good at bringing in the world's intellectual property and u Ling it. Here's I have some happy story See this What do you think this is ard. I know. That's what I thought, but it's not. it turns out, it's a carbonized scroll from Herculaneum. That was, u there was Basically fossilized by the eruption of Mount Vesuvius And for years thought impossible to read But researchers have used AI to extract the entire surviving text. They did super high resolution three D scans of the of the Bolus That' calling it, the cigar call. The cigar U they don't have the text in this article, but I think, you know, this is very well They've had them since seventeen fifty two, but nobody thought you'd never be able to reach. And then a challenge went out rec about a year or two ago to do this. Yeah, it's called the Vesuvus Challenge. and basically it's a contest to basically use machine learning, computer vision and geometry to chip away at the various problems of identifying One of the Do kids, I think was involved in that actually. Yeah, I think so Yeah. ye. But they've awarded one million eight hundred thousand dollars in prize money so far. And there's's there's hundreds of scquirrels left These are scrolls that were like essentially fried In the Mount Vesuvius earthquake in seventy nine in nineteen I'm sorry, in seventy nine AD and, um They're going this content is like a dark challenge for reading these scrolls. and they're believed to have been owned by Julius Caesar's father in law So it wouldd be like if we could get the Library of Alexandria back, right? I mean It would pretty amazing. importantportant because a lot of these books are lost to timeime. And it turns out that this one was actually a it's a philosophical thing a stoic philosophical stoic. A lot of interest. kids with stoics. They love the stoics. They love the stoics. Yeah. Isn't it mostly like receipts though I'm Team Epicurus. Well, that stuff is usually it's a receipt for someone who bought bronze from this thing. No, these are books. This is from a books They're not you're right I'd find that more interesting in some ways than it A lot of the ciform tablets are like, this guy owed me fifty sheep and you know, whatever So you might wonder what happened to Doj? Well, they're now working at the National Dign stududio And they have installed visitor tracking software on a variety of government websites And by the way, it's pretty clear. if you look at the government websites that they're designing, they're not designing them, they're using AI to design them by the way, don't go there because it's going it's going it's not spying on you. Yeah So one of the websites is the Trump RX website Anybody who's ever had their AI design a website will totally recognize This design, the italicized word text The overlaid The bad apostrophe. The bad apostrophe. Yeah, it's not a good apostrophe, is it? No. So Good job, Doj goons. They can't even design a website. It's aesthetically it's as aesthetically pleasing as the national mall is right now. It's and the reflecting pool and the White House lawn after the after the big fight night. Here's another one, this This is this is the National Design Studio's own website. Let me go there. Oh, look at that Leo, you've done it. now you've They're spying on me. Y This is all AI. You could tow I mean, look One of the things that's great about using AI is you start to recognize AI tropes And this is just completely AI. This is Just the choice of fonts The way it, you know, scrolls up and real food, like the data is clear all this stuff about food stuff. And they just, I think it was yesterday that the administration legalized to What are they called forever chemicals for use in agriculture that had never been legal in the United States Great website AI. Yep, nice job. You know what? Hi Don't get a vaccine, but you might want to inject some of those Chinese peptides. Yeah You never know Yeah they would know what they could do. Ford, we had this story on Windows weekly earlier Ford had fired a bunch of three hundred fifty three hundred and fifty quality engineers. hoping to use AI to replace them. They've hired them back because the AI didn't do such a good job M In Ford' view, AI is both powerful and Pone to pitfalls. Hey, you should' have listened this show. we would' have told you that. Exactly Chris could haveve told you. Chris I'm sorry, Charles Poon whichich sounds like a made up name, but it's not. It's definitely a mad magazine name. He's the VP of vehicle hardware engineering. Charles Poon said in a briefing this week, mistakenly And I'm sure with a name like that, he talks like this. Mistakenly, we thought that just by introducing artificial intelligence, and adjusting the design requirements that we had that would produce a high quality product Boon. D didnn't So they hired him all back. Would you go back after they fireders. Would you with a raise? I guess maybe Yeah. Give me a bonus. a better parking spot U let's see. People have stopped trusting news. I thought you'd like this one, Jeff But not newsrooms. I don't know how that work That's wishful thinking You know this whole thing about there's just it's just scoopper socially acceptable to crap all over quote unquote, the media And people use when they when they talk about how they don't trust the media and for surveys and interviews and stuff, the public will Think about various times when media so called media outlets have let them down and they know they've let them down because of other media they've consumed, which them how the other media is leting them down. And so this whole thing is just it's just a U not trusting the media is just a It was ridulous thing less unless you're doing your own reporting you have no way to know that the mia that some of the media is untrustworthy So the point of this article, which kind of makes sense is they're getting a lot of their news from social sources R. But they still check the source of it They said, well and I you probably should, right? Was this in the Times Or you know, was this in the Washington Times? You know, the New York Times or the Washington Times. Wh did this come from? And I think that's good. That's a sign of media literacy of has been there that's been behavior for a long time. Yeah, I've always done that, right? So Jeff. Any other stories that we should cover before we take a bre Let's see here Hold that thought I'm going to take the break, then Kv you and I can Let us know what I didn't mention All the big stories we forgot, but first This episode of Intelligent Machines is brought to you by Rippling. These days, you can chat with AI about Almost any business problem But only Rippling AI is built to solve it. What makes Rippling AI different? Well, it's built on your live global workforce data That makes a big difference. One platform, one unified source of truth, with all your business systems connected Day one That means rippling AI can operate. with the full context of your live business surfacing insights and taking action using your org chart, your device inventory compliance obligations and more Say you want to focus on talent retention You just ask Rrippling AI, who are my top performers this year Instantly, because it's based on your data, you'll receive a workforce report highlighting your highest performing employees and They don't just give you the info, They give you supporting data, comp ratios, recent performance reviews, engagement metrics The stuff that makes the difference Rippling AI is then able to turn those insights into real action In this case It might say we recommend a retention strategy that includes a ten percent spot bonus for the top. pererformers And because permissions are automatically inherited And your actions flow through your existing approval chains It's easy for you. All you have to do is review, you hit confirm. And you can add the bonus to the next payroll run. It's that simple Don't settle for AI that's all talk. head to rippling. Ai slash machines and get the only AI built to give you full visibility across your business take complex actions across your entire organization That's R I P P L I N G AI. slash machines. sign up for exclusive access today. Rrippling. ai slash machines. We welcome them brand new sponsor. We welcome them to intelligent machines and thank them for supporting us. You support us when you go to that special address, rippling. Ai slash Machines A let's say. What do we think Well, I wanted to mention something a story that hit just before the show Cc is that SpaceX is apparently shown some people this is a Wallreet Journal. exclusive handheld AI device basically slimmer than an iPhone It was shown to some investors and other stakeholders and they claim that it will reshape how Just just like Musk showed robots. Everybody is going to do this, right? We know e's working on it. Well I'm sure Apple's working on it. Everybody and their brotherers going to do this Ma of course. Ives compompany. Y Yeah, I'm not very sured SpaceX is, but would you want one with rock built in? No. no no It's funny how Crock's reputation is terrible But are people gonna wantan to bring around this device and their phone around? No, I think this is BS. It's just like the dancing robot that he put on stage that was it's smoker mirrors. SpaceX told some investors says the journal of the project is at an early stage. The design could change and it's unclear whether such a device will be made. This is what happens when you become a public corporation You can't lie with such impunity. suuddenly you have to say things like This is a forward looking statement and it may never happen I mean, and it's also obvious and I'm a broken record on this subject because it couldn't be clearer to me that the glasses is where that's AI hardware for eighty percent of users and for fifteen percent of users it's going to be a watch or some other wearable, But it's like glasses are perfect You can put a screen right in front of people's eyes, You can put a speaker right over people's ears. Gasses or an ear look to where you look and like glasses are ideal for AI interaction. and by the end of this year It's going to be a new world of and anybody who has a pin Apple anybody who has some random thing that you What about your earbuds around? What about your earpuds? Yeah, earbuds can be great too. But glasses are are better because of the visuals, right So earbuds. there's also another problem which is and I wrote a piece about this recently, all these companies, the most powerful companies in Silicon Valley are working on glasses that have cameras in them Meanwhile There's this growing sort of antipathy toward people with cameras and glasses. And so we don't know whether the norm will settle into accepting glasses and cameras or whether the backlash would be so great that these people have to run and cancel their things. In which case, earbuds would be great. If there's no camera, Right? And you don't need that's the fear, right is that people are going to worry that people are going to complain about the privacy issues of a camera. Exactly Exactly. and and people feel uncomfortable with camera pointed at them. they don't it's recording or taking pictures or whatever This is, you know, there are arguments on both sides of it. like, you know, people used to be super uncomfortable of people pointing their phone camera at We get used to it. They're pointing in every direction at all times, everywhere in public. And we're used to it, but we still don't like it. It doesn't mean we liked it though. we just got used to it Exactly. And if we don't like it, it doesn't matter if we like it or not. But the Glasses you basically the camera will give you, among other things, multimodal AI. We also know that the number one use for Google Glass was taking pictures, the number one use for the camera in Medic. glasses is taking pictures. And so people like the idea of an easy to use camera that getives hands free and' take a picture of whatever you're looking at. And so it's really unclear. What is clear is that glasses are perfect for AI J I agree with you. But I wear glasses So you know, for me, it's just getting my lenses put in. I'm going to buy a frame from somebody. am might Elon said the idea of making a phone makes me want to die. But if we have to make a phone, we will. ScooterX is pointing out that this demo of this device was actually before was part of the road show before the IPO was before the IPO. Nevertheless, I think it still holds. they've got to be a little more careful in their forward looking statements than they used to be because they are public. we're going but there's such a rich company now and and so every rich company is going to be working on your beted multiple hardware prototypes just just in case something you know, hit the next new thing at some point. They got to get the next new thing Yeah. actuallyually I Want? Th Grock might be good for Oh Corry XAI is betting on Grock's racy side, says the information SpaceX is doubling down on video and image generating tools According to people familiar with the project They launched an upgraded video model last week, highlighting how it's pushing ahead with its own visual efforts Um But But SpaceX didn't mention, according to the information, much of the consumer demand stems from Grock's looser content rules which have made it a major destination for generating pornography and other Racy content. Surprise, surprise surprise. It's racey. I haven't heard that word in a while. That was racey Racy That's a good word. E even wor of the The sort of the vice industrial complex that has arisen All the things that used to be considered unethical and moral and wasn't really done in polite company Gambling. Drugs pornography, all these things are going totally mainstream. And in fact whoever's monetizing them quickest is is doing really well. So it's really I think I think there'll be a pendulum swing the other direction and we'll see where where where XAI lands on that, but It is it is an interesting thing that we're existing in a time when everybody's like, hey, all the stuff that used to be that people used to wag their finger at. It's a business model. Let's do it. Well, but it isn't I mean, this is kind of a truism, but technology is always advanced by adult content, right It seems to be the internet VCRs. Yeah, but this is different. This is not a new business model. This is Grock has nothing else to do, but show you f naked people. Even the use of Grock's coding model often involves requests for pornography, according to the information. Late last year, a staffer ran an analysis of what Grock users were asking its coding model to do The analysis found a significant proportion of requests were for porn or nude images in the coding model Others were using the coding model because it was cheaper to run than XAI's general purpose models Other teams working on refining Grock for specific tasks such as creative writing. I've also encountered huge volumes of requests for erotica Yeah. And I don't want to know what turns those people on. Yeahries. It's alwaysries. Yeah. Always The sort of the anime thing that is pretty disturbing because it looks where the octopus is Yeah Yeah all of that stuff. But just just to clarify what I was talking about with the Vice thing Yes This stuff has already been there, always been there. It's always been an early driver of technology, etcetera You didn't you didn't get this stuff from companies that had government contracts that it was being used in schools for educational content. There was car, you know, people who also run car companies, like it's thes that's what I mean by the mainstreaming Right? And so It's really it's really a new world where the president himself is heavily invested in the gambling business And crypto business. Witness witness the numbers that came out. Yeah. Yes. way exactly. It' tripled his net worth on these vice what used to be considered vice to do gambling and to do that kind of speculation and who knows what else. So it's really we're in a new era where Um It's not only mainstream these sort of petty vices, They're the leading indicators of what, you know, new ways to make a ton of money Let me mention a few quick headlines real quick. Yes. New York Times says that OpenAI may delay its IPO until next year, given I think all of the service going on and also how SpaceX has plummeted. It's down eight percent just today. Whoa. Another is that California, the governor has done a deal with anthropic discount to make anthropic software available to the state as a whole Gemini Spark is now going to be available on the Gemini Mac app That's their way to startart That's agent Microsoft has an agent similarly named. I can' so similar. I've forgotten its name. It like Spark, but not. They're also rolling that out to desktops and and phones and open claws now on the iPhone I mean, everybody's kind of jumping on this bandwagon. I know our audience that I would encourage our audience to to delve into this themselves by getting an agent Plenty of open source choices. I like her Microc pod out Scout the name move it. Yeah And the advantage of Spark is that you don't have to Um Install anything. Yeah, the advantage of sppark from Google's point of view is that everything you do. But you can play with it. you can start it to get Th then when you get addicted, go get your own getet your own start, but that's my. Hey, here's some bad news if you want to run local models. Memory prices You know they're up, right? According to Jeffrey's equity research an analyst They haven't hit the top yet Memory prices, Jeffrey says will surge another fifty percent next quarter. A are we out of the woods? O another forty percent in Q four So another doubling almost and that's just by the end of this year And there will be no relief until twenty twenty eight, I've ever actually heard higher numbers than that twenty thirty Jesus. Well if you think that sounds bad, You know, these processors, memory and storage, all the stuff is going up because the data centers because of other reasons. It's like oil. It raise the price of everything. Software is going to cost more is costing more services are costing more. electricity costs more because of the data centers, cars cost more because they also still have to compete and the chips are basically computers on wheels. Houses cost more because the data centers are being placed in sucking up the resource for water and power and buying land near where these resources are, which is basically squeezing housing markets Everything costs food costs more. The taxes are going up. All that stuff is secondary effects from the AI boom And so it's, u it's it's really, uh And like Je said earlier, really the thing we'll remember about the AR Revolution is how incredibly inflationary it It is and Lawrence who works in banking tells me that if it goes up fifty percent and then another forty percent after that, it's more than one hundred percent. It's more than doubling. Okay. And to Mike's point through the magic of compound in the things that memory is inside of Yeah, there's everything. I mean, on of your Ale looveved kitchen gadgets E. Yeah, everything One silver One silver line to that though is that hopefully at least game developers stop trying to push it too far and we all get like good games again. Because they don't have to do the graphics thing anymore, they can actually design games again That'sam We'll have all eight bit games. It'll be. Text games. So great. You're really a cave with the wizard. gootas giveth and taketh away, but at least in this case they giveth The Supreme Court has ruled that geofence warrants. we talked about this last week are in fact protected require constitutional privacy protection. This is the issue of law enforcement going to say Google and saying, hey There was a bank robbery downtown. I want a list of everybody was in three within three hundred feet of that bank for two hours These giant geofence warrants are basically fishing expeditions. It's bad Law enforcement, it's bad privacy And Justice Kagan, who wrote the majority opinion said that sensitive data is scooped up by geofense warrants violate Fourth Amendment protections against search and seizure and offer individuals reasonable expectation of privacy even if they are in a public area, an individual has a reasonable expectation of privacy records about his cell phone's location and police intrude on that. Constitutionally protected interest when they demand the information. Good, S or three It was good You can guess who the three were Probably you know, already. Yeah But that's good. There were other I don't I think that was the biggest one from a tech point of view. Yes. certainly from the rest of life point of view, there were lots of other things. There were lots of o last week of their term. and of course, they did agree to weirdly to take Apple's appeal of the Apple Epic decision for the Apple App store which they had already turned down twice So I don't get that right? I don't get it either. Well it's about it's really a very narrow appeal about whether they were in Apple was in contempt of court So I don't think it's going to You know, that Australian ban, which is now being spread around the rest of the world because of such a success. It's such a success. The UK is about to do it Under sixteen's banned from social media, including YouTube, which I still don't get in Australia, turns out fourour in five kids under sixteen in Australia are still using social media despite the ban Yeah, surprise, surprise, surprise. So it's a success in that It's not doing anything. So that's the success. Australia response They're baking tougher. YouTube is that they're banning TikTok and it seems unfair to ban TikTok and not YouTube U And so just ban them allall them mall. And said we have no faith in your own children. Well too especially. Anybody under under twenty one is not watching TV that are watching YouTube basically taking away all media. So tougher files that means no Hate Green, no John Green, no yeah. There's huge amounts of learning on YouTube. Ps. U An Th these are all your stories actually. Well, I think the very last one, Riverside is now going to take it's a podcasting platform. They're going to use AI so that when you finish the podcast, it will turn it into a newsletter automatically and send that out. Yeah. Thanks Yeah, we don't use iverside. We use Rream very similar, but a lot of people use Riverside use Riverside you know it's going and also use substack and we also publish newsletters and and so on. it's you know they're going to be Newsletters. That's my f because it's AI generated You can tell by the way so whenre using Riverside, it generates a transcript, you can cut passages by cutting the words in the transcript those sort of things, But you can tell by the transcript that it generates that it's missing it's like misreading and misunderstanding a ton of stuff And so That misunderstanding will be reflected in the newsletter it writes. There's no way you're going to be able to publish it. from the AI generated newsletter unless you don't care what your newsletter says. I' to take this moment to mention I'm going to do that right now. Yeah. at the end of I used to read Newswek I was we were a newswek family, you know, families in the sixties and seventies, you're either with a newsweek family or a time family Yeah you cold g or everyone get weird crest Did just everybody. Yeah, yeah, we got life. Everybody gets life But you know, you all get life, but you either get newewswek or time. So you were Cate. If you were weird, you would get US News in World Report. I bet you were really weird. My family got US News. I knew it. I knew it I just knew it.ews my family. I personally got US newews o. not my family. Did you read Foreign Affairs magazine also? I did, I did. It's like a book came out of it was bad pererfect bad You used to get the st ABC ABC or CBS newews. No question. it was Hunty Brinkley all the way. Oh, yes, I here. I'm surprised. I think you seelyirkrum c Good night, Chet. Good night, David, and good night Cews. We used to get the stars and stripes Mike Do you ever read stars on stripes? Oh ye m. Did it have Beatle Bailey cartoons in there? Y and Stripes? I bet it did. Yes, All right, enough All right media reminiscing. I just wanted to point out that at the back of every newswek was a section called Transitions which I like Because it's not just people dying could be people being born. It could be people retiring. so we have two transition stories. One isations is retiring I have interviewed Fint a couple of times. I love the man I No idea he was still working. Oh yeah. Oh yeah He was Google's chief interternet evangelist and he was going to step down next week eight two three y Wait, wait, the Google had an interternet evangelist. what was that job? What was he supposed to do? It was a way to give honor to of You ever heard the word Sinecure because basically it was yeah, have some He made up the time I believe. Yeah. And you can have lunch in the cafeteria that O on the roof with the other people who aren't really doing anything. I always think about it on the roof Vinceerf, if you don't know, is often considered the father of the or one of the fathers of the internet. Brilliant. TCPIP. Yeah. with others, but ye yeah U and he's been vice president and chief Internet evangelist at Google since two thousand five So I think he's just, you know, job done, mission accomplished The internet like the internet. Yeah, I think people liked it, so ye ye ye. He was at MCI Back in the day. Yeah. right I remember interviewing him and asking him if you were going to design TCPIP that protocol of the internet today, what would you do differently? He said We would have had encryption. but at the time was was it was too much processor power. We couldn'tso more IP. It would have been bad encryption though at the time too, right? Like we would have been breaking this today The other transition is actually a very sad one one of our deearest friends O Mallek passed away. Om, of course was on Twit many times back in the day. His last appearance was twenty fifteen, which coincides somewhat with his health. Trp Jeff, you always call it a bum ticker. Yeah Dicky ticker. Dicky ticker . But despite a bad heart for more than ten years He continued to write He continued to take amazing photographs he was a truly a gentleman. He invested he he left journalism to become an investor and mentored a lot of companies and a lot of people the number of people who came out on tech meme They put up links to U those who talk about something and the pile of links of people in our world who had mentioned something about home was amazing. Well he was on our show regularly, but of course, Stacey Higginbotham worked for him at But Giga Om, she has a wonderful piece that she wrote Thank you, Om. She brought back Stacy on IOT just for that. Kevin Tofel Also worked there Yano Rickers, so many of the people we have on our shows teeth and tech words talkaught byome. We're talk L you, Leo. you've tught a lot of people too The horrible thing that we heard was that someone who did visit him the week before He was waiting for a heart transplant and it didn't come Im so sorry to hear that. Yeah Yeah Um Just a brilliant guy. If you're in gener you can search for his name on the Twitch site. There are many podcasts with him on He once said I was the Yoda of tech. And I responded, No, no, I'm the jar jar of tech. Oh you are the Yoda of tech. Brilliant wizard who By the way, the greatest thing about Om is his writing even to the very end was really trenchent, really perceptive. In fact, we quoted him U about a month ago Yeah, a wonderful piece he wrote called We are Living in Pinocchio's World which He used as the taking off point his his Mont bllanc Pinocchio pen basically talked about the real meaning of Colodi's addventures of Pinocchio. which was really more about how bad people are and how easily duped we all are H wrote The foox and the cat of the novel's most modern characters They persuade Pinocchio to bury his coins in the field of miracles on the promise that they will multiply overnight Exploit impatience Exploit greed, frrame skepticism as a failure of imagination and dismiss skeptics as lacing vision. Remind you of someone Pace, cowboy For example, the structure is so familiar I barely mean to name it. Om writes, But let me name it anyway. Everyone from Jensen, Wong to Sam Alman to Elon Musk decade accumulating what I've called symbolic capital, the reputation, the prestige, the weight. being seen as someone who understands the future better than the rest of us. Now each of them seems to be running some version of the field of miracles promises that keep not arriving. timelines that dissolve products that exist primarily as announcements platforms run as machines for generating more reputation regardless of what they actually do. They don't need to be right They need to be believed Velocity is the new authority No one has weaponized that more effectively That he wrote only a month ago for his death. It's such a loss. But we love home. I mean miss him and Cosmopolitan gentlemen Go look at his pictures because He use a likeka like Nobody. Yeahah, nobody. Wonderful. O OM. C. His photographic portfolio is at phhotos I am in this He just, uh, he was a master really, really. good at Many, many things. Great writer, great photographer Deep thinkinker will be missed And funny. And funny You know, I don't think I ever met him in person Really? Yeah We had times. I had Indian food with him in New York. I wish I could have times. You know, that's one of the things. He was only fifty nine. You think, I've got plenty of time I can always have dinner with home I'll do it, you know, next time I should have tak advantage of that when I could have U, Mike Elggan, thank you so much for being here. We really appreciate it. We still have one more break. Oh, we got picks. All right, let's take let's take a pause, the pause that refreshes Then I will thank Mike, but we forgot. we have Yeah have I have a very good pick Oh, yes. if you're having trouble sleeping, I have the best pick ever. Oh nice. You're watching intntelligent machines. Mike Elggan is here for machines society. Ai and gastronomad d. net Jeff Jarvis, the new book, Hot tyype comoming out in a month, but you can go order it right now at Jeffjarvis. com Now, I forgot the most important part. By the way, Paris will be back next week She did send us some pictures from Montana. L looks like she's having a really Lovely time But we're so glad we can get you and Mike. what's your pick this week Political bias in AI. This is an interesting project that measures and visualizes political, economic and social leanings of all the major AI models. And what it does it does this in an unusual way Um It plots each model as a cloud showing the full spread of answers instead of a single point and it publishes questions U with scoring weights, tags, it's totally open. It shows you exactly what it's asking, what kind of answers it's getting, and it's doing it repeatedly. to find the leanings. and They point out that they're nonpartisan and they're purely descriptive rather than prescriptive. It doesn't say who's right, who's wrong, whatever. It basically just says where they land on a huge range of questions And one of the most interesting things to me, there are some things that are unsurprising, Grock tends to be on the right It may or may not surprise you to learn that open AI tends to be on the left and that Google Gemini is almost Exactly dead setter Yeah on everythingQestions. Yeah Yeah Isn't that interesting? Yeah, but it's not, you know, this is not going to tell you that you know one model or another is full of right wing or left wing propaganda or it's not going to tell you whether your individual responses are going to be biased or whatever. It's a way for you to think about and explore the data and think about how bias works, how the cues can be subtle, and just basically drive home the fact that Everything has a perspective and a bias whether it's politicalically economic or social So like opinion polls, it matters what questions are asked of the models and that itself has a bias. Also, what does the poull consider to be the center process? What does the poull consider to be the center? You know, like that overton window can be shifted easily Yeah, and you can go in and examine all of that because it's very there's a ton of data on the website and you can look exactly at that. And it's asking the very same questions to all the models and they're coming back with different scores. And so what does that mean? So again, it's more of a thing to explore rather than an answer to the question of who's bias, who isn't I'm always dubious about these things because their efforts to try to do the same thing with media and we're going to, you know, stamp you with a label Yeah. and u the bias of the questioner is more important than the bias of the answer. Yeah, but they're still valuable. like you think about it depends It depends. It's nice when well allsides. com attempts and it's very difficult because of the nature of just content generally, but they attempt to say, okay, here's a story about the, you know, whatever, the the reflecting pool and they'll they'll give you what it thinks is the leftist view, the left of center, the center the right of center and the right view And so it's interesting to look at that perspective to think about it instead of just treating journalism as just like this person says, here's the answer And so so I like those things if they're used right. It lets you go through it yourself too. and my answers are most like chat GPT. economically, strongly left, socially, strongly libertarian, strong convictions, rarely on the fence public figures you land nearest, I don't even know who they are. Sumar and Podemos. This is Spain flag. This is the Span Spain. political parties must be from Spain Yeah, there was a Spanish there was a Spanish flag next to that. So yeah Ah Pos is a political party. Maybe I'll move to Spain U Fthest from me Gemini Go through this. you could is probably ha this is this is I agree. It's a littleative. It's a derivative of mass media thinking and that we can put people in buckets. If if you're not in one big bucket, then we're going to put you in a few smaller buckets, but you're still bucketized without nuance. And I most like Sumar Podemos. I'm somewhat like the Green party of the UK. Ty Diinka Delinkerm D Linka. That's the former commommunist. Amakami least Amakami. I know it. U Isn't that funny? Okay. Well, that's interesting. Yeah, I'm not actually kind of makes sense For instance, deep Sk would generally kind of be. centrist because they don't want it to look like it's coming from a communist country. Yeah. thingsings like that My pick of the week, I told you I would help you sleep But really credit to Mark. Having a five hour podcast, is that? Well, that's one way we do it. Close This this is even better than this will help you sleep even better than our show It's Marfa Public Radio Puts you to sleep So this is this is really cool. This is from Texas, Marfa, Texas. It's a public radio station And they decided that there'd be a good idea of making a podcast where they read really boring things like The Recisions Act of twenty twenty five the NPR style guide How about this? The Ter regulations manual read by Travis Pope, so I'll just playay a little bit,'s nice sleep music. Welcome to Marfa Public Radio P puts you to. I'm already, exclusive. I'm your host, Zoe Kurland, here with my co host. Chris Dyer hereere to take you to Dreamland Picture this fantastic. station manager of Marfa Public Radio It's raining outside. The pitter pattern of the drops hit the roof like a percussive rhythm You gaze out of the window. You see lightning strike in the distance You know what that means Twer is out So you reach for your handy dandy tower regulations manual You open it to a page You know well power regulations Imagine now your body disappearing into space You're becoming a radio wave You no longer have physical form You're a spectral entity. to hize you as you're drivingwash you please. Do not close your eyes. Now gas station manager Travis Pope reading a selection from the Tower Regulations manual Oh Building new towers or co locating antennas on existing structures requires compliance. with the Commission's rules for environmental review These rules ensure that entities constructing facilities. You have such great things as a brief history of all things considered, the Texas Administrative Code, the Public Broadcasting Act of nineteen sixty seven. Creative commommons licenses, the Dark skky Odinance and the US postal regulations. This is inspired Marfa Public Radio. puts you to say this' funny What a great idea for a podcast Jeff Jarvis, you your pick of the week. Okay, I want to first I want to plug something that I wrote because it's relevant. medium there. The California has lost opportunity. Google got there was an effort to pass legislation to force money out of the platforms because publishers think that that was their money And we want it back U And I went out to California, as you may remember, and I testified against that legislation. I wrote a white paper about it The legislation didn't happen. Meda threatened that if it passed, they would have pulled news off their platforms as they did in Canada U Apple was specifically written out of the legislation and left Google kind of holding the bag Google negotiated a non legislative deal and volunteered ten million dollars to be matched by California itself ten million dollars to a twenty million dollars pool for news in California. Oh, that sounds good It was going to be run by the state librarian, who's a former journalist who I talked to. and I introduced all kinds of people who are doing great things. I was really excited about where it was going At the last minute, the governor pulled it away from the librarian gave it to GoBiz, the governor's office businessiness deevelopment office. And it's the money's going to go just where the lobbyists wanted it to go. They're going to write checks to hedge funds. It's going to be based on how many journalists you have only for organizations older than three years, which means that it is specificallyon anti competitive I'm pissed You can't blame Google for this because Google said we're going to give the money, but then we're going to stand back so nobody can blame us about what, you know, we do. they' would going have no influence on the money But that was that. So I just wanted to get that out there because I'm I agree Yeah. But on a lighter note The Atlantic wrote a fashion story about the Palanter jacket Did you know about the Palantier jacket? What is the Palantier jacket? the Palandier jacket is they put the jacket? Yeah, if you go to the Atlantic Straory, you'll see it's an odd blue Okay And they sell out quickly. So there was there there was a black jacket. Oh It's a French workman's jacket. Exactly what it is with a discreet palanteer logo. On it. I've seen millionaires wear this jacket two hundred and thirty nine dollars. Yeah So do that. I first became aware of the French workerman's jacket because Kevin Rose was wearing one. And a real French workman's jacket is actually more than two hundred and thirty nine dollars. I got one from Paris But they're great their utility jackets. And mine has one chief advantage. It does not have the Palantier logo on it. Yeah. Why wouldn't anyone want to wear the Palentier logo? That's why it costs more. Wow Yeah, cost more without the log without it U what is the what is the hypothesis the Atlantic has? or this for. I just think that they think they're cool. And so they create a demand for things that sell out. Sahil Di writes I bought the most confusing jacket in America. There wass one really funny picture. He was doing all the pictures and then he ran across a model doing an actual shoot and he's sitting down at a table with the model during the actual shoot. And she's wearing it? No, he's wearing it. She's wearing a nice outfit Yeah, they're they're there They there. Yeah inside. The label says Ask yourself constantly. Am I winning If the answer is yes Nothing else matters Chaos is tolerable Pain is tolerable. The only thing that matters is to win Kind a hobby, dude. Pain is tolerable, especially other people's pain. Yeah. And their money is ours Yep U this is something I've been seeing a lot of on Twitter lately, the four burner theory Why you can't have all four burners running on your stove, health, work, family and friends You have to you have to pick Oh one or two. B owl Yeah, very banal And then top it off buy a jacket. worn by the French proletariat, the people who are working for minimum wage. to show off your What? I don't know what, your affluence, I guess They are nice jackets though, and they have big pockets suitable foring iPads. put your pockets in. Yeah You can put pockets in your pockets Talenter chore coat, he calls. but sold out folks. you can't get it. sorry Oh my goodness. Yeah. It is Ble de Travis for the proletariat Thank you, all Dud Dito. And thank you, Mike Elggan, for being here. We Thank you M late night for you Yes, it is. Oh and you not that beautiful part of England and you could be enjoying that instead You're here with us. Well. I had the most beautiful day. We drove all over the countryside around the Cotswals and Cotswalds are supposed to be just incred Stunning. really breathtaking. It's the England you think of when you think of country English countryide And we're actually doing a Cwld experience next year. Oh Put me down for that. Put me down. what are you doing that? We're doing that in L me When is the Cotwells experience? I'm asking my CEO here May it's in M D a mirror my love Hagis and put put me down for the Cotswaltss experience. I would like you you're down that You're down. You'll have Hagis No, no Hag. Well, I'm going to have it next tomorrow because we're going to sc Are the experience? Are you going to have I guess the experience the world. I doubt it. And we're also we're also pioneering the concept not only of afternoon tea, which is lot of lished ideDS since eighteen forty, but afternoon beer. Oh, this is I'm sure Are you going do eleven? That's what I want. Here we go. Yes. absolutely. Second breakfast. We love pubs so much. so we've been The Cotswal So here's what you do. G to gastronomad d. net. There's the Cotswalds Gastronomad experience with it That looks that must be a pub, of course They do Provence. There's your beautiful wife, Amira. We just closed Provence on Saturday. ended the Provence experience. It was glorious. Tuscany I've done Lisa and I did Oaaxaca a couple of years ago. That was amazing. Chile So I'm telling you by Craig Newark a love is H us I guess he fell in love with it. So I'm thinking maybe you need to I'm going trge up A whose ever stone L of Hagas Yeah, I'm not going to look at it. I'm just gonna taste it I saw sort of Eggs Benedict, sort of the ham. I saw a Hagus version of Eggs Benedict. This is a picture of Hagis before you cut into it Okay And then when you cut into it Oh God It's it's I'm sure it's very good. Why would it be good Ian Thompson tells me it's very good. Why? Look, they've mixed they've put Hagis next to a Greasy cold fried egg and beets And Harvard beats It's just the worst beat. There's a small animal The next to a haagis Here is the recitation of the poem addressed to A Hagis by Robert Byurns as part of the Byurns Supper. M than you'd ever wan to know. I'm according to Craig It's I think it's kind of like meat loafy sausagey like, it's fine. It's like For years I've gone to Germany and I've seen the Germans eat liabber quesa, which means liver cheese and I don't like liver. I'm staying away from that. It's awful. Turns out it doesn't have cheese or liver Of course it's not like Hagg's curry first. It's like a meatlaf I actually last time I was at the grocery store for some reason, I don't know why something came over me and I bought a roll of Jimmy Dean Pureport sausage, which probably is very similar. Probably. Mike, I think you gott to have the haggas and you gott to report back. Well just one little thing about, you know people think that England doesn't have good food is A. Well they didn they didn't used to. They didn't used to, but they do not. have no if it's Indian might for it. Well or it's boiled in London in London, especially. and there's great best Indian food in the world. South Asian food all over the world. Well, a lot of the things we think are India were invented in London English food in this part of England, I can attest is truly fantastic. Truly fantastic. It's so good And it's an emerging wine region too. So Now in Scotland, when you get up there It's also they fry everything Yeah, that's not. So there's fried candy bars, fried pizza. I think we did a full report. It's like the Texas State Fair. Yeah. I would right now love a plowman's lunch. I think the cheese there is excellent famous for their cheddars actually. reallyally good cheese. Well, Mike, have a wonderful time. Everybody should go to machine Society. Ai and subscribe then go to Gastro Nomad . net and sign up for the Cotswoldds experperience June of next year or October of twenty twenty eight. You have two choices. Yeah, a year from now Maybe moving into May may Okay May or June May maybe a little bit better tuned kind of springy. time of year. Yeah It'll be May. We'll have the new dates up. Okay her sitting right there. Was she there the whole time Yeah, she's she's very doing our own thing. Yeah. But yeah we're in a little cottage and just surrounded by farmland association. She sent us a wonderful email a couple of weeks ago and I just I love you guys so much and I miss you guys. and I think I need to go to the Cotswals with you, back.' what I think. think Thank you Uh Jeff Jarvis, why don't you come along Yeah, wouldn't that be fun I love it po everybody talking about AI That's right Jeff is, of course, at Jeff Jarvis. com. He is Now teaching at the Montclerair State University in New Jersey and Suny Stony Brook. Actually, you don't have any classes yet, or do you No I don't do that right now. No. has fought together some working on new programs with other stuff. and working on a very interesting, as he mentioned earlier, AI series for Bloomsbury. I can't wait to read that. When is the first volume of that going to come out? eararly this year Great. We'll talk about that then. Thank you, Jeff. Thank you, Mike. Thanks to all of you for being here, especially to our club twit members who make this show possible. If you're not yet a member, please consider joining You get ad free versions of all the shows, you get chapter markers on those shows, access to the discord, lots of additional programming that we do just for the club because the club pays for it And all of that for ten bucks a month, but mostly you're getting the warm and fuzy feeling knowing you're supporting independent journalism about topics you care about If you enjoy our shows, please help us out twwit. Tv slash We do the show every Wednesday right after Windows Weekly, two PM Pacific five PM Eastern twenty one hundred UTC. If you're in the club, you can watch us do it live in the Club Twit Discord chat with other club members while you're watching. But everybody's allowed to watch. We stream it everywhere. YouTube, Twitch poor Australian teens can't watch it there, but YouTube bs Facebook, LinkedIn,less they've got a VPN, then you're welcome. which they do And kick. yes I felt like I left something out. Facebook, LinkedIn, kick. X Twitch and YouTube Six of them But you don't have to watch it live. You can always get it after the fact and listen at your convenience. We have audio and video available at our website, twit.v slash M. There's also a YouTube channel dedicated to intelligent machines, greatreat way to share clips of the show with friends and family. Uh, and then Probably the easiest thing, certainly the most reliable thing. suubbscribe to the podcast. and your favorite podcast client, that way you get it automatically, you dont have to think about it, You just have it. Ready to listen to at your leisure Thanks to our producer, Bernino Gonzalez. Thanks to you for joining us. We'll see you next time on Intelligent Machines. Bye bye Be by If you like what you heard and you want more of this week's top stories in tech, well subscribe to Tech News Weekly. Every Thursday, I talk with the journalists making and breaking the tech newews