Understanding the Token Maxing Phenomenon

No one goes to Hank's for his spreadsheets. They go for a darn good pizza. Lately though, the shop's been quiet. so Hank decides to bring back the one dollar slice. He asks Copilot in Microsoft Excel to look at his sales and costs and help him see if he can afford it. Copopilot shows Hank where the money's going and which little extras make the dollar slice work Now Hanks has a line out the door. Hank makes the pizza Copilot handles the spreadsheets. Learn more at M three hundred sixty five copilot dot com slash work. Aaron Levy is the CEO of Box. He's one of the most insightful and fun voices on the future of this technology. He actually was the fourth guest ever on big technology podcast and the guest the only other time we did a live podcast together So with that, I am thrilled to welcome Aron Ley. Please join me inlc Great to see you. Good to see you. Thank you. Let's dive right into it. I'm going start you know, again, since you're not at one of the labs, let's start to talk about the biggest most controversial moment now which is the anthropic fable situation Let me put to you what I call the Jassie mystery. So what we know about I'm sure he liked that Dame He didn't show up so we can.. o What we know about the fable ban or the export controls on anthropic Yeah is that Amazon found a vulnerability in the software And Andy Jassie maybe made a call to Dario, definitely made a call to the White House And then very soon afterwards, there were export controls that were put on Anthropics frontier model. Those two fact patterns are probably not ideal So The mystery is why did he do that? Yeah. And here's one hypothesis. This is from Chimoff. He said Google, Amazon, Microsoft Meta now have a serious nonzero opportunity to tank the frontier labs. G to the government, kncap the lab's motion of putting the latest models out into the wild, become the trusted gatekeeper between labs and the public by having the labs go through their clouds. Pausible. Um, I would I would say u I mean, anything's plausible. I prefer Occam's Razor on this one, which is, you know, ever since Mythos you know Mphis very clearly was this event that basically said You know, AI is obviously getting super powerful. It has all these risks associated with it. We are going to give it to some, you know, small trusted partner network. They're going to go evaluate their own tools. They're going to evaluate these capabilities. There's been a lot of sort of It's a very kind of dramatic, you know kind of rollout of a technology. And I think what that has done is it's created this flywheel where it almost incentivizes even more drama and more research and more depth in security being the primary space. In a way that we could have already been doing since GPT four if we wanted. L you can go and deploy these things to go find lots of vulnerabilities. you can use them offensively or defensively. But Mythos kind of created that extra air of seriousness and uncertainty around it for good reason because it's an incredibly powerful model. I go with Acam's Razor, which is Amazon obviously has security research teams. They're any company at that scale, you know, we try and you know test and push the limits of models in our particular domain of use cases Clearly at Amazon scale you have a very large security team. They're try trying to jailb break models all the time And so almost by definition, there's already a public private partnership on all forms of jailbreaking models, trying to push them to the limits. As a part of that and especially with the surrounding atmosphere of Mythos I think it would be very natural for Andy to either share that research or his team to share that research and that escalates and then that sort of creates its own flywheel. But the idea that there's some kind of board boardroom levels, you know, sort of strategy meeting that says, we now need to kind of like cooo op the technology, become the only interface of the government. This kind of puts us in the pole position I think it's less likely that and more likely this is a this is a situation where the mythos Mentum continued Fable obviously, you know, had ways of getting back to kind of the Mythos level capability. and researchers, you know, sort of shaare that information. and I don't I think there's like a basically, you know, very limited small percentage chance that then Andy and team knew that like the very next event would be They'd stop the model. and that's not even good for Amazon, like strategically. like Amazon makes plenty of money the more that Fable gets used in the world. So I don't think you would I don't think you would do some kind of like, you know kind of maneuvering to create this. So I kind of just go with this is It's a very chaotic kind of environment right now. The government has you know, only a few tools at their disposal at any given time to deploy against these things. Those are going to be kind of blunt instruments U And and this stuff is coming together very quickly because of you know, in some cases, the lack of technical capability of the government compared to how powerful these models are. It's like you don't, you know, when you see something that seems very scary, like, oh my gosh, the thing could be We can jailbreak the model and get back to Mythos level capability and Mythos was the thing we're supposed to be scared you know, just like stop it. L I think that's a very natural reaction based on the atmosphere that we've created in AI recently. So I just go with that as the answer I mean, I like that you say the atmosphere that we've created in AI lately. Everybody but me. Yeah. Well, I I didngr it. The company on the receiving end of this though, is anthropic. And you know you talked about these mythical capabilities. they called the model Mythos. They put in the documentation that it broke out of its containment and wrote the engineer while he was having a sandwich in the park. Y that's surprising that this is one of the downstream impacts? Yeah, I mean if you put that in your announcement blog post, you know people might be able to kind of extrapolate and get pretty pretty scared of things. I think it's interesting. So U you know, on the anthropic front, first of all, I have I have a huge amount of respect for the entire kind of stack of researchers and policy folks across AI I happen to have disagreements with some of the categories, but But I think there's a deep, let's say, if you were, if you imagined a continuum of the most like you know, if you u U if you kind of had like like the most And I mean it's only in like a polite way. It will sound impolite, but like I mean, like if you're the most doomer on one end of the spectrum and the most like like accelerationist on the other end of the spectrum. H hereere's kind of the views. The most doomer possible is was afraid of like GPD three And GBD three was going to like, you know sort of accelerate and you, kind of achieve some kind of unstoppable continual improvement And you know, the accelerationist says like we need like fable twenty as soon as possible. right? So that's that's sort of the continuum. I'm probably like you know, maybe two thirds up to the accelerationist kind of side of things. But if you were the on the Doomer and I'm trying to say the polite version of Doomer. like you're deep in AI safety, you're very scared of the technology, you think There's as much likelihood of bad things happening as good things happening. We have to win the race and control and kind of stamp down on the technology. We don't want this to be the sort of thing that runs in the wild. If you're in that end of the continuum The thing that happened this weekend is actually the best case scenario for you So, so like you actually want there to be these sort of like valves and like buttons in the government that just is like, we're just going to stop it. I mean like Dario's position, do you think he's happy with what's going on? You know, I'm not I won't try and guess any of that. But I will just say if you had to establish a if you had to establish a regulatory regime That said, we are going to review models We're going to push the limits of models and we're going to have the ability to either roll back access to models or prevent their release in the first place. We want that to be a regulatory approach You would need an event like Fable to effectively create the precedent for that environment. you're not going to wait for Congress to vote on this being the new kind of process. You would kind of need something that sort of shocks the system into that kind of regulatory framework. So all I'm saying is that If you're if you were on this end of the continuum This is actually an outcome that is sort of almost desirable. Now maybe you would wish that there wouldd be more technical evaluation, more back and forth, Maybe you wish the policy people were different on the other end, who knows But the idea that we now have established that the government can press a button and prevent the rollout of AI U is actually like a probably a positive update for an entire cohort of people. Now, unfortunately, I don't know that any of your guests represent that cohort. But I think you could easily get some people that would be like, this is the greatest thing that's ever happened in AI safety because now we actually have we've created the case law essentially for this. now we know we know the tool exists And then the next messy process is when should we use the tool again? What should the real, you know kind of ongoing process look like But I think that you know, probably I wish this wasn't the case, but I think practically In the next three to five years, we probably have to end up in an environment where models do get evaluated by the government. There is a sort of collaborative approach between the government and research and the labs know, the government has to kind of green lightight the release of the model. I think it's probably become either too scary of a technology or too economically powerful of a technology. for governments to not want to be in that position. I think that that has massive implications when you kind of unpack it, like just totally massive implications. O being Other countries now have far more incentive to stand up their own sovereign AI initiatives. So it's actually like maybe net negative for the US economic position in AI. this is the outcome I think somebody could take the other side and say, no, we'll always have the most powerful models. And so this puts us in the best position because now we can do like horse trading with other countries of do you want access to our stuff So I think I like the fact that this is a super interesting debate. and I have like a huge appreciation for every part of the continuum because I think it's like so intellectually interesting. I still land on the, hey, we probably want to treat this technology more as a substrate technology and then regulate the applied use cases. So we should regulate if you use AI to break into something. We should regulate if you use AI to do bio, you know kind of research that leads to dangerous things, we shouldn't regulate the model itself. But I totally understand the other views that are on the other end of this and I think it's kind of very natural with this import of a technology that has to be somewhat of a deemocrat process of how we decide to regul it. Yeah You you remember there were all those petitions? Y month pause and everyone kind of laughed at them? Yes. This is effectively the best way to do that type of pause. Yeah, I mean, this is if you were in the pause AI movement, this is, again, like this is a great outcome. We now have proven how we can pause AI. Now it's an interesting kind of like mechanic that they chose. It's sort of this export control thing. But effectively, if you have an export control where non U S nationals can't use the technology Eective, that's pause because your end API users of these models almost have no way to fully ensure at all times that their end users don't sort of fall into some kind of criteria that's off limit. So And there's already companies that are pulling back. JP Morgan, for instance, has told its Hong Kong users no more claw. So so okay. so now if you really war game this out meant like two to three, four more years out, this is this is kind of interesting. So We have this like sovereign cloud kind of comparison The cloud you know For better or worse basically became a commodity. Like whether you're running in a cloud in, you know, there's like lots of, you know, performance implications, like some are faster, some are cheaper, but like largely like, You can get a web server built out wherever you are in the world, You can get storage built out wherever you are in the world. We can build sovereign clouds. Sovereign AI is a different kind of know has some intricacies that are different, right? L intelligence just is not commoditized yet. We don't have everything having the same model capabilities. So there's lots of really interesting implications, which is, well, what if like one country has access to frontier intelligence before the other country? You know, what does that mean geopolitically? What does that mean economically? Obviously now if you're another country, you have so much commercial incentive to make sure that you can build out labs and have access to frontier intelligence as a kind of hedge against the U.S So Wh's a net winner in that? probably China. And so what's interesting is like you end up I don't know if, you know, probably most people saw the Dor Kest Jensen interview. And can you can actually it's a Rorschach test. You can watch that through two totally different lenses. You can have one lens which is which is like Dorash is totally right. We have this huge lead, like this stuff is so dangerous, but if we control it, then like we're going control everything The other lens, which is probably more of the Jensen angle is like Actually, you, these other countries have a lot of incentive to also get this right. And so even if it's like a five hundred billion dollars problem for them, they just might deploy that much capital on this problem and they will eventually get it right. And so at the outcome, actually, we haven't gotten any gains as better intelligence from the rest of the world, But what we have lost is our economic superiority and this technology category because what we've caused is a catalyst for all the other countries to have to build out their own stacks And if they build their own stack, it's probably going to be, you know, chips from China, models from China, etcetera, which I don't have any like, you know, reason to be be against other than just like I want America to win the economic know angles on this. And so so this is sort of this debate that happens on like where should you apply export controls? And what are the implications of that downstream And even this week, you know post fable, we see that, you know, you have models that are Certainly not fable performance level, but Opus four point seven, four point eight level, which is a big update for a lot of people on what is now possible with open weight models that we just didn't have kind of visibility into before. Yeah, I think you shared recently that the open weight model, open source models, the capabilities are not that far away from the frontier. And in fact, like as these models get smarter U they're only they're almost going to saturate with intelligence where there's not going to be such a big difference between Let's say the smartest open source model and the frontier, donon't you think? And then so won't this push people to open source? So the big the big ongoing conversation and I think you have some guests that can really represent, you know, what they're seeing on the front lines is is do you have a sort of fast takeoff scenario of model capability in progress U and with some kind of continual learning kind of, you know, self improvement dynamic. and then and then it stands a reason that like The company with the most compute or the country with the most compute And you get the fast takeoff, you get sort of a virtuous flywheel That maybe is sort of has some compounding you know benefits to it that are just you know, unreachable by anybody else. That's a scenario. Another scenario is, is that that's another just incremental capability. Everybody kind of catches up to it and you always have this sort of, you know, kind of two loops going at all times and the with the clos the closed providers and the open providers they and they're kind of always within three to six months of each other The world is so different from a market structure standpoint, whether we end up in an outcome where we have sort of an exponential progress in the in the models that kind of continually learn vers versus the clos source models. And it's like a five year gap in progress and that just goes again kind of exponential. Totally different market structures The one that where we have this exponential progress is again, it's probably actually net positive for America in that case. In which case, the export controls probably worked. It means our kind of top three, four labs have this incredible superiority. We control access to this technology. That's actually a good scenario. like economically speaking, it might not be like a total net good scenario for society, but it's good for the US Let's just say that's one scenario. A lot of people are betting that that is that's where we're at with research The other scenario, and like China sits around, and they probably bet on this scenario is no, we're going to be able to keep up. We're going to throw more computeer at the problem. We're going to get more data We're going to build our own, you know, flywheels and it's always three months out and you know, kind of behind. And if it's three months behind and it's an open wight provider that has more of a commoditization kind of business model approach because they just want to, you know, sell more infrastructure or chips or they just want to reduce our you know, superiority in the space, which is actually like strategic for China to do. Like there everybody wonders like, why are they doing this open weight stuff? It actually makes totalense You're just reducing US's dominance in a field, and it might be worth a couple hundred billion dollars to do that for something that might be worth,, ten trillion dollars So so if that keeps up because there is real economic advantage to doing so Then You have this new kind of know sort of dynamic that plays out, which is maybe the layer of incremental value shift is effectively the applied layer of AI. So if you think about it, there's the lab layer, and then there's the applied lay cursors. The cursors, the Harveys, the Sierraas, the Deagons, the boxes, which is amazing because everyone said there's just a thin wrapper on top of large language models, but now maybe that's where the value comes Yeah, so so and you know, it's one of these things which is like we just have to not be binary about it. Like everything I'm saying, I think the frontier models still make way more money in the future than they do today becausecause what happens at the routing layer is you still sort of say, hey, I want fable. U orr GPT five, you know, five or whatever the next model will be. I want that to be the orchestrator. I want I need like the super intelligence of the orchestration layer And I need super intelligence at the review and sort of like you fix and check the work of the other agent. And so so you have like a Barbell, you know, maybe U shaped model where you use frontier intelligence. But then everything in the middle, you can just say, no, I'm going to take that to Nemotron or Kimy two six or GLM five five two or whatever. And then all of a sudden it's like suuper high cost you know, inference in one part of the workload. super low cost, still pretty good inference in another part of the workload. But who has the incentive to do that? It's the applied layer of AI. because like the business model of the applied layer is obviously like our job is to give you the best model for the job, not just the model from just our lab So it's cool because we actually now have a good kind of push pull between frontier labs and the applied layer where you probably wouldn't want it to be that we're all sort of only in the orbit of one or two companies, you know, commercially and economically, you'd want to make sure that there's some good tension there. And so I think that's kind of the direction things are things are headed between kind of the token costs the open source models becoming so good and then maybe even some of this regulatory dynamic I think the applied layer sort of incrementally get gets more more of that opportunity, which is right, which is obviously great. So you've talked about open source and you've just mentioned China. But what what can you tell us about La Chanton Fat? So it's great memes. So folks, La Chanton Fat is a rumored Open source model from Mdrawl and has been the subject of great fascination from the internet, wouldn't you say? There's great comedy. John D Can we show people what we're talking about? Let's roll image A. All. This is L Chanton fat The number one model from Europe. Y My the world? My French rudimentary French, It translates to the very fat kitten. Can we roll B This is a standard day in Paris now Yeah. But it does show something that there's so much eagerness for AI that there's now fan art for this potential model from M Draw. We've reached a really important sort of phase in the cycle. I do think that it is kind of cool because what Some of the things that you maybe discounted the importance of all of a sudden just like have so much more importance. L I'm watching the I don't know if folks are watching like the fireworks Bace tenent space as an example Like it's pretty cool that we now have these open weights models that you can effectively post train on your particular domain of task And you can go and you know, ek out another five or ten points of of performance on these types of models. U And again, that's only possible because of the mistrawals. because of the Chinese, you know, kind of open weight models. And the cost curve has gone down so much that there are actually some situations which is oh, actually maybe I should train a model just for my use case becausecause it's literally like economically now, it's not even like I want control. It's actually economically advantageous for you to do so So is this the answer to like the big token maxing hype where like everyone's spending all this money on tokens and not really Understanding where they're going or whether there's an ROI that Yeah. I mean, I think that in practice, that phase probably lasted two and a half weeks. like from the moment that Meta token mag the media over hype token max. No, I would never claim that. We should go through your various podcast headlines, but we're not going to do that. We got the cat pictures, and that's it ye No, but like, I mean, if I had to like capture the cycle of like the first token maxing, you know, meta has a leaderboard, uses the most tokens possible to now, you know, the last weeks of rumors of like we're shutting down everything. no one can use AI. you know, it's about a two month period. So it's, you know, people need to like probably you know, always kind of step back and just be like, okay, is what we're doing like a pragmatic, you know thing for for a work or we just sort of like like getting kind of hyped up too crazily on something? What's interesting is this phase was so short that I don't ever think it reached outside of the tech industry. was we kind of host these CIO dinners in every city that we go to. and I we had a dinner like within three days of like the token maxing like initial spike on like Google trends, like the word finally emerged and Three people had heard about it. and so I feel confident that diedight They don't heard it because their employees are out spending the token. Yeah, fair point, fair point. hopefully it will have completely died by the time it reaches reaches the rest of the world and then we can just move to more normal environments. And but the thing that is true of the phenomenon is that these agents are just using hundreds of times more tokens than they were before. And so You know, when we launched our first kind of AI use case within Box our product The average number of tokens that was being used on a task was like five thousand, ten thousand, twenty thousand tokens Now our latest agents might use A million tokens or five million tokens on executing a task. And so that's, you know, in some cases, that's a hundred X increase in number of tokens. And the reason for that is obviously like what's happening is right as we solve one use case When you would think that we can drive down the cost curve of that one use case, all of a sudden, a model capability allows us to now add another use case that's much harder. And then our appetite just grows to solve harder and harder and harder problems. And so it's this funny thing because people get confused. They say, I thought AI was supposed to was supposed to be getting cheaper. And it's like, yes, you can actually think about it as cheaper if you looked at like the unit of intelligence. The reason it's more expensive is because we're now taking on bigger tasks. And so we're getting confused because we're like Why is this the one, you know tech trend that doesn't have sort of the Moore' Law phenomenon? It's because actually, no, we're outrunning imp the efficiency improvements in our appetite for what these models can go and do And so it's actually what you need to do is have like a way to normalize the cost of the tokens to the tasks that you can now deploy. And then if you look at that, then that starts to look cheaper on a per task basis. It's just again, our tasks are getting bigger or more accurate or more effective. and that's going to happen for quite some time. The reason why token maxing took off as a concept is because people saw the exponential revenue, right? The fact that anthropic and open AI were at zero twenty twenty now they're like going to do fifty billion dollars this year at the very least. And so people are looking for an explanation. And either the question either the answer is this is real or it's somehow inflated. And that's why people go to token maxing. So if I'm hearing you right, what you're saying is all this spend is much more legit than some of the online discussion makes it out to be Well, I think if I had to like officially provide my own takeaway for my own point, it would be it would it would sort of be there's There's sort of always this experimentation phase of a new technology. and this happens to be a relatively expensive technology. So thus the experimentation phase is expensive And then what will happen is enterprises will deploy AI and then they'll sort of peel they'll start to see like where are the real use cases, where are the ones that aren't as real? They'll wind down the ones that aren't as real. The ones that are real, they'll then look at it and they'll say, is there a way to do it at a lower cost once we understand it enough? Or do we still need the frontier intelligence for everything we're doing And that's actually just like a pretty normal I think process that everybody's going through right now. But you know I think about it like Our engineering team We are not token maxers in the sense of like there's no leaderboard. We're not incentivizing overuse of tokens. We're just saying, use it as effectively as possible to get your work done faster. Our growth rate of spend is exponential, and we're like totally happy about it noobody internally is other than like, ah we got to like shift some things around and make sure we plan for this even more next year. that's obviously a stressful conversation, but we're not stressed about the idea that we're spending on AI. Like we're quite excited about the productivity gains that we get. And so I think what's happening is every enterprise is having to kind of go through their own journey on that. They're deploying in some teams and some teams are saying, Ohh my gosh, this is this is like the greatest thing of all time. And then other teams, you kind of look at what they're doing. You can't see any kind of measurable improvement in the output of that organization. And so then you're like, okay, you know, like maybe maybe it's not as effective there. But I am I would say like I think it's very easy to kind of capture one or two anecdotes and then kind of over extxtrapolate on the overall themes, I would say the vast majority of the current agentic spend that's happening is sustainable because because partly because it's actually coming mostly from engineering and engineering related tasks. and this is an audience that is kind of technically capable of you know, determining whether they like the work, you know, product that's coming out of the AI Maybe as it gets to other parts of knowledge work, you know those people will not be as sort of familiar with how to do the RI measurement and then it'll get even messier. But like so far, I think it's actually been largely totally reasonable. We' have a couple minutes left. Let's do a small lightning round. Oh So my first take here is that series is really good. It's going to be really good now. Yeah What do you think I agree Fire Collaborate Is it lightning round or is it you want to hear a five minute answer round have like a sixty second. mean. What could be easier than pressing a button on your phone and talking to it? And if you know at least based on the announcement, they've taken Gemini, which is a very good model and been able to, I don't know if it's fork or distill or something within there is sort of Gemini grade intelligence So if you get Gemini gr greatade intelligence and voice on your phone, press a button, I think you're just going use that for a lot of things. and then you know, I think the exciting thing is like imagine that hooked up to various apps on your phone and you're like, hey, order this thing for me or, you know, you know, you know go and add this calendar entry. Like I think those are very plausible daily use cases that we will have And it's exactly the sweet spot for Apple to kind of own that space. Yeah. No, I think Apple did it finally. That good. That was good time. Are you going to tell me if my answer iss right at the end of each one Okay, That's yeah. good. Okay. okay. so we agree. One for one. How about this one? perermanent underclass U I don't like this one. This one, I don't like at all. U notot only do I disagree with it, but I think it's just like a bad meme to have in the atmosphere. I think it's like not good for college students coming into the workforce of having so much stress about, you know, what company to join and and what's going what's going to, you know, kind of play out. I do think companies actually do The job market a disservice though by not being as clear on their own philosophies on this, which some of it is reasonable because it's like, man, we're just like We're getting thrown through a loop, There's so much innovation. But I do think that companies need to be somewhat clear on, hey, here's how we want to use AI. We want to use AI to accelerate our work or accelerate our technical innovation or accelerate our ability to hit customers versus, you know, no, we're actually like our metric is as few employees as possible, you know, with AI. Like you kind of do want to, you know be able to have some stance. And I think companies have been very confused and that lets this meme somewhat persist Uh, uh, you know, for, u for for for, you know, The interternet Okay, I won't write that one. Thank you. All right, last one is the SpaceX performance good or bad news for open eye and anthropics Well, it's obviously good news. You don't think Elon took some of their money because he pitched the market on an AI company That's where the money got funneled into. I'm not sure I've seen like a limit of appetite for I mean, there's there's a literal limit of money in the world, but I don't know that I don't know that that is zero su at this stage. So think I think people are pretty clear that know, if the revenue of this entire category of the frontier models and the infrastructure stack is measured in the trillions, then you can have twenty companies that all take a piece of that at different layers of the stack. So so I'm not sure I would be convinced that that would be zero sum. Did you buy SpaceX? Ictually did U you know, I'm u I don't know if I'mbarrassed, but I'm not gonna say the amount of shares, but I wanted to like be a part of the movement. So I'm on Robinhood find my retail shares of SpaceX. I'm up like fifteen bucks now, per share, per share. But so I'm happy. Yeah. Amazing. Well. Earon, you know, you answered my email when we were just at the very start of this podcast four episodes in ame on the show. I feel like every single time we talk, something crazy is happening. That's a guarantee at this point, so arere we in the thick of it? Yeah. Aesome. Good to see. Thank you so much, Aaron Thank you everybody