Imagining a Future Siri Agent

Hello, and welcome back to another episode of App Stor ies. This episode is brought to you by none other than Vitally. I am John Vorhes, and with me is Federico Vittici. Hey Federico. Hello, John. Hello. How are you? I'm doing great. How about yourself? I am feeling relaxed after the weekend. We're recording this on Monday. And uh yeah, it's uh I think it's gonna be a busy week. Uh I feel like uh you know by the time this episode comes out for members, maybe we will have a WWDC announcement or maybe not. But when we're recording this, there's no way to know. So there is no way to know. Yeah. Yeah. And and you know people, by the time this comes out, people will know that we have updated our setups too on the website. So go check out Macstories.net because slash setups because Federico and I try to update our setup so like four times a year roughly, you know, once a quarter or so. Yeah, once a quarter more or less more or less more or less and it's just an if you're interested at all in the kind of gear that we use for the various things that we do it's a good way to do that. So uh check that out on the site. But today's topic is not gear. Today's topic is open claw, but by Apple. Alright. Good luck . This is an a this is imagining a world where Apple did something similar to open claw. And I think before we even get into the depths of this, I think we really kind of need to set out what the big deal is about some of this a agen cy stuff because I feel like you know there are a lot of people who have not uh tried open claw. I've tried it and abandoned it and have moved on to something different, similar to what you're doing, which we'll get to in a bit. But I feel like a lot of people saw the hype around it, but also saw a lot of the security warnings and rightfully said, you know, I don't want to mess with that. I could probably figure it out for myself, but I don't have the time or the energy to do that. And so I think that there's still a lot of people probably who probably are using uh you know chat GPT or Claude and what they're doing is they're mostly using the chatbots still. And so I think it it kind of helps to set the stage to talk a little bit about what our experiences have been with using agents because it really is different than where where you where you would be if you were still just using a chatbot. I still use a chatbot. It's just that this agentics stuff is kind of next level in terms of the kind the things you can accomplish. Yeah, I've been thinking about these a lot, especially especially over over the past week with some of the changes that Anthropic has announced in Claude . And um and I think I've blended on this answer, like what's the difference between a chatbot and an agent? And so I think the chatbot is I mean it's right there. It's it's the place for chats, for like an uh answers and questions and the back and forth. But chatbots they they have a different hardness. So so when you hear the word hardness, it means like the way that the model underneath there's always a model, there's always a large language model, right? The hardness is what these companies like OpenAI and Anthropic refer to as the very basic instructions at the core of the product and the like a like the template that that the model is supposed to follow when it's in the context of a chatbot or whether it's in the context of an agent like cloud code, for example, or co-work. And so the chatbot has uh these instructions that by default, no matter how hard you try, from my experience, chatbots never work as long as agents do. Like even when you set them to work on a long task, like hey, I want you to analyze, I don't know, my three months worth of email. The way that the chatbot experience is designed it's not gonna work for like thirty or forty minutes unless you kick off like a deep research or unless you use open AI, chat GPT Pro. Those are separate experiences. They are. And a lot of times those longer sessions that where you do that kind of thing involve using MCP where you're involving other other tools in the process. I I personally think of chat the chatbot experience is a lot closer to doing a Google search. I think it was designed in that way as almost like a replacement for that. It's evolved from then from there. It it's no longer just answers to questions. As you said, it can do things, but it can do things and it can work sort of independently for a while. But even if you kick off the same task, and you can run this experiment if you don't believe me, uh put together a question that involves you know multi-step research or taking ac tions in you know I don't know in your Gmail in your Google Calendar and then cross-referencing maybe their information with the web or with stuff that you keep in notion and try the same question, try the same request, both in a claw the regular chat versus cloud code or cloud co-work. You will see two very different experiences. Uh I am willing to bet that the cloud chat, the chat bot will finish earlier and maybe it will even run out of context for the conversation whereas the agent in co-work or in co or in cloud code will keep working on it. And it'll work in isolation independently. Maybe it'll use sub agents. And it'll work for twenty, thirty, forty minutes, it'll work for an hour, and you will see, you know, this parallel tool call ing. It'll kick off the Gmail integration on one side and the notion integration on the other, some web searches on the other side. Like it'll parallelize operations a lot more. It'll use tools a lot more . It'll use sub-agents so that it will not pollute the context of the main session. It's a lot more optimized for working on its own for a while, using multiple tools , often in parallel, whereas the chatbot experience, as much as it's been improved over the past year with you know external tools and connectors and all those things, it's still primarily designed to have a shorter context wind a smaller context window right and for back and forth for chats not for you to ask a question and then leave for 45 minutes and then come back and find an answer. Like it's a very different type of experience. Right, right. No, yeah. And that's kind of how I use it. I mean I use it for it's that conversational stuff where you might have follow-up questions and often it's very much a one-off question. It's just like uh uh a starting place. Although I will say that it does work well w alongside things like Claude Code and Cowork in that you can kind of explore do the some initial exploration of an issue. F kind of get to a point where you're like, okay, well, I really want to do something with this information that I've gathered. Have Claude the chatbot create a report from that, you know, just like a markdown file or something. And then that bring that over to something like cowork or clog code or whatever it is and use that as your jumping off point because the chatbot is also you know where you can still do things like projects where you can uh you know do things like have it help you create prompts because you can I mean one of the one of the projects that I know you and I both have used and I don't use it for every project but if I have a really complicated for how to structure a prompt with XML to make it easier for the LLM to understand and follow the instructions. So, you know, it even though it is shorter, a shorter context window and more of a conversational short -term thing, it is a good jumping off point a lot of the times, I think, for the ultimate tool you might use after it. Yeah. This episode of App Stories is brought to you by our friends at Vitally , the AI-powered workspace for customer success managers. Vitally's purpose-built AI is for scale customer success. It's there to help you better understand customers to reduce churn, unlock growth, and always stay a step ahead. Basically, all the things that make you great at your job vitally delivers the clarity, automation, and AI-ridven insights your team needs so you can move faster, stay aligned, and drive customer outcomes at scale. Vitally AI services insights from every customer interaction to unlock another level of CS productivity, visibility, and collaboration. It's why Vitally is trusted by more than 600 leading B2B SaaS customer success teams. And how is this for a great offer? Vitally is offering a free pair of AirPods Pro for every App Stories listener who takes a qualified demo call. So if you're a customer success decision maker, schedule your call by visiting vitally.io slash app stories. Again, that's vitally.io/slash app stories for a free pair of AirPods Pro when you take a qualified demo. Our thanks to Vitally for their support of App Stories. And since I mean since you know this episode is about the idea of of an open claw for you know made by Apple or for Apple platforms you don't need to use OpenClaw if you want to get a taste of what an experience like OpenClaw is like. Exactly. Because it can be you know it can be a little confusing, it can be a little overwhelming to set up open cloud from scratch. And obviously, I mean the first place to start, I think, would be the cloud app on your Mac and using co-work or using this patch, uh, which is a feature that requires the Cloud for Mac application to be running on a Mac, but you connect via this patch from your iPhone or from your iPad. Both features, uh all of this is based on co-work. Co-work itself is a more visual flavor of cloud code. Cowork is basically clock cloud code for people who don't want to code. It's based , I believe it's the same hardness or very similar to cloud code. It's built for agentic work. So it's built for doing things in the background, for working independently, for using multiple tools, for uh you know working with folders on your computer. And it's that's you know when people you know gush over open claw I mean uh I I did it I do it uh because I really think this is the future of using AI for you know productivity work. Uh but this is what they mean. Like you you can ask a question and it's not just about you know the AI going on the web and searching for an answer and then coming back to you. It's about the AI the AI actually doing work and so actually using multiple tools. Like a person would have you know one window for notion, another window for their email, and then maybe a calendar. And so these agents can also, you know, use all these integrations in parallel and they can do things. They can, you know, just today for example, I use uh uh cloud cowork on the Mac to analyze three months worth of conversations in our Slack with our web developer , finding all the things that we said, oh, we should work on these at some point, like finding all those items, cross-referencing them against the items that our developer kept track of in an ocean database, finding the missing pieces, like the things that we said, oh we should do this, but we actually never did them, and compiling a list uh for those things that I can then I'm I'm supposed to discuss with you, John, at some point. Um so that's the kind of experience that these things can do. If you don't want to use Cloud, um you can I would I I would say you should check out Perplexity Computer. Yeah. Which is which is another uh of these agents alternatives. And again, that's a that's a pretty uh also like you know, proving exactly what we were discussing. Like perplexity now literally has two tabs where you can ask or you can go to computer. And that's exactly what we're talking about. Like ask is the chatbot. Computer is the agent doing things. And we're not talking about open AI a lot here because they really haven't gotten into this nearly as much as anthropic and perplexity, although they do have an agenic tool, which is Codex, which is for developers largely for making code. You can use it for other things, but it really is, as we talked about recently on the on the show, it really is designed to be for developers more than anything else. And I don't know, Federico, have you used codecs at all for things that are kind of non scripting or non-coding? I have tried. It's really not that good for for that because it one of the differences between there are then the differences between agents, right? And and even if codex is an agent , but it's a kind of coding agent that is designed for reading documentation about a code base up front and then executing on you know changes that you want to make to that code. And if you try and use codecs for non-coding things, that part of its nature still kicks in. So I tried, for example, to use codecs for task management with the to-doist MCP. Any thought about Todoist and read my entire to-doist account up front for a long time. Oh yeah. Probably looked at all of your completed tasks and everything. Yeah, and I mean that kind of thing makes sense for a code base. So even though you you know these agents can have the same hardness, but then you gotta optimize them for productivity , which is exactly what anthropic is doing with cowork, uh while still keeping claude code for programmers, right? And so yeah, I have tried to use codecs for non-coding things and it's not nice. Um, but now technically they have Peter Steinberger from you know the creator of OpenClaw. Now works at OpenAI. Now, technically, OpenAI did not purchase OpenClaw, they just hired the creator of Open Claw. So who knows what that means. And also there was the report from from last week in the Wall Street Journal. They kind of own it. They put it in a foundation, right? And they put it in a foundation. They so they th I guess they control it in the sense that it's not part of the corporation open AI, but it is adjacent to it at a foundation. But I know I wonder what you mean. They Peter's a smart guy and they wanted they wanted to hire Peter more than anything else. Yeah. And there was also that report from the Wall Street Journal saying that OpenAI has realized that they haven't done enough for productivity and for the enterprise and now they wanna shift their focus from these experiments like Sora or Image Generation or you know or the browser to a more productive chat GPT experience and I mean um I mean duh right I mean they realize that there's money to be made in in the productivity space and the enterprise space, which is something that Anthropic I saw they posted a breakdown of the revenue uh uh a few days ago and they showed this graph where twenty percent of Anthropic's revenue comes from subscriptions, from people subscribing to Cloud Pro or to Cloud Max, and eighty percent of the revenue comes from enterprise money. So I mean I have an aside for you, Federico. I was listening to Hardf the Hard Fork podcast over the weekend and they they said that Anthropic had told them that their number one customer, and I believe this was they're talking about individuals, their number one customer had spent a hundred and fifty thousand dollars in one month on Claude tokens. One hundred and fifty thousand dollars in one month. What are they doing? No, because there's this you know there's this whole thing going around now. This is a whole other topic, but these leaderboards for who's using the most tokens as a way of of evaluating um employee performance. If you're using a lot of tokens, the notion is that somehow you're you're more productive, which is if you've used these things, that's there's not a lot of correlation there necessarily any more than there is correlating lines of code to productivity. But but yeah, there there are people out there using lots and lots of tokens. More far more than us, I guess. Yeah. Yeah, I guess. Anything else we should oh I guess I should also mention if you want to try open claw but you don't want to set up OpenClaw you want something very similar to it, I have been having a really good experience with the new Claude channels that they announced uh la uh last week and it's the basically anthropic building open claw it's literally telegram and discord integration for cloud code and I like it so much, John. I mean there are there are you know it's a 1.0, there are many things still to figure out, many missing features compared to open claw. I have stopped using open claw, I have migrated everything to claude, uh all my skills, all my automations, like the cron jobs, the schedule tasks that were running on my Mac Studio server. And I have been using open uh not open claw, but claude c oding Telegram for the past few days and it's been quite lovely and I don't run into , you know, oh claw just disconnected from your open claw again, like because it's all integrated, it's all native. And um I mean they are copying open claw quite aggressively, but hey, this is what happens with open source projects. Yeah, I I yeah exactly. I set it up on Saturday myself. I mean I I I essentially had all the pieces in place already, all the kinds of projects and recurring tasks and things that I have Claude already do through scripts and things on my computer. But I didn't have it integrated in that way where things would happen in concert and reports would be sent to me over Telegram. And there is something about having a messaging app to communicate with these chatbots well with these you know these agents because it gives you that well it it takes you away from your desk because a lot of what we're talking about a lot of these tasks and a lot of mine are claude code tasks that really have to happen in the terminal, but now those are happening using Tmux in the background. So I have like virtual versions of Claude running on and always on Mac and I can just communicate it with it wherever I am using Telegram, which is just a nice way to kind of get away from dealing with the terminal and being at a desk at a Mac and being able to do these things with with a messaging app . So it's a it's just another one of those conveniences and another kind of piece of the puzzle. Because I think what people like, I think really it's pretty clear from chatbots that people like to chat with these things. And that's essentially what a messaging app is adding, but to a different part, a different flavor of the LLMs that we're talking about, the more code and agentic ones. And they like getting things done quickly, having someone do the the boring work. I mean, this is like this is exactly what you and I are doing. We're like delegating a lot of administrative work that we would do and being able to do things that we wouldn't have had time to do if we're just you or me doing it, doing that with things like cloud code. And that's that's kind of where I think all this is going. Yeah. So anyway, all that context up front to say could Apple even have an open claw alternative? I think we should imagine it whether they can or not. Because realistic realistically they don't even have a large language model right now. So Yeah, this is there are a lot of pieces in place. But to me, like as I was putting these notes together, the fascinating thing to me about this is that the companies like OpenAI and Anthropic, they came to this party with the large language model but without the tools. Yeah. And Apple is coming to it with the tools but not the large language model, right? That is a great point, which is maybe you know exactly why Apple realized, oh, a match made in heaven. We should partner up with some of these model providers to fix our AI story on our devices. Which I mean, that is you summed it up perfectly, John. I think that's why Apple ultimately decided to uh uh you know to team up with Google for Gemini and the next version of Siri will be integrated with Gemini because at this point they are so far behind making a large language model that it's it was probably the only way out. Uh but they do have the ecosystem. They do have the tools. They have the apps and the computers that people like to use. Right. They just don't have the glue to kinda do what they call in the AI world orchestration. Orchestration. Right. So assuming that this chatbot new series experience rollout goes well and it's powered by Jem and I I should tell you John. Um the the my uh the I use perplexity a lot. Uh-huh. And I gotta tell you, I really like using Gemini 3.1 Pro in Perplexity . Interesting. I never use Gemini. I never use I never go to Gemini.google.com. I never use the Gemini I don't anymore either, even though I subscribe still to Gemini in perplexity is very good. Oh it searches the web a lot. Um it it it has access to YouTube . It it I think by default, since it's Gemini, so made by Google, it has access to more sources and it thinks a lot, reasons a lot, and it doesn't sugarcoat things. Gemini 3.1 Pro in Perplexity is very good. And I think that would be an example of what you can expect from Gemini inside something else, right? Gemini inside Siri, for example. It's it's interesting because yes, uh Google does have better a better world knowledge primarily because they don't let you block Gemini without also losing access to Google search. And that I mean that's that's a whole uh political ball of wax. But I mean that's that's I think w the reason it is is that you can't shut down uh you can't shut down Gemini as a publisher without also cutting yourself off from Google searches, which I think despite the decline in Google searches , very few publishers are willing to do. And I have noticed a lot of publishers are blocking um blocking LLMs these days. Yeah. Still. The try cutting one off deals with them. Trying to at least. Yeah. Trying to. Uh yeah. So um let's dig into it, Federica. Yeah, I g so I guess I guess. Assuming that that that a chatbot you know that Siri powered by Gemini goes well. Like okay, so first thing like the first obvious part for me would be to have a Siri app, like a place where it's not just ephemeral voice questions to your assistant on your phone, but there's an app where you can have a chatbot experience where you can pick up a previous conversation where maybe you can organize those conversations by folder. So I don't think Apple would call them projects because you know Apple does make productivity apps, but I think you know they also want to design something for the masses and the masses understand folders. So I think uh you know a place where you can organize your chats by folders. So I think for Apple to so after this first step, Gemini in Siri, a new AI story, a Siri app in iOS 27 . Let's imagine like an open claw made by Apple. I think it's still gonna be Siri, right? And I think they would probably pitch it as now you have a Siri agent on your device that can use the same advanced Apple intelligence of of the of your main series you know uh conversations, but this time we optimize it for getting work done on your devices. I'm trying to just imagine a product so if you were Apple would, you put that in your s in your imaginary Siri app where you have the c where you have basically the chat? I would do I would do something like Perplexity where you have uh tabs. I mean even Anthropic is doing they have Claude, they have co work and they have code. Sure. I would I would do Siri, you know, world knowledge like uh questions and answers. And shortcuts though. So yeah uh here's here's what I think I I don't think people would open the shortcuts app. I think the short I think I would use the shortcuts app as the okay use a hot take. I could see the shortcuts app potentially going away even eventually if this is all successful and sort of being rolled. Just becomes the back end and it's no longer yeah just becomes the back end. Um and I could see I could see a section of the Siri app that is called Siri Agent or S I don't know. Hey Siri shortcuts Si.ri shortcuts is right there . Once again. Yeah, fine line becomes meaningful. Obviously, we activated Siri Sumware. So there's that. More details on Apple.com, as always. Um no, but I could see a section there where it's more work oriented. And so a place where Siri can uh give you a UI. Because I do think that if Apple gets into this space it'll be very visual as an experience. Oh yeah, it'll be for s it'll be s for sending your invites out, you know, the uh through the invites app. No, but I mean but I mean that'll show you that'll show you all the apps you have installed, they know and maybe they'll let you do. I can imagine a hybrid of perplexity computer and cloud co-work more than OpenClaw itself. A place where you can set up your projects and a project maybe involves some documents or maybe a folder in the files app and maybe you can say oh I want to work with these and these apps and so you pick and choose like Fantastical and you choose some of the actions, there are permission prompts. Like I can see Apple fully embracing, you know, um the shortcuts backend, but in a visual way that is more similar to these products like cloud co-work and perplexity computer, right? Yeah, I would I would hope too that in that scenario, what Apple would do is those permission prompts would be part of the setup process of whatever you're trying to do as opposed to being a runtime thing. Because that's one of the problems with shortcuts is that they interrupt what you're doing. And it would be nice to be able to just kind of go through the UI and and do those permissions as you go, as you're working on the project and setting it up at the front end. But yeah, and look, maybe Apple doesn't think that it's necessary to split these two these two experiences between chatbot and agent. Maybe they think no, we're just gonna roll it all into the main Siri app. But the idea I think will be the same that they should be tapping into the potential of their App Intense ecosystem. The fact that App Intense can already show you UI in the form of widgets, in the form of snippets. And I think you know I I could see a scenario where in IUS twenty eight, right? So we're gonna get twenty seven this year, it's gonna have Gemini , Siri, and the new Siri app. And I could see in 2027 for iUS twenty eight Siri becoming more productivity oriented for things like this land and uh I I guess what I'm imagining is hmm I wanted to say something and then I thought about it and then I changed my mind . I could also see, and maybe this is the most obvious answer of them all. I could also see Apple building an OpenClaw adjacent product in iMessage with Siri in iMessage. Because maybe they think, oh, the reason why these products like OpenClaw are so popular is that people are using their messaging apps of choice. And I could see Apple saying, Well, now you can also have I mean they wouldn't call it Siri Claw, obviously, but they now you can also iMessage Siri in iMessage to get things Yeah, I could too. I could too. I I one of the things that I think is interesting from Apple's perspective is that when you think about what anthropic and perplexity are doing, you know, to for them to be on mobile, they need to have apps that are tapping back either to the cloud or to your desktop computer. And I would think it would be interesting to see what Apple could do both for a combination of on-device models for simple tasks and then private cloud compute or whatever wherever Gemini happens to be you know running, whether it's on Google servers or Apple's and orchestrating from the cloud and with uh you know a piece of it on device so that you could do things a lot more directly on your iPhone without having to worry about running, you know, like right now you have to worry about, oh, I got to remember to keep Claude open on my Mac so that and running so that I can use dispatch or, you know, that all that. Or the same is true with perplexity, isn't it? I think with uh computer. Yeah with personal computer is gonna be the same thing. So it to me there's an advantage there. Now obviously there's like a lot of considerations of things like battery life and all that, but I do think that Apple has done a very good job with things like handoff and all the continuity features, where they these devices have a certain amount of awareness of each other. And it seems to me that, if you have multiple devices in your life, you know, an iPad, an iPhone, maybe a laptop, and you have even things like AppNap, where apps are doing certain tasks even when your laptop is closed, that they could come up with a system to distribute this work, this agentic work across devices, even locally, depending on, you know, what's the battery life, battery life. What type of Wi-Fi network are are they on? Do they have to pass off to the cloud to do real heavy lifting? And orchestrate that together not just from a orchestrating tools, software tools, but or orchestrating hardware too, which could may I think could really really, be a differentiator if Apple could get to it. But I mean there are you know, th that's many steps down the road from where we are today or where we'll be even in the next I think two years. Yeah. Yeah. Uh look, I don't know. I I I really don't know. If I mean obviously you you the point of the episode was to imagine this product. And and I think we imagined it. I think um there's the I think Apple the the the other thing I wanted to stress I think they have the potential to make this kind of experience so much more user friendly. Yes. Um than it currently is. Especially because it you know, these apps would be running on your device. So like you mentioned, there would be a privacy angle that's n you know, that's quite important. It would mean that you would have proper user inter faces for all of these apps, uh thanks to the work that's already been done with widgets and and snippets in in app intents . Um but it's a lot of work to get this orchestration done right, and you gotta start in a much, much simpler fashion. Yeah, because in a way it's its own operating system when you think about it. It kind of is. It's like a a whole new layer of the of the OS that has to span a whole bunch of different kinds of devices. Yeah. Yeah. And I you know, I talked about a Siri app. I wouldn't discount the potential for IMessage integration. I would also think about that. I I don't think it's I don't think it's a silly idea. But who I mean the other thing is we're talking about we're imagining open claw made by Apple now, but and we don't even have a basic chatbot on iOS. Who knows what the market it's gonna be like a year from now? Yeah. We don't even know what the market is gonna be a month from now. So no, and it's I mean look, and developers don't even have access to private cloud computes. So they've got they've got a four thousand token contacts window still on device, and that's that's just downright primitive in today's day and age, unfortunately. I mean, maybe we'll see a lot at WWC this year, but you know, the catching up has never been I I feel like Apple is catching up but they're catching up too slowly. Yeah. Yeah. So we'll see. But it's fun to imagine things. It is fun. It is fun. And I think look, it really is with it it's the age old reality with Apple is that the coordination of both hardware and software together, that's powerful mix that by controlling that whole stack, you could do things that right now are maybe a little clunky and workaroundy for the likes of anthropic and perplexity and open AI, but Apple could do it. It's just they need to get the last piece. The big piece is the LLM piece. And so we'll we'll see how see how all that goes. All right, Federico. Well, great conversation. I've got a we've got some stuff to talk about in the post show. I want to talk about where AI ranks in your mind and in kind of the lifetime of technologies changes that we've seen. So we'll talk about that in the post sho w. In the meantime, you can find me and Federico at MacStories.net. And we are also, of course, writing for Club Mac Stories, which is baked right into MacStories.net these days. You know, check it out if you haven't ever checked out the club. Just go there and you know poke around a little bit. You can see some of what we're writing about because we've got the the first parts of that paywalled stuff up there uh visible for everybody to look at for each article. And uh yeah, we'll be back in another week. You can find us on social media too, where Federico is at Fiti ci. That's V-I-T-I-C-C-I . And I'm at John Vorriz, J-O-H-N-V-O-O-R-H-W S. Talk to you next week, Federico . Ciao, John .