Xcode Evolution and Closing Thoughts

Hello and welcome to another episode of App Stories. Today's episode is brought to you by albums, controller for Home Kit and Revenue Cat. I'm John Voriz. And of course, I have Federico with me, Federico Viti, that is, here in person. That will be me at Apple Park. And then again for another year. And we have two very special guests, two good friends of ours, mister Mike Hurley to my left. Hello, hello, Michael. And Chris Ferli. How you doing? Chris. I'm doing . I actually have something for Federico though. No , no, what could it possibly do? I have something hanging on hang on . Federico, I would like to present you with a doctor Pepper, zero sugar with cream soda , all yours . Nobody has touched, well, I mean, I just touched it, but that's all yours. You can have that. That's a lot of adjectives attached to the ones closed. No, you gotta drink it at some point. I'll be deeply offended. You could just text them later and be like, oh, it was so good. It happens in the show. No, it's fine. Okay, it does now. It's good. So you're living this here as a threat, as a visual threat to destroy the podcast space. Yeah, for those who don't know last year, Chris imposed one of these on Federico at eight in the morning on the last day of WWDC and yep, I did . I did. And I nearly died. He didn't nearly die. It was good. It was horrifying. Thank you for bringing me to the show. It was a different year and you was a poisoning. Yeah, Federico has an adventure every year this week every single year . And that was on me that gave him food poisoning that year. I was on. I took him to a subshop that didn't work. Enough about that. All right, all right. So we have Mike. Hello and Chris. Hello. Thank you . We had a busy day so far. Yeah . Went to the keynote . First of all, first question, vibes . How are we feeling? Vibes were good. Vibes are good. Good. It was good. We had a nice introduction at the very beginning before the livestream started where Craig came out and kind of hyped up Tim . And then Tim came out. Got the big stand in O that I thought he was gonna get in . They were able to cut it off. The man said thank you about sixteen times in a row. It was pretty good. Yeah, it was great. I've got some good pictures of that, right? Like all the phones like looking at him and he's waving. He's cool. Yeah, it was really cool. We actually, Federico and I started our day out by getting here literally before anybody else. We were literally the first two people in line for the media the media line and then to get across . Yeah, we got rejected at the we did initially we were a little too eager. We got there a little too early . Oh way . Stand over there s,ir. Still ready for you . Yeah, but it worked out, it worked out. We all came in, we had a good time. And the sitting outside is always interesting because where we typically sit with the press is that it's very sunny, but slowly but surely the shade come the sun moves across the sky, the shade comes in. There's a moment when it cools. And you're like , We're in that moment. And if you're like forty five minutes , it's like and I believe my theory is that they usually time the best announcement for that moment . Yeah . Right when you think you're about to die of sunburn and you're thirsty going on. Go back in previous keynotes, the boring stuff in that portion of like the really hot sun . That's the bad part of it. They hit you right in that elite of moment. Yeah. And like last year I pedal ass, it was just when the sun went beyond the roof. I was like, now you're chill and here's the good part of the keynote . I wanted to mention that after the keynote , we the three of us you were doing something else . We went to a tech talk . Yeah, yes. Now, tech talks are not a new name for Apple first developers. Usually for developers . This was a tech talk at the what's it called, the developer center little auditorium at the one end of the building. Yeah. Hosted by Craig Federigi himself , joined by members of Michael Cockwell Amaza Mermanian. You're good with names going. Sebastian, I can't remember . Marines , yeah. Yes . Members of the Apple Intelligence. Apple intelligence and IOS teams. And there was about fifty minutes of a technical conversation about with diagrams. Good diagrams. Diagrams. We're going to talk about that later. We're going to talk about that later. Good overview, I think. Demos. Demos live stage . Yep, yep. And questions taken from the press who were there probably a couple hundred percent. Not live Q and A. No, but they were submitted in advance, yeah. Yeah. Two questions and they didn't answer them. They didn't answer ours either. No. Obviously, the other very important sort of while we're talking about the vibes and sort of the backstories. Very important topic . Little Finder Guy. He showed up. Little Finder guy is an official pin . Comes with its own little house . Separate from the others pins. that you get My theory is that it was a late edition . They had a pin set. They have the pin set, right? It's in black. Same kind of packaging. Find a mascot came in a little and I think it was they were like, people really like this . Let's go . this But here is my request to Apple now, right? So this find a mascot has shown up in TikTok videos. Yeah , the community's taking this thing. They love it. They made a pin and it was in the keynote peaked out from the side of a laptop like it was in TikTok. Apple name it because everyone's coming with their own names now I want them to say the name of this get old character. What is the character merchant at some point too where could see shirts and they should, yes, that's why people love this little guy. Yeah . Yeah. I want a name. I think it should just be findy. Find his good. Find his . We set all together in the keynote. Yeah, same group of people, I would say. Yeah, pretty much the same group as one of my favorite favorite moments from the keynote. There was that slide that was like a thousand words or I have a story about that. Yes, which is so Federica took an image or got an image and then just fed it to an element like summer . It's rather simple. All right, all right, all right. So before I leave Sylvia back at home, she's like, Oh, make sure you turn off your computer . I mean, she's not gonna listen to this anyway . So I have this Mac Studio server at home that I keep running at all times with tailscale and it's got all my agents running. It's mostly Kodaks running . And I said, yeah, sure, I'll turn it off. I didn't turn it off because I knew I was going to use it. And during the keynote, we got that image . So I grabbed the screenshot in four K from the keynote. And I opened Codex on my phone , connected to the Mac studio and said, Perform OCR on this image and gave me a list of all the features that were mentioned in this image . And organize it by topic? Yep. Yeah. And that was done in about, I don't know, like a minute or something . And centered off to I kid you not, I did the same thing. Look at that. Screenshot . With Codex. Yeah. Yeah, that's great. It was great. I mean, we send it off to our team. We have Devin and Jonathan back at home based. Doing their writing. Yeah, they're working on our individual home bases. Yes, they're individual home bases. Arkansas and London, you know, a little bit different. We have iOS twenty seven on my iPhone Air . This has iPad OS twenty seven is iPad Pro . I have IOS twenty seven on a sixteen . Okay , Mike . I'm not going to put it on my iPhone because I need to go home, right? It's important for me to get home again . I was thinking maybe I'll put it on my iPad but it's like, yeah, but I've got an eleven hour flight during the day . So you're gonna need to watch something . Yeah, you don't want to take it. And then it's like , I have another phone. I have an iPhone here. But I can't put it on that phone because I need that phone for widget smith. So I can't be like, here's about work. That's really a great point. But like I have to make content for Widget Smith. Oh, sure. Like photos and videos. And like one, I can't have the UI not look correct in case it and also what if all I know maybe the app completely is broken . That's why I went with my sixteen E because even though I use as my webcam, when we don't, we're not surrounded by many, many iPhones filming us here, I have other webcams and I can use those. My plan is I will put on it my iPhone when I get home.. Yeah Or I believe you have a wallet with you. Oh yeah, Google buy some more stuff. There is no need. There is not no store. There is no need. There are no need. One year when I needed to have like a spare device . You bought a homepod here once. So for W WE I purchased once. I have a purchase history for WWE. I purchased a homepod and I have a pro with you ?es Y. A spare iPad Pro. Me and me, you and Steven have a picture of you outside the infinite loop. With that iPad. Yes. I got an iPad air, I think, in the year where I was stuck in a hotel room. Yes , I need ifed something to occupy myself. So it's like I will buy an iPad and play with stage manager, which is what a sad thing was not like stock you know what you can you can get out the door. I wasn't allowed to leave . He got jammed in the door. We don't need to talk about what happened. But it's not like he got stuck . I mean, Booney the Pooh. It was kind of like that. It was kind of like that . It kind of like Now , overall, what do you think the overall reaction to the keynote has been? Oh, interesting. I've not been online. So you're going to have to tell me how people Well, we were at the platform state of the Union and the developers were very happy, I would say because they're what are the top levels We know we'll get into a little bit more in a bit, but there's like high level X code, there's a lot of changes to and it's it's addressing a lot of things that we've heard from other developers over the last year or two that were, you know, pain points, friction points. And so there was a lot of applause. They were hinting in the keynote that was going to was being . I was very, I can't wait to hear more about it. did this line were building X code with X code, which is usually the kind of line that you hear from anthropic or Open AI when they say we built Cloud with Cloud . And it was kind of funny to hear the same line from Apple. I like that. That's a nice, that's nice. Yeah. It's like we know too. Yeah. Because that's like the big thing this year is, you know , we're making things better, right? Like that's there are the three temp roles of the operating system story. One of them is we're making things better. We're taking away pain points. Like that is obviously they've been their focus for the year because the platform teams were given the freedom to do that because the AI team was doing what they were doing covering bases for everyone, I guess. In that Word Cloud, there is so many things in there that Word,Cloud sl ide of we made this thing faster . We made the iPad switching between iPad windows faster. We made opening apps faster . There's so many of those little things. There was a lot of that platform state of the Union too, a lot more detail about that in terms of how the different components work under the hood and how that's accelerating things for people. Yeah. You know, I think Frederick were bearing the lead about who we sat with during the Tech Talk. Yes. Yeah, I think we really are. Once now , well, yes, and John Grimmer, and Mike . But we had a whole there was a row right in front of us that was . Yes. And so here's the deal. So John and I this is our vove . It's not our first rodeo is. It's not our first rodeo . And we have picked up a few tips and tricks along the way, such as w walalkk f fastast, . And if you can be first to anything, especially when there's movement of PRP wonders away you went Oh, we look we like move. If you can be first to be first and especially if when you walk into some kind of theater or room with seats , ask , can I sit in the second row? Usually the first row is reserved, right ? For Apple people . And so John and I were first to walk into this developer center auditorium . And the first row, we can hear the security people and sort of the Apple employees saying the first row is for reserve sitting. So I asked, but can we sit in the second row? And they're like, yeah sure. And so I go right in the middle of the second row . And we can tell, obviously over the next couple of minutes that the row is fitting up. And there's Jos h with a bunch of senior PR people . And there are these two seats that are being left open . I also picked up that something was going on they started telling these messages about like emergency exits and like things to do in case of emergency, which usually doesn't happen. And sure enough, in front of us, a few minutes later, there's John Turnus and a couple of minutes later Tim Cook , sitting right next to him right was in front of Federic o and John Turners was right in May. So we got the pretty good seats, I would say. Did you pass him mini notes, you know, like no? No, we were chatting . Sure, yeah. Here's my wish list. Yeah, I was whispering in his ear whispering all the time. I'm the John Whisper. Yeah, I get up . That's how but we were able to say hi at the end. That was nice, very nice and we said congrat s to Tim and thank you for everything. I believe Tim mentioned it was his twentieth. fifteenth. fifteenth. fifteenth consecutive WB. Wow , that's pretty amazing. Yeah. And the joke of today is Jason's thirtieth . Tim will never catch him. Four guy at this point. So Jason, he's done thirty, consecutive. Thirty consecutive executive. That's amazing. I've done thirteen. What would they talk about in the first WWC ? Like years I said please make apps to make apps because this was the days when WWC did not sell out WWC did not sell out until it used to be very very yeah. It wasn't until the iPhone. Right. And it was very exciting. Going for thirty years. Yes , which is why on our new podcast coming soon called Design in California, wow, we'll be talking about things like that. I bet you that is my life. Go back, Mike and Jason started them. We can go check it out. They're doing excellent work. Thank you. Yeah. Excited to hear us . This episode of App Stories is brought to you by Albums. Albums is the app that cares about your music library as much as you do. It's an album focused Apple music player and library manager with features like album shuffle and album to album playback controls that put the album experience first . It's made by one music obsessed indie developer, Adam Linder, and it's been on O iS for about seven years with a Mac version that arrived in late twenty twenty five . Your queue syncs across all of your devices through iCloud and you can build custom collections with tagging, including wishlist tags for albums that aren't in your library yet. Albums goes deep on stats with graphs, weekly, monthly, and yearly report s . Plus, you can browse liner notes and production credits pulled from music brains and discogs. Insight collections resurface old favorites and album anniversaries . There's also robust last.FM support with real time and manual scrobbling and full history import . Plus a release feed keeps you up to date with new music from your artist s and labels . There's even a cardplay app, interactive widgets and app intents and shortcuts for nearly every action , and by design, there are no AI features. Learn more at albumsth ot com slash app stories. That's albumstheap dot com slash app stories. That's albums. The app that cares about your music library as much as you do. Our thanks to Albums for their support of the show . Shall we talk about the actual stuff? Yeah, we should. We should dive into kind of an overview of the platforms. Yeah . Yeah. So what's different I think this year is that we didn't go platform by platform in the keynote. At all. At all. Not they showed a list. It's like here they are . It the' nsames of all of them. These are case you couldn't guess . Here's the names. And here are the but here are the three things we talked about. Yeah, but like in this it affects all of them . Exactly. And they did dig in. Oh, sure. Yeah. They were dipping in and out. There were many things the only one that didn't get like a any moment, right? Like there are TVOS features , but all of the others they were shown in some way. Right. I don't believe TVOS was like TVOS was not shown here and's a TVOS thing. Yeah, like even watch got like a very quick the watch got a quick demo of the series the new series. But the sort of the overarching idea would be we're focusing on improvements and making things better everywhere, both in terms of design and actual performance. Yes. On the design side , so I got twenty seven on my phone . When you set it up, you update, there's a new welcome onboard experience and you immediately get the slider in the onboard info. So it starts liquid glass. The liquid glass slider that lets you go from extremely clear li,quid glass , almost IOS twenty six beta one clear glass to a very tinted frosted glass on the right side of the slider. And you immediately get the slider. So a little more three D three dimensional too. The material itself feels more three D. Feels like it's popping out of the screen a little bit. Yes . And the other thing that I noticed is that and I mean, Chris, you also have it on the on the iPad. Yeah, yeah. Even if you have d party apps that have not been updated for twenty seven , they immediately get the new liquid glasses. And they don't even need to recompile. No passengers. On twenty seven, even if you have an old third party app that never adopted twenty six , now you get the new keyboard, the new software keyboard. So I didn't realize that. I'm happy about that, but that is odd. That is a little bit odd. I don't think I've seen that in the past. In any other IOS version. There's a thing about this year that breaks on apps . There are going to be there are a lot of features that are coming for free for developers that they don't really have to do anything. It's like the liquid glass like you mentioned, but also of course all the under the hood stuff that's going to that's optimizing things like the speed of launching apps on devices , speeding up some of the transitions and moving around windows , making long lists scrolling more smooth and that sort of thing too . There's a lot of changes on the Mac. You know, I think a lot of people were very critical about the design on Mac OS Tahoe with especially liquid glass. It felt to me like when I did my review last year, it felt to me like incomplete, like unfinished to some degree because it had different degrees of application depending on what system part of the system you were in. And this year they, you know, Apple's really focused on things like the toolbars , which stand out a little more from the background, especially when you're an app like something like photos or music where you often will have very random colors behind these different elements . It diffuses the light more so that you get a more uniform look and the buttons actually look more they're just easier to pick out against the toolbar that sits behind them. That was one of the things that I really liked. And then I think another thing that some people had trouble with is that the sidebars on the Mac kind of their own separate element that sat on top of the window and didn't reach all the way to the edge of the window. And sometimes depending on what the overlap of your windows were, sometimes that looked as though it were almost a separate window that was two windows on top of each other, not one actual window with a sidebar. And that has changed too, because the sidebar now stretches all the way to the very edge of the window. I didn't mind that old look, but I do like the change too. I mean , I'm just happy that they reconsidered it. That's why I care about more. I was not as against this as many of our friends were. Right. I think I'm a little bit more forgiving because I'm less of a like stallwall. Right? Like I don't hold , I don't think I really hold any operating system with such a regard that any change would mean I would not update it. Right? I'm the same way. But I appreciate that they've gone back in and taken time because the Mac felt like it had the least . I don't think the Mac was the worst place. I think like watch OS maybe suffers the most from a design perspective in some areas , but on twenty six , the MAC was clearly the one where they did not have or were not able to spend enough time . Didn't feel as comprehensive. It didn't hold together quite as well. I don't know if you see it the menu icons they've gone they're gone. Their menu I've gone through the iPad. Also on the iPad as well in the menu . And I believe there's an API for developers to say, No, I want to bring back the icon in the menu or by default, that's just gonna yeah . And the corner radius is, I mean, look , my co hosts are going to do an hour and a half just on the corner radiuses . A thing I never've noticed it once,. save I don't know what people can say. I didn't notice it until goes in the comfort zone, also on the Mac Stories Network. My coach , Matt Neilon, I didn't notice it until he pointed it out to me and I broke me. And now I can't stop noticing it. Now you'll be fine. Now I'll be fine. You'd be good. Yeah. Very nice. What else is there? Federica? That's kind of across the wish you mentioned the iPad menu bar . So with the iPad slowly becoming more and more like a Mac? Yeah, I mean you can now make the menu bar persistent just the way it is on the Mac. Right. So just the way it is on the Mac, it's now left aligned oh that's good. I never liked that it was there . Now there's a still the clock on the left side . So it goes clock app menu bar . So it's still that way. Like it almost is just like begging to move the clock to the right side now but even you can choose to always see the name of the foreground app in the status bar , even when you have the menu bar hidden . Yeah, I believe that's just there when you're in the window mode or stage manager. If you do the full screen mode that goes away. Yeah. Yeah, it's nice. Yeah. Yeah, that's a nice little cue as to what is actually the active app on your device, which you didn't always have. On the Mac, they brought the color back to the sidebars and only the color shows in the active app. So it's like a number of thing to help you on that. That was another regression from before . The other thing that I picked up in a bunch of different places during the keynote and the State of the Union is all the different ways that Apple is telegraphing the idea of an iPhone that will change shapes. Yes, yes, yes , yeah. When unfolded by not saying, Oh, we're making a foldable phone. For example, developers can now inexcode, there's a new device hub with places simulator places the simulator based. New simulator UI integrates with agents. We can talk about that later . But one of the things they can do, they can literally just grab the preview and resize it. Why? Why? Keeps going . On the iPad, when you can , you know, you've always been able to use iPhone apps on the iPad. Well, now that iPhone app you can resize and also the iPhone mirroring on the map. iPhone mirroring, you can resize the iPad. And I can do one more, Apple Weather on the iPhone so the weather app, right? Most people will use the weather app in portrait. Well now, you can also use it in landscape. Why? Yeah. So these are all the little ways that these are the things we were looking for. Yes. Right. Like history has shown there will always be something exactly. There is always an indication and the only way we were going to believe they're actually gonna do this is if these things happened We mentioned please use auto layout Why ? Well and on the MAC side of things, there is now drawing and free form and notes and there is touchscreen improvements for sidecar . Who knows what that is leading to why? Think about that, but yes, that's obviously also a thing to consider . Yeah, right. How will you draw? I don't worry about it Interesting. Yeah . We tried the new shortcuts generative feature . It's not called Shortcut's Playground . You got lucky. I was wondering when they were building up to it. twenty two. Like, what is this gonna be called? Two. The name does it have to describe a shortcut.cribe a short Des cut. It's the new default screen for when you open the shortcuts app and you create tap on new shortcut , the default is now describe your shortcut. So it looks essentially looks like it's field or like a chat right like yeah where like you're given like a prompt. Yeah, you're given a prompt and the three column thing I think is really cool we'll mention that you got the default view is the prompt UI. There's an icon in the upper right corner that lets you switch back to the default editor. On the iPad , there's a new three column view with your shortcuts library on the left , editor in the middle and the right sidebar on the right. And when you click on the shortcuts in the library, it changes the editor. It's a lot easier to kind of navigate shortcuts that way. We tried describing a shortcut didn work't for me? Yeah , I got it to work. So there was an example on Apple's website that when you take your iPad and connect it to say like a magic keyboard , it'll you can have it automatically turn on windows . So I did that and then I did the opposite of I take it off and it went into full screen mode. And that worked just fine. But anything that was kind of more on the complex side of things, that's where it definitely started to fall down. Yeah . Right now it sounds like it's a little closer to app shortcuts, right? Which are single action things, but this is more . Isn't that right though? Yeah. Like the most complex shortcuts are desired by the people who probably know how to do it, right ? Yeah, okay, I probably shouldn't say complex. I would say middle of the road shortcuts. Ten action shortcuts. That's a pretty that's a pretty advanced shortcut. Okay. Right. Like the user user the user who would use this, like this feature the person doesn't know how to do this. That's totally fair. And they would never do this like text so right, like text my partner when I leave work on it. That is a perfect example of a thing that all four of us know how to build and can build it for you in five minutes because we just understand it's also something that is very useful to people but it's just outside of , I would say , being understandable. Right. There's a couple of actions and a personal automation. And those automation is where it falls down. Right. Like do you do people even well, do people even know shortcuts exists, right? And this is like a, I think this is a good way and it's also can help people understand how to use the app as well because they can look at what they made and can tweak it. I haven't played with it, but I think this is a very good thing and I think is doing what I expected. And what's interesting is if you go into the automations tab, there's no plus button to create a background automation anymore. You describe the automation you want. So that's interesting. That example . You don't go in and say like when I leave this location trigger 'cause it was it very was a very a separate UI. Pretty good. Yeah, that is good. That UI is just way more complicated than it needs to be, which I understand why. I always loved it personally. Well, of course you got it because you and I would get it, but like it's a lot of steps. Yeah, right And so like these things are much more easily easy to explain by just speaking them, right? Yeah. Yeah . Anything else that we picked up in the different platforms? Well, one other thing on shortcuts, there's an if else statement. Oh , yeah, good to me. That's speaking of accessible consumer features . That's total opposite Yes, it is. That's even it's one that we've all wanted. I wanted it. We've asked for it for how many years like and there's ways kind of like fake it, your way around it. Everybody has their methods . There's stuff. There's no point spending more time talking about this, right? Seriously. Right. Like that's what we should. Yeah . This episode of App Stories is brought to you by Controller for Home Kit. Controller is a third party Apple home companion for the iPhone , iPad, and Mac , and the new version nine point zero introduces controller AI. Here's how it works. You pick a home scene, a home automation, or a controller workflow , and you simply describe what it should do . Controller AI assembles it for you, choosing the right sensors and time windows and building multiple automations when that's what the job needs. Workflows go beyond what Apple Home can do on its own with conditions, weights, branches, and even weather data, so you can set up something like efficient garden watering that actually responds to the forecast . The idea is that you decide what to create not how to build it. For automations to run in the background , you'll want a controller hub , which can be an iPhone, iPad, Mac or Apple TV running controller. A wall mounted iPad can even become an Apple homepad style display. There's a LIDAR powered three D floor plan scan that gives you tappable spatial overviews of your home plus backups , history charts, advanced notifications , secure setup code storage, widget s, NFC tag support, and an Apple Watch app. There's a seven day free trial and App Stories listeners get twenty five percent off the first year . That's essentials for twenty nine ninety nine cents or plus for fifty nine ninety nine cents at controllerforhomekit com slash abstories. That's controllerfomekit. com slash abstories. Take control of your smart home with Controller for Home Kit. Our thanks to Controller for Home Kit for their support of the show . We should talk about Seri and Apple intelligence. Yeah. So first of all, it's called Siri AI. Yes. Good name? Yes. So I think so. Here's my thought on this . Apple intelligence doesn't really exist anymore . Right? Like Apple intelligence is a broad set of features. So like, you know, essentially the machine learning stuff that they used to do. They're just called Apple Intelligence . Serious AI is now the tool. It seems like all of that all of the things that were that we didn't get in Apple Intelligence, they are now all in Siri AI . So I think that that is the brand. They've decided to do I don't know why they've decided to do that. Like why they're calling it Siri AI, I don't think they needed to do that, but they have . I think that this is now the brand moving forward for these types of features. Yeah, that makes sense. I think you're probably right. Bifurcating the brand between Siri and S iri AI , it lets you easily say, Oh, you're using the old Siri versus Siri AI and it allows them to also include it in the iCloud plus pricing plans, which we're going to talk about that later . But yeah, so it's called Siri AI . There's a new Siri app on the phone . Something which we all wanted, and we talked about an awful lot. I mean, I think that because my biggest concern was that Siri would remain an audio only feature no longer just audio. It's multimodal or that even to get to your previous conversations you'd have to open Siri do the dynamic island or whatever. I like that there is just an app that I can have on my home screen and I can just go there . And it's very approachable. I really like the design of it the way it's done with tiles. If there's a tile that represents conversation with Siri, and it will have like images that maybe will kind of remind s me of the journal app. It is a little bit like that. Yeah , that's a good thing. So there are new ways to activate the new series. There's new design. You can still hold the side button. You can invoke Siri via voice or you can swipe down from the dynamic islands. You get this new floating bubble design with liquid glass. I assume that one goes straight to text as the I think when you yeah like you're going to respond to something. Like if you're if you're talking from nothing, like looking at my phone and I don't want to speak. I would assume if I do the swipe down one that would go probably in that case would make sense. Yeah. We should say the Siri app is not in beta one . That's something you have to request access. Yes. There's a there's a well it doesn't come with the beta, but it should be in beta one though, right? Yes. Yes. It seems like it's a separate thing. Like it will come to you once you're through the white list. Yes. Right. Yeah. There's a waitlist you can sign up for. Just what twenty six in twenty twenty four. I was eighteen was eighteen, eighteen. You had time to put your name, you say, I want this, and it was the same with image playgrounds as well when that came. Now, obviously, before we talk about the features, I think we should talk about the foundation of all of this. Yeah . We've heard obviously leading up to this the rumors of no longer even rumors. They put out a joint statement Apple and Google saying we're collaborating with Google to for the new versions of Apple Intelligence Foundation models . And we got a bunch of them, and I'm going to try my best to explain them. All right. So we have two new on device models and three new cloud models . And behind all of them there's the system orchestrator . That's a prompt writer. It's a prompt writer and the apple equivalent of the routing model. Yeah. Interesting because when open AI and Antropic tried a routing system. It didn't really go well for their audience. Yeah, it didn't really anywhere. Yeah, so we'll see about this. Anyway, on device, we have the third generation of AFM , which is now called AFM core . So the on debis local model, there's a new version of AFM called AFM core and there's AFM core advanced . These are two different models. AFM Core is what used to be called AFM . It's a small dense architecture, three billion parameter model , not multimodal. So it's text only. It's what developers have been able to use so far . That's been the newer version of this . It's a newer version of that. Yeah, being developed by Apple only , and it's still a pretty small text only model. Right. AFM Core Advanced , that's been built by Apple and Google together . That's a sparse architecture LM on device twenty billion parameter model. Now this is interesting for a couple of reasons. A sparse architecture model it means that when you when you write a prompt to the model , in a dense architecture model, all the parameters get loaded around time by the model all at once. Into the memory, into the memory. Yeah. And the CPU. In a sparse architecture , only some of the parameters in the model get activated at runtime. The problem is that with traditional sparse architecture models , those the tokens and the parameters that get activated , they keep shuffling depending on the length of the question. So let's say that you ask a really long question and you covered the spectrum of your imagination with this question. Like you start by saying, hey, I want to buy a birthday gift to Mike. Mike is really into Hawaii. And oh, by the way, it's also my mom's birthday in Cambodia. So and imagine that in a sparse architecture model, the model starts by saying, okay, I got to worry about Mike. And then he keeps reading the question. Oh no, I'm gonna worry about Tech's model. Right. And so even though it's a sparse architecture model, the problem with the default of those models is that yes, they activate fewer parameters , but they keep reshuffling in memory and in the CPU. And that's not really power efficient or memory efficient for a phone. So what Apple has done, and they have I believe they have a white paper dimension on this. They invented it. They invented it a few years ago. It's a new generation of sparse architecture model where twenty billion parameters total one to four of them get activated one to four billion. One to four billion of them. No one to four One to four billion get activated and they stay fixed so they don't change that allows them to have this pretty big model that is multimodal. So even supports image input on dev theice , run on a phone and the audio and it does deal with the audio delay the new series always synthesizer for Siri, but there's a catch and the catch is that it requires devices with twelve gigabytes of RAM . So it's not available to the eight gigabytes of RAM devices that were able to run the regular AFM core, which is why they probably split this into two models. Isn't it funny that we're back here again? Back here at the twelve This was what happened with the first generation of Apple Intelligence. It ran on just the highest end or the newest devices and we're back because it's the iPhone Air and it was logical that it would happen. It would run on the air, right? Air in seventeen Pro. Yeah. And I could see this being a theme over the next several years in that as these advances are made in AI very want to be cutting edge. That's what I mean. Yeah, yeah, exactly. And if you want to have something running on device , right? And also like , what are the lead times of these things? You know, like are the software team just working faster than the hardware team, you know, your hardware teams years in advance for these chip designs, right? And so like they're always going, I assume they're going to be on like the thin end of that wedge for for time to come. But a twenty billion parameter model running on devices no joke. Like when I saw the RAM requirements, I was talking to Jason and I said, Oh, that sounds at a minimum like an eight to twelve billion . Yeah, that's what we were talking about in days before. B model on device that's pretty impressive, especially when it's multi modeled and it even supports audio, any images and text. Yeah. Sounds pretty serious. That's pretty serious. I wonder about the context window and I wonder exactly what happened there . My theory is that this is an AFM model, so an Apple made model that's been distilled with Gemini output. I think that that's probably Yeah. they No know, they mentioned refined refined refined what that meant for it was in the tech talk that we went . It was in the tech talk . But they mentioned the refinements for AFM cloud. So on the cloud side . So in private cloud compute, we have three models AFM Cloud, AFM Cloud Pro , and ADM Cloud. So ADM, it's the diffusion model, it's the image model that deals with special reframing. We can talk about that image playground playgrounds, all of that image stuff. AFM Cloud and AFM Cloud Pro , so they mentioned the AFM cloud is the one they're optimizing for lat ency and serving costs. And AFM Cloud Pro is the one that deals with reasoning behind the scenes and agentic tool calling. So they didn't mention which actual Google Gemini model they're using as the foundation ? If I were to guess, didn't they say that they weren't using Gemini? They were using the Google Frontier model? They were using Google Frontier. So what is a Google Frontier model? At the moment, Google Frontier Pro. If you go family by family , flashlight is three point one , flash is three point five and pro is still three point one . I think that's right. So the three point one, that is the number of their frontier model. I'm trying to simply it depends on what they're using. Okay . So we need to infer what they meant. So for AFM Cloud, we're optimizing for latency and serving cost. That means it needs to be fast. And it needs to be cheap. Yeah . That sounds to me like flashlight . During the keynote, we saw and in the talk talk, we saw some demos . And we got a range latency times . We saw responses that took two to three seconds . And I was manually counting. I was sitting in there . It was coding in my head too. We got one response that was like nine to ten seconds . If I were to guess, I would say that Gemini Pro is not involved in this at all. You cannot get Gemini Pro to give your response in nine seconds. That's a big chunky model that loves to reason, loves to think and has a lot of latency, and it's expensive to serve. Flash three point five is able to reason. It's speedy enough. It's a pretty intelligent model. I think AFM Cloud Pro Prediction, right? Yes . I think AFM Cloud Pro is based on Gemini Flash three point five . We will try to confirm this. We will try to confirm however we have not controlled what I found most interesting about the Pro is how they're managing it from a hardware perspective. Yes , that model is lives in Google Data Centers. Yes. And they've partnered with Google and Nvidia and Intel to get the hardware to where it needs to be. And they're using all of these companies priv' acy technologies and then adding on the components they need for it still to class as all of the things they care about for privacy. Right. They explained it in the sense that these other companies had stuff. They're their own things that are similar , but they didn't have all the pieces. And then so Apple also brought in their own contributed their own pieces to get it to where it's and then add their security stuff on it like Apple can send code to these servers , no one can open them all this kind of stuff. Right, exactly. It's all code that's been , you know, that 's signed code. Yeah, it's signed by Apple, right? Yeah. And then if your phone ever sends anything, it has to get that hand shake that this is only Apple's ever touched stuff . I found that to be fascinating. Yeah, it was fascinating. And you know, I mean, I think we've we heard about private cloud compute in the past. And I think the message was very clearly this is the same technology. It's implemented differently, but it's the same level of security and privacy that you got when with what was described a year or two ago. Yeah , it's almost like private cloud computers become a standard rather than a piece of technology. Yeah, exactly. The actual physical technology which we're still using. So the other one, the AFM cloud, cloud . That one is still Apple serving that. Yep. AFM Cloud Pro. I don't know about the diffusion model, but AFM Cloud Pro. That is happening at Google. I would imagine that Google is serving that. Yeah, for the similar reasons. They never mentioned it. It's I was telling you this, I think, during the keynote. It's very clear to me that it's based on either the Nanobana model for image combination and editing . And when they showed off the spatial reframing . So the way that this works, we can talk about some of these features now. The way spatial reframing one of the new photo editing features in IOS twenty seven and the other platforms, it allows you to reframe a photo by sort of almost changing where the camera was positioned. It's a perspective thing. It's a perspective thing. And since you're changing the position of the camera and if you angle the camera and it's now capturing something that didn't exist , like say that I'm angling the camera more to the right and there's, you know, there's supposed to be something behind you on the right that wasn't there before . Now the model needs to fill that part of the image. But Apple said the way that we're doing this is we're not touching the original portion of the photo with your original moment that was captured. We're just filling that part of the image. How do you do that? That just happens to be a Google Gemini API. It's the image segmentation API that allows the model to receive precise coordinates, like x and y points on an image to say, well, I'm supposed to fill in the canvas at this coordinates, leave the rest alone. Right. And it's also taking advantage of the spatial stuff because you are kind of rotating the subject of the photos or scenes parts that would otherwise have been kind of hidden. And you can think of it a little bit like when you're trying to, if you try to reframe a shot when you don't have this cutting technology, what you end up doing is you end up zooming in because you're turning the photo and you're cutting off the cor ners and so you end up with something that's really zoomed in. This doesn't do that. It lets you do that and then those parts that would otherwise have been blank get filled in with additional background generative background What I found really interesting is the idea that so you got all these AI labs now talking about the models having a memory about you and they're all these proprietary locked systems, right? And they're now actually sort of almost waging wars against one another. Claude can import your memories from Ch ipiti, ChajpT can import your memories from Germani and so forth. And here's Apple say'ings, well, we don't have a memory system in so far as your device is your entire memory. Your personal contacts much better. It is. I don't have to tell serious things about me and to say, remember to do this or remember that also means it's not going to do, I expect the thing that Ch ibT loves, which is to like bring everything back to this one piece of information it tells about me. Yes. It's like you just need to chill out So I made the mistake of telling Chip that I have two dogs, right? Yeah . And occasionally it comes in handy. Like when I'm researching a restaurant and they're saying, Oh, and by the way, it's also it's also dog friendly so you can bring Zelda and ginger. A few days ago, I believe I was asking a question about I don't know which is funny. I was asking a question about a bathing suit and it was like, hey, is this like the right type of cotton for this bathing suit? The dog friendly. GPS said, Well, yes, it's great for you, but if you're thinking about it for the dogs suit . It's like if I ask any question it's a podcast, right? It's like, oh, this would be so good for state of the workflow. It's like no, I do more than one shit. I do more than one . You know this about me . Leave me alone. I've had the same thing with automation and shortcuts where I've said, Oh yeah, I like to automate things. And it's like everything, well, this would be perfect . Because we know you love automation. Yeah, that's what those companies do because they don't they love to get information about you and some of that gets used for training, but most of it they want to put up this appearance of we're building a personal assistant for you. And they also want they also want you to ask them another question. Yeah. So they kind of frame these things as like, would you like a suggestion for how this could work for your dogs? Like they want you to ask them more questions. Yes . Apple is taking a different approach. Yeah. So the personal context that was announced two years ago, it's real, it's here. It seems to be working . Yes. There's it goes hand in hand with the new semantic index , which is what Spotlight gathers on device and organizes and keeps up to date . I believe they mentioned during the SOTU that they're using a local rag search. There was a slide up. I was expecting there to be some kind of rag to search that because that some parts of that local on device spotlight index, they get searched by the on device model, right? So for example, when you want to find an I message that Chris sent you months ago about a doctor Pepper with American Dr. Atrocities Zero Sugar with cream soda. Yes, they're not a sponsor of the show. Seriously, Dr. Pepper, we will allow it. And then what happens is that the on device model runs a local search on spotlight, filters by iMessage, filters by Chris, filters by Dr. Pepper and says, Okay, here are the messages . Only then those messages get sent if needed to the cloud, right? So they have this on device index . They seem to be supporting all the things that they mentioned two years ago . All the domains, all the schemas, calendar stuff, emails, third party apps. App intents. Obviously s now what they called? Action app actions. Well, app intents is that a different thing. They're calling them actions . Yeah. Actions are an app intent can provide both an action and something else. So is app actions the new app intents or I don't work at Apple. Okay, I don't know. You went to the Tech Talk, I didn't. They didn't talk about that. That was I think maybe Actions is the more consumer friendly . I think that's exactly sound more of the marketing term, but you don't wanna say app intense in like what does it intend? Intent, what? It almost sounds threatening. What is the intent life on you? It's like, you know, actions fits where like, you know, how messages it can show you a couple of little buttons. Yes, right, right . This episode of Abstories is brought to you by the Revenue Cat Shipatun. The Shipaton is the world's largest mobile hackathon and it's all about shipping real apps to real app stores. The rules are simple. Ship a new app to the app store or Google Play between august first and september thirtieth and use the Revenue Cat SDK for in app purchases, subscriptions, or ads. There are prizes across a ton of categories, including four hundred thousand dollars in cash and that pool keeps growing as more sponsors come on board. Winners can get their app on a billboard in New York City, a shippy trophy, and a trip to New York City for the Shippies Award and the Revenue Cat App Growth Annual Event , with even more prizes still unannounced. Every participant gets a ship kit full of discounts and credits and there are ship attun IRL events hosted all over the world by community organizers. Applications are open if you'd like to host one and there's a track for university students and teachers too. If you've been sitting on an app idea, this is the push to finally build it. One past Shipponat winner, hearing buddy , which took home the peace prize, was just nominated for an Apple Design Award . Pre register and find all the details at Shipaton . com that's hippa t ot com . Join the mobile app hackathon that's all about shipping. Our thanks to RevenueCat Shipaton for their support of the show . I think we should talk a little bit more about some more of the actual features of the Sir AI app because I mean, there's a lot of pieces to this, right? Because it's showing up in all different places in the system . For instance , as you talked about before , you pull down from the dynamic island and you can ask a very brief question and get your answer. But then you can continue to pull swipe down . And then that's when you can do the follow ups and things like that. And which I thought was really interesting. There's elements . They never mentioned the word chatbot once. No, that's a good thing. It's always conversations . Yeah. And there's . And also there are different ways to interact with it on different platforms. For instance, I was impressed to see that it's been built into the Apple Watch because that's not something you can do with other chatbots really. It's probably going to the phone, right? Well, sure. Yeah. But it's at least an input method, which you can't do with a lot of there's no such thing as a clawed Apple Watch . No, for instance. There's been a lot of Applees W catchut off from Watch OS twenty seven for the first ultra is gone and series nine and below is gone. Oh yeah, it's the blood bath when it comes to the Apple Watch. Interesting. I didn't see that. Yeah, and the M one iPad Pro is no longer supported as well. Or the the M one iPad Pro twenty two . Wow. No , Max or M three and I are . Yeah . Wow. Yeah. Well, that's what the models. That's where that's what it does. Yeah, yeah. No, no, no, no. Right. That's the good point. The watch stuff was for the OS . Yeah, that's that is quite so keen to know the download size for the twenty billion model on device. That's a chunky model I bet. It is chunky, I bet. What do you think of the Vision OS little globe that I thought I thought it was really cute. I thought it was cool to look at it and just ask a question. Brilliant. I want a physical product that is actually a globe that is a homepoint. It's like a little lighter. Like an atom snow gl obal series get really confused. Well , I'm looking in the real one and then which one am I talking? You just find yourself staring at it as a home and asking it questions. We got a lot of demos in the Siri app for showing you Siri taking actions in apps and showing you not just the output of the action, but also a visual preview of the app . Like a reminder showing up . A reminder showing up a note from the notes app showing kind of event. They showed taking a picture of a schedule and having all the calendar events at one time could be added. Like if you were at a conference like WW remind me ed of there was a version of Syria when it was still linen where it used to do that. Yes. It would put a little UI in, I think was like the first time that it was like what was the phrase they had like you could you could have certain types of apps . Right. Like did this rid share stuff like that? Reminders I don't have a name but I don't remember Syrika No but there was like there was a list of things you could do . And there was like a very short list. Very narrow domains, like ridesharing, calendar. It doesn't matter. I kind of remember it. So the UI of your app inside. You ruined it. For Minaqi, that's all I need to think about. Well, we're just going to have to dismiss you from the show then. It's some almost right. Go sit in the green room. I have to route you, I'm sorry. No, no, it's fine. What I think is really impressive here is this is the kind of chat bot experience that I wanted Apple to offer . And I think over the past couple of years, all of us collectively, we have gotten used to these other chatbots , right ? And if we wanted to have more integrations with the app that we wanted to use , a lot of us moved to web services and web apps because those were the only ones that were integrated with Cloud or Chad GPT, right? Right. And it has an MCP or API of APIs. And that has an API or MCP, right? And I think what is happening here is something similar in concept , but totally different in execution. Now if this all works and people adopt it. And people adopt it , you can just You can just keep using your native apps just the way you always did . Right. So in a funny way , this Apple and Google collaboration actually helps the Apple lock in effect if you think about it . That's the whole point And I think if it works, and if it works for , especially if you have a lot of apps on the device , I mean, no, you have a problem. I do have a problem. I have like five hundred. Yes. We're not hoarder. But if it works, like can you imagine how nice it'll be to just have a conversation with Siri and be like, okay, I want you to save this in notes and then play some music on shuffle and then send a message to I will probably break it immediately with my five minutes. I'm sure. My index is going to be huge. There is like a moment in WWC when you're in the glow of the announcements. Yeah, right. And we're in that moment right now. Sometimes not always, but sometimes the globe breaks, right? That like you find out actually this thing you thought was going to happen doesn't happen the way you want it to manager. Yeah, sorry. But that is a great one. Or shortcuts originally was one, right? Where it was like, oh, I'll be able to do so much. It's like, yes, but like in this glow right now , I can imagine that I wouldn't need Church PT anymore in my life , right? That like something like Claude, I use cowork. That is a very specific set of tasks that I am doing. Right. And it was producing tools. Complex stuff stuff, pulling in files for my mat. Like that is like a very specific experience. But for me, Chat GPT is like searching very simple stuff, right? Or like what do you think about this or I think I hurt my knee at the gym? Can you help me work out what I could have done, right? Yeah I don't need any particular app for that. Like Chat GBT is the one that I use because it's the one I have used. So it's like I couldn't be bothered to move, right? I know that you guys move more frequently, but for me, it's like I started using that one. I'll just keep using that one. But I feel like my hope right now is I'll just use Siri though. Yeah. I think so. I think that's the big deal because built right in. Yeah. It has a system shortcut. You don't need to open another app. You just hold down the button which is why find very, very intriguing the onscreen awareness part of it. That is a good . ide Ita's very exciting to me to be able to just say like what's this, do this? And just talk to the phone and have it done. Can I share something a little bit sort of a conspiracy, adjacent? Oh, you know, I love it. Yeah. Yes, don't you think it's pretty perfect coincidence that both open AI and anthropic over the past few months , they seem to have suddenly realized that they want to pivot to agents and cloud code and codex and not so much more the chat bot because that 's on the wall, right? Google's already captured Android. Apple's obviously gonna capture IOS . You have to start getting away from it. Why would there's no money in that anyway? Why would I use the regular THAGP chat bot or cl ou chdat bot when I have Siri that is actually smart and integrated with my apps, but also at a time where the general consumer is starting to turn against this more. Yeah, but like so it makes more sense for the pure AI companies to go for the business angle . People are more likely to accept it, it's not the individual making the choice. It makes more sense for them to go in that road, especially when they've clearly lost the ground. And what I think was interesting is that during the tech talk, Craig mentioned, we've seen some of these companies to retrofit the work that they did for their coding agents consumer things. So it was basically saying we have seen Codex from Open AI and people using Cloud Co work, which is based on a coding hardness and trying to sell it as a productivity agent and they didn't say we're working on an agent, but I think also Mike Rockwell said we're aware of these other use cases and this is a flexible system that will scale. I mean, he's at all those things. He said it's built in an aigantic foundation and it can now it can scale the future. Yeah . You know, Federico, before we go, I do think it's worth mentioning X code because one of the things that I know you and I have experienced a lot in recent months is using things like Codex or Cloud Code to build web apps and other tools both for our own use and Mac Stories and some of the things that you've published on the website . And I found myself having some background in building apps and having used Xcode in the past , not using Xcode anymore. I was using terminal and I, was using various CLIs and other tools like Claudcode and Codex with its graphical user interface. And it occurred to me. It's like, well, this is so much nicer in a way for somebody who's doing it for themselves because it's just so much simpler than X code. Apple's really taking X code sort of in that direction, which I'm pretty excited about. I'm really interested to try and see how it works because there's a new view where for instance , one of the things you do with Cloud Code is you'll plan a project , then you'll implement the project, you'll test the project, and you'll do code reviews on the project. It's kind of like a layer cake of steps. And now that's kind of built into Xcode where you'll have that view, which is nothing but a chat window basically, and you'll do a plan and it will walk you through it. And then it'll give you because it's got things like the previews, theift Sw UI previews built in, you'll see what it's going to look like. And you can then prompt against those previews to refine the UI that way and see chunks of code as they're being generated and be able to interact with them individually . And it's all in that kind of similar similar cloud code or codex kind of view, but then for people who are, you know, kind of more maybe comfortable with Xcode and really getting up to their elbows in the actual lines of Swift, they can do that too because it's just another view within that app. And I think it looked a lot more user friendly than anything I've ever seen from Xcodexico in the past. That's cool .. Great. That's great Before we go, yeah, I was just peeking through the social media. Yeah . They found references in the IOS twenty seven code to fold state and angle degrees. There you go. That's not . It doesn't mean anything. . Thank you both. Thank you that. Sorry. And closing thoughts? You know, as a gift to you, I will not make you drink the doctor Pepper. Thank you so much. And I'll keep a lookout for good coffee for you. No promises. Thank you. Yeah, no, I'm excited. I was happy to be with all of you. It's been great. And I'm very keen to see what this bayout looks c likeyc willle look like. Yeah. Well, it's always excellent to get back together with all you guys during this week and everybody else who's not in this room with us and also thank you to Apple for putting this on for us because it's you're not seeing all the hard working people behind the scenes are helping us make this episode possible . And I want to thank our sponsors too. We had albums and controller for Homecat, as well as RevenueCat. And I guess that's it until very soon I'll be talking to you again here in this studio . Talk to later Federica. Tao John . Bye Mike. Bye Loosen for Mike and Chrisp sorry