The Future of Synthetic Data

Hello and welcome to Virt Cast, the flagship podcast of music that sounds yearily but not exactly like other music. I'm your friend, David Peerce, and today on the show, we're talking about training data It's the raw materials of everything that is AI And we probably don't understand it or talk about it enough I'm talking to Alex R Reisner, who's a staff writer at the Atlantic. And over the last couple of years, he has spent a lot of time investigating training data and how All of these books and all of these articles and all of these YouTube videos and all of these songs are compiled into these gigantic data setets. AI companies then use to form the basis of their models The way that these models are created and the sources of these data has a lot to do with the way that we feel about AI, particularly generative AI as a creative expression. and understanding how this data works, where it comes from, and how it gets used think is really important. So we're going have Alex on, we're going to dig into it. I'm very excited about it. But first, here's everything else happening on the verge today This is ninety seconds on the verge for Thursday, june twenty fifth, twenty twenty six Apple just raised the prices of a huge number of its most important products. iPads and MacBooks in particular are now up anywhere from one hundred to five hundred dollars. and even like the Apple TV is way up. It's now two hundred dollars, roughly the price of nine hundred Amazon Fire Ss. We knew this was coming, but this is still a big moment Apple is as well managed a supply chain company as anybody and exists at really high margins. If it can't keep prices down in this era of AI driven shortages of memory and storage Nobody can. And I suspect this is going to get even worse. My advice from yesterday still very much holds, which is Prime Day is not over. G get your deals while you still can'. And here's a way to help you pay for all of those more expensive gadgets. Go get some money from Disney If you have UCuTV or directTV stream, you might be eligible for a cash payout from a new fifty million dollars settlement The case started in twenty twenty two, and the argument was essentially that because Disney owns ESPN and Hulu, which are both powerful streaming services in their own right, Disney was able to drive up the prices of all of its rivals to unnecessary heights. Live TV is lucrative and competitive and is going to keep being messy in this way. This will not end things. But hey, go get some of that fifty million dollars. And finally, a quick PSA to advertisers everywhere Yes, I get it. The whole idea of using AI creative and AI targeting to put your AI ads in front of AI people It is exciting, I guess But maybe check your ads to make sure that you're not, I don't know, advertising a bike with two sets of handlebars like REI did this week that may not go over so well Just a little free advice for me too You can read more about all this at the Vverge dot com dot That's ninety seconds on the Verge for Thursday, june twenty fifth Support for the show comes from service Now AI is moving fast across the enterprise, but without visibility, it's just chaos Different tools, different models, different teams using AI in completely different ways Service now turns that chaos into control At the AI control toer, you see all your AI across the business in one place. What it's doing, what it's done And what it's about to do, so you stay in control To put AI to work for people, visit serviceNow. com We've all been there. You pop into the shop for five minutes and all of a sudden you've forgotten where you parked Car Core Unfortunately, that lost feeling is what it's like trying to manage your policy with other insurers Here C, come out, come out wherever you are. Please. With Gaico, you can use the app to easily manage all your policies in one place. Did this parking lot have a waterfall? I think you've wandered too far, mate. It feels good to find what you're looking for. It feels good to Gaico. All right, let's talk training data. Alex Riser from the Atlantic is here Hi Alex. Hey, David, thank you for having me Very excited to talk to you. I think we have talked a lot on this show about why people feel the way that they feel about AI and the sort of instinctive reactions to the whole idea of AI And a theory that I have is that a lot of it is about training data. And I am I just want to talk about that You've done a lot of work on this stuff and you've been investigating how AI models get trained for a long time now. But I want to just lay a little bit of groundwork here And just to ask the very obvious question upfront, why does it matter data is used to train these products. What difference does it make what's inside of these models I think it is potentially the most important aspect of a model is what it's trained on. I mean, if you a model and you train it on, let's say it generates music and you train it on nineteen fifties jazz That model will be very good at generating music that sounds a lot like nineteen fifties jazz. If you train it on a recent hip hop, it's going to generate music that sounds like recent hip hop C You know, these models have names like Chat GPT and Claud, but I think you could make an argument that The right name for a model is actually the a description of the data it was trained on because that is a description of its capabilities. That's what it can And so I think that the training data is really It's really fundamental to the model, maybe more than the architecture to some degree. Interesting. And I think I mean, that kind of answers to some extent my next question, which is Why is training data such a closely guarded secret from these companies And it seems to me that there is both a Straightforward business answer. and maybe a slightly more nefarious answer as to why these companies would be so secretive What is your read on why this is such a closely guarded secret, what data is being used to train these models? Yeah, I mean, the companies have argued that they need to keep this secret because the data that they have selected to train on is their competitive advantage, right? Like Anthropic has done a better job at selecting data than Google and open AI. and if they were to let that come out in a court case or be public in some way, they would lose their competitive advantage Um you know, there's another pretty obvious reason, which is that they have gone about acquiring a lot of this data in ways that the people who've created the data, the authors of the books and the creators of the videos and the music would not be happy about. in a lot of cases, they just don't. know that the they're their work is being used and when they find out, they're not happy about it. And I think it's a conversation that the AI companies try to just avoid having A big part of what you've been up to recently has been sort of reverse engineering this process to like peel the model apart to figure out what data is inside of it. And it seems to me that you've had to do sort of the reverse of what all of these companies have done, which is go figure out where these mass sources of books or web articles or videos or songs R. So Tell me a little bit about your process. How do you go about finding these things that are kind of otherwise closely held secrets Yeah, My process is actually maybe not that different from the company's process I am a programmer by by I worked in tech for twenty years. I you know built websites and apps and did some statistics work. and so I've been aware of these models for a long time. and at a certain point I started hanging out in the forums where AI developers were hanging out talking about the work and You know, they're talking about what data they're using to train these things And so that was a useful source. And I've also been reading a lot of their research papers you know, selecting the data is reallyally challenging. There's a lot of effort that they put into it. They have to come up with some notion of what is high quality data, like what do and don't we want in the model and They write papers about that. you know, I think they want to be involved in this conversation that's happening about training data within the industry. One thing that's been really helpful to me is that there's an open source AI development community, and they believe that the work should be done more out in the open. And I think a lot of them are doing really good and important work and they have really interesting things to say about AI just philosophically and socially. And so they they're pretty transparent, pllaces like Allen, AI and E Luther about what they're using to train. And so that's been helpful. But then even at the companies, AI AI world is a little bit like academia in that it is good for your career to publish papers. And the companies don't like that, but they also acknowledge that they have to let the employees publish something And so the lawyers will go over it and tell them what they can and can't say And over time, they've clamped down more. And so it' compomanies are revealing less through the research papers, but yeah, a lot of my research is just from reading a Google paper, you know, for example, for this last article where they said we trained on you know tens of millions of songs Yeah, I remember not that long ago going through a big cultural issue with this with its AI team because Apple is so secretive and so reluctant to let any of its work be public that all of its researchers were like, well, if you're not gonna let plish, we don't want to work here And it did that that push and pull seems like it has It has kind of morphed in a bunch of different directions over time. but Interesting to hear that it is definitely it is Everybody's retrenching a little bit As this space gets hotter and hotter. Yeah, it's not twenty twenty one anymore. The research papers read very differently now than they did a few years ago. That's really interesting. But tell me a little bit about these databases. One thing I think I had not given enough thought to until I started really reading your work is There's a real business in making and maintaining these databases Like one company you wrote about common crawl I think is like a aoroughly fascinating player in this and they they essentially, as far as I can tell for all the internet and make it available to whoever wants it, which is weird and complicated, and we should probably come back and talk about Chrome and Crawl sometimes. but It seems like if you look around a little, these gigantic databases of books and articles and videos exist. Like are people making these and selling them? Why do these databases exist? mainly, well, I mean, for AI training, why else would exist? That's a it's a huge, you know, it's an extremely labor intensive process to find, you know In one sense, they're just you just go and download all of Library Genesis or all Vana's arrchive. That's one sort of naive thing you can do, but the companies realized pretty early on that they need to filter the stuff pretty carefully. So the organization you just mentioned, C and crawl Yeah, they've been crawling the web since the late two thousands, maybe two thousand nine or something like that And they just make the whole thing available. Every month. There's a new they've scraped a few more one hundred million web pages. and it's available to anyone who wants to do any kind of research with it. In fact, it's mostly AI researchers who are using it But all the early Large language models were trained on common and crawl. If you go back and read those early open AI papers Everyone was training on Cic crawl and at first and the models were terrible. Because if you train a model on the whole internet, it's just, you know,' you get all the it says all the junk that people say on the internet along along with the intelligent things. Yeah. mostostly junk. Yeah. And so it's statistically mostly junk. Statistically Yeah, mostlyk. And I think the early large writers' models were proof of that Aome But yeah, I think they they're, you know, Con Crawl is a nonprofit. so They would argue it's not a big business. They do get a lot of money from AI companies and AI investors U But yeah, I think this The topic of training data selection, the challenge of selecting the right data for a model. is still really hard. The AA companies, I would say, are still have a very primitive understanding of Data will make their model better. It's an area of research that I think even even they at this stage are not very good at U they they do it mainly by trial and error as far as I can as far as I can tell. So interestnteresting. Again, so the reason these you're asking, why do these data sets exist? I think it's people trying to share what they've learned from, you know, curating dataets in different ways and training yalls with them Yeah, I mean, part of the reason I ask is I think one of the things I have come to believe about the AI industry is that This shift that went from AI research being fundamentally a research thing. Like if you if you go way back Op AI was basasically a research organization, right And you talk about Concrawl and I think its early users were largely researchers. And it these things were academic things for academic purposes and From what I understand, the kind of rules of the road are different, right? that like if you want to make copies of a bunch of things for academic purposes, these things are generally considered less problematic, right? But if you then do it and become a trillion dollar company on the back of it, people are going to rightly feel differently about the way that you went about getting that information. And just the speed with which AI commercialized All of these companies just moved so fast from we are essentially an academic thing to oh my God, we're making so much money. Everyone's filthy rich that it feels like they just hoped everybody would ignore the ways in which they got this information. And so that's why I'm particularly fascinated by where these data sets even come from. because it does seem like in many cases, like you're saying It's not that there is some gigantic business in being the one to sell the songs to somebody. It's that stuff being this stuff is being compiled for other purposes It's just that now the main purpose because of the sheer volume of work. is AI research. All this stuff has been sort of co opted from every other purpose. to AI And it just all feels so concentrated now Do that Do that feel right to you? Does that make sense? Yeah, that I think that I agree with most of that I do think that I'm not sure how much these datasets were really collected for other purposes Comic Praul likes to talk about, I think they're a one case. They probably have the strongest argument that their data could be used for other purposes. But when you go back, you know, there's they've been cited by over ten thousand papers. I didn't read all ten thousand but I read a lot of them and they're mostly AI Right And it's early. a lot of it is stuff that peoplen' wouldn't mind as much as with generative AI, right? Common Crawl, I think without Con crawl know, AI translation tools might not be as good as they are. I think it was really a huge hell because they scraped web pages the same page in multiple languages and people were able to train translation models based on that. So That was helpful. but thing that You know, there is still a what I would call a data laundering network whereere the AI companies are still relying on they'll do a collaboration with the university And they'll have un the university download, you know millions of images to train a model or download millions of articles to train a model And the company can say like, well, we didn't do it. This was like an academic thing You know, the same goes, Common Crawl is not the only nonprofit that's like doing a lot of the scraping for the AI industry. One of the data set I reported on h and the music, the article about music. training data is this organization based in Europe called Lion. They have a data set of twelve million songs from YouTube Anyway, this is like, is it academic? like not Not really, like this is, you know, technically yeah, those are universities and nonprofits, but They're all receiving money from the AI industry When I scraped my car in that parking garage, I was worried that it could be a long process to take care of it L like a landscaper's first day trimming a hedge made. I have definitely already been here Now is it left, right or right left mayaybe I'll cut a path out and find my way back later. It wasn't like that. I filed a claim in under two minutes on the GaIiCo app, and they handled it from there. It was taken care of almost as quickly as it helped. It feels good to get help quick. It feels good to GaICo. Support for this show comes from Vettch pet inssurance. Do you have a pet? Every six seconds, a pet owner in the US gets hit with a vet bill of over a thousand dollars And it's almost always an unwelcome surprise That's where Fetch pet insurance comes in. Fetch is the most complete pet insurance. Get paid back up to ninety percent of vet bills. You can use any vet in the US and Canada. All vets are in network. Go to fetchpet dot com slash save right now for your free quote. That's fetchpet dot com slash save Avoiding your unfinished home projects because you're not sure where to start Thumbtack knows homes, so you don't have to Don't know the difference between matte paint finish and satin orr what that clunking sound from your dryer is With Thumbtack, you don't have to be a home pro You just have to hire one You can hire top rated pros, see price estimates, and read reviews all on the app Download today Heat up your fourourth of July at the Home Depot with our wide variety of grills under hundred dollars and make every gathering one to remember. Give your outdoor space a glow up. Whatever your budget is, the savings on seasonal plants starting at five dollars. With the grill fired up and your backyard set to perfection, you'll be able to invite friends and family over to kick off the party Start celebrating with low prices guaranteed at the Home Depot. Prices may vary h stor ex as to a price see H home Depot com sl priceash for details This is kind of a diversion, but this I'm so struck in reading all of your work by how often YouTube appears as just It is it is everybody's favorite source for everything. And like It's what, you know, it's what OAI used allegedly to create whisper. It's what a lot of the music stuff is using. It's what there's a lot of stuff based on videos like What is your sense of YouTube's role as an AI training force because it seems to be everywhere. Yeah, that that's accurate YouTube is an extremely common source I think one reason is there are tools for downloading from YouTube that are really that work really well. They're really easy to use And it's pretty common for AI developers to just use those tools. and it's just kind of a become acc custom But also, you know, and that's That includes If stuff on YouTube is just less protected, I think is one way of saying it. Like there's music, you know, if you're a musician, your song might be on Spotify But Spotify's website has digital rights management protections. It's really hard to download from Spotify It's much easier to get the same song from YouTube and so many songs are alsoso on YouTube. So I think it's just ease of downloading YouTube. obviously would say out loud that this is not allowed, right? That it violates assums of service. And yet It seems to have either either it can't or it just hasn't done anything to stop this, really Yeah That's that's a question that's been in the back of my head for a long time. and which I've asked YouTube and which they they they don't really answer. they did, they have said that they considerered a violation of their terms of service to be downloading their videos. But yeah, they haven't, you know, years have passed. and is It's just as easy to download from YouTube now as it was a few years ago. Yeah, I downloaded a YouTube video this morning. It is just a thing you can do. It's really true T You mentioned the sort of data laundering stuff, but it also seems to me that more and more people Do the AI training are just completely unapologetic about it likeike you quoted Rich Srrenta, the CEO of Common Cirl who literally said to you, like if you don't let AI robots crawl your data, you essentially don't exist. like you're going to miss out on the future of the interternet. And I think about, you know, Mark Andreistin even a couple of years ago being like, none of this would work if we couldn't just take the training data that we needed. Mh There is this almost like manifest destiny sense of the AI industry that we can have this data because the thing that we're doing is so important that we must have it no matter what time What is your sense of the trend there? Because it occurs to me that that's happening even as the backlash from people hate the experience and feeling of AI in part because of the way that this stuff is trained just keeps getting worse. L These things are just running away from each other at like record speeds Yeah I That's a huge question. I think it You know, I think it has something to do with the fact that we just have not doneone a great job. in this country with establishing the value of and who should be able to have data, right? This is something that privacy advocates have talked about Jaron Lineier, I think was one of the earliest people to be talking about this. I think he wrote in two thousand seven about that you should get paid like you're being surveilled basically and companies are taking They're monitoring everything you do. They're generating data from your online activity. That data is extremely valuable to them might seem like nothing to you, but you should be getting paid for that People thought that was crazy back then, and I think a lot of people still think that's crazy now But you know, taking people's music to build models that generate songs that compete with them is just the next version of that. and it's just going to keep going. if we don't acknowledge that this data is incredibly valuable and figure out a way to write laws around that or just have you know better business practices or something Um this is just, it's just going to get worse. There's just going to be more exploitation. more mining The next step I think a lot of people have perceived for a while to be synthetic data, right? that eventually we're going to get AI models that are so good at making new things that then we can use those things to train new models and eventually they don't they don't need existing recorded music, that synthetic data is the future and that's how we get to everything. And especially now, I mean you look at Spotify and there are tons of AI generated songs on Spotify that some people are listening to There are a billion AI generated podcasts out there. likeike the content is being made Is that the next phase of training data is synthetic data coming into its own in such a way that we're going to start to see the next generations of these models built on the things made by the last generations of these models Absolutely not don't I don't think that there's any evidence that that actually works I think when AI companies talk about training on synthetic data, they choose their words very carefully and they always exaggerate the extent to which is happening. There's a ofap there's a lot of research out there on a phenomenon called model collapse which is what happens when you train a model on its own its own outputs. It's very quickly It doesn't get better. It very quickly degrades And it's not hard to see why that case, right? Like AI is kind of an averaging machine Right? It's defining statistical average between different types of content putting that into some new new kind of more average type of content and there's just not enough weirdness or interestingness or something like that, there's some quality in the work that humans do, that's not in the work that AI does. and I think that that's actually proved by the model collapse. phenomenon. So I'm amazed that AI companies are going around talking about synthetic data still when there's so much evidence that doesn it doesn't work So is there a next untapped place filled with data. I mean, these models keep getting bigger, they keep needing more data They got to go somewhere, right? there Is there a next sort of unturned place to go for these companies I think they justay pay people to make it. I think there's already a gigantic industry of writers who are writing for AI, musicians who are u making music for AI, you know after I published this story, I got a an email from a company that is doing this. They claim to have paid creators over ten million dollars just to make things for AI It's very strange, but this is this is the next like AI as your audience is the next frontier for as a deeply strange thing to think about, but also like, you, we're gonna put this on YouTube and it's going in there anyway. so who knows? Maybe this is all of our destinies no matter what I certainly hope not, and I don't think so. I think we can have a conversation and arrive at a more reasonable future for ourselves and the culture I'm with you on that All right, well Alex, thank you so much for being here. Re really appreciate it. Thank you David. That's great. All right, that's it for the show. Thank you to Alex for being here and thank you as always for watching and listening. If you have thoughts, questions, feedback of any kind, if you have a favorite AI song you want to send me It's not the Puerto Rico song. I already know that one. Send me all your favorite AI songs. No judgment. I just want to hear all of them