Comparing AI Answers to Reliable Sources

John Lister Jason wrote in to say not me said recently a colleague asked me how I knew a piece of information. I wanted to say that I looked it up, but I hesitated because I actually had asked ChatGPT. Did I look it up if I asked a chatbot? Would I have looked it up if I had typed the same thing into a search engine instead? Uh you did not look it up when you asked the chatbot, unfortunately. Uh search engine maybe, but unfortunately, lots of sort of AI summarization results are are near the top. I would say he could say he looked it up if he followed ChatGPT's links to a place. It doesn't always provide them, but if the information you got was from the link, you looked it up. Now you may have looked it up on a uh in a source that is bad. Like say you had a set of encyclopedias from the 60s and they had incorrect information . Yeah, tell me about Leningrad and Rhodesia. It's either out of date or or it was just wrong in that edition of the encyclopedia. Right. It's not the correctness of the information that means you looked it up. It means that you found a source where this information was placed and that's where you looked it up. And it could be wrong, it could be right, but you looked it up. But if when we take all that stuff and put it into a big stew and do a statistical uh probability uh you know next word algorithm thing on it, that's not looking it up. That's just, you know, that's just a a a word soup that you got out. Uh it may be useful and it may be right and it may be wrong but that's not looking it up. Yeah. Your I would say your colleague is asking you to cite your sources. Yeah. Where did you look this up? Because if you just say uh put it this way, if you had uh turned to your buddy next to you and asked them, and they told you you would not say you looked it up, and that's very equivalent to asking Chat GPT. You asked a thing and it gave you an answer, but you didn't look it up. You didn't unless your friend is like the world's premier authority on whatever thing that you asked them. But looking it up means you found it in a in a source that was created that to contain it in the right. All you did was ask Chat Chat GPT and get an answer. You have to say, I got it from ChatGPT. You can't say I looked it up. In which case they just say, Well, I don't care. Try again. Yeah, I don't believe you. Because it's probably wrong. Like there is, as we're recording this, there was an ad campaign for a smartphone that has AI in it that involves a woman asking the smartphone for recommendations about flowers to add to an order of flowers that she's gonna buy and which at which I shout, um, don't believe it. And then will these last until Saturday? And it responds confidently, yeah, these should last till Saturday. And I think, don't believe it . Don't don't Don't believe it. Maybe it's right. You're selling the flowers by asking asking your friend, hey, if I buy flowers now, do you think they'll stay alive and your friends say, Yeah, they probably will. Yeah. Yeah. I would believe that more than I would believe the the AI. So yeah. I love the c confidence of like, yeah, they'll la Yeah, they'll they'll probably last till Saturday. I'm like, d no, no . No, definitely not. Sorry, uh sorry there, listener Jason. You didn't look it up. Rob or not