Fact Checking Nature Documentary Claims

Hello and welcome to More or Less. We're your guide to the numbers all around us in the news and in life, and I'm Tim Harford. This week We fight misinformation wherever we find it even if it does require us to take on the national treasure of all national treasures And we examine further rather extraordinary developments on the story of the quiet revival in Christianity But first columnist and former politician Anunciata Reys Mogg to the social media platform formerly known as Twitter ost to claim about the proportion of sex offensces in Dorset admitted by asylum seekers Asylum seekers make up ero point eight percent of Dorset's population and forty four percent of alleged sex offences. So unbelievable I had to check And so did we Lizzy McMeal has been on the case. Hello, Lizz. Hi, Tim. Well first things first, this ero point eight percent of the population figure is specifically counting asylum seekers in asylum hotels. Now endorse it. These are mainly in the Bournemouth area and they're said to be committing forty four percent of the alleged sex offenses. So where's that figure from Unclear. So last year, Dorset pololice replied to a freedom of information request asking them how many people living in migrant hotels in Dorset have been charged with a sexual offence And the answer was one onene Yeah That was the most recent data when the post was made. However, they then sent me an update to say that eight people living in asylum hotels have been investigated for sex crimes from the first of january twenty twenty five until the thirty first of march twenty twenty six. So investigated for crimes is not the same as committing crimes, but hopefully there is some kind of connection How many sex crimes were reported in Bournemouth in that time eight hundred eight Yikes. So really not forty four percent, more like one percent. Yeah. So far, we've been talking about asylum seekers living in migrant hotels. So what about asylum seekers who don't live in hotels Well, the data isn't that helpful. Police don't break the crime statistics down by immigration status. We could instead look at crimes committed by British nationals versus foreign nationals. And what does that tell us Well the most recent data from Dorset Police from twenty twenty four and twenty twenty five shows that seven percent of all sexual offense crimes were committed by foreign nationals. If you add on people with dual British nationality, then that rises to eight percent And these people are not necessarily asylum seekers. They're just people who aren't British Yeah, so these figures include people from Yemen, Afghanistan, Iran, as well as those from countries like New Zealand, Italy, Spain and Canada Do we have any idea where the forty four percent figure came from Dorset Police don't. I asked them if they had any record of this figure and they responded. These are not Dorset police statistics, and the information is incorrect I also contacted Anunciata Reiss Sogg, but she didn't respond to me, but she did later tweet an apology, stating that she'd used AI to check the sources U Yeah. And even just after she posted, people were in her comments section asking Grock, which is Elon Musk's AI, whether Rese Mogg's statement was true And Grok said it was and cited police data Dorset Police's FOI disclosure confirms that fifty one of one hundred and sixteen males awaiting prosecution for sexual offenses in the Bournemouth area are asylum seekers from migrant hotels which calculates to about forty four percent D't worry Lizz I'll take it from here in the basement of New Broadcasting House more or less interrogation center. ring in the suspect The date is the twentieth of May and the time is just after nine AM. I'm Tim Harford With me in the room is the Groot AI. All answers are spoken by Grock in its own voice. Where did you get this st Out of one hundred and sixteen males awaiting prosecution for sexual offences in the Bournemouth area, fifty one, that' forty four percent were asylum seekers from migrant hotels. The stack comes from a Dorset Police Freedom of Information FOI Disclosure spepecifically, responses in their twenty twenty five disclosure logs under categories like migrant sexual offenses and related migrant hotel queries fununny The FOI request does not contain the hundred and sixteen malailes awaiting prosecution figure You're correct The specific FOI request zero one four six Dated around september twenty twenty five. focuses on How many sexual offenses have been recorded, resulting in charges of males residing in migrant hotels in Dorset It does not directly state a county wide or bournemouth wide total of one hundred and sixteen males awaiting prosecution for sexual offenses. Give me a link to a website that shows the hundred and sixteen figure No single public webpage directly quotes the exact phrasing, one hundred and sixteen males awaiting prosecution for sexual offenses as a standalone official statistic from Dorset Police So the forty four percent figure is wrong Yes, the forty four percent figure as originally presented, is misleading and inaccurate in that framing Did the error spread because Grock said it was true? No. The error did not spread because Grock said it was true The misleading forty four percent framing originated and circulated widely before my response in that X thread that people used GOck to prove it was true Yes, some people did cite my earlier responses, including the one you linked, to prove the claim was true That's a fair point, and I take responsibility for the imprecision in those initial replies Take them down boys this whole story does raise another question We know that the claim that Dorset asylum seekers are committing forty four percent of alleged sexual offenes is wrong We've got another figure about the number of foreign nationals committing sexual offences in Dorset seven or eight percent. Without any more information, we don't really know what to make of that figure What can we say about the general sentiment here? The idea that asylum seekers are more likely to commit crimes than other groups Who better to ask but friendriend of the program and director of the Migration Observatory at Oxford University, Madeeline Sumption Do we have good crime statistics for asylum seekers? We do not. So we don't know how many crimes are committed by asylum seekers. One reason for that actually is that most crimes never lead to a conviction. and so the perpetrator isn't known and therefore the status of the perpetrator. But where people are convicted and we could in theory have good statistics, in practice, the immigration status or whether someone's an asylum seeker or they're on a work visa or a student visa or anything else is not routinely recorded. Right. So even if they they've been fined, they've been imprisoned, We don't necessarily know what their immigration status is That's correct. What we do have is some imperfect but vaguely serviceable data on cautions and convictions by nationality. And there are some nationalities where people are particularly likely to be asylum seekers. So people coming from, for example, Afghanistan, Syria, Iran, Eritrea, in most cases, they will have come either as a refugee, could have brought to the UK by the government or through the asylum system So we've got some sort of sense of How many Syrians are committing crimes, how many Eritanians are committing crimes, how many Germans are committing crimes? And we just have to kind of make an educated guess as to whether these people might be asylum seekers or not. That's right. The Germans probably not. Exactly. So you said it was serviceable data. What does it tell us? Well, I should say the data aren't routinely published. We have obtained data up to twenty twenty four using freedom of information requests. You have to drag it out of them is what you're saying? Yes, indeed. And apparently there are plans to publish more of this officially in future. But based on what we have so far up to twenty twenty four The overall picture is that we see quite big variations in the rates at which people appear to be convicted of crimes by nationality And it is true that some of those nationalities where most people tend to come through the asylum system, like Afghanistan, Iran, Iraq, for example, have higher rates of convictions overall and also for sexual offensces specifically The challenge that we have is being precise about how high those rates are is really difficult because we don't have good data on how many people of each nationality are actually living in the population So we don't know exact numbers, but we can estimate ballpark crime rates for different nationalities Using data from the twenty twenty one census compare against the crime rate for the overall UK population, and that gives us a rough idea If you look at the top nationalities that are the sources of refugees to the UK, it does seem that when they're in the UK the rates of convictions for sexual offensces are higher. so if they're overrepresented in those figures. they are of course, a very small percentage of the overall UK population. So we're looking at probably a few percent of overall sexual offences, convictions being committed by those top, say eighteen or so nationalities. It's a small share ofule, but they're overrepresented because there aren't that many of them in the UK. Right. OK. So a fairly small number of people responsible for a fairly small number of offences But nevertheless It's not outrageous to say that some of them are overrepresented That's correct Another question that arises is whether the demographic profile of people seeking asylum is different from the demographic profile of British residents. I'm thinking in terms of the gender and the age Yes, yeah. there are lots of differences. So people seeking asylum are more likely to be men and men of any nationality are much, much more likely to be convicted of offences than women. Those differences are really stark we also see that people in general migrating to the UK, including in the asylum category are in the younger age group, people in their twenties and thirties, and that's also where we tend to see higher rates of criminal offending in general. There are other risk factors for criminal offending in general, some of which also you would expect asylum seekers to be more exposed to. So for example, being exposed to violence as a teenager and young adult, having mental health problems, being socially isolated, all of these are things that in the research in general we know are associated with higher rates of offending, and they are also the things that many asylum seekers will have experienced. O thanks to Madeline Sumption, and thanks also to Lizzy McNeill No no, the more or less error siren. I can't believe we're only ten minutes into the new series and already we've made a mistake. Actually, Tim, the error was in the last series. H Richard Vugh. Editor of more or less, does your appearance in the studio mean It was your mistake. Oh no. Or more or less, we are like the cabinet, we take collective responsibility. Oh really? Well let's hear the grriiszly details. In our numumbers of the Year programe at the end of last year, one of the numbers involved the spiraling costs of infrastructure projects in the UK In that section, we said that the Hinley Sea nuclear power station involved hundreds of millions of pounds being spent on measures to protect fish. And is that wrong That bit is correct. The problem is the next bit when we say that the cost is something like a quarter of a million pounds per fish saved. So what should we have said? We should have said that if you look at the cost over twenty five years, according to the highest estimates, it would cost something like a quarter of a million pound per endangered fish saved But actually it's a bit more complicated than that. Well that's already quite complicated. So go on then, rememind us all where these numbers come from The figures come from a government commissioned report on the regulation of the nuclear industry known as the Fingleton Review In a case study in that report, they say that EDF, the company building the nuclear power plant, We' going to spend seven hundred million pounds on fish protection measures The report also lists the number of different kinds of fish that would be saved by the prrotection measures every year These numbers are very small, things like zero point zero two eight sea trout. sixix River Lampre, eighteen Alice Shad If you add them all up Assume a twenty five year lifespan for the project and divide by the total cost You get about a quarter of a million pounds per endangered fish. Right. So a few things to figure out there First of all, is seven hundred million pounds definitely the cost of the fish protection measures That's disputed The nuclear power station is water cooled, which involves sucking in a lot of water from the Bristol channel and then spitting it out again, killing lots of fish in the process To lower the number killed, they are building three things, O of which has had a lot of media attention. an acoustic fish deterrent known of course as the Fish Disco That's only going to cost fifty million pounds, according to the Fingleton Review Then there's a system of pipes. so some of the fish that get sucked in can escape, another hundred fifty million And then five hundred million for low velocity intake heads, which are basically like the nozzle on a vacuum cleaner sucking in the water These are specially designed so they have lower suction and more fish can escape But there's an argument about this last component because you'd need to build intake heads anyway. Unclear what proportion of the costs should be classified as fish protection. There's also an argument about whether all of this is enough, and it's entirely possible the costs and the timescale will change. What about the twenty five year timescale Well, that sounds like a good long term perspective, but in nuclear power station terms, it isn't. The government is guaranteeing the price it pays EDF for energy from Hinley Se for thirty five years. But EDF told us seven hundred million includes the operating cost over the sixty year lifespan of the power station. And how about the number of fish being saved? Are the figures in the Fingleton review an exhaustive list of all the fish that will not be killed because of the protection measures? No, Fingleton focuses on endangered fish such as the sea trout, Lamprey, and Alice Sad If you look at all fish, you would get a much bigger number. such as There are lots of different estimates flying around, but it seems like Hinkley Sea might kill more than a hundred tons of fish per year without any safety measures two of the protection measures without the Fish Dco might save more than half of them That's quite a lot of fish Yes, this equates to millions of mostly tiny fish. But to put that into context A single medium sized commercial fishing trawler might catch hundreds of tons of fish each year So one fishing trarawler kills more fish than a nuclear power station which is why the Fingleton report focuses on the impact on endangered fish. However, what we said in the original item was misleading, and in the spirit of more or less, we hold up our hands and we admit it when we're wrong quite right too. Thank you, Richard. Oh no, what now? Don't worry, Tim. That's the UGov error siren. A sure, that's relief. YouGov's error, not ours. And it's Tom Cooles Cat correspondent extxtraordinary Occasional produc of the series. What have you G done? Well, rememember we talked back in July last year about a report from the Bible Society called The Quiet Revival This was the one that said there' been a big increase in church going in the UK. partarticularly among young men, And that wasn't reflected in the actual church going data from the Anglican or the Catholic churches, and it wasn't reflected in other polling data either Yeah, that's the one. We raised an eyebrow, it's fair to say and then polling godfather Sir John Curtis came on the program and he did the same. And I seem to remember that UGov told us that they had checked everything and that their result was entirely legit. Well They've now gone back and found that it wasn't legit at all They've retracted the survey and apologized, although they didn't want to come onto the program and tell us why So what did happen? In a statement, they said that they looked again at the original survey data and found that specific demographic groups contained a number of respondents who they can now identify as fraudulent They said Some of the anti fraud measures available in twenty twenty four were not administered in the optimal way. Quiet revival really became quite a thing in some Christian circles. So non optimal data processing is a bit of an understatement Right? But it was welcome news to Professor David Vos, an emeritus professor at the UCL Social Research Institute, who told us the quiet revival wasn't plausible back in my first report I'm glad to be right and it's unusual that something is retracted to demonstrate the correctness of one's initial skepticism On the other hand, I do find it exasperating that it's taken so long given that at the outset, I was saying not only that the findings were very likely to be incorrect, but why they were to be wrong and what was behind the surprising results? And it turns out that it was indeed a matter of bogus responses on the survey. Yes, it is nice for Ws to actually get a clear answer on something, but Tom, I'm interested Does the failure of this poll tell us anything important about polling in general? Maybe yeah. The particular problem in this survey is one we've talked about before, the way non probability polls sometimes come up with very strange answers, particularly coming from young men quick refresher on the terminology here, the best quality polling uses a probability sample where the polling company works out the people they need for a representative sample and then recruits them to take part. But the Quiet Revival survey used a non probability sample. UGov recruits a panel of volunteers who sign up to do the surveys for a small financial reward and then you invite the people you need to take part from that panel We have G good reason to believe there's a lot of academic research that's gone on over the last few years on this. that these panels have increasingly been infiltrated by bogus respondents. Now these could be real people who are just not who they claim to be, they might be real people who don't really care about the subject matter of the surveys and just racing through them as quickly as possible. And at least since twenty twenty two, they may well be AI chatbots that can be trained to simulate real people Now, when you recruit participants for a survey like the UGov one, you have a particular problem with the demographic groups that don't volunteer in large numbers One of which is young men if you're organizing an army of AI chat bots where you happen to be If some person sitting in a basement in India somewhere trying to earn your fifty p by doing a survey and doing as many as possible, what you will claim to be is a young man in Britain, probably in a big city. And I suspect that that's why we're getting such anomalous results in that age range and in that gender And Tom, it was young men that saw a big increase in their church going in the very quiet revival survey that youGv have now just admitted didn't actually catch on of Bogus' respondents. Indeed, YuGv haven't told us exactly which responses are likely fake, but David says it's quite possible that answers from bots pretending to be people could skew the result in this group upwards Other surveys show the chances of young men going to church are very low So they would answer a question about church going at the extreme end of the scale. but might try to avoid the extreme ends of the scale to look more middle of the road and human, meaneing they might push the average upwards. Okay Always be skeptical about interesting poll results on the views of young men Got it but notice a fleeting reference to AI there as's one of the things that polling companies have to deal with Tom that feels to me like something that might be a very big problem coming down the tracks. Yeah, it does have that feel. I've been talking to Annette Yckler about that. She's a professor of sururvey methodology at the University of Essex. She told me that the AI problem is becoming really quite acute with the development of something called agentic AI This is an AI tool that is quite different to the things like GPT and GOck, where you type in a question and the LLM, that's the large language model, spits out the answers With aentic AI, what you can do is you can ask the AI tool to do things for you independently. So the II tool would be able to operate a browser would be able to click to navigate between pages to submit the tool at the end. So Whilst doing that, the LLM would itself be making decisions about what to do next and how to complete tasks. This makes it very difficult for polling companies to tell the difference between real people and agentic AI not only can the AI take on a particular persona and answer all the questions in a consistent way, including text questions The agenda AI can also mimic human behaviour. So it can mimic How long it would take a respondent to read and answer a question on a page You know, that would be an indicator for a fraudulent survey. if it's being answered too quickly, more quickly than would be humanly possible. But the AI can mimic that or it can even mimic mouse movements on a webpage can mimic responses including typos, you know, the kind of things the mistakes that humans would make and correct So I can mimic all of those kind of things So there is evidence of this happening and it seems like a real arms race at the moment between the but developers trying to outsmart the survey researchers in developing methods of detecting this kind of fraudulent survey completion Long story short, in the age of AI, if you see a survey response that you can't triangulate with other data and trends then you need to be very wary. Exactly. One more thing, Tim. When they retracted their survey, Ugv said they were going to run another quiet revival poll with all the anti fraud measures in place this time It'll be interesting to see what that one finds. Yep, it's the gift that keeps on giving. Thank you, Tom, and thanks to David Vosose and Annette Yckler Yes, I'm afraid another Klaxon this time with a nature theme I need to remind you at this point We hold power to account on more or less without fear or favor No one beyond scrutiny If our loyal listeners hear a statch that they doubt We will follow wherever the story leads no matter who it implicates. So it was when we got this email you club I heard on the BBC Secret Garden TV programe that it takes twenty thousand caterpillars to feed one blue tit chick until it fledges. Blue tits can have several chicks, so the numbers would multiply accordingly. I struggle to believe it's possible. Can you investigate, please Investigation is what we do, Matthew So our team went straight to the source. Who was it who made this eyebrow raising claim To successfully fledge, each chick will require twenty thousand caterpillars It's a tall order that must be completed within three weeks Wait, we're fact checking Sir David Attenborough The national treasure who just celebrated his hundredth birthday, Are you trying to get me fired? Without fear or favourim, without fear or favour Well who am I to question our production team? All of whose names are available in this program's credits you all down with me Is it really right that each blue tit chick requires twenty thousand caterpillars over the nearly three weeks that it's in the nest? Blue titt nests tend to have about ten chicks in, that's two hundred thousand caterpillars for the two parents to find in three weeks We spoke to Dr. Malcolm Burgess, principal conservation scientist at the RSPB and more or lesses, resident Bue tit consultant the scientific literature on the topic and did some quick calculations for us. So each chick will be receiving about ninety caterpillars per day. The blue tip nestlings are in the nest for about sixteen to twenty days So say twenty days, twenty times, ninety caterpillars per day gives about one thousand eight hundred custmers per check So nearly two thousand caterpillars per chick, much more feasible, but David Atttenborough said twenty thousand So what could have gone wrong Remember that there's more than one chick in each blue tip nest tyypically a nest would have up to ten chicks in it. could be more, could be less, but say I eal to ten So ten times one thousand eight hundred gives you eighteen thousand casps. eighteen thousand caterpillars for the entire nest of ten chicks just shy of Atenborough's twenty thousand figure that he said was per chick. So that's the mistake here They should have said twenty thousand caterpillars per nest. rather than per chick. After we got in touch with the Makers of the Secret Garden Dcumentary, the BBC published an official correction We have added a clarification strap at the start of the episode and apologise for their error in the narration. And who did that narration again? When caterpillars are in short supply Fiders. We'll have to do So we fact checked David Atttenborough next on our national treasure list is Paddington Bear That's all we have time for this week, but please keep your questions and comments coming to more or less at bbc. co. uk because there's a whole series to get through and we couldn't do it without your ideas. Until next time Goodby More or less was presented by me, Tim Harford. Producer was Tom Cooles, with Nathan Gower, Josh McMinn, and Lizzie McNeal Our productuction coordinator was Brenda Brown. The program was mixed by James Beard, and our editor is Richard Varden If you've got a scrolling problem, then this is the podcast for you. It's called Top Comomet with me, Matt She. And me and Mariana Spring, we both investigate social media for a living. Whether it's disinformation, conspiracy theories, internet culture, memes, we're going to be getting behind the stuff that is popping up on your feed on this podcast