Last year, The Bible Society published a report on what they deemed a quiet revival in church-going. According to their analysis, which drew on data collected by the respected polling company YouGov in 2018 and 2024, the proportion of people attending church at least once a month grew by a whopping 50% during that time. The biggest proportional increase was, surprisingly, among the younger generations, with the proportion of 18-24 year old church-goers quadrupling.
These findings were naturally surprising, even to The Bible Society itself. Breaking the figures down further, church attendance came out highest among young men, with 21% or more than one in five 18-24 males saying they attended at least once a month. Bear in mind that the survey was specifically asking about Christian church-going, so the comparatively healthier state of mosque attendance, for example, has nothing to do with it. What’s more, the findings contradict the Church of England’s own estimates that attendance fell 19% during the pandemic, only recovering by 4.5% in 2022-23 and by a further 1.5% in 2023-24.
NatCen’s John Curtice was first on the scene, questioning the accuracy of the survey since it plainly contradicted what NatCen’s British Social Attitudes survey had to say about religiosity and church-going during the same period. The long-running BSA had the proportion of people identifying as Christian staying at 40% between 2017 and 2024. And while the BSA did show a modest increase in the proportion saying they ever attended church (from 32% to 35% during that time), within that the proportion attending once a month or more frequently fell. Other robust sources such as the Labour Force Survey and the census don’t show even a hint of a revival.
The YouGov surveys for The Bible Society look like a clear case of bad data. But why? What went wrong? YouGov charges good money for its polling services and has a reputation to uphold. If this was a major slip-up, it wasn’t noticed by the people doing the polling. They had followed all the usual polling protocols, and happily passed data on to The Bible Society despite it contradicting basically everything else including common sense.
Polling bots and trolls
What polling experts and researchers who’ve looked into this now seem to agree is that this wasn’t what you’d call a traditional failure of polling. Classic things that can go wrong with polls are that they don’t include enough people, that certain groups of people are missed or that not a broad enough range of people are sampled. Things can also go wrong on the technical side, such as responses being wrongly weighted.
What happened here seems to have been something much harder to guard against than any of these typical dangers: people were deliberately giving answers that were not true.
Most polls these days, certainly of the quick kind that YouGov does, are via opt-in online surveys. This is in contrast to traditional polling methods which were chiefly over the phone or face-to-face. It’s actually very hard to even select and contact people using those old methods now, so the internet is the only option for getting quick responses at scale.
Pollsters have been noticing problems with opt-in online surveys already for years. Pew Research found in 2020 that around 7% of respondents to opt-in surveys where people got paid to answer were ‘bogus.’ These were people who took the survey multiple times, gave non sequitur answers, or always answered ‘yes’ no matter what was asked. These could be real people, rapidly clicking through dozens of surveys per day for small amounts of cash, possibly without even reading the questions. But ‘bogus’ respondents could in some cases even be bots. Dartmouth professor Sean Westwood recently proved that large language models can be programmed to answer surveys and even to generate different personas each time to evade fraud detection.
One of the biggest tells for these bogus or bot respondents is consistently saying “yes” even when that is by no means the logical answer. Pew ran an experiment to test which respondents were paying attention by asking whether they were licensed to operate a class SSGN (nuclear) submarine. A somewhat surprising 12% of their adult respondents under 30 answered ‘yes.’ Seventeen per cent also answered yes to having purchased a private jet, climbed a peak in the Karakoram Mountains, learned to cook halusky (Slovakian potato dumplings), or played jai alai.
Jai alai always gives me a chuckle as it reminds me of Mad Men and the campaign to popularise this bizarre sport in the 1960s. Safe to say there’s absolutely zero evidence that that’s having a real revival either.
Opt-in surveys about Christianity in particular seem to consistently produce wild over-estimates of the active Christian population. This could be because of the general ‘yes’ bias from bogus respondents but that doesn’t seem enough to explain the scale of it. The Bible Society’s survey suggested more than one in five 18-24 year old men in their sample was a regular church-goer. Could it be that one in five young men answering the survey was just clicking through, answering anything? I think it’s also possible that with Christian surveys in particular, some of these respondents are trolling. Perhaps they aren’t just clicking aimlessly: they’re reading, and they’re deliberately constructing a fake persona, for a laugh.
The quality of opt-in internet survey data is clearly a big problem – even an existential threat to polling. At least we still have surveys people appear to answer more honestly, like British Social Attitudes, for now.
Demographic change
So the Christian revival is almost certainly not real, but it also seems logical that there has been some change in the level of religiosity in the population purely from population change in recent years.
The Bible Society, in its Quiet Revival report, pre-empts what it assumes will be people’s first question upon seeing Christianity surging back among the young by including a section which asks, ‘Is this just down to immigration?’ They say no, and even if their polling results were accurate, immigration hasn’t been high enough to quadruple the level of church-going among the young.
This is fair, in the sense that if there’s been an effect from immigration it’s likely a small one. But I do also wonder if surveys like British Social Attitudes and the Labour Force Survey could be under-estimating this modest change.
In the UK, for several reasons, we have a lack of up-to-date demographic data. One reason is that we’ve never really had it; another is that the Office for National Statistics (ONS) is in the middle of transforming the way it produces population estimates, so these are currently even less detailed than they used to be. Although the ONS produces annual estimates of the population by age and sex, there’s nothing equivalent for ethnicity – and certainly nothing for religion. This is in the context of the UK population having grown by around 2.5 million or by around 4% since before the 2021 census, due to immigration.
Having up-to-date data on age, sex, ethnicity, and ideally nationality, is crucial for weighting survey data. Again, the population grew 4% between 2021 and 2024, in large part due to immigration from outside North-Western Europe or, in other words, from countries with different patterns of religiosity to our own. My question is, given the lack of official, detailed demographic estimates since the 2021 census, are we sure this change is being reflected in survey weighting?
I’ve been doing some modelling recently, along with colleagues at the Migration Observatory, to roll forward 2021 census data, using estimates of population change since then, in order to estimate the foreign-national population in the UK now.
Using these rolled forward figures, it’s also possible to make a very rough estimate of how the level of religious affiliation might have changed due to high immigration in recent years. I’ve taken what non-UK passport-holders said about their religious affiliation in the 2021 census and assumed that the same pattern broadly holds true for new arrivals. The method isn’t perfect but I’m confident in the overall approach.
The result is that given the likely profile of new arrivals, we should be seeing an increase of one or two percentage points in the proportion of younger people who are religious. This doesn’t tell us anything though about church-going.
If there’s anything I’m worried about with this method it might wrongly estimate religious affiliation in new non-UK arrivals if that is significantly different to the pattern observed in non-UK nationals who have been here for any length of time. It would be possible to refine the model further by tailoring ‘expected religious affiliation’ more closely to, say, the nationality composition of new arrivals – I may do this at some stage.
A similar model can be used to estimate how the proportion of people in each age band who are in each broad ethnic group may have changed since 2021. This model is a little more precise as I’ve controlled for the global region of nationality (using Home Office visa data) of new arrivals.
Broadly speaking, people in the Black, White and Mixed ethnic groups are most likely to affiliate with Christianity. Collectively, the proportion of young people in this grouping has shrunk, so it would make sense to also see a reduction in the proportion of those age bands who are Christian. This somewhat contradicts what the previous chart showed, so it’s a bit of a mixed picture overall.
This is just a rough analysis and could certainly be improved by controlling for more factors. The bottom line though is that opt-in online polling data on this subject should be taken with a huge grain of salt and that data from traditional survey methods is surely much closer to the truth. That said, when we’re talking about the size of minorities, a percentage point here and their in your estimate makes quite a big difference, and our ability to be precise on this topic is ultimately hamstrung by bad data.
Until next time, God bless.
Originally published on Substack: https://georginasturge.substack.com/p/the-gen-z-christian-revival-polling