Collecting statistics: When the numbers don’t add up

Source: Radio New Zealand

Stats NZ had a seven percent cut to its budget, but the Government Statistician, Colin Lynch, denies the cut has affected the quality of information gathering or public distribution of statistics. Screenshot

A former government statistician is sounding alarms about our ditching of the five-yearly census, saying we risk losing a rich seam of information.

It’s getting harder and harder to nail down the numbers when it comes to collecting statistics.

People are increasingly wary of filling out surveys, data is becoming politicised and the results are becoming harder to follow.

In the last several weeks, the Ministry of Business, Innovation and Employment has struggled to produce accurate figures on fuel shipments; the health ministry has had to sort out confusion over Covid death figures; and the chief holder of information, Stats NZ, has bungled the food price index.

“We certainly endeavour to make no mistakes,” says the Government Statistician, Colin Lynch.

“That’s the ambition – everything’s 100 percent correct – and you need to put that in context of 270-plus releases every year and hundreds of thousands of series.

“But for the food price index, we did get it wrong. I did what I think was right which was to be transparent about it. And it didn’t have any impact, nor would it have had any impact on the CPI or GDP or any other macro-indicators. I have a review underway so that we do completely understand what happened … and we actively learn [from] the error and what underlying causes we might need to change in our systems to make sure we don’t repeat that mistake.”

He says trust is at the centre of everything they do, and he’s very focused on it.

Lynch denies the department’s seven percent budget cut has affected the quality of information gathering or public distribution of statistics.

“We’re unapologetically focused on delivering value for money for New Zealanders,” he says.

While the head count has reduced in recent years (media estimates suggest by about 300 people) he’s confident they have the right people in the right roles in the right places.

Former government statistician Len Cook says Stats NZ’s website is “almost impenetrable for an ordinary person to use” and Lynch replies to that by saying it’s being rebuilt.

He says the world is increasingly complex and increasingly AI-led.

“My focus is ensuring New Zealanders and others can get access to that information in that world. So with that in mind we are rebuilding our website because what we’ve discovered, I’m sure like many others, is that more and more people are consuming our data through AI search or AI-related tools. We’re seeing less individual visitors to our website because we know that AI’s the main source of information, with AI searches.

“We will have a new website by early next year and we’re also looking that all our data tools that are on that website are up-to-date and AI-enabled.

“People will still be able to visit the website and get what they want directly, but we also need to make sure that when people put a search for GDP in AI they get high quality Statistics New Zealand official statistics.”

There’s a huge change coming with the way we collect those statistics – the traditional five-yearly, knock-on-every-door census is going. There’s an amendment bill going through Parliament now (it’s passed its first reading) that would get rid of it in favour of an administrative data-first approach.

Stats NZ has done a Regulatory Impact Statement on the change, including warnings that it would mean some reductions in data accuracy, detail and coverage, especially initially for groups such as disabled people, rainbow communities and small ethnic communities.

But it also points out that there’s growing public resistance to complete census forms, and rising costs – the 2023 census cost about $326 million dollars to carry out.

“What’s changing is the way we collect the census,” says Lynch.

From 2030, yearly surveys of about five percent of the population would be run alongside the collection of data from government departments, such as prison numbers from the Justice Department, welfare figures from WINZ, car registration numbers from NZTA, etc.

Lynch says it will mean richer data available for New Zealanders.

But Len Cook is raising alarm bells about the move, including that no independent expert has been asked for advice.

He’s sent a 10-page list of concerns to MPs and anyone who might be thinking about making a submission on the amendment bill. Among those concerns is that data will be of lower quality and leave gaps – including of society’s ‘invisible’ people, such as caregivers for relatives, new stay-at-home parents, the disabled, religious groups, small communities and Māori.

And he says we’re making the change in the midst of a population storm, when, for example, there’s a huge outflow of young New Zealanders. This has repercussions for the projected tax take, housing planning, education and much more. Cook worries that there will be turmoil if decisions are driven by political instincts rather than sound statistics.

New Zealand isn’t the only country making changes around the census but Cook says in many of them, there’s also been a move towards national ID cards to keep track of the population – something that New Zealanders aren’t keen on.

Also on today’s podcast we speak to two journalists who’ve had head-banging moments when it comes to obtaining official statistics, Newsroom’s Jonathan Milne and The Detail’s own Amanda Gillies.

Check out how to listen to and follow The Detail here.

You can also stay up-to-date by liking us on Facebook or following us on Twitter.

– Published by EveningReport.nz and AsiaPacificReport.nz, see: MIL OSI in partnership with Radio New Zealand