Trevor Butterworth makes a point during his talk on using data in news stories.
news stories, do the math
Trevor Butterworth did not go into journalism to write about statistics, and still considers himself to be bad at math, even after years of working with statisticians and mathematicians on a daily basis. What he knows how to do is how to ask for help when he needs it.
Butterworth is editor at large at STATS.org, based in Washington, D.C., and director at Sense About Science, a United Kingdom-based nonprofit that assists STATS.org in providing help with statistical analysis, and regularly contributes to a variety of online publications. His talk, titled “Does it all add up? Using Big Data,” Friday, Feb. 19, at the New England Newspaper and Press Association’s winter convention, was attended by about 30 people.
Butterworth, considered to be an expert on data use in journalism, ended up in the field essentially by chance.
“I had the opportunity to be unemployed or I had the opportunity to take over a failing, dying almost moribund nonprofit called STATS.org,” Butterworth said.
While Butterworth was making his decision about whether he wanted to take on that failing organization, he read a story in Science Magazine about potential cancer risks created by PCBs found in farmed salmon. After reading the full story, Butterworth wondered how likely it was that he would actually get cancer from eating the fish. The most elemental statistics were left out of the Science Magazine story and out of a similar story in Butterworth’s then-local newspaper, The Washington Post.
“I wasn’t out of school that long, but I thought, “Isn’t that kind of the basic question you would ask here?’ ” Butterworth said. “If someone tells you there’s a risk of rain, you go, ‘Well, how likely?’ And then you take an umbrella. But not only did the Post not tell me, but most of the news stories around the world didn’t actually tell me what my risk was.”
That story inspired Butterworth to work with a toxicologist to determine what the risk was of getting cancer from PCBs in farmed salmon. They discovered that the concentrations were about 26 parts per billion, which does not constitute a significant risk of cancer by scientific standards. News outlets around the world were calling for readers to avoid consuming the fish, however, and some people stopped eating it.
Butterworth gave numerous other examples of studies, mostly in the nutrition field, that used data in incorrect ways and generated misleading trends, the exact problem STATS.org is designed to solve.
STATS.org is a resource for reporters who want to double-check the use of data in studies they cover or in their stories. It is a network of statisticians and mathematicians who have volunteered to offer their expertise. Reporters can ask questions through the STATS.org website about data or statistics, and a volunteer will respond as quickly as he or she can. Butterworth noted that journalists must give adequate time for volunteers to respond, because they have full-time jobs.
“We have this idea that we are now in a fourth paradigm,” Butterworth said. “The first paradigm was experimental, second was theoretical, third was computer science, and now we have data-intensive. So our ability to record data can give us the answers to questions we didn’t even know we could ask.”
After Butterworth’s presentation, he answered audience members’ questions, mostly focusing on how a reporter can best detect “bad data.”
Butterworth warned against taking observational studies as fact, because unlike a scientific experiment, those studies are just systematic observations and do not include a control. Because of that, correlations can be drawn, when in reality, the observed effect could have been caused by an entirely different variable. Butterworth gave the example of a study that showed a direct correlation between the number of Nicolas Cage movies released and the number of pool deaths each year, which clearly are unrelated occurrences.
He also suggested being wary of trends that seem either too good to be true or too bad to not be true, such as studies suggesting a certain food will cause cancer in anyone who eats it.
He urged journalists to make use of STATS.org or local resources whenever they were unsure of their analysis of data or statistics, and always to get a second opinion.
Sarah Schwarz is an undergraduate student in the Northeastern University School of Journalism.
© Copyright 1998-2016 New England Newspaper and Press Association. All rights reserved.