John Steinbeck on Story telling...
Image by Jill Clardy via Flickr

Two thought-provoking articles published last week in JAMA make compelling and complementary arguments to the rhetorical power of both numbers and words in conveying the message of science.

The first, “Reporting of Effect Direction and Size in Abstracts of Systematic Reviews,” by Elaine Beller and others, looks at the reporting of statistical results in the abstracts of reviews that contain meta-analyses.

Meta-analyses are important in medicine for synthesizing the research literature. They do this by pooling the results of many individual studies and attempting to estimate an overall effect. Unfortunately, many clinicians are not confident in interpreting statistical results, and some findings can be counter-intuitive. For example, the odds ratio –a statistic that reports the relative probability of two distinct outcomes– uses 1 as a reference point, not zero.

An odds ratio of 2 means that one outcome was twice as likely to occur as the other, but just half as likely when the odds ratio is 0.5. Depending on how the researcher set up the analysis, odds ratios above 1 are usually interpreted as beneficial, but not always. Reporting one’s results with numbers and words is critical to avoid ambiguity or misinterpretation especially when a reader consults just the abstract of the paper.

Of 182 abstracts covered by their study, 77 (42%) did not report the direction of the effect in words; in 34 cases (19%), the direction of effect could only be determined by interpreting the numerical results; and in one-quarter of the abstracts (24%), there was no measure of uncertainty such as a confidence interval or a P value. Beller writes:

Although abstracts should present estimates of effect and confidence intervals, interpretation of the results should not require statistical knowledge. Given the high level of innumeracy among journal readers, the main results should be presented in both words and numbers.

In a second article “Narrative vs Evidence-Based Medicine — And, Not Or,” Zachary Meisel presents a compelling argument that scientists need to understand the power of narrative in translating dispassionate clinical studies into stories that people can comprehend. He writes:

Scientific reports are genuinely dispassionate, characterless, and ahistorical. But their translation and dissemination should not be. Stories are an essential part of how individuals understand and use evidence.

From a public health perspective, Meisel does not need to reach far back to describe how scientists failed to control public hysteria over a possible link between childhood vaccination and autism armed with just facts and figures. In this battle, scientists lacked a convincing counter-narrative to illustrate how avoiding vaccination was far riskier behavior.

For example, the story from a distraught mother convinced that the MMR vaccine was the cause of her child’s autism could be countered with a story of an infant who became gravely sick after being exposed to an unvaccinated child. In the latter story, the child is portrayed not as a victim, but as a public health risk, and the mother as an irresponsible guardian. Meisel writes:

Those who espouse only evidence — without narratives about real people — struggle to control the debate. Typically, they lose.

While this paper is about communicating science and not science publishing, I’m reminded of early public debates surrounding open access policy in which anecdotes from distraught mothers was met with dry accounting data from academic publishers. Like the MMR vaccination controversy, publishers lacked a convincing counter-narrative and the rest, well, is history.

One of the most brilliant narratives that I’ve seen recently was a story about Jonathan Eisen, one of the academies’ strongest proponents of open access publishing and the Editor-in-Chief of PLoS Biology. The story, appearing in Wired Magazine, opens with a photo of Eisen’s father, an NIH researcher who took his own life at the young age of 45. While the piece touches upon the economics and politics of science publishing, the narrative of this story is essentially the quest of a son grieving for his departed father, and the first time I read it, tears came to my eyes.

No academic paper, spreadsheet or policy document has come close to conjuring the same emotional response. This is the power of narrative.

Phil Davis

Phil Davis

Phil Davis is a publishing consultant specializing in the statistical analysis of citation, readership, publication and survey data. He has a Ph.D. in science communication from Cornell University (2010), extensive experience as a science librarian (1995-2006) and was trained as a life scientist. https://phil-davis.com/

Discussion

9 Thoughts on "Statistics and Storytelling — Why We Need Them Both in Science"

What a depressing post!

“Given the high level of innumeracy among journal readers, …” Ugh

“Those who espouse only evidence — without narratives about real people — struggle to control the debate. Typically, they lose.” Thus “The debate” will be won by those who are best able to conjure up a story without regard to the connection to the facts.

One sees why “debate” is used as an anti-science tool. If William Lane Craig debates Richard Dawkins and shows the latter to be tongue-tied, evolution will no longer exist. If Monckton were to debate Gore and show the former to be emotive and the latter to be cold-hearted, we will have fixed Global Warming. Ugh.

Dave, I think you are misunderstanding Meisel’s argument. He is suggesting that storytelling accompany science not replace it.

The question, Phil, is “accompany science” where? Journal articles, including meta analyses, have a deliberately calm and understated language. Science-related policy advocacy, on the other hand, is often emotionally hyperbolic, as is all advocacy. Which are you talking about?

I had it drummed into me from the start of my career in market research that you must “tell a story” when you present or write up results. The goal is not simply to provide charts or written commentary on every single question or quote from a study but to put it all together in a way that answers the objectives and makes sense to someone who has had no involvement in the research. The other important point I had drummed into me was to “write for your audience”. When writing a Journal article you are generally aiming at a specific community of people, so one type of language is appropriate, when communicating the research outside of this community a different approach may be necessary which might include a more narrative approach with more examples and when communicating to the general public a different slant should be taken again. It might not necessarily mean that a Journal should use anything other than “deliberately calm and understated language” as David suggests above, but when this research is picked up and placed in a more public domain the way to defend or promote this research should change. This would be the time to definately include a more narrative approach.

I agree Christine. back in the day I worked as a journalist, reporting science and technology for the electric power industry via Electricity Daily. We had to introduce the study then present and explain the results, saying why they were important. It was certainly a story. But is science journalism scholarly publishing?

I completely agree about making a strong narrative. Not only is it important in communicating with the wider public but I think it is equally as necessary to use narrative when addressing those in public policy. These are the people who are the decision makers about implementing the research and if it doesn’t make sense to them or has a place to lodge in their memory then our 50 page whitepapers are for nothing. Colorado is using story telling in communities about health issues and then are connecting the story telling to tangible initiatives done at the community level. http://pitchforksoptional.com/2011/11/22/the-benefit-of-using-empathy-and-story-telling-to-change-health-care/

Comments are closed.