Controversial Topics, Education, Research

Statistics and Storytelling — Why We Need Them Both in Science

John Steinbeck on Story telling...

Image by Jill Clardy via Flickr

Two thought-provoking articles published last week in JAMA make compelling and complementary arguments to the rhetorical power of both numbers and words in conveying the message of science.

The first, “Reporting of Effect Direction and Size in Abstracts of Systematic Reviews,” by Elaine Beller and others, looks at the reporting of statistical results in the abstracts of reviews that contain meta-analyses.

Meta-analyses are important in medicine for synthesizing the research literature. They do this by pooling the results of many individual studies and attempting to estimate an overall effect. Unfortunately, many clinicians are not confident in interpreting statistical results, and some findings can be counter-intuitive. For example, the odds ratio –a statistic that reports the relative probability of two distinct outcomes– uses 1 as a reference point, not zero.

An odds ratio of 2 means that one outcome was twice as likely to occur as the other, but just half as likely when the odds ratio is 0.5. Depending on how the researcher set up the analysis, odds ratios above 1 are usually interpreted as beneficial, but not always. Reporting one’s results with numbers and words is critical to avoid ambiguity or misinterpretation especially when a reader consults just the abstract of the paper.

Of 182 abstracts covered by their study, 77 (42%) did not report the direction of the effect in words; in 34 cases (19%), the direction of effect could only be determined by interpreting the numerical results; and in one-quarter of the abstracts (24%), there was no measure of uncertainty such as a confidence interval or a P value. Beller writes:

Although abstracts should present estimates of effect and confidence intervals, interpretation of the results should not require statistical knowledge. Given the high level of innumeracy among journal readers, the main results should be presented in both words and numbers.

In a second article “Narrative vs Evidence-Based Medicine — And, Not Or,” Zachary Meisel presents a compelling argument that scientists need to understand the power of narrative in translating dispassionate clinical studies into stories that people can comprehend. He writes:

Scientific reports are genuinely dispassionate, characterless, and ahistorical. But their translation and dissemination should not be. Stories are an essential part of how individuals understand and use evidence.

From a public health perspective, Meisel does not need to reach far back to describe how scientists failed to control public hysteria over a possible link between childhood vaccination and autism armed with just facts and figures. In this battle, scientists lacked a convincing counter-narrative to illustrate how avoiding vaccination was far riskier behavior.

For example, the story from a distraught mother convinced that the MMR vaccine was the cause of her child’s autism could be countered with a story of an infant who became gravely sick after being exposed to an unvaccinated child. In the latter story, the child is portrayed not as a victim, but as a public health risk, and the mother as an irresponsible guardian. Meisel writes:

Those who espouse only evidence — without narratives about real people — struggle to control the debate. Typically, they lose.

While this paper is about communicating science and not science publishing, I’m reminded of early public debates surrounding open access policy in which anecdotes from distraught mothers was met with dry accounting data from academic publishers. Like the MMR vaccination controversy, publishers lacked a convincing counter-narrative and the rest, well, is history.

One of the most brilliant narratives that I’ve seen recently was a story about Jonathan Eisen, one of the academies’ strongest proponents of open access publishing and the Editor-in-Chief of PLoS Biology. The story, appearing in Wired Magazine, opens with a photo of Eisen’s father, an NIH researcher who took his own life at the young age of 45. While the piece touches upon the economics and politics of science publishing, the narrative of this story is essentially the quest of a son grieving for his departed father, and the first time I read it, tears came to my eyes.

No academic paper, spreadsheet or policy document has come close to conjuring the same emotional response. This is the power of narrative.

About Phil Davis

I am an independent researcher and publishing consultant specializing in the statistical analysis of readership and citation data. I am a former postdoctoral researcher in science communication and former science librarian. http://phil-davis.org/

Discussion

9 thoughts on “Statistics and Storytelling — Why We Need Them Both in Science

  1. What a depressing post!

    “Given the high level of innumeracy among journal readers, …” Ugh

    “Those who espouse only evidence — without narratives about real people — struggle to control the debate. Typically, they lose.” Thus “The debate” will be won by those who are best able to conjure up a story without regard to the connection to the facts.

    One sees why “debate” is used as an anti-science tool. If William Lane Craig debates Richard Dawkins and shows the latter to be tongue-tied, evolution will no longer exist. If Monckton were to debate Gore and show the former to be emotive and the latter to be cold-hearted, we will have fixed Global Warming. Ugh.

    Posted by Dave Pullin | Nov 15, 2011, 8:52 am
    • Dave, I think you are misunderstanding Meisel’s argument. He is suggesting that storytelling accompany science not replace it.

      Posted by Phil Davis | Nov 15, 2011, 8:56 am
      • The question, Phil, is “accompany science” where? Journal articles, including meta analyses, have a deliberately calm and understated language. Science-related policy advocacy, on the other hand, is often emotionally hyperbolic, as is all advocacy. Which are you talking about?

        Posted by David Wojick | Nov 15, 2011, 9:26 am
  2. Phil, there’s a lot of research in this area under the Public Understanding of Science banner. We also held a great event a few weeks ago on this topic with Alice Bell talking about the need to use the media to communicate more effectively. There’s video from the debate online – more here http://sageconnection.wordpress.com/2011/11/09/social-scientists-role-in-strengthening-trust-in-scientific-evidence/

    Posted by Mithu Lucraft (@mithulucraft) | Nov 15, 2011, 10:21 am
  3. I had it drummed into me from the start of my career in market research that you must “tell a story” when you present or write up results. The goal is not simply to provide charts or written commentary on every single question or quote from a study but to put it all together in a way that answers the objectives and makes sense to someone who has had no involvement in the research. The other important point I had drummed into me was to “write for your audience”. When writing a Journal article you are generally aiming at a specific community of people, so one type of language is appropriate, when communicating the research outside of this community a different approach may be necessary which might include a more narrative approach with more examples and when communicating to the general public a different slant should be taken again. It might not necessarily mean that a Journal should use anything other than “deliberately calm and understated language” as David suggests above, but when this research is picked up and placed in a more public domain the way to defend or promote this research should change. This would be the time to definately include a more narrative approach.

    Posted by Christine | Nov 15, 2011, 10:28 am
    • I agree Christine. back in the day I worked as a journalist, reporting science and technology for the electric power industry via Electricity Daily. We had to introduce the study then present and explain the results, saying why they were important. It was certainly a story. But is science journalism scholarly publishing?

      Posted by David Wojick | Nov 15, 2011, 10:54 am
  4. I completely agree about making a strong narrative. Not only is it important in communicating with the wider public but I think it is equally as necessary to use narrative when addressing those in public policy. These are the people who are the decision makers about implementing the research and if it doesn’t make sense to them or has a place to lodge in their memory then our 50 page whitepapers are for nothing. Colorado is using story telling in communities about health issues and then are connecting the story telling to tangible initiatives done at the community level. http://pitchforksoptional.com/2011/11/22/the-benefit-of-using-empathy-and-story-telling-to-change-health-care/

    Posted by Joy Twesigye (@forkoptional) | Dec 2, 2011, 3:40 pm

Trackbacks/Pingbacks

  1. Pingback: 111115 Cirrus | Clyde Street - Nov 15, 2011

  2. Pingback: Measuring the impact of research access in the developing world | Elsevier Connect - Jan 7, 2013

Side Dishes by Stewart Wills

Find Posts by Category

Find Posts by Date

November 2011
S M T W T F S
« Oct   Dec »
 12345
6789101112
13141516171819
20212223242526
27282930  

The Scholarly Kitchen on Twitter

SSP_LOGO
The mission of the Society for Scholarly Publishing (SSP) is "[t]o advance scholarly publishing and communication, and the professional development of its members through education, collaboration, and networking." SSP established The Scholarly Kitchen blog in February 2008 to keep SSP members and interested parties aware of new developments in publishing.
......................................
The Scholarly Kitchen is a moderated and independent blog. Opinions on The Scholarly Kitchen are those of the authors. They are not necessarily those held by the Society for Scholarly Publishing nor by their respective employers.
Follow

Get every new post delivered to your Inbox.

Join 14,562 other followers

%d bloggers like this: