Two thought-provoking articles published last week in JAMA make compelling and complementary arguments to the rhetorical power of both numbers and words in conveying the message of science.
The first, “Reporting of Effect Direction and Size in Abstracts of Systematic Reviews,” by Elaine Beller and others, looks at the reporting of statistical results in the abstracts of reviews that contain meta-analyses.
Meta-analyses are important in medicine for synthesizing the research literature. They do this by pooling the results of many individual studies and attempting to estimate an overall effect. Unfortunately, many clinicians are not confident in interpreting statistical results, and some findings can be counter-intuitive. For example, the odds ratio –a statistic that reports the relative probability of two distinct outcomes– uses 1 as a reference point, not zero.
An odds ratio of 2 means that one outcome was twice as likely to occur as the other, but just half as likely when the odds ratio is 0.5. Depending on how the researcher set up the analysis, odds ratios above 1 are usually interpreted as beneficial, but not always. Reporting one’s results with numbers and words is critical to avoid ambiguity or misinterpretation especially when a reader consults just the abstract of the paper.
Of 182 abstracts covered by their study, 77 (42%) did not report the direction of the effect in words; in 34 cases (19%), the direction of effect could only be determined by interpreting the numerical results; and in one-quarter of the abstracts (24%), there was no measure of uncertainty such as a confidence interval or a P value. Beller writes:
Although abstracts should present estimates of effect and confidence intervals, interpretation of the results should not require statistical knowledge. Given the high level of innumeracy among journal readers, the main results should be presented in both words and numbers.
In a second article “Narrative vs Evidence-Based Medicine — And, Not Or,” Zachary Meisel presents a compelling argument that scientists need to understand the power of narrative in translating dispassionate clinical studies into stories that people can comprehend. He writes:
Scientific reports are genuinely dispassionate, characterless, and ahistorical. But their translation and dissemination should not be. Stories are an essential part of how individuals understand and use evidence.
From a public health perspective, Meisel does not need to reach far back to describe how scientists failed to control public hysteria over a possible link between childhood vaccination and autism armed with just facts and figures. In this battle, scientists lacked a convincing counter-narrative to illustrate how avoiding vaccination was far riskier behavior.
For example, the story from a distraught mother convinced that the MMR vaccine was the cause of her child’s autism could be countered with a story of an infant who became gravely sick after being exposed to an unvaccinated child. In the latter story, the child is portrayed not as a victim, but as a public health risk, and the mother as an irresponsible guardian. Meisel writes:
Those who espouse only evidence — without narratives about real people — struggle to control the debate. Typically, they lose.
While this paper is about communicating science and not science publishing, I’m reminded of early public debates surrounding open access policy in which anecdotes from distraught mothers was met with dry accounting data from academic publishers. Like the MMR vaccination controversy, publishers lacked a convincing counter-narrative and the rest, well, is history.
One of the most brilliant narratives that I’ve seen recently was a story about Jonathan Eisen, one of the academies’ strongest proponents of open access publishing and the Editor-in-Chief of PLoS Biology. The story, appearing in Wired Magazine, opens with a photo of Eisen’s father, an NIH researcher who took his own life at the young age of 45. While the piece touches upon the economics and politics of science publishing, the narrative of this story is essentially the quest of a son grieving for his departed father, and the first time I read it, tears came to my eyes.
No academic paper, spreadsheet or policy document has come close to conjuring the same emotional response. This is the power of narrative.