HBO’s John Oliver has joined the ranks of Daily Show alumni who have gone on to absolutely delight in their own subsequent endeavors (see also Steve Carrell, Stephen Colbert, Ed Helms, John Hodgman, etc.). Oliver’s show, Last Week Tonight, takes the Daily Show’s format and extends it more into longform analysis, rather than the quick hits scored by Jon Stewart and company.
Last May, Oliver had a perceptive (and funny) piece about the way the media covers controversial science, and I’ve been wanting to post it here for a while. But a recent Scholarly Kitchen post by Phil Davis has made me rethink my initial enthusiasm for Oliver’s approach.
Oliver’s segment is below, and it gets to the confounding nature of news media which requires everything to be “fair and balanced”. When any statement is made, no matter how factual, there seems a requirement to present someone with the opposing viewpoint. Which brings us to climate change, and a recent poll showing that 25% of Americans are skeptical about it. As Oliver states,
You don’t need people’s opinions on a fact. You might as well have a poll asking which number is bigger, 15 or 5? Or do owls exist?
As he points out though, every news media outlet presents established fact as something open to debate, requiring one pro- commenter and one anti-. This is non-representative, given that the vast majority of the scientific community falls on one side of the supposed debate. Oliver’s response is to show the numbers more accurately:
It’s a funny bit and it gets to the real point about how flawed our media currently presents facts. If someone claims the earth is round, a flat earth representative must be included to represent the opposition, an author writing about the Holocaust must be accompanied by a neo-Nazi denying the whole thing ever happened.
I was really happy with Oliver’s piece until last week, when Phil Davis did yet another masterful analysis of a shoddy piece of research regarding the question of how citation relates to access for scholarly articles. If one looks at the raw numbers of studies done asking about the effects of open access on citation, the results are overwhelming: dozens of studies suggest a citation advantage, while only one (at least to my knowledge) shows that no such advantage exists.
So, using Oliver’s criterion, the question is answered. But the raw numbers don’t tell the real story in this case. One must look at the actual research itself, separate out the observational studies which suggest a correlation from the one study that performed the actual experiment and included the proper controls. The single experiment shows that the observational studies are showing correlation and not causation. But if you just tallied up the raw numbers…
So while Oliver is right in his skewering of the media, it’s hard to endorse a “majority rules” approach to deciding which science is right. Science sometimes works by having groundbreaking papers that overturn decades worth of dogma and well established beliefs. Quality matters more than quantity and while there’s an infinite number of incorrect theories, there’s only one right answer.
Perhaps a better approach is to try to separate facts from opinions, and not require an opposing viewpoint for the former, rather than trying to take a headcount on every issue.
*Please note that the comments below will be heavily moderated. We don’t want to go down the rabbit hole of climate change debate (there are plenty of outlets on the internet for that) and prefer to stick with the relevant subject here, how the media presents science to the public and how we make judgements about what’s right. Off topic comments will not be posted.
Discussion
11 Thoughts on "In Science, Should the Majority Rule?"
The opposite case is also a problem, namely reporting speculation as though it were facts. This is often due to a news article beIng written from a press release, which typically overstates the findings. News articles are typically written quickly, in hours not days.
In the debate case, debate is often a better hook than the facts. (I was a science journalist for a decade or so.) For example, I have been tracking the reporting on DOE’s announcement of their public access plan. Most articles and some of the headlines feature OA debate over explaining the plan. See for example:
http://www.washingtonpost.com/blogs/the-switch/wp/2014/08/04/why-open-access-advocates-arent-thrilled-with-the-does-plan-to-expand-public-access-to-its-research/
Of course in the climate case the public debate is extensive, as you note, so it can hardly be ignored. What is interesting here is the great numbers of people who are interested in physical science, as opposed to health and nutrition, the usual public interest. This is because of the policy implications of the science, not because the science is intrinsically interesting. So the debate becomes the hook. It is all about the readers, not the science.
David, there are about a half-dozen observational studies that report no citation advantage; however, they are clearly outnumbered by the observational studies that report a difference. Several years ago, I called out Alma Swan for using a paper-counting approach as a guide for declaring who won the OA-citation debate, see: http://scholarlykitchen.sspnet.org/2010/03/11/rewriting-the-history-on-access/
It seems to me that when a scientific study is politicized and/or popularized the first item tossed out the window is the science.
Harvey, I do not know what you mean by tossed out the window, but if the findings relate to a policy or public issue then that is important. How important depends on the audience, Sciencemag and WashPo being very different. Expository writing has a branching tree structure so the more attention given to one sub issue the less given to all the others, including the science per se. There is nothing wrong with this zero sum game. It is the human condition.
This is what meta-analysis is for.
There are different standards for different kinds of assertion. An opinion should be “defensible” whereas a factual assertion requires “proof” and a theory has to “fit” the phenomena under study. All of these supports can and usually are present to one degree or another. They are not commonly absolute.
Given that messy reality, those who are so inclined have a good bit of latitude in arguing for or against assertions they find congruent or incongruent with their world view. How free we are from these emotional and idealogical motives that skew our thinking is probably more a difference in degree than a difference in kind.
We have met the enemy and the enemy is us (Pogo).
Perhaps a better approach is to try to separate facts from opinions, and not require an opposing viewpoint for the former, rather than trying to take a headcount on every issue.
This is the tough part, though, since “trying to separate facts from opinions” is not only what science is about, but also what the debates over science are about. The purpose of science is to establish what the facts are, but science is always wrong — that’s why we have to keep doing science. It’s not just about discovering and establishing previously-unknown scientific truths, but also overturning previously-established scientific “truths” that turn out to be not actually true.
This is what makes the issue of “scientific consensus” so difficult. If a great majority of scientists agree on a fact, then surely that means something. On the other hand, the whole history of science is a history of existing scientific consensus being overturned by new and better science, which produces a new consensus that will itself be overturned eventually — except for some of it, which will turn out to be really and truly true and will never be overturned. But none of us can actually know what subset of today’s version of “The Science” that will be.
One thing I have noticed: all of us claim to love science as long as science tells us what we want to hear. As soon as science says something we don’t want to hear, we start talking about how science is corrupted by politics and special interests. (And I find this tendency to be true all across the political spectrum.)
I agree with this, and it is well said, except for the metaphor of science saying things. Science is actually several million people, saying many millions of things, to several billion people. There is thus no such thing as what science says, apart from what certain people say. Much follows from this. As you note, consensus is important (else there could be no progress) but not definitive. Debate too is important.
My favorite analog is rebuilding a ship at sea. Everything gets changed but you always have to have a place to stand.
In the STM arena, there are publishers of journals, ranked academics as editors and the editorial boards. The question here is “who has the say” in what gets published. In different journals and throughout the history, articles that are “controversial” get published and subsequent issues published comments and critiques of these articles. Today, there are many alternatives to respond, often influenced by the issues of “academic publishing”. Often responses and exchanges appear in different journals and depend on difference in interpretation of the same data sets as well as questioning of these data sets. Often the choice of where to publish research is dependent on a “reading” of who is on the editorial board.
Climate change is but one of the many issues in STM but highly visible because it has been taken beyond the STM research community and into the public arena. This makes the question more interesting in weighing the role of publisher and editor/editorial board.
