There’s no fresh start in today’s world. Any twelve-year-old with a cell phone could find out what you did. Everything we do is collated and quantified. Everything sticks. — Selina Kyle (Catwoman), in “The Dark Knight Rises”
Last week, I wrote about an interesting controversy involving authors who analyzed blog posts and public comment threads on sites talking about climate change and discovered a correlation between climate change deniers and conspiracy theorists. The paper, entitled, “Recursive fury: Conspiracist ideation in the blogosphere in response to research on conspiracist ideation,” was published in a journal called Frontiers in Psychology, from the publisher Frontiers. The paper went through a number of ups and downs before finally being retracted a few weeks ago due to what the lawyers involved called an “unclear legal context,” wording that sounds a little like “because we don’t want to be sued over this one.”
Upon retraction, Frontiers went on the record as saying that:
In the light of a small number of complaints received following publication of the original research article cited above, Frontiers carried out a detailed investigation of the academic, ethical, and legal aspects of the work. This investigation did not identify any issues with the academic and ethical aspects of the study.
However, Frontiers recently changed its tune, stating that there was an ethical issue, one having to do with identification of what they call “human subjects,” a perhaps disingenuous use of the term given that what was analyzed were blog posts and online comments:
For Frontiers, publishing the identities of human subjects without consent cannot be justified in a scientific paper. Some have argued that the subjects and their statements were in the public domain and hence it was acceptable to identify them in a scientific paper, but accepting this will set a dangerous precedent. With so much information of each of us in the public domain, think of a situation where scientists use, for example, machine learning to cluster your public statements and attribute to you personality characteristics, and then name you on the cluster and publish it as a scientific fact in a reputable journal. While the subjects and their statements were public, they did not give their consent to a public psychological diagnosis in a scientific study. Science cannot be abused to specifically label and point out individuals in the public domain.
This seems a little simplistic. Human subjects are defined by the Department of Health and Human Services as:
. . . a living individual about whom an investigator conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information about whom includes a subject’s opinion on a given topic.
Further in the definition, there is an interesting phrase about “private” information — i.e., it is information “which the individual can reasonably expect will not be made public.” The blog posts and comments used in the “Recursive Fury” paper were public, as were the identities of those making them. I don’t see how there can be any expectation of privacy.
There is also a difference between human subjects in a clinical study and observing the information trail left by humans in their natural environment. Imagine the fields that would vanish — anthropology, history, economics, sociology, literature — if we needed permission to study anything left behind by other humans.
One complaint is that the people leaving these posts and comments behind can be identified. And, in fact, reading the paper, it wouldn’t take long for anyone with a browser to find out who made the comments. This only underscores that their identities were not private, but public, at least to the extent of being “identifiable.” And this is probably as it should be. Consider the alternative — that the paper includes completely anonymous and unidentifiable statements rather than giving some specifics that could lead to someone following a public statement to a public author. The complaint I’d imagine from the anonymized approach would have been even more difficult to dispute or defend — the researchers would be accused of inventing data or showing their bias by fabricating comments or being unwilling to share their data. Clearly, that’s not a viable research alternative, especially on topics such as this.
Any expectation of privacy seems to require a reality based on Google and the Internet and databases not existing. Remember, in 2002, using anonymized patient data and publicly available records, a researcher was able to identify the health records of the Massachusetts Governor at the time, William Weld. Recommendations for purchases follow us from sites to sites. Retailers can figure out if someone is pregnant, using only shopping events, which seem much more private than public blog comments and postings. Can we really act offended, shocked, and appalled when public statements in the public domain or observable public behaviors are analyzed and ultimately traceable back to us? Is that really a basis for retraction? Or just a new level of reality?
The researchers used nothing private to reach their conclusions. But there is more. One part of the complaint seems to be that they proposed that the people expressing certain opinions, as a group, were both climate change deniers and conspiracy theorists. Rather than disproving this, the group has responded by claiming this is part of a conspiracy, inadvertently confirming what the paper found while trying to undermine how the research was done.
It’s also worth mentioning that this case is in an even grayer zone, because the researchers are accused not of making libelous statements, but of labeling the individuals’ own statements in a libelous manner. However, labels aren’t inherently defaming or libelous. Calling someone a “climate change denier” or a “conspiracy theorist” is not necessarily defamation. Just like calling someone a “couch potato” or “radical liberal,” labels may be, to some, a badge of honor or merely accurate. Numerous court cases show that labels — from “dumb ass” to “satanist” — aren’t themselves defamatory, especially when they are used in a descriptive manner. In the case of a scientific paper, they clearly were used to describe comments and posts with shared attributes, all within a hypothesis-driven framework.
From a publishing perspective, what’s disturbing about how Frontiers is handling this is the lurching and erratic nature of its decisions. First, there was a hair-trigger when one complaint was received, and working with the authors to fix a couple of things. Fine. Not a problem, really, especially if the authors agreed. While it was tampering a bit with the scientific record, let’s cut them a little slack here. Next, a long interlude ensues, and then finally the paper is retracted after what is called an “investigation,” but only because of an “unclear legal context,” not for academic or ethical reasons. Most recently, the rationale for retraction has been publicly expanded to include ethical concerns — namely, using identifiable information without someone’s permission.
One change Frontiers could make would be to join the Council on Publication Ethics (COPE). The organization is not currently listed as a member. The flowcharts and decision trees from COPE might help guide future decision-making around thorny issues like this.
Frontiers is making a lot of noise about supporting its authors and reassuring its editors, just as it did with the first statement of retraction. However, actions speak louder than words. This latest round is sure to chill authors and editors more, as even after the retraction was official, Frontiers felt it necessary to back the car up to see what it had retracted.
This isn’t a blog about climate change, conspiracy theorists, or psychological research per se. It’s a blog about publishing. To me, Frontiers could be better positioned by joining COPE, could have held its decision until it had a full decision to make rather than stutter-stepping in public, and could have done more to reassure authors and editors that researchers won’t be thrown under the bus when things get dicey.
To me, this still smacks of a lack of backbone and author advocacy. As I wrote in February, one of the most important roles of a publisher is to protect authors from unwarranted tampering with their published results. When a paper is published, a bond of trust is forged between authors, editors, and publishers. They should stand united behind it, even if it takes time and resources to defend it. In this case, I have yet to see anything indefensible, so it’s particularly strange to have Frontiers collapsing like a wet taco.
The most worrisome aspect of this dispute is that the scientific record was intimidated into retraction by anti-science forces like those identified in this paper. For Frontiers, this could be a long-term problem, for as the quote that started this reminds us:
There’s no fresh start in today’s world. Any twelve-year-old with a cell phone could find out what you did. Everything we do is collated and quantified. Everything sticks.
44 Thoughts on "Keeping It Real — Ethics and Privacy as the Frontiers "Recursive Fury" Case Continues to Churn"
Given Frontiers statement and new policy, it seems they will be unable to publish any meeting review articles. Talks at meetings are given by human subjects and they might be identifiable from a meeting report. Also, they may not have given their consent to a public interpretation of their results. This also means no book reviews (books are written by human subjects). Similarly, they will need to ban all review articles. The previous papers mentioned were written by identifiable humans, and I know of no journal that requires authors to sign a consent form for public analysis of their work.
Really, Frontiers should not publish any papers that contain citations. The human subjects who wrote those previous papers could be identifiable from the author names included in citations.
I know nothing about this realm of law, but could one argue that this was a medical diagnosis, and an adverse one at that, so the privacy rules are rather more stringent than for reviews? Or is it common in psychology to publicly diagnose people from their writings? This seems like the central ethical question, one of medical ethics, not general scientific ethics, defamation, etc. Telling the world you are nuts, as it were.
This sort of analysis is common practice, and a policy like the one above would remove a wide swath of research from ever being conducted or practiced:
This seems all very simple. According to the common IRB rules “Research using sensitive information (e.g., information that reasonable persons would not want disclosed) that is identifiable is not exempt, even when it is publicly available.” (From http://www.umbc.edu/irb/IRBResearchersGuide_09_13_11.pdf). Research with human subjects (or any animal subjects) should have gone through IRB approval. If it did, the paper should have stood, and if there was any heat, it should have been on the IRB, not the authors. If it did not go through an IRB, the paper should never have been published, and retraction makes sense simply on these grounds. The decision of the journal editors should simply be that. (There are, at least in the US, IRB “companies” – I don’t know if they are non-profits – that can approve experiments from institutions that do not have their own IRBs.)
But I don’t think it’s all that simple. The research did not, in any way, use sensitive information or “information that reasonable persons would not want disclosed” as the information used in the study was all publicly disclosed by those who wrote it and publicly posted it.
The argument here is that those who publicly wrote that material might be unhappy with the conclusions of the study, but that’s an entirely separate issue from how the study was conducted.
What I mean is (or should be) simple here, is that from the POV of the journal, either a work is inhuman subjects was IRB approved or not. It clearly uses identifiable human subjects, so it clearly needs to have been checked by an IRB. Journals shouldn’t be making research ethics questions, and their answer is simple to complainers and lawyers alike: If the work uses human subjects, esp. Identifiable ones, and was IRB approved, don’t bother us. If it wasn’t, we don’t want it. This puts the burden of ethics back on the authors and their institutional IRBs, which is where it belongs. I realize that this means that every anthro and sociological project has to go to an IRB, which is a pain in the ass, but that’s what scientific ethics is about.
Cornell has an IRB document on the “Use of Social Networking Sites or Mobile Devices for Human Participant Research” (http://www.irb.cornell.edu/documents/IRB%20Policy%2020.pdf). The document admits that the field is not clear on informed consent, but recommends the researcher avoid making his/her own decision, send in the proposal anyway and allow the committee to decide. Note the following guideline, phrased “as a courtesy.”
As a courtesy, we recommend that individuals not be individually identified or that the information on the individuals be combined in such a manner that the identity of the group or individuals can be readily ascertained.
I understand that this may be a matter of authority, and that journal editors or publishers should not be in the position of making an ethical decision. On the other hand, given the ambiguous grounds of informed consent with regard to social networking research, this appeal to authority seems to be based on the subjective interpretations of individual IRBs.
I agree that referring things to an IRB is a good way for a journal to pass the buck here.
But I would also argue that no human subjects were experimented on in this study, just as no human subjects are used in analyzing the works of Shakespeare.
If i do a study in the depiction of lions in medieval art, does that qualify as an animal study and require IRB approval?
Citing the Common Rule is largely irrelevant, as the research was conducted in Australia. In the U.S., this would have been fine under exemption 4 if there were no interaction with the subjects, which is not at all clear here. The Australian view appears to be stricter. However, this one was tacked onto a previous IRB approval and so was not fully reviewed. Moreover, they began the study before even seeking that.
The examples you have linked are studies about the following individuals:
Jules Holtzapfel (died 1866);
Now, if you consider these valid examples to support your position, it’s obvious to me that either you lack some basic logic skills, or your reasoning abilities are impaired by some ideological bias. Sure, that’s just my opinion. I just wonder how would you like this very opinion to expressed by some psychologists in a peer reviewed article about the ideological biases of climate change proponents. Or do you think this is impossible because, come on, everybody knows climate change is happening and therefore you have a right to support in any way you want?
What do they all have in common? They are humans, are they not? Or are you saying the line is drawn depending strictly on whether one’s subjects are alive or dead? A psychological analysis on a living author based on his writing should be prevented until he dies?
Sure, they’re dead. They’re even more than dead, they’re historical figures. Really, can’t you see the difference? For instance, your second link points to an article about “the medical history of Thomas Jefferson”. Do you think that a similar article about “the medical history of Barack Obama” could or should be published? Even Wikipedia has specific standards for the biographies of living persons, and famous writers ask for their private memories to be published after their deaths. So yes, there is a strong distinction between living subjects and dead subjects. As for your last question, sure, you can’t publish a *scientific* psychological analysis of a living author, as that amounts to a diagnosis and you can’t publish a medical diagnosis of recognizable individuals without their consent.
Sorry, can’t agree. An article analyzing the motivations of Obama based on events described in his autobiography seems fair game to me. Ditto an analysis of Putin’s inner thinking based on his speeches. Or the mindset of Pynchon as seen through his books or Hockney from his paintings. Academia is rife with such analysis. A world where we were not allowed to examine the writing and other public outputs of the living would be a world we understood much less.
First off, scientific ethics applies to animals as well as to humans, albeit somewhat differently. But also, for many reasons it is not reasonable to worry about analysis of these people’s writings. Clearly the analysis of (presumably) living (or at best recently deceased) blog posters is of much more reasonable concern, and a conservative ethical position would suggest asking the IRB about it. One might wonder, for example, about DNA analysis of hitler, etc. as this may affect living persons, and any such experiment should be discussed with an IRB. It is, in the end, a set of judgement calls among reasonable persons, but reason suggests that unless the authors of the paper at hand had IRB approval, it indeed should have been retracted, and also that if they did, the journal should have stood its ground.
Those are all articles about people who are no longer alive and therefore unlikely to take offence.
No analogy at all.
As I asked above, is that where the line is drawn? We are only allowed to analyze and report on statements a person makes or that person’s writings, paintings or creations after they have died? What does this mean for our understanding of the world in which we live? Can we only retrospectively understand our own culture?
“As I asked above, is that where the line is drawn?”
Well I would say there is a pretty clear line between being alive or dead. So I think this would alert most people to the fact your claim the Recursive Fury retraction implies a policy that “remove[s] a wide swath of research” on long dead people, as being rather strange and melodramatic!
I think you should know that the Recursive Fury paper draws attention to its own unique approach here:
“Unlike previous analyses of web content, the present project was conducted in “real time” as the response to LOG12 unfolded, thus permitting a fine-grained temporal analysis of the emerging global conversation.”
I speculate that a large part of the problem between the journal and authors here derives from a lack of mutual understanding about this unique approach. I wonder if it was not fully discussed between the journal and paper authors?
It certainly seems more clear, looking at the correspondence between Lewandowsky and The University of Western Australia ethics committee, that for what ever reason (possibly he did not yet have the methodology fully planned out) Lewandowsky did not alert the ethics panel to the fact he was about to do something “unlike previous analyses of web content” using a “real time” process.
I certainly think heaping all the blame on the journal here seems unjustified without knowing the full details of what went on in the negotiations between the authors and journal.
Let me try to restate my point, which has been repeatedly missed. I do not wish to defend this particular article. But if a journal is going to retract an article, a retraction policy should involve transparency and honesty. If this article should have been retracted because it was a bad article or inaccurate, or if it was a mistake to publish it, that’s fine, just say so.
But it is a mistake to hide behind a policy that blames the retraction on improper use of human subjects because the article mentions the writings of humans and they might be identifiable from those mentions. As the original post states, this would make it impossible to do research in fields such as economics, sociology, history, literary analysis, art, music, political science, etc. As I stated in a comment above, it would seem to rule out meeting reports, book reviews, review articles and potentially even citations.
Suggesting a policy where we are only allowed to analyze the works of a human after they are deceased is equally absurd, and would result in an inability to progress or understand the current world.
Research involving human subjects and informed consent is particularly problematic in fields like Communication, which has a long history of doing media analysis. Informed consent was never an issue when analyzing newspaper and magazine content, and still isn’t for those looking at its modern Internet equivalents. Still, there is a strange grey zone one enters when one wishes to talk to another human being (interviews), or gather feedback from multiple respondents (surveys), even in informal groups (focus groups). While I understand the history of informed consent in the medical research world, applying it too broadly would make many studies in the social sciences too difficult to conduct within the ivory tower, meaning that these types of studies would simply move out of academia and into industry, or simply not done at all.
But does not the fact that this was a psychological analysis of specific people make it medical? Perhaps that is the central grey zone. I have not read the paper but it sounds like there was a finding of irrationality, which might be considered medical. Or does psychology straddle the line between medical and social science, with one foot in each? It is certainly sometimes medical.
There was no private information shared. It was all public. It was therefore a social psychology analysis.
The private information that was made public in this case would be the diagnosis itself, of the blog writers. But David C. says this is a common practice so there seems to be no issue here.
It’s not common practice. See comments in a different branch: ‘According to the common IRB rules “Research using sensitive information (e.g., information that reasonable persons would not want disclosed) that is identifiable is not exempt, even when it is publicly available.”’ This is (or should have been) an IRB issue, not a publication-level issue.
I couldn’t read the article (removed), but perhaps there was a problem with terminology. A person who posts in a conspiracy blog is not necessarily a “conspiracy theorist.” That is an inference which may or may not be demonstrable. They are merely a “person who posts in a conspiracy blog,” which is a fact. If that’s the problem, then a bit of good review and editing may have avoided the situation.
Editors and publishers have an obligation to see that authors get their facts straight and can support their conclusions. If you’re afraid somebody will sue because you publish the truth, you belong in another field.
“Imagine the fields that would vanish — anthropology, history, economics, sociology, literature — if we needed permission to study anything left behind by other humans.”
Anthropology, economics, and sociology already operate under the IRB process. Indeed, I’m guessing that certain aspects of history and literature do as well, for example when interviews or surveys of living persons are involved. Similarly for any field. The whole purpose of IRBs is to deal with this exact case, and the right thing to do is to err on the side of caution by asking your IRB whenever there are living people involved, and in some cases when dealing with the recently deceased as well, for example regarding DNA or letters.
It appears the publisher was cowered by some lawyers. What a shame. Will we all now be threatened by these people on the right and the theists who live among us? Science is controversial. Are our companies prepared to go to court?
Next week I’m teaching classes on intellectual freedom and intellectual property; self-censorship is a threat to both and this is a classic example. The research was sound and ethical but the results were offensive to a group of people who predisposed to be offended–Frontiers voluntarily censored itself out of fear and shame on them. Perhaps a more balanced approach would have been to suggest that those outraged replicate the study and publish their results and subject their findings to peer review and public comment. End of my reasonable approach and indulging in a face palm.
(There seems to be a (very reasonable) depth limit on comment branches, so this is in response to “DAVID CROTTY | APR 23, 2014, 12:41 PM”)
“But I would also argue that no human subjects were experimented on in this study, just as no human subjects are used in analyzing the works of Shakespeare. If i do a study in the depiction of lions in medieval art, does that qualify as an animal study and require IRB approval?”
I think that the IRBs would generally disagree with whether or not there were human subjects experimented on, but as noted by another commentator, there is room for disagreement. However, within reason (which your Shakespearian and medieval examples are not), if one has any doubt, one should at least drop an email to one’s IRB. If the IRB says that you don’t need to worry about it, then that’s IRB approval. If the IRB has any concern at all, submit the project and get them to give an opinion. And (again, within reason,) in the grey zone, editors should simply require at least an email from the IRB indicating that this no IRB approval is required. The paper in question was clearly in the grey zone.
This extreme caution would pretty much put an end to research. Okay, that’s an overstatement. Rather, it would cause extreme delays in research that would lead to a reasonable percentage not getting done.
I tried running a standardized survey that no one was going to look at the resulting individual responses by IRB, and it took 6 weeks to get the expected waiver. That was without everyone on campus taking your extreme approach and adding to their deluge of proposals requiring review.
That, and you seem to be unaware of a great deal of literary criticism of the last 50 years or more.
Yes, it would cause delays. I have generally found the IRBs that I have dealt with to be reasonable and rapid. However, I concede that that might not be universally true. It’s certainly universally true that I’m unaware of literary criticism at any time ever. However, unless literary criticism counts as an experiment, which seems unlikely, I doubt it is relevant to the present discussion.
David Crotty (3:07) wrote: “Sorry, can’t agree.[…]”
What you think doesn’t directly matter. The IRB standards are the law, and the IRBs are the police. If you think you’re about to break the law, it would be wise to ask a cop first. If you don’t agree with the law, go through the process of having it changed. Flout it at your own risk.
I just went to the paper itself, which is available in various forms via google scholar search.
(The version I looked at was reached here:
accessed just now.)
There is no obvious mention of having gone through any sort of IRB process (although I only skimmed for keywords).
@David Crotty – I’m replying here given the thread depth limitation.
You say: “An article analyzing the motivations of Obama based on events described in his autobiography seems fair game to me.”
Are you sure? Do you seriously think that someone could or would publish, now, a peer reviewed article that scientifically characterizes the psychology of Barack Obama based on information extracted from his published biography? I think you’re missing the difference between opinion pieces (where everybody can advance his own opinion about any subject) and a scientific paper that attempts to establish a certain, universal and impersonal truth. I’m pretty sure this distinction is immediately obvious to anybody who considers the matter seriously.
I will not accept a complete ban on all analysis of living persons or artifacts left behind by human beings. Any study in any field that reaches any conclusions must be accurately done, must pass peer review and must pass the established guidelines for that field. If an “article that scientifically characterizes the psychology of Barack Obama based on information extracted from his published biography” can meet those criteria, then I have no problem with it.
You can argue the merits of this current study, and debate whether it meets those criteria (and I’m not making an argument for or against this particular paper). What I am arguing is that blanket statements that disallow examination of the public record because it was generated by humans is antithetical to science.
Although the resolution of this tif does not bear on the present post, and anyway is an issue for the IRBs, not for you two, or any of us, possibly this should be taken offline. However, it does suggest that a ripe domain for some social science experimentation would be to send experimental probes to IRBs, for example, replicating the case of the recursion paper, or of this little argument between you two, and see how various IRBs respond. (Of course you’d have to somehow get IRB approval for this experiment, which would be a tiny bit tricky, but I think that if you were to go in person to the head of the IRB and explain what you are doing, you might be able to work around that problem.)
the only lawyers that were involved were the authors and the journals…!
additionally, one or more of the co-authors at the time of the research were publicly attacking people named in the paper.
one of the co-authors – Michael Marriott – was publicly attacking people named in the paper – before, during and after the research period.. (he has said he is at ‘war’ with the sceptics
he was also directly interacting with them, attacking the, goading them etc, on the blogs he was ‘researching’ no declaration and a deception
the ‘ethics approval from UWA -(such that it was) stated – Observe – no direct participation or interaction of any sort.. Both Marriott and Cook are involved (Cook the founder) in the Skeptical Science website, and Cook is on record stating that people named in the paper, are his direct opposition..
this is not a story about whether, comments can be analysed by researchers, but the conduct of the authors and whether they were conflicted.
Perhaps the author of this article has only mainly heard one side of this story (the authors)
The co-founders of Frontiers – Professor Henry Markram ( a neuro-scientist) wrote an additional article about this paper.
in the comments he added this:
My own personal opinion: The authors of the retracted paper and their followers are doing the climate change crisis a tragic disservice by attacking people personally and saying that it is ethically ok to identify them in a scientific study.
They made a monumental mistake, refused to fix it and that rightfully disqualified the study.
The planet is headed for a cliff and the scientific evidence for climate change is way past a debate, in my opinion. Why even debate this with contrarians? If scientists think there is a debate, then why not debate this scientifically? Why help the ostriches of society (always are) keep their heads in the sand? Why not focus even more on the science of climate change? Why not develop potential scenarios so that society can get prepared? Is that not what scientists do?
Does anyone really believe that a public lynching will help advance anything? Who comes off as the biggest nutter?
Activism that abuses science as a weapon is just not helpful at a time of crisis.” – Henry Markram
This is interesting because it shows that skeptics are not the only one’s offended by the paper. Markram’s “headed for a cliff” position is actually rather extreme on the warmer side. (I study the climate debate.) What we see here is an instance of the relatively widespread belief, on both sides, that the virulence of the debate is damaging the public view of science. It is certainly historic. Each side loudly calls the other side “anti-science” and this is tearing science apart in the public mind. It is like a public trial that has gone on for decades.
This is a very muddled post.
You seem to have been misled by the stream of media stories by activists presenting their own distorted version of the story, and perhaps by your own prejudice and bias.
How many articles in the media are there written by those who criticised and complained about the paper?
Frontiers did not “collapse like a wet taco” and were not “intimidated into retraction by anti-science forces like those identified in this paper”. They have made this quite clear in two statements following the retraction. These are false claims put about by Lewandowsky and his apologists.
Two of the people falsely labelled as conspiracy theorists were professors of climate science. Are you saying that Professor Richard Betts of the UK Met Office is an “anti-science force”?
It’s really very simple. Calling people “climate change denier” or a “conspiracy theorist” is derogatory and could lead to harm to them, which is contrary to all the basic ethical principles of the field. Again, this is all explained in the Frontiers statements.
What is remarkable is that these false claims keep resurfacing, and the obvious basic facts are repeatedly missed. Now that is a phenomenon worthy of a psychological study (it’s related to what is sometimes called “noble cause corruption”).