Sage The Bad Naturalist saw a TikTok showing what seemed to be an implausible scientific experiment, so she set out, in the best manner of the research world, to debunk it by seeing if it was reproducible. And then, well, things went awry. Really awry.

Watching the video, I was reminded of a post from 2017 on the significant level of knowledge and ability required to accurately reproduce many experiments. The two lessons of that post remain ever important to this day:

  1. To accurately reproduce an experiment, you need to know the detailed methodology that the original researcher used. This is why Open Methods are likely more valuable to reproducibility than Open Data (even if I can see your data, I don’t know if it’s any good if I don’t know how you collected those data).
  2. Even with a detailed experimental protocol, it can take a significant investment of time and effort to master some experimental techniques. That graduate student you’re trying to debunk probably spent years of their life mastering that particular experimental method — are you willing to put in the same effort?

Regardless, hopefully you’ll enjoy Sage’s effort (and realization of how far off she was). Also a trigger warning for those made queasy by things a moldy and slimy.

David Crotty

David Crotty

David Crotty is a Senior Consultant at Clarke & Esposito, a boutique management consulting firm focused on strategic issues related to professional and academic publishing and information services. Previously, David was the Editorial Director, Journals Policy for Oxford University Press. He oversaw journal policy across OUP’s journals program, drove technological innovation, and served as an information officer. David acquired and managed a suite of research society-owned journals with OUP, and before that was the Executive Editor for Cold Spring Harbor Laboratory Press, where he created and edited new science books and journals, along with serving as a journal Editor-in-Chief. He has served on the Board of Directors for the STM Association, the Society for Scholarly Publishing and CHOR, Inc., as well as The AAP-PSP Executive Council. David received his PhD in Genetics from Columbia University and did developmental neuroscience research at Caltech before moving from the bench to publishing.

Discussion