Editor’s Note: Today’s post is by Tony Ross-Hellauer, Samantha Hindle, Gary McDowell, and Jessica Polka, who are early-career researchers with backgrounds in information science (TRH) and biomedical sciences working at Know-Center, bioRxiv, Future of Research, and ASAPbio, respectively.
Scholarly Kitchen posts often serve up predictions of how technological, social, and political trends will affect the publishing industry and the researchers it serves. Sometimes these prognoses are negative, and a common theme is how researchers might be harmed by processes of change. As advocates of open practices, however, we believe that sometimes it is not the effects of new processes themselves that are the risk; rather, it is the uncertainty caused by unclear policies during periods of transition.
TRANSPOSE is a grassroots initiative to build a crowdsourced database of journal policies to bring clarity to three areas: open peer review, co-reviewing, and detailed preprint policies. Our goal is to foster new practices while increasing awareness among authors, editors, and other stakeholders, and we seek to provide resources to assist journals in setting, sharing, and clarifying their policies. After getting started at the Mozilla Sprint earlier this year at the recent Triangle Scholarly Communications Institute in Chapel Hill, North Carolina, we’d like to invite everyone to contribute and, in particular, we are seeking verification input from journals and publishers.
Why is a resource like this needed?
Consider the case of a researcher wanting to publicly post a preprint of their manuscript. Given the relative newness of preprints in some fields (such as the life sciences), and the range of different preprint acceptance approaches adopted by journals, the authors would need to seek out each journal’s preprint policy to ensure that posting a preprint will not jeopardize the acceptance of their manuscript by any of the multitude of journals on their wish list. Thankfully, certain aspects of preprint archiving information can be easily found at SHERPA/RoMEO. But which preprint servers does the journal support? What licenses can be used? Which versions of the manuscript can be posted without compromising the journal’s publication process? And what types of blog or media coverage of the preprint would constitute an unacceptable breach of journal embargo? Varied and vague policies create a minefield that confounds author choices, and these constraints multiply with each additional journal considered.
The distributed, inconsistent way that journal policies are reported make it difficult for authors to find publishers with desirable practices, and even slows the appreciation among authors that different approaches are possible.
Furthermore, policies vary not only in their substance, but also in their location. Sometimes they are found in instructions to authors, other times in more obscure places on a journal’s website, and not unfrequently spread across multiple locations as well. The path of least risk and resistance to the researcher may simply be to not post a preprint at all.
The situation is worse if a researcher wants to select journals based on practices for which there are no databases, such as peer review (at least for journals that are not listed as partners with Publons). For example, if a graduate student is asked by their advisor to help with a peer review, they might want to know if the journal allows this and whether the review form enables them to be acknowledged (e.g., via a text box to name co-reviewers in the review submission form). At present, the only way to find this out is to either contact the journal directly or to find someone with experience reviewing there. Alternatively, our graduate student may wish to submit to a journal that will anonymously publish the content of peer reviews (believing that these will be more constructive, well-prepared, and professional) or to find a list of journals that publish peer reviews to read good and bad examples as a training exercise. The distributed, inconsistent way that journal policies are reported makes it difficult for authors to find publishers with desirable practices, and even slows the appreciation among authors that different approaches are possible.
Researchers are not the only ones who can benefit from taking a birds-eye view of journal policies.
Benefits to editors and publishers
It is difficult to track the prevalence of emerging policies. This muddies understanding of how common and well-accepted those policies are and can slow progress, even when the need for change is obvious. Journal editors or publishing staff might want to benchmark their journal’s peer review and preprint policies against others to best serve their research communities, as well as to remain competitive against other journals in their field. Additionally, crafting updates to journal policies is understandably not on the top of the priority list for many busy editors; having a handy single source of a range of journal preprint and peer review policies will lighten their load and help to streamline the process. Furthermore, schematizing policies can increase awareness among authors who sometimes find it difficult to sift through lengthy instructions to authors. Making this important information available in an easy-to-find, author-friendly format may increase submissions and, at the very least, will assist in author compliance. Moreover, the community as a whole will benefit from the ability to track trends and changes in policies over time.
A good example showing the value of such information for academia can be found outside of publishing: graduate programs in the life sciences dropping their requirement of the Graduate Research Examination (GRE). Preparing for the GRE can act as a financial barrier to graduate school. Despite a wealth of evidence that the GRE is powerless to predict a student’s future performance in graduate school, few programs seemed willing to drop it as an application requirement — until the past year. Joshua Hall, the Director of Graduate Admissions and PREP in the Office of Graduate Education at UNC Chapel Hill’s School of Medicine, created the #GRExit hashtag and began cataloging the schools and programs that had dropped the requirement for their applicants. This information about the actions of peer institutions — in a format that is easy to access and visualize — was used to encourage other programs to consider a change, and arguably accelerated the tide of #GRExit.
Moving this back to the publishing world, there are many researchers in the basic life sciences who assume that journals will not consider manuscripts that have been posted as preprints, and as a result, they are reticent to experiment with this medium. Right now, their resources for broadly understanding journal policies are limited. Wikipedia has an impressive list of journals’ preprint policies that can help correct inaccurate assumptions. The color-coded list is a fantastic resource, but it is far from comprehensive, and it is limited by a lack of structure and visibility among researchers. It’s helpful for authors who would not have the energy to sequentially look up policies on websites or SHERPA/RoMEO, but we believe we can take these efforts one step further.
Gathering more granular policies (for example, which versions can be posted as preprints, and how those preprints can be covered in the media) will also enable future exercises in gauging the growth or decline of such practices. Furthermore, a survey of existing policies can make sense of confusing, conflicting, or vague terminology. One of us (TRH) has undertaken a systematic study of the term “open peer review” and teased it apart into seven different dimensions. Similar efforts could be undertaken to create a taxonomy of other journal practices, too. That work could enable the emergence of more precise terminology that will facilitate further experimentation and change.
More pragmatically, making lists also enables actors to compare notes about the outcomes of policies. To continue the #GRExit case, faculty now have a list of programs to contact if they want first-hand reports of the effect of making a policy change. This is greatly expedited by collecting this information in one place. Similarly, a list of journal policies could help editors identify colleagues with experience in the practices they are considering implementing. For example, the experiences shared by signatories of a letter on publishing peer review can address common fears associated with making this transition.
Won’t writing policies down rigidify practices, and potentially slow change?
Some might argue that codifying policies could stifle rather than accelerate progress by either generating a herd mentality in which journals simply copy what others are doing or discouraging authors from pushing the envelope by breaking policies they don’t know exist. We acknowledge that both of these effects could occur but believe they will be more than outweighed by the greater visibility for new models and reduction of uncertainty caused by unclear policies, not to mention the increased conversation and attention that will ensue.
What if the listings go out of date?
TRANSPOSE is not intended as a stand-alone piece of infrastructure. Our interest is in fostering new practices while mitigating risks through provision of information. Our aim is hence to collect urgently-needed in-depth information, use it to define schema for categorizing journal policies, and maximize its potential for reuse. All original information collected will be released under the CC0 waiver, and in addition to being periodically archived (e.g., on Zenodo), we intend to eventually explore opportunities to merge it into more permanent initiatives.
What if a volunteer enters incorrect information about your journal?
While we expect that volunteers will act in good faith, we are happy to address quality concerns (please feel free to get in touch at firstname.lastname@example.org). However, the most expedient way to ensure accuracy may be for journal editors to contribute their own journal-verified record here. “Journal-verified” records will be displayed as such in public versions of the database, and links to edit those records will be hidden from public view. We will periodically monitor the records and notify the provided email addresses about the presence of the records to ensure that the information comes from a genuine source.
Requesting your help
TRANSPOSE uses Google Forms to allow anyone to update journal policy records stored in Google Sheets. This allows contributors to build a complete journal policy record from multiple submissions, giving us the potential to crowdsource information. To find a list of all existing journals or to add a new journal, please click here. It’s easy to create or verify records via Google Form — we estimate this will take between 5 and 15 minutes per journal. By making this process as easy as possible, we hope that users will find it trivial to update TRANSPOSE when submitting to (or reviewing for) a new journal. The TRANSPOSE team will periodically review and moderate these records.
The most valuable source of information is of course the journal editors themselves. Editors are most familiar with the policies and practices of their journal and are equally positioned to benefit from increased awareness of their forward-thinking policies. Therefore, we not only welcome contributions from journal staff but would be delighted to highlight these first-hand contributions. At the end of the form, you will be able to assert that you are a journal representative.
Scholarly publishers, as stewards of the scientific record, have a great deal of power to steer researcher behaviors. Unclear policies could unnecessarily limit uptake of new practices, hinder scientific progression, and deter authors from submitting to your journals. In contrast, increasing awareness of forward-looking, author-friendly policies could draw in new submissions. Please join us in making the submission process easier and rendering journal policy information more transparent to those who need it!
Thanks to Alison Mudditt and David Crotty for helpful feedback on drafts of this post.