Editor’s note: Today’s post is by Donald Samulack and Jason Roberts. Donald is President, U.S. Operations of Editage, a division of Cactus Communications, and Jason is Senior Partner at Origin Editorial.

File this post under “looking back to look forward”, but we thought it prudent, one year after the quadrennial International Congress on Peer Review and Scientific Publication, and during this year’s Peer Review Week, to reflect upon a central theme from both occasions in 2017: transparency – also a theme of yesterday’s post.

peer review week logo

Rather than rehashing the detailed arguments and carefully stated nuances, we would like to propose it might be time for the industry (namely publishers, societies and bodies such as COPE, CSE, ISMTE, SSP, as well as academic organizations, industry organizations and system providers) to join together to provide formal – standardized – guidance for journals on how to implement a move toward greater transparency of journal processes. It is true that this plea is probably simplistic, and certainly idealistic, but the debate has persisted long enough to now warrant moving toward a solution.

So, are both the publishing and academic worlds clamoring for action? Well, yes and no.

Is it likely that agreement could be reached on what defines an acceptable level of transparency? In all likelihood, no, and maybe that is not desired anyway. Individual journals, because of scope or discipline, may want to set their own thresholds. But, perhaps this guidance we are calling for can at least propose the issues to debate internally, ranging from what information to disclose on your journal homepage, to how editorial office policies are enacted (for example, at its most basic level, precisely how text comparison reports are interpreted after using iThenticate).

Furthermore, while academics continue to debate loudly about greater transparency in peer review, it struck us that the typical editorial office is left wondering where to start to begin tackling the issue in a considered, consistent, approach. The more granularity that is added to the debate, the less likely anything will get done collectively.

Is it the responsibility of publishers to tackle this alone? We can answer this question ourselves: emphatically, no. But publishers working with other parties represent the best opportunity to broadly disseminate highlights of the debate and some straightforward principles or policies that every journal can work toward.

Before going any further, let’s stop to understand that transparency usually references “openness”. Essentially, there are two meanings in play here. The first, applies to ensuring various stakeholders (authors, reviewers, and editors) are transparent in their words and deeds (usually manifested through activities such as publishing peer review comments or requesting authors more precisely describe their study methodologies through the use of reporting guidelines such as CONSORT). The second meaning, quite literally, refers to being more open about how one handles a manuscript that has landed on your doorstep; describing your processes from triage to publication. It is this second definition that we wish to primarily focus on here, not least, because it seems like the most achievable for all journals.

The cold hard truth is that the average author has no idea what actual processes are involved in the handling of their manuscript. The Instructions to Authors tend to be very technical about manuscript structure and (perhaps to a lesser degree) calling authors to account for their behavior in preparing and submitting the manuscript (e.g., author contribution, access to data, etc.). On the other hand, authors do not know how initial editorial triage and desk rejection decisions are made. They do not know whether their manuscript will be screened for plagiarism and figure manipulation through the use of an industry tool, or how the results from such a screening are applied. They do not understand what criteria Editors and Reviewers are encouraged to apply when assessing a manuscript and, consequently, why it takes so long to complete peer review. They do not understand how ORCID has become an essential element in the “plumbing” of the publishing industry.

Referencing this year’s Peer Review Week theme of diversity in peer review, authors almost certainly know nothing about whether a journal takes steps to eliminate the potential for bias, wherever it may lay, conscious or otherwise.

If the author did understand all of this, and if a journal were to disclose all of these processes, this transparency would reflect well on the brand, and take the mystery (and possibly much of the “misery”) out of publishing.

Within the current publishing landscape, if a journal is looking to differentiate itself from others, or even from the so-called predatory or illegitimate journals, simply disclosing what you do, and how decisions are taken from the time the author submits the manuscript, to the point of publication makes you distinct. As the publishing industry moves toward greater transparency, the journals acting ethically will become easily differentiated from the journals that do not. The author will become better informed about such issues, and the decision to attempt to publish in one journal versus another will become a much more informed decision.

Furthermore, for those publications that may outwardly seem “predatory”, but in fact are simply struggling to meet good publication practice standards, an industry-directed set of criteria on transparency could help – and this is our crucial point. Journals of all kinds strive to fix procedural gaps, raise practice standards, or sharpen the focus on certain policies; especially those relating to publication ethics. Distilled, and easy to follow industry guidance related to transparency is very much needed.

So, why aren’t journals already doing this? Simply put, this is likely due to a lack of awareness regarding the issues. This is particularly true for the under-resourced editorial office, or the journal, where (for example), a new editor who, though quite brilliant within their research field, has received little-to-no guidance on how to move the journal on a path of greater transparency.

At the recent ISMTE meeting, we informally polled a small sample of session attendees (18 responded) and asked, “Have you created AND published a ‘transparency’ policy regarding your journal’s peer review processes?”; 17% (n=3) responded “Yes, for over a year”; 17% (n=3) responded “We’re working on it”; 33% (n=6) responded “No, not yet”; and 33% (n=6) responded “What is a ‘transparency policy’?” While this was an unscientific poll, the take-home message was that the full disclosure of internal journal policies is not something that is commonly done, thought about, nor even understood.

Why should we care? Every stakeholder in the publication process has an investment in making journals more transparent. For the industry, as a whole, transparency can equate to accountability. For authors and readers alike, such a move would allow for judgments to be made on how a journal validates the research it publishes. For reviewers volunteering their time to assess a submission to a journal they may be unfamiliar with, criteria could help them better position their critique. For editors and editorial offices, beyond the reasons we have already outlined, the implementation of such transparency criteria to develop policies could help protect them against accusations of peer review “rigging”. For publishers, though collectively they have not asked to be assigned the role of ombudsmen, a set of transparency criteria could help them identify journals that are potentially going rogue. In short, everyone should care!

Such a call for transparency is not new. Far from it. Indeed, the call itself was made just last year on the Scholarly Kitchen. COPE, naturally, has begun steps to address the issue. The July 2018 COPE Digest declared, “transparency is ‘mission critical’ for COPE”. Other initiatives exist as well, such as the ASAPbio open letter on the publication of peer review reports. Also, ideas have been floated in the past about developing a transparency metric, though that would require defining the assessment criteria. Regardless of all the discussions of what the criteria consist of, there must be an accompanying implementation strategy. There needs to be simple and clear documentation on why these issues are important. Perhaps this takes the form of a modular stakeholder table that tabulates a set of definitions and concepts regarding how each group intersects with, and is affected by, each criterion that is included in a transparency policy. It would be up to individual journals to implement the guidance that is relevant to them as they develop their own transparency policy; similar in practice to the Center for Open Science TOP guidelines, but with greater scope.

For journals, the scope of criteria may focus on clearly identifying who owns the journal, who makes decisions, a description of the decision-making process, and the model of peer review employed (e.g., is it double blind; is a statistician deployed; etc.), information on how reviewers are selected, the criteria used to assess a paper, expectations on what the editor wishes to see, publication ethics policies, and so on.

The criteria may also be used to better shape author and reviewer behaviors via journal practices; for example, perhaps a standard or routine inquiry on authorship, funding, competing interests, reporting standards (especially for clinical journals), or expectations for both good practice and incisive comment from reviewers.

As transparency, in all of its forms, starts to permeate the publishing industry landscape, and “open science” becomes more “open”, we really need to be open and transparent ourselves. We need to get our house in order, much like one does before inviting friends over for dinner. An orderly house in the case of a journal, includes full transparency, and that transparency translates into brand. And this “getting the house in order” does not mean metaphorically shoving things into cupboards to give the outward appearance of orderliness. It means genuine clarity in policy definition and enforcement, and being entirely clear in presenting this information.

As an industry, we need to move toward providing definitive guidance regarding transparency – for journals, authors, and reviewers. We can no longer push off coordinated efforts to become operationally transparent. Such guidance must be evidence — and consensus-based – perhaps driven by one or more of our industry bodies. We hope that this post will help begin this conversation and that, over the coming months and years, we can collectively start to develop standardized cross-industry guidance that everyone can support.

Once we, as an industry, turn this corner together, the benchmark of transparency, ethical standards of conduct, quality of publication, and differentiation away from journals that do not make such disclosures will become readily transparent.

We welcome your feedback and ideas!

Discussion

7 Thoughts on "Guest Post: Transparency – “This is What We Do, and This is What We Expect”"

It would be interesting to know:
a) the positions of the persons who were informally polled in your survey and how representative they are.
b) who do you anticipate will create the transparency document?
c) who of the VOLUNTEER peer review professionals do you expect to carry out such transparency processes
d) What additional fiscal resources will the journal publishers allocate to create and administer both the journal editors’ functions and that of the academics doing the actual review in order to expedite such in a timely fashion.

I can offer some initial thoughts to your queries (others may have views as well):

For (a), the survey was a live Sli.do interactive poll at the beginning of a session that Jason and I ran at the ISMTE meeting, addressing a similar scope of issues as this post. It was in a session of approximately 50 people, who were, for the most part, either managing or technical editors working at a wide spectrum of journals. The ISMTE organization provides practical learning for editorial office members. While there may have been some people in the room who were vendors, through a show of hands at the beginning of the session, I can’t remember any. Yes, the poll was truly unscientific, but it shed light on the need for the discussion.

For (b), I believe the transparency document tailored for a particular journal would likely be written by the Managing Editor (after full stakeholder input at the journal) and signed off on by the Editor in Chief, and deployed in a variety of ways, over time. It does not need to be (and should not be) a one-shot deal — it should be a living document that has visibility throughout a variety of operational touchpoints that stakeholders come in contact with (online systems, published opinion pieces, stakeholder communications, etc.). In my opinion, a transparency policy for a journal is not just a document, or series of statements. It is a culture of journal operations.

For (c), I do not see the peer reviewer activities changing in any major way, other than the peer reviewers complying to ethical disclosure criteria, and being accountable to their role as per activities requested by the journal. Which, in essence, is what they should be doing presently.

For (d), it is not that a call for transparency is a call for excessive burden or expedited timeliness. It is a call for transparency and accountability for what processes that exist. As processes change over time, that too should be conveyed in a transparent manner.

As an industry, we need to move toward providing definitive guidance regarding transparency – for journals, authors, and reviewers. We can no longer push off coordinated efforts to become operationally transparent. Such guidance must be evidence — and consensus-based – perhaps driven by one or more of our industry bodies. We hope that this post will help begin this conversation and that, over the coming months and years, we can collectively start to develop standardized cross-industry guidance that everyone can support.

Once we, as an industry, turn this corner together, the benchmark of transparency, ethical standards of conduct, quality of publication, and differentiation away from journals that do not make such disclosures will become readily transparent.

Just whom is making this clarion call for transparency? Should the author and the reviewer(s) be made known? If both, fear of retribution arises. Just how does OA and transparency conflate? What has one got to do with the other? Lastly, I thought ethics was rather comprehensively covered by COPE.

Just whom is making this clarion call for transparency?
— the article references many bodies who have made the call prior.

Should the author and the reviewer(s) be made known? If both, fear of retribution arises.
— unblinding is the call of the journal. The call for transparency of journal process is simply that. If unblinding is the process of the journal, then it should be clearly stated by the journal that such is the policy of the journal.

Just how does OA and transparency conflate? What has one got to do with the other?
— this is probably best addressed by the paragraphs in the article that point to how generally confused authors are about the journal processes, and the differences between a journal acting ethically/transparently, and one that is not.

Lastly, I thought ethics was rather comprehensively covered by COPE.
— as pointed out in the article, even COPE is making the call for transparency in the July 2018 issue of COPE Digest (a good read: https://publicationethics.org/news/letter-cope-co-chairs-july-2018).

Hi Harvey,
I think you might be confusing the two different definitions of “openness” given in the post. As noted above, this is not a call for “open peer review” in terms of revealing all identities and publicly releasing the peer review reports. It is instead a call for journals to be “open” or better, “transparent” about their practices. For example, a journal could say, “Submitted articles will go through desk review by one of the following people, followed by single blind peer review by X experts selected by this method, along with analysis by plagiarism and image manipulation tools.” Is that too much to ask?

As we close out Peer Review Week for this year, I thought I would add a few itemized thoughts surrounding what I think is important regarding issues to address in a journal transparency policy as it relates to the journal, authors, and peer reviewers; many of these concepts have already been discussed in the article. It is my hope that by offering these bulleted points – under each could be several other sub-bullets – it prompts others to add their related thoughts as well.

For Journals:

• Ensure that the journal owner (e.g. Society) and publisher are clearly identified
• Clearly identify the Editor/Editorial Board and their institutional affiliations
• Describe, in detail, each stage of the manuscript handling process and how decisions are made on manuscripts
• Explain what type of peer review model is followed, how reviewers are selected (mitigating for bias of any kind), and how (if any) peer reviewer recognition is to be managed
• Describe in detail how the journal handles potential conflicts of interest or ethical irregularities related to the manuscript, authorship, etc.

For Authors: (beyond the traditional Instructions to Authors)

• Follow and ethically abide by checklist-driven declarations regarding processes that you can expect regarding how your manuscript will be handled, as well as a definitive guidance of how the journal expects you to have behaved prior to, and at the time of submission
• Define how authorship was determined, versus other forms of attribution or acknowledgement
• For submission to clinically oriented journals, ensure that you have followed specific reporting guidelines like CONSORT, PRISMA, STROBE, or others

For Peer Reviewers:

• Follow and ethically abide by checklist-driven declarations, to disclose any forms of bias, competing interests, or conflicts of interest
• Declare to the journal the amount of time you will require to review the manuscript and submit your review, as a criterion for accepting to review the manuscript
• Declare acceptance and accountability to the publication timeline if the review is not submitted on time
• Define expectations (if any) of how peer reviewer recognition is to be managed

Leave a Comment