Trust is the theme of this year’s Peer Review Week, and we can’t think of anything more important or timely.  Peer review runs on trust. Trust is both a noun and a verb; both are central to how knowledge develops and is shared through research. And yet trust seems in short supply in our fractured and fraying world. Trust in institutions and processes was always a privilege, and people for whom processes and institutions have never worked well — or even worked at all — may never have trusted either of them. As we learn in more detail about how science has reflected the same biases as society, including racism and sexism (a prominent example is the history of Henrietta Lacks and HELA cells), is it any wonder that trust is in short supply? But there are also new and potent forces for disinformation, including social media platforms and politicized traditional media to spread it. Anti-science movements, whether anti-vaccination or anti-mask wearing, come from a different but equally long history of mistrust.

We can’t offer an answer to the many and enormous challenges our societies are facing, but from our different personal and professional perspectives, we suggest that it is crucial for scholarly communications to elevate trust in all we do, and to operate with both the noun and verb form of trust front and center. With this in mind, we have scoped out what we think are the key elements that constitute trust in peer review, how they relate to trust in research more broadly, and what processes are essential to maximize trust.

It is critical to both research and dissemination that review is trusted. The classic peer review process, of a piece of research for publication, involves the identification of appropriate reviewers, ethical review of the research by the reviewers, and editorial oversight of the reviewers’ selection and conclusions. But trust is baked into a much larger set of review practices, from grant funding to awards and more. We need to have confidence that these practices and processes, while never a certain guarantee, will both minimize risks and add quality.

Trust — and mistrust — in review practices is iterative and cumulative. Previous Peer Review Weeks have focused on quality, diversity, and transparency — all of which are central to trusted practice, we would argue. When we have ethical practices, we earn trust; when we act on behalf of ethical review we are investing in a trusted practice.  We act, and we invest, because of a sense of a larger trust — a public trust in research and scholarship to act for the greater good.

How does trust in peer review relate to trust in the whole research process?

Surveys continue to show that, despite some partisan and demographic differences, a large majority of Americans, for example, still trust scientists over pretty much every other leadership group. However, the level of the American public understanding of science is lower. Only two thirds of respondents to this 2019 Pew survey saw science as an iterative process, ranging from 56% of respondents with no higher education to 85% of those with postgraduate education. Alarmingly, 15% of respondents overall believed that the scientific method “produces unchanging core principles and truths,” including 9% of postgraduates! How the public understands and trusts other kinds of research, for example, in humanities and social sciences disciplines, is more ambiguous.

Improving public understanding of the research process in all disciplines — and the central role that peer review plays in it — is critical, in order to increase public trust in research findings. Without trust, we cannot move knowledge forward. As noted previously, we’re not just talking about peer review of a journal article or other publication, we’re talking about the peer review that happens throughout the whole research ecosystem: from the graduate school application process through defending a dissertation; from applying for a grant through hiring, promotion, and tenure; and from evaluating a dataset to reviewing a manuscript. Does this guarantee that all research results will be error-free? Of course not! But, in the same way that a quality assurance process builds trust by minimizing the risk of faults in an appliance, so the peer review process does the same for a piece of research (or, indeed, a researcher). Not only that, but peer review is also additive — an essential part of the iterative process of constant improvement, through trial and error, that constitutes good research. The continual feedback provided by peer review in all its forms enables more people to participate in the research process at every stage, and to improve knowledge production — and trust — with each input they make.

Building trust through this process of constant checking and improvement is vital not only for other researchers in the field, but for the wider research community, as well as for policy-makers and the general public. While fellow researchers need to trust that research in order to cite it, or build on it in their own work, policy-makers rely on it in order to make decisions that may have literally life-or-death implications (think COVID-19 again — and not just from a medical perspective, but also in terms of the socioeconomic impact). And, in order for those policy decisions to be effective, the general public also needs to have faith in the research on which they’re based. Researchers — and, arguably, scientists in particular — don’t always help their own case, by talking in terms of possibilities rather than absolutes. This makes sense, given how the scientific process works, but it can, at best, cause confusion and, at worst, increase the spread of misinformation (think the use — or not — of masks to prevent COVID-19). For disciplines like history, contingency and complexity — in other words, few absolutes — have their own costs in terms of public perception.  Exposing the process of scholarship helps all round.

What does ethical peer review look like in practice?

Peer review is a process that depends on the application of trust. It requires that every person involved behave responsibly and ethically. By consistently engaging in the process, again and again, we work to improve it; and, when needed, we develop more trustworthy practices. Submission by submission, our organizational resources should display that our processes are valid and that we are practicing ethical peer review. When implementing our processes, we may often overlook the practices, which might not align. In being overly focused on the process the measurement of trust is skewed. Success becomes defined by what we have on paper — policies and procedures — but in practice we’re failing. Like anything we put into practice, peer review should be a method of learning and of acquiring experiences, and of taking those experiences and reassessing if there are better ways of doing what we do.  Peer review should transform research into solid and trustworthy foundational knowledge. In order to ensure we are practicing ethical peer review, we cannot assume that our processes and practices are equal.

There is often a tendency to equate the practice of peer review to the process of peer review. However, the practice provides more than the process. In the practice of peer review we may find it necessary to leverage many processes in support of those practices. Looking at this from a journal publisher perspective, there are several processes involved as part of the practice to help our organizations perform and maintain ethically sound review. The practice concepts introduced in the COPE Ethical Guidelines for Peer Reviewers provides organizations with the framework to develop adaptable principles and standards.  It also helps to guide researchers, journals staff, editors, and other research participants on the practices that must be followed to safeguard against unethical peer review processes.

Following a practice-based approach to peer review allows organizations to leverage concepts that help build and evaluate trust, such as transparency — operating in such a way that it is easy for others to see what actions are performed; and, it implies openness, communication, and accountability. If reluctant to expose any internal actions, policies, or procedures, then you are likely lacking transparency in that area. Work to expose these spaces. Don’t allow doubt to settle into any part of your process, thereby giving way to mistrust. Are you unsure of where distrust can creep into peer review and start to deteriorate trust? Here are few examples to think about:

• Not revealing conflicts of interests
• Power dynamics being abused, such as graduate students actively performing reviews, but not being recognized as official reviewers
• Outdated policy and procedures, which does not account for new data/information
• Undervaluing the publisher’s role in the academic ecosystem, statements such as “they don’t know what they’re doing so let’s do it ourselves”
• Not properly understanding the rules and policies and/or the rules and policies being opaque, hard to find, or jargonistic
• No on-boarding or training, which leads to not knowing if everyone is on same page
• No clear career pathing, lack of transparency in the selection process, which leads to questions such as how did they get there, and, how do I get there
• Bias (conscious or unconscious) in decision-making

Failing to fully consider how to improve in any of these areas, ruptures trust in peer review. And once trust is broken, it is hard to re-establish. Mistrust is not always, or even often, associated with unethical peer review; but flawed peer review, will inevitably lead to mistrust. So challenge your practices. If necessary, force your practices  to contend  with inherent biases. Take time to pull out that old policy and procedure manual or revisit your instructions to authors. Dust off  past (mis)understandings of diversity, equity, and inclusion. Courageously tackle your data disparities, and be specific in your approaches to address problems. For example, when there is a lack of Black editors serving on your editorial boards, find the glitch in the system that allows this to happen. Show your community that you’re stronger than your weaknesses by taking ownership, being honest, and talking to them directly. Have radical conversations, accepting that they may be difficult. Remember that change begins at the end of your comfort zone.  Develop — or re-establish — better evaluation measures for competency, reliability, and integrity. Frequently assess your education and training resources. Lastly, keep in mind that it’s when you shore up your processes and procedures through meaningful actions, and step away from checking boxes that you begin to apply trust.

To end where we began, trust is a noun and a verb: to gain trust, we must act in ways that are worthy of trust. Our peer review processes must be ethical, and transparent, and fully communicated to the widest possible scholarly and public communities. And then we can, ourselves, trust and be trusted.

The infographic above and other Peer Review Week resources are available on the PRW website.