Editors’ note: Today’s post is by Chef Ashutosh Ghildiyla, Mathïs Fédérico, and Maria Machado. Mathïs is a French engineer whose career has spanned academic research, industry R&D, and artificial intelligence. Maria is a physiologist turned consultant who has explored different formats of peer review, and attempts to bridge the gap between researchers and the academic publishing industry.

Scientific progress depends on two complementary activities: exploration and validation. Yet, our systems of scholarly evaluation have become better equipped to validate established forms of knowledge than to recognize contributions that challenge them.

Exploration generates new questions, hypotheses, conceptual frameworks, and connections between ideas. Validation subjects claims to systematic scrutiny, determining whether the available evidence and methods provide sufficient support for confidence in the claim within a defined scope and against current standards of evidence. It does not establish that a claim is complete or universally true.

Modern scholarly communication has largely evolved around the requirements of validating empirical knowledge. As science became an increasingly trusted source of knowledge for public policy and decision-making, the need grew for reliable mechanisms to distinguish robust evidence from speculation. Scholarly publishing has developed practices such as peer review, methodological reporting, and efforts to improve transparency and reproducibility to support the systematic assessment of empirical claims. But is the way we evaluate scholarship equally well suited to every kind of scientific contribution?

Different Contributions, Different Questions

When every contribution is routed through the same evaluation model, reviewers may spend substantial time applying criteria that are not central to the work’s intended contribution. Aligning evaluation with purpose can direct reviewer expertise and effort toward the questions that matter most for the work being assessed.

• For validation, the central question is whether the evidence provides sufficient support for confidence in the claims. Therefore, its evaluation focuses on the adequacy of the evidence, methodological soundness, and reproducibility.
• For exploration, the central question is whether the work offers an original, intellectually coherent, and potentially significant contribution to understanding, even when its central ideas have not yet been empirically validated. Evaluating this type of work may need to focus on the quality of the reasoning, the coherence of the conceptual framework, the originality of the connections proposed, engagement with relevant knowledge, the treatment of counterarguments, and the potential to open a productive line of inquiry.

In both cases, the contribution must provide rigorous reasoning to support its claims or propositions. What counts as sufficient support, however, depends on the purpose of the work — exploratory contributions may use thought experiments, conceptual analysis, or other forms of reasoning to ground a proposition and define what would need to be tested, while validation-oriented work relies on evidence generated to test the claim.

The Paradox of Expertise

Innovation often begins by questioning assumptions that have become so deeply embedded within a discipline that they are no longer recognized as assumptions. Thus, ideas that challenge those assumptions can be difficult for scientific communities to recognize. Thomas Kuhn’s account of paradigms captures one part of this tension: expertise helps communities define legitimate questions, acceptable methods, and convincing evidence, but those same frameworks can make departures from established expectations harder to recognize on their own terms.

This tension is particularly well illustrated by Alfred Wegener’s theory of continental drift. In 1912, Wegener proposed that the continents had once been joined. His hypothesis challenged prevailing geological thinking and lacked a convincing mechanism for continental movement, so it was widely rejected. Decades later, advances in plate tectonics provided evidence and a mechanism that supported Wegener’s central insight. The lesson is that the absence of evidence sufficient for validation at a specific point in time does not necessarily settle the value of an exploratory contribution. 

Peer Review Within an Unfolding Science

If knowledge is continually evolving, peer review cannot establish that a contribution is finally true or complete. It is a form of critical examination applied to a contribution at a particular point in the development of an inquiry.

Peer review can also contribute to the development of inquiry itself. Reviewers can help authors examine assumptions, clarify reasoning, consider alternative interpretations, and recognise possibilities that may not have been apparent within the original framing of the work. This does not make peer review less rigorous; it recognises that critical examination can serve both the assessment of a contribution and the development of the ideas within it.

A paper that has undergone peer review becomes part of the accumulated record while remaining open to further observation, criticism, replication, reinterpretation, and discovery. Its conclusions may be strengthened, refined, challenged, or overturned as the scientific frontier moves. Peer review is, therefore, one stage in a continuing process of examination rather than the endpoint of evaluation.

Seen in this way, the question for reviewers is not whether a paper represents the final word on a question, but what the contribution currently establishes, what remains uncertain, and whether its reasoning and evidence are sufficiently sound, transparent, relevant, and intellectually defensible to support further scholarly examination. What constitutes sufficient scrutiny, however, depends on what the contribution is intended to accomplish. This is where purpose becomes important: different contributions require different questions, forms of evidence, and expressions of rigor.

Purpose Shapes the Expression of Rigor

If peer review is one stage in the continuing examination of knowledge, then the question is not simply whether a contribution meets a universal checklist. It is also what the contribution is attempting to accomplish within that process. A study seeking to establish an empirical claim, a conceptual paper proposing a new framework, and an interdisciplinary synthesis connecting bodies of knowledge may all be rigorous, but the grounds on which their rigor can be judged are not identical.

A natural objection follows: doesn’t broadening the criteria for evaluation create a loophole, allowing weaker work to escape scrutiny simply by labeling itself “exploratory”? It would, if purpose-aligned evaluation meant replacing evidence with subjective judgments of originality. Thus, the safeguard must be operational: the article’s purpose should be explicit, the relevant assessment criteria should be visible to those assessing the work, and it should be clear what further development, evidence, or scrutiny would be needed for the contribution to progress.

Scholarly rigor has two complementary dimensions that are particularly relevant here (see Figure 3). Both are essential to scientific progress, but their relative emphasis can vary with the purpose of the scholarship. They are different expressions of the same underlying values: intellectual honesty, transparency, critical inquiry, and openness to challenge.

As AI makes aspects of procedural and technical competence easier to automate, those features become less reliable proxies for scholarly value. Evaluation must then look beyond the fluency or technical quality of an output toward questions of originality, significance, contextual judgment, and conceptual contribution. Purpose does not loosen the standard of rigor; it changes what rigor requires you to look at (see Figure 4).

Three Principles for Editors and Publishers

For publishers and editors, this is also a question of how scarce editorial and reviewer capacity is used. As submission volumes grow and reviewer availability remains constrained, the challenge is not simply to review more work, but to ensure that available review capacity is directed toward the questions that matter most for each contribution. A purpose-aligned approach can help by clarifying what a work is intended to contribute and, consequently, what reviewers need to assess.

1. Identify the purpose of scholarship, not just its format. 

Empirical research, conceptual scholarship, and interdisciplinary synthesis make different kinds of contributions to scientific progress and should not automatically be evaluated using identical criteria. Exploration and validation describe different purposes and stages of inquiry; they do not map neatly onto article types. A conceptual paper may be exploratory, while empirical research may itself be exploratory or validation-oriented, and a single work may combine empirical, conceptual, and integrative elements. Hypothesis generation may form part of conceptual or interdisciplinary scholarship, while metascience describes a domain of inquiry rather than a distinct contribution type.

Journals should define the primary purpose of each form of scholarship in their author guidelines alongside article types such as editorials, reviews, and case reports, including what editors, peers, and the community are expected to assess. The purpose should determine the primary questions asked of the work: is it establishing sufficient support for a claim, advancing conceptual understanding, reframing a problem, generating a hypothesis, integrating knowledge across fields, or examining how knowledge itself is produced?

2. Match the evaluation model to the contribution being assessed.

Research whose primary purpose is to establish or test claims should continue to undergo rigorous peer review appropriate to the nature of the research, examining the strength and limitations of the evidence and methods. Other forms of scholarship may benefit from different approaches, including editorial assessment, review by non-academic experts or other interest-holders, structured scholarly dialogue, or post-publication review.

For exploratory or conceptual work, structured scholarly dialogue could complement formal peer review by creating an opportunity for authors and reviewers to examine assumptions, clarify the contribution, question alternative interpretations, and explore where the argument might lead. The purpose would not be to reach agreement, but to subject the contribution to critical examination while also allowing the interaction to generate questions or possibilities that may strengthen the work. This approach draws on David Bohm’s concept of dialogue, in which participants examine the assumptions underlying their understanding.

Such a process would not replace evidentiary evaluation where validation is required. Nor would it imply that exploratory work is exempt from rigorous assessment. Rather, it recognizes that peer review can serve more than one function, namely examining what a contribution currently establishes against standards appropriate to its purpose, while also helping that contribution develop through engagement with other perspectives.

Some existing models, including Registered Reports, eLife’s published peer reviews with editorial assessments, and F1000Research’s open post-publication review, demonstrate that scholarly evaluation can take different forms and be structured around different stages and purposes of inquiry. Whatever model is used, journals should clearly communicate how the work was evaluated and the standards applied.

3. Hold every evaluation pathway to explicit and enforceable standards of rigor.

Methodological robustness, conceptual coherence, transparent reasoning, evidential support, critical engagement, and openness to alternative perspectives are essential to rigorous scholarship. The relative emphasis placed on these characteristics may differ according to the work’s purpose, but the standard of rigor should remain high regardless of the evaluation pathway.

For exploratory work, a reviewer should be able to flag issues in a submission like:

• its central proposition is unclear;
• its reasoning depends on unsupported leaps;
• relevant evidence or credible counterarguments are ignored;
• its assumptions are hidden;
• its novelty is asserted rather than demonstrated;
• its claims extend beyond what the argument can support;
• its proposed contribution has no plausible significance.

An exploratory contribution is ready to enter the scholarly conversation when:

• it states a clear question or proposition;
• distinguishes evidence from conjecture;
• makes its assumptions and limits explicit;
• engages seriously with existing and competing explanations;
• develops a coherent line of reasoning; and
• shows why the resulting idea could productively change future inquiry.

Peer review is a judgment about what a submission establishes against the standards appropriate to its purpose at that time, not a final verdict on an idea’s ultimate validity or truth. A contribution enters a scientific record that remains open to further observation, criticism, replication, reinterpretation, and discovery. Purpose-aligned evaluation allows a contribution to be examined according to the question it is intended to address, while making clear what remains uncertain and what further evidence or scrutiny could deepen the contribution.

Putting This Into Practice

Here is one concrete way to start: pick a single article type your journal publishes often and examine the criteria reviewers currently use to evaluate it. Do those criteria actually assess what that work is intended to contribute, or are they simply the same checklist applied across different kinds of scholarship?

Editors can also look at their own recent experience. Rather than asking whether they support the idea in principle, look at specific cases. Where have reviewers applied criteria that were not central to a submission’s intended contribution? Where has a potentially valuable submission struggled because its contribution did not fit the journal’s usual evaluation model? Where have editors had difficulty deciding whether a work was rigorous because it did not fit familiar categories? These experiences can help reveal where the current approach works, where it does not, and whether a different approach might be useful.

The aim is to ensure that rigorous standards remain high while evaluation is responsive to the purpose of the work. This can help valuable contributions receive the scrutiny they need and give important new ideas a fair opportunity to enter the scholarly conversation. Good evaluation should help valuable science emerge, not inadvertently hinder it.

We would like this piece to start a conversation about how scholarly work is evaluated. Tell us where you agree, where you don’t, and what you have already tried in your own journals or communities.

AI Use Disclosure: AI tools were used to assist with language editing and to support the development of some visual concepts, which were subsequently translated into final images by a designer.