Today’s post is by Leslie McIntosh, PhD  Leslie is an academic turned entrepreneur, she founded Ripeta (now part of Digital Science) in 2017 to improve research quality and integrity by developing trust markers. She is currently the VP of Research Integrity at Digital Science focused on trust and mistrust in science. 

While the responsible conduct of research has always been a pillar of the scientific process, in a growing computational research and publishing world, methods for identifying and analyzing misconduct have evolved in such a manner as to warrant their own academic field of research. As an active participant in the community of research integrity, including sleuthing, I have come to recognize the increasing importance of defining this field of inquiry for ourselves and our colleagues and suggest the name for this emerging field: Forensic Scientometrics (FoSci).

Erosion of public trust in science and scientific research threatens the adoption and implementation of critical policy, technology innovation, clinical outcomes, and national and global economic and security interests across the spectrum of public life.  Recent high-profile cases of research misconduct and security breaches highlight the need for established, evidence-based policies, tools, and common practices for ensuring research integrity in order to fortify the public trust.

The complexity of maintaining research integrity is driving the development of a new disciplinary field dedicated to the study of research integrity forensics. Currently, efforts to uphold the integrity of scientific activities are dispersed across various stakeholders, including researchers, librarians, independent scholars, research institutions, journalists, government officials, funders, and lawyers. These efforts, while valuable, are often siloed within their respective disciplines, leading to a fragmented approach to addressing lapses in research integrity. By establishing a specialized field focused on the forensics of research integrity, we can foster a multidisciplinary collaboration that leverages the expertise of all relevant actors. This new discipline would provide a structured framework for studying and combating the proliferation of disinformation, ensuring a coordinated and comprehensive approach to safeguarding the integrity of science and scholarly communications.

What’s In a Name

New disciplinary and academic fields emerge with changing societal needs — and we have a pressing need to fortify the scaffolding of trust around science. We argue that the work of the unpaid volunteers as well as those who have job responsibilities for research integrity at publishers, universities, funders, etc. should be recognized as being in the field of applied forensic scientometrics. This applied work is not new; assigning a name to the field is new. 

Historically, new disciplinary and academic fields emerge when research communities coalesce around a theoretical framework or set of practices coupled with changing societal needs. The last decade or more has seen the way we communicate and share research expand with the growth of open science and online communication. While these technological advances and scientific practices have opened vast possibilities for the rapid sharing and verifiability of science, so too have they introduced new challenges to the integrity of research — especially in terms of the growth of paper mills, citation cartels, and dubious authorship. The clean-up from this damage to the scholarly record will take years — and that is with people actively engaged in mitigating misinformation. Yet, the work to retract articles or check pre-prints — which is only a small fraction of what needs to be done to correct scientific malfeasance — needs dedicated time and committed budgets. Thus, this effort necessitates a distinct field and a name.

But why forensic scientometrics? The forensics of science (or forensic science) may be a more accurate term, but that has been long used in the field of criminal investigations to describe the application of scientific principles and techniques to matters of criminal justice. In the case of FoSci, the focus is not on the use of scientific principles to solve crimes, but rather the use of investigative principles and methods to further check and assess the integrity of the science in a quickly evolving scholarly communications landscape. 

In essence, FoSci is an emerging field that combines elements of forensic science with scientometrics, which is the study of measuring and analyzing scientific literature. In this context, “forensic” refers to the use of scientific methods and techniques to investigate questions related to authenticity and integrity within the realm of scientific research.

Signals of Emergence

The emergence of forensic scientometrics as a field is reflected in several signals or indicators:

Increased Attention to Research Misconduct: There is growing awareness and concern about issues such as plagiarism, data fabrication, and other forms of research misconduct. This has led to a demand for more sophisticated tools and methods to detect and address these problems.

Increased Availability of Bibliometric Data: The availability of large-scale bibliometric databases has provided the necessary data infrastructure for testing the veracity of scientific claims and verifying many aspects of scholarly communications. 

Advancements in Data Analysis Techniques: The development of advanced statistical and computational methods has made it possible to analyze large datasets of scientific literature and identify patterns indicative of unethical behavior or sloppy scientific practices.

Greater Emphasis on Research Integrity: Funding agencies, academic institutions, and scientific journals are placing more emphasis on research integrity and ethical conduct. For example, the US Office of Research Integrity at the National Institutes of Health and institutional research integrity and security offices address research misconduct issues of funded researchers. Newly formed or expanded research integrity offices within publishing organizations are monitoring and reforming publishing practices (e.g., the STM Integrity Hub). 

Collaboration Among Disciplines: The field of forensic scientometrics involves collaboration among disciplines such as statistics, computer science, and library and information science. This interdisciplinary approach is a hallmark of emerging fields.

Disciplinary Specialization: As academic fields mature, specializations and subfields develop. Within FoSci we’ve already begun to witness this among researchers who have specialization detection — from nefarious networks to image manipulation to tortured phrases to hijacked journals. 

Growth of Community: Perhaps the most significant signal of field emergence is that individuals doing this work see themselves as in a community of practice. They recognize each other as doing “work like mine” and offer each other support, engage in knowledge exchange, and network. One example of this is the World Congress on Research Integrity (WCRI), held biannually, which has led the way in this space. It started years ago in a foresightful movement to convene interest in globally upholding the integrity of science. The WCRI aims to provide a platform for participants from around the world to share experiences, best practices, and strategies for promoting research integrity. It addresses various aspects of research integrity, including plagiarism, data fabrication and falsification, authorship issues, conflicts of interest, peer review, and responsible conduct of research.

Working in FoSci

All of this work is not without costs, which strengthens the case for creating a field. There is an emotional toll of working in forensic scientometrics. The thrill of investigating crimes against truth may fascinate and enthrall many, but this comes with the emotional complement of delving into the deception of a field you trusted. People you believed in. Knowledge you believed in. 

The threats to forensic scientometricians are alarming – including revealing personal information like home addresses (doxxing) in an attempt to intimidate the individuals. Real or threatened lawsuits target people and organizations striving to do the right thing, even those without institutional support. Moreover, lawsuits against publishers aim to prevent retractions, and both individuals and institutions face the burden of lawfare – the exploitation of legal systems and institutions to undermine, discredit, or exhaust an adversary.

In response to this concerning trend, the newly established Scientific Integrity Fund aims to cover legal expenses for scientists facing lawsuits due to exposing fraudulent research. This initiative emerged after a surge of lawsuits against scientific forensic investigators like Data Colada, and fertility researcher Chelsea Polis, who faced legal repercussions for their work.

Considering my experience with one paper, I recognize the considerable time and effort required to navigate such complaints. Adhering to COPE guidelines, both the editor and the authors must carefully consider and respond to each complaint. Additionally, responses must undergo a thorough review by respective employers, further delaying the resolution process.

For many FoSci researchers, the real challenge lies in these bureaucratic hurdles that consume valuable time and impede their ability to work efficiently. The researchers may gain notoriety, but that doesn’t mean they will get professionally promoted.  The establishment of FoSci may offer some shields, defenses, and opportunities for those involved.

Forging a Recognizable Identity

By embracing FoSci as a specialized and necessary field, we can galvanize interest, foster the development of a community of practice, and signal the importance of this crucial work. Moreover, formal recognition of FoSci as a distinct field will pave the way for the development of standards, methods, and training initiatives. Looking ahead, envisioning a future where FoSci plays a pivotal role in preventing and detecting scientific misconduct underscores its potential impact in safeguarding the integrity of the scientific record.