Earlier this year, David Moher, Larissa Shamseer, Kelly Cobey, and colleagues caused a bit of a stir when they published an article in Nature showing that, contrary to some (many?) expectations, it’s not just authors from low-income countries who publish in so-called “predatory” journals. In fact, their analysis of nearly 2,000 biomedical articles from more than 200 predatory journals found that “more than half of the corresponding authors hailed from high- and upper-middle-income countries as defined by the World Bank.”

A just-published follow-up article in Nature Human Behaviour focuses on what constitutes a predatory, or in Moher et al’s words, illegitimate journal, and, critically, on what can be done to help authors avoid publishing in them.

Wolf In Sheeps Clothing

In this interview, Kelly Cobey and Larissa Shamseer (both of the Centre for Journalology, Ottawa Hospital Research Institute) share their thoughts on what is meant by the term “illegitimate journal”, why these publications are proving so successful at attracting authors, and the community actions needed to stop their spread.

One of the problems with predatory publishers is defining what that term means and, therefore, which publishers are genuinely predatory rather than, say, well-intentioned but unable to follow what we consider to be best practices. What’s your definition and why?

The term ‘predatory journal’ was coined by Jeffrey Beall to describe publishing entities masquerading as typical journals but failing to uphold expected best practice standards. He chose “predatory” to reflect the intentionally deceptive nature of these journals. For instance, these entities may claim to use peer review, but based on several ‘stings’ we know they do not; they may claim to be open access but don’t seem to follow the tenets of open access publishing laid out in the Bethesda Statement (i.e., appropriate copyright for re-use and derivatives, published in a suitable electronic format, or deposited in repositories). Further, they are known to use aggressive tactics and trickery to solicit manuscripts from authors.

In a study published earlier this year, we described some unique, evidence-based features of journals once listed on Beall’s lists of predatory journals distinguishing them from traditional open access and hybrid/subscription journals. It is possible that some of the characteristics we identified may be common to journals that are from lower income countries, or are low resourced (e.g., low article processing charges, spelling/grammar errors). In our view the term ‘predatory journal’ does not appropriately describe these types of journals. However, any journal not meeting best practice standards, whether due to intentionally deceptive practices, low resourcing, or lack of knowledge, or otherwise, in our view are nonetheless a poor entity to ‘publish’ in. The term ‘illegitimate journals’ may better reflect the range of journals that fail to meet expected best practice standards.

You describe predatory journals as “contaminating” all domains of science. What do you mean by that?

The problem of predatory journals has crept into many areas of science and is not limited to biomedicine (our discipline). Scientific publishing isn’t perfect. Issues with the reporting quality and reproducibility of the published literature have received growing attention. In biomedicine, problems such as publication bias and selective outcome reporting can mean that the evidence-base informing treatment decisions is biased in favor of interventions that may not actually be effective or safe. Predatory journals introduce a further concern relevant to reproducibility. If content is not vetted (e.g., via peer review, use of plagiarism detection software, enforcing ethical standards) it may result in the publication of very low quality, poorly reported, and, sometimes, fraudulent research. If their content is able to be found by readers, it may be mistaken as trustworthy and used to inform future research or decisions. Perhaps more problematic is that real research, potentially containing negative or null results may appear only in predatory journals (for lack of a better option) and these journals remain unindexed and as such, not consistently identifiable.

We also have to look at the system of incentives and rewards for researchers to publish. We exist in a system of ‘publish or perish’ where publications are the standard we are measured by. The more, the better. Quantity seems to be valued over quality, leaving room for researchers to publish just about anything, anywhere, and put it on their CV for future consideration tenure/promotion and grant committees. For researchers with less nefarious intents, they may simply not be attuned to the fact that publishing is a for-profit business rather than a scientifically-minded endeavor. One assumption may be that publication (in any venue) infers quality when in fact, that is not the case.

Earlier this year, you demonstrated that corresponding and senior authors of papers published in predatory journals are just as likely to come from upper-middle and high income countries as from low income countries. Why are so many researchers choosing to publish in these sorts of journals?

The running narrative had been that predatory journals were mainly a problem affecting lower income countries. The fact that this is not the case really demonstrates the universality of the problem – researchers across the globe lack training and knowledge on how to select an appropriate (and legitimate) venue to publish their work in. One of us, Kelly Cobey, is the publications officer at the Ottawa Hospital and has consulted with researchers who have inadvertently submitted their work to illegitimate/predatory journals. The experience can be very stressful for those involved. The motivations for selecting a predatory journal, over another journal, need to be examined in more details. It is possible that some researchers are being duped, but that others are consciously using these illegitimate outlets to accrue publications quickly and without barriers.

In your latest paper you call for “stronger decisive action” to stop the spread of predatory journals. What specific actions are you recommending each group of stakeholders to take?

We identify a number of key stakeholders who we note need to work concertedly to address the problem of predatory journals. A complete explanation of consequences of predatory publishing and actions each stakeholder can take is listed in our recent article in Nature Human Behaviour. Examples of key changes these stakeholders can make include:

  • Researchers: Self-educate on the characteristics of predatory journals as well as on how to choose a journal.
  • Academic institutions and libraries: provide support, policies, and education to guide researchers on the risks of predatory journals and how to avoid them.
  • Funders: Perform audits of where funded research is published and develop guidance and policies discouraging publication in predatory journals.
  • Legitimate journals: Increase transparency in publishing practices so that they are publicly auditable (e.g., open peer review), endorse and adhere to transparency initiatives (such as study registration and reporting guidelines), and if open access, register with the Directory of Open Access Journals.
  • Regulatory bodies: adopt policies that tie regulatory approval to dissemination plans, including specific plans for publication in legitimate journals.
  • Patients and the public: Before participating in research, seek assurance that the resulting publication will not be published in a predatory journal; become an advocate for best publication practices by participating in institutional ethics boards.

How are you hoping to make your proposals a reality? Who are you working with and what are the main challenges?

For these recommendations to be effective, there needs to be a standard and agreed on definition of what constitutes a “predatory journal”. The definition and agreement must come from all stakeholders, not just one group in isolation. Our hope is to raise funds to be able to bring together a group of leaders from each stakeholder group for an in person meeting to gain consensus on this issue before proceeding with further activities to address the problem. It will be important to evaluate and track the success of any interventions generated to address the problem.

What’s next — are you planning to continue your research in this area or will you be turning your attention to other topics in future?

At our Centre for Journalology at the Ottawa Hospital, we have multiple research projects on the go. In addition to our work on predatory journals, other main areas we carry out research in include: evaluating incentives and rewards in academia, developing and implementing reporting guidelines, and developing core competencies for biomedical editors.

Pertaining to predatory publishing, we aren’t done yet! Now that we have helped to identify to scale of the problem we would like to actively contribute solutions. While we’ve received little external funding for our work, we have manage to keep going, thanks in large part to the generous support from the Ottawa Hospital Research Institute. Research evaluating publication best practices and meta science more generally tends to be difficult to acquire funding for as it often falls outside of agencies remits. At present, we are wrapping up an evaluation of funder policies regarding journal publication, i.e., whether current funding policies address where funded research should be published. We hope to carry out similar audits of other stakeholder practices in the future.

Alice Meadows

Alice Meadows

Alice is Director of Community Engagement & Support for ORCID, responsible for communicating the why, what, and how of ORCID for researchers and their organizations. Alice is on the Board of Directors for the Society for Scholarly Publishing and received the 2016 ALPSP Award for Contribution to Scholarly Publishing.

View All Posts by Alice Meadows

Discussion

14 Thoughts on "Illegitimate Journals and How to Stop Them: An Interview with Kelly Cobey and Larissa Shamseer"

I have done publishing workshops for doctoral students and junior faculty, and I include information on predatory journals–how to identify them and why to avoid them. Every campus has at least one editor of a legitimate journal, or a university press, or some other publishing entity. Providing this sort of educational opportunity is a good start, though perhaps a mere stopgap in view of the overall problems of the publish-or-perish syndrome.

Thanks for sharing your experiences, Beth. We agree that education is an important part in addressing the problem of predatory journals. Libraries/librarians have taken much initiative to create relevant outreach material, but in our view more needs to be done to successfully disseminate these tools and ensure their evidence base. Initiatives like DORA (http://www.ascb.org/dora/) will be important in addressing the important and related issue of the system of incentives and rewards in academia.

Isn’t this problem why we rely on citation metrics, however imperfect? It’s easy to understand that authors are tempted or ignorant but do their articles really get cited much? If scholars realize their citation metric will become unhealthy if they fill their CV with junk food, maybe they will see the benefit of choosing better venues.

You are right in raising the issue of perverse metrics as being related to researchers’ decisions around selecting journals and publishing their work. Thanks for sharing your view. The systems used to decide on tenure and promotion or hiring of faculty vary between institutions. Some institutions use publication count as a metric to success, without vetting journals or the quality of work published. Policy and practice changes are needed to address this issue.

There seems to be a drive for creating a dichotomous world and placing each publication (or publisher) into one of the other, e.g., legitimate/illegitimate, predatory/not predatory. It seems to me that, once we move out of the realm of fraudulent (where I am personally comfortable with a fraudulent/not fraudulent judgement), there are really a great variety of factors to consider. I’m thinking of a journal issue that is invite-only … thus fails at one or more of the best practices criteria but I would hope no one thinks illegitimate. If labels must be given, there would likely be far more than only two categories?

Thanks for your comment, we agree very much that there is a spectrum. Journals that are poorly resourced, and fail to meet best practice standards, are not synonymous to predatory journals (i.e., those that are intentionally dubious to make a profit). In our view both journal types present issues and both require attention. We think it will be important to develop a clear consensus on what features constitute ‘predatory’ activities. This will help distinguish journals with nefarious intents from those that are well-intentioned. To do this effectively we will require collaborative input from broad stakeholders. From a pragmatic perspective, in order to develop policies, it may be that grouping journals into categories will be necessary.

Luey and Hinchliffe as well as the authors interviewed (Cobey and Shamseer) keep brushing at the edges of the issues at hand:

a) The recognition, but then stepping around, the “publish/perish” syndrome which is manifest in a variety of ways such as dependence on “metrics”as an evaluative criteria rather than a measure of the value of the content.

b) Rules/regulations of an industry, as well as licensing and certain laws, under the guise of protecting the “public” are vehicles for restricting access both for new entrants and those who could benefit.

Excellent examples are seen in healthcare (the focus of the authors), law, and other professions as well as more mundane areas such as the “taxi” business, real estate, etc.

While there is legitimate concern for the most egregious of publications, one would be hard pressed to accept that trained academics in recognized and vetted research institutions naively or unknowingly published in what might be labeled a first order predatory journal.
As mentioned in this article, academic publishing, with exceptions, are profit-making, often with exceedingly high margins. There is a vested interest of these to maintain that position. At the same time, authors, under pub/perish see the demand to publish while dancing between Scylla and Charybdis.

In the health care arena, the issues of licensing nurse practitioners, in publishing OA of various stripes and similar examples give one pause to ask whether, as with Uber in the “taxi” area, Legal Zoom in law and others, there is more to the increasing alternatives to STEM publishing and academic publishing in general.

Thanks for sharing your thoughts on the article and blog. Below are some thoughts on the interesting points raised:
a) We agree that the publish or perish syndrome is a significant driver of researchers decisions on where/when to publish. It also contributes to related issues like publication bias, selective outcome reporting, and p-hacking. It’s clear to us that we need to move away from measuring quantity and develop novel approached to assess research quality. Our Centre is interested in this important issue and we are actively pursuing work in this area.
b) This is a great point. There is currently little regulation in the publishing industry (compared to others which are heavily regulated, i.e., the pharmaceutical industry). Currently there is no system of audit and feedback or clearly established industry standards. We agree that this would be important to discern practices at different journals, but also to drive evidence based decisions about journal operations.
Concerning your point about trained academics being unlikely to submit to predatory journals unknowingly, our experience and research is counter to this. Training on writing and publishing is often learned on the job, and researchers may have trouble keeping pace with changes. In my role as Publications Officer I have met a number of researchers, some of whom are very established, who have found themselves in that situation. A few examples are describe in a recent World View piece: https://www.nature.com/news/illegitimate-journals-scam-even-senior-scientists-1.22556
We also recently found that researchers from around the world are submitting and publishing in presumed predatory outlets. You can read more about that here:
https://www.nature.com/news/stop-this-waste-of-people-animals-and-money-1.22554
Work in Italy shows similar findings, with an increasing trajectory:
https://journalology.ohri.ca/account/login#!
New or varied models for legitimate open access are popping up all around us., PeerJ, eLife, and F1000 are examples.

The referenced study by Shamseer, Moher et al. (which suggests a snappy, shareable ‘Ten Salient characteristics of potential predatory journals’), which is supposedly ‘evidence-based’ is probably the worst example so far of a group of researchers in the Global North failing to understand publishing outside of their academic bubble. The study is methodologically flawed – for example the sampling bias of comparing Beall’s List journals with Pubmed-indexed and subscription journals…and nothing in between. It is incoherent – it warns about taking into consideration the challenges of publishing in low-income settings, then proceeds to ignore this and suggest chucking journals who don’t have LOCKSS or a correct copyright statement into the ‘predatory’ bucket. They also ignore their own data that shows ‘credible’ journals also have language that targets authors and have no archiving policy. For these reasons and many more, it is discriminatory towards publishing in the Global South. I see they have backtracked slightly from this now and have created a new category of ‘illegitimate’ journals to describe most of academic publishing in the developing world. Still pretty outrageous to be honest, and still hopelessly confused about the difference between ‘predatory’ publishing and publishing that fails to meet ‘best practice’.

Worst of all, they are still peddling the myth in this interview that low APCs are an indicator of predatory/illegitimate publishing. While most journals in the Global South are free-to-publish non-APC open access journals, an increasing number are charging small APCs (less than $100) to authors in their countries, to ensure the sustainability of their publications, which are mostly small, volunteer-run or with limited funding from their university/society. Who is going to be the first to tell these guys that this is ‘illegitimate?’ and that they should be charging $2000 to their local authors, just like the big publishers in the US?

Like one of the comments has already pointed out, creating these false dichotomies of publishing standards is not helpful, and fails to understand a variety of factors. In global journal publishing there are many shades of grey in between Shameer, Moher et al’s ‘best practice standards’ and ‘predatory’ journals, and I’m not going to take Ottawa Hospital’s Journalology team seriously until they make an effort to go out into the world and try understanding this a little better.

Thank you for spending time thinking about our work. I hope to address some of the points you mention, below. Given your leadership on Think, Check, Submit, we’d also be very happy to set up a time to speak regarding your perspectives on optimal research designs to investigate deceptive/predatory publishing.
– We agree there is conflation between journals from low-income countries and intentionally predatory/deceptive/illegitimate journals and have stated as much here and elsewhere.
– In the study you point to (Shamseer et al BMC Med 2017), our sampling was done in 2014 (submitted for publication in 2016). At that time, as a community we knew far less than we do now about so-called ‘predatory’ journals. Additionally, Beall’s list was still widely considered a go to source for many in publishing and academia for identifying predatory publishers. Due to its prominence in the field, we chose to investigate it. There were no known alternatives to use to identify and sample journals presumed to be predatory. For our legitimate biomedical journal sampling, you may recall that the DOAJ was undergoing a transition at that time so not a reliable source of OA journals. In consultation with co-authors (some of whom active professional in the publishing industry, and not academics), we opted for PMC-listed journals.
– In that paper and in our subsequent work, we have adopted the phrase ‘illegitimate entities’ as a more holistic term describing potential predatory/deceptive/low quality journals, as they are not abiding by typical/standard journal practices. As this field progresses, we aim to keep up with evolving terminology.
– In agreement with yours and others’ comments, we too do not think dichotomizing predatory vs non predatory journals/publishers is helpful. This is why we haven’t suggested the reliance on non-evidence-based lists (e.g., Beall’s, Cabell’s, etc.) in our stakeholder recommendations. Rather, authors need to be aware of hallmarks of both legitimate and illegitimate journals. We need research, not opinion, to identify these distinctions.
– In addition to Table 10 of salient characteristics, as described in the paper, they are meant to highlight factors that are more frequent in predatory journals than mainstream journals as well as those which are important and easily assessed. For instance, while archiving policies were not stated across all journals, being indexed in Medline/PMC ensures that archiving requirements are met. Increased transparency around these features as we suggest in our Nature Human Behaviour paper is in line with the recommendation of many others in the publishing industry.

We welcome the opportunity to speak offline about alternative methodological approaches to future research.

Hi Larissa,

Thanks for your graceful response to my rather caustic message, and explaining some of the thinking behind the study.

I was posting in a personal capacity, and I am not on the committee of the Think.Check.Submit team at all – I am merely a (grumpy) voice on the periphery who occasionally feeds into the thinking from conversations I regularly have with researchers and editors in low-income countries – this means that sometimes I get quite animated about representing their views and the challenges that they face.

Focusing on the positives of your work, I think that it’s good that you’ve been carrying out some detailed analyses on this problem, and great to see you looking to collaborate with others to find solutions – I’m sure TCS would welcome being part of these discussions.

I agree that an evidence-based approach is the best way of understanding this ‘predatory’ publishing phenomenon, even if we disagree on this particular study – I take your point about the date – things are changing quite rapidly in this area, so I look forward to seeing the results of any future studies. However, I am dubious that a simple ‘red flag’ checklist is achievable in a complex global publishing environment, and that we should be more concerned with educating researchers on how they can identify *appropriate* journals for their research (and recognising that these journals might be low-APC, and might not always tick all our ‘best practice’ boxes).

Super disappointed in this piece. I agree with Andy Nobes.

I suggest reading this piece in C&RL News byShea Swauger “Open access, power, and privilege”
A response to “What I learned from predatory publishing”
Shea Swauger is head of researcher support services at the University of Colorado-Denver’s Auraria Library, email: shea.swauger@ucdenver.edu
© 2017 Shea Swaugerhttp://crln.acrl.org/index.php/crlnews/article/view/16837/18435

Ann’s post with reference to the piece by Shea needs serious consideration by the SSP and particularly the “Chefs” of The Scholarly Kitchen. There are several levels here. The first, as Shea addresses in part is the substance and value of what is published whether its in the traditional journals, the “Grey Literature” or those journals outside and often labeled as predatory, almost whole cloth. The second is the publishers of the journals with a vested interest in protecting their fiefdom. It is that latter which has been discussed in the larger socio/economic literature (c.f. “The Captured Economy”). It is that which Andy’s comments, in part, addresses.

While one can agree that the quality and substance of the scholarly work and its reporting are critical, it is the publishers that choose, via allocation of fiscal resources, that limit both the volume and depth of review of scholarship while at the same time spending an inordinate time via such metrics as impact factor to validate the journals as journals and where the publishers have increasingly stepped across the Chinese Curtain between themselves and the editorial management of content.

The rise of journals, some of which are “predatory”, points to stress within the established academic publishing arena. This has been seen early with OA in publishing but also in many other “gated” areas such as in healthcare. We saw this in the past, in physics, with the rise of “letters” which eventually became the equivalent of “journals”. We see this today with the metered proliferation of increasingly narrower focused journals. And we see this with the Internet with the increasing number of ways to “publish” and the push to “streaming” of articles as accepted making journals a point of entry or gate for knowledge to pass through.

Whether it is scholars in the South or even in the “North”, the rise of alternative paths for sharing vetted knowledge rests with and always has rested with the publishers who are seeing the alternatives becoming more prevalent. The idea that scholars in ranked institutions might choose alternatives is real (and there are numerous examples of how such choices are made have and are being seen today). This issue is one of the publishers’ own making much as the monopolist licensing in healthcare and even in other service industries are seeing today.

Here is perhaps a model of what an information literate society might look like: The Electrochemical Society or ECS. I just found out about ECS via my Twitter network.

ECS seems to have embraced information literacy as Shea Shwauger in his C&RL piece suggests, and moved on, complete with videos including one called “Free the Science: Taking Down the Walls”.

Do other librarians use ECS (http://www.electrochem.org/) as a model?

ECS makes embracing open access doable. So refreshing. ECS has done the work of information literacy and upgraded their thinking about their role in scholarly communication, just as they would in their labs with techniques or equipment. ECS seem to take this stance: schol comm presents both societies and authors with “challenges”! And that has apparently freed up ECS resources and energy to address the challenges.

An example excerpt from ECS blog dated 6 December 2017: http://www.electrochem.org/redcat-blog/finally-action-predatory-publisher/

” Commercial publishers represent a major challenge for a scholarly society like ECS to uphold those standards and meet our mission; but the open access environment has created even more challenges. Authors have these same challenges. So it bears repeating that there are some “best practices” to which we should all do our best to adhere: carefully vet journals where we are submitting; create awareness with our various constituents (members, students, colleagues) that there are “good” and “not good” places to publish; and do a periodic Web scan of own own names from time to time to make sure we have not been added to an editorial or advisory board without our knowledge, which can compromise our personal identity and standing in the community.

ECS will continue to work toward its Free the Science goals to create a more open, but responsible, scholarly communications ecosystem.”

Leave a Comment