Editor’s Note: Today’s post is by Christos Petrou, founder and Chief Analyst at Scholarly Intelligence. Christos is a former analyst of the Web of Science Group at Clarivate Analytics and the Open Access portfolio at Springer Nature. A geneticist by training, he previously worked in agriculture and as a consultant for A.T. Kearney, and he holds an MBA from INSEAD.

On the last day of 2020, the Chinese Academy of Sciences issued the first Early Warning List of International Journals. While it includes just 65 journals, the list disproportionately targets journals that are fully open access (OA) and attract a high volume of Chinese content in absolute or relative terms, notably IEEE Access, several titles published by MDPI, and four from Hindawi. It has been three full months since the Warning List’s release, and the impact is already visible on publication volumes. For example, IEEE Access has had its worst quarter since early 2019 and MDPI’s listed journals are getting less content from China.

As Europe doubles down on OA with initiatives such as Plan S, the Chinese administration, intentionally or not, seems to have chosen a different direction. The Warning List serves to remind that global cooperation among policy makers is just as likely as a policy rift. International publishers who try to accommodate conflicting regional policies may find themselves caught in the crossfire.

Warning road sign against a blue sky.

Few journals, many OA papers

The journals on the list account for a negligible 0.2% of the 26,000 journals that are indexed on Scimago (Scopus data). However, they are primarily fully-OA (43 of the 65 journals), large (40 published more than 1,000 papers in 2020), and they publish plenty of papers from China. As a result, based on 2020 volumes, they account for 51k Chinese papers and 43k fully-OA Chinese papers, which represent 6% of all Chinese papers and 22% of all fully-OA, Chinese papers respectively. At the stroke of a pen, almost a quarter of fully-OA, Chinese content is under warning.

chart showing number of papers and journals affected
Figure 1. The Warning List in numbers. Calculation (1) assumes 65 listed journals, and 26,199 scholarly journals in 2019 per Scimago; calculation (2) assumes 144k papers for the listed journals in 2020 per Dimensions and publisher websites, and 2,994k papers in 2020 for all scholarly journals (10% growth on Scimago’s 2,722k papers in 2019); calculation (3) assumes 51k Chinese papers for the listed journals in 2020 per Dimensions, Lens, and publisher websites, and 787k Chinese papers for all scholarly journals (15% growth on Scimago’s 684k papers in 2019); calculation (4) assumes 43k fully-OA, Chinese papers for the listed journals in 2020 per Dimensions, Lens, and publisher websites, and 191k fully-OA, Chinese papers for all scholarly journals (assumes 24.3% papers in China were fully-OA, same as in the rest of the world in 2019 per Scimago); calculation (5) assumes 728k fully-OA papers for all scholarly journals in 2020 (10% growth on Scimago’s 662k DOAJ papers in 2019).

Multiple criteria that come down to one

The list cites multiple criteria for the inclusion of a journal. Some criteria, such as the volume of retractions, can be linked to the editorial rigor and quality of a journal. Other criteria, such as a journal’s self-citations, are more relevant for the development of citation indicators than the development of warning lists. Then, some other criteria, such as the rejection rate and whether the journal charges an APC, are bizarre and cater to outdated notions about journal quality.

It all matters little, because in practice, as hinted in Figure 1, the list targets international journals that get a high volume of fully-OA Chinese content in absolute or relative terms. Surely it includes some unsavory titles that have frequented other warning lists such as the infamous Beall’s list, but the common denominator is not the predatory nature of the journals but that they attract considerable APC payments from China.

The listed journals are assigned to three warning levels: low, medium, or high. As shown in Table 1, journals with medium or high warning get about half of their content from Chinese researchers.

table of journal and content characteristics
Table 1. Journals and content characteristics in 2020 by ‘warning level’ (data for MDPI journals and IEEE Access coming from publisher websites; global paper volume for other journals coming from Dimensions; share of Chinese papers based on 2019 data from Lens)

MDPI, IEEE, and Hindawi stand out

The Warning List includes a couple of journals from many major publishers. Yet it is MDPI, IEEE, and Hindawi that stand out, as the list includes 22 journals from MDPI, the megajournal IEEE Access, and four titles from Hindawi. All but one of MDPI’s titles are in the ‘low’ warning level, whereas IEEE Access and three of Hindawi’s titles were in the ‘medium’ warning level.

Given that MDPI published 166k papers in 2020, the list puts under threat about 11% of its content (19k papers from China). In absolute terms, less of Hindawi’s content is coming under threat (5k papers), but in relative terms almost 20% of its content will be threatened (per Dimensions, Hindawi published about 25k papers in 2020). The relative risk is even higher for IEEE Access, which in 2020 it got most of its content from China.

table of journals and content characteristics by publisher
Table 2. Journals and content characteristics in 2020 by publisher (‘other major’ publishers include Elsevier, Springer Nature, Wiley, Taylor & Francis, Sage, and Wolters Kluwer; data for MDPI journals and IEEE coming from publisher websites; global paper volume for other journals coming from Dimensions; share of Chinese papers based on 2019 data from Lens)

As shown on Figure 2 below, most of Hindawi’s titles, as well as those of Spandidos, have heavily relied on Chinese content, which accounted for more than 60% of their overall content in 2019. Many titles of other publishers also got a high proportion of their content from China. MDPI is the odd one out, as its titles received a relatively low share of content from China with most titles staying below 30%.

One possibility is that the Warning List is targeting MDPI’s journals collectively for the absolute volume of Chinese content that they received. Or it might be that other criteria came into play, unrelated to the quality of the journals and the volume of Chinese content that they publish. After all, 20 of the 22 listed MDPI titles rank in the top half (Q1 and Q2) of Clarivate’s JCR in their respective categories.

Figure 2. Volume and share of Chinese papers by journal in 2020 (data for MDPI journals and IEEE Access coming from publisher websites; global paper volume for other journals coming from Dimensions and share of Chinese papers for based on 2019 data from Lens)

Impact so far

In a previous Scholarly Kitchen post, I suggested that megajournals are a risky business, and that the successful megajournals enter a period of turbulence and slow decline after their Impact Factor drops. The Impact Factor of IEEE Access peaked at 4.1 in summer 2019, and dropped to 3.7 in summer 2020, halting the journal’s spectacular growth. While publications did not outright decline, the last two quarters in 2020 were flat in comparison to the previous year.

Then came IEEE Access’ inclusion to the Warning List to show that one riskier product than a megajournal is a megajournal that is overly reliant on one region. In the first quarter of 2021, IEEE Access experienced its lowest quarterly publication output since mid-2019, marking an annual decline of 24%. March was down 34% on an annual basis.

The decline has been driven by an exodus of China-based authors. It started after the Impact Factor of IEEE Access dropped below 4.0 (China-based authors have been particularly responsive to Impact Factor fluctuations) and it accelerated after the publication of the Warning List. Interestingly, the journal has been growing in other locations (11% growth in March for papers without China-based authors vs 60% decline for papers with China-based authors), but the over-reliance and decline in China have dictated its course.

chart of volume growth for IEEE Access
Figure 3. Quarterly volume of papers and annual growth for IEEE Access (data coming from IEEE’s website)

Similarly to IEEE Access, the Chinese content for MDPI’s listed journals has declined. Contrary to IEEE Access, MDPI’s portfolio is better positioned to absorb the impact. That’s because (a) MDPI’s listed journals have less Chinese content, (b) the journals that are not listed are still growing in China, and (c) all journals are growing tremendously in the rest of the world. In the first quarter of 2021, MDPI grew by 50% on an annual basis, despite its Chinese content shrinking by 8%. MDPI is on track to publish more than 200k fully-OA papers in 2020. Thanks to its portfolio approach and regional diversification, MDPI thrives while others falter.

chart showing annual growth for MDPI's jopurnals
Figure 4. Annual growth for papers published in the first quarter of 2021 for MDPI’s journals (RoW stands for ‘rest of world’; data coming from MDPI’s website)

The publication of the Warning List on the last day of 2020 should have affected journal submissions by January. Its impact on publications is getting observed with a lag, depending on the publishing speed of each journal and publisher. IEEE Access and MDPI publish very fast (IEEE Access typically publishes within 4-6 weeks from submission, and MDPI had a median time to publication of 35 days in 2020), and as a result the impact of the list is already visible in their Q1 performance. The impact should become increasingly visible by Q2 for other, slower journals and publishers.

A miscalculation or a cynical calculation

While the list clearly impacts international, fully-OA publishing, it is less clear whether this is the result of a miscalculation or a cynical calculation. The miscalculation would be about misinterpreting OA for poor quality. The cynical calculation would be about China benefitting from the increase of OA content from other countries while reducing the OA, APC payments by China. Chinese institutions get to pay less for subscriptions to read foreign content, and China-based authors get to publish for free. Heads I win, tails you lose.

It should probably be noted that the list does not include any Chinese, fully-OA titles that get most of their content from China-based authors, arguing against the ‘miscalculation’ above.

Adding to the perplexity of the matter, the heavily targeted MDPI hires thousands of employees in China (about half of their workforce) and runs at a very slim margin, which implies that China is a net beneficiary of the APC payments that are made to that publisher. Undermining MDPI is probably not in the country’s interest, but it may serve narrower interests.

Implications for foreign publishers and policymakers

The question that should occupy the minds of international publishers and policymakers is whether the targeting of OA, APC publishing by the Chinese administration is part of a trend. Last year’s policies introduced APC caps of 20,000 RMB (about 3,000 USD) and limits to the papers that can be paid for through the special fund of the National Science and Technology Plan. Does the warning list build on that policy, and will it lead to further anti-APC action?

Moreover, will China see the growth of international OA as an opportunity to curb its publishing costs? Or will it ease OA, APC restrictions once an increasing volume of international content becomes freely available thanks to global pro-OA policies?

In a worst-case scenario, is this the start of a regional policy rift whereby international publishers struggle to accommodate both European and Chinese policies? Would you rather ‘transform’ your journal in order to accommodate European funders or would you maintain its subscription status to avoid drawing the ire of Chinese funders?

In any event, the warning list shows that the Chinese market can be risky. Seemingly healthy revenue streams can come under threat with little warning and unclear justification. Over-reliance on the Chinese market is not a healthy long-term strategy, and scholarly publishers and information providers that are overly exposed to the Chinese market should plan to diversify their revenue streams.

Christos Petrou

Founder and Chief Analyst at Scholarly Intelligence. Christos is a former analyst of the Web of Science Group at Clarivate Analytics and the Open Access portfolio at Springer Nature.

Discussion

16 Thoughts on "Guest Post – An Early Look at the Impact of the Chinese Academy of Sciences Journals Warning List"

A very interesting analysis indeed.
Reliance on certain markets heavily is always risky for any business. Only four months into the introduction of the Warning List by China proves that once again. Now it would be interesting to see how the megajournals respond to the changing trends as shown in this article.

Christos, could you share a copy of the full list of criteria being used by the CAS to compile this “warning list”? It would also be really helpful to know how the criteria are weighted. I’ve spent some time looking around for this information and have yet to find anything that wasn’t in Chinese…

Hi Rick, I worked through the Chinese text with a native speaker. You won’t get anything different than when using Google translate.

I guess that if it were just for that document, it could have been dismissed as something vague or unimportant. But clearly it has already had real life consequences.

Sorry I can’t be more helpful than that!

Translating directly from the announcement:
[…] In particular, by going through a thorough evaluation of aspects including the number of articles in a journal, the international scope of the journal’s authors, its rejection rate, APC, [ranking??], the rate of articles which self-site, and its retraction rate, so as to identify scholarly journal which display potential issues in quality or risk characteristics. (具体而言,就是通过综合评判期刊载文量、作者国际化程度、拒稿率、论文处理费(APC)、期刊超越指数、自引率、撤稿信息等,找出那些具备风险特征、具有潜在质量问题的学术期刊。)

Source: Chinese Academy of Sciences Documentation and Information Center formally issues its “Warning List of International Journals (for Trial Implementation)” 中科院文献情报中心正式发布《国际期刊预警名单(试行)》 http://www.igg.cas.cn/xwzx/kyjz/202101/t20210101_5849507.html

Where have all of the articles that would have been published by IEEE been published instead? Are they going to domestic Chinese journals? Could that be the motivation?

Having previously worked in a Chinese institution that was obsessed with publishing in Quality Journals with high Impact Factor in order to increase our rankings (and indeed, it is very highly ranked): my guess would be that the motivation would be to increase reputation of Chinese research rather than increase submissions to Chinese journals.

The list only states journal titles. This can have a devastating effect on journals that have the same title. An example: both IndiaScienceTech and American Scientific Publishers publish a journal titled “Journal of Nanoscience and Nanotechnology”. Is there a list of the titles that include additional metadata such as ISSNs?

Great piece, Christos. I thought SK readers might like to know that of the 65 journals listed by CAS just ONE is on Cabells’ Predatory Reports (Journal of Biomaterials and Tissue Engineering). In addition, there are seven that appear similar to the nomenclature of some of the MDPI journals (eg ‘OA Journal – Energies’ and MDPI’s ‘Energies’ on the CAS list) that are published by EnPress, and this also highlights additional problems for journals where the titles are similar to others or include any errors.

Chinese Academy of Sciences has its main library and a journal publishing company called Science China Press (https://www.scichina.com/), both in the same building. The library produced this now famous list of journals. None of our MDPI journals deserve to be added there according to their criteria, but more than 1/3 journals on the list are from MDPI. Soon after the release of the journal list, the head of Science China Press contacted me, suddenly, and wanted to collaborate with MDPI and to invest.

Christos, thank you for mentioning MDPI here and in your previous blog paper (https://scholarlykitchen.sspnet.org/2020/08/10/guest-post-mdpis-remarkable-growth/). People often ask me why MDPI is fast. I normally argue that the manuscripts submitted to a journal for consideration are finalised by the authors and presumably ready for immediate publication. It is always good if we work hard to do our job correctly and publish papers very fast. I also answer the question by saying that Elsevier is number one and will stay as the largest because it perhaps has been always very fast. Obviously Elsevier will publish more than 1 million journal papers in 2021. I also replied once that to avoid the attacks, the old horse must run forward very fast.

As many of the comments pointed out, there isn’t enough information for readers to figure out how this list was produced. We know which elements were considered but don’t know how they were weighted. CAS has been producing a category which puts journals into 4 blocks by journal’s 3-year average impact factor. The top 5% of titles are Block 1, top 5-15% are B2, top 15-50% are B3, and the rest are B4. For years this annually updated category has been used by many institutions as criteria to judge the staff and students’ academic achievement. In my opinion, this is why the CAS warning list is so influential. It is made in response to the policy published last year (https://scholarlykitchen.sspnet.org/2020/02/27/new-chinese-policy-could-reshape-global-stm-publishing/), but any institution can make a list. CAS list could be the most influential of all though. It was published quietly on a new year eve, but received over 100K views almost overnight. I also don’t believe that CAS is targeting at OA journals, because CAS Library is the strongest advocate of open access.

Leave a Comment