Over the past month, my Chinese social media feeds have been flooded with news, discussions, and research papers about the COVID-19 virus. But earlier this week, all my contacts in the academic community were discussing two government documents.

The first one is called Some Suggestions on Standardizing the Use of SCI Paper Indexes in Higher Educational Institutes and Establishing Correct Evaluation Orientation, and the second one is “Some Measures to Eliminate the Bad Orientation of “Papers Only” in Science and Technology Evaluation (Trial)”. Essentially these two documents mark an effort to largely reform the research and higher education evaluation systems in China. The first document is a set of guidelines and the second, marked as a “trial”, contains many detailed regulations. Here are the key takeaways from these policy documents, which could  potentially shift the landscape of global STM publishing, since China is now the world’s largest producer of scientific articles.

Chengdu, Sichuan, China at Anshun Bridge.

Differentiated evaluation criteria

Currently, researchers, research teams, and organizations are required to supply lists of papers published in their applications for government grants and their reports on the results of those grants. Research papers have been a primary measuring stick used to determine funding and career advancement. Key considerations have been the quantity of papers produced, publishing those papers in journals listed in the Science Citation Index (SCI), and publishing in journals with high Journal Impact Factor (JIF) scores. Institutions in China have tailored their practices to meet these criteria, putting pressure on researchers to publish as many papers as possible.

The new policy states that publication of papers will only be used as a main evaluation indicator for basic science and technology research, and not for applied research and technological development. This removes the publication burden from clinicians and engineers and others working in more applied areas.

For the basic researchers, a “representative works” system will be used. Under this system, only a limited number of a researcher’s or an institution’s most important papers count. No less than one third of the representative papers must be published in domestic Chinese journals. The quantity of papers published and the JIFs of the journals that the representative works appear in are not going to be used as a measurement for performance or research ability.

Researchers and institutions are encouraged to publish or present their most important works in:

  1. Domestic STM journals with international influence
  2. Top international journals universally recognized in the research field
  3. Top domestic and international academic conferences

These are called the “three types of high-quality papers” that are most sought after for evaluation purposes. The academic committees of individual organizations will determine the specific scope of journals and conferences that meet these standards, based on a “fewer but better” principle. Specifically, the list of domestic journals will reflect those included in the earlier policy guidelines offered by the Action Plan for the Excellence of Chinese STM Journals.

In principle, the number of papers (the “representative works”) that will count for basic research evaluation in grant applications or reports are limited in each evaluation cycle as follows:

  • Individual researchers are limited to five representative works
  • Key subject innovation teams: no more than ten representative works
  • National Laboratories/State Key Laboratories and other scientific and engineering research bases: no more than 20 representative works
  • Central level science research institutions: no more than 40 representative works

Developing high quality domestic STM journals

To accelerate the implementation of the Action Plan for the Excellence of Chinese STM Journals, the new policies seek to establish a “science citation index” system with Chinese characteristics and international impact. It encourages the publication of publicly funded research papers in high quality domestic STM journals.

Monitoring of journal quality

The new guidelines propose monitoring the quality of academic journals and the regular publication of an Early Warning list of domestic and international journals. Titles on the list will be tracked continuously and adjusted over time. Those with poor management, poor academic reputations, and those that put commercial interests first will be blacklisted.

Publishing cost management

For representative works, publishing expenses can be paid for through the special fund of the National Science and Technology Plan. Expenses for other papers not considered representative works cannot be paid out of this national fund. If the publishing cost of a single paper exceeds RMB 20,000 (approximately 2,850 USD), it can only be paid for after passing an examination of the necessity to publish the paper and with the  approval of an academic committee at the corresponding author or first author’s institution.

If a paper is published in a title on the blacklist or early warning list, the publishing related expenses will not be covered by the National Science and Technology Plan fund. For those who violate these regulations, any publication funds spent will be recovered and the remaining funds of related projects will be taken away.

Universities and research institutions will need to audit the necessity of publishing any paper, and will no longer be allowed to use incentive systems for researchers based on the number of papers published or a paper’s JIF (or other journal metrics). Implementation of these measures will be monitored and reviewed closely.

What does this mean?

These documents are worth further analysis and much will depend on how the new rules are implemented, but we can draw some potential conclusions now.

  1. There is likely to be a significant decrease in the total number of paper submissions from Chinese authors to English language journals. The dramatic increase in recent years and current large numbers of Chinese submissions to English language journals are largely due to the use of counting the quantity of papers published (especially papers published in journals indexed by the SCI) as one of the main metrics in the evaluation of scientific research. This practice is now officially discouraged. With the reduction of pressure to publish quickly, the quality of papers is likely to improve in general.
  2. The decrease in submissions will be seen mostly in low-quality journals. Publishing with top, high-quality journals is now more encouraged than before and those journals may even see increases in manuscript submissions.
  3. One third of high-quality papers will now be expected to flow to domestic Chinese journals. However, there are currently only 280 titles selected by the Action Plan for the Excellence of Chinese STM Journals, half of which are Chinese language titles, leaving the question of whether there is enough capacity to handle these papers. Most English language Chinese journals are co-published with international publishers. If they have a CN journal number, they are regarded as domestic Chinese journals.
  4. The threshold of 20,000 RMB for publishing costs, which is less than 3,000 USD at current exchange rates, may become one of the criteria used when authors choose journals in which to publish. This may be problematic for fully open access titles where a higher APC is needed to cover the costs of selectivity. Authors may instead seek to publish in subscription or hybrid journals to avoid these fees. Journals with significant page charges may also be at risk.
  5. Where in the past it has been common for Chinese authors to seek publication in broad, comprehensive journals due to their high JIF scores, top specialty subject area journals with somewhat lower JIF rankings will be at less of a disadvantage. Under the new policy, the top specialty journals will have more opportunities to get the best Chinese papers in their field, but bringing in these papers will require more communication and promotion to the Chinese academic community.
  6. In order to attract the best Chinese papers, journals and conferences need to get included in the “good list” (recommendation list for the “three types of high-quality papers”), and avoid being included in the “bad lists” (blacklist and early warning list).

How these new policies will be put into practice remains to be seen, but they  are already the subject of many intense discussions among Chinese publishers. They are being both applauded and severely criticized. Given the increasingly important role that Chinese research is playing, both in scientific progress and scholarly communications, the new sets of rules have the potential to significantly reshape the journal landscape. Top universities and other organizations are expected to start submitting implementation reports by the end of July, 2020.

Acknowledgements: The author wishes to thank David Crotty for his insights and help in the writing of this post.

 

Discussion

18 Thoughts on "New Chinese Policy Could Reshape Global STM Publishing"

The move to limit the number of papers published that can count for professional advancement seems to me an entirely healthy development, similar to what Harvard Medical School started doing a decade or more ago. Quantity should never outweigh quality.

It is generally regarded as good news for researchers, allowing them focus more on research. But for journals, not all good news. The majority of Chinese journals, many of them have existed for decades, are excluded by the national funding under the new policy. They may have to reform, one way or another.

Thanks Tao, another excellent insightful post on what’s happening on the ground in China, and potential implications outside of China, give pause for thought, and no doubt more detailed risk assessment.

On the ‘blacklist’ or ‘early warning list’, is there now a published ‘blacklist’ showing titles/publication names, and any more detailed information/criteria used for how titles are added on to ‘blacklists’, and also, how to get off the ‘blacklist’ or ‘early warning list’, if you find your journal is on it?

No blacklist or early warning list have been provided in China yet, and it is unkown which organization will be responsible for the issue and how the lists will be built.

Completely agree – very grateful for the contribution – and yes, Transformational. Journal publishers take note!

Thank you Adrian. I know big publishers with offices in China keep track of these policy changes closely, but most western academic publishers do not enjoy that convenience. These changes will potentially affect everyone, as long as they publish content from China. Dr Wang has answered your questions about the lists.

Thanks, very insightful.
I think The policy is quite self-contradictory. According to it, high-quality paper are those publish in “three types of high-quailty journal list” . So-called three journal lists in China are mostly measured by IF. It is conflicted with Chinese government’s another intention of reducing “excessive reliance” on SCI and IF.
I am also wondering if this regulation also applies to research funded by NHC and other government departments, except MoST. because it states that it applies to research projects funded by MoST, but doesn’t mention if other projects funded by NHC, NDRC or MoA also need to follow this requirement.

That’s very true. The focus of discussion around the new policies is what new evaluations will be used to replace SCI and IF, and how objective and fair will they be. The journals are calling for new evaluation system, too. The new Measures are made by the MoST so it does not speak for other departments, and there are researches funded by local government or by the institutions themselves.

Superb article and important information, Tao.

Thanks!

Thanks a lot, Tao, for your very valuable contribution and insights. Personally, I believe that these policies, and those yet to come, do primarily affect Chinese (STM-) scholar- and authorship, and less so the international STM publishing business. You are correctly stating that only a few hundreds of so-called CEL – STM journals do exist, most of them under a co-publishing arrangement with one of the known Western Publishers. These CEL journals do amount to max 5% of the world’s total, while China’s journal article output is estimated to sth between 15-20% of the world’s total. It seems that Chinese Scholars are encouraged under these new policies to either publish in one of those few CEL journals under Chinese ownership. Alternatively I believe that the pressure to publish in one of the absolute top international journals will only get more fierce for Chinese authors. To me, it looks like that the new policies will make it even more difficult for Chinese Authors to publish their works internationally. Question: Will the recent acquisition of the French STM press EDP-sciences by CSPM, Beijing, play a role for the implementation of any of the new Chinese policies known and/or yet to come? Time will tell! best regards, Matthias Wahls, The Netherlands.

Thanks Matthias. These policies will affect Chinese authors, as you pointed out, and because Chinese authors contribute a large portion of global research output, their change of behavior could potentially change global STM publishing industry. I agree that competition to publish on top journals could be more fierce than before, but top journals with relatively lower IF have more chance to get the best Chinese papers, too.

Good article addressing both the new criteria and the August policy document from the State Council.

I’m not sure I entirely agree with some of the conclusions, however. Quality not quantity has been in place in many areas of the world without necessarily affecting published volumes. The reasons for this are simple: paper growth and journal growth correlate with the number of researchers rather than any other metric. (See STM Report 2018 for more on this and references to the literature.)

Many parts of the world have been looking to move away from the JIF, although given that it is a signifer of other factors it is unclear how successful or radical this would be. In most communities, scholars are well aware of the standing and peer assessment of titles through a set of associations with their brand identities, so I am not sure research assessment will be as affected as some may think.

The requirements on Chinese researchers to use more Chinese journals will result in bottlenecks given there are only 250 top tier titles, but the bigger problem will be with perceived quality and impact generally. International collaboration and authorship is one of the strong factors that make for reputation and the idea that there can be “nationalist” science is an oxymoron. Science is international.

Nevertheless this will stimulate more joint ventures, I think, as the pressure to get more and bigger Chinese outlets (even if co-published by or with Western publishers) grows.

Thanks for this post Tao. All of the above seems to apply primarily to STM publishing. Do we know if there are similar moves afoot in HSS?

There is no official policy announcement in HSS, but it is generally believed that move of weight from quantity to quality will be seen in HSS evaluation as well, and there is more calling for publishing in Chinese journals.

Thank you for sharing this important update. Tao, what is your opinion on the practicality of implementing such policy? It seems like the gate-keeping responsibility to monitor researchers are following these guidelines is heavily placed on libraries and universities. How practical is this to uphold?

Comments are closed.