I recently programmed and chaired a one-day conference for UKSG, with a theme of The scholarly communications ecosystem: understanding and responding to evolving expectations. I invited some researchers to consider the extent to which they find current systems for disseminating their work fit for purpose, and what improvements (if any) they might seek. Meanwhile, publishers, librarians, and providers of related technology and services talked about the efforts they are making to keep up with researchers’ evolving needs. As we started the closing panel discussion, I asked the participating researchers to rate the current “system”. The result? Reasonable, reasonable, poor. I assumed the frustrations would be focused on things like access issues or clunky formats. And yet my notes from the event show that I had probably set the entire day up with the wrong premise. Once again, in focusing on the role of publishers, libraries, etc., we are tackling the symptoms, not the cause of the problem. The real frustration for researchers, which came up time and again, from different perspectives, is that institutional systems for evaluation (rather than publishing-related systems for dissemination) are no longer fit for purpose. I’ve summarized the most thought-provoking points from the day below.
- Stockholm syndrome? Early career researchers love the work that they do, but feel unfairly treated by institutional structures for recognition. Some of those interviewed by Anthony Watkinson and his team for the CIBER Harbingers project categorized themselves as “slaves to a publishing-based reputation system”, but others didn’t like the use of slavery as an analogy, acknowledging that their adherence to the system is self-imposed.
- Buridan’s ass? Which format do you choose for publishing your work — the one that will get it out fastest, or the one that is most respected when it comes to evaluating your performance? Jane Winters talked about “edited collections” (of e.g. conference papers) as the place where the newest humanities research is likely found, while noting that in terms of impact and visibility, publishing in an edited collection is like “throwing your research down a well” (the format being insufficiently mainstream to contribute to readership or impact for your work). Current evaluation of publishing formats leaves researchers with a tough choice between speed and impact. Winters also talked about the challenges of “making your first book sufficiently different to your thesis” in a world where theses are commonly published online, and where early career researchers are expected to have a “balanced publishing portfolio”, including monographs and journals, but also popular periodicals, blogs, newspapers and so on. She advocated for more guidance from publishers, institutions and societies to help scholars know how and where best to focus their efforts.
- Square pegs, round holes. Several times during the day we touched on how difficult it is to get interdisciplinary research published, because it doesn’t fit the established journal verticals; Micheál Ó Fathartaigh had joined with others to set up a new journal, partly to address this. Seth Cayley brought the topic to life with a closer look at some digital humanities projects (hands-down favorite was the study/ngram that showed that, for one year only — 1964 — the Beatles were indeed bigger than Jesus), the serious point behind the fun being that the cross-disciplinary nature of such projects makes universities nervous about putting them forward for evaluation within traditional frameworks — institutional conservatism meant that even award-winning research was not considered excellent enough to be put forward for the UK’s most recent research evaluation exercise (REF 2014) for fear that it would not “score” well. (Ironically/encouragingly, as Jane Winters pointed out, those digital humanities projects that were put forward actually did quite well).
- Etiologies of open? Though not expressly on the agenda, open access (OA) and open science ran like threads through the day — one of our researcher panelists was a co-founder of the OA journal Studies in Arts and Humanities, our library speakers had collaborated to form an OA repository and press, our publisher speaker talked about re-usability of published research (look out for the imminent launch of the Scigraph.com linked data platform from SpringerNature). You could take from this that open is mainstream, an established rather than evolving need, not something we needed to focus on specifically. On the other hand, Sabina Michnowicz talked about researchers having to fund OA from their own pockets, and Jane Winters lamented the institutional conservatism that means humanities scholars still consider OA too “risky” a model for communicating their work because it is conflated with a lack of rigorous peer review. The framing of the issues around open has moved on: a newcomer to our sector might have concluded from the discussions that it is institutions, not publishers, who are blocking progress towards open.
- You can lead a horse to water, but you can’t make it drink. The concern about outworn higher educational processes was also evident in discussions around researcher evaluation. Winters talked about the many activities expected of a “good academic citizen” (and reminded us of the relatively low prominence of publishing by showing where it sits in the guidance being given to early career researchers by her discipline’s society). Steven Inchcoombe talked about practical ways of helping institutions evaluate the “bigger picture”, such as enabling the time invested in the review process to be better recognized and credited. However, Michnowicz’s letter to the Higher Education Funding Council for England, asking that peer review contributions be recognized in research evaluation, and received a scant, and negative, response.
- Keepalive signals? The publication process did not, of course, get off scot-free. Ó Fathartaigh characterized peer review as the process of watching a loved one sail away, not knowing when they will be back, not hearing from them for months, “wondering how she’s faring”. “Is it too much to ask for the occasional update?” he pondered. Inchcoombe had earlier cast “inefficient” peer review (siloed and linear submission/rejection processes) as “publishing’s nasty secret”, and talked about how improvements can be made both in terms of systems (a new manuscript submission service has reportedly reduced turnaround times for selected SpringerNature journals by 50%) and in terms of processes (SpringerNature’s “transfer” service undertakes the reformatting when a submission is transferred between their journals — well received by the academics in the room who had joked about 30 page submission guidelines). The publishers among the audience/twitterstream thought updates during the peer review process “do-able”, and brainstorming around this might make a useful workshop session at a future conference — at what wayposts during the submission/peer review process could updates be sent to authors? (Do any Scholarly Kitchen readers already have such a framework in place?) Acknowledging the role of academics themselves in slowing down review processes, Andy Miah proposed book-sprint style hack days for peer review to incentivize and reward quicker review actions. Meanwhile, James Harwood presented Penelope, a Publisher Natural Language Processing tool (PNLP – geddit?) which performs a range of automated checks on submitted manuscripts (formatting, etc.) and marks up documents with simple “looks good” and “check this?” comments to speed up author feedback/manuscript processing times. It has been used by about 500 authors so far, and typically finds 10 errors per paper. Cengage now provides text and data mining tools (e.g. data visualization software) to help researchers make the most of the content on its platforms without needing to be digital experts in their own right.
- Trees falling in a forest? Researcher panelist: “it would be great if libraries organized events like today — a showcase of all the services out there to help researchers”; librarians in the audience: <mass implosion>, because I think nearly every librarian there had organized something exactly along these lines and had scant attendance by researchers — the panellists themselves acknowledged that they ignore emails from their libraries, with Andy Miah mentioning that he really only communicates via What’s App. And don’t expect scholarly collaboration networks to provide the answer: all the evidence on the day was that researchers aren’t using these sites for communicating, collaborating or discovering. They are simply shop windows, places to ensure they are visible; collaboration is still growing primarily from face to face connections at conferences. (I don’t think the day provided any answers as to how best to ensure researchers are connected to the guidance and support being offered by libraries and publishers; my own position on this is simply “multi-touch, multi-channel”: tell people over and over and over again, through every available medium, what you are offering. The majority aren’t annoyed by the repetition, they’re oblivious to it.)
- Change happens. Winters criticized organizations that alienate humanities researchers by using “science” in their messaging — among others, she flashed up the Publons’ homepage with its “speeding up science” tagline. Her point was widely tweeted. Before she’d finished her talk, the Publons homepage had been updated! If only all change were as simple to effect. But what a welcome reminder that change can happen.