Last week, Public Library of Science (PLOS) CEO (and Scholarly Kitchen Chef) Alison Mudditt released an open letter to the community in which she outlined the organization’s priorities going forward. Included in the letter was a note that development on PLOS’ ambitious workflow system, Aperta, was being shut down. While disappointing to many in the community who had high hopes for much-needed improvements in the manuscript submission process, this serves as yet another reminder of the complexity involved in building such systems.

Back in 2013, Joe Esposito wrote about the one universal point of agreement among all researchers:

What do almost all scientists agree about? The manuscript management systems employed by most publishers. Scientists detest these systems. They find them to be cumbersome and they express real frustration and sometimes outright anger over them. I will forebear naming names here, but some of the most prominent publishers and vendors of workflow management systems come under fire.

Having spent the last few weeks meeting with journal editorial boards, I can attest that editors aren’t too keen on most of the current offerings either. On both sides of the process, the systems in use seem needlessly complex and hard to maintain, with powerful capabilities often hidden from the user. One assumes that these systems, like most other commonly-used software, have accumulated a great deal of cruft over the years. So it would seem to make sense to start from scratch and build a new, modern manuscript submission system from the ground up.

But clearly that task is a lot harder than one would think. From the outside it seems a straightforward activity, but as Cameron Neylon explained back in 2015, this is an area where scale becomes a major problem, rather than an advantage:

If documents came into this process in a standardised form it would be easy. But they don’t. And they don’t on a large scale. One of the key elements we will see over and over again in this series is that scholarly publishing often doesn’t achieve economics of scale because its inputs are messy and heterogeneous. Depending on the level of finish that a publisher wants to provide, the level of cleanup required can differ drastically. Often any specific problem is a matter of a few minutes work, but every fix is subtly different…

…The few examples of systems that users actually like are purpose built by small publishers. Many people have said it should be easy to build a submission system. And its true. It is easy to build a system to manage one specific work flow and one specific journal, particularly a small one. But building something that is flexible(ish) and works reliably at a scale of tens or hundreds of thousands of articles is quite a different issue.

Neylon gets to the heart of the problem faced when building a submission system, namely the complexity with which they have to deal. From what I’ve heard, even the smaller bespoke submission systems he talks about require a significant amount of manual intervention, something that’s a complete non-starter for a large journal or set of journals.

In the intervening years since the post was published, the complexity of the material coming in to journals has only continued to increase as articles increasingly contain new types of media and new types of content (think data, or code for example). But perhaps even more than the variation in incoming documents, the great variability in editorial processes within and across journals is a major confounding factor.

In speaking with Allison about Aperta, the initial work involved modernizing and simplifying the user interface for authors, and she sees this a success. Where they ran into trouble was in working with editorial teams using the system. Each editorial office has its own personality, its own go-to features and its own “can’t live without” functionality. Peer review standards and processes vary from journal to journal. This makes it very difficult to build a system around a standardized workflow because there is no agreement among journals as to what that should be. Trying to accommodate even a limited number of editorial offices turned Aperta’s planned lightweight system into something heavy and cumbersome, much like the products already on the market.

There are no easy answers here. The editorial practices and workflows are important differentiators between titles. Standardizing all journals on one homogenous system would remove variation and choice from the market.

As Neylon notes, Elsevier has spent years and reportedly tens of millions of dollars trying to build a better submission system, and still runs a variant of Aries’ Editorial Manager. Thus, it’s perhaps not surprising that PLOS hit the same roadblocks that have plagued other attempts. Making these sorts of bold gambles has been a strength for PLOS, and not every gamble will fully pay off. On the positive side, the improvements in Aperta’s user interface offer value, and as Mudditt notes, “we’ve had great feedback on the interface from users and hope to be able to repurpose that experience in the future.”

So if not the planned revolution, at least there’s hope for some evolution to ease a major pain point for scholarly authors and it’s worth keeping an eye on how PLOS puts into practice those lessons learned.