If you spend anytime online, you have likely participated in an A/B test without knowing it.
A/B testing allows a web-based service to randomly direct users to two or more variations of an interface (A or B) in order to measure some kind of outcome like clicks, purchases, or recommendations. An online store may be interested if you were more likely to purchase a related item if the products were displayed on the right side or at the bottom of the screen. A political campaign may be interested in whether slight differences in wording affect how many people donate or join a mailing list. A news organization, like BuzzFeed, may be interested in measuring clicks to different variations of a headline, and those who produce long form journalism or podcasts, like NPR, may be interested in testing different introductions to the same story.
And while scholarly publishers were early adopters of web-based technology, interface development is not a high priority for many organizations. After considerable time, effort, and resources have been spent, a single online interface is launched and remains largely unchanged for years until some executive or subcommittee is charged to investigate something new. Rarely are multiple interfaces tested simultaneously, and the ultimate decision that will lock the publishers’ interface into another cycle of stasis is often based on the organization’s Highest Paid Person’s Opinion, a term in organizational behavior that receives its own acronym, HiPPO.
This got me wondering whether this is something different about scholarly publishers that dissuades them from putting much emphasis on interface design or the process that leads to incremental interface improvement.
None of the major journal platform providers who responded to my inquiry currently offer A/B testing, although some now work with third-party providers to provide A/B functionality. The American Medical Association, which hosts its journals with Silverchair, uses Optimizely — a software that can redirect traffic to different versions of a website, gather and summarize user statistics. Matt Herron, web developer and product manager for the AMA told me that he has used A/B testing for online registration, branding, and testing various elements on journal articles. HighWire customers who use the open source content management system, JCore, (see Science Advances and The BMJ) can implement A/B testing although those publishers using HighWire’s native H20 interface cannot. Similarly, A/B testing is not offered to Atypon customers, but should be available sometime in the future, according to Gordon Tibbitts, EVP of Corporate Development for Atypon.
A/B testing is not going to solve large interface problems but it will allow publishers to engage in incremental testing and improvements so that website evolution becomes a more gradual process rather than an episodic event.
There are costs to adopting an incremental change approach, and small publishers lacking sufficient staff may decide that old-school is good enough because it allows them to focus on generating high quality content rather than chasing web design trends. What worries me about this approach is that publishers do not have a monopoly on their readers’ experience. Scientists may be just as likely to retrieve a journal article from a large subject-based repository, personal websites, or to a growing extent, from commercial products like ResearchGate and Academia.edu, as they are from a journal website. These non-publisher conduits of literature do not invest in the creation of content, but rely on a free supply of papers uploaded by their users or scraped from other sites. And by serving only as content providers — rather than content producers — they can focus entirely on providing superior user experiences.
If your organization is entirely focused on creating content and doesn’t care where or how it is consumed, then interface should not be a concern for you. However, if there are compelling reasons to keep readers coming back your site (community cohesion, branding, advertising, COUNTER downloads, among others), than journals must consider improving the user experience. It all begins by making a choice between A and B.