A recent story in BusinessWeek caught my eye. It dealt with allegations that Apple, Intel, Adobe, and Google had colluded to control the compensation of software engineers and programmers by agreeing not to raid one another’s staffs, even going to the extent of having written agreements to this effect. A class-action suit brought by 64,000 engineers and programmers resulted in a settlement of $324 million for the plaintiffs — hardly damaging to these giant firms (it represents 0.4% of their valuations), but certainly a sign that technologists are a rare commodity worth fighting for.
This has some bearing on our own experiences with technology, which continue to be underwhelming, especially as platform providers attempt to be “one-stop shops” for our technology needs, and we continue to have the expectation that they can be. I can’t visit another publisher or talk with a colleague without the topic arising — we don’t have enough platform providers to choose from; those we have are too limited or difficult to work with; option A is good at this, but not good at that, while option B is good at that but not at this; and we want everything at prices our mostly non-profit firms can afford.
Of course, these complaints are rife with contradictions. We want unlimited horsepower at low prices; we want more platform provider options but a “one-stop shop”; and we want them to be both unlimited in their scope and simple to work with.
Our customers provide fuel for these contradictions. They want journals to be cheaper, believe technology is inexpensive to build, operate, and maintain, and compare us to multi-billion-dollar firms and their technological capabilities (“Can’t you just do it like Amazon does?”). Even with resources essentially semi-pooled at the platform providers, we don’t have anything approaching the kind of financial heft needed to meet these expectations. As noted above, even the multi-billion-dollar firms have been caught capping and trapping expensive engineering and programming talent.
One response to the situation has been for publishers to hire staff to deal with technology, but these publishers seem to have one of two experiences:
- They have a hard time hiring qualified technology people, underscoring the experience their platform providers must be having
- They hire people who are good at dreaming up and/or describing requirements, but don’t prioritize or winnow their demands to a few key initiatives, which only exacerbates the problems associated with limited bandwidth while also clogging deployment channels
This latter problem bears some examination.
There is a good deal of workplace credibility to be gained by establishing yourself as extremely persnickety about interfaces, user flows, online design, online functionality, and so forth. Yet, often these same people are not asked to justify their demands in economic or financial terms — i.e., is there a practical difference in getting it “just right” by their standards? More importantly, they are not asked to justify their demands in terms of actual user needs or desires.
Search engines provide a good example of this kind of behavior. They are a special magnet for unique demands and idiosyncratic expectations, for many reasons.
First, search engines are at the nexus of the problem of input/output (or GIGO–Garbage In Garbage Out, otherwise known at help desks call “the PICNIC problem” — Problem In Chair, Not In Computer [also known as the “I.D. 10T” problem]). This refers to those users who put in search queries that nobody with any facility with searching would ever enter. In most search engines, less is more. But some users will enter what they think was the full title of the article, something that often sends a search engine into tailspin, especially if conjunctions and articles are included, and if the internal logic is “or” instead of “and.” Or they will enter punctuation, extra spaces, or typographical errors. Even after doing so, any substandard result is the search engine’s fault. Maybe so. But worse, any frustrating experience becomes an excuse for that person’s inner dramatist to emerge, and for a number of vague requirements to stream forth. Woe to the online development crew who has a person of influence who has these habits and takes a shine to the task of “getting it right.”
Second, there is the vanity problem, which invariably leads to authors searching for themselves and being disappointed when the results are either incomplete or overly inclusive (many search the references as well as the author fields). Glaring problems can occur for a few reasons. In some papers, authors were listed by initials, but by full name or some combination in others. In some cases, authors were acknowledged outside the actual authorship line, so they were not included as authors per se, but still feel they were authors. In other cases, search engines crawl references to return papers that include mention of the name being searched, believing this to be a service. In one memorable case, the author in question realized that, whoops, he hadn’t published in the journal he was searching, but in one with a very similar name. No matter what, it’s a problem with the search engine.
But is it worth fixing, in an era where most article searching comes from outside your own site?
A third situation that can arise with search engines is that editors have editorial ideas about how search engines should parse responses into certain conceptual bins or linked zones they can imagine. These ideas can be compelling for the editors involved, but a quick dose of user testing or a guided session of prototyping often breaks the spell.
Pick your technological mire — video players, audio players, article displays, home page designs, tablet versions, mobile designs — it doesn’t matter. In any situation, we mostly encounter a two-sided problem — publishing staff members who gain rewards and recognition by pushing technology into a more perfected state, and technologists who are spread too thin, are hard to find, and who are expensive.
Clearly, this can cause discord, just as any situation where the expectations are far greater than the other party’s ability to deliver. And when it comes to centralized services like platform providers, it can lead to finger-pointing or hard feelings, which often makes the relationship all about the relationship, and less about the customer and the business.
Our technology partners and providers can and often do find ways to improve offerings and increase our capabilities, and many provide great custom offerings. I think this is because they, like many of us, find the rewards of mission-related work in the sciences elicit dedication and strong contributions. In short, we give more because we care, and we get more because they care.
But when multi-billion-dollar companies are so protective of their software engineers and programmers that they basically agreed to a four-company “no touch” policy, it’s no surprise that our little online publishing industry has a difficult time finding the talent to do complex engineering and software development work at reasonable prices. Remember, we’re an industry in which 81% of the companies have less than US$25 million in annual revenues. There are very few companies in our industry that can afford champagne solutions. Heck, even Apple and Google were trying to keep things affordable by colluding to keep their technologists captive.
There are three solutions I’ve found that work — first, focus on the revenue-generating aspects of your online properties, not on the aesthetic or technical aspects; second, supplement platform providers with other providers to compensate for the inevitable gaps these providers will have; and, finally, test ideas with users beforehand, because often the idea you find so appealing will fall flat with actual users, making blind pursuit of it an exercise in futility.
Technology is one of the most difficult things to manage in the modern publishing organization, a moving target with an evolving ecosystem of standards, expectations, and possibilities. It is also something not many customers are willing to pay for. Managing all three of these within budgets that are nowhere close to limitless is, to me, part of managing technology relationships — when resources are stretched thin and the stakes can easily be made to seem higher than they are.
16 Thoughts on "Keeping It Real — Are Our Technology Expectations Out of Whack?"
Kent a great article. One of the most difficult tasks is maintaining a membership or subscription list. I forgot how many ways a person can enter their name but is exceeds a dozen!
This is a great example of how something you’d think would be so simple becomes very complicated to implement technologically. We just spent months creating a unified database of our rather small set of customer records. With long-term customers, who enter their names in a variety of ways, and who move a few times over the course of the relationship, or who use home and work addresses in some cases but not others, even managing a customer list can be expensive and time-consuming, requiring lots of behind-the-scenes technology.
ask yourselves as to why these companies need the software engineers that they have, and more:
a) the systems that they are developing work across many clients who want the same or similar intelligence that Kent has outlined. That is a system with “intelligence” that is adjustable
b) the systems such as Google and Amazon find “you” with infinite misspellings because their search engines have very large data bases that every day get better and better at not only identifying who, but what, where and a few more “w’s”
These systems make the companies vast returns for accessing and interpreting what books you want, what foods and merchandise you like, possibly facts that you wish were dark and figuring out how you have mangled the title, size, color and country of origin.
As you point out, a small industry can’t come close to getting this and amortizing the cost, not PLOS and not Springer or Elsevier unless they are able, like Google, etc to amortize the cost over other uses/clients.
if clients such as government intelligence agencies use these systems, what is the rationale for a small industry and smaller members trying to sort out all the issues that Kent has described.
c) there is more than faint hope that the industry can regain credibility by providing AI to replace or at least supplement a faltering and fatally flawed peer review system overwhelmed by the demands on the peers from many directions and not having access to the intelligence that these systems can provide. I am not asking for IBM’s Watson. But it does address the many issues that close reading of the peer review system would reveal more than age spots beneath the cosmetics.
What is at stake, or maybe steak, is a Soviet style industry encumbered by, perhaps 150 years of tradition, potential loss of jobs, and similar issues faced by disruptive innovation.
You hit the nail on the head with this one. There are few options for platforms and none are perfect. Future strategies for content development center around technology these days. How can the publishers make it better, faster, more useful, etc? We are competing with what has become the expected customer experience with Amazon and online news services, etc.
One might be tempted to reach outside the traditional publishing platforms but that too is a risk. It is costly and you (the publisher) end up spending a ton of time explaining things that are unique. These technology companies don’t have a clue about ORCID, Fundref, DOIs, CLOCKS, etc. The publishers are not in a position to coach a new vendor through all this and not doing it correctly can have disastrous results.
You mention the new Publishing Technology positions popping up at societies everywhere. This has become a really critical position but finding people who understand the content and have the technical chops is difficult. To add to that, people who do have the technical chops may find better paying positions elsewhere.
Reaching outside our industry to other technology providers shows you one thing in two guises — that each industry has its own foundation of business and custom solutions. When we go outside, we’re unlikely to find our general business and custom solutions. In addition, we’re very likely to find some other industry’s business and custom solutions. This bilateral reality creates a gap that can be impossible to close, as neither side is willing to abandon core activities to accommodate the other.
There’s more than money at stake. Technology personnel know things that you’d rather the competition not learn. Companies like Google, Apple and Samsung go to great lengths to prevent engineers from developing a public profile on the theory that you can’t target what you can’t locate.
Good post. I am sitting here with a sight chuckle as in yesterday’s SK’s post and a number of comments laid out a plan for University Presses to expand their offering to include sophisticated commerical sites and how easy it is to build and maintain web storefronts. Then today you cover have difficult it is to mange a hosting platform and to retain programming staff. My observation having worked with a number of publishers that hosting platforms, access control systems, fulfillment systems and manuscript tracking systems all require careful management. Sales teams often sell licensing agreements that operations cannot support. Having worked with various size publishers has demonstrated how difficult it is to keep up with technology and hold on to your hat when you decide to switch systems.
Managing technology is the most difficult part of publishing.
Dan, no irony here at all. You misunderstood yesterday’s post. Building a Web store is a very different thing from a hosting platform. For a Web store there is a set of established suppliers (Bibliovault, CoreSource, etc.), for a hosting platform the options are fewer, but I would say still very good.
Not at all Joe, I have not seen any really good systems at any UP operation. Maybe I am missing something. I thought the life bread of the UP was textbook adoptions. I am not including OUP, CAP, or U of Chicago. Libraries shift to on demand acquisitions programs has hurt UP sales. Journals and database sales have captured much of the sales activity leaving book sales in a difficult position. I have not found many publishers that are satisfied with their hosting platform. I had a conversation today with a publisher on that very topic.
I can say “amen” to that, having been director of a small university press during its transition into the digital age. There were so many challenges to confront, so few monetary resources to devote to them, so much pressure to make the transition quickly, and so few people one could trust to do the IT work responsibly that it’s a wonder our press was able to make the transition at all. But we did, and our secret was partly that we had the tremendously good fortunate to find an IT specialist who had the breadth of knowledge about software and hardware and the necessary vision and ability to prioritize that we were able to accomplish much with little. This process involved making key decisions about what to outsource and what to try developing in house.
Fantastic article. As a start up we have to compete for this same talent and as Nathan Myrhold (founder of Microsoft) once said, “one really motivated and great software engineer is not twice as good as a mediocre engineer, they can be 10,000 times better”. This is absolutely true and I see it every day. Great software engineers can make money anywhere, but working on things they care about is something they all want at some point.
Finding and getting that kind of talent and motivating them to solve big problems is the art of modern business. It is what Apple, Amazon, Google and others do at scale which is not easy. I have been in startups and I have worked at Amazon and the common denominator is a driving sense of urgency.
I hear so many old line companies saying “we are launching our new App” as if that is the end of the equation. Being good at digital is a 24/7/365 business and largely an exercise in constant self-cannibalization. The DNA to be good at it cannot be imported or taught. It has to be woven deeply into the psyche of the teams working on the problems and how those folks think, live and work. The work only starts when you “launch the App” or your “digital strategy”. People oversimplify that all the time and this article helps much in dis-spelling that myth.
On the other side, it has never been cheaper or easier to develop world class technology if you have the right approach. For businesses leveraging the cloud correctly, operating costs are a fractions of what it costs old line businesses and vendors to do the same thing. What once used to be a market advantage (we can afford to run our own data center and hire hundreds of engineers!) is now an albatross.
Small, lean companies without oversized management layers, etc. can upend markets quickly. The companies doing this right and passing on the cost savings to customers will create winning businesses. People talk about Amazon.com changing bookselling and publishing, but it is a drop in the bucket to the impact they have had on the world with Amazon Web Services.
Again, great insight and perspective. Thank you.
“But is it worth fixing, in an era where most article searching comes from outside your own site?”
This is the exact answer I get from our publisher when I pass along author complaints about the search function on their platform. My question is why don’t publishers just remove the search engines from their sites altogether rather than having search functions that they know are not actually functional? Why frustrate your users this way?
Kent, great set of thoughts here. You could replace “publisher” with “library” in your article and not lose much in the translation. The challenges – and the typical responses – are similar.