With the year coming to a close, it’s a good time to take stock and reflect on what we have observed — since the last time we stopped to reflect and made mostly the same observations.
So bring out the soapbox and head for the exits.
It’s standard practice nowadays to talk about “the business model.” This conversation takes place in companies large and small, for-profits and not-for-profits (NFPs). The term doesn’t necessarily mean the same thing to everybody. For some, it’s a simple answer to the direct question, “How does this company make money?” (Answers vary: “We sell advertising,” or, “We sell subscriptions,” etc.) For others, it refers to a sophisticated financial exercise, almost always performed (yes, performed, like complex gymnastics) with Microsoft Excel. These Excel models have become works of art.
I saw one not long ago that consisted of a series of linked spreadsheets that covered assumptions and multiple scenarios. What happens if the average price received for Segment 9 drops by $0.32? Well, everything gets recalculated instantly; note in particular the changes in Segments 3, 11, and 26 and the impact on cashflow, which in turn will alter the borrowing rate — which of course was recalculated. It’s stunning — and cool. This particular model had sufficient complexity to manage a transaction of some size — say, the acquisition of Reed Elsevier by GE (this rumor started here). But the particular case for which it was developed was for an Internet company with no revenue. And all the assumptions of the spreadsheet proved to be wrong.
Welcome to the world of premature quantification, where the allegedly scientific appeal of numbers masks an absence of sufficient forethought and strategic intelligence. It is not curable by taking a little pill; to fix the problem requires a major overhaul of organizational culture. I am not optimistic and expect that I will be reposting this essay next year.
Premature quantification strikes for-profits and NFPs alike, though usually in different ways. In the for-profit world, the classic instances of the ailment take the form of a Christensen effect, where a financial analyst attempts to “model” a new venture and finds it lacking because it is not forecast to generate as much profit as the organization’s existing businesses. This is not a trivial problem, and one has to feel some sympathy for companies that fall into this pit. Take Google, for example, which has a stunning high-margin business in advertising sales. Just about any new business Google gets into will have lower margins than the existing advertising business, which makes it hard to get into new activities, as they will make the company as a whole seem less profitable by some measures. For such companies, the quantification is premature because the analyst cannot possibly anticipate all the ways a new capability could be deployed. Indeed, even Google had no idea how they were going to make money when they started. Had someone insisted on a detailed financial model at that time, the company may not have gotten its original venture funding.
If Google seems like too exotic an example, think of what it must be like to review new opportunities at hugely successful publishing companies such as Reed Elsevier, John Wiley, Springer, and Wolters Kluwer. Dear Mr. (Ms.?) CEO, the memo begins, I have a great idea for a new service that will deliver a whopping 5% return on sales! Well, let’s move on to the next, the more serious, proposal. Most of the time that means acquiring another company. What this leaves out is the possibility that the 5% business could grow into a 20% business or better in the years ahead.
In NFPs, premature quantification appears to be a relatively recent development, as straitened financial circumstances have made funding bodies and governing boards demand greater “accountability.” That’s a great idea in theory. What it often means in practice is that staff with little experience with numbers (and who may have come to the NFP environment precisely because they sought a mission-based rather than a commerce-based professional life) now must quantify things they never thought of in this way before. One recourse is to hire a consultant (I personally have been the beneficiary of this many times), whose task it is to come up with the “business model” that will make everything seem just right. Of course, as a former colleague used to say, you can’t perfume a pig — a financial model is only as good as the thought that went into it, and rarely does that kind of critical analysis and strategic discussion get enough attention. Okay, let’s put this into numbers: the time put into strategic exercises should be 99% of the total, with the quantification of the strategy taking up the other 1%.
I mentioned this topic to a friend recently, who remarked that the recent crisis on Wall Street would not have been possible without the widespread use of spreadsheets. How’s that again? His point was that the orderly presentation of financial data on spreadsheets invites the perception of greater certainty than may be justified. Thus, people made major decisions based on the appearance of careful analysis where in fact there was none. An alternative strategy would have been to talk with some of the people before they bought homes they could not afford and to the mortgage brokers who gleefully lent them money. But that’s not “scientific,” that’s merely anecdotal. The democratization of finance has been a terrible mistake. Numbers are hard, as we know when we see our kids struggling in their algebra classes. Spreadsheets turn everybody into a wannabe financier.
This is not an anti-quantification post. Publishing companies — for-profit and the NFPs — are full of people who will have nothing to do with numerical estimates. They appeal to intangibles (“Nobody knows how many copies we will sell”) or to cultural mission (“We are here to do this kind of thing even if it will not make money”). Bad or premature quantification is good for nobody, but intelligent financial analysis is very important. The real question is the quality (and quantity) of the work before the financial model is put together. In fact, organizations have to create a culture of quantification, but the numbers have to be good numbers, and they have to support a deep analysis of strategy.
What gets lost when the spreadsheets come out too early — and serve to put an end to the discussion of strategy — is the iterative nature of operating a business. Think of the Public Library of Science, which was launched with a business model (premium journals with high fees to authors and a demanding peer review system) that later proved to be financially untenable. When PLoS began to adjust its model, many critics declared (and some jeered) that PLoS didn’t have a sound business model; some went so far as to indict the very notion of open access publishing. They were wrong. After some false starts with pricing, PLoS launched PLoS ONE, which operates on an adjusted model (lower fees, higher acceptance rates, a different form of peer review), which is proving to be successful. What the critics failed to see is that the real business model, if that is the proper term, is not something easily rendered in financial forecasts, especially for early-stage enterprises. The real model is the intuition and flexibility of the management, which performs an experiment, evalutes the results, adjusts the parameters, and tests once — and twice and three times — again.
One hopes that as we look forward to the New Year, we will indeed count our blessings. Let’s tally one point for the creative individual; another point for the special dynamism of creative people working together; a point for serendipity; another for the social institutions that gave rise to the creative staff in the first place. You get no points for making a mistake, but 5 points for learning from a mistake; and 10 points for learning from the mistakes of others. In the end it all adds up.
And with that we begin the countdown to the New Year.
5 Thoughts on "Breaking the Silence Around Premature Quantification"
The role of quantification for NFPs is important but also limited in ways that relate to the prioritization of mission. As director at Penn State Press, I had to balance concern for not running up a large deficit against the overarching goal of enhancing the University’s reputation by publishing books of high quality, which would win prizes. My working assumption was that pursuing high quality as an editorial strategy, rather than basing decisions on marketing projections, would ultimately serve the Press’s best interests, and that ultimately quality would also produce quantity in revenue at least as much as pursuing sales maximization directly.
For decisions at the operating business level, quantification could play a greater role. As Joe well knows, since he was involved as an advisor in one of these episodes, the Press three times went through the exercise of analyzing the costs and benefits of maintaining the Press’s own order-fulfillment operation. The business manager and I both favored outsourcing, and that’s what Joe recommended also, but since the Press benefited financially from not having to pay any rent for its warehouse space provided by the University, in strict quantitative terms the decision not to outsource made sense. Part of the problem in making the case for outsourcing was speculating about the point at which demands from large vendors would require investment in new technology that would tip the balance financially. The business manager and I knew that day would come and wanted to start the process, but the immediate negative impact on the Press’s finances for making the change proved decisive every time. So, how does one plan for the future if quantification ends up acting as a constraint in the short term?
Quantification is often used as a proxy for caution, in my experience. It looks rational and reasonable, so its practitioners are insulated from negative effects. But as Joe rightly points out, the lack of a strategy is often part and parcel with this practice, so asking what the strategy is remains vital to proper planning. As Sandy points out in his comment, if the Press had identified “future-proofing” as a strategy for its infrastructure, what was quantified and the interpretation of the analyses might have been very different.
I think this also has some relevance to Michael’s earlier post this week about core competency vs. what business you’re in. Content providers (publishers) have come to believe that fulfillment and printing are core competencies, when really they’re not. CRM and content creation are closer to the mark.
A very thoughtful piece, indeed.
Yet, the false precision that is applied to numbers is not limited to finance — a field that aspires to scientific standards — but to science itself.
As a reviewer of scientific manuscripts, my most common critique is the authors’ over-specificity of accuracy, and part of this blame needs to go to spreadsheets and statistical software, which often report data with a standard number of decimal places. (In the case of Excel, it is 8 decimal places.)
So a small study with a few dozen respondents may include statements like, “24.138% of respondents feel X,” when it would be more appropriate (and no less accurate) to report “about one-quarter of respondents…”
CFOs aren’t the only ones dazzled by numbers.
I love the phrase “premature quantification.” I agree with you that so much of the time forecasts are driven by incomplete data and inaccurate assumptions.
I also have been guilty of creating inter-dependent Excel spreadsheets that trigger multiple results by changing just a single assumption. But I am also very careful of making sure these assumptions are defined as such — educated guesses.