“Agile” can mean different things to different people. That’s partly because that’s what buzzwords tend to do, but it’s partly by design. Agile isn’t a prescribed project management framework like Waterfall, it’s a general philosophy of working that encompasses a wide variety of practices and processes. These range from organizational and time boxing structures for developers (the most common are scrum and kanban), to people ops Human Resources practices that focus on self-organizing teams and co-created organizational structures.

What these views of Agile don’t address are the fundamental business reasons that make Agile a good idea for certain activities. What’s worse is that if the reasons why Agile is good are not understood, it can and will get used for the wrong things, resulting in poor outcomes, and loss of confidence.

Technology and product managers that are trying to become more Agile often decry a lack of executive buy-in. There is certainly plenty of well-meaning advice in the blogsosphere about how to secure buy-in and quite a few companies exist that offer c-suite Agile training. The problem with almost all of the advice available is that it’s too operationally focused, with high-level explanations of things like scrum and cross-functional teams. What’s often missing is how Agile impacts strategic decision making and planning. In other words, How does this help me?

The temptation is to appeal to cost. After all, isn’t that what organizational leaders want? More revenue and lower costs to boost profit margins? Once a strategy or product is decided upon, changing the way a company delivers innovation should surely only be a good idea if it’s cheaper, right? Well, no.

Cost isn’t the real enemy, it’s risk.

Lemonade stand economics

Imagine you’re setting up a lemonade stand. You need lemons, sugar and water. Fortunately, the water is open source, so you only need to pay for the fruit and the sugar. When you go to the store you have a choice between basic lemons or tree-ripened, locally sourced organic lemons. If you buy the basic lemons, you’ll spend $10 (or £10/€10 if you like) and will be able to sell the resulting lemonade for 15. If you buy the fancy lemons, that’ll cost 15 but you’ll sell the lemonade for 25, that’s twice the profit, right?

Except for the fact that if the dog from next door knocks over the jug, all the lemonade will go down the drain. Then you’ll have to tell your Dad, who capitalized this misadventure, that you lost his $15. You know that’ll make him less likely to lend you money in the future.

Perhaps it would be better to spend less of Dad’s money this week and use the profits to buy your lemons next week. You will have spent more on ingredients by the time you start making premium lemonade, but you will have reduced your risk (and maybe adjusted your recipe a bit based on early adopter feedback).

Innovation as an ongoing process

Businesses often seek to defer the costs of investments on the balance sheet. It makes sense, particularly for physical assets like offices, factories, and computers that those costs are measured against the future income generated from the investment. 

There’s an inherent danger here, particularly if a project is risky, like innovation and IT projects often are. Nobody wants to be in over their head having capitalized significant expenditure on a project that doesn’t deliver. It’s one thing if the realized revenues are lower than expected or the project overruns, but the consequences can be more serious if it has to be scrapped.

Take, for example, PLOS’ much talked about 2017 financial overview. As fellow Chef, Joe Esposito points out in the comments section of that post, the poor on-paper financial performance is at least partly due to having to write down the cost of a large technology project. To be clear, PLOS were using Agile development methods on Aperta, but they did capitalize a very large amount of spending on a single program that was serving the company poorly enough that it had to be cancelled. It was undoubtedly the right thing to do, but it would have been better to do it before the penalty for failure had climbed to around a third of PLOS’ annual revenue — ouch!

PLOS isn’t the only publisher that has faced such challenges. We work in an information industry and information is complex. The systems we work with, from conventional submission, content management and distribution platforms, to cutting edge workflow and research management products, often support multiple stakeholder workflows and interact with multiple data sets. There will inevitably be unforeseen use cases, requirements will change and compromises will be made. Even if product owners were omniscient, market conditions and technology landscapes shift over time so that large projects can be out of date even before they launch.

Agile, with its focus on delivering value in short, iterative timescales, reduces exposure to these risks. In cases of truly innovative or disruptive products, it’s possible to push a minimum viable product (MVP) to market and get direct customer feedback within weeks of a project starting. If a product doesn’t validate, it can be failed quickly with far fewer consequences.

Having worked for and with a number of companies that use Agile development practices, experimenting with feature sets and even whole products is an integral part of Agile innovation. It’s hard to imagine creating a cutting edge research management application or a new way for researchers to share their data with collaborators without having the freedom to test things without excessive risk.

It would be possible, of course, to develop a minimum viable product using traditional, Waterfall style project management. The difference is the feedback loop and iterative learning that is key to Agile’s effectiveness. An MVP is pointless from a product development point of view unless you actually learn from the market feedback.

On the flip side, it’s not always necessary to actually push something to market to create an Agile feedback loop. Jeff Patton’s seminal blog post and book on user story mapping describe how user-driven development can be used even for products where big releases with complete feature sets are needed. One case study he gives is about Brazil’s largest media company Globo.com. They use story mapping to plan the development of a fantasy football app that accompanies the Soccer World Cup. They don’t get to test that in the market — it has to be released on time and has to work. The approach relies on product owners representing the end users through a series of personas — imagining who they are and how they’ll interact with the product. Alpha testers and development client-partners can also help with this.

Iterative learning is the key to risk reduction

The shorter the distance you travel down the road before you realize you need to adjust your steering, the less likely you are to hit a pothole. PWC put it like this in their 2017 white paper Agile Project Delivery Confidence: Mitigate project risks and deliver value to your business.

Regardless of the delivery method used, the core underlying project tasks, dependencies and environmental sensitivities remain the same. Agile is not a silver bullet. There are still risks, but when doing Agile Project Delivery, the team has the opportunity to respond to risk earlier in the delivery lifecycle due to ongoing visibility and continuous improvement.

Introducing Agile approaches within an organization does not necessarily make innovation cheaper or faster. In fact, depending on the operational maturity of an organization, it can cost more and take longer, at least at first. What it does is significantly reduce risk by shortening feedback loops and making work more visible. This offers the opportunity to learn and improve before things get out of hand.

A good example of this approach is Hindawi’s use of Coko’s open source code stack. I spoke with a couple of members of their executive management team earlier in the year, and learned about some of the course corrections that they made as they developed the first phase of their platform. They’re currently using the new system on a single journal as they continue to incrementally add features and workflows to the point where they can scale to more of their portfolio.

So, should your organization undergo an Agile transformation and use it for everything that you do? Probably not.

In a way, Waterfall vs Agile isn’t really a fair comparison because they are two different things. Waterfall is a specific project management framework that is well suited to situations where the requirements of a project are fully knowable. Agile, on the other hand, is a philosophy that favors fast feedback loops and shared goals. It has led to the invention of a number of methodologies. The two approaches are therefore not mutually exclusive. Organizations can and do mix them very successfully. 

The key to that success is to recognize that Project/product/program management choices are not purely operational, they have associated strategic risks and opportunities. In order to make the right choices about the frameworks and methodologies used, it’s necessary to understand their consequences at every level.