unicorn on prow of ship
Agile methodologies promise greater flexibility and responsiveness, but they may only work in limited circumstances PHOTO: Neil Williamson

All of you software development methodologists and theorists, I’m here to ruin your day. 

I am about annoy you to call your baby ugly. 

How? By calling out just how poorly your pet development methods — Agile, user-centric design, continuous deployment and DevOps — are working in real life. 

Agile: Nice in Theory

Nowhere in the development world do we see the difference between theory and practice so clearly than in these new, “modern” development methodologies. 

They are supposed to enable companies to better respond to changes in the business climate by delivering higher quality software at a faster rate. Agile especially is meant to drive rapid iteration and hence, faster evolution of software. 

That’s nice in theory. 

The truth is much different, especially for large organizations trying to develop software at global scale. 

Without a doubt some number of companies have found success using these new methodologies. Yet as I have come to know more and more people who are trying to implement these methodologies in their organizations, I’ve begun to wonder if the success stories are the exceptions, not the rule.

Common Problems With New Development Methodologies

The following problems consistently arise with Agile, DevOps, etc.:

1. Agile is not just a development strategy, it’s a whole different structure for business

The entire organization must be aligned to developing software in flexible and collaborative teams. That’s great if your business is developing software. If it’s not, then other aspects of the business are less likely to be able to or have the incentive to adapt to these methodologies. 

If your business is health insurance, why would you structure your company around software development rather than delivering insurance plans? 

In most companies, software development is meant to serve the business, not the other way around. The majority of workflows and organizational structure is meant to deliver a product or service other than software.

2. In any given company, much of the development is still Waterfall. 

Let’s face it, change in a big company comes slowly. Implementing a new way of doing anything is a gradual process. Unfortunately, without a near fanatical devotion to these methodologies, they tend to flounder. Try matching existing Waterfall project management and an Agile one. They won’t sync up without superhuman effort.

On top of that, the people who have worked in an organization for the longest time, the most experienced developers and stakeholders, are used to a long, drawn out process that defines everything up front. Trying to adapt to fast moving, independent and integrated teams with much less up front definition is difficult and, for some, impossible. 

Now imagine both forms of projects running at the same time while being interdependent of each other. It’s chaos.

3. Most developers don’t understand Agile to begin with. 

Sure, they can develop in short spurts of work we call Sprints. That doesn’t begin to address the all-in, collaborative, highly empowered approach of true Agile development. 

Half-Agile efforts provide all the stress of Agile without any of the right type of team support or independence.

One of the worst mistakes that companies make is throwing software at this problem. Just buying an Agile development suite such as IBM BlueMix or JIRA doesn’t create a collaborative and empowered environment — it only supports it. 

Too many Agile development efforts put in the software and the sprints but fail to create the right environment or give the teams the autonomy they need to make Agile a success. 

4. Speaking of resources …. Agile and continuous deployment assume ongoing and constant development.

Just because a team produces a minimally viable product doesn’t mean that’s the end. Agile is a commitment to continuing development forever. 

That does not align with big company budgets and resources. No one seems to properly resource for Agile. Teams members get yanked away to other projects or move on to other development after the first major end user release. 

Software development budgets are rarely designed for the long term. Project budgets are not like operating budget but are more akin to capital budgets. You can’t fund Agile projects with moonshot budgets.

Ultimately, big companies are used to funding projects, not building long-term capacity in software development. That’s not at all strange when you consider that software is not their primary business.

5. Agile, DevOps and other new, more collaborative, methodologies generally only work when there is co-location. 

Teams using modern methodologies need to be in constant contact with each other, otherwise many small impediments add up to a major drag on the development process. 

Not only do developers need to be in easy proximity to each other, but analysts, product owners and major stakeholders must be able to communicate with each other easily. Time zones and travel distances can slow a project to a crawl while waiting for responses. Even the best collaborative software can only alleviate — not eliminate — the time dilation effect that distance brings to a project.

This doesn’t translate well to large global companies that may be running multiple teams and projects throughout the world. A 12-hour difference may mean a two day turnaround for answers to simple questions. It’s ask, wait, discuss, research, wait, answer.

6. Back-end constraints are major impediments to making what customers want. 

Ideally we would create user-centric products with intuitive interfaces all the time. That might work when starting from scratch. 

That’s not how it is in most big companies. Already established systems are monumentally expensive or risky to change. The capabilities of ancient systems and heavyweight middleware drag new development down, since they were never developed with fast and flexible development in mind.

7. DevOps is not even the thing most companies think it is. 

It is a philosophy of interdependence in IT, implemented as collaborative processes and teamwork. Politics, old divisions and lack of attention by senior management make this more wishful thinking than a practice. 

Developers and systems operators don’t communicate as much as they should, let alone create integrated teams.

Worse yet, when many companies approach DevOps, they look at it from a software tools point of view. They throw all types of orchestration tools that, while useful, can’t really affect the end-to-end process problems that DevOps is supposed to solve. 

Why? Because of all those pesky people with real personalities that get in the way. 

DevOps is a threat to the individual fiefdoms that have developed over the years which provide great power and comfort to different constituencies. Few managers want to tackle that problem head on and instead opt to invoke DevOps, toss some tools around and hope that will solve the problems.

8. Continuous releases sound great in theory but confuse customers and stakeholders alike with a constant flow of new features. 

A constant flow of new features can be hard to absorb. So even if Agile, DevOps and continuous deployment work smashingly, end users still need to manage the intake of new or changed features.

That’s fine when you are adding a couple of features a month to productivity suites such as Microsoft Office365. It’s much harder when you are changing mission-critical software in a regulated environment. 

End users need to get on with the business of the business, not with learning new software every month.

Sure, every release doesn’t have to equate to a customer or end-user feature release, but the purpose of these methodologies is to allow for greater change over shorter timeframes. 

The average knowledge worker can only absorb so much change. After their absorption rate is exceeded, new features can become a distraction and negatively affect productivity. 

Failing to Reach the Promise of Agility

What worries me the most about these new methodologies is that they may be hard to push to global scale. 

It’s one thing to have a handful of Agile projects. It’s quite different to synchronize several large-scale development efforts with major interdependencies. The management overhead of multiple small independent teams can be high enough to weigh down these projects. 

More than anything though, large companies are still hierarchical command and control structures. That type of environment is toxic to Agile, DevOps and other new development processes.

It may well be that the few success stories are just anomalies or that these methodologies only work at small scale or for certain types of companies. 

Either way, I consistently see large companies trying to adopt these methods, grabbing for a promise of a more flexible and responsive software development process, only to fall flat. Maybe it’s time we rethink all of them.