If all (or even most) systems projects went well, we wouldn't need to talk about this subject …. But they don’t, and those failures compromise the progress and integrity of the information world.

The selection and application of formal methodologies is an important ingredient in the success or failure of most projects. In today’s automation project world, choosing the right project methodology sometimes seems as important as the project itself. Choose the right methodology from among the top contenders -- Six SigmaAgileLean and SDLC -- and, we are told, you are on the road to success. Choose poorly and you are alone on the windswept plain of project failure or mediocrity at best.

But even projects that claim application of rigorous methodology often fall short of their hoped for outcome. How can that be?

Most of today’s most popular methodologies, at least under their current names, have been around for a while -- Six Sigma since the 1980s, Agile since 2001, SDLC since the 60s, Lean since 2003 -- each seeing major growth as a required ingredient for management of systems projects.

The question for many people is how well we have done selecting and using these methodologies.

Outside each methodology’s cheering section, the answer appears to be mixed, with greater success in more detailed, technology focused projects, but less in the growing percentage of more lifecycle focused efforts involving groups and considerations throughout and external to the organization.

Getting Down to Basics: The Underlying Principles

A close look at the philosophies underlying most methodologies uncovers an interesting fact: all are based on a set of time tested precepts, expressed in differing ways, but the same nonetheless:

  • Include all relevant groups (customers, developers, managers, etc.) and encourage bonding and team spirit among them.
  • Establish and maintain strong communication, collaboration and information sharing among all participants.
  • Distribute authority for important decisions where and when appropriate.
  • Use rapid and iterative prototyping to identify, clarify and respond to user needs.
  • Use rigorous control methods where appropriate.
  • Keep entire picture in mind as project proceeds.

If this is true, then we might agree that a successful project must include these. The question then is how to make sure they are properly included. Experience with use of the current methodologies tells us that how things are done today isn’t the answer.

The problem is partly due to the fact that starting from the core principles, each methodology develops and applies increasingly detailed tools and techniques -- some say to claim bragging rights -- for implementation in specific projects. The problem with layering of specifics on general principles is that given the choice between the two, many projects find themselves applying the specifics, but ignoring the foundation principles that are the real objectives.

Moreover, while failures in projects based on these approaches are often chalked up to failures in application and lack of management support within the organization, some also blame the fact that most of the methodologies, developed in and for a manufacturing environment, were designed to improve repetitive processes -- Six Sigma from manufacturing at Allied Signal and Motorola, Agile from widespread frustration with the ponderous methods left over from early software development and Lean from automobile manufacturing -- and may actually not be useful in the one-off world of systems development projects.

It appears that checking the right boxes, assigning the participants catchy names and holding the appropriately titled meetings do not a successful project make.

Concurrent Engineering, Antidote for Misplaced Detail?

So what should we do when faced with a complex, lifecycle oriented systems project -- the type increasingly encountered in today’s connected world?