Successful Automation by the NumbersIf you’re contemplating a moderate to major automation project, the odds are against success.

Studies suggest that projects involving a significant degree of technology usually cost too much, take too long and under perform, often creating major negative effects on the organizations that attempt them. While technology has raced ahead, our ability to deploy it has lagged. Indeed, we are scarcely better at successful projects than we were decades ago, a sad fact often obscured by organizations’ natural reticence to talk about their failures.

So why is this true and what can be done about it? While there are no magic bullets to remedy a failing project, there are some proven elements common to successful efforts, hence the following:

1. Starting Right: You can’t win if you don’t know your team

The largest impact and determiner of success, believe it or not, will be your organization, staff and current procedures. You may not like your current processes but like them or not, those processes and the people who perform them are the foundation on which your tomorrow will be built. These people will make your new engine run, and if you don’t know who they are and make them an active part of your effort at the outset, you will probably not be fully successful. They are also the people who will spell success or failure for whatever technological tools you decide to implement.

If things don’t work better, faster and more transparently when you are finished, all the technology in the world still spells failure.

2. The Good, the Bad and the Ugly: What works, and what really needs fixing

Most projects focus on the new technology to be acquired. But the setting in which that technology is to be used will ultimately determine its success, and the only way to understand and prepare that setting is to focus on the people, content and procedures in place at the outset. To use an only somewhat exaggerated analogy, buy any vehicle you wish, Fiat to Ferrari; its function will ultimately be based on how and by whom, it is driven.

The best way to get the impacted organizations involved is by creating functional teams -- a good working maximum is ten per team -- covering each impacted area of the operation, preferably with at least one knowledgeable representative from each area supported by a systems/technical “mentor” to help them ask the questions and express the answers in usable form. In Concurrent Engineering, these groups are called “cross-functional teams” and they work remarkably well when properly deployed and managed. They also create, as a side benefit, the review mechanism you will need when decisions are made along the project path.

Charge your teams with reviewing and documenting, in brutally honest form, their part of the life cycle of content and processes involved in the project. Asking people to critique their own functions can be difficult, but most will if they believe their input will be valued, and they often know things that will amaze you.

This will involve some painstaking investigation of current procedures and hours at the blackboard charting -- sometimes haggling over -- how and how well things flow. A good rule of thumb for this phase is “follow the content”: focus on the nature and journey of data through your operation with an eye toward minimizing and simplifying the operations required to get it where it needs to be in the formats required. You’re likely to find that much of what you are doing today is partially based on workarounds required by the limitations of earlier efforts, their supporters and their associated technologies.

This analysis is your roadmap to progress and without it, you have little chance of fixing or improving very much.

As you progress, you are likely also to notice that a majority of solution factors involved in your project are not purely technical but involve functional and procedural remedies. Indeed, getting a handle on what isn't working properly today will partially define the type of technology most appropriate for tomorrow. If you don’t do this first, you may not get another chance.

This departs from conventional wisdom in the systems and IT community and may generate some friction with the technical groups. You would hope not, but if so, better now than later when key decisions are in play and positions have hardened.

3. Automate the chaos and you end up with … Automated Chaos

The groups you are asking to share the spotlight -- IT and operations -- do not typically work well together so this phase will take some time and be … a little messy. But if you attempt to automate without addressing your underlying functional problems, the resulting “automated chaos” may be no better than the original chaos.

Next, using your functional analysis, have your teams describe the software tools you will need. Do this in strictly functional terms because before you can decide on what software to buy or build, you must know what you expect from it and what changes in your operations will be needed to make it truly useful.

Including non-technical people in developing technical strategy sounds scary and can be challenging, but it is the only way to find out how your project will impact and be impacted by your operational setting. Some technologists will sneer that operational people can’t do requirements analysis; they are usually wrong.

4. What you get depends on what you ask for; RFI or RFP

Now … issue a Request for Information (RFI) to vendors, large and small, who might have products and services that meet your functional needs (a market survey will help you identify them.) Make clear that you want each vendor’s best thoughts on how its expertise and offerings can address all or any major part of your needs. Ask also how well each vendor’s tools integrate with others: their level and scope of standards compliance for example. Request general pricing information but make clear you are not looking for a detailed quotation.

While there might seem little difference between RFI and RFP-based means of requesting responses from the vendor community, there are real and meaningful differences.

When most vendors receive a formal RFP, their sales group goes into high gear, preparing a bid, often passing up the technical groups and going straight to the price list. This happens because sales groups are penalized for not bidding virtually every request they get, yielding responses from even unqualified vendors who just don’t want to face criticism for “no-bidding.”

The RFI on the other hand, especially if you make clear that you are serious but not buying yet, usually gets to the design and technical parts of a vendor’s organization, often producing their best thoughts on how their offerings can help you. These groups are also much less fearful of admitting they can’t meet your needs, thereby qualifying the market for you.

5. Find the Vendors that can really help you

When you have reviewed the responses (many honest vendors who can’t meet your needs will voluntarily opt out) call back the best for in-depth sessions to present their solutions. Emphasize that you want their technical folks and best thinking, not just sales people and presentations. If they can’t (or won’t) meet that proviso, they probably don’t really want or deserve the business.

From these presentations, you can now:

  • issue a formal Request for Proposal to the most promising vendors … or
  • skip the competitive RFP and negotiate directly with the best vendor/s you found (if some really stand out and you aren’t required to conduct formal competition.)

Armed with your functional analysis and selected vendors -- all of whom have agreed that their offerings meet your functional needs and have given you a right of return if they don’t (most software contracts exclude functional warranties, but you can get close with a sufficiently detailed and accurate functional description) -- you can move to implementation. You will be far ahead of most projects at this point, knowing the impacts likely to grow from your efforts, having vendors truly prepared to provide what you need, and having operating groups involved and prepared for the impacts of your project.

6. Do Your Technology with a Full Hand

As you move into development and implementation, your functional analysis will provide the basis for scheduling and in-process reviews to keep the project on time and within the boundaries you have set for functions and impacts. If you have a preferred development strategy, -- Agile, SDLC, waterfall, etc. -- you can use it, but must take care not to interrupt the functional approach you have begun.

This takes time and effort well beyond that expended on most automation projects… but is highly characteristic of successful projects across many industries and application areas. It also tends to reduce the cost of technology deployed by reducing operational costs thereby paying for a portion of its cost.

We live in an age of incredible technological power, often deluding us into thinking that our salvation is in our technology. But there have been successful and failed projects at every stage of technology, from the days of punched cards forward, suggesting that the differences between success and failure aren't in the technology itself but in how and how well it is planned and used.

We can ignore this reality at our peril or use it to reap the rewards our new technological world offers us.

Title image courtesy of Lightspring (Shutterstock)

Editor's Note: To read more from Barry, check out his The Battle for Data Supremacy: The Cost of Ignoring XML