My absolute favorite tool of the Enterprise Architect as he or she navigates amongst the sea of paradoxes that face them as they identify and stitch together the packaged, open, and custom solutions that make up the modern enterprise is the good old-fashioned “bake off”. It’s also sometimes referred to as the “beauty pageant”, but this metaphor doesn’t work for me as a well designed bake off is intended to reveal the ugliness beneath the slick marketing sheen of either a COTS package or a SaaS offering.
Enterprises need the right solutions to their problems AND they need them in a time frame that will allow the business to meet the needs of the rapidly shifting marketplace. In short, they want to dictate the speed of delivery and the position of the thing delivered (where the position in question is defined as “the right solution” for the enterprise). The problem as stated seems to defy the laws of physics. Werner Heisenberg famously proved that, when it comes to particle physics, the more precisely one property of a particle is measured, the less precisely the other can be controlled, determined, or known.
Try as they might, many technologists and their business counterparts cannot transcend this problem. The discussion goes back and forth with the following cycle:
Business owner -- “Our current tool to manage process X isn’t meeting our requirements anymore. Can we replace it with something that meets our needs?”
Technologist -- “Sure! Given that we will have to live with this decision for more than a few years and that this is a big decision with a lot of dollars attached to it, I think we should do a platform selection.”
Business Owner -- “That sounds good. How long will that take?”
Technologist -- “A formal selection can go anywhere between 6 to 12 months, depending how many alternatives you want to look at.”
Business Owner -- “So you want me to wait a year to pick something and we still won’t have it replaced? I can’t sell that to my leadership.”
Technologist -- “I’m sorry, but picking an expensive package is complex. You don’t want to jump into bed with a vendor and then find out you’ve recreated the same problem you started with.”
Business Owner -- “I’m sorry too, but a delivering something in 18 months or more is not a solution at all.”
After this discussion is over, both parties walk away with a common thought: “He/She just doesn’t get it!”
Luckily, Betty Crocker has a way to solve this Zen koan in a time frame that is a little bit longer than baking a cake but still comes in way shorter than the formal platform selection process; The Bake Off.
Ingredients: An inscrutable business problem, an impatient sponsor, several vendors and a multidisciplinary team capable of being comfortable in ambiguity.
Step 1 | Compose a long list: Using a variety of reference sources, identify a “long-list” of possible vendors. This can be done either by working with the analyst community or even through reading the trade publications or talking with trusted colleagues in different companies.
Step 2 | Capture requirements: Collaboratively, identify and document high-level business requirements. It is important that this not be done too thoroughly as it will not actually provide any value at this point. The goal here is to be just detailed enough to be able to combine the critical high level requirements into somewhere between 6 to 10 use cases.
Step 3 | Create a brief RFI: Work with your legal and procurement teams to put together a high level brief of your business problem and ask respondents from the long list to submit a company profile, pricing models, and a selection of relevant reference implementations with customer contacts. You can also ask for compliance statements to verify that their solution will meet the high level requirements in a out-of-box fashion, with customization, or not at all, but this is actually a waste of time for you and the vendor, because they will find a way to answer with what they think you want to hear.
Step 4 | Design the bake off: While the vendors are creating their responses, create 6 to 10 high-level use cases. Each of the use cases should:
- Begin with the phrase “Demonstrate how your product can be configured to…”
- Directly correlate to a “must have” high-level business requirement.
- Contain some complexity or constraint that will, when addressed, reveal how hard it will be to configure and/or maintain the product relative to your changing business needs.
Additionally, you can create a scoring matrix, but experience tends to show the final decision will ignore the scores anyway and will be made qualitatively. A possible alternative is to create a “showstopper list” of items that could eliminate a vendor from consideration (e.g., no support for PCI compliance in an ecommerce package, no integration with SSO or a corporate LDAP).
Much more important than the scoring matrix are the complexities and constraints in the use cases. These are the critical ingredients as they must be balanced between being simple enough to allow a rapid prototype but complex enough that they will force the vendor to show you how the system actually works in the real world and how the artifacts and components are customized and maintained.
Step 5 | Create a short list: Based on the written responses, cull the list down to 2 to 4 vendors and notify the vendors of the phase completion with instructions for the next phase to the short listed participants.
Step 6 | Bake off: Go visit the short list vendors and have them demonstrate their approaches to meet the functionality described in the use cases over a period of 1 or 2 days. Alternatively, have the contestants come to your shop and demonstrate the solution in your environment. If you choose to have the vendors come to you, be prepared to hear excuses about the complexity of getting things to work so rapidly without being able to control the environment.
As each vendor completes their demonstration, have the observation team record how the solution will respond within their environments and have them pay special attention to the “ugly compromises” and “maintenance kludges” that are intrinsic in all software offerings.
For the more adventurous teams, you can have the contestants set up their solutions in a production or pre-prod type of environment to run a battery of performance tests on the systems.
Step 7 | Select a winner: This one can be hard and will often pit competing interests against each other, but that is surprisingly unimportant when one realizes the answer to the koan. There is no “right” solution.
The Answer Is Mu
Mu is roughly translated as "no thing" and basically means that the question cannot be answered as it is currently formed. The reply of Mu in Zen koan terms is stating that the question needs to be unasked before it can be answered. As Malcom Gladwell has explained on the TED stage, there is no perfect pickle!
The object isn’t actually to pick the right solution, because there is no “right” solution. The actual desired end state for the bake off is for the enterprise to know what it is signing up for before it jumps into bed with the vendor, and if you crafted your recipe well enough, then all the peccadilloes of the selected vendor will be known in advance and can hopefully be planned for.