Selecting a CMS: Managing Product Demos

By Seth Gottlieb (@sggottlieb) Mar 11, 2010

If you followed the advice from the first two articles in this series (How to build a short list and Developing scenarios), you should have a good idea of what you are looking for and with what products you might find some content management system bliss. This next article provides guidance on how you can start evaluating actual products against your defined requirements.

This next phase of the selection process is where you evaluate the products against your requirements. Successful completion of this phase will mean that you have selected a product/vendor that is compatible with your content and your way of working. The product satisfies both your objective and subjective criteria.

Failure in this phase means that you will either be swayed by the most charismatic salesperson or that you will be stuck in a never ending sales cycle that doesn't drive you towards an informed decision. Neither case is very appealing — so lets avoid both.

Take Product Demos Seriously

The vendor presentation and product demonstration is one of the most critical components in a CMS selection process. You will learn more from seeing a product in action than reading an analyst report or a RFP response from a vendor.

But to be effective, a product demonstration needs considerable investment from both sides. You won't learn anything by occasionally peaking up from your email to glance at a canned demo about a fictional business that has nothing to do with your company.

Instead, you should partner with the vendor to develop a prototype that supports the scenarios you have given in the RFP. In this exercise, you get to experience what it feels like to be a customer working with the vendor to achieve success. If you run a demonstration properly you will be able to answer the following questions:

Does the vendor understand your business and the way you work?
Will you be treated like an important customer?
Does your company and the vendor have good chemistry?
How naturally does the product fit your vision?
What customizations or compromises would have to be made to use this product?

Construct a Thoughtful Invitation

The first thing you need to do is connect with the vendors on your short list to tell them that you are evaluating their product and could use their help. This is where most companies go wrong. By pulling out their standard RFP template and loading it up with demands one shoots oneself in the foot, and early.

The worst of such offenses have a glob of irrelevant boilerplate text and then a long feature checklist. One CMS vendor I know even received an RFP that asked if the product contained any radioactive materials — clearly this was language designed for another type of procurement and the customer was too lazy to even read what he sent out.

Trust me, when you do work like this, you are sending a signal to the vendors that you don't care. Some vendors will not engage at all. Others will play along but invest as little as possible in the opportunity — they know that the sales cycle will be long and unpredictable.

You get what you give: an RFP with a lot of boilerplate text will get responses with a lot of boilerplate text. Plus, you will get stuck in "qualification" queue until you show some signs of intelligent life.

The good news is that, if you have done the work of developing scenarios, you have a lot of information that shows (a) you are serious about this initiative and, (b) you know what you want.

Vendors love scenarios because they efficiently tell the story of content in your organization and help them understand what you need. In addition to your 10 most important scenarios, your RFP should contain the following information: