#jboye09: Selecting a CMS - Pitfalls and Best Practices

We’re live from the City of Smiles, where the J. Boye conference kicked off today in Aarhus, Denmark. The first day is the Tutorial Day with topics ranging from content strategy and governance to persuading people with digital content.

One of the tutorials -- CMS selection: the process, the pitfalls and the best practices -- was presented by analysts Jarrod Gingras of CMS Watch and Peter Sejersen of J. Boye.

The session was prefaced with a statement that there’s no perfect system. Making your CMS requirements too special may make it nearly impossible to find a system that will fit them.

Dos and Don’ts in Choosing a CMS

When looking for a CMS, look for both a software vendor and a system integrator, if they’re not part of the same package. Make your project attractive for bidders. Make it easy for them.

Here are some additional dos and don'ts that were offered:

  • Do not employ a detailed scoring methodology
  • Do not develop a full list of requirements
  • Buy a scoping exercise before buying a full-blown
  • Control the dialog with vendors
  • Do not look beyond 3 years

What to consider:

  • Project timelines
  • Stakeholders
  • Budget
  • Creation of a business case
  • Objectives
  • Requirements
  • Marketplace
  • Process
  • Decisions

In the best cases 3-7 months, worst case a year or more, according to Sejersen, who quoted J. Boye’s report on Selecting a CMS. Your level of complexity will affect the timeline, added Gingras. This is not a trivial process, but you need to move fast. In EU, the open tender process may make the process even longer, when vendors have around 40 days to respond to your RFP.

Who Needs to be Involved

Marketing folks, developers, a project manager, system admins, application admins, website manager, etc. This mix of groups depends on your organization. Engaging the project champion and the project manager are absolute musts, said Gingras. It’s a political battlefield you want to navigate carefully, when many (or too many) people are involved and you need to make a decision. You need to communicate and make it clear what their role is. They’re not making the decision, but provide input.

Don’t select a CMS before developing your requirements. Gather the requirements only after the business case is completed. How do you justify acquiring a Web CMS system? There are 3 dimensions to a WCM Business Case:

  1. Greater efficiency and reducing costs and frustration largely through automation
  2. Reduce risks (a system can provide insurance, but also introduce new risks)
  3. Enhance value (look to get more value from your content, improve quality by implementing standards)

What Should be Included in the RFP?

Understand internally what you’re trying to achieve. The short and effective RFP is not more than 10 pages. It can start with a brief intro to the project and your organization and include project scope, goals and characteristics. As well as:

  • Specific needs and business scenarios
  • Technical architecture and standards
  • Structure and design of the platform
  • Role of the vendor
  • Schedule and evaluation criteria

Do not issue checklist RFPs – unless it is absolutely required in your organization. When vendors get it they throw it to the most junior person in the organization to fill out based on a previous example. In many cases not a lot of time get put into these answers, and the possibility of cut and paste from a previous checklist is higher. A checklist alternative is providing specific, testable scenarios. But don’t go overboard – 5 to 7 scenarios should be enough. Entering a CMS vendor contract, is like entering a marriage. You have to like the people you work with.

Key vendor differentiators:

  • Geography
  • Budget
  • Size
  • Technology
  • Vendor intangibles (chemistry, company focus, financials, trust, quality and experience)


You don’t want too many vendors on your shortlist. Somewhere between 5 and 7 vendors is ideal, and more than 10 is too much. Do your research and get input from blogs, peers, network and external analysts. Do not assume that there’s a best vendor/product. The analysts acknowledged that many reports and analyses exist but noted that they are useful as sources of information, but will not tell you which the best option is for your specific context.

Reading and Evaluating Proposals

After you get your RFPs back, look for indications of heavy copy and paste activities. More importantly, check if vendors understood your scenarios. Check their references. Many are willing to share their experiences (the good and the bad) and can provide candid feedback. And, of course, look for pricing and license models. Finally, develop an evaluation matrix, so that you don’t have to waste time on the losers.

Vendor Demos

When running a demo, make sure your team is well represented, but keep your team size limited, as 100 plus people is probably an overkill. Beware of canned demos. They often look good but fail to address your specific needs.

Give vendors the rules up front and tell them what you want them to show. This way they know to keep focused on your scenarios. Ask questions about pricing and be sure to understand which modules are part of the core system and which ones are add-ons which might cost more.

If you’re thinking of involving a system integrator, do include them in your demo processes. Keep in mind that licensing cost is usually the smallest part of the total cost of ownership of the CMS. Usually, it is recommended to have a scorecard to evaluate each demo. Formats may vary, but look for pros and cons in two areas:

  1. product fit
  2. vendor fit.

With open source CMS, you have to look at the vibrancy of the community. Make sure to debrief immediately after the demo, so you document the impressions properly and don’t forget who said and showed what. One of the ideas in the audience came from Martin White of Intranet Focus who suggested to videotape the demos (even though vendors may not like this) and compare them later.

Bake-off and POC

When you’re down to 2 or so vendors, you can do a bake-off using real scenarios and real content to get the real feel of the product and work with real people from the vendor or integrator side. It can be expensive, but what’s the cost of a failed implementation?
Do a proof of concept before you dive in completely. One of the benefits is to get your users to try the CMS.

Making the Final Decision on a CMS

How do you make the final decision? Blindfold, roll the dice, flip a coin? Do the due diligence and evaluate the vendors from the point of view of:

  • Viability and stability
  • Support and community
  • Services and channels
  • Strategy and roadmap

You can also attend group meetings and user conferences, which would be perfect opportunities to check references and get honest opinions. You’re not just buying software -- make sure that you meet the people you’re going to work with.

It is very challenging to decipher the licensing models, as they can be CPU-based, user-based, server- and domain-based, etc. etc. Don’t take this exercise lightly and don’t underestimate the costs. The advice is to budget 2 to 4 times the license cost to allow for integration, consulting and customization expenses. Open source systems are not entirely free, keep that in mind when making your vendor selection and deciding on the budget.

Finally, yet importantly, don’t just sign the contract before you negotiate.