In a previous contribution I made suggestions about how to use scoring in an effective way in selecting a vendor shortlist. Now, based on the observations made at around 150 presentations I have endured over just the last four years, let's talk about the management of vendor presentations.
It is so easy for these to degenerate into simultaneous Q&A sessions between individual members of the evaluation team and the vendor team. Only a few days later does the evaluation team realize that they still do not have a clear idea about which vendor to select.
Here are ten recommendations to make sure that the evaluation meetings are win-win for both parties.
1. Allow Three Hours for the Presentation
This may seem a long time, but you are going to make a business-critical decision. An hour is totally inadequate, and what will usually happen is that the vendor senses you are not a serious prospect and sends in the B-team just to avoid canceling out on you.
After brief opening statements from both sides I would suggest giving the vendor (and implementation partner) 90 minutes to go through the technical presentation. Take a 20 minute break away from the vendor team and agree on some of the issues that need to be clarified. That will take a further 30 minutes. Then give the vendor 20 minutes to summarize and allow a final 20 minutes to run through a provisional project plan and schedule. That’s why you need three hours.
2. Agree in Advance What you are Looking For
This may well be different for each vendor. It could be image management with one, and metadata management with another. Every member of the evaluation team should have this list available at the meeting.
3. Use Scenarios
In the tender document you should have set out some use cases for the CMS. “A writes the news story which B then reviews. C then adds in images and links and D does a final quality review”.
In setting up the presentation provide these in more detail to the vendor, together with sample content. In the presentation you are looking for the number of screens, clicks and drop down boxes needed to accomplish these tasks.
4. Keep IT Staff Off the Evaluation Team
A vendor should not be on the short list if IT has unresolved concerns about server architectures and technical support.
5. Have a Content Contributor on the Evaluation Team
They will have a very different perspective on the ease-of-use of the CMS, especially in intranet usage where they may only be contributing a few items a month.
6. Keep the Evaluation Team Consistent
You are probably going to do four meetings over two days, and then have a final wrap-up meeting. It is essential that the membership of the evaluation team is consistent. If someone cannot make a commitment to beat all four meetings find an alternate who can. Circulate bios and roles of both the evaluation team and the vendor team in advance so that 20 minutes is not wasted in introductions.
7. Video the Presentation
After four presentations it will be very difficult to remember just what was said, and by whom, no matter how good the note taking. Just point the video camera at the screen and record the conversations. Offer a copy of the video to the vendor for training purposes and most will be very happy to know that you are taking the evaluation meeting very seriously indeed.
8. All Questions Should Go Through the Meeting Leader
No matter how disciplined the evaluation team, within 30 minutes the meeting can become a Q&A riot. The meeting leader should have the authority and wisdom to know when to allow the question and when to keep moving on and catch up later
9. Don’t Allow the Vendor to Mock Up your Site
If you do then it is easy for the questions to focus in on irrelevant details about the color palette the vendor has chosen, and totally miss the point of the evaluation process. It is much better for the vendor to use that time practicing the scenarios!
10. Make Sure the Vendor Project Manager Turns Up
This is the opportunity to decide whether you can work with the project manager. Excuses about the person concerned not being available won’t wash with me. If the vendor wants your business they will make sure they bring the project manager along. You will want to see them taking a major role in the presentation, and not just sit there looking bored.
Finally do the wrap-up meeting and make a decision as soon as possible after the evaluation meetings. If the decision is still too close to call then it is always better to set up a further meeting than toss a coin.