A Rubik's cube on a desk
How can you really pick winners and losers from a marketplace quadrant? PHOTO: Karlo Pušić

Facing highly fragmented technology marketplaces with many competing vendors, enterprise customers naturally seek out "horse race-type" analysis — who's winning and who's losing — to help narrow their choices. 

Major analyst firms have traditionally responded with 2x2 quadrants to show which vendors are ahead and behind in any given marketplace.

Quadrants Are Useful

Quadrants remain quite popular visualizations because they serve several useful purposes. They are easy to understand, offer a simple window into deeper research, and can provide a quick summary of a marketplace. 

No wonder they are ubiquitous. In fact, an emerging software category getting designated its own quadrant obtains a powerful signal that it has "arrived."

But Quadrants Are Often Misused

While it is easy to see the seductive appeal of quadrants, more often than not, they are misused. Of course, vendors will promote their favorable placement in a quadrant as proof of their virtue and it remains a key part of their marketing arsenal. They pay analyst firms for reprint rights to advertise their leadership status.

Yet it is surprising to see technology buyers unthinkingly bow to these shamans. Sure, a lot of analytic and quantitative rigor may go into the quadrants, though we could debate some of the criteria traditional analyst firms use. 

But what nearly all quadrants have is common is a tendency to rely on a single base use-case that — once generated — takes a life of its own. Vendors get designated winners and losers in a very generic way. Hence the intense pressure for vendors to appear in the top-right quadrant, and avoid at all costs getting consigned to the lower left, famously spoofed in this video.

Snapshots Are Misleading

There's a big problem to this snapshot approach. Depending on what you the customer actually needs, the same vendor can be either a winner or a loser.

For example, take this quadrant generated based on RSG's Enterprise Portals research. This research evaluates eight enterprise portal vendors, with detailed evaluations and quantitative ratings laid out in a 250-page report. 

Portals Quadrant #1

You'll see that all four quadrants are populated. RSG's RealQuadrant is a sophisticated decision tool but roughly speaking, the X-axis reflects product suitability while the Y-axis reflects the vendor company and ecosystem weighted to your preferences. 

Now take a look at the figure below. You get a different snapshot — with different winners and losers.

Portals Quadrant #2

How is that?

The underlying research remained unchanged: the vendors are the same, the products are the same, and the report narrative didn't change. But when I weighted some of the strategic considerations differently to reflect a different operational or risk/reward profile of a customer buyer, vendors shifted left and right, and a different snapshot emerged. Both are equally valid for the parameters chosen. The permutations are endless and you can have a parallel universe of different winners and losers.

The moral of the story is that you will want to weight certain factors differently than your neighbor, and such charts are very sensitive to underlying assumptions and parameters. 

You Are Unique ... Just Like Everyone Else

The second and perhaps more pernicious problem of using quadrants for technology selection is that they don't adequately convey the diverse set of potential enterprise business requirements, and how those will vary from customer to customer.

When an analyst builds a quadrant, they may be a using a lowest common denominator of requirements. Would you use your neighbor's requirements when buying a car for yourself? So, but why would you do that for technology?

Let me illustrate. Say you want to procure enterprise portal software so that you can improve internal collaboration. You are also looking for a solution that offers comparatively lower Total Cost of Ownership (TCO). Here is the list of best-fit vendors for you. 

Portals Quadrant #3

But let us say you are looking for a portal server that lets you easily surface business intelligence reports and dashboards from back-end ERP systems. 

Your management wants to get this done in a hurry, so you are looking for a vendor who offers professional services or has a channel with strong professional services. That generates a very different quadrant, with vendors shifting up, down and across the axes.

Portals Quadrant #4

As Bob Dylan sang - 

And don't speak too soon
For the wheel's still in spin
And there's no tellin' who
That it's namin'
For the loser now
Will be later to win...

The charts above illustrate just one software segment, but the song remains the same whichever category you pick.

Hidden Costs of Misfit Technology

Selecting the right technology up front is not sufficient for your success, but it remains an important pre-condition. The costs of a misfit technology are not readily obvious. 

Sure, it may not result in outright spectacular project failure. Instead you end up with budget overruns and project delays as you try to slap the misfit technology into shape. The blame often gets shifted to systems integrators and even your project managers. 

Even when you finally stand up the technology, user adoption may not take off and you'll be left wondering what went wrong despite the enormous resources you've spent on the project. 

CYA Cannot Be a CIO Strategy

You've heard the old enterprise saying, "Nobody ever got fired for hiring IBM." Picking a vendor from a generic quadrant is today's equivalent.

But the stakes have gotten higher today, as digital upstarts topple industry incumbents and leaders. CYA cannot be a CIO strategy. Make sure you pick the best-fit technology based on what matters to you - your business scenarios, strategic considerations, and your customized selection criteria.