Gartner's been getting a bit of attention lately. The Gartner Enterprise Search Magic Quadrant released in July resulted in criticism from Miles Kehoe, Stephen Arnold and Charlie Hull. Nuix heavily criticized the MQ on e-Discovery and Scott Liewehr has reservations about the Forrester Wave on Digital Experience Delivery Platforms. And now the lawsuit.
My own views on the Gartner Search MQ were a little less forthright. However the Search MQ raises issues which are much wider than whether the companies in the top right hand quadrant (Leaders) deserve to be there.
Is a Picture Worth 1000 Words?
Here lies the problem: Consultants (and I’ve been in the business for over 35 years) love to be able to create some form of graphic to capture the attention of current and prospective clients and seeks to establish some form of intellectual property protection for their analysis. For example, Gartner just released a guidance note on Copyright and Quotes which runs to over 4000 words. Despite its title it makes no reference to the doctrine of "fair use" under the US Copyright Act.
I’ve developed many such diagrams in the course of my career and learned that the skill lies not in the development but in knowing when the diagram no longer fits the purpose. Occasionally someone develops a framework that stands the test of time, with the information management maturity model developed by Doug Laney (now with Gartner) in 2002 being an excellent example.
Although I have no quantitative information to support this assertion I would guess that a four-quadrant presentation is by far the most popular. It fits nicely on a PowerPoint slide, can be easily created without any design experience and generates a convenient set of four explanatory slides with the minimum of effort.
The scientist in me questions just how sensible it is to reduce complex issues to four quadrants, despite the description from Gartner as to how the Magic Quadrant is developed. The only person to win the Nobel Prize for (in effect) a set of diagrams was Richard Feynman, and he was a genius!
Search is Not a 2-D Technology
Gartner stresses that just because it names a vendor a Leader doesn't mean it will be a good fit for a particular organization. While true, try telling a CIO who knows little about search technology that her organization will be better off with a Challenger or a Niche vendor rather than a Leader. Vendors that use the Gartner MQ as the equivalent of the ultimate accolade aren't helping matters, but it does say a lot about the reputation that Gartner has built up over the years.
In the rubric for the MQ, Gartner notes that “There are good reasons to consider market challengers. And a niche player may support your needs better than a market leader. It all depends on how the provider aligns with your business goals.” Reading that, there seems to be no benefit at all in considering Visionaries.
The search business needs visionaries to find solutions to some of the near intractable business challenges that organizations of all sizes are facing. That innovation is coming mainly from the open source community, which does not fit easily into an MQ format. Elastic Search is a good example. It has attracted roughly $100 million in investment from venture capital institutions that have many other sectors and companies to choose from, so it is difficult to understand why it is not included on the MQ.
Understanding Business Requirements
Gartner emphasizes that organizations need to understand their search requirements before making a decision, but that is a complex process which few organizations have the skills and/or experience to undertake.
Sadly there is not much competition for Gartner and IDC. One of the best reports on Enterprise Search was published by Ovum in 2011, and that ran to nearly 300 pages and 20 vendors. The Real Story Group used to publish very good analytical reports on search but no longer do so. The rapid adoption of open source search and the much improved search functionality in SharePoint 2013 are just two of the many factors that IT and business managers need to take into account in developing an enterprise-wide strategy for search.
Garter MQ report includes a sentence in the Market Definition/Description section that really intrigues me:
Organizations are no longer satisfied with a list of search results -- they want the single best result.”
Now that really would be magic!
Title image by Syda Productions (Shutterstock)