A client is in the process of selecting an enterprise search application. The project started in February, and we hope to begin a Proof-of-Concept validation in September. The two major surprises for the client have been: 1. how long the assessment process is taking and 2. the difficulty of comparing the claims and capabilities (often quite different) of vendors at both the RFI and RFP stage. We have considered over 70 functional and non-functional elements during the assessment process.
2D or Not 2D? – That Is the Question
Reducing these diverse capabilities to the two-dimensional 2-by-2 schematic approach adopted by Gartner and Forrester looks impressive, but how useful is it? For starters, Gartner and Forrester differ in their vendor coverage. Their selection criteria omits over 50 other vendors. The reports are a reasonable starting place to get a sense of the opportunities and issues involved in selecting an enterprise search application. But they are only that — a starting place.
Both Gartner and Forrester agree Coveo and Sinequa have been the "best in class" for the last eight years. However, the similarities between the two companies are small compared to their differences. Based on these reports, that will not be obvious to IT directors looking for search solutions.
To me, Coveo's primary focus appears to be on support function search, given its wide range of partnerships with productivity-related applications. The openness Coveo is showing at present in supporting the SIGIR 2021 ecommerce data workshop is impressive, so if I was working on an ecommerce project, Coveo would certainly be on my RFI list. Sinequa would not make the list despite its high rating because it makes no pretense to be in the highly specialized ecommerce business.
In the case of Sinequa, I see a significant global competence in a range of industry sectors, notably biosciences, manufacturing, government and financial services. Most enterprise search applications involve supporting North American and European operations, so the fact that Sinequa has major offices in both the U.S. and Europe is helpful. In addition, many organizations in this sector want an on-premises solution. Coveo only offers a SaaS platform.
Related Article: How Much Does Enterprise Search Cost?
Connectors, Languages and AI/ML
For all three of these deliverables — connectors, languages and AI/ML — the devil is in the details and not in the marketing pitch. All search vendors promise a wide range of connectors and seamless federated search, but unless the user interface can offer usable and predictable access to the information in these applications and manage the security issues, the number of connectors is irrelevant. The value of user-managed interfaces is now widely recognized, but Sinequa and Coveo have adopted significantly different approaches here. One isn't better than the other, they just support different business cases.
Counting the number of languages a platform supports (something the analysts ignore) is also not an indicator of either index or query-time capabilities. And then there's AI, which too often is treated like a technology and not a concept with no agreed-upon definition. The days of black box search are over. If a vendor is unprepared to adopt the principles of explainable AI, reject it and move on!
Related Article: Has Microsoft 365 Been Clinically Tested?
Enterprise search has matured significantly over the last three decades, which means making a decision based on a scoring matrix never works. The non-functional attributes involved in enterprise search should strongly influence your choice for the best platform for your requirements.
- An excellent knowledge of how information is used in industry sectors.
- Significant industry and technology skills in the vendor and integrator project team.
- A well-developed project management capability.
- A coherent proof-of-concept approach.
- Training and comprehensive post-implementation support.
- A customer community scheme.
Requirements invariably emerge once teams have begun to assess the power of the incoming application. All too often the specification is written with the existing application as the basis rather than starting from current and anticipated user requirements.
Installation and Implementation
Installation is a project, implementation goes on forever! Both Gartner and Forrester emphasize integration success, overlooking (because they cannot assess it) the balance of roles between a vendor, its integration partner and the customer in solution delivery. Selecting and then implementing a multi-language global search application is a complex project, which might take 12-15 months from user research to user satisfaction. For me, a crucial issue is the extent to which an early-stage implementation can be stress-tested by real users. Ideally this needs to happen within three to four months of contract signing.
Search Is Not Simple
Enterprise search is a classic "wicked problem," caused primarily because organizations fail to regard information as an asset and ignore content quality. The core issue with the two-dimensional approach is it seeks to demonstrate how simple implementing enterprise search is. If only!
So often I see vendors rejoicing in the number of connectors they offer or the speed of search in delivering highly relevant results on the first page of search results. Unfortunately, there is no link between search speed and search quality. Optimizing the balance between precision and recall is very challenging. Analysts that present vendor assessments in 2-D, with vague comments and false equivalents, are not helping the situation.