storm trooper figurine with a growth chart
PHOTO: Daniel Cheung

"What is the return on investment of a search application?" I've lost count of the number of projects which started with this as the primary objective, either to justify additional investment in the current application or to buy a replacement. With so many suggestions on how to build a business case for enterprise search, I thought it would be worthwhile to review the options.

Return on Investment of Enterprise Search

When starting a return on investment project, I always ask for the ROI justifications used for other applications, saying they will help me understand how the company allocates overhead costs for enterprise-wide applications. Invariably I discover they don’t exist and that search is a special case.

Let's start with the investment side of the equation. The first question is whether this is just the software costs or should include professional services costs and internal staff resources. Keep in mind that the typical license costs for enterprise search are unavailable up front. Potential vendors are always reluctant to provide a firm quote until they are assured a contract is forthcoming. Moreover, the license costs may only represent 30% of the total investment. Take a look at the cost structure for Elastic Cloud as an example of a vendor that wraps up costs in a ‘processing unit’ formula. In effect you have to develop a Fourth Dimension costing model to account for the full range of potential costs.

How are you going to measure return? Are you looking for a financial return or performance return? (I’ll come to productivity in a moment). Whatever measure you use, the final challenge with an ROI approach is defining what is an acceptable ROI figure.

Related Article: How Much Does Enterprise Search Cost?

Does Search Lead to Productivity Improvements?

Search software vendors often argue that an investment in a search application will result in an increase in productivity. Businesses measure many performance metrics, but productivity — outside of manual work linked to manufacturing or logistics — is rarely measured with the degree of accuracy needed to make a productivity case for the staff involved in research, corporate services, marketing or strategic planning, to name a few.

James Robertson assessed the lack of credibility of productivity-based business cases for an intranet in 2008. His basic argument also applies to enterprise search today. Search vendors issue press statements about the increase in productivity their application will deliver, but never include any quantifiable indicators of before and after. As with ROI, the question businesses cannot answer is what level of increased productivity will make a definitive business case? Is it 1%, 2.5%, 3.2% ... or some other number?

The Myth of Reduced Time to Search

A core element of the productivity approach is that a faster search equals a better search. No research justifies this claim as this analysis demonstrates. The range of search types is so broad that averaging out a session time is completely inappropriate, as is the myth that enterprise search users only use one search term. Very little reliable research has been undertaken on the factors that affect search satisfaction, the notable exception being the work of Paul Cleverley inside a major oil and gas company.

Related Article: Diagnosing Enterprise Search Failures

One Search to Rule Them All?

Then there is the “one search to rule them all” justification. This at least recognizes the multiple search applications in use at any given time in an enterprise and raises the issue of how a master application fits into the picture. A recent study carried out at the NASA Jet Propulsion Laboratory illustrates the complexity of the search application environment. Many of these applications have user-specific interfaces. Take a look at research on how different groups of users rate search functionality. Ask yourself how many times each week do your employees need to search across every one of these applications? Does this make a business case? I have my doubts.

Return on Information

The only metric with any merit is the impact on ensuring that decision making happens on the basis of the best available information and that the discoverable information will ameliorate business risk. Corporate risk managers own a corporate risk register which scores every potential risk and the actions that can be taken to manage the onset and impact of any element of search failure. It does not help that search can fail in so many ways, as ClearBox Consulting has explored in some detail.

If you want to get a sense of how senior managers use information, the French telecommunication company Orange undertook a very good case study that gives a sense of the value of information use at a senior level. Presenting a business case with this level of insight into information needs is probably all you need to get the green light.

Related Article: Why Should You Care About Enterprise Search Research?

What It All Comes Down to ...

In the final analysis, the only approach that makes sense and delivers value is to fully understand not only what information employees are looking for but also why they are looking, how they use the information and what will happen if they cannot find the information. You need both an information management strategy and a search strategy. After 20 years in this business, my experience is that without them any business case you make will give a zero return.