a closed industrial container with "search" written on the sides
PHOTO: Martin White

The clouds are darkening around the future of Autonomy and its IDOL search application. Its current owners, Micro Focus, recently decided to part company with executive chairman Kevin Loosemore, who joined the company in 2005 to guide it through an IPO. Under Loosemore's direction, the company prospered. Then it bought Autonomy from HP in 2017. In the post-acquisition euphoria, the Micro Focus share price was £3448 ($4468). As I write this, it is down to £772 ($1000). 

Can Autonomy be sold off again? I doubt it.

The search industry as a whole can learn some important lessons from the Autonomy story. 

The IDOL Black Box

First, a little history. Autonomy grew out of Cambridge Neurodynamics, a company set up in 1991 by Mike Lynch to commercialize the work on signal processing he undertook for his PhD. To cut a very complicated story short, the basic principle of the intelligent data operating layer (aka IDOL) technology was the detection of patterns in the text of documents without any need to use any natural language processing techniques. Indeed, IDOL was language independent as (for example) it would recognize patterns in Chinese without understanding the language. 

The second element was the use of Bayesian statistics, which enables probabilities to be calculated that if one document was relevant then a second document with similar patterns was also relevant. (I know that is a travesty of the work of Reverend Bayes but for this column, it will suffice).  

This approach was undoubtedly innovative but the implementation was a significant challenge. Autonomy licenses were already expensive without the added significant costs of the professional services fees to get it up and running. 

Once running, the novel ranking approach worked well at times, but just as often the results were all over the place. Even experienced support teams had a difficult time working out and fixing what was happening without causing problems with other queries. 

Related Article: HP Sues to Recover $5.1B From Autonomy Deal

5 Lessons From Autonomy for Vendors

1. Technology differentiation is very difficult

Vendors often focus on presenting their software as a set of modules, replete with vendor-specific acronyms and associated jargon. While this approach may impress procurement managers and some IT managers, it does nothing to convey why the total end-to-end solution will meet specific customer expectations.   

The only people who fully understand the technology of enterprise search are either working for a software vendor or are external consultants. Yet every vendor tries to sell its technology (always incorporating AI!) as being the perfect fit for any enterprise search problem. The impact of content quality and the need for skilled search managers are never considered.  

2. Vendor intangibles are important

Real Story Group's Tony Byrne has posted a very good list of generic vendor intangibles. These are rarely mentioned in the web PR stack and yet are often the reason for selecting one vendor over another. Autonomy won many customers by being a) British and b) in the top 100 companies (FT100) on the London Stock Exchange. “Let’s go for Autonomy — what can possibly go wrong?” Prospective customers are increasingly paying much more attention to these intangibles. 

3. Who owns the customer?

Post-installation there is often confusion about who owns the customer. Is it the software vendor or the search implementer? In the case of Autonomy, it was always a huge challenge to find someone in the company who could provide support. Customers had to first talk to their integrator and then the integrator would manage the dialogue with Autonomy. This was a particular problem with the analysis of search logs as no one could make sense of them. 

Related Article: Who Needs Cognitive Search When We Lack the Resources to Make it Work?

4. AI black box

Adding in AI to relatively standard ranking models turns an application into a black box. As we saw with IDOL, when results appear which seem to bear no relationship with the query, the user quickly loses trust. Paradoxically the problem is made worse when some results are highly relevant and some totally irrelevant. The person conducting the search will not want to revise their query in case the relevant results vanish. People worry that an AI algorithm over which they have no control is potentially precluding them from content.

From a recent Accenture post: “Many consumers feel the automated techniques brands are using to serve them are not effective. The algorithmic recommendations brands make to them, based on their demographics and past behaviors and purchases, don’t accurately reflect their intention.”

For consumers, read "employees" and for "brands," the company.

5. Preventing customer – customer relationships

Autonomy’s greatest secret was its customer base. The company was always very unwilling to put customers who might have common objectives or problems in touch with each other. The concept that a satisfied customer is your best salesperson was lost on Autonomy. Even now user groups are very unusual, and I am at a loss to understand why this is the case. 

Related Article: The Essential Glue for Your Digital Workplace

Break the Cycle

If vendors do not start to educate the market and sell in a way that can help customers achieve defined information discovery options, many of the roughly 50 search software vendors will disappear. They will be replaced by newcomers with even more complex technology who will in turn make exactly the same mistakes. And so on, ad infinitum!