Horseback riding
PHOTO: Thomas Peham

We are all of us at one level or another beset with information management problems. These problems might be considered information governance issues or technology problems. But for every one of us who is inventing good practices on the leading edge, 20 more are struggling to get a good grip on our information. 

Some organizations still worry about their employees understanding what information constitutes a record, some have moved onto more flexible governance structures allowing them to focus on business problems. The split between on premises infrastructure and the cloud keeps some up at night, while others worry about the proliferation of applications and systems. Still others are frustrated by the constant complaints from colleagues that they cannot find anything on the intranet and others are needled by their legal department with e-discovery questions.

In short, our organizational maturity is all over the map. Therefore, no “one size fits all” solution suits all of our information management and information governance issues.

ML and Search Technology to Information Management's Rescue?

For years I have said that implementing technology alone will never offer the answer to any business problem. I am not suggesting otherwise now: people, process and technology remains a key mantra. However, reading recent articles by CMSWire contributors and search experts Martin White and Miles Kehoe, combined with recent conversations I've had with vendors, has me wondering if a technology solution can in fact help us with our information search and discovery woes. 

The magical ingredient here is machine learning (ML) and how vendors are applying it to enterprise search.

If you work in a largely homogenous environment, for example, if you're all in on the Microsoft Office 365 environment, things might not be too bad. Office Graph, hybrid scenarios for cloud and on-prem SharePoint search, Microsoft AI technology applications (no, not Cortana, the useful back end stuff), the ability to build connectors to surface content in Teams, even the discovery and information governance features: everything is on a constant product improvement cycle. If you have an information access or discovery issue, either Microsoft is already working on it, or a third party within its ecosystem of partners is. With SharePoint's underlying FAST search technology, the Office Graph with Delve, as well as other applications as the front end, we have a pretty good technological base for finding the information we need to do work.

But what happens if you’re in a highly heterogeneous, hodge-podge environment? Exchange on-premises, some SharePoint (because everyone has it), Jive, Salesforce, SAP, a little pocket of iManage here, and a slightly larger one of OpenText Content Server over there .... This might sound more familiar to many readers, as our organizations have historically taken “organic” growth paths. Mergers and acquisitions can play hell with even the best enterprise architecture road maps.

Related Article: The Clue to Your Digital Workplace Problems Lies in Your Employees' Work-Arounds

Enterprise Search's New Chapter

Thirteen years ago Martin White explained to me all of the issues the federated search tool I was trying to implement would bring: the mixture of results from different indexes and search tools, with no way in the user interface to differentiate or compensate for the different algorithms, each algorithm dealing with recall and precision and security permissions in different ways. 

Thankfully, a lot has happened in the ensuing years. Advances in new technology, the scalability of cloud infrastructure, more user friendly interfaces, and the application of machine learning to process separate results from a highly federated search have taken care of many — if not all — of those previous issues.

BAInsight, with its SmartHub extensible Enterprise Search user interface and its Unified Information Access federated search tool, is taking an approach which allows it to leverage Microsoft’s Cognitive Services, or Google’s Cloud Machine Learning. It can extend Microsoft SharePoint and FAST, or it can use Elasticsearch (which is an implementation of Apache Lucene).

Coveo is another vendor applying artificial intelligence to solving enterprise search problems. Its machine learning technology can also be utilized via connectors to Salesforce or SiteCore, and it also works with Elasticsearch.

The idea is that the advanced processing machine learning provides creates an enterprise search facility that can break down silos in even an extremely heterogeneous environment. Machine learning makes it possible to provide relevant search results with the required precision while still offering a great user experience. What's more, it should be able to do this with advanced analytics so search managers can prove to information owners why certain results are being presented, giving credence to information governance efforts, and facilitating any adjustments that are needed.

If you have a great enterprise/federated search setup in your organization, please let us know how it works for you in the comments section.

Related Article: What's Behind the Elastic-Swiftype Acquisition