Editorial
A common request clients make during the early stages of an enterprise search project is for a single search box to search across all of their applications. These requests, delivered more in hope than anything else, envision a "silo-breaker search," one that provides a single point of access to all of the information resources in the organization.
In principle, this is an admirable vision, but it is very difficult to deliver in practice. The technology is often referred to as federated search and many vendors claim they can deliver on this requirement. The aim of this column is to provide some insights into what is involved so you can have an informed discussion with your vendor.
Aggregated vs. Federated Search
On the way to federated search, aggregated search is worth a mention. In aggregated search, a search query presents a list of links to content items (usually referred to as information nuggets) that the software assembles into a page for display. Information nuggets can come in multiple formats (text, image, video, etc.) and levels of granularity (document, passage, word, etc.). Each of these formats is defined as a vertical. This is the approach behind the search cards you see in Google and Bing. Selection of the nuggets and the order of presentation takes into account the outcomes of the pages that users clicked after their initial search and is not as easy as it might seem.
The objective of federated search is to query the search applications in multiple collections simultaneously and then present a list of results from each collection which are ranked in order of relevance to the user. Ecommerce comparison sites regularly use of this approach but translating it to enterprise search poses a significant challenge.
Related Article: Unravelling Federated Search
2 Approaches to Federated Search
Query Time Federation
Federated search is available in two distinct architectures. In query time search, the search API issues a query to all search applications, collects the responses and assembles them into a list of results. Issuing the query is fairly easy. The query-to-client interface is usually Open Search. Now the API has to decide how much time it will wait for all of the individual applications to respond to avoid a substantial display latency.
Assuming that (for the purposes of this column) results come back from all of the applications, the next challenge is how to present these results. Although the results may come back from each application in a ranked list, this ranking is based on a specific collection. That makes presenting an integrated list of results in relative rank order a nightmare. At best the results could be presented in multiple windows or interleaved in sequence, but even so the snippets from each search application may well be different.
Index Time Federation
Learning Opportunities
Webinar
May
30
ChatGPT and the Future of Conversational AI
This discussion explores various aspects of AI implementation, ethical considerations, and challenges for your organization.
Webinar
May
31
The Evolution of Employee Listening - 2023 Trends & Best Practices
Learn proven effective methods and solutions to supporting an inspirational future of Employee Voice
Webinar
Jun
1
How organizations can design, build and run Generative AI into the Customer Experience
Discover how to drive enhanced CX through Generative AI
Webinar
Jun
6
The Future of Social Media: Emerging Trends and How to Stay Ahead of the Curve
The impact of emerging technologies like AI on social media
Webinar
Jun
7
The Building Blocks of Personalization
This webinar will explore how personalization can create powerful ecommerce shopping experience.
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
May
30
ChatGPT and the Future of Conversational AI
This discussion explores various aspects of AI implementation, ethical considerations, and challenges for your organization.
Webinar
May
31
The Evolution of Employee Listening - 2023 Trends & Best Practices
Learn proven effective methods and solutions to supporting an inspirational future of Employee Voice
This architecture has a single master index of all of the content in all of the collections. A search is run against the master index, which then produces a ranked list of results against all the documents in the collections. This is usually a substantial number and given the common limit of 10 links a search page, results from some of the applications may not appear anywhere near the top of the ranking list. This can cause problems for the person making the inquiry as when they use the search engine of a specific collection they may end up with some highly relevant results that didn’t make it into the first 100 returned from the master index.
A variation on this architecture compiles the master index from the search indexes of each application rather than from an index of the original content. The index schemas do need to be pretty close for this to be an option. Searching across a mix of classic inverted indexes and knowledge graphs, or on-premises and cloud, is not a good idea.
Related Article: Enterprise Search in 2018: What a Long, Strange Trip It's Been
Start With the User Experience
When considering federated search it is essential to start with the user interface, as both architectures inhibit the quality of the interaction between the search application and the search user. A number of search vendors offer solutions (in my view partial solutions, but make up your own mind) to the challenges of federated search with the inclusion of some clever tweaks and short cuts. Vendors often talk about the number of connectors they have, but federated search isn't just about connector integrity but presenting a user interface that achieves an acceptably high level of search satisfaction.
When you start your discussions with vendors, ask them to demo the user interfaces they have developed for other clients and take a very hard look about how well these might meet the needs of your users across the range of collections you have at present. If they don’t have any, be clear about the development costs. Also ask about what happens when you add a new application either internally or through the acquisition of a business, and what the disaster recovery options are when one of the individual applications goes down. You really want to avoid a complete re-index! Finally, talk about early and late binding security management. This could — and should — be a long meeting.
Related Article: When it Comes to Intelligent Search, Don't Expect Magic
Learn how you can join our contributor community.
