During a presentation I gave to the Search track of the IntraTeam event in Copenhagen last month I caused some consternation by suggesting that Best Bets for a search application were a well meaning, but totally misguided approach to improving search performance.
Why do companies invest time and effort for less than ideal results?
Cracks in the Methodology
The initial list of Best Bet query candidates is often taken from the top 100 (or more) search queries. The Search Manager has to then find people who will know what the definitive document is to match the term -- sure to guarantee 100 percent precision -- and then not only add the document to the top of the results page, but ensure that the Best Bet remains the Best Bet in the future.
The first question to ask is whether the search logs are reliable. Often the terms are listed as single words --"bearing" -- even though the query may have been "track runner bearings." Assuming that the search logs are being used effectively, the second question is why the information is not more readily available on the intranet or similar applications.
If information on bearings is organized hierarchically, users will be led through a route which progressively refines their requirements and -- when they reach a possible document -- take them to a landing page with context and related links. A Best Bet will not have this context and will rarely include URL that enables a user to work back through related content.
One justification I often hear for the introduction of Best Bets is that users cannot find the information they need on the intranet because the information architecture (IA) is fast becoming a total mess. Search and browse are -- with monitor -- three different and related ways of finding information and it is important to find a balance between all three. Search will never offset poor IA.
The current focus on filters and facets as a way of helping users refine their searches means that little attention is being paid to providing enhanced query management along the lines of "Did you mean X?" This information will be generated by the index file and so can be displayed to the user in near real time as they start to add keywords. The only downside to this approach is that the index file will almost certainly not be controlled for permissions, which might result in query terms being displayed which could be embarrassing.
'Users Click on Best Bets'
A common defense of Best Bets is that users click on them. That is not surprising, considering that was the idea of adding them to the results page. The more important question is "what do users do after clicking on a Best Bet?"
In an ideal world, they wouldn't need to click on any other result in the session. But a check of the search logs will show that is not the case. In some circumstances the Best Bet will provide useful information but users will be concerned that the result isn't the most current or comprehensive document so will look at other results to make sure.
The biggest challenges facing Best Bets are deciding which search queries deserve a Best Bet, identifying the document and an owner, and then making sure it remains current. These require the time and effort of your under-resourced search team. There are much better uses of the time that would otherwise be spent (and, I would argue, wasted) managing Best Bets.
Things get much more interesting in intranets with multiple language content. If a German engineer searched for "Rollenlager," should the result be presented with a German language Best Bet or the "agreed" English language Best Bet? I cannot answer this question. But again, considerable time and effort will be spent in discussions of the optimal solution.
Second Order Best Bets
There could well be a query situation where a document exists that may be relevant to a search but not directly related to the query. As an example, take a chemical company where any search on a range of defined hazardous chemicals could also bring up relevant health and safety information. One country market might list a document that highlights the company’s policies on ethical business conduct as a Best Bet. I call this a "second order" Best Bet. They still take time to develop and maintain, but at least do not try to turn the search application into a 100 percent precision performance application. Also, it's possible that a single Best Bet (on hazardous chemicals) could match a wide range of queries.
Talk to Your Users
The final decision must be based on solid user research, which most organizations lack due to under investment in the search team. Without a good understanding of what, why and how users query, any Best Bets initiative will be seen as a very visible sign that the organization fails to understand the value of search.
Editor's Note: Read more from Martin on enterprise search in Search as a Decision Support System