There's some confusion about what I mean when I say managing search is a contact sport.
Basically, enterprise search, while a well-established technology, is not one you can install once and forget about if you still expect happy users. This holds true even when your enterprise search is a Google Search Appliance.
Getting on the Road to Better Search
Start by knowing your users’ intent. Your external SEO folks have dozens of different tools to help with this — sadly those tools don’t easily to convert to enterprise use cases. And often the web solutions are based on the SEO term gaming their own search platform at the same time they try to game Google.
You need to undertake your own Enterprise SEO, or ESEO. I first wrote about ESEO in 2009, and it’s as relevant today as it was then — and suffers the same lack of tools even now. However, there are methodologies you can use.
Start by looking at query terms people use on your sites. If you are lucky, your search platform is one of those rare platforms that has sufficient reporting to provide this data easily. If you are really lucky, you can also get the IP address.
Create a plot showing the queries and the number of times each is used. Typically, you’ll see a small number of queries used frequently, and then a large number of queries used sporadically. If you can get the frequently used query terms to produce great results, you will have improved your search platform for the queries people use most often.
With a little extra time — or an intern — you can create a cluster map of terms. Manually identify related or synonymous query terms, and begin to cluster them together. For example, "lunch" and "menu"; "conference" and "conferences"; "vacation" and "holiday" — terms employees may use to search for the same documents. If two synonymous terms occur 25 times, you really have 50 queries with the same basic intent.
Sort the "intents" list as you did the query terms, and you’ll likely see the same break between frequently and infrequently searched intents. Specifically, when you have multiple intents that seem similar, group (and count) them together.
Go though the list of top intents and look at the results. This will probably require some serious effort, and it may not feel that important, but knowing which documents your top queries return is an important step in improving results.
If results for one of your top queries/intents are not very good, note the term and document. We’ll get to fixing that shortly. If you are not a subject matter expert, reach out to your users for help. Bring them in, show them the results for a top query, and ask them if the top documents are the best. If not, have them identify a better document or two. When I’ve done this exercise with customers, I’ve never seen subject matter experts who were not willing to give their time for the promise of better search.
Remember, you are not trying to fix every query — just the frequently used ones. Start with the top 10 queries/intents, and see how things are going. If you have a little more time, start with the top 25. No matter how many you start with, you’ll be improving search results. Ten terms a week is 520 in a year — my guess is you’ll see less and less improvement after the top 25 to 50 terms, but feel free to keep going as far down the list as you want (or have interns for).
Fixing Your Results
Even the most basic search platforms usually offer the ability to use synonyms and stop words: use these tools to effectively boost or block certain terms. Once you've identified the most relevant documents for your top queries, use these tools (and any others available on your search platform) to boost the "good" documents and penalize the "bad" documents.
If you have a search platform that supports boosting and blocking results, the task becomes even easier.
Lather, Rinse, Repeat
As you work through your list from top queries to less frequent queries be aware that fixing a less common term may break a more frequent term that you already fixed. Relevance is an ongoing process, so after you’ve taken a pass at the top 25 queries, go back and verify that the top 10 queries still work as expected.
I’ve said before that Google is so good at search because it analyzes every query from every user. You may not have the technology to do it all, but the methods I’ve described here can get you stared on the road to search success.