Organizations are investing heavily in enterprise search and information accessibility, but a surprising number have no understanding of their users’ search behavior. We recently sat down with Otis Gospodnetić, founder of Sematext (news, site) and co-author of the book Lucene in Action. Otis shared some key lessons learned via the emerging discipline of Search Analtyics.
The Importance of Search Analytics
Search analytics captures metrics about user search behavior and search performance. Why are search analytics important? According to Gospodnetić:
Search analytics are super important because it should be driving a number of decisions around search [such as] relevance, tuning for search and design decisions around how search is exposed on the site. [Search can also indicate] documents that are important to audience and topics that people are after, but people don’t use it.”
Now that data sizes have grown to a point where categorization and directories aren’t practical solutions in most organizations, most of us depend on search functioning well. But without search analytics, there is no way to determine if search is working according to expectations, other than qualitative feedback, which is subject to perception.
This is the situation within many organizations. Although Sematext specializes in open source tools, such as Lucene/Solr, Otis' search recommendations are tool-agnostic. If companies are going to invest in search with any tool, they should also define requirements, thresholds and collect metrics to determine if search is performing optimally -- just as they would with any other software or process implementation.
Must-Have Tools for Enterprise Search
The majority of organizations should monitor two components: Search performance and user behavior.
Organizations should consider implementing search-specific performance monitoring. Most operational monitoring solutions capture general data about server performance, but few provide adequate details to evaluate search. Key Performance Indicators (KPIs) include cache utilization and the time to warm up a new index. There are also software-specific metrics that may be applicable.
Search analytics provide insight because they capture what users typed vs. traditional "click stream analysis,” in web server logs. Click stream analysis requires an assumption about why users clicked a link.
- percent of queries failing (no results returned)
- results with no clicks
- most popular queries (top topics)
- most popular words
As organizations become more sophisticated, they can consider adding:
- which results were clicked the most
- who is performing the most searches (users and roles)
- amount of search traffic over time
- correlated searches (users who searched one item also searched for another item)
In addition to collecting metrics, organizations can implement simple tools to improve the search experience. For example, auto-completion of queries is a functionality most organizations should implement, and is common in public search engines. However, few organizations actually have this in place internally.
A New SaaS Option in the Oven
Organizations that do want to enhance their search analytics and practices, but struggle with the budget or resources to do so will soon have an additional option. Gospodnetić indicated that in the near future Sematext is launching a near real-time search analytics Software-as-a-Service offering that will analyze search data and provide reports and recommendations. Stay tuned to this space.