In the 40 years since I first started using search applications (initially on online remote access services such as Dialog, SDC and BRS) we have seen steady progress in search performance. But this progress is nowhere near the scale of development every other area of computer technology has witnessed.
I’m in the middle of preparing a presentation about the future of search for the Findwise Findability Day in Gothenburg, Sweden, on October 1. I’m looking forward to throwing out some challenges to both commercial and open-source search software vendors.
At the same time I have been working on an intranet project for a global organization, along with a number of other consultants. The organization invested in a cloud-based project collaboration application to aid with the exchange of documents, comments and ideas. However the pace of the project means that over the course of a working day, a document or comment can be many pages below items posted at the end of the day. So search becomes a very important application.
19th Century Search Lives!
I’m not going to name the vendor — tempting though it is — because the very poor quality of the search application is not uncommon with cloud-based collaboration and social network applications. I can say that the vendor concerned prides itself on innovation and a high-quality customer experience.
I needed to locate information on a discussion around page templates, and so just entered the query [template] and waited. It took 30 seconds for the query to be processed, and in subsequent checks the typical query time was 20 to 30 seconds.
So much for a leading-edge application.
The first surprise in the results was the lack of hit count — I just received a list of results. The “template” term was clearly highlighted in the summary text, but not when it appeared in the title, which I found intriguing.
The only filter options were by content format or by people. However, when I filtered by people I was told there were no results, even though every result had the name of the author. Not a helpful response. All messages of this type should suggest solutions and options, such as checking spelling, searching in another repository, etc.
Another challenge with social applications is that it is quite common for members of the team to add comments to a posting, but on the results page each comment is displayed as a specific document.
The project I am working on has almost as many external consultants as internal team members. The external consultants may only dip into the site on an occasional basis. In addition, the external team might post documents to a different folder than the internal team. Already there are postings from members of the internal team asking where a particular piece of information has been filed, a good indication that search does not work.
This illustration is from a project collaboration application, but I have seen a similar lack of search sophistication in hosted enterprise social networking applications.
21st Century Search?
Organizations work in teams. I have commented before about the way in which enterprise search remains a one-person/one-screen application. Research published by Professor Elaine Toms and her colleagues at the University of Sheffield suggests that in teamwork, a single member of the team carries out the actual search process, but often using a query approach (and possibly repositories) suggested by the team. Once a set of results has been derived, the team may then discuss the results and select items to share with the group. Collaborative searching may therefore require some form of workflow management application that integrates multiple individual search episodes and manages the overall information flow.
My concern is that the vendors of collaboration platforms don’t understand the value of search and that search application vendors do not understand the way in which teams work together to solve problems and capitalize on opportunities.
There are significant business opportunities here. Hopefully one or other of these two communities will wake up to the scale of the opportunity and develop some next-generation team-search applications. The amount of research work in this area is now very significant, and I was surprised to find recently that there is an emerging discipline of coordinated exploration.
Surprisingly, the authors of the paper make no reference to the work of Chirag Shah at Rutgers University, who is one of the pioneers in seeking to understand user requirements for collaborative information seeking (CIS).
I expected someone to develop CIS applications in 2015. It looks as though I will be disappointed. Will someone make it happen in 2016, please?