Looking back, 2012 has been quite a year for search.
From a business perspective Lexmark acquired Isys-Search, Lucid Imagination changed its name, Attivio gained a US$ 37 million investment, Coveo followed with an US$ 18 million investment, Apache Lucene and Solr moved to Release 4, ElasticSearch set up a commercial arm and Microsoft announced a seriously well-featured SharePoint 2013 search application.
Gradually a picture of search implementation is beginning to emerge, thanks to surveys from Findwise, MarkLogic, Oracle and AIIM. The picture is not a pretty one. All agree that information is a business-critical asset but companies have failed to understand the urgent need to provide technology and support for search.
Publishers saw business opportunities as well, so along with my own book came excellent titles from Susan Feldman, and Tony Russell-Rose and Tyler Tate. Stephen Arnold continued to track and probe the search business to the benefit of the entire search community. And then came the HP-Autonomy saga, which is clearly not going to play out any time soon.
More to Come in 2013
So far I have not mentioned "big data." The Exalead tweet machine (@3dsexalead) highlights a dozen news items a day and every search vendor has invested heavily in a big data hype machine.
The upside of the interest in big data is that it is causing IT managers in all types of organizations to look, probably for the first time, into the requirements for data and information discovery. Data without an information context has very limited value.
The Findwise survey (backed up by the Digital Workplace Trends Report) indicates that less than 20 percent of organizations have a strategy for search even though many of them will be supporting multiple search applications. I expect this figure to improve markedly by the time the 2013 Findwise survey is presented at Enterprise Search Europe (of which I am Conference Chair) and the Enterprise Search Summit in May.
The 2011/2012 search vendor acquisition frenzy took out most of the mid-range vendors. In 2013 we will find out whether smaller commercial vendors can attract the investment they need to bring their technologies to a wider market or whether the space will be taken by open-source applications.
The evidence may be along the lines of LucidWorks, Attivio, IntraFind and Polyspot, in which Lucene/Solr is blended with a range of open-source and proprietary modules. That is not to say that internal or external developers cannot build excellent open-source applications but many IT managers feel reassured by a post-implementation contract with a company that has investors to keep happy.
Search technology dates back to the early sixties, and it may come as a surprise to many to learn that the RAND Corporation was thinking about Bayesian statistical approaches to information retrieval in the late 1950s!
Over the last couple of years content analytics, search-based applications, semantic search, social search, clustering, multi-lingual search, multimedia search, unified information access and entity extraction have all moved on significantly. Another important trend is the way in which the information retrieval and enterprise search communities are beginning to see the benefits of working together.
Search is About People and Technology
The barriers to the wider and more successful implementation of search applications are all people-related. The Findwise survey indicated that the majority of organizations have, at best, only one person supporting all the search applications (including the corporate website) and that is ridiculous. Search touches every person in the organization. Try telling the IT manager responsible for the ERP application that they will have to manage with just one support person in 2013!
The problem is finding people with the skills and experience needed in development, pre-sales, sales and support positions. These skills are in addition to those of data scientists. The problem is that universities cannot suddenly modify degree courses and in any case the need is immediate, not just in five years’ time.
The recruitment industry also needs to understand these requirements. Some recent personal experience suggests that recruitment consultants have little idea about the specialist skill sets required, and there are still far too few training courses that are not run by organizations with an interest in pushing their own particular solutions.
Action Next Month
You are not going to be able to cope with the opportunities and challenges of 2013 without taking the following actions next month
- Appoint a full-time Search Manager and make sure that search logs are reviewed in detail on at least a monthly basis.
- Assess search satisfaction and search requirements across the organization.
- Ensure that a senior non-IT manager is responsible for a corporate search strategy that links technology and staff investment to maintaining competitive advantage.
- Find out what skills the organization already possesses in information and data discovery and analysis.
- Start to build networks with other organizations using your particular search technology stack. If your vendor is reluctant to help, ask them why!
- Plan to be at either Enterprise Search Europe or the Enterprise Search Summit in May.
Editor's Note: To read more of Martin's writings on search, try Is There a Future for Enterprise Search?