David Fishman, Vice President of Marketing with Lucid Imagination, said that search customers were running into certain problems with conventional commercial search software. Specifically, that it's becoming a constraint on their flexibility. He defines two axes of the cost of growth when it comes to enterprise search:
- Realizing that you have new requirements after implementing the search solution, such as new content formats that you want to index, and discovering that the new formats require paying for a new module
- Your content continues to grow until you're indexing so much data that you have to pay more due to higher search volumes
In the case of those using search appliances in particular, you can also run into limits on your ability to tune performance.
In response to these issues and the fact that Lucid Imagination has found that today's enterprise site developers want fine-grained control over many low level issues such as relevancy, the company created LucidWorks Enterprise. This product is based on Apache Lucene (news, site) and Apache Solr (news, site), as you might expect from a company built on providing professional services around those projects.
LucidWorks Enterprise is aimed at people who want to build search as an application. Fishman said that as part of that focus they had to lower the bar on how complex it is to build a search application, plus create a pricing model that was more flexible.
One method of making applications easier to build was to take Lucene and Solr configuration from XML files, which can cause catastrophic failures through typos and other little mistakes. Instead, they've built a set of REST APIs to expose the search functions for configuration without touching the files. Unlike editing raw XML, the APIs will send back an error if you, say, try to set a parameter that must be a digit with the letter O instead of a zero.
Click Scoring Framework
Another feature they've added is a click scoring framework, a technique search giants like Google (news, site) use to make the engine smarter by having it watch what results people click on after a search, and then to value those results higher the next time someone does the same search.
While it's of course possible to build your own click scoring framework with Solr and lots of developer expertise, Lucid Imagination is banking on the fact that many would prefer to have a pre-built option.
They've also simplified security features and set what they consider smart defaults so that, for example, if you're importing HTML documents, the server is already configured to handle them properly. The work in curating your own build has also been reduced, since they've included new client libraries such as those for .NET . LucidWorks Enterprise ultimately contains 75 open source projects, a complete version of Solr, and proprietary code from Lucid Imagination.
As to cost, they're offering a two-server deployment subscription for US$ 68,000 per year with consulting and incident management included. By the company's estimate, a 4 CPU server with 16GB of RAM can handle 10 million documents, which Fishman says can cost 6 or 7 figures to index with most commercial search vendors, and that cost wouldn't include incident management or support.
Will your company consider the LucidWorks Enterprise alternative rather than setting up an open source search solution on your own? We're interested in your take.