time lapse photography of a highway at night approaching the Golden Gate Bridge
PHOTO: Wilson Ye

Enterprise search has been going through a dramatic shift as of late. We've watched as some of the leaders in search, those platforms usually found in the upper right quadrant on Gartner reports, have fallen off through acquisition or from simply not keeping up with the market.

But behind the scenes an even bigger shift is taking place: from proprietary kernels to core technologies based on open source projects. 

Some, like Lucidworks, have always been based on the open source Apache Solr project. Others, like Coveo, have joined the open source movement by offering the choice of using its traditional proprietary kernel or licensing the Coveo user experience built on top of the Elastic kernel. 

I suspect the shift taking place in the enterprise search market will likely grow in scope. More and more commercial products will begin to replace their complex and expensive proprietary kernels — the code under the covers that actually does the real work — with open source technology.



Related Article: Enterprise Search in 2018: What a Long, Strange Trip It's Been

Why These Enterprise Search Trends Will Only Accelerate

The product interface offers the true value for users

What goes on under the enterprise search covers is not important to any product users. If a company can focus its limited resources on the product rather than on the plumbing, their engineers can focus on improving the part of the product that users actually buy: the user experience.

Open source projects — especially from reputable organizations like the Apache Project — are finally being viewed as reliable, robust and well tested

And, since the code itself is freely available for download, the chances that a bad actor can insert malicious code are relatively low.

It hasn't always been this way. Several years ago, a friend of mine who worked at a large, well-known consulting firm attended an Apache conference. He and two older colleagues from his firm were in the back of the conference hall during one of the keynote talks, probably the only people in the building wearing coats and ties. At one point, one of the senior managers leaned towards my friend and said, "I like everything I hear about open source — except the people in this room."

Needless to say, many open source projects are still populated with engineers who don’t dress like traditional corporate programmers and developers. But the part of the product that employs open source code is hidden under the wrapper of the commercial product supported by sales and technology staff perhaps more acceptable to senior management. No one ever sees it.

The true value of software is functionality, capability, usability and reliability. Of course, documentation and training are important as well. So smart software vendors have started to reduce their development costs, and improve their products and their margins by utilizing open source technologies under the covers. In fact, some software companies have entire teams of developers whose jobs are to create, maintain and enhance the open source projects upon which the commercial product is built. The software companies benefit by incorporating features that other developers donate to the project, leaving their developers to focus on capabilities that enhance their proprietary commercial product. As a result, software companies can be profitable sooner than they might otherwise have been, and the product can be sold at a more aggressive price because many of the people writing and maintaining the open source code are not on their payroll.

So go ahead, it's fine to embrace open source software, and perhaps even consider using it for your internal development projects where it makes sense. Why should only software companies enjoy lower development costs, faster time to market, and well-tested and supported tools?

Related Article: Enterprise Search Has an Open Source Secret