looking behind the blinds
PHOTO: Chris Nguyen

Last November, Elastic surprised many by acquiring hosted search company Swiftype. Elastic is the company behind the open source Elasticsearch search engine. At first it struck me as odd that the company excelling at what I call the best search for the internet of things would buy a company that hosts primarily public-facing web content.

But then I started looking at the acquisition through the lens of intellectual property, and suddenly everything came into focus. To understand why this acquisition makes sense, we'll dive first into a little background on the two vendors.

Clearing Up Elastic Confusion

Since its founding in 2012, there's been some confusion about Elasticsearch differentiates from Solr, another open source search platform.

Like Solr, Elastic's popular Elasticsearch search engine is open source, free for anyone to download and use. Both sets of code use the very friendly Apache 2.0 software license, and both are based on Apache Lucene, the low-level API under Solr. Lucene and Solr are products from the Apache Software Foundation site, while Elasticsearch is only available from Elastic.

Unlike Lucene and Solr, my understanding is that only employees of Elastic can be committers. Thus, Elastic controls what goes into each release of the product. Solr has a large and diverse group of committers from a number of companies — often competitors — and anyone willing to put in the time and effort can become a committer and submit code for consideration. And the capabilities that are included are determined by developers from a number of companies.

Elastic Strengths and Weaknesses

When I learn of a new search platform, I like to dig in to understand its strengths and potential vulnerabilities to establish its niche. It becomes clear then what sort of projects the product is suited for and which might not be as good a fit. 

My evaluation of Elasticsearch was that it's the right search technology for search for internet of things (IoT) applications and perhaps not a turnkey product for traditional enterprise search. 

Elastic's architecture lets it shines with dynamic data and dynamic shards. Kibana, another tool from Elastic, is fantastic for easily creating attractive graphics and charts that help you understand what IoT data is telling you, the kind of reporting upper level managers like to see. There is even an open source version of Kibana, called Banana, available on GitHub.

Many of my colleagues use Elasticsearch to implement enterprise or site search for their customers. It certainly can shine as a search index/retrieval platform, but requires some assembly for traditional enterprise search to function. You need to add a crawler or spider to find content to index web content and file systems; a tool to convert binary files like Microsoft Office documents and PDF formats; and a way to tag documents, files and repositories for LDAP or Active Directory security; and finally, a graphical management console to manage it all.

It's possible to build this out with open source solutions, but most enterprises want support — "do it yourself" is not a popular approach in large organizations. Elastic now offers these capabilities and more in its X-Pack commercial product, but still requires assembling components.

Swiftype: Part of a Hosted Search Service Renaissance

In the last year or so, I stated seeing a resurgence in 'hosted search' services, a concept close to my heart. In the early days of the dot-com craze, my co-founder Mark Bennett and I started New Idea Engineering to provide vendor-neutral search consulting. By the late '90s, we recognized the need for a hosted search service, and we started Searchbutton. Even then, we had a competitor: Atomz. The dot-com crash hit right when we were starting to see commercial sales and our investors shut us down. Timing is everything!

A renaissance in hosted search services is now taking place. Algolia, Coveo, Google, Lucidworks, SearchBlox, Swiftype and others are providing great "search as a service" capabilities, generally with full enterprise search capabilities and a great price. It's exciting to see the re-birth of a great concept. Swiftype competes favorably with the other available services. 

With this background in mind, let's see how these parts fit together.

Elastic-Swiftype: A Smart Acquisition

Swiftype is a service, and as such, probably isn't appropriate for internal enterprise search where security is critical. By acquiring Swiftype, Elastic acquires the technology to market a hosted enterprise service. But more importantly, it acquires the potential tools — in the form of a pretty nice web crawler— to enter the world of enterprise search where ingestion and document level security are critical.

(Side note: Elastic appears to support SAML-based Single Sign On (SSO) with tools from Bitium. Bitium also appears to offer LDAP and Active Directory solutions. So perhaps we might expect another Elastic acquisition in the future? Note, this isn't based on inside information, but rather is a logical guess.)

Will Elastic tweak the Swiftype crawler to be an enterprise tool? If so, it would enable Elastic to compete with the other leaders in 'traditional' enterprise search. The two companies announced plans to continue the Swiftype service, but haven't offered many further details beyond that.

Elastic may just morph into an enterprise search company after all. Stay tuned!