Anyone who has ever tried to search through masses of unstructured data looking for pertinent information knows how frustrating it can be. 

Unless you structure your query just right, you’ll get a haystack to look through and by the time the needle is discovered, the prime of opportunity for action has passed. In the age of big data, algorithms and analytics, can’t we do better than that?

Yes, you might think. There are tools like machine learning and artificial intelligence to point to and they’re great if you have the talent, tools and time needed to train models (and do a better, faster job at it than the competition).

But there might be another way that can yield the same, if not better, results in near real time.

That’s precisely what Elastic, the commercial entity behind the world’s most popular search engine, Elasticsearch, just introduced to CMSWire. 

It’s called Graph and it’s supposed to make it easy for anyone to uncover, understand, and explore the relationships that live in their data. It works by combining the speed and relevance-ranking of search (Elasticsearch) with graph exploration and provides visualization through (Kibana) which is part of the Elastic Stack.

Elastic graph

“Graph adds a relevance lens to search results,” Steve Kearns, senior director of product management, Elastic, told CMSWire. 

It works by surfacing the most meaningful connections in your data and makes it easy to answer questions that previously would have involved multiple systems, batch jobs or machine learning,” he said. Though it hasn’t been available until yesterday, Kearns expects that it will be used to answer complex questions and address use-cases such as behavioral analysis, fraud, cybersecurity, drug discovery, personalized medicine, and to build personalized recommendations based on continuous real-time data.

Look for the Interesting

One simple way to understand how it’s different is through an example. Take the case of a reported credit card breach. 

The first thing many providers might do, after triaging the first individual occurrence, is to see who else might have been affected. The logical question to explore might be “where did the breach occur?” The tracking begins with purchases applied to the said credit card. 

If a latte was purchased at Starbucks that day, it might one option to explore, but detecting a pattern where such a high volume of transactions occur could be challenging. It might not be the first, best place to look. 

Another choice would be to look at the local gas station where the victim last filled-up. What time did that purchase occur? Who else fueled-up at around the same time? If seventy percent of the gas station’s customers had their information stolen between 3 and 5 that day, the answer seems pretty simple.

“It’s often the uncommonly common data that holds the key,” said Kearns.

It’s worth noting here that the questions that would be asked and answered in this situation would be discovered visually via data and patterns.

Is Graph a Big Deal?

“We see graph data as an under-appreciated opportunity, because it’s so often relegated solely to use cases around recommendations and social networks. Graph makes a great deal of sense to add to the Elastic stack, with a graph-based Elasticsearch API and Kibana visualization applicable to common use cases like IT infrastructure, social networking and its implications, and recommendation/search,” 451 Research analysts Donnie Bekholz and David Immerman wrote in a report earlier this month. 

Graph had been briefly discussed at ElasticON, the company’s user conference earlier this year.

Title image "Stretching" (CC BY-SA 2.0) by @lattefarsan