Hadoop distro providers like Cloudera and MapR don’t have to worry about going toe-to-toe with Hortonworks anymore.
That's because Hortonworks is no longer a one trick pony.
(Ok, maybe we should have said one trick elephant.)
Either way, the provider of the only 100 percent Apache Hadoop distro, HDP, now has a new Apache-based big data offering in its portfolio. This morning the Santa Clara, Calif. based company announced it has acquired Onyara, the creator of and key contributor to Apache NiFi.
Keep It Moving
Instead of focusing on data-at-rest, like Hadoop does, Apache NiFi is all about data-in-motion.
While some may interpret that to mean information generated by the Internet of Things (IoT), Hortonworks takes it a step further.
“We call it the Internet of Anything (IoA),” said Shaun Connolly the company’s VP of Corporate Strategy.
He pointed out that IoT generally refers to data generated by devices/sensors while IoA also includes the Internet. “Stuff like social streams, server logs, clickstreams,” he added.
Hortonworks’ Apache NiFi-based product will be called DataFlow. One of the biggest advantages it provides to customers is its ability to help predict changes discovered in streaming data and to react as they occur, the company maintains.
Good Enough for the NSA
Not only that, but NiFi is also tried and tested. It was in development for eight years at the US National Security Agency (NSA) then spun out of the NSA Technology Transfer Program and donated to the Apache Software Foundation.
“This is hardened/baked tech--NSA–grade stuff,” said Connolly.
The significance here is that the enterprise-grade features which often take lots of money, manpower and time to build are already built-in.
They have also been used in highly sensitive settings to do things like inform Traveler Threat Detection, Connected Car Tracking, Smart Meter reading and the like.
Though there are technologies, some custom built by individual organizations, that attempt to do what DataFlow does, there are a few distinguishable differences worth noting. “DataFlow is bi-directional, quite robust, feature rich and approachable,” said Connolly.
It could become instrumental in the future of smart clothes, smart buildings, energy and transportation.
What’s Hadoop Got to Do with It?
At this point DataFlow and HDP haven’t had much time to play together, but they’re likely to make for a smart combination— one looks at data at rest, the other at data in motion. But you don’t have to buy one to use the other.
“These are two separate products. They’ll be sold separately.” noted Connolly.
Will DataFlow distract Hortonworks’ Apache Hadoop committers?
Nope, said Connolly. They’re laser focused on the work that they love and Onyara brings to Hortonworks its team of Apache NiFi committers.
Title image by OakleyOriginals.