Look, I’ll be the first to admit that the so-called “Hadoop Wars” can be kind of interesting. Who isn’t going to click on a link that says something like “The Hadoop Wars: Cloudera and Hortonworks' Death Match.”
Or read beyond this first sentence:
“Another day, another set of choice words hurled at one Hadoop vendor by another. This time, it’s Hortonworks doing the hurling, claiming that Cloudera’s business model isn’t designed for today’s big data market.”
But sometimes, when we get so busy consuming this kind of content, we fail to remember the other story -- how Hadoop came to be and why, as a technology, it’s been able to evolve so quickly and become one of the major catalysts ushering in computing’s third age.
There’s history being made that goes well beyond vendors fighting for position (which companies in a new industry that want to go public clearly have to do).
A Respite From the Feuds
Yesterday the attendees at the Hadoop Summit had the opportunity to focus on that when Doug Cutting, who is often called “the creator of Apache Hadoop” and earns his paycheck from Cloudera, appeared on stage with Arun Murthy, who led the development of Hadoop 2 and earns his paycheck from Hortonworks (he is also one of its founders).
What was significant about the conversation is that, despite being employed by fierce competitors, Cutting and Murthy showed genuine appreciation for each other and respect for each other’s contributions to the Apache Hadoop project.
Murthy practically thanked Cutting for teaching him about Apache Open Source and what it took to commit code to an Apache project (while they both worked at Yahoo). Cutting spoke about how Murthy helped get Hadoop ready from prime time -- where it is today.
They spoke about Hadoop’s evolution: how Cutting began its development to solve a search problem while he worked at Yahoo. Murthy explained how the idea for YARN came to him early one morning (3 AM?) out of a desire to do things better than MapReduce -- a main component of Hadoop -- could at the time.
Bound By Community
Both Cutting and Murthy talked about Apache Hadoop as a “we” project and about the community that contributes code to it. Some of these committers, and Apache Hadoop Project Committee members, work at Hortonworks, others at Cloudera, and still others at competitors (or companies that sell/resell related products and services) like Altiscale, IBM, Intel, MapR, Microsoft, Pivotal, VMware and WanDisco.
Yet there are companies who also employ Hadoop committers whose revenue generating activities have no direct connection to Hadoop. They include eBay, Facebook, InMobi, LinkedIn, Twitter and UC Berkeley, among others.
And it’s also fairly widely acknowledged that anyone who has ever reported a bug in Hadoop or submitted a fix has also made a contribution.
As Murthy said, looking out at the audience, toward the end of the discussion, “If you’re in this room and you used Hadoop four to five years ago, I can’t thank you enough.”
Whatever happens between competing vendors in the world of Hadoop in the future, for a few minutes on June 4, 2014, Hadoop was in a sacred place and many of us got to watch.