You don’t have to convince Jason Atlas that yesterday’s databases weren’t built for today’s world. The Vice President of Engineering at IID experienced it for himself when the popular database his company was using almost puked because it couldn’t handle the momentum of data coming at it.
And while a little latency might be tolerated in some industries, at Atlas’s company it is not; after all it provides a platform where enterprises can share data about the latest cyber threats and ongoing attacks and be warned before the trouble that’s on its way -- and it’s always on its way -- can wreak havoc.
Doing this requires gathering and crunching huge amounts of data from a large number of sources, so much so that the term “big data” and its three v’s (high volume, high velocity, high variety) seem barely sufficient to describe it. And this data, mind you, not only needs to be collected, crunched and analyzed, but this also needs to be done in a highly secure and transparent way so that the financial institutions and other businesses, who share their information with IID, have full confidence that it is protected, uncompromised and used for intended purposes only.
The stakes here are extremely high, according to Atlas, because it’s only by sharing data about attempted intrusions, threat indicators and confidences that IID’s customers stand a chance of shielding themselves against trouble.
The criminals are as organized and probably more capable than we (enterprises) are. They’re working together and we’re not,” says Atlas. “They share with each other in ways that we don’t. There are sites out there where criminals buy, sell and trade things like bots, worms and Trojans.”
Suffice it to say that IID, which sometimes acts as “mission control,” needs to leverage the best technologies the world has to offer to get its job done. Speed, 100 percent up-time, security and transparency are the basic requirements and traditional databases like MySQL and Oracle won’t suffice because they weren’t built for this data driven world.
So when Atlas looked to replace the database he was using, he considered a number of alternatives -- MPP databases and most NoSQL databases, but they didn’t meet his needs until he found one that did -- open source Apache Cassandra.
“NoSQL does not equal NoSQL,” he explains, noting that different NoSQL databases are built for different things. One is not necessarily like the other and they don’t always compete.
Atlas chose Cassandra for IID because he was sure it could get the job done. It offers 100 percent up-time, scalability, transparency and high availability, without compromising performance, as well as proven fault-tolerance on commodity hardware. For mission critical data, like he works with, it was a no-brainer, says Atlas.
But there were other things on Atlas’s nice-to-have list that open source technologies don’t come with when you take them “off the shelf” (and mind you, the Apache open source version of Cassandra is free), like enterprise-grade support, training, integration with other platforms and technologies, and so on.
So Atlas, who says he’s be terrified of working on mission-critical projects without commercial-level support, had to find a remedy. The salve? DataStax’s Enterprise platform.