EMC knows how to grab headlines.
The opening feed on their website right now streams Rock Star Report — Are you ready for the hottest job in tech?
The job they’re referring to, of course, is that of a data scientist. Earlier this month the storage giant released the results of a large study it sponsored that concluded that there is an extreme shortage of data scientists. EMC’s PR team then did a bang-up job publicizing the study’s results and news outlets all over the web picked it up.
And…VOILA! The fact that there’s a shortage of data scientists became BIG news.
But it wasn’t actually news. Most everyone in the industry already knew that data scientists were in high demand and short supply. It’s a subject that’s been addressed regularly and repetitively at Big Data Conferences hosted by the likes of GigaOM, O’Reilly and EMC itself.
Not only that, but news of the shortage even hit mainstream media outlets such as Business Week and Forbes, as well as web publications that proclaim things like “Big Data is new oil” and then point-out that there aren’t enough data scientists to usurp value from all the cumulative data that’s been created.
So, why the sudden, well-publicized shortage of data scientists?
There are a few ridiculously simple reasons.
The Term “Data Scientist” is Still New
Word on the street is that the term “data scientist” didn’t even exist until 2007 when Jeff Hammerbacher, who at that time worked for Facebook (he now works at Cloudera), coined it to describe the roles and the skill sets of the team that he led. He reasoned that their yet-to-be determined job titles belonged partly to “data analysts” and partly to “research scientists”.
Hence: data analyst + research scientist = data scientist
“Data Scientist” Jobs = Near Zero Until 2010
It should be, almost, needless to say that if the term “data scientist” is a new, then the supply of professionals who have worked carrying that job title is likely to be very small. Not only that, but the likelihood of academic institutions offering programs that graduate data scientists is also likely to be small.
Case and point? Mega job search engines like Indeed.com indicate that the demand for workers carrying a “data scientist” job title was virtually non-existent prior to 2010.
Large Data Set Computation Becomes Faster and Cheaper = Skyrocketing Growth for Data Scientists
It wasn’t until after Big Data was evangelized that the industry realized that the gold in their corporate and communal data stores could be brought to the surface and made usable in a cost-efficient and time-efficient manner.
It would take data scientists to get that job done and most employers didn’t have that kind of talent in house; hence data scientist job growth skyrocketed 6500% from February of 2010 until present.
It should be noted, however, that “data scientist” job postings represent less than .006 of all job postings today.
Data Scientist Demand Will Rise as Awareness Grows — The Facebook/Linkedin Effect
Companies are only now beginning to see the value they can usurp from data they own (and can buy). They’ve watched companies like Facebook and Linkedin create products from their data stores and they’re beginning to think that they can produce something of value too.
In order to do that, they need data scientists.
Is There a Shortage of Data Scientists?
You bet. There will be a shortage at this time next year too.
Will that be news? Nope. The news will happen when the demand for data scientists equals data scientist supply and that won’t occur for a very long time.