Big data is no longer the sole domain of big companies.
As the perception of big data moves from futuristic hype to real-world opportunity, the promise of improved decision making, increased operational efficiency and new revenue streams has more organizations actively engaging in data analysis projects than ever before.
That no longer just means more enterprise organizations, either. Midmarket companies are jumping on the big data bandwagon in a big way.
In fact, a recent survey by Competitive Edge Research Reports indicates an astounding 96 percent of midmarket organizations are either already in flight with a big data initiative or plan to start one in the next year. That’s a whole lot of companies whose big data projects are either going to sink or swim in the very near future.
Learn from Experience
Companies who are later to the game in the adoption curve of any technology cycle have opportunities to learn from those who came before them. In the case of midmarket companies about to embark on big data projects, that means capitalizing on lessons from their enterprise forerunners.
By learning from the mistakes of big companies and taking steps to avoid them, smaller firms can position themselves to enjoy greater success. Here are the five most important lessons:
Lesson 1: Lack of alignment with executive stakeholders will derail any project
Data analysis done right is not about technology. It’s about business. Before you start any big data analytics project, you first need to secure the support of the company’s executive stakeholders.
There are two primary reasons this is so important. First, these stakeholders are the ones who can ensure that you have the resources you need — whether it’s the right team, the required budget, or the necessary data access — to position your project to succeed.
But there’s an even more important reason. Data analysis is only effective if someone is willing to act on it. This is a lesson many enterprises learned the hard way. If your key executives aren’t prepared to make tangible business decisions based on the findings of a big data project, the project itself will have served no purpose.
Lesson 2: Don’t fixate on infrastructure savings
Many big companies initially thought moving their archive data off legacy databases with expensive license requirements and onto the nearly free clusters of databases such as Hadoop would yield significant cost savings.
While shifting data to these unstructured sources can in fact save your company on licensing costs, the labor required to architect, deploy and manage these systems can be significant – so significant that many large companies are finding that all they’ve done is shift costs from licensing to labor.
The takeaway for midmarket companies is this: Factor labor costs into your return on investment calculations, but don’t fixate on infrastructure savings at the start. Focus instead on outlining and answering questions that are critical to your business. That’s where true savings are ultimately found.
Lesson 3: Data scientists aren't quite unicorns, but they’re close
Simply put, labor requirements in the big data realm are difficult to satisfy. Though new educational programs are now being created with increased regularity, universities and professional training services were not initially equipped to handle the tremendous demand for so-called data scientists.
The number of people needed to support the deployment of big data technologies has overwhelmed the pool of IT resources. If deep-pocketed enterprise companies can’t go out and hire the talent they need, chances are you won’t be able to either. There simply aren’t enough many data scientists in the world today, nor will there be in the foreseeable future. Instead of focusing on finding a single data scientist, you should instead focus on building data science teams from within your organization.
Train team members in-house to manage your customized big data initiatives. Find enthusiastic DBAs and business analysts willing to learn and take the next step and offer the on-the-job training they need to take it.
Lesson 4: Native analytics technology on big data platforms is limited
Just because you have an enterprise-grade big data platform in place, doesn’t mean you have true data analytics capabilities. Having a big data cluster isn’t the same as being able to run meaningful big data analytics.
Many enterprise companies have found it difficult to access and analyze data despite the implementation of a top-end big data platform. Fortunately, big data analytics is one of the hottest markets in all of IT. New providers with new offerings are sprouting up on what seems like a daily basis. Review the marketplace of analytics solutions and find one that provides the best fit for your organization.
Lesson 5: Collaboration is key
This is another lesson most enterprise organizations learned the hard way. Line of business leaders in marketing, sales and other functional departments were led to believe they could successfully embark on data analysis projects without the help — and in some cases, without the knowledge — of IT. As they soon found out, however, while they might be tremendous innovators, line of business leaders are not equipped to manage, govern and scale data analytics products.
- Are You Too Old to Work in Tech? IT's Midlife Crisis
- EMC Should Sell Documentum, HP Should Buy It
- Customer Success is a Failure
- If Hadoop Disappears, Will the Label on Your Distro Matter?
- Inside Acquia's Gartner Ascension, Web CMS' Next Road Trip
- 7 Deadly Signs of Career Burnout [Infographic]
- Connecting Workers to Information in the Digital Workplace