Between the massive volumes of data, new storage platforms, and latest processing and analysis methods, IT’s scope of responsibility has expanded dramatically.
IT leaders are left wondering how to manage the new technologies being thrown their way — especially when it comes to managing the new world of big data in combination with existing business intelligence (BI) initiatives.
Modern enterprise data architectures need to unite big data, which is stored and processed by platforms such as Apache Hadoop, with the cleansed and structured information traditionally held in the enterprise data warehouse (EDW).
Doing so will enable business users, like business analysts and executives, to make better decisions backed by the broadest, most recent and most detailed data.
A hybrid enterprise data architecture that combines big data, traditional EDW and BI is achievable, and some enterprise customers are already moving in this direction.
Keep these best practices in mind if you're trying to turn data into business-ready information:
Adopt Future-Proofed Data Architectures
Today, most organizations manage complex, multi-layered data environments that involve multiple data sources and a variety of data storage, warehouses and processes. With these current environments it’s challenging to gradually introduce Hadoop into data architectures without interrupting business processes, data access, user data flows and other things end users have grown used to.
Doing so requires architectures that can incorporate big data, while still leveraging existing investments in environment, processes and people. An analytic platform that can leverage an organization’s data warehouses as well as big data sources will help that organization connect and leverage existing hierarchies, data dimensions, measures and calculations, and also connect users to data pipelines developed in Hadoop.
Don't Waste Resources on Managing Operations
Hadoop complements BI investments, but it requires the manpower of skilled teams to manage Hadoop effectively over time.
Beyond the initial setup, it can often require extensive ongoing effort to maintain. The operational challenge of ongoing Hadoop management is one of the key reasons a leading analyst firm estimates that 70 percent of Hadoop implementations will fail to meet revenue and cost objectives over the next few years.
Outside Hadoop operational support can free up your teams to focus on data exploration and data insights that improve the business, while also limiting the operational costs of scaling Hadoop.
Make Big Data Available to Broader Business Groups
To date, most Hadoop users have been the more technical data scientists and developers, but business analysts and executives also see significant value in big data and are looking to tap into this valuable information asset. Business users will be slow to adopt big data if the the data is difficult to explore and analyze.
The needs for end-user data interfaces will vary across departments, roles and skillsets. While some users may require responsive dashboards, others may need intuitive visual discovery. Regardless of how the data is served up, it must be based on data that is consistent and accurate.
Big data uncovers a world of opportunities. To take advantage of them, IT leaders must ensure that the strategic and tactical architectural decisions that they make today will position them well for tomorrow. Keeping these three considerations in mind will help enterprises successfully navigate the business transformation to big data.