The growing use of Apache Hadoop, increasing data warehouse volume sizes and the accumulation of legacy systems in organizations are all factors fostering structured data growth. To manage it, organizations are being forced to look at data archiving and how to get non essential data out of mainstream enterprise applications.
The findings appear in Gartner’s first Magic Quadrant for Structured Data Archiving. This is the first time that Gartner has produced such a Magic Quadrant, which features 13 vendors in total including the four leaders.
While many enterprises are yet to embrace data migration and archiving, Gartner estimates that migration will represent about 20 percent of all information governance projects. To help enterprises do this, four vendors have emerged as the leaders in the archiving space: HP, IBM, Informatics, and Solix.
In this article, we will explore this quadrant and take a look at emerging trends and the market in general. In a second article, we'll take a deeper look at the leaders and what they are offering.
A New Need?
Given that many enterprises are currently buckling under the weight of structured data, it is surprising that this is Gartner's first Magic Quadrant for this area. But maybe that’s because the need for this kind of capability has only started to be verbalized recently.
In fact, a constant feature — and a surprising one — that we find when looking at information governance is that archiving, in many enterprises, appears accidental rather than part of any coherent strategy. That is rapidly changing, however, as data creation continues to experience massive growth and the need to keep and archive data increases as regulatory regimes become increasingly demanding.
The amount of information that enterprises are being forced to keep has grown so rapidly and to such volume that by 2016, 75 percent of structured data archiving applications will incorporate support for big data analytics, Gartner reports.
There are a number of other factors that are driving the structured data archiving market, not least of which is the fact that it just doesn’t make sense for enterprises to be running up storage bills by holding on to information that can be removed from direct line of business processes.
Gartner also points out that organizational mergers and acquisitions, data center consolidation, and migration to cloud based applications have accelerated the requirement to retire legacy and redundant applications.
Application retirement presents numerous cost benefits and efficiencies that further fuel this trend, the research says. Summing up, the report concludes:
The benefits of using this technology include reduced capital and operating expenditures, improved information governance, lower risk, and access to secondary data for reporting and analysis."
Structured Data Archiving
Just to be clear about what Gartner researched in this Magic Quadrant, the definition of structured data archiving being used is:
The ability to index, migrate and protect application data in secondary databases or flat files typically located on lower cost storage for policy based retention. It makes data available in context and protects it in the event of litigation or an audit."
It address four issues in particular:
- Storage optimization: Reduces the volume of data in production thus reducing capital and operating expenditures. It also provides access to primary and secondary data for reporting and analysis
- Governance: Keeps appropriate data for compliance and regulatory issues, particularly structured data related to financial issues.
- Cost optimization: Structured data and application retirement can result in significant data costs savings.
- Data scalability: The technology can manage large volumes of nontraditional data resulting from newer applications that can generate billions of small objects. Scalability to petabytes of capacity is required in these cases.
Data Archiving Market
Gartner estimates that the market was worth $270 million last year, but also that it is growing at a compound rate of 10 percent every year. There are, however, four trends that will dominate this market over the coming years:
Big Data: The market, however, is changing and expanding due to growth in data, application retirement, information governance and big data analysis opportunities. The use of Hadoop and data warehouses means that organizations can keep more information, as well as social and machine data that doesn’t fit into databases.
Information Governance: Structured data contained in applications, especially legacy applications, is an easy target for external auditors. In particular, they are quick to identify applications and associated lack of controls around the protection of sensitive information. If the data is contained in legacy systems that are so old they are no longer supported, or badly managed, they may expect enterprises to migrate to newer versions or entirely new applications. The preparation and response to e-discovery requests is also a core element in structured data e-discovery and governance.
- Why Agile As We Know It Will Disappear
- SWAM: When LinkedIn Locks Down Social Networking
- The Metamorphosis of the Social Enterprise
- Intranet Search: Where Documents Go to Die or KM Enabler?
- Just How Badly Does Microsoft Want Your OneDrive Biz?
- Pivotal Revs Its Big Data Play, But There's a Better Story
- ROI Is the Wrong Tool to Justify Social Investments