Some 90 percent of the world’s data — maybe more — was created in the past two years. 

Each digital move a person makes is now being watched and stored in an ever-expanding global universe of servers and data warehouses. By 2020, the size of this digital universe is expected to grow from 2,837 exabytes to 40,000 exabytes. 

That’s big data.

In addition to data from “traditional” sources (mobile, tablet, desktop), we now need to consider the impact of the Internet of Things. Cisco estimates 50 billion devices and objects will be connected to the Internet by 2020. 

The costs and risks associated with managing this data are ongoing concerns for many organizations.

In a global study of 350 companies, IBM determined the average cost of a data breach in 2014 was $3.79 million. Beyond the initial cost of breach management and recovery efforts, the increasing cost of lost business due to fears of identity theft and loss of confidence in a brand is expected to grow from $1.33 million to $1.57 million in 2015.

Respect the Data

Both companies and consumers face very significant risks associated with the management and security of data. 

While companies often have resources and back-up plans in the event of a data breach, consumers who have their identities stolen — and perhaps their bank accounts wiped out — may not. 

And while most public discourse focuses on the responsibility of the organization, consumers must also show up to the conversation if they continue to expect ever more personalized experiences from brands and institutions.

Transparency and respect go hand-in-hand when it comes to big data. 

When an organization is collecting information on a person, it should inform that person that data is being collected, disclose how it is being used and allow the customer an opportunity to weigh in on the decision. 

Upon learning preferences of individuals, organizations should strive to respect boundaries and honor requests in order to gain trust and develop stronger customer advocates in the long run.

When individuals are consistent and accurate with the data they provide, the value for the customer goes beyond a customized coupon from their favorite coffee shop.

Customers who have control of their data in the future will have a near-complete picture of their lives, according to MIT professor Alex “Sandy” Pentland in a recent article for HBR: “Seeing all the patterns of your life allows you to personalize medicine, personalize insurance, personalize finances.”

Secure and Accurate

Once the data is given, both organization and consumer play a role in keeping information secure and accurate. Practices like network partitioning, encryption, server hardening, system integrity monitoring, multi-factor authentication, centralized logging and altering, and vulnerability scanning need to be on the radar of all organizations managing customer information and records.

It’s important to note that this applies to all types of data, as even unstructured data can be used to identify individuals in a few simple steps. 

Keith Carter, professor at the Business School of the National University of Singapore, gives the example of a seemingly anonymous set of GPS records held by a government or organization.

“You would easily be able to find out who they are by identifying the address they regularly come from at seven or eight in the morning. You would be able to see the school or office where they then show up. You would be able to learn where they went back to in the evening," he said.

Consumer Responsibilities

And for consumers, while it’s fantastic to get your next batch of Tide by pushing a dash button on your washing machine, consumers can no longer just sit back and hope that their data is secure. 

Individual consumers need to recognize that the cost of convenience comes with responsibility as well. As the big data conversation continues to evolve, consumers today need to educate themselves, participate in the conversation and take responsibility to manage their digital identities.

At the very least, consumers need to develop a diverse set of passwords across accounts, store them in a secure location and update them regularly. 

When organizations misuse personal information, the consumer should communicate this information via a survey, phone call or email. These feedback loops are a critical element in helping organizations understand what is working, what is not working and what can be done better.

Forrester defines big data as “the practices and technologies that close the gap between the data available and the ability to turn that data into business insight.” 

I would argue that big data has potential to extend beyond mere business insights into every corner of our lives. Big data can help us understand and improve many things beyond sales margins, from disease to politics to the universe as we know it.

However, even if the bottom line is your primary concern, companies and organizations that support a stronger system of big data management have a lot to gain in the years ahead — as Pentland reminds us: “The economy will be healthier if the relationship between companies and consumers is more respectful, more balanced … a balanced approach can bring greater stability and profitability.”

Title image by Nicolai Berntsen.