How did analytics enable the ancient Egyptian high priests to achieve tremendous power?
Egyptian priests’ ability to calculate allowed them to “predict” the Nile’s annual flooding, explained Austrian mathematician Rudolf Taschner in his book Die Zahl die aus der Kälte kam (The Number That Came in From the Cold).
Because ordinary people had no inkling of what the priests were doing, they believed the priests had supernatural powers and revered their pronouncements about sowing and harvesting.
It should come as no surprise, then, that the high priests had no interest in sharing the source of their knowledge with the people. That would have reduced their power significantly.
Similarly, companies and organizations have long limited data exploration and model creation to a small group of people. While this group may not have had the status of the high priests, without approachable tools ordinary people were nonetheless excluded from the inner circle.
The hoi polloi could passively receive results but were unable to perform analyses on their own.
Self-Service and the Democratization of Analytics
Today solutions like self-service data exploration and data visualization enable business users to explore their own data, generate results and test analytical models on their own.
This is a huge advantage, as business experts know the history and the background of the data better than anyone else and can assess findings from a business perspective.
Consider a few examples:
Life insurance companies use risk assessment systems to score applicants for insurance. Based on this score, the companies approve the applicants for a normal insurance rate, request a higher rate, or decline to issue a policy.
Insurance companies frequently employ medical doctors to evaluate their scoring systems from a medical perspective, because physicians understand the underlying relationships in detail. If these physicians have access to data, they can search for relationships and are likely to find the relevant relationships in the data to enable the life insurance companies to rate applicants and price policies in a more granular way.
In performing predictive maintenance for industrial equipment used in refineries, windmills and gas fields, it’s important to predict the remaining lifetime of pipes, drilling devices, engines and so on. It is usually much cheaper to replace these devices before they break to avoid large damages and out-of-service intervals. Engineers can analyze the data from a technical point of view, identify correlations between technical parameters and build models that predict failures. These models and the findings are an important input for the development of maintenance procedures.
Marketers who create campaigns can use analytical CRM to analyze customer behavior data to quickly identify reasons people respond to an offer. Data about customer behavior on specific campaigns can be generated more quickly, and marketers can use the findings to fine-tune offers in the short term.
Performing More Interactive Analysis
Previous reliance on analytics experts once caused bottlenecks for business users, as business people had to refer to technology experts each time they had a new question and these experts were frequently backlogged.
Often business people needed to have a back and forth conversation with the analysts, which could lead to misunderstandings and add to the time required for analysis. And business users were often limited to simple reports on their business results.
Access to self-service analytics allows business users to look behind the scenes and learn more about relationships. Users can touch the data to directly enter variables into or remove variables from the analysis, fine-tune models and see results instantly. Instead of simply looking at static reports, they can interactively analyze data or subgroups, filter segments or redefine groups.
They can even present results in a business meeting, taking advantage of a modern, graphical look and feel to more easily communicate with senior management or the board of directors. If necessary, they can even rerun the analysis in minutes in order to review “what if” scenarios during a meeting.
Should Analytics Experts Fear for Their Jobs?
In a word, no.
Self-service doesn't mean that users can run BI and data visualization applications without any involvement or intervention from analytics experts. It simply means more people can generate findings much quicker and share the results of the analysis in a collaborative and visually engaging way.
To make the transition to self-service analytics, analytics experts and business users will require new methods of collaboration.
Rather than analytics experts simply providing data, self-service means that experts will build production models, provide guidance and help the user gain autonomy in using the models.
Analytics experts will also need to evaluate the findings from the business users to decide whether these results are statistically reliable and how they can be included into the production analytics models.
Overall, my experience has been that the more an organization eats, the greater its appetite for analytics.
The more people in the organization deal with data analysis, the more knowledge and ideas they generate. Consequently, organizations are even more likely to rely on analytic expertise to make their most important decisions.