It’s hard to admit after 25 years that the business intelligence (BI) stacks we all worked to build are failing us, but to make progress, we must. If we, as technologists, take a hard look at what we’ve learned from the current generation of big data and other analytics technology and keep the following principles in mind, we can build empowering solutions.
1. Easy access to data
Traditional BI protected users from raw data and mandated a high degree of data normalization. This was a mistake because users can also benefit from raw and intermediate forms of data. The BI machinery for data modeling and normalization is also hard to adjust to ever changing requirements and lacks much needed agility. We should create a simple way to get unstructured, structured, machine, human-generated, real-time, historical and any other type of data through a universal repository, or at least a universal search.
2. Add context
Systems should enable users to ask relevant questions in the context of their work and the objects they are familiar with. What data would they like to inspect and in what timeframe? What are the key objects?
Users should be provided with the data they need and tools to analyze it. The system should add context and enrich data using machine-learning techniques for correlating data and adding relevance. If call detail records have cryptic internal IDs, the system should substitute phone numbers. User IDs should be linked to data about the user, including transaction history.
3. Advanced analytics for everyone!
We trust a GPS to get us where we want to go, but have no idea how it works. Similarly, in the future, business will be guided by systems using machine learning and advanced analytics. We must make advanced analytics easier to use and to trust. We must understand when the system can do the job and when it needs guidance. Data scientists can be a tremendous asset for a company, but you shouldn’t have to employ an army to run advanced analytics. Analytical systems that more people can use are more valuable than feature-rich systems usable only by experts.
4. Build on search
When people want to learn, they search. Let’s build on that.
Casual users can use natural language and assisted search techniques. We must make it easy to set the stage for analysis, jump into an experience, and jump out to keep searching. Search also makes it very easy to correlate data that is isolated in siloed data marts or in disparate systems. Advanced search technologies provide mechanisms to group events or matching records together, as transactions for example, making it easy to understand otherwise hidden relationships.
5. Provide speedy interactions
Responsiveness is implied in the search paradigm. Users understand that they may have to wait for complex requests, but they are happy to get partial results. Building results in an incremental fashion and streaming data continuously to the user interface supports an iterative workstyle for users to refine their requests with each iteration without waiting for long running batch jobs to complete. It won’t be easy to make things fast, but we must keep trying to bring instant value to users.
6. Make real-time information easy to use
In most companies, real-time systems are seen as specialized and advanced, not something you use every day. Most environments weren't built to handle a real-time display and required significant efforts to reduce the latency between data being created e.g. by a transaction and the event showing in your analytical environment.
However we must realize that the cost of acquiring and using real-time information has dropped dramatically. And while there are the scenarios where real time capabilities are irrelevant, we should always consider how real-time information could help. Particularly most operational systems will greatly benefit from very low latencies.
7. Make visualization pervasive
Often visualizations are the big finish of a long train of data analysis, a fancy painting on the wall. While the results are impressive, it also means visualization is often an afterthought. Visualization should be used more often, not just at the end of the process, but in exploring data, in reviewing interim results. BI should be more like a video game — primarily visual. Visualization tools are important to help overcome data overload when presenting information derived from big data and to help people recognize actionable insights. Information workers need to be empowered to explore data without preconceived notions.
8. Embed collaboration
Almost every analysis involves working with others. But if we want to collaborate and share an idea, we must stop what we are doing, move to email or IM, and record our thoughts. Often, it is hard to invite others to see what we’re looking at. The systems we create in the future must be collaborative and allow knowledge to be shared so that advanced practitioners can help others. Social networks are rapidly spreading into the corporate world and transforming communication and information sharing.
- Microsoft's Secret Social Weapon: Why Yammer Will Replace Traditional Extranets
- The Future of Digital Marketing: 8 Trends
- 6 Predictions for SharePoint, Office 365 in 2014
- HP Turns the Page ... to Cool?
- Enterprise Content Management Changes, Challenges in 2013 - Leading Vendor's Views
- Inside the Road to Acquia's Possible IPO
- HootSuite Enters the Enterprise with IBM Connections Integration