In The Ever-Expanding Importance of Analytics: True Intelligence for Business, I initiated an analysis of the current state of “business intelligence" -- the kind of intelligence that truly benefits enterprises derives from classic definitions of the word: Learning, understanding and applying knowledge and experience to new situations to make better decisions. Until recently, BI has mostly derived from practices and technology that address structured data and business processes inside the firewall. BI will now have to include unstructured or content sources in analytics, to add in missing subtleties, context and insight that cannot be pulled from structured data sources.

Now let's further analyze new directions for business intelligence and analytics. 

New World Focus on BI, Predictive Analytics

Some of the significant changes to analytics solutions relate to how final artifacts are “formatted." There has been a growing transition from traditional report formats to interactive visualization tools, and to collaborative processes that enhance the final artifacts. More companies want BI to help with current and future needs and goals, rather than measuring the past. And these companies should be inviting more individuals (via collaboration) to help refine the accuracy and contributions of analytics outputs. Newer on-demand BI/analytics solutions are providing “right now” intelligence, though still these are still works-in-progress, for the most part.

A number of articles lately have made the point that BI initiatives have not been successful for many customers, while making another important point: "Failure” actually is part of the BI refinement cycle. But for costly enterprise-style BI solutions, “failure” can be quite expensive. Gerry Brown of Bloor comments:

However, there is little doubt that the number of BI users overall is increasing. With low-cost operators such as QlikTech, Tableau, LogiXML, Pentaho and Jaspersoft, you get an awful lot of BI software for $25,000. Conversely, most enterprise BI vendors don't traditionally get out of bed for deals of less than $50,000. Low cost (or 'free') BI software encourages trial and experimentation with little risk, and this is what most enterprises prefer. After all, BI project 'failures' are common."

Referring again to the graphic for “BI Has Hit the Wall”, Stephen Few sees that “the traditional BI software vendors and most of the industry’s thought leaders are stuck on the left side of the wall” -- while the future of BI will come from more non-traditional BI solutions:

The software vendors that are providing effective data sense-making solutions -- those that make it possible to work in the realm of analytics on the right side of the wall -- have come from outside the traditional BI marketplace. Vendors like Tableau, TIBCO Spotfire, Panopticon, Advisor Solutions and SAS tend to either be spinoffs of university research or companies that have ventured into the BI marketplace from a long history of work in statistics.

In an interesting and thought-provoking Gartner Magic Quadrant for BI platforms, the analysts see significant changes in the solutions sought by enterprises:

…there is significant, if not euphoric, satisfaction with, and accelerated interest in, pure-play BI platforms. This is particularly true for smaller, innovative vendors filling needs left unmet by the larger vendors. To understand this paradox, it is necessary to consider a number of factors that are driving the BI platform buying decision today.

Other vendors to watch for new ways to perform and use BI / analytics include Lyzasoft, Predixion Software and GoodData. Lyzasoft and Predixion recently became partners for cloud-to-cloud analytics-as-a-service. GoodData provides BI PaaS to help speed solution deployments.

BI SaaS adoption, while very low today, will grow steadily as maturing BI SaaS solutions are delivered in private and public clouds and in on-premise and off-premise configurations by trusted vendors. …innovative, pure-play vendors offering highly interactive and graphical user interfaces built on alternative in-memory architectures to address their unmet ease-of-use and rapid deployment needs. The perceived benefit is so compelling that business users are making this choice, despite the risk of creating fragmented silos of applications and tools.

Business users in particular showed a growing impatience with the time to deploy and complexity of traditional enterprise tools, which led to a rise in departmental buying of alternatives.

Collaboration and the Human Element

The newer area of interest for BI/analytics solutions is the inclusion of collaborative activities to add contextual and qualitative layers to the output of BI processes. To achieve authentic intelligence, contextual / qualitative layers can provide strong basis to test, fine tune and filter the artifacts of analytics. Analytics can benefit greatly from human filters that bring experience, knowledge, creative thinking. Context has a big role here: context for sources, context for outcomes, context for usage with other data points to achieve the best Intelligence for “making better business decisions”.

Collaboration is far more than distributing and sharing documents. It is interactive, inclusive, cross-enterprise. If collaboration is to be part of analytics, then iterative collaborative processes should be established throughout analytics cycles. Collaboration for analytics means bringing in disparate people to test assumptions, validity of data sources, accuracy and relevance of the outputs. These additional participants should provide a wealth of experience, complementary concepts and other essential data and perspectives. Having to prove the relevance and accuracy of analytics results to sympathetic as well as less sympathetic individuals should strengthen BI / analytic processes and projects.

The possibilities for new applications of analytics increase with collaboration. Inviting in many-to-many interactions also opens up processes to new ideas from participants. Gartner found that social venues and collaboration help to track and capture outcomes of the decisions made based on BI / analytics:

Gartner's user surveys show that improved decision making is the key driver of BI purchases. However, most BI deployments emphasize information delivery and analysis to support fact-based decision making, but fail to link BI content with the decision itself, the decision outcome, or with the related collaboration and other decision inputs. This makes it impossible to capture decision-making best practices. Solutions are emerging that tie BI with social software and collaborative tools for higher-quality, more transparent decisions that will increase the value derived from BI applications.

Vendors embracing collaboration as an essential part of BI / analytics include Lyzasoft, Predixion and Tibco Spotfire. Keep in mind that these vendors are all in early stages of building in collaboration as a significant aspect of their analytics solutions.

Analytics Aren’t Always Fun and Games

While there are very interesting analytics solutions that are “friendly” to business users, it is essential that safeguards are in place to ensure that business users understand what they are doing with analytic models and whether the resulting “intelligence” artifacts are correct, meaningful and useful. Collaboration with others who know the data, understand analytics, visualize the big picture and so on, is one safeguard to ensure reliable analytics outcomes and correct usages.

In a particular enterprise, do enough people know what to do with analytics, both to start processes in meaningful ways and to audit outcomes to validate accuracy and relevance? Are users chasing the right problems or questions, providing the right data sources, including enough pieces? Do they have the understanding to work in a “big-picture” sense? Yes, analytics vendors should build in methods for validation, testing, guidance, but is that enough? Bob Warfield addresses this concern:

Few enterprises have the right analytical talent.

I was musing not long ago with VC and fellow EI Evangelos Simoudis that very few people actually know how to ask questions in a way that solves problems. It is something of a Sherlock Holmes conundrum. All the data is available. It is shatteringly obvious once someone connects the dots. Yet, very few know how to step across the stones that peek above the raging torrent of data to get to the other side where the answer lies without falling in and getting wet.

“You see Watson, but you do not observe.”

Beyond the traditional users of BI / analytics, there is an additional “challenge” to vendors to reach potential users who “need predictive analytics but don’t know it”. The challenge comes in two parts: first to find them and connect meaningfully with them; and then to provide guidance for every step of using the analytics solution to ensure proper outcomes. To connect with these “new” analytics users, vendors have to help them understand why they need analytics and how to use it. Self-serve training modules must be available for using the solution but also for understanding what analytics mean to the user’s role, industry and business needs. Detailed templates and best practices organized by industry scenarios are also important for success. It will be interesting to see how successful various vendors are for empowering such non-traditional users.

The Elephant in the Room: Reliable Structured Data

Although I’m not addressing data warehousing in this article, I do want to point out the need for basing any data-driven intelligence / analytics processes on reliable data. While it’s the right direction to provide analytics tools that work well for business users and increase ease-of-implementation and ease-of-use, diligence must be exercised to guarantee that data sources fed into analytic processes are trustworthy. Otherwise GIGO.

Teams comprised of IT and business must collaborate to create and maintain excellent processes for data profiling, data cleansing / quality and data integration. The business users know how source data is used and the context for analytic processes. IT is the partner to help with the data profiling, data quality and integration steps. Data profiling tools can be used by business users to audit data sources, test for errors and/or for the lack of the right data, and to understand different data sources.

Intelligence is nothing if not an institutionalized black market in perishable commodities"

A lot of data and content that matter for better “Intelligence for the Business” come with a short shelf life of significance. The new direction for many analytics solutions should bring this information more quickly into intelligence processes. When enterprises succeed in achieving richly integrated knowledge and context, authentic INTELLIGENCE is likely to result for “making better business decisions”.

Disclaimer: Julie Hunt is not affiliated with any of the vendors in this article.