Neon sign: "This is the sign you've been looking for"

Here's Why Insight Engines Are the Next Big Thing

5 minute read
Kamran Khan avatar
With enterprise search increasingly influenced by intelligent enterprise assistants, the launch of the MQ for Insight Engines represents a logical update to Gartner’s Enterprise Search MQ.

Gartner recently released its new Magic Quadrant for Insight Engines, effectively replacing the Stamford, Conn.-based technology research company’s previous Magic Quadrant for Enterprise Search. 

Gartner defines insight engines as enterprise search that [provides] “more-natural access to information for knowledge workers and other constituents in ways that enterprise search has not.”

Improving Workplace Search and Collaboration

With enterprise search increasingly influenced by intelligent enterprise assistants, the launch of the MQ for Insight Engines represents a logical update to Gartner’s Enterprise Search MQ.

In fact, for several years now, our company’s customers have been asking for question answering systems like Siri that can improve their workplace search and collaboration functions. Now, with Google Assistant and Cortana, these systems are starting to become more common and therefore, in greater demand.

The Emergence of Natural Language Processing

It’s becoming evident that enterprises can benefit from question answering systems or insight engines in the workplace, particularly as emerging technologies such as natural language processing (NLP) and chatbots become increasingly common in today’s workplace.

As a component of artificial intelligence (AI), natural language processing is a computer program’s ability to understand human speech as it is spoken. It’s fast becoming essential to many business functions, from use in chatbots and digital assistants like Alexa, Siri and Google Home, to compliance monitoring, business intelligence and analytics.

Demand for NLP Currently Exceeds Supply

Consider all the unstructured and semi-structured content that can bring significant insights to an organization — queries, email communications, social media, videos, customer reviews, support requests, etc. NLP tools and techniques are helping businesses process, analyze, and understand these kinds of data to help those businesses operate more effectively and proactively.

But, how can you make NLP work in the enterprise? Today’s digital assistants only understand very broad, generic subjects like places, recipes, biographies, etc. But that’s not typically what today’s enterprise users are looking for in their daily work.

Personalizing the Insight Engines

We find that customers want to create insight engines that encompass their own worlds, whether that is intranets, ecommerce catalogues, employee databases, publishing databases or public sector content.

What’s more, each company has its own language, acronyms, metrics and processes, so each company requires its insight engine to understand its unique needs and context in order to answer questions or execute actions like:

  • What was the revenue for a certain product sold in North America last quarter?
  • Schedule a meeting with sales.

Tools for Fine-Tuning Insight Engines

Although insight engines have the capabilities to provide metadata management, natural language interfaces and knowledge discovery beyond search, these systems need to be heavily tuned to be able to handle questions specific to your workplace.

However, there are wide ranges of open source and commercial text analytics and NLP tools that can help with the tasks of processing the data such as text mining, extraction, tokenization and entity/phrase extraction.

Learning Opportunities

Here Come the Chatbots

Chatbots are taking over the world and the workplace. Powered by NLP, chatbots enable conversations between humans and computers in everyday business interactions. They bring deeper natural language understanding to bear, not only to enhance search, but also to provide entirely new ways for employees to use corporate data more productively.

For instance, an enterprise chatbot combining NLP, semantic search and voice recognition can interact with users to acquire necessary information such as dates, times and order details or integrate with business systems to complete common office tasks such as making reservations or processing orders.

Putting the Chatbots to Work

Here are just a few business use cases where chatbots may help improve productivity or customer service processes:

  • Handling customer services like filling orders, order status and tracking
  • Helping employees do basic tasks like reserving conference rooms, processing timesheets and submitting expenses
  • Providing automated responses to common customer support questions

Outside of the traditional office, organizations can also leverage chatbots for uses such as interactive games, controlling equipment, etc.

A Good Bot Is Hard to Find 

Enterprise chatbot architecture can integrate with your business data, search engine, NLP algorithms and in some cases, voice recognition, to provide answers or perform specific tasks.

But creating a good business bot isn’t easy. Not only should the chatbot respond well to natural language queries, it must also be able to handle rare cases or exceptions. So you’ll want to make sure your chatbot fits your business needs and will integrate well with your existing systems. And to implement your chatbot, you’ll most likely need ongoing access to a high level of technology expertise.

Fueling Discovery and Collaboration

As enterprises embark upon building their own insight engines, this will, in turn, fuel additional information discovery and collaboration in the workplace. Open source and cloud technologies are helping to accelerate the development of these applications and enable organizations to extend beyond traditional search as they head into the era of big data and AI.

The Digital Workplace Experience conference (June 18-20, Chicago) features the latest on intelligent workplace. For more major digital workplace topics, speakers and more, visit

About the author

Kamran Khan

As an investor and president and CEO of Pureinsights, Kamran provides the company with clear, pragmatic leadership. With over 25 years in the search industry, Kamran was a key executive at Convera before co-founding Search Technologies in 2005.

About CMSWire

For nearly two decades CMSWire, produced by Simpler Media Group, has been the world's leading community of customer experience professionals.


Today the CMSWire community consists of over 5 million influential customer experience, digital experience and customer service leaders, the majority of whom are based in North America and employed by medium to large organizations. Our sister community, Reworked gathers the world's leading employee experience and digital workplace professionals.

Join the Community

Get the CMSWire Mobile App

Download App Store
Download google play