Report: Why You Need to Understand Text Analytics' Rising Importance

By Dee-Ann LeBlanc | Published Nov 4, 2009

Forrester (news, site) recently asked how much of what you know about your business ecosystem comes from controlled, structured sources like surveys, product registration forms, customer interviews and market reports? If the answer is "just about everything," then you're leaving a huge amount of data on the table and then tossing it in the trash.

This data is in the form of unstructured content in social media, email, call center notes, help desk tickets, news articles, patent applications, contracts and more. The problem is that even if you want to take all of this data seriously, it's impractical to hire enough people to sort through and classify it.

Mine Data For Business Intelligence Patterns

The information is out there. You just have to sort through it, collate it and make use of it. To this end, a recent report from Forrester, "Text Analytics Takes Business Insight To New Depths: An Obscure Technology Has Found Its Killer App" highlights a class of software called text analytics.

According to Forrester, text analytics is the process of extracting and analyzing information patterns in text collections. Typically these tools come in vertical market, specialized forms, using domain-specific dictionaries to seek out common terms.

Text analytics takes both structured and unstructured information and analyzes it for patterns, image provided by Forrester Research, Inc.

How Text Analytics Works

Text analytics products generally work as follows:

The software examines the content you've asked it to mine, using its custom dictionary to extract references to people, products, locations, and related terms and concepts—a process known as entity extraction.
It then groups similar pieces of information together, such as the names of competitors as known from the dictionary.
It then starts mapping relationships between the extracted references, such as the name of a journalist who wrote about a competitor's product and the name of the article they wrote.
It finishes by performing sentiment analysis to determine whether the tone of the reference was positive or negative.

Of course, the above is a huge oversimplification, but it gives you the general idea.

Text analytics software spiders the data, prepares it, analyzes it, reports on it, and then delivers its results—image provided by Forrester Research, Inc.

Making Smart Use of Text Analytics

Forrester emphasizes that text analytics needs to be used in a context. Before giving any thought to what software you might choose, begin by defining the business problem you want to solve, and then:

Define the range and quality of sources you need to analyze, such as free-form blog posts versus call center notes from a relational database.
Understand the semantic capabilities of your existing investments, you may have software that can already do some basic analytics.
Clarify what you want to do with the annotated output, such as what questions you want to have answered and in what formats.
Determine whether a single technology will support your cross-functional needs, to ensure that everyone from market intelligence and product management has their needs met.