Swimming Through Sentiment Analysis with Lexalytics
Everyone has the right to their own opinion, but add the web in the mix and it's not uncommon for companies to find themselves buried under a hill of negative sentiment. Enter Lexalytics (news, site), a company that helps keep track of what's being said and how it's being said. 

Sentiment Analysis 101

The definition of Sentiment Analysis is a broad one, but it essentially refers to technology that aims to determine the mood of written text. This means that anyone looking to gain insight into customer opinions expressed on blogs, social networks and other platforms can do so without having to sit there and actually read through them.

An easy example is Scout Labs (news, site), a company that listens to the voice of the customer over social media outlets like Twitter. Another is Thomson Reuters (news, site), an information company that uses sentiment analysis machine readers to help investor trading.

Behind the curtain, the Reuters machines assign numerical “sentiment scores” to words or phrases which are then processed to give an overall positive, neutral or negative score. These scores can then be added together to calculate the overall sentiment for a company, sector, index, etc. Meanwhile, the ScoutLabs application examines opinions, judgments and emotions about topics across the Web, and can then extract helpful quotes, or create graphs that visually represent brand or product reception.

Lift the curtain behind the curtain, and you’ll find Lexalytics, the provider of the sentiment analysis technology that fuels both of these companies, and several others as well.

Lexalytics 101

Though you might not have heard of them, Lexalytics has been doing what they do best since 2003. In addition to Scout Labs and Thomson Reuters, the company’s core text analysis and sentiment software, Salience, powers big names like Cisco Systems and Lithium Technologies, and has also been deployed as part of search solutions in conjunction with Microsoft and Endeca.

According to the company, Salience can work its way through any sort of English language text, and has been integrated into systems for business intelligence, reputation management, automated trading, survey analysis, market intelligence, search and retrieval, customer satisfaction, etc. Because it primarily focuses on the clients' applications, Salience can integrate with BI, dashboards, analytics and data warehouse systems. 

Use cases include:

  • For analysis and measurement providers: Specifically for measuring sentiment across social media platforms like Twitter. 
  • For manufacturing companies: Salience can expose engineering defects, and highlight competitor weaknesses.
  • For content and media companies: By picking through content and highlighting what works, Salience can boost advertising rates, content re-use, and site stickiness.

Unsurprisingly, the age of information overload has continued to keep Lexalytics busy this year. Let’s look at a couple of the most recent enhancements:

Sentiment Analysis of Short Form Content

This month Lexalytics enhanced reporting around Twitter by creating sentiment analysis for commonly used emoticons and acronyms. For example, 

  • LOL (Laugh Out Loud) -- Does not carry sentiment, nor does expanding it add any value to the resulting lexical processing; treated as an interjection
  • FTW: (For The Win) -- Carries positive sentiment
  • IDK: (I Don’t Know) -- Is useful when expanded out to its individual words

Positive:

  • :D
  • 8)
  • ;)

Negative:

  • :/
  • :<
  • :(
  • :S

Further, the symbols @ and # trigger additional processing to help discover associated sentiment and themes. 

lexalytics_emoticon.jpg

Lexalytics sentiment analysis

“We spent a few months enabling our software to better deal with such content,” said Lexalytics CEO Jeff Catlin. “The improvements we made make processing this content significantly more valuable.” 

And it's true--social speak is becoming more prominent, but is traditionally difficult to gauge out of context. The enhancement is very new, but if successful, could boost social media monitoring offerings in a huge way.  

Opinion Mining

The latest version of Salience also offers "opinion mining", which expands the ability to analyze and pull opinions out of indirect quotes such as:

  • Seth then asserted that this was a truly awesome feature.
  • Tim agreed that Bill was unduly angry.
  • Paul explained that the code was broken.

lexalytics_opinion_mining.jpg

Lexalytics opinion minning

lexalytics_opinion_mining_2.jpg

Lexalytics opinion mining

Sentiment Analysis + Content Management

Lexalytics mines in-house content as well (CMS people, rejoice). An application called Lexascope Web Service can break documents down into topical elements and drill in on concepts, people, and whatever else is going on on a topical basis.

"This is not Google," insisted Catlin in a phone interview. "This is taking large content sets living in a CMS and navigating them in ways that are uncommon in order to expose things you might not have thought about. It's not a gurantee you're going to get great stuff, but it is a way to ask better questions and learn things you might not've asked about otherwise."

“If it’s text, and it’s English, we can read it and add value to it,” said Seth Redmore, VP of Products. “The Lexascope Web Service can make the content you’re writing more valuable by extracting metadata that you can use to boost SEO, or can take mounds of content coming in and give you a high level view of what’s in that content so that you can make better decisions.”

Problems and Solutions  in Sentiment Analysis

Accuracy has been a challenge for sentiment analysis since day one. 

"No single approach today provides the desired accuracy," said expert and author on Web data mining, Bing Liu. "Thus, in practice, it is important to have novel ideas that can cleverly combine existing methods to accurately classify or predict sentiments and emotions. I expect some innovative techniques to come out in the next few years." 

Greg Radner, global head of PR Services at Thomson Reuters, guessed at one of those innovative techniques earlier this year: "The next thing beyond sentiment analysis is understanding what the nature of those conversations are. It’s only so helpful to be able to say something has a positive or negative tone, but that doesn’t itself give insight into the nature of the conversation, into what people are really saying." 

Meanwhile, Lexalytics claims that the importance of accuracy and automation differ depending on the solution. "An example we often use where a technology-based automated solution really shines is in financial services where the trends across a collection of stories are what users are most interested in," wrote former senior marketing manager, Christine Sierra. "They care less about the accuracy of every document detail, and more about the sentiment across a corpus of data that needs to be processed quickly."

On the other hand, reputation management is another story. "It could be said that automated sentiment analysis was born in this space, and was invented because of the amount of time people spent hand measuring the tone around products and brands," added Sierra. "I bring up these two contrasting uses because it's important for people to think about their specific needs and requirements before they jump into using any vendor's solution. Make sure the solution you're looking at is well-suited for the problem you're trying to solve."

Lexalytics is one of the vendors worth checking out in the sentiment analysis market, if you're looking to get your feet wet.