Creating customer profiles is a surprisingly difficult task, especially if for example you only have sparse information from a person's last couple of purchases. Consider a single woman who bought diapers for a baby shower. This person is going to want different recommendations the next time she shops than her friend who had the baby, but most recommendation engines don't recognize that. One answer to this dilemma is unsupervised learning, according to Danielle Swank, founder and head of product at Fomoro AI. “It can create brittle heuristic methods to deliver recommendations that people actually want,” she said.
Unsupervised learning is one of two classes of machine learning algorithms within machine learning that uses completely unlabeled data to train a model. The other class is "supervised" learning, which uses labeled data and aims to learn the relationship between two sets of variables in a dataset.
Businesses may be more familiar with this latter form of learning, as it is usually used to answer a specific question such as forecasting revenue for the quarter, said Pavel Dmitriev, vice president of data science at Outreach. But unsupervised learning, which has been around for a long time, has seen a recent spurt in popularity, Dmitriev continued. “That has been due to the huge increase in the amount of data being produced, which is very expensive to store and process in its original form.”
What is Unsupervised Learning
Unsupervised learning, part of machine learning, identifies patterns in data without explicit human supervision through labeling — or put another way, it tries to find structure or patterns in the data without the "help" that labeling provides. This is important because, as Dmitriev noted, storing and processing data has become expensive and using unsupervised learning can save time and money. “Labeling a single self-driving car image can cost upwards of $10 per image,” Swank said.
“A robust machine learning dataset might need a minimum of 10,000 to 100,000 images,” Swank said. Labeling costs, in other words, can quickly add up, she added.
Related Article: What Is Explainable AI (XAI)?
'Here is Some Data...Tell Me What You Find'
There are other reasons why unsupervised learning is a useful technique. It can be used when the model cannot be described and calibrated data does not exist, said Kristian Simsarian, founder of Collective Creativity and also a professor of technology design at the California College of the Arts. “Often it is not even known what success might be — a machine learning designer may simply be saying ‘here is some data, it may be interesting, tell me what you find.’” In the example of the woman buying diapers for a baby shower, unsupervised learning can help pinpoint her true buying needs by analyzing many different dimensions of customer attributes, product attributes and transaction attributes.
It is also useful for price optimization for this customer by categorizing her and others into micro segments of similar willingness to pay, said Michael Wu, chief AI strategist at PROS. “This gives a company a way to understand which customers are performing on or over target and which ones are underperforming,” he said. “As a result, this allows the sales team to negotiate for an optimal price with confidence, knowing what other customers within respective segments have historically paid for similar products.”
Unsupervised learning can also be applied to slow, manual processes that eat up a lot of resources, said Madison May, machine learning architect and co-founder of Indico. For instance, it is being used to build machine learning modes for contract analytics, audit planning and reporting, RFP analysis and composition, sales opportunity workflow automation, customer support analysis and automation, appraisal and claims analysis and invoice reconciliation. “AI with unsupervised learning has the potential to eliminate a lot of inefficiency there,” Walsh said.
A related use would be its support of a system that is able to better learn from messy data, said Peter Eckersley, director of research at the Partnership on AI. “In other words, AI that has the instincts to see that a row in a spreadsheet looks fishy [can] skip over it rather than being mathematically confused by it.”
Other areas where unsupervised learning can be useful include grouping news stories by related topics and identifying fraudulent credit card transactions, said Bruce Tannenbaum, product marketing manager of AI at MathWorks. “There are still many untapped opportunities for unsupervised learning,” he said.
Related Article: Why the Benefits of Artificial Intelligence Outweigh the Risks
Black Box AI
Unsupervised learning will also be key to Explainable AI, or XAI, said AJ Abdallat, CEO of Beyond Limits. “Some enterprises that have experimented with AI have learned that conventional AI approaches are ‘black boxes’ that cannot explain the reasoning behind their answers,” he said. The neural networks that are often used to produce results are literally too complex for humans to comprehend, he said. This opaqueness is starting to have a backlash as increasingly users want to know how AI arrived at its conclusions. “It’s one thing to trust a machine’s recommendation of a TV program or clothing accessory, it’s quite another to entrust AI with making investment decisions or medical decisions on its own,” Abdallat said.
The Limits of Unsupervised Learning
To be sure unsupervised learning has its limits. When used alone the results are unpredictable — and a certain kind of result is what people are usually aiming for when they use AI, said Natalie Rutgers, head of product at Deepgram. That said, unsupervised learning can be combined with supervised learning to get even more out of the dataset and train on new data. “Unsupervised learning is probably not going to outperform supervised learning on the same tasks,” said Swank. “It’s at its best when labeling is either not possible or too expensive.”