Improving the Consumer E-commerce
Experience Through Text Mining
IndustryView | 2015
Text mining, the process of analyzing textual information in order to identify patterns and gain insights, is increasingly being used by e-commerce retailers to learn more about consumers. By identifying customer purchase patterns and opinions on particular products, e-commerce retailers can target specific individuals or segments with personalized offers and discounts to boost sales and increase customer loyalty.
To learn more, Software Advice conducted a survey to determine how consumers feel about e-commerce companies using text mining practices to personalize product or service offerings. This report will help online retailers understand which text mining practices are most likely to improve the customer experience.
The rise of the digital age has resulted in a host of new information that online retailers can use to improve their marketing efforts and differentiate themselves from competitors to gain customers’ business. Many are turning to technology such as text mining software to tailor the shopping experience to consumers’ personal preferences.
Text mining solutions are used to analyze digitized text from different written sources (e.g., search engines, blogs and forums) and social media platforms (e.g., Twitter and Facebook) to identify patterns and trends on brand affinity, product preferences, consumption patterns and more.
Fiona McNeill is the global product marketing manager for analytics provider SAS and co-author of "The Heuristics in Analytics: A Practical Perspective of What Influences Our Analytical World." She explains that there are two complementary approaches to text mining. Basic text mining starts as a “bag of words approach” that provides data on the frequency of certain words. The linguistic approach (natural language processing or text analysis) then provides the semantic meaning of the words.
Sentiment analysis is used to determine if a conversation about a brand is positive or negative. In other words, do people like or dislike a product, and what do they like or dislike about it? This information can then be used to provide online shoppers with a more personalized shopping experience and more appealing pricing options.
To help retailers make better use of text mining solutions, we surveyed consumers to find out how they feel about retailers’ use of this technology to improve their e-commerce experience. This report highlights our most important findings.
One major benefit of text mining software is that it can help retailers understand which products customers are most likely to purchase. For example, a company can use this technology to monitor what a consumer is searching for on their website to see what they might be interested in.
Or, it could follow conversations on social media channels (such as Facebook, Twitter or LinkedIn) to learn what products and services someone is talking about. This information can then be used to offer consumers personalized discounts, greatly increasing the chance they’ll make a purchase.
Our data indicates personalized discounts can be highly effective: The majority of respondents (50 percent) say they would be “much more likely” to choose a retailer that offered personalized discounts, while 39 percent say they would be “somewhat more likely.”
Differential pricing has been used by retailers for some time. Lowes, for example, uses customers’ ZIP codes as a pricing tool on their home-improvement site. Online shoppers receive whichever is lower: the online store price or the price at their local Lowe's store, as indicated by their ZIP code. These pricing schemes can reduce the need for direct price competition.
IBM explains that companies can also use social media data to identify customer impressions of pricing, and then combine that information with the location (or other statistical) data that either Twitter or mobile devices provide to customize the shopping experience.
The benefits of this strategy are clear: Retailers increase the chances of making a sale, and stand to make a greater profit on specific products, while customers receive deals on the products they’re interested in. NPR also suggests that customers feel special when they get personalized discounts, which encourages brand loyalty.
But targeted pricing is not without risk. Orbitz came under fire for charging Mac users as much as 30 percent more than PC users for a night’s lodging, while Amazon was forced to apologize for practices where consumers were shown different prices for identical products.
Companies run into trouble when the pricing is arbitrary. “If there’s a rational reason for it, then that needs to be a part of the value proposition,” McNeill says. “If it’s an artificially derived notion, then absolutely, they should be called out on it.”
Instead of diving head first into complicated pricing schemes, Andrew Fast, chief scientist at Elder Research, Inc., a data mining, predictive analytics and text mining consulting company, says retailers should begin with a limited test or pilot study.
“Start small and see what works,” he advises. “Instead of starting with individually tailored offers, try hitting a specific demographic where you can make an offer that is more personalized than an offer made to everybody. Then you can increase the personalization as you improve.”
When asked what channel they preferred to receive personalized discounts, the majority of respondents (59 percent) cite email, while just 24 percent cite the retailer’s website.
Fast says that personalized discounts are “the holy grail of text mining,” but are more useful when a customer is thinking about buying rather than when they are already shopping. This is because, since popular companies like Amazon often include recommendations at nearly every point of the online retail experience, customers have been “trained” to quickly dismiss them or ignore them altogether.
“[Recommendations] look general on a website,” Fast explains. “An intensely personal offer on a website is viewed a bit more negatively than a personal email. Email offers less pressure and special discounts are less in-your-face.”
McNeill agrees with the idea that consumers might be attracted to the intimacy of email interaction. “There’s the notion of, ‘If you’re really going to personalize something for me, then send it to me personally,’” she says.
In other words, email serves as a low-pressure reminder for people who weren’t quite ready to pull the trigger when they visited a store but may still be ready to make a purchase if offered a discount.
Moving on to customer preferences for receiving product suggestions, the data is nearly the opposite of the previous chart: 52 percent of respondents say their preferred channel is the website homepage, while just 24 percent cite email.
Whereas offering discounts may be a more effective strategy when a customer is debating a purchase, Fast explains that “customers who are visiting a retail website are already in the mood to buy. It seems natural that that’s the appropriate time to show them product suggestions.”
Customers also have high expectations for online shopping, which can influence their preferences. “Consumers want to be able to find products as intuitively as they can in stores,” explains Marc Hayem, vice president of platform transformation at RichRelevance Inc., which provides an online recommendation engine service for retailers.
As such, tailoring a retailer’s website homepage to a customer’s unique needs and habits offers these type of conveniences, at the perfect moment—when the customer intends to buy.
Of course, product suggestions must be attractive to the customer if they are to result in increased sales. The challenge comes in transforming the information gathered from text mining into a recommendation that’s both usable and relevant. Counting product mentions on social media channels (Twitter feeds or search history, for example) can say a lot about whether a product is on a consumer’s mind. But, it doesn’t always identify whether that attention is positive or negative.
Seth Redmore, vice president of marketing at Lexalytics, a provider of text and sentiment analysis software, explains that the challenge is that text is often contextual. The word “sick” is usually negative for a health care company, for example, but would indicate positive sentiment for a video game retailer.
Adopting a text mining solution that features advanced natural language processing engines can give companies the ability to customize their sentiment analysis settings to pick up on these subtle differences. If “sick” is indeed a positive term for a retailer, the company could program their text mining software to assign a positive score to the word.
Text mining invariably involves issues around privacy. To better understand consumers’ perceptions in this area, we next asked respondents to rate their comfort level with text mining of various data sources.
While only 37 and 34 percent are uncomfortable with text mining of product reviews and website search history, respectively, a significant 76 percent say they are uncomfortable with text mining practices that involve social media.
There’s a wealth of data to be found on social media channels. If a clothing company wants to know how well a new product launch will be received, for example, they could monitor the number of Tweets related to the product and see if the sentiment is positive or negative. The intelligence gathered could then be used to tweak the design or tailor the launch strategy.
Redmore notes that, when it comes to using information gathered from social media text mining, both timing and the nature of the existing relationship are critical to engage successfully with a customer.
“If you notice that someone is tweeting that they broke their camera, and you happen to have a business relationship with them (and you sell cameras), that is an excellent time to step in and say ‘Hey, bummer about the camera, how about 20 percent off to replace it,’” he says.
But privacy is a sticky issue; hence respondents’ discomfort with the analysis of their social media data. Many find it unsettling when businesses make use of information that wasn’t directly provided to them. This is one reason why many customers balk at the mining of their social media data, yet are comfortable with the text they enter in the search bar of a retailer’s website.
Digital Trends notes that customers are essentially bartering their data in return for the convenience and experience that comes with retail personalization. While some might be comfortable sharing their data with retailers they already have relationships with, many worry about whether their data is being shared with third party sites.
Customers also tend to have a poor understanding of what’s publically visible, and assume that more privacy exists than actually does. “For a lot of people, when they use social media, they do believe there is some degree of privacy,” McNeill explains. “But depending on their settings, there may be little or none.”
Still, U.S. social commerce sales of physical goods through online social networks are projected to grow by 93 percent per year in the U.S., reaching $14 billion in 2015. Clearly, many consumers are still willing to make the trade-off for more convenience, more choices and better deals.
“This all comes back to subtlety, transparency and usefulness,” Redmore says. “Much is forgiven if you have a better product or a discount that comes at the right time.”
Finally, McNeill adds that there’s a distinct line between the anonymity that people believe they’re entitled to on social media and using social media for direct communication when someone has not opted in. Retailers must be very careful not to cross it.
“When you single out someone in social media based on what they consider to be an anonymous conversation, that's when you run into problems. Any retailer that wants to be successful in communication with consumers will want to have opt in flexibility so that someone can choose to participate or not.”
Of the survey respondents, 56 percent identify as male and 44 percent identify as female.
Half of all respondents are between the ages of 25 and 34, and nearly a quarter of respondents are between the ages of 35 and 44. Fourteen percent of respondents are aged 18 to 24, and another 15 percent of respondents are aged 45 and older.
Over half (60 percent) of respondents report a household income of under $50,000. Nineteen percent report a household income of $50,001-75,000. Thirty-six percent report a household income of $75,001-100,000.
In the e-commerce world, text mining is extremely useful in allowing businesses to learn a great deal about customer sentiment and preferences towards products or services.
For those retailers considering the implementation of text mining software, McNeill advises them not to wait.
Because there is so much unstructured data that has been historically untapped, we are finding that, at a minimum, [text mining software] is improving the work that companies are already doing. At the other end of the spectrum, it’s actually negating previous notions of what was important. You get much more of the conversation and much more of the meaning from text.Fiona McNeill, Author
To find the data in this report, we conducted a six-day online survey of seven questions, and gathered 380 responses from random customers within North America, Europe, Asia, Africa and Australia. We screened our sample to only include respondents who made a purchase from an e-commerce site within the past six months. Software Advice performed and funded this research independently.
Results are representative of our survey sample, not necessarily the population as a whole. Sources attributed and products referenced in this article may or may not represent client vendors of Software Advice, but vendor status is never used as a basis for selection. Expert commentary solely represents the views of the individual. Chart values are rounded to the nearest whole number.
If you have comments or would like to obtain access to any of the charts above, please contact firstname.lastname@example.org.