As the retail landscape continues to evolve, many companies rely on quality customer reviews to elevate brand awareness. This is true for many of our clients, including a national retailer using customer reviews across its website and email communications. When you visit the website, you are greeted with targeted customer reviews based on your location.
When the brand transitioned its customer review technology from BazaarVoice to Medallia, the thousands of reviews generated from customer feedback surveys was unmanageable. And manually moderating them was time consuming and redundant.
The retailer was familiar with machine learning and asked DEG to implement a solution that used machine learning to automatically process reviews and provide an application for employees to provide manual intervention if needed.
To make the review moderation less painful, our team looked at the client’s current technology roster to leverage its existing tools. Since the brand was already managing its website on Sitecore—a digital experience platform—a custom application within the content management system (CMS) platform was ideal for the manual intervention feature.
We designed a dashboard called the Review Moderation Application to manage the reviews, making human moderation or intervention quick and intuitive. We also evaluated historical review data to see why reviews could or would be rejected, identifying patterns and themes for the machine learning capabilities needed.
Based on the data, our technical experts identified a group of Azure functions for automatic processing of reviews.
To meet the retailer’s needs, we chose the Azure Cognitive Services—a family of AI services with APIs—specifically selecting the Content Moderator Cognitive Service. This machine-assisted content filtering software automatically moderates content for profanity and inappropriate language. We also leveraged the Sentiment Analysis API to evaluate text voice and tone for overall sentiment, producing a score that can be applied to reviews. This allows us to automatically flag any customer reviews that have a negative sentiment score.
Our team also identified the need for custom-built moderation services to identify price, competitor reference, and personally identifying information (PII). If the Azure functions identify any of these in the review text, a code or flag is attached to the review in the database before it is sent to the Human Moderation Application dashboard.
How the Review Moderation Application works
When someone writes a customer review, here’s how the data is collected and sent to the Review Moderation Application.
- Medallia’s customer review technology collects completed survey data from an email campaign sent to existing customers.
- Reviews are sent to the brand’s data warehouse, where all reviews are stored including those sent to the Review Moderation Application.
- New reviews are pulled from the database and Azure functions are applied to automatically moderate reviews.
- The reviews are then sent to the Review Moderation Application.
Since the Review Moderation Application is built in Sitecore, staff has the ability to moderate and intervene if necessary. When a review comes into the application, it includes any flags identified by the Azure Cognitive Services.
A review with no flags is approved for use in marketing materials, while any review that comes into the application with a flag will be automatically rejected. A staff member then can add or remove flags if they disagree with the services’ findings. An example of this may be a review that isn’t great, but the reason for rejection doesn’t fall into an existing code category.
With the Review Moderation Application, this retailer’s moderators can now add manual attributes to review that are manually or automatically rejected. These include general rejection comments, marketing categories, and marketing value scores. Updated moderated reviews are saved back into the brand’s data warehouse, helping DEG leverage additional, more complex Azure functions in the upcoming phases of the project.
So far, the Review Moderation Application has processed hundreds of thousands of reviews, and DEG is continuing to enhance the capabilities and quality of the data collected. Additional Azure functions are being evaluated as we work toward a goal of zero human intervention.