A Zappos Lesson in QA Management

About three years ago, Zappos decided their QA program needed a makeover. Agents didn’t have a lot of transparency into how they were being scored, and as a result, developed a negative perception of the QA team.

“We made a lot of mistakes early on. The QA team was off in a corner and rarely, if ever, interacted with the agents they were scoring,” Zappos Supervisor Dylan Morris says. Like most QA programs, the evaluation team would score calls and send forms straight to management, who would then coach team members based on their grades.

“There wasn’t an opportunity to close the dialogue loop between the team providing the scoring and the people being scored,” Morris added. It was starting to impact the ability of managers to effectively coach agents; instead of learning from their mistakes, agents would contest the validity of their score. Since the team’s process is to take calculated risks and adapt quickly based on lessons learned, management identified this challenge as an opportunity for change.

A snapshot of the Zappos support team at Customer Service Week.

A New Approach to the Team Dynamic

So, Zappos made a few changes that leveled the playing field between the QA team and the agents, and provided more opportunities for the two groups to interact. Today, each “advocate” on the QA team (now called The Learning Center team, or T.L.C.) spends at least five hours a week taking calls on the floor, just like an agent. This way they’re actually “walking the walk,” and stay connected to the team they are scoring.

Additionally, after scores are delivered to the people on the floor, agents have the opportunity to contest the evaluation if they feel like something wasn’t scored in accordance with the form.

“It’s never just, ‘here’s the review of your call.’ There’s always active participation from every party,” Morris says.

A Form That’s Weighted to ‘The Zappos Experience’

While making over the QA process, Zappos also decided to make changes to their form. The most important factor to creating a “Zappos Experience” is forging a personal emotional connection with every customer (which you can read more about in this article: “A Zappos Lesson in Customer Service Metrics”), and management didn’t feel like this was accurately reflected in the form.

So, they created a new document called the “Forming Lasting Experiences Starts Here,” or the F.L.E.S.H. form (The TLC team is zombie-themed, hence the acronym for the form). Their motto is “Quality is infectious! Let’s start an epidemic!”).

Click here to view the form.

The form evaluates calls against several very common characteristics, such as empathy and call courtesies (addressing the caller by name and using standard greetings and closings), but notice that each section is weighted differently towards the overall call score (“x3,” for example, means that score is weighted three times as heavily as sections with an “x1”).

As in most call center evaluations, the “Solution” section, which evaluates how well the agent actually solved the customer’s problem, is among the most heavily-weighted. However, more unique to Zappos, the “Connection” section is also one of the most important. This section most closely aligns with creating a “Zappos Experience.”

“We want customers to feel like they just got off the phone with a friend,” Morris says.

Advocates grade at least one call per team member, per month, though that number can increase if the team member requires additional coaching and feedback. If the agent misses the grade on the first call evaluated by the advocate, the agent’s manager will score at least two more calls. This can increase depending on whether the agent still needs improvement.

How Zappos Takes QA Management Above and Beyond

In addition to innovating their team dynamic and QA form, Zappos has also implemented a few unique tactics for QA training and employee engagement. These include:

Integrating Self-Check Sessions

About every six months, Zappos evaluates its QA form to make sure it is “continually raising the bar,” as Morris put it. At that time, agents are asked to grade their own calls using the form, so they get a first-hand look at how they will be evaluated going forward.

“This gives them the chance to get more comfortable with the form,” says Morris. “That way, they don’t feel like expectations are changing on them and they have no control.”

This is the only time that self-checks are mandatory, but agents are also given the option to listen to and grade their own calls at any time. Team members will commonly elect to do this if they are having trouble increasing their scores. Zappos has found this works better than one-on-one coaching for some agents.

“Surprisingly, team members on the floor tend to be their harshest critics. They’re quick to call out the things they missed,” Morris said.

Getting the Customer Involved

Like many companies, Zappos also uses customer feedback to manage call quality. This includes traditional methods, such as Net Promoter Scores, as well as more unique tactics, like “sharing great calls” and “customer props.”

“Sharing great calls” enables agents who feel like they created a really exceptional connection to personally reach out to the customer and ask for their take on the call. The customer feedback is then shared with management, giving them a “full circle view” of the connection, Morris says. Team leaders often save these call recordings, so they can be shared with other team members as an example they should follow.

“Customer props” happen when customers proactively reach out to Zappos to thank an agent for an exceptional experience. These are then emailed to the entire company, including to CEO Tony Hsieh.

Empowering Managers to Customize Coaching

In addition to letting agents elect to self-check their calls on an ad-hoc basis, team leaders are empowered to individualize coaching in other ways. For example, managers have learned that some agents learn best by listening to them take a call. In those instances, the agent will sit with the manager while they take a call and grade it just like a QA advocate would.

In other cases, the team leader might choose a peer-review method, where two agents on the same team (teams are typically between nine and 12 agents) will listen to and score each others’ calls, so they can learn from each others’ experiences.

“Everyone learns differently, everyone processes information a little differently, so we really trust the team leaders to decide what’s best based on the relationship they have with each individual agent,” Morris says.

Defining a Clear Path to Progression

Zappos provides a clear path for agents to move up within the contact team. They offer different support tiers that are based on advanced skillsets, each of which involves a pay increase. The higher tiers, for example, handle calls that take longer, or that involve customers who need to be escalated from a frontline team member.

Having a clear path for progression gives agents a reason to continually strive to improve their quality: they have something concrete to work towards. Along the way, agents might also get on-the-spot awards, including additional paid time off, gift cards or preferred parking. These are awarded when an agent gets a 100 percent quality score, has perfect attendance or exceeds other performance metrics.

Improve Through Constant Innovation

Above all, Zappos attributes its QA success to constantly fine-tuning its process. From the team leaders to advocates to agents on the floor, everyone is encouraged to suggest changes they think will improve the QA process. When everyone strives to improve, they ensure continuous innovation.