Here is how I built a support QA program from scratch. A support QA program is a routine for reviewing a sample of tickets against a shared, weighted scorecard so you can measure quality and coach agents. It needs five parts: the scorecard, a fixed monthly sample, calibration between reviewers, coaching and a short report. The core formula is QA score = points earned ÷ points possible × 100.
The free QA score calculator handles weights and criteria that do not apply. In Part 2 we cleared a growing backlog without hiring. This part makes sure the tickets your team now solves faster are also solved well.
Why do you need QA if you already measure CSAT?
CSAT gives you the customer's view. QA gives you the inside view. A polite reply that solves the wrong problem can still get a good rating, and a correct answer can get a bad one because the policy behind it is unpopular. You need both.
Start small. Four criteria applied the same way by every reviewer beat a twenty-line form nobody trusts.
What should a support QA scorecard include?
Describe an excellent ticket, then group it into four to six weighted criteria.
| Criterion | What reviewers check | Weight |
|---|---|---|
| Resolution and accuracy | Correct, complete fix; next question answered | 35 |
| Communication | Clear, right length, plain language | 25 |
| Process | Tags, priority, identity checks, escalation rules | 20 |
| Tone | Acknowledges impact, owns the issue | 20 |
Use Yes, Partly or No (full, half or zero points). Ten-point scales create more disagreement between reviewers. Add one auto-fail rule, such as sharing account data with an unverified contact.
Free download
Get the Erlang calculator (call center staffing) spreadsheet
The Erlang calculator (call center staffing) as an Excel file with the formulas set up, ready for your own numbers. Works in Google Sheets too.
The download button appears here right away; nothing is sent to your inbox. Your email also joins Clarivoxx Insights (a weekly article and a monthly newsletter). Unsubscribe in one click. Privacy
How do you calculate a QA score?
QA score (%) = Points earned ÷ Points possible × 100
Points earned = sum of the points given on each criterion
(Yes = full weight, Partly = half, No = zero)
Points possible = sum of the weights of the criteria that apply to this ticket
Auto-fail = score is 0 regardless of other criteria
Say a ticket gets full marks for resolution (35) and communication (25), and half marks for process (10, wrong tag) and tone (10). The score is 80 ÷ 100 = 80%. If process does not apply to that ticket, remove its 20 points from both sides: 70 ÷ 80 = 87.5%. Check your own scorecard in the QA score calculator.
How one ticket scores 80%
QA score = points earned ÷ points possible × 100. Here (35 + 25 + 10 + 10) ÷ 100 × 100 = 80%. If process does not apply, remove its 20 points from both sides: 70 ÷ 80 × 100 = 87.5%.
How many tickets should QA review each month?
Reviewing every ticket is not realistic. A fixed monthly sample per agent is fairer. Mix random tickets with a few targeted ones, such as low CSAT or escalations.
Three samples, one fair picture
Random sample (6 per agent)
Drawn evenly across channels and weeks
Measures typical work fairly
Basis for trend reporting
Targeted sample (2 per agent)
Low CSAT or reopened tickets
Escalated or long-running cases
Feeds coaching and root-cause review
New-hire sample (double for 60 days)
Every ticket type the agent is cleared for
Weekly feedback instead of monthly
Ends when two months score above baseline
Reviewer hours per month = Agents × Reviews per agent × Minutes per review ÷ 60
For 12 agents with 8 reviews each at 12 minutes: 12 × 8 × 12 ÷ 60 = 19.2 hours a month. Double the sample for new hires in their first 60 days. If that is too much for one lead, share it with a second lead or trained peers.
How do you calibrate QA reviewers?
A score only means something if two reviewers would give the same ticket the same score. Every two weeks, have all reviewers score the same five tickets alone, then compare and discuss every gap. Write the decisions into the scorecard guide with examples.
I put anything more than 5 points from the group average on the discussion list. After two months, monthly calibration is enough. If you want a formal measure of agreement, Cohen's kappa corrects for the agreement two reviewers would reach by chance. For a small team, the 5-point spread is usually enough.
How do you turn QA scores into coaching?
Scores without coaching feel like surveillance. Share every review with the agent, allow a simple way to dispute a score, and use a monthly one-to-one to pick one criterion to work on, with two or three real tickets. Ask agents to score one of their own tickets first. The gap between their score and yours is often the best conversation.
QA also supports earlier parts of this series. The resolution criterion helps first contact resolution, and accurate tagging makes the splits from the ticket backlog playbook reliable.
What I've seen in practice
When I first wrote a scorecard, I gave "tone" the biggest weight because partners complained about it. After a month of reviews, the real issue was wrong refund timelines, said very politely. We moved the weight to accuracy and the coaching conversations changed straight away.
On one team, our French-speaking and English-speaking reviewers scored "tone" quite differently. What felt warm in one language read as too casual in the other. Two calibration sessions with real tickets from both markets fixed most of it.
The trap I watch for now: QA rising while CSAT falls. When that happens, the scorecard measures the wrong things, and I go back to the criteria before I coach anyone.
What should a monthly QA report show?
The overall average hides where coaching should go. Report each criterion separately.
Process drags the team score down
Coach process and compliance first: tagging and priority errors also distort backlog reporting.
Put QA next to CSAT on the same page, using the CSAT calculator and the CSAT method I use to keep the customer view consistent. A monthly report needs five lines: team score, score by criterion, auto-fail count, calibration spread and the trend over three months.
The QA dashboard I send every month
This is that report as one table, with the example team's scores.
| Metric | Target | This month | Last month | Change |
|---|---|---|---|---|
| Team QA score | 85% | 81% | 80% | +1 pt |
| Resolution and accuracy | 90% | 84% | 83% | +1 pt |
| Communication | 90% | 89% | 90% | −1 pt |
| Process and compliance | 80% | 68% | 63% | +5 pts |
| Tone and empathy | 85% | 81% | 79% | +2 pts |
| Auto-fails | 0 | 1 | 3 | −2 |
| Calibration spread | 5 pts or less | 6 pts | 9 pts | −3 pts |
Process and compliance is still the lowest row even after the biggest gain, so it stays the coaching focus. If the calibration spread is above 5 points, I fix that before I trust any other row.
What to do this week
Write your four criteria and weights on one page. Then ask two leads to score the same five tickets alone and compare. The disagreements will tell you what to clarify before you launch.
Frequently asked questions
What is a good QA score in customer support?
There is no universal benchmark, because every team's scorecard is different. I set the first target a few points above the baseline from the first two months of calibrated reviews.
Who should do QA reviews?
Team leads at first, then trained senior agents. Whoever reviews must take part in calibration, or their scores do not count.
Should QA scores affect pay or performance ratings?
Not in the first months. Use them for coaching until calibration is stable and agents trust the scorecard.
How long should one QA review take?
About 10 to 15 minutes for an email ticket on a four-criterion scorecard. Longer usually means the scorecard has too many lines.
Sources
- Customer service quality assurance: the ultimate guide, Zendesk
- Cohen's kappa, Wikipedia
Run your own numbers
Free, no sign-up, with the formula and an Excel version.
Read next
How to Reduce Handle Time Without Hurting CSAT
How to reduce average handle time without hurting CSAT: find the searching and waiting inside each ticket, cut it with five levers and set guardrails.
Read the article
Go further with the ebook
Head of Support
The blueprint to build, staff and lead a 50-agent support department, with the dashboards I used.
$39.99 · 233 pages · View the book