# How to Build a Support QA Program From Scratch

> How to build a support QA program from scratch: a weighted scorecard, the QA score formula, a sampling plan, calibration and coaching that sticks.

Source: https://clarivoxx.com/articles/how-to-build-a-support-qa-program · Author: Abdessamad Ghanem · Clarivoxx · Customer Support · Updated 2026-06-04

Here is how I built a support QA program from scratch. A support QA program is a routine for reviewing a sample of tickets against a shared, weighted scorecard so you can measure quality and coach agents. It needs five parts: the scorecard, a fixed monthly sample, calibration between reviewers, coaching and a short report. The core formula is QA score = points earned ÷ points possible × 100.

The [free QA score calculator](https://clarivoxx.com/tools/qa-score-calculator) handles weights and criteria that do not apply. In Part 2 we cleared a growing backlog without hiring. This part makes sure the tickets your team now solves faster are also solved well.

## Why do you need QA if you already measure CSAT?

CSAT gives you the customer's view. QA gives you the inside view. A polite reply that solves the wrong problem can still get a good rating, and a correct answer can get a bad one because the policy behind it is unpopular. You need both.

Start small. Four criteria applied the same way by every reviewer beat a twenty-line form nobody trusts.

## What should a support QA scorecard include?

Describe an excellent ticket, then group it into four to six weighted criteria.

| Criterion | What reviewers check | Weight |
|---|---|---|
| Resolution and accuracy | Correct, complete fix; next question answered | 35 |
| Communication | Clear, right length, plain language | 25 |
| Process | Tags, priority, identity checks, escalation rules | 20 |
| Tone | Acknowledges impact, owns the issue | 20 |

Use **Yes, Partly or No** (full, half or zero points). Ten-point scales create more disagreement between reviewers. Add one **auto-fail** rule, such as sharing account data with an unverified contact.

## How do you calculate a QA score?

```
QA score (%) = Points earned ÷ Points possible × 100

Points earned   = sum of the points given on each criterion
                  (Yes = full weight, Partly = half, No = zero)
Points possible = sum of the weights of the criteria that apply to this ticket
Auto-fail       = score is 0 regardless of other criteria
```

Say a ticket gets full marks for resolution (35) and communication (25), and half marks for process (10, wrong tag) and tone (10). The score is 80 ÷ 100 = **80%**. If process does not apply to that ticket, remove its 20 points from both sides: 70 ÷ 80 = **87.5%**. Check your own scorecard in the [QA score calculator](https://clarivoxx.com/tools/qa-score-calculator).

## How many tickets should QA review each month?

Reviewing every ticket is not realistic. A fixed monthly sample per agent is fairer. Mix random tickets with a few targeted ones, such as low CSAT or escalations.

```
Reviewer hours per month = Agents × Reviews per agent × Minutes per review ÷ 60
```

For 12 agents with 8 reviews each at 12 minutes: 12 × 8 × 12 ÷ 60 = **19.2 hours a month**. Double the sample for new hires in their first 60 days. If that is too much for one lead, share it with a second lead or trained peers.

![How to Build a Support QA Program From Scratch — A photorealistic scene in a dark navy modern office with teal accent li](https://clarivoxx.com/article-images/customer-health-score-formula.jpg)

## How do you calibrate QA reviewers?

A score only means something if two reviewers would give the same ticket the same score. Every two weeks, have all reviewers score the same five tickets alone, then compare and discuss every gap. Write the decisions into the scorecard guide with examples.

I put anything more than 5 points from the group average on the discussion list. After two months, monthly calibration is enough. If you want a formal measure of agreement, Cohen's kappa corrects for the agreement two reviewers would reach by chance. For a small team, the 5-point spread is usually enough.

## How do you turn QA scores into coaching?

Scores without coaching feel like surveillance. Share every review with the agent, allow a simple way to dispute a score, and use a monthly one-to-one to pick **one criterion** to work on, with two or three real tickets. Ask agents to score one of their own tickets first. The gap between their score and yours is often the best conversation.

QA also supports earlier parts of this series. The resolution criterion helps [first contact resolution](https://clarivoxx.com/articles/how-to-improve-first-contact-resolution), and accurate tagging makes the splits from the [ticket backlog playbook](https://clarivoxx.com/articles/how-to-reduce-ticket-backlog) reliable.

![How to Build a Support QA Program From Scratch — A photorealistic scene of a team lead and a support agent in a one-to-o](https://clarivoxx.com/article-images/customer-success-manager-kpis.jpg)

## What I've seen in practice

When I first wrote a scorecard, I gave "tone" the biggest weight because partners complained about it. After a month of reviews, the real issue was wrong refund timelines, said very politely. We moved the weight to accuracy and the coaching conversations changed straight away.

On one team, our French-speaking and English-speaking reviewers scored "tone" quite differently. What felt warm in one language read as too casual in the other. Two calibration sessions with real tickets from both markets fixed most of it.

The trap I watch for now: QA rising while CSAT falls. When that happens, the scorecard measures the wrong things, and I go back to the criteria before I coach anyone.

## What should a monthly QA report show?

The overall average hides where coaching should go. Report each criterion separately.

Put QA next to CSAT on the same page, using the [CSAT calculator](https://clarivoxx.com/tools/csat-calculator) and [the CSAT method I use](https://clarivoxx.com/articles/how-to-calculate-csat) to keep the customer view consistent. A monthly report needs five lines: team score, score by criterion, auto-fail count, calibration spread and the trend over three months.

## The QA dashboard I send every month

This is that report as one table, with the example team's scores.

| Metric | Target | This month | Last month | Change |
|---|---|---|---|---|
| Team QA score | 85% | 81% | 80% | +1 pt |
| Resolution and accuracy | 90% | 84% | 83% | +1 pt |
| Communication | 90% | 89% | 90% | −1 pt |
| Process and compliance | 80% | 68% | 63% | +5 pts |
| Tone and empathy | 85% | 81% | 79% | +2 pts |
| Auto-fails | 0 | 1 | 3 | −2 |
| Calibration spread | 5 pts or less | 6 pts | 9 pts | −3 pts |

Process and compliance is still the lowest row even after the biggest gain, so it stays the coaching focus. If the calibration spread is above 5 points, I fix that before I trust any other row.

## What to do this week

Write your four criteria and weights on one page. Then ask two leads to score the same five tickets alone and compare. The disagreements will tell you what to clarify before you launch.

## Frequently asked questions

### What is a good QA score in customer support?

There is no universal benchmark, because every team's scorecard is different. I set the first target a few points above the baseline from the first two months of calibrated reviews.

### Who should do QA reviews?

Team leads at first, then trained senior agents. Whoever reviews must take part in calibration, or their scores do not count.

### Should QA scores affect pay or performance ratings?

Not in the first months. Use them for coaching until calibration is stable and agents trust the scorecard.

### How long should one QA review take?

About 10 to 15 minutes for an email ticket on a four-criterion scorecard. Longer usually means the scorecard has too many lines.

## Sources

- [Customer service quality assurance: the ultimate guide](https://www.zendesk.com/blog/quality-assurance/wem/customer-service-quality-assurance/), Zendesk
- [Cohen's kappa](https://en.wikipedia.org/wiki/Cohen%27s_kappa), Wikipedia