RevelirQA performs AutoQA to the standard of a human quality analyst. Revelir calibrates it with each customer until, when that customer's own QA analysts review a sample of evaluations, more than 95 out of 100 are judged correct. Scoring 100% of conversations is table stakes that most auto QA and automated quality assurance tools already claim. The method itself has three steps. Slice QA scores by contact reason and by individual scorecard metric. Find the colleague who consistently passes the metric another agent keeps failing on that same issue. Pair them for short, structured sessions built around real tickets. Revelir tags every ticket with a contact reason and retrieves the SOPs for that reason before scoring, which is the data this method depends on.
TL;DR
- RevelirQA performs AutoQA to the standard of a human quality analyst. It is calibrated with each customer to over 95% accuracy, as validated by that customer's QA analysts on a sample of evaluations. Scoring 100% of conversations is table stakes that most tools already claim.
- Overall QA scores hide specialisation. Pair agents on contact reason plus failed metric, not on "top performer."
- The best peer coach is often a near peer who passes that specific metric, not the highest average scorer.
- Most agents are strong on some contact reasons and weak on others, so pairs can run in both directions.
- Judge each pairing by the learner's later scores on that contact reason.
About the Author: Revelir AI builds RevelirQA, AI customer service QA software that Xendit and Tiket.com run in production on thousands of tickets per week. RevelirQA scores every conversation against each customer's own SOPs and attaches a contact reason to every ticket.
What is peer coaching by contact reason?
Doing it by contact reason means the "specific task" is defined by the issue the customer raised. Examples include refund requests, account verification and booking changes.
- Example: an agent who keeps quoting the wrong refund amount on payment disputes pairs with a colleague who gets that metric right on payment disputes.
Why does pairing with the "top performer" usually fail?
Building on that definition, the common shortcut of pairing struggling agents with the team's highest scorer misreads what an overall score measures. An overall QA score works like a decathlon total. A decathlete can win on points while being average at hurdles, because strength in nine events absorbs one weak one. An agent's high average can hide the same kind of weak spot.
- Contact mix skews averages. An agent who mostly handles simple order-status tickets can outscore a colleague who takes the hard escalations.
- Stars become bottlenecks. One person coaching everyone stops handling tickets well themselves.
- Status gaps change the dynamic.
What data do you need before pairing agents by contact reason?
The pairing is only as good as the scores underneath it. If an agent is wrongly scored as an expert, they will pass the wrong habit on to a colleague. You need:
- A contact reason on every ticket, applied consistently rather than left to whatever agents pick at wrap-up.
- Metric-level results, not just a total. "Correct escalation path: fail" is coachable. "78/100" is not.
- Enough scored tickets per agent per reason. Small manual samples can leave an agent with only a handful of reviewed tickets on a given issue. A single bad ticket then swings the result.
- Checked accuracy.
- Evidence on each score, so the coach can show the exact exchange instead of arguing from memory.
RevelirQA covers these through ticket enrichment, custom metrics that can be configured per contact reason, and a full reasoning trace on every evaluation. Each failed metric shows the quoted agent text and the SOP passage it was checked against.
How do you build the pairing matrix step by step?
With trustworthy data in place, building the pairings becomes a filtering exercise rather than a judgment call.
- Choose three to five contact reasons that carry high volume or high risk.
- For each reason, list the scorecard metrics that fail most often.
- Pull each agent's pass rate on those metrics, within that reason only.
- Shortlist coaches who pass consistently over many tickets. Read a few reasoning traces to confirm the passes are genuine.
- Prefer near peers who work the same channel, language and shift as the learner.
- Limit each coach to one or two learners.
- Re-run the matrix monthly as contact mix shifts.
| Contact reason | Metric often failed | Learner | Peer coach | Why this coach |
|---|---|---|---|---|
| Refund request | Correct refund amount quoted | Agent A | Agent C | Passes consistently, same channel |
| Account verification | Required disclosure given | Agent C | Agent B | Strongest on disclosures, similar tenure |
| Booking change | Correct escalation path | Agent B | Agent A | Reliable escalations, same language queue |
Illustrative example. Note how each agent coaches in one row and learns in another.
How should a peer coaching session around AutoQA scores run?
- Start from two tickets: one the learner failed and one the coach passed, both on the same contact reason.
- Read the SOP passage together before discussing either ticket.
- Have the coach narrate their reasoning, including what they checked before replying.
- Agree on one behaviour to change, not five.
- Keep it out of performance reviews. Scores are evidence, not verdicts.
RevelirQA supports that last point. Agents can contest scores, QA leads can override grades, and the platform triggers no warnings, rankings or discipline. Decisions stay with the QA lead.
How do you know whether a pairing worked?
The final test brings the method back to the scores that started it. Track the learner's pass rate on the targeted metric, within that contact reason, over the following weeks. Compare it with similar agents who were not paired. Check that the coach's own scores hold steady, then rotate the pair once the gap closes. RevelirQA shows scores per agent over time. Through its MCP connection to Claude, a CX lead can ask questions such as "Which contact reason is growing fastest?" to decide where to aim the next round of pairings.
Frequently Asked Questions
How many scored tickets per contact reason are enough?
There is no universal number. You need enough that one ticket cannot flip an agent's result, which is why scoring every conversation helps.
Should team leads be the coaches?
Usually not.
What if nobody passes a metric on a contact reason?
Then you probably have a process or SOP problem rather than a skill gap. Fix the policy or knowledge base first.
Does automated quality assurance rank agents for you?
RevelirQA does not trigger warnings, rankings or discipline. The QA lead builds the pairings from the scores.
Can AI chatbot conversations be included?
Yes. RevelirQA scores AI and human conversations on the same scorecard. This shows which contact reasons the chatbot handles worse than your human agents do.
How accurate must the scores be?
Revelir calibrates with each customer until more than 95 out of 100 sampled evaluations are judged correct.
About Revelir AI
Revelir AI is a Singapore-headquartered company that builds RevelirQA, an AI quality assurance platform for customer service teams worldwide. RevelirQA scores human and AI conversations against each customer's own SOPs, which it retrieves through RAG, and attaches a full reasoning trace to every score. It enriches tickets with contact reason, sentiment and custom metrics, which turns QA results into a usable map for coaching. Xendit and Tiket.com run it in production, with scoring live in English and Indonesian.
If you want peer coaching built on scores your QA team already trusts, learn more or get in touch at https://www.revelir.ai/.