The four things people call deflection

Start by writing down what your tooling actually records. In most setups the deflection figure is derived from a single observable fact: somebody opened the assistant or the help centre, and no ticket appeared with their name on it afterwards. Everything else is inference layered on top of that fact.

The four things hiding underneath are these. First, a conversation that ended without a ticket, which is simply an absence of evidence. Second, a ticket genuinely avoided, meaning a person who would have written to you got what they needed and did not. Third, a question actually answered, correctly, which is the thing you were trying to buy. Fourth, a customer who gave up, closed the panel, and either went to a competitor or is now sitting on a problem you do not know about.

All four produce the same row in the log. That is the whole problem, and it does not get solved by choosing a better tool. It gets solved by deciding which of the four you are willing to claim credit for.

This is the weakest of the four and the one most reporting defaults to. The conversation ended. That is all you know. The person may have been satisfied, may have been browsing, may have been a bot, may have been testing whether you had a chat panel at all.

It is not useless information. Volume of conversations tells you something about demand, and a sudden change in it usually means something changed on the site rather than in the customers. But treating an ended conversation as a saved ticket is an accounting choice, not an observation, and if you never say out loud that you made that choice, it hardens into a fact within a quarter.

This one is real and it is the reason anybody buys this kind of thing. Somebody had a question that would have arrived in your queue, they got the answer somewhere else, and your queue stayed shorter.

The awkward part is that avoidance is a counterfactual. You cannot observe a ticket that did not happen. What you can observe is the shape of your queue over time against the volume and content of assistant conversations. If the assistant is answering large numbers of questions about your returns window, and tickets about your returns window fall, that is evidence. It is not proof, and it is worth saying so internally, because somebody will eventually ask and a claim you cannot defend costs more than a modest one you can.

The practical version of this measurement is boring: track the composition of your ticket queue by subject, not just its size. A queue that shrinks while its composition stays identical is probably seasonal. A queue that stays the same size but loses an entire category is deflection.

This is the only one you can verify directly, and hardly anyone looks at it, because it requires reading conversations rather than reading a chart.

Take a sample, or better, take everything from a quiet week. For each conversation, decide whether the person asked something specific, whether the reply addressed the thing they asked, and whether the reply was true. Three yeses is an answered question. This is slow, it does not scale, and it is the single most informative hour a support lead can spend in the first month after any automation goes live.

The reason it matters more than the aggregate number is that a correct answer and a plausible wrong answer are indistinguishable from the outside. Both end the conversation. Both count as deflection. Only one of them helps.

Here is the case that should worry you. Somebody arrived with a real question, got something unhelpful or evasive, decided the whole channel was a waste of time, and closed the tab. They did not write a ticket, because writing a ticket now felt like the second attempt at something that had already failed once.

In the log this looks exactly like the first case: a conversation, then nothing. It may look better than a successful one, because a person who gave up early produced a short clean transcript while a person who got real help asked three follow ups.

There is no automated signal that reliably separates abandonment from satisfaction. What there is, is a set of shapes that correlate with it. Short conversations that end immediately after a refusal or a non answer. Repeated rephrasings of the same question in one session, which is somebody trying to find the words that will work. Conversations that end on a question rather than on an answer. None of these is conclusive on its own and all of them are worth reading.

Since the log cannot separate them, put something in the conversation that can. The cheapest instrument is a route out that is always visible: a way to reach a person that does not require the visitor to argue with the assistant first. If somebody takes it, you have converted an invisible abandonment into a visible handover, which is a strictly better outcome even though it makes your deflection number worse.

That trade is worth naming explicitly to whoever reads the dashboard, because it will look like a regression. A team that adds a clear escalation route and sees deflection drop has not got worse at deflecting. It has stopped counting people who left in frustration as people who were helped.

The second instrument is content. Every question the assistant could not answer is a gap, and gaps are where abandonment lives. If you are logging refusals separately from answers, you have a list of the exact things people wanted that you did not have written down. That list is worth more than the deflection figure it reduces.

Report three things and stop reporting the composite. Report conversation volume, which tells you about demand. Report the number and subject of questions the system could not answer, which tells you what to write next. Report the change in ticket composition by subject, which is the closest honest proxy for avoidance you will get.

If somebody senior insists on a single number, give them the third one and explain what it is. A support lead who can say we get fewer questions about delivery timing than we did, and here is the category breakdown, is in a much stronger position than one holding a percentage nobody can define.