Metrics and terms
Asking again is the complaint nobody has to write
A visitor who rewords a question and sends it a second time has told you, without pressing anything and without filling in a form, that the first answer did not work. It is the most honest negative signal the system produces. Every message needed to spot one is recorded. Nothing counts them.
recorded, not on the insights page
No repeat rate is computed. Every message is recorded with its role, its text and its place in the thread, and a conversation's messages are retrievable in order through the interface, so the evidence exists in full. Nothing compares one question in a conversation with another, nothing counts a second attempt, and no figure derived from repeats appears on the insights page. The list there is headed "Unanswered questions", which groups refusals by the question text and does not know whether the same person asked twice.
What it means
A repeat question is a second attempt at something already asked inside the same conversation, usually reworded rather than retyped. It matters because of what it rules out. A refusal tells you the material was missing, which is useful but is the failure the system already admits to. A rating tells you how somebody felt, from the small minority who press buttons. A repeat tells you that an answer was given, was read, and was judged insufficient by the person who wanted it, in the moment, with no prompting and no incentive. That is a rare quality of evidence in this field. The catch is that recognising one requires deciding whether two differently worded questions mean the same thing, which is a reading judgement rather than a comparison, and it is why no figure for this exists in the product.
How it is actually calculated
What is recorded, and in what order
Every message in a conversation is stored with its role, its text and its position in the thread, and the messages of one conversation are retrievable in order through the interface. The raw material for this is therefore complete: if a visitor asked the same thing three times, all three are there, with the three answers between them.
The assistant itself sees a short window of the thread when it produces the next reply, so it has some awareness of what came before within a conversation. That is a property of how an answer is written, not a report, and nothing derived from it is counted or displayed.
The rule you have to write yourself
Counting exact text matches will find almost nothing. Nobody retypes a question character for character; they rephrase, shorten it, or drop the polite framing and ask the bare thing. A useful rule has to group by meaning, which is a person reading two lines and deciding.
Then there is the denominator. Repeats per conversation and repeats per message give different figures, and both are distorted by the fact that most conversations hold one exchange and cannot contain a repeat at all. If you compute anything here, compute it over conversations that had room for the failure to occur.
The window is yours too. This is not on the insights page, so there is no thirty day convention to inherit, and a week of conversations read carefully will teach you more than a month counted loosely.
Two repeats that mean opposite things
A repeat after a refusal means the fallback did not send them anywhere. The material was missing, the assistant said so, and instead of leaving or asking for a person the visitor tried again in different words. The fix is a document, and also possibly the wording of the fallback, because a refusal that offers nothing invites exactly this.
A repeat after an answer is the more serious one. Something was said, with citations, and the person on the other end did not accept it. That is either a wrong answer, an answer to a nearby question, or an answer so hedged it did not commit to anything. None of those appears in the unanswered list, which is precisely why this signal is worth the effort of looking for it.
One thing that can hide the second attempt
Answers are cached by meaning, and a rewording close enough to a question already answered for that assistant is served the previous answer rather than a fresh one. A visitor who rephrases the same question can therefore be handed back an answer identical to the one they were unhappy with.
Two consequences worth knowing. From a transcript this reads as an assistant being consistent, when the visitor experienced it as being stonewalled. And the second attempt spends nothing, so a run of frustrated rephrasings is invisible in every cost figure while being the clearest quality problem on the page.
How the number gets moved without anything improving
How the count comes out low without the repeats stopping
Match on exact text. The number lands near zero on any real traffic, the report looks excellent, and every genuine repeat is missed because people rephrase when the first attempt fails. This is the standard way this measurement is quietly neutered, and it is usually nobody's decision.
Average it across every conversation. Most conversations hold one exchange and cannot contain a repeat, so a real problem in the long conversations is divided into invisibility by the short ones. Restrict the denominator to conversations with room for the failure and the same data reads very differently.
The accidental version comes from the cache. A rephrased question that gets the identical answer back looks, in a transcript, like the assistant being reliable. Reading for repeats means reading what the visitor did next, not what the assistant said.
What to look at instead, or alongside
- Conversations holding three or more visitor messages, read in order, which is where every repeat lives by definition.
- The unanswered questions list, which catches the half of this that ended in a refusal and names the missing topic.
- Negative ratings read individually, since a visitor who gave up on rephrasing sometimes presses a button on the way out.
- Enquiries left immediately after an answer, which are repeats that escalated rather than repeats that were abandoned.
Questions
- Is a follow up question a repeat?
- No, and keeping them apart is most of the work. A follow up moves forward: they got the delivery time and now want the cutoff. A repeat goes back over the same ground in different words. If you cannot tell which one you are looking at, read the answer in between, because the answer is usually what decides it.
- Why is this not just counted for me?
- Because deciding that two differently worded questions mean the same thing is the judgement the whole metric rests on, and automating it would mean publishing a figure whose definition nobody could inspect. The messages are all recorded, so the counting is possible for anybody who wants to define the rule and own it.
- How many conversations should I read to get a feel for this?
- Ten of the longest ones a week. Length is the cheapest filter available and it concentrates the sample on conversations where somebody kept trying, which is where both the repeats and the reasons for them are. It takes about twenty minutes and it is the highest yield reading in the product.
Keep reading
- Conversation volume: what a conversation count can tell youHow many people opened the widget and said something. A useful denominator, a poor headline, and a number that moves with your marketing.
- Containment rate: why this product does not compute oneContainment counts conversations that ended without a person. A widget cannot see whether one did, so no containment figure exists here.
- First response time has no meaning without a queueThis metric measures how long somebody waited for a human to pick up. There is no queue in a widget, and no first response time is computed.
- Every metric and termWhat each one means, and which of them this product actually computes.
Try it on your own material
Upload a document or point it at your site, paste one line of HTML, then ask it something only your business could answer.