Containment rate counts the customer who gave up as a success

A customer messages an order status bot on WhatsApp at 09:12: "where is order 48213". The bot reads the carrier feed and replies "Your order is in transit." At 09:13 the customer writes "it said that yesterday, when will it arrive". The bot classifies the turn as order_status again and sends the same sentence. The customer stops typing. After thirty minutes of silence the conversation closes, and on the dashboard it lands in the contained column: every turn answered, no handover requested. The next morning the same customer opens web chat and types "agent".

Containment measures what the bot did. It cannot tell a customer who was answered from one who gave up, so a bot tuned on it learns to deflect. The figure to put beside it is resolution: a conversation counts as resolved only when the same customer does not come back about the same thing, on any channel, within a fixed window. I have built an omni-channel chatbot across Messenger, WhatsApp, Telegram and web chat, and that return window is the measure I would wire in before any intent accuracy figure.

The customer closes the conversation, not the bot

Give every conversation an outcome state that the bot cannot set on its own. When the bot sends its last reply and the customer is silent for 30 minutes, the state becomes bot_closed with a closed_at timestamp, and that is provisional. A job moves bot_closed to resolved only once 48 hours have passed with no return. A return is a new conversation from the same customer key, classified to the same intent, carrying the same entity where the intent has one, and started inside that window.

The new conversation records returned_from with the old id, and the old one moves to returned, dated to the hour it closed, so yesterday's figure is restated rather than quietly left alone.

flowchart TD
  A[Open] --> B[Bot closed]
  A --> H[Handed over]
  B -->|no contact in 48h| R[Resolved]:::accent
  B -->|same intent in 48h| T[Returned]
  B -->|agent typed in 48h| T
  T --> N[New conversation]

The return check runs when a new conversation gets its first classified intent, not when it ends, because the point is to flip the earlier record as soon as the customer reappears.

// simplified: called once the new conversation has its first intent
async function linkReturn(c: Conversation): Promise<void> {
  const since = new Date(c.startedAt.getTime() - 48 * 3600 * 1000);
  const prior = await conversations.findOne({
    customerKey: c.customerKey,  // account id, or the channel handle
    intent: c.intent,            // order_status
    entity: c.entity ?? null,    // 48213, or null for intents without one
    state: 'bot_closed',
    closedAt: { $gte: since },
  });
  if (!prior) return;
  await conversations.update(prior.id, { state: 'returned', returnedBy: c.id });
  await conversations.update(c.id, { returnedFrom: prior.id });
}

A handover request needs its own rule, because "agent" carries no intent and no entity. Treat any handover request inside the window as a return of that customer's most recent bot_closed conversation, whatever its intent.

Run the 48213 case through it: the WhatsApp conversation closed at 09:43 as bot_closed; at 08:20 the next day the web chat turn "agent" arrives from the same account; the earlier conversation flips to returned 22 hours and 37 minutes after it closed. By noon the order_status row reads 94 per cent contained and 61 per cent resolved. That gap is not a classifier problem. The bot understood every turn; the carrier feed it read was 18 hours stale, and the customer knew it.

Where the window lies

The customer key is the weak joint. Each channel identifies a person differently, and an anonymous web chat cannot be matched to yesterday's WhatsApp thread, so the "agent" return above only links if the customer logged in or gave the order number. Mark every conversation whose customer could not be matched as unmatched, report that share beside resolution, and never fold it into resolved. When the unmatched share is larger than the gap between contained and resolved, the gap means nothing yet, and the fix is identification, not the bot.

Intents that recur for honest reasons need the entity in the key. A second order_status question about order 48377 the next day is a new conversation, not a return of 48213. For an intent with no entity, such as opening hours, any recurrence inside the window is a return, because the first answer should have settled it. The window itself is a trade-off: 48 hours catches the customer who sleeps on it and misses the one who waits for the weekend, and a longer window delays the figure by the same amount, so put the conversation's start date on the report and refuse to tune thresholds on the last two days.

Add the returned state and the linking job first, with the window as configuration. Then put resolved beside contained for every intent, with the unmatched share next to them. The intents where the two figures diverge are the ones to fix, and the fix is usually in the answer's data, not in the dialogue.