Deploy your first support-triage agent, human in the loop

Intercom’s Fin charges $0.99 for every ticket it resolves. A resolution includes a customer who simply doesn’t reply after the answer. Teams that read the transcripts found some of those “assumed resolutions” were customers who gave up and emailed someone instead.

The gap between what the dashboard calls solved and what a person would call solved is where most support-agent projects go wrong.

The rule that keeps them on track: the agent owns the first response, and a person owns the close. Get that right and a triage agent is one of the few agent setups that reliably works in 2026.

1. Pick one category

Choose one high-volume, low-stakes kind of ticket. Order status, password resets or “where is my invoice”. Leave refunds and cancellations alone.

An agent’s errors multiply with the range it handles. One category gives you a clean baseline, a small set of content to ground it in, and a contained mess when it slips. You can widen later. A burned customer stays burned.

2. Ground it, then read what it pulls

Connect the agent to your help centre, your macros and a sample of solved tickets in that category. Ask it the ten most common questions. Read which sources it pulled.

Nearly every wrong answer traces back to a gap or contradiction in the source content. So fixing the documents fixes the agent. It’s the cheapest improvement you’ll make.

3. Define “resolved” before your vendor does

Write one sentence saying what counts as resolved for your team. Put it next to the vendor’s billing definition.

Intercom cites 67 percent resolution across more than 40 million conversations. Its own case studies show 42 to 50 percent. Forecast on the audited figure.

Per-resolution pricing has one thing going for it. You only pay for successes. Per-conversation pricing bills the failures too.

4. Put a person between the agent and the customer

For the first two to four weeks, every draft goes to a human queue. Log whether each one was approved, edited or rejected.

That log is your test set. It shows true accuracy on your tickets, in your voice, before any customer sees an unchecked answer.

Set a bar beforehand. One in ten edited is fair for a low-stakes category. When the edit rate falls below it, let the agent send directly and keep reviewing a 10 percent sample.

This system prompt enforces the split. Change the brackets and keep the structure.

You are a support triage assistant for [COMPANY]. You handle ONLY
[CATEGORY] questions.

Rules:
1. Answer only from the provided help articles and ticket macros.
   If the answer is not in them, say so and escalate. Never guess.
2. You may look up order/account status via the [TOOL] tool.
   You may NOT issue refunds, change account state, or promise
   timelines not stated in the articles.
3. If the customer is angry, mentions legal action, churn, or a
   payment dispute, escalate immediately with the tag [ESC-HUMAN]
   and a one-line summary.
4. End every reply by asking if that resolved the issue. If the
   customer says no twice, escalate.
5. Output format: {category, confidence: high/medium/low,
   draft_reply, escalate: yes/no, reason}.
   Anything below high confidence routes to human review.

5. Build the exits before launch

You need at least three:

Customers forgive a bot that hands off quickly. They don’t forgive one that goes round in circles.

6. Watch three numbers

Track these weekly for the first month:

They tell you when to widen the category, when to loosen the gate and when something upstream has broken.

How does it go wrong?

The most common failure is skipping step three. At renewal you find you paid for hundreds of “resolutions” that were customers walking away. Reading 50 random resolved tickets a month takes half an hour.

The second is a prompt that never says what the agent may not do. An agent that can look up an order will sooner or later try to change one.

The third is rolling out to the whole inbox after a good pilot. Each new category puts accuracy back to unknown. Each one deserves its own two gated weeks.

Once your plans include refunds, plan changes or anything else involving money, it’s an engineering project. You need permissions, spending limits and audit logs. At the enterprise end, vendors like Decagon price to match, from around $95,000 a year with weeks of deployment. Before you extend that trust, test the agent properly.

FAQ

How do I set up an AI support agent for the first time? Start with one high-volume, low-stakes ticket category. Connect the agent to your help centre and solved tickets for that category only. Send every drafted reply through human review for the first weeks. Allow direct sending only once the human edit rate drops below a bar you set in advance. Add categories one at a time, each with its own review period.

What resolution rate should I expect from an AI support agent? Plan around audited rates of about 40 to 50 percent for a well-grounded agent on suitable ticket types. That matches Intercom’s published case-study range of 42 to 50 percent for Fin. Vendor headline claims run higher (Intercom cites 67 percent across 40M+ conversations), partly because billing definitions count customers who just stop replying. Check transcripts monthly and forecast costs on your own measured rate.

Should an AI agent be allowed to issue refunds? Not in a first setup. Refunds and account changes carry money and trust risk that an unproven agent shouldn’t own. They also need permissions, spending limits and audit systems beyond a triage setup. Keep money-touching actions behind human approval until the agent has months of measured accuracy and a tested evaluation and guardrail layer.