How to integrate an AI chatbot: knowledge, systems and the handoff to a person

The short version

• An AI chatbot integration has three connections: a knowledge source, the business systems it reads or changes, and a handoff to a person.
• Start read-only, because a wrong answer can be corrected and a wrong change often cannot.
• A chatbot should only see what the person talking to it is allowed to see, and that check belongs before retrieval, not after.
• The handoff to a human is part of the integration, not a fallback added after launch.
• A test set built from real past questions with known answers is the most useful thing a chatbot project produces.

What does AI chatbot integration involve?

AI chatbot integration means connecting a language model to three things: the content it answers from, the business systems it reads or changes, and a person who takes over when it cannot help. The chat window on your website is the visible part. It is also the smallest part.

Most chatbot platforms will put a widget on a page in an afternoon. What that widget knows, what it is allowed to touch and what happens when it is wrong are the questions that decide whether it helps or embarrasses you. Those questions are the integration.

Treat each connection as its own piece of work with its own risk. A chatbot that answers from a stale document gives wrong answers politely. A chatbot with write access to your order system can make wrong changes politely. A chatbot with no exit traps a customer who needed a person.

What are the three connections, and where should each start?

In what order should you integrate an AI chatbot?

1. Collect real questions from past tickets and emails, each with the answer a good member of staff would give.
2. Pick one knowledge source, make sure someone owns it, and connect the chatbot to that alone.
3. Build the human handoff before launch, and pass the conversation across so the customer does not repeat themselves.
4. Connect business systems read-only, using credentials scoped to the person in the conversation.
5. Launch to a small group, read the transcripts, and fix the content the chatbot keeps getting wrong.
6. Only then consider letting it take actions, one narrow action at a time, each with a confirmation step.

Why should a chatbot start read-only?

Because a wrong answer can be corrected and a wrong change often cannot. Starting read-only lets you watch how the chatbot behaves on real traffic while the worst it can do is say something incorrect.

Read-only does not mean low value. Order status, booking details, account balance, the delivery date on an invoice: plenty of what people ask a business is a lookup, and a chatbot that answers those correctly is already useful.

When you do add actions, add them one at a time and keep each narrow. 'Cancel this order' with a confirmation step is a defined action you can test. 'Manage my account' is not. Log every action with who asked, what the chatbot did and what the system returned.

How do you stop the chatbot seeing data it should not?

Resolve who the user is before the chatbot fetches anything, and fetch only what that user is entitled to see. The check belongs before retrieval. Filtering the answer afterwards still means the model received data the user could not access.

The tempting shortcut is a single service account with broad access, shared across every conversation. It works in the demo because the demo user is allowed to see everything. In production it means the chatbot can be talked into reading another customer's records, and prompt injection makes that a design problem rather than a hypothetical one.

The safer pattern is to pass the user's own identity through to each system, or to issue short-lived credentials limited to that user's records. Anonymous visitors get public content only. If a system cannot scope access per user, the chatbot should not be connected to it yet.

How should the handoff to a human work?

The handoff should be visible from the first message, should pass the full conversation to the person taking over, and should trigger whenever the customer asks, not only when the chatbot decides it is stuck. It is part of the integration, not a fallback added after launch.

Do not assume customers want AI service. In a Gartner survey of 5,728 customers conducted in December 2023, 64% said they would prefer companies did not use AI for customer service. A chatbot that hides the route to a person turns that preference into a complaint.

In practice that means a clear 'talk to a person' option, a ticket or live chat created with the transcript attached, and honest expectations about response times outside working hours. The customer should never have to retype what they already said.

Does a chatbot work better for customers or for your staff?

Start with staff if you are unsure. A staff-facing assistant keeps a person between the model and the customer, and the field evidence we can point to is for AI that assists support agents, not for bots that replace them.

Brynjolfsson, Li and Raymond studied a staggered rollout of a generative AI assistant across 5,179 customer support agents, measuring issues resolved per hour. Productivity rose, with the largest gains among less experienced agents and minimal effect on the most experienced. That is a study of a tool that helped people reply, not of a bot talking to customers on its own.

The integration work is nearly identical either way: the same knowledge source, the same scoped system access, the same tests. Starting with staff lets you fix the content and the permissions while a human still reviews every answer before a customer sees it.

How do you test a chatbot integration before launch?

Build a test set from real past questions with known good answers, run the chatbot against it, and score the results before every change goes live. Include questions it should refuse and questions that should go to a person.

The test set is the most useful thing the project will produce. It tells you whether a new model, a rewritten prompt or an updated help article made things better or worse, which you cannot judge by trying a few questions by hand.

Three kinds of question belong in it. Ordinary questions with a single correct answer. Questions the chatbot should decline, such as requests for another customer's data. Questions that need a person, such as a complaint or a refund outside policy. After launch, add every real failure from the transcripts to the set.

When is AI chatbot integration not worth doing?

It is not worth doing when question volume is low, when there is no maintained source of truth for the answers, or when nobody will own the content after launch. In those cases a better help page or a shared inbox is cheaper and more reliable.

Adoption surveys can make it feel urgent. The U.S. Census Bureau's business survey finds that firms with at least 20 employees are the biggest AI users, and a U.S. Chamber of Commerce report found that a majority of small businesses in all 50 states say they are embracing AI. Neither tells you whether a chatbot will help your customers.

The test is simple. If you cannot name the source the chatbot will answer from and the person who will keep it current, fix that first. A chatbot amplifies the state of your content. It does not repair it.