A business chatbot should have a clear job: help someone find information, prepare a response or complete an authorised task. A fluent conversation is not enough to establish that it understands a policy or can safely change a customer’s record.
Begin with the expected users, the information they need and the actions the system may take. That scope determines the suitable technology, integrations and evaluation.
Compare rule-based and model-based approaches
A rule-based bot follows a designed flow or selects from approved responses. A language-model-based assistant can interpret more varied phrasing and generate an answer. A hybrid can use a model for interpretation while keeping important actions within a defined workflow.
| Approach | Useful characteristic | What still needs testing |
|---|---|---|
| Designed flow | Predictable steps and explicit choices | Missing paths, unclear wording and recovery |
| Search with approved answers | Responses tied to controlled material | Retrieval quality and source freshness |
| Language model with retrieval | Flexible phrasing grounded in selected sources | Unsupported claims, omitted detail and access boundaries |
| Assistant with business actions | Can connect a conversation to an operational task | Authentication, authorisation and transaction validation |
A more flexible interface introduces different failure modes. Do not choose a model solely because it produces more natural wording in a demonstration.
Separate knowledge from permissions
Decide which documents are authoritative and who maintains them. Remove contradictory versions and define how corrections become available to the assistant.
With retrieval-augmented generation, the system selects relevant material and supplies it to a model. Poor retrieval, outdated sources or misleading context can still produce a wrong answer. Asking the model to cite sources does not prove that those sources support its claim.
Apply access controls before information reaches an unauthorised user. A prompt telling the assistant not to reveal confidential material is not a substitute for permissions in retrieval and application logic. Read RAG for companies for the wider design considerations.
Keep actions within a controlled workflow
Looking up an order, changing an address and issuing a refund are different permissions. Authenticate the user where necessary, validate inputs and restrict the actions available to the assistant.
Treat text in retrieved documents, messages and external pages as data that may contain misleading instructions. It should not be able to grant the model new privileges. Use separate approval steps for consequential actions and record the actual outcome returned by the business system.
Design for retries. A lost response must not cause a second booking or payment. Provide a way to reconcile uncertain transaction status before repeating an action.
Build an evaluation set before tuning
Collect representative questions, including ambiguous wording, unsupported requests and cases requiring a human. Keep a separate set for evaluation so that tuning on the examples does not become the only evidence of quality.
Review factual accuracy, relevant sources, refusal behaviour, clarification and task completion. Test whether one user can obtain another user’s information. Include an unavailable integration and an outdated document.
A confidence threshold is useful only when its relationship to actual errors has been evaluated. Do not interpret a model’s confident tone as a calibrated estimate of correctness.
Make escalation easy to use
Customers and employees need a visible route to a person. Preserve useful context for the receiving team within the approved data scope. Explain what happens outside staffed hours and when an action cannot be completed.
Monitor repeat contacts, reopened cases and complaints. A high containment rate can reflect successful service or a frustrating inability to reach anyone. Sample conversations and subsequent outcomes to tell the difference.
Compare the full cost of ownership
Cost depends on the channels, knowledge sources, integrations, access model, volume and service requirements. Include content preparation, testing, review, model usage, monitoring and maintenance as well as the interface.
Compare ready-made applications with a custom implementation against the same requirements. Verify vendor prices and licence conditions for the configuration you actually need. There is no universal deployment price or payback period for a chatbot.
Questions before choosing a chatbot
Does connecting documents prevent hallucinations?
No. Retrieval can provide relevant evidence, but the system can still retrieve the wrong passage or produce an unsupported answer. Evaluate retrieval, source support and behaviour when evidence is missing.
When does a chatbot need a person?
When the task is outside its approved scope, the evidence is insufficient, an action requires approval or the user needs human support. The handover should be designed and tested as part of the workflow.
Choose the next step
A defined assistant can be discussed through Q-Dev Agentic. If the starting question concerns the company’s processes and use of AI, explore Q-Journey. The agreed scope determines the delivery and operating responsibilities.