A useful company knowledge base depends on reliable information, clear ownership and a way to check whether people can find the answers they need. AI can support retrieval and answer preparation, but it cannot make incomplete or contradictory documents authoritative.
This guide covers the decisions involved in moving from scattered documents to a maintained search or question-answering service. The appropriate architecture depends on the work people need to do.
Start with the questions people ask
Collect representative questions from the people who will use the system. Include straightforward lookups, questions that require several sources, ambiguous requests and questions the system should decline to answer.
For each question, record the authoritative source, the expected answer or relevant passage, and who is allowed to see it. This becomes an evaluation set for comparing approaches. It also reveals where the real problem is missing documentation rather than search.
Map sources and ownership
Inventory the repositories that may contain useful information: document stores, wikis, approved communication archives, business systems and technical documentation. For each source, record:
- the owner and intended audience;
- document types, volume and update frequency;
- available export or API access;
- access permissions and retention requirements;
- known duplicates, contradictions and outdated material.
Do not assume that access to a repository authorises indexing every item. Private messages, personal records and confidential customer information need explicit inclusion rules. Information known only to experienced staff may need to be documented before technology can retrieve it.
Choose between search, RAG and a graph
| Approach | What it helps with | What needs attention |
|---|---|---|
| Keyword or full-text search | Exact terms, identifiers and familiar document names | Vocabulary differences and ranking |
| Semantic search | Finding related concepts expressed in different words | Model choice, relevance evaluation and permissions |
| RAG | Preparing an answer from retrieved material | Unsupported claims, citations, source freshness and refusal behaviour |
| Knowledge graph | Navigating explicit relationships between entities | Schema design, evidence for relationships and ongoing maintenance |
| Combined approach | Questions that benefit from both passages and relationships | Additional complexity and evidence that the combination improves results |
Document count alone is not a reliable selection rule. Test simpler options against the actual questions before adding graph extraction or answer generation. A graph can represent who owns a project or which document describes a process; it does not automatically make every relationship correct.
Prepare information for retrieval
Documents may need parsing, OCR, deduplication and metadata extraction. Keep a stable link to each source and enough context to identify its owner, version and access rules.
Chunking splits documents into passages that a retrieval system can use. Fixed-length chunks are easy to implement; section-based chunks can preserve headings and context. Test both against representative documents rather than assume one method always works best.
Embedding models should be evaluated on the languages and terminology in the source material. For Polish documents, test inflected forms, abbreviations and domain-specific names. Exact-match retrieval can remain valuable alongside embeddings, especially for product codes and contract references.
Design the interface around the task
A search interface should show why a result is relevant and make the original source easy to open. A generated answer should distinguish quoted information from synthesis and provide citations that actually support its claims.
For question answering, define what happens when sources conflict, are too old or do not contain an answer. A confident-looking response is not a substitute for evidence. Give users a way to flag errors and route those reports to someone responsible.
A graph explorer can help users inspect relationships, but it is useful only if those relationships are understandable and traceable. It need not be the main interface for every employee.
Evaluate before wider rollout
Use the question set created at the start and keep a separate set for later checks. Useful measures include:
- retrieval precision and recall on labelled examples;
- whether citations support the answer;
- whether unsupported questions are declined appropriately;
- whether restricted information stays restricted;
- task completion time, corrections and user feedback.
Agree acceptance criteria for the intended use. There is no universal accuracy or time-saving percentage that establishes readiness for every organisation. Compare the proposed system with existing search and include human review time in the comparison.
Plan deployment and maintenance
List every place data may travel: ingestion services, embedding models, language-model APIs, logs, backups and analytics. Self-hosting one component does not mean that every other component runs locally. Deployment location and applicable transfer rules need to be assessed for the actual architecture.
Assign responsibility for infrastructure, content, access reviews, retrieval quality and incident handling. Source changes and permission changes need to propagate to the index. Deletion and retention rules must cover derived stores as well as the original documents.
The budget should include preparation, integrations, evaluation, hosting, model usage, maintenance and staff time. The schedule depends on source quality, access to systems and the acceptance requirements; it should be scoped individually.
Where BeHive fits
BeHive is QA10’s open-source research engine, available under the MIT licence. It supports source-based research, claim assessment and exploration of relationships between entities, using a language model selected for the research setup.
pip install behiveThose research capabilities should be evaluated against the intended task. A company knowledge base also needs its own source integrations, permission model, interfaces and operational arrangements. This article does not establish that a ready-made enterprise knowledge-base deployment is included with the package.
FAQ
How much does an AI knowledge base cost?
The cost depends on sources, data preparation, permissions, interfaces, evaluation and ongoing operation. Estimate these components for the proposed scope. A package licence does not determine the total implementation or maintenance cost.
How long does implementation take?
There is no fixed duration. Source access, document quality, integration work and acceptance criteria determine the schedule. A limited pilot can help establish what is feasible before a wider deployment is planned.
Does it replace Confluence or Notion?
It may complement an existing authoring tool. Documents can remain in their source system while an additional retrieval layer helps people find them, provided integrations and permissions are designed for that arrangement.
How should sensitive data be handled?
Map the full data flow, restrict access according to the source permissions and assess model providers, logs, backups and retention. Neither self-hosting nor choosing an EU region establishes compliance on its own.
Who maintains the knowledge base?
Name owners for content, infrastructure, access controls and retrieval quality. The required staffing depends on the sources, update rate and service expectations; it should not be inferred from a standard headcount ratio.
Discuss a defined need
Describe one task people struggle to complete with the current knowledge sources. Contact QA10 to discuss the next step. An application developed after an audit belongs to Q-Journey’s second stage, including when it does not use AI.