Knowledge bases

Index documents into a knowledge base, generate FAQs and chunks, run hybrid semantic search, and proxy questions to a separate QA Agent service. Access is controlled by knowledge_bases:read / knowledge_bases:write (and data scopes when enforced).

Feature	Status	Description
SPA i18n	✅	List + detail UI use knowledgeBase namespace (frontend/src/i18n/locales/{en,zh-CN}/knowledgeBase.json)
KB management	✅	CRUD via /api/knowledge-bases; KnowledgeBaseList with create/edit/delete; resource ACL per KB (knowledge_base); create bootstraps owner ACL; mutations require write on the KB when ACL applies; delete requires manage
KB sharing	✅	Settings → Sharing sub-tab (ResourceSharePanel); GET/PUT /api/resource-acl/knowledge_base/{id}. New KBs: owner-only ACL bootstrap (Others denied). Pre-existing KBs (before migration h8i9j0k1l2m3): seeded Others r/w/m to preserve open-by-default.
KB documents	✅	Add/remove documents to KB (join table); Documents tab paginated (GET …/documents?offset=&limit=); Add Documents dialog: channel tree + searchable document picker with pagination
KB wiki spaces	✅	Dedicated Wiki spaces tab on KB detail (same pattern as Documents): link spaces (GET/POST/DELETE …/wiki-spaces); per-row Index wiki pages queues run_kb_wiki_space_index (replaces that space’s wiki chunks only; one page → one chunk when ≤ 8000 chars); full kb-index also uses page-as-chunk rules for wiki pages (not KB chunk_config)
FAQs	✅	Manual create/edit/delete FAQ pairs; LLM-based FAQ generation from documents; FAQ list shows source document; paginated list (offset, limit); Edit FAQ modal with key-value form for document metadata (from KB metadata_keys; channel label_config/extraction_schema for object_type/list[object_type])
FAQ generation	✅	Two-step: POST /faqs/generate returns preview; user reviews, removes unqualified; POST /faqs/batch saves selected; configurable prompt in KB settings and modal; when multiple documents selected, generates one-by-one with progress in dialog
Chunks	✅	Document or wiki-page chunks stored with pgvector embeddings; configurable chunking strategy (fixed_size, markdown_header, paragraph); default chunk_size 8000 characters (UI + kb-index fallback). markdown_header / paragraph split by structure first, then subdivide any section longer than chunk_size (metadata may include split_part / split_parts); paginated list (offset, limit); GET single chunk by id for deep links; Edit Chunk modal with content and document metadata (same key-value form as FAQ). Editing content clears the chunk embedding until the next index job recomputes vectors
Semantic search	✅	POST /api/knowledge-bases/{id}/search using pgvector cosine distance over chunks (document-backed and wiki-page-backed) and FAQs; wiki-sourced chunk hits include wiki_page_id and wiki_space_id for linking; with a QA agent URL and no filters, hybrid recall (BM25 + dense + RRF) loads a small dense candidate pool (max(2×top_k, rerank_recall_top_k) via force_dense) then BM25 re-scores that pool for RRF (no full-corpus chunk/FAQ crawl). search_type chunks | faqs | all chooses scope; metadata filters apply to stored doc_metadata on chunks (wiki pages can populate this when metadata_keys match page metadata). include_historical_documents (default false) includes chunks/FAQs tied to superseded or out-of-window documents; force_dense forces the dense-only path (skips hybrid). Search tab: All/Chunks/FAQs toggles, one collapsible Search options panel (max results, dense-only, and metadata filters when metadata_keys is configured), Change vs last search after a rerun, and optional Retrieval panel per hit (chunk_index, retrieval_mode, retrieval_debug). Returns 503 with install instructions if pgvector missing
QA proxy	✅	POST /api/knowledge-bases/{id}/ask proxies to configurable agent service URL
KB settings	✅	Settings tab with General (agent URL, embedding model, chunking, FAQ prompt, metadata keys, index job) and Sharing sub-tabs. Save on General only. On wide screens General uses a two-column layout (connection & chunking left, FAQ prompt and metadata keys right)
KB indexing (CLI)	✅	openkms-cli pipeline run --pipeline-name kb-index --knowledge-base-id <id> [--wiki-space-id <id>] --api-url <backend> — full run chunks linked documents (KB chunk_config) and linked wiki pages (one page per chunk when ≤ 8000 chars; oversized pages split at markdown headers). --wiki-space-id re-indexes only that linked space (deletes its prior wiki chunks first). Embeddings via GET /internal-api/models/kb-embedding-credentials. See Indexing with openkms-cli below.
KB indexing (job run)	✅	run_kb_index (full KB) and run_kb_wiki_space_index (one linked wiki space). Settings tab: Queue indexing run → POST …/index-job. Wiki spaces tab: row action → POST …/wiki-spaces/{id}/index-job. Same Job runs listing as document parse runs
QA Agent service	✅	Separate FastAPI + LangGraph project (qa-agent/); RAG via backend search API; hybrid recall then optional cross-encoder rerank (POST …/v1/rerank; off by default — set OPENKMS_RERANK_ENABLED=true when your gateway exposes /v1/rerank, or set OPENKMS_RERANK_BASE_URL to a host that does). Langfuse (optional): requires LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, and LANGFUSE_BASE_URL — without base URL, tracing is off. With LANGFUSE_HEALTHCHECK (default true), qa-agent probes /api/public/health before callbacks; while down, all sessions skip Langfuse until LANGFUSE_HEALTHCHECK_RETRY_SECONDS elapses, then retries; OTLP exporter loggers are capped so failures stay non-fatal. LANGFUSE_TRACE_STREAMING defaults true; set false to trace only /ask. Bearer token is not copied into trace metadata. Skills include ontology and page_index
Q&A	✅	Q&A full-page chat uses persisted threads under GET/POST /api/knowledge-bases/{id}/agent-conversations (shared agent_conversations / agent_messages tables with surface=knowledge_base). Sending a message calls POST …/agent-conversations/{cid}/messages with stream: true (NDJSON includes persisted user then deltas/tools, then done with user / message ids and sources). POST …/ask/stream remains available for stateless proxies. done.sources: retrieval chunks unless Page Index (document_section) or successful ontology run_cypher_tool (then ontology rows: object-type list + Cypher summaries, not chunk snippets). Chunk sources may include chunk_index, retrieval_mode, retrieval_debug (same shape as search). Thumbs up / down on completed assistant replies (session-local); Save as FAQ appears after thumbs up and opens a review dialog (editable question/answer, optional POST …/faqs/polish to rewrite the answer with the judge model, then POST …/faqs to create; queue indexing to embed). Back (sidebar or mobile toolbar) clears the trace session id for the next visit. Layout: no extra main-column padding, global app header hidden while Q&A is open (wider thread + composer); only the message rail scrolls and the composer stays at the bottom of the viewport, with auto-scroll to the latest as messages stream. Phone (≤768): chat-primary + full-width Chats panel (same pattern as Agent Projects Sessions), not a stacked 38vh rail. Without an agent URL, the button is hidden. SPA: ontology source chips/cards use --color-ontology-* tokens in KnowledgeBaseDetail.scss. Detail chrome (tabs, section actions, Index/Q&A) uses ds-compact-label / --tap-min on phone; document picker stacks channel tree above the list.
FAQ assist (threads)	✅	Parallel persisted threads at /api/knowledge-bases/{id}/faq-assist-conversations/… with surface=kb_faq — same qa-agent contract as Q&A (optional drafting / notes). Requires knowledge_bases:read and KB visibility

Agent thread permissions (KB)

Routes under /api/knowledge-bases/{id}/agent-conversations/… and …/faq-assist-conversations/… require knowledge_bases:read (FastAPI dependency) and the same KB visibility checks as the rest of the KB API (get_kb_scoped / knowledge_base_visible when group scopes apply).

Indexing with openkms-cli (documents + linked wiki spaces)

1. In the app: link documents and/or wiki spaces to the knowledge base.
2. Optional — from the app: open the KB Settings tab and use Queue indexing run to enqueue run_kb_index on the worker (requires a saved embedding model). Track progress on Job runs (same listing as document parse runs).
3. Install CLI extras: cd openkms-cli && pip install -e ".[kb]".
4. Configure openkms-cli/.env: at minimum OPENKMS_API_URL (e.g. http://127.0.0.1:8102), OPENKMS_AUTH_MODE (local or oidc), and credentials (OPENKMS_CLI_BASIC_USER / OPENKMS_CLI_BASIC_PASSWORD for local, or OIDC client fields per openkms_cli/settings.py). Embeddings use GET /internal-api/models/kb-embedding-credentials with that auth (same pattern as VLM document-parse-defaults); optional OPENKMS_EMBEDDING_MODEL_* overrides base URL, model name, or API key.
5. Run:

openkms-cli pipeline run --pipeline-name kb-index --knowledge-base-id <KB_UUID> --api-url http://127.0.0.1:8102

--api-url defaults from OPENKMS_API_URL if omitted. The job replaces all chunks for that KB and re-embeds FAQs; wiki pages are included only for spaces linked to the KB.