Data layer interaction design¶

UX for the two Data-layer apps — Datasets (Catalog + Model tabs) · Data Insights. Goal: a data engineer, a data BA, and a business user each reach their task with zero training. Phase 2–3 surfaces (quality, metrics, AI dashboards) are sketched, not specced.

Companion docs: Data layer plan · Implementation design · Design system

Intuitive = four rules¶

Reuse the mental models users already have. Every new surface reuses openKMS patterns — EntityViewShell, rail layouts (ManagerNavRail / ExplorerNavRail), <Dialog>, <MetricGrid>, <EmptyState>, useConfirm. No new interaction language.
Progressive disclosure. Catalog → detail → action. A list never shows columns/config that only a detail page needs; a detail page never shows a form until the user asks to edit.
The picture is the primary artifact. The ER map is drawn, not tabulated. Tables are the fallback, not the default.
The empty state is the next action. Every new surface's empty state is a single button that does the obvious thing: Upload data / Create relationship / Ask a question.

Layer grouping in the App Rail¶

Three visual groups, separated by a subtle divider, collapsible per group:

 ┌── APPLICATION ───
 │  … (Agents, Documents, …)
 ├── ONTOLOGY ──────
 │  🕸 Ontology Manager
 │  🧭 Object Explorer
 │  ⌘ Function Editor
 ├── DATA ──────────
 │  🗂 Datasets (Catalog · Model)
 │  🔎 Data Insights

Collapsed (56 px rail): icons only; a thin divider between groups; the active icon carries the group's accent.
App Launcher (full-width): three titled sections (Application / Ontology / Data) — this is the primary discovery surface, so the section titles are the first thing a new user reads.
Console stays separate (shared platform) — it is not a fourth layer, so it stays where it is today.

Why it's intuitive: users already think "I need data → data section; I need business meaning → ontology section." The grouping names the thing, instead of a flat 15-icon list.

Datasets — catalog, detail, upload¶

Landing (list / map)¶

A catalog, not a database admin table. A view toggle switches the same datasets between:

List — search + filters (domain, tag, sensitivity, data source — chips, not a form), server-side pagination (Pagination, default 25/page); each dataset shows its display name, domain tag, sensitivity badge, row count, freshness, and a mini quality dot.
Map (ER) — datasets as nodes and relationships as directed FK edges (many → one); click a node → open the dataset. The shape of the model lives here, next to the datasets it connects.
Primary action (always visible): 「Upload data」 — the one thing a first-time user should try.

Detail (reuses `EntityViewShell`)¶

Tabs, in reading order:

Tab	What the user sees	First impression
Overview	Read-only summary — `MetricGrid` (columns, rows), the dataset's catalog facts (source, name, description, domain, tags, business definition, sensitivity) and who depends on it (object types / link types that use it as a backing source, each linked). One action: Edit in Settings. It never saves.	"what is this dataset, and who uses it"
Data	paged table preview (TanStack: server sort + inline cell editing when row editing is on) with a filter bar: one `column · operator · value` per condition, AND-ed, type-aware operators, the row count reflecting the filters, and the view reproducible from the URL (design)	"what's actually in it"
Columns	column list + type + ontology-type mapping + description (editable); schema history	"what's in it and what it means"
Relationships	the FK edges this dataset takes part in (`from → to`, column pairs, INNER/LEFT)	"what does it connect to"
Quality	declarative rules + run history	"can I trust it"
Settings	the single edit surface: identity + catalog fields, row-editing toggle, keys (primary + unique), AI context, and the delete zone — one Save in the header	"what can I change about it"
Sharing	existing `ResourceSharePanel`	"who else can see it"

Read vs write is the split: Overview reads, Settings writes (both used to be forms with their own Save, over the same PUT /api/datasets/{id}). The Settings page follows the Model Settings shape: FormField everywhere, one concern per panel, one Save in the header, the delete zone last.

Add tables from a data source¶

The non-file path (the data engineer's most frequent action):

In the Datasets landing, 「Add tables」 → pick a data source → the backend lists its tables (GET …/from-source/{id}).
Select tables (multi-select); each shows schema.table and a preview affordance.
Register → creates Dataset rows (one per table), then land on the first dataset's detail.

Why it's intuitive: it mirrors "connect a folder of files" — pick a source, check tables, done. No schema knowledge required.

Upload flow — a three-step wizard¶

Complexity is hidden behind a wizard, one question per screen:

Drop / choose file — a large drop zone (reuses the document-upload pattern). After choosing, the file uploads with a progress bar. One field: the target data source (pre-filled with the default data-layer source).
Preview & confirm — the only screen that needs care:
Inferred table shown as a grid: column name, detected type (editable via a type dropdown per column), and the first 5 sample rows.
Editable: table name, schema, header row on/off, overwrite vs append.
A "looks right?" footer: 「Load」 / 「Back」.
Loading → done — a job runs (status pill); on completion, a success card with a single button: 「View dataset」.

Why it's intuitive: the user only ever answers one question at a time (which file → is the inference right → done), and the preview screen makes the machine's guess visible and correctable instead of a black box.

Model tab — semantic models¶

The Model tab lists semantic models — named scopes over the shared datasets, fields, and relationships (which live under Catalog). Opening a model shows its dataset scope, its metrics (primary) and its fields. Function spec: semantic model spec · definitions/surfaces: data layer.

List → detail — New model opens a short wizard (details → dataset scope → metrics); then open it. An Import (per-model document) lives in the header.
Metrics — the model's aggregate expressions (Ossie expression.dialects), authored by a data engineer with AI-assist; run read-only (timeout + row cap).
Playground — reachable from the sidebar nav: pick a metric, then group-by fields from a type-ahead menu (a field is a dimension; time dimensions carry dimension.is_time and use date_trunc with a query-time granularity) → a results table + the generated SQL + the run's context (ad-hoc, not saved). See Playground design.
The model page uses the same entity shell as the dataset page — identity + navItems in the sidebar, one section in the outlet. The sidebar identity is deliberately thin: the name (the back link already says which list you came from) plus the model-wide document action; the dataset scope is not repeated there — long scopes and long names would turn it into an unreadable block (entity-view__meta holds at most one short fact, clamped to two lines). The scope is the Datasets section's job:
Metrics — model-owned; authored here.
Datasets — the model's scope. Each dataset shows the catalog facts it references (source, description, keys, AI context — read-only, with Open in Catalog) above its fields, which the spec nests under a dataset; the fields themselves are model-owned (add/edit/delete per dataset, searchable across the scope).
Relationships — a read-only view of the FK edges among the scope (from → to, column pairs, INNER/LEFT, AI context). Relationships are catalog-owned and shared by every model, so they are edited in the Catalog (link on each row).
Playground — the model's query builder: metric + group-by fields + filters + order + limit, answering with a table, the generated SQL and the run's context (rows, limit reached, duration, null share per column, and why it is empty). The field picker is a type-ahead menu (grouped, each field with its label and data type; ↑/↓/Enter/Backspace/Esc) with selections mirrored as chips. Read-only and stateless (saved answers belong to Data Insights); a failed run is classified (definition / data / request) and explained in the answer pane.
Settings — model identity/scope, AI context, and delete. The model's Ossie document is a read action in the sidebar identity block (View YAML: a right-docked panel with a collapsible tree, the raw YAML and Copy) — one click from every section, and no header row spent on it. Downloads are gone; the export API stays for the skill/agent.
The split is owned vs referenced: the model owns its fields and metrics (edited in the model), while datasets and relationships live in the Catalog and are only referenced here — their ai_context is edited once, in the Catalog, so two models can never disagree about the same dataset.
Fields — the model's semantic fields (column ref / expression + Ossie dimension {is_time} + ai_context), added explicitly (a field marks a time dimension or adds AI context); the same column can carry a different dimension/ai_context per model.
Document — the model's Ossie document is viewed from the shell header (View YAML: tree ⇄ raw YAML + Copy), not downloaded; Import stays on the model list. The export API is the agent/skill path.

Why it's intuitive¶

Structure is shared (Catalog); a model is just a named scope — so the same table can serve several models without re-declaring its columns or FKs.
A metric is an expression, a dimension is a field role, and grouping is a query — matching how people ask (“revenue by month”).
Relationships are the join graph, so “can I write a cross-dataset metric?” is answered by the Catalog map.

Data Insights — ask, then see¶

The entry point is a question, not a chart builder. Mirrors the Agents surface users already know.

Shipped (P0): a report binds to a dataset or a semantic model, carries blocks (each with the query it came from), and Run re-executes every block through that source's engine — the narrative, insights and the block queries themselves are authored by the agent or a person, and the app renders and re-runs them (design). Next: visual charts (the block already carries columns/rows) and the question-first streamed dashboard below.

Main screen¶

Top: a single prompt box — "Ask your data a question" — with example chips underneath ("why did this stock underperform its industry?", "show monthly turnover").
Answer (streamed): an analysis narrative, then a dashboard grid of charts. Each chart has an 「Insight」badge that expands the AI-authored note: what the chart shows, anomalies, possible causes, suggested follow-up.
Provenance always visible: every chart/number carries its source dataset + metric, so the answer is traceable.

Saved reports¶

A left column (or a "Reports" tab) lists saved dashboards — named, dated, re-openable. A report is a first-class artifact persisted in the Data Insights app (decision #9). Export reuses the Reveal + Plotly static-HTML path.

Who uses which part¶

User	Path
Business	type question → read narrative + charts → optionally open a saved report
BA	ask exploratory questions, then save the useful ones as reports
Data engineer	rarely here; they live in Datasets

Why it's intuitive: business users never see a query editor or a schema — they ask in words and read a sourced answer, exactly like the KB Q&A they already use.

Mobile (≤ `$bp-md-min`)¶

Layer grouping: App Rail hidden; App Launcher shows the three sections full-width (already the phone pattern).
Datasets detail: tabs collapse to a top scroll + drawer (same as ontology second-rail drawer).
Model ER: canvas goes full-screen with a bottom-sheet dataset picker; join dialog is a bottom sheet.
Data Insights: chat-primary (same phone IA as Agents).

Intuitiveness checklist (applied to every screen)¶

Before shipping any Data-layer screen, it must pass:

In ≤3 seconds, can a new user name what this screen is and what the next action is?
Is the primary action a single visible button (not a menu)?
Does the screen reuse a pattern the user has already seen elsewhere in openKMS?
Is complexity behind a wizard or dialog, one question at a time?
Does every chart/table/number show its source?

Build order for the UX¶

Phase 0: layer grouping in App Rail + Launcher; Datasets moved out (existing list/detail reused, re-labeled).
Phase 1a: Datasets — catalog fields + Data tab + upload wizard (steps 1–2–3).
Phase 1b: Model tab — ER canvas (read) + join dialog (list-select) + Relationships tab.
Phase 1c: ~~Datasets — Lineage tab~~ removed: read-time "lineage" could only name the backing data source (nothing records how rows arrived) and repeated the Usage + Relationships tabs. Real lineage needs ingest/pipeline provenance first.
Phase 2: Model tab — Quality + metrics + queries editors (sidebar forms, not canvases).
Phase 3: Data Insights — prompt → dashboard grid → Insight badges → saved reports.