# Progress Log — Semantic Search & AI Chat # Branch: ralph/semantic-search # Started: 2026-02-15 ## Codebase Patterns - `@xenova/transformers` pipeline with `pooling: 'mean'` and `normalize: true` returns a Tensor; use `Array.from(output.data as Float32Array)` to extract the 384-d vector - Scripts live in `scripts/` and run via `npx tsx` (tsx is not a project dep, npx fetches it) - tsconfig `include` only covers `src/` — scripts are type-checked by tsx at runtime, not by `tsc --noEmit` - Project uses `"type": "module"` in package.json - Palette item IDs: `exp-{consultation.id}`, `skill-{skill.id}`, `proj-{investigation.id}`, `ach-{0-3}`, `edu-{0-3}`, `action-{0-3}` - `buildEmbeddingTexts()` in `src/lib/search.ts` returns `Array<{ id: string, text: string }>` with IDs matching PaletteItem IDs — use this for both embedding generation and chat context - `src/data/embeddings.json` is an array of `{ id: string, embedding: number[] }` — 42 items, 384-d vectors, IDs match PaletteItem IDs. Vite imports JSON natively. - `src/lib/embedding-model.ts` exports `initModel()`, `embedQuery(text)`, `isModelReady()` — check `isModelReady()` before calling `embedQuery()` - `initModel()` is called fire-and-forget in `App.tsx` on mount — model loads during boot/ECG/login phases - `src/lib/semantic-search.ts` exports `semanticSearch(queryEmbedding, embeddings, threshold?)` and `loadEmbeddings()` — embeddings are normalized so cosine similarity is dot(a,b)/(mag(a)*mag(b)) - CommandPalette uses `semanticResults` state + debounced `useEffect` for async semantic search, falling back to Fuse.js when `isModelReady()` returns false or on any error - `loadEmbeddings()` and `paletteMap` (Map) are precomputed via `useMemo` — no re-computation on each search - ChatWidget is mounted in DashboardLayout alongside CommandPalette and DetailPanel — z-index 90 (below command palette z-1000) - `prefersReducedMotion` pattern: read `window.matchMedia` at module level, use in framer-motion variants to skip animation - ChatWidget stores messages as `Array<{ role: 'user' | 'assistant', content: string }>` — same shape as LLM message format, ready for Gemini integration - ChatWidget `isOpen` state controls both panel visibility and button icon (MessageCircle ↔ X) — panel rendering handled by AnimatePresence - `src/lib/gemini.ts` exports `sendChatMessage(messages)` (async generator), `isGeminiAvailable()`, `parseItemIds(text)`, `stripItemsSuffix(text)` — ChatMessage type is `{ role: 'user' | 'assistant', content: string }` - Gemini API uses SSE streaming: POST to `:streamGenerateContent?alt=sse&key=KEY`, parse `data:` lines as JSON, extract `candidates[0].content.parts[0].text` - System prompt built from `buildEmbeddingTexts()` — instructs model to end responses with `[ITEMS: id1, id2, id3]` for portfolio item linking - `isGeminiAvailable()` checks `import.meta.env.VITE_GEMINI_API_KEY` — when missing, chat panel shows "unavailable" message but button remains visible - Assistant messages store item IDs as `` HTML comment suffix for US-010 to parse — `getDisplayText()` strips this before rendering - Conversation history capped at 10 messages (`MAX_HISTORY`), metadata stripped before sending to API - Icon/color mappings (`iconByType`, `iconColorStyles`) live in `src/lib/palette-icons.ts` — shared between CommandPalette and ChatWidget - ChatWidget accepts optional `onAction?: (action: PaletteAction) => void` prop — same pattern as CommandPalette's `onAction` - `DashboardLayout` passes `handlePaletteAction` to both CommandPalette and ChatWidget for unified action routing --- ## 2026-02-15 - US-001 - Installed `@xenova/transformers` (^2.17.2) - Created `scripts/generate-embeddings.ts` with main() that loads `Xenova/all-MiniLM-L6-v2` and embeds a test string - Added `"generate-embeddings"` npm script - Verified: outputs vector length 384 and exits cleanly - Typecheck passes - Files changed: `package.json`, `package-lock.json`, `scripts/generate-embeddings.ts` - **Learnings for future iterations:** - `pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2')` auto-downloads and caches the ONNX model (~23MB) - First run takes a few seconds for model download; subsequent runs are near-instant from cache - The pipeline's `pooling: 'mean'` and `normalize: true` options handle mean-pooling and L2 normalization in one step — no manual tensor manipulation needed - `output.data` is a `Float32Array`; wrap in `Array.from()` for a plain number array --- ## 2026-02-15 - US-002 - Added `buildEmbeddingTexts()` function to `src/lib/search.ts` - Imports all raw data files (consultations, skills, kpis, investigations, documents) - Generates natural-language paragraphs for each palette item type: - Consultations: role, org, duration, history narrative, examination bullets, coded entry descriptions - Skills: name, category, frequency, proficiency %, years of experience - Achievements: title, subtitle, full KPI explanation + story context + outcomes - Investigations: name, methodology, tech stack, results - Education: title, type, institution, duration, classification, research detail, notes (from documents.ts) - Quick Actions: title + subtitle - IDs match PaletteItem IDs (e.g. `exp-{id}`, `skill-{id}`, `ach-{i}`, `proj-{id}`, `edu-{i}`, `action-{i}`) - Typecheck and lint pass - Files changed: `src/lib/search.ts` - **Learnings for future iterations:** - Education items in `buildPaletteData()` are hardcoded arrays (not iterated from `documents`), with ids `edu-0` through `edu-3`. The mapping to `documents.ts` entries is: edu-0→doc-mary-seacole, edu-1→doc-mpharm, edu-2→doc-alevels, edu-3→doc-gphc - Achievement items are similarly hardcoded with ids `ach-0` through `ach-3`, each linked to a KPI id - Quick action items are `action-0` through `action-3` - `documents.ts` is imported but wasn't previously used in `search.ts` — now used for education embedding text --- ## 2026-02-15 - US-003 - Updated `scripts/generate-embeddings.ts` to import `buildEmbeddingTexts()` and generate full embeddings - Script embeds all 42 palette items sequentially using `Xenova/all-MiniLM-L6-v2` - Outputs `src/data/embeddings.json` as `Array<{ id: string, embedding: number[] }>` - Each embedding is a 384-dimensional float array - File is ~453KB (42 items × 384 floats with pretty-printed JSON) - `npm run generate-embeddings` regenerates the file successfully - Typecheck and lint pass - Files changed: `scripts/generate-embeddings.ts`, `src/data/embeddings.json` - **Learnings for future iterations:** - `import.meta.dirname` works in tsx/Node ESM scripts — use it instead of `__dirname` (which isn't available in ESM) - `@/` path alias works in `npx tsx` scripts because tsx resolves tsconfig paths automatically - The embeddings file is ~450KB with pretty-print; could be reduced with compact JSON but readability is preferred for now - Processing 42 items takes ~10-15 seconds on first run (model cached after first download) --- ## 2026-02-15 - US-004 - Created `src/lib/embedding-model.ts` with three exports: `initModel()`, `embedQuery()`, `isModelReady()` - Module-level `let extractor` pattern avoids React re-render issues - `initModel()` uses `loading` guard to prevent duplicate pipeline loads - `embedQuery()` uses same `pooling: 'mean'` and `normalize: true` as the build script - `initModel()` called fire-and-forget in `App.tsx` `useEffect([], [])` — runs during boot phase - Silent failure: try/catch swallows errors, `isModelReady()` stays false - Typecheck, lint, and build all pass - Files changed: `src/lib/embedding-model.ts` (new), `src/App.tsx` - **Learnings for future iterations:** - `FeatureExtractionPipeline` type is exported from `@xenova/transformers` and can be used for the module-level variable - The `loading` boolean guard prevents race conditions if `initModel()` is called multiple times (e.g., React strict mode double-mount) - `initModel()` is intentionally not awaited — it's fire-and-forget so it doesn't block the boot animation - Consumers should check `isModelReady()` before calling `embedQuery()` — it throws if model isn't loaded --- ## 2026-02-15 - US-005 - Created `src/lib/semantic-search.ts` with cosine similarity search and embeddings loader - `semanticSearch()` computes cosine similarity, filters by threshold (default 0.3), returns sorted by score descending - `loadEmbeddings()` imports `embeddings.json` via Vite's native JSON import and returns typed array - Typecheck and lint pass (0 new warnings) - Files changed: `src/lib/semantic-search.ts` (new) - **Learnings for future iterations:** - Vite handles JSON imports natively — `import data from '@/data/embeddings.json'` just works, no dynamic import needed - Since embeddings are already L2-normalized (from pipeline's `normalize: true`), cosine similarity simplifies to just the dot product. However, the full formula is kept for correctness in case non-normalized vectors are ever used - With only ~42 items and 384-d vectors, brute-force cosine similarity is fast enough — no need for approximate nearest neighbor libraries --- ## 2026-02-15 - US-006 - Integrated semantic search into CommandPalette with Fuse.js fallback - When `isModelReady()` is true: debounces query by 200ms, calls `embedQuery()`, runs `semanticSearch()` against preloaded embeddings, maps result IDs back to PaletteItems via O(1) Map lookup - When model is NOT ready: uses existing Fuse.js search (behavior preserved exactly) - Results maintain `groupBySection()` grouping and section ordering - Existing keyboard navigation, action routing, and UI unchanged - Semantic results state is cleared when palette opens/closes and when query is empty - Error handling: any failure in embedQuery/semanticSearch silently falls back to Fuse.js - Typecheck, lint, and build all pass - Browser verified: Fuse.js fallback works correctly; ONNX model loads asynchronously during boot and activates semantic search when ready - Files changed: `src/components/CommandPalette.tsx` - **Learnings for future iterations:** - Semantic search is async so it can't live in a `useMemo` — use `useState` + debounced `useEffect` pattern instead - The `useRef + setTimeout` debounce pattern works well here: set `debounceRef.current = setTimeout(...)`, clear it in the cleanup function, and in early-return paths - `isModelReady()` is a synchronous check — call it before setting up the debounce timeout to avoid unnecessary delays when model isn't loaded - The ONNX model takes several seconds to load in the browser (downloads ~23MB first time, then cached in IndexedDB), so initial searches will always use Fuse.js fallback - `loadEmbeddings()` is cheap (just returns the already-imported JSON) — safe to call in `useMemo` without performance concern --- ## 2026-02-15 - US-007 - Created `src/components/ChatWidget.tsx` — floating chat button with toggle state - 48px circular button (40px on mobile <640px), fixed bottom-right, teal accent background, white MessageCircle icon - Entrance animation: fade + translateY(8px→0), 1s delay after mount, via framer-motion variants - Respects `prefers-reduced-motion` — skips animation, shows immediately - Hover: shadow-md → shadow-lg + scale(1.05), 150ms transition - z-index 90 (below command palette z-1000) - onClick toggles `isOpen` state, swaps icon between MessageCircle and X - Mounted in `DashboardLayout.tsx` alongside CommandPalette and DetailPanel - Typecheck, lint (0 errors), and build all pass - Browser verified: button visible at bottom-right, toggle works (Open chat ↔ Close chat) - Files changed: `src/components/ChatWidget.tsx` (new), `src/components/DashboardLayout.tsx` - **Learnings for future iterations:** - Responsive sizing via Tailwind classes (`h-10 w-10 sm:h-12 sm:w-12`) works well with inline style for non-Tailwind properties (boxShadow, border-radius) - `AnimatePresence` is already imported and ready for the panel animation in US-008 - The `isOpen` state lives in ChatWidget — US-008 will add the panel UI inside the same component - Hover effects use `onMouseEnter/Leave` with direct style mutation (same pattern as other dashboard components) --- ## 2026-02-15 - US-008 - Built chat panel UI inside `ChatWidget.tsx` with header, message area, and input - Panel opens above the floating button with scale+opacity entrance/exit animation via framer-motion `AnimatePresence` - Messages stored as `Array<{ role: 'user' | 'assistant', content: string }>` in component state - User messages right-aligned in teal-tinted bubbles (`var(--accent-light)` bg, `var(--accent-border)` border) - Assistant messages left-aligned in light gray bubbles (`var(--bg-dashboard)` bg, `var(--border-light)` border) - Message corner radii differ: user bubbles have small bottom-right radius, assistant bubbles small bottom-left (conversational feel) - Input area: textarea with Enter to submit, Shift+Enter for newline. Send button enabled/disabled based on input content - Empty state shows placeholder text when no messages yet - Auto-scrolls to latest message via `useRef` + `scrollIntoView` - Auto-focuses input when panel opens (200ms delay for animation) - Responsive: on mobile (<640px), panel is full-width bottom sheet with rounded top corners; on desktop, 380px wide positioned above the button - Panel entrance: scale(0.95)+opacity(0) → scale(1)+opacity(1), 200ms. Exit: reverse, 150ms - Respects `prefers-reduced-motion` — skips all animation - Close button in header triggers `setIsOpen(false)` (same as floating button toggle) - Submitting appends both user message and placeholder assistant response to state - Typecheck, lint (0 errors), and build all pass - Browser verified: panel opens/closes correctly, messages display, input works, Enter submits, close button works - Files changed: `src/components/ChatWidget.tsx` - **Learnings for future iterations:** - `AnimatePresence` with `key` prop on the panel div is needed for exit animations to work - Panel uses `transformOrigin: 'bottom right'` for natural scale animation from the button corner - CSS-in-JS `