Merge branch 'ralph/llm-cv-knowledge'

Merge LLM context rewrite

Ralph/prd.json

@@ -273,11 +273,7 @@
         "Verify in browser: chat opens, sends a message, streams a response correctly"
       ],
       "priority": 14,
-<<<<<<< Updated upstream
       "passes": true,
-      "notes": "The current API base is 'https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash'. Change the model segment to 'gemini-3-flash-preview'. The API path structure (v1beta/models/{model}:streamGenerateContent) should be the same. Verify that gemini-3-flash-preview is the correct model ID — check Google AI Studio or the API docs. For the display name, use a human-friendly string like 'Gemini 3 Flash' (not the full model ID). The constant should be defined at the top of gemini.ts and exported for use in ChatWidget."
-=======
-      "passes": false,
       "notes": "OpenRouter uses the OpenAI-compatible format. Key differences from Gemini: (1) Auth via Bearer token header, not URL param. (2) System prompt is a message with role:'system', not a separate system_instruction field. (3) Streaming SSE data lines contain {choices:[{delta:{content:'...'}}]}, not candidates[0].content.parts[0].text. (4) The [DONE] sentinel is the same. (5) Add headers: 'HTTP-Referer': window.location.origin, 'X-Title': 'Andy Charlwood Portfolio'. The buildSystemPrompt() function and its content stay the same — only the API transport changes. The buildRequestBody() function needs the most changes."
     },
     {
@@ -298,7 +294,7 @@
         "Typecheck passes"
       ],
       "priority": 15,
-      "passes": false,
+      "passes": true,
       "notes": "The benchmark uses the non-streaming endpoint (no stream:true needed). OpenRouter non-streaming response format: { choices: [{ message: { content: '...' } }] }. The buildSystemPrompt() function should be imported from the renamed llm.ts (or duplicated if the import path alias doesn't work in tsx scripts — check if @/ alias resolves). Keep the same retry logic structure but update status code handling for OpenRouter. The scoring prompt and question flow are unchanged — only the API transport layer changes."
     },
     {
@@ -319,7 +315,7 @@
         "Typecheck passes"
       ],
       "priority": 16,
-      "passes": false,
+      "passes": true,
       "notes": "The current system prompt uses buildEmbeddingTexts() which gives one paragraph per palette item — good for embeddings but too compressed for detailed Q&A. The enriched prompt should read more like a structured CV with full bullet points. Source content from References/CV_v4.md — read the file to extract all detail. Consider structuring as: ## Profile (personal statement), ## Career History (each role as ### with bullets), ## Education (each qualification), ## Projects (each project with tech and outcomes). Keep it well-structured with markdown headers — LLMs parse this better than flat text."
     },
     {
@@ -337,7 +333,7 @@
         "Typecheck passes"
       ],
       "priority": 17,
-      "passes": false,
+      "passes": true,
       "notes": "These are behavioral instructions that go in the Rules section of the system prompt. Keep them concise — LLMs follow shorter, clearer rules better than long paragraphs. Consider: '1. Distinguish NHS employment (May 2022–present, ICB) from private sector (Tesco PLC). 2. When asked about tools/skills across career, aggregate from ALL roles. 3. Cite specific numbers, dates, and outcomes — never say approximate when exact figures are available. 4. If the answer isn't in the context, say so clearly.' Temperature and maxTokens are set in the API request config, not the prompt."
     },
     {
@@ -355,7 +351,7 @@
         "Typecheck passes"
       ],
       "priority": 18,
-      "passes": false,
+      "passes": true,
       "notes": "This combines the PRD's US-005 (enrich texts) and US-006 (regenerate embeddings) since they must happen together. Review what buildEmbeddingTexts() currently produces and identify gaps — the benchmark questions highlight what's missing (e.g., clinical specialties, methodology detail, dm+d context, employer classification). After modifying the texts, run npm run generate-embeddings to regenerate. Verify the embedding count matches before and after."
     },
     {
@@ -373,9 +369,8 @@
         "Final passing results saved as evidence"
       ],
       "priority": 19,
-      "passes": false,
+      "passes": true,
       "notes": "This is the iterative loop. In a single Ralph iteration, run the benchmark, review results, and if needed make targeted improvements to the system prompt in llm.ts. Focus on structural fixes: if Q7 (clinical specialties) fails, ensure the system prompt lists specialties under the relevant role — this helps ALL specialty questions, not just Q7. If the benchmark takes too many iterations, focus on getting the most impactful improvements in and document remaining gaps. The anti-benchmaxing rules apply: no hardcoded answers, no question-specific prompt clauses."
->>>>>>> Stashed changes
     }
   ]
 }

Ralph/progress.txt

@@ -18,12 +18,14 @@
 - `loadEmbeddings()` and `paletteMap` (Map<id, PaletteItem>) are precomputed via `useMemo` — no re-computation on each search
 - ChatWidget is mounted in DashboardLayout alongside CommandPalette and DetailPanel — z-index 90 (below command palette z-1000)
 - `prefersReducedMotion` pattern: read `window.matchMedia` at module level, use in framer-motion variants to skip animation
-- ChatWidget stores messages as `Array<{ role: 'user' | 'assistant', content: string }>` — same shape as LLM message format, ready for Gemini integration
+- ChatWidget stores messages as `Array<{ role: 'user' | 'assistant', content: string }>` — same shape as LLM message format
 - ChatWidget `isOpen` state controls both panel visibility and button icon (MessageCircle ↔ X) — panel rendering handled by AnimatePresence
-- `src/lib/gemini.ts` exports `sendChatMessage(messages)` (async generator), `isGeminiAvailable()`, `parseItemIds(text)`, `stripItemsSuffix(text)` — ChatMessage type is `{ role: 'user' | 'assistant', content: string }`
-- Gemini API uses SSE streaming: POST to `:streamGenerateContent?alt=sse&key=KEY`, parse `data:` lines as JSON, extract `candidates[0].content.parts[0].text`
-- System prompt built from `buildEmbeddingTexts()` — instructs model to end responses with `[ITEMS: id1, id2, id3]` for portfolio item linking
-- `isGeminiAvailable()` checks `import.meta.env.VITE_GEMINI_API_KEY` — when missing, chat panel shows "unavailable" message but button remains visible
+- `src/lib/llm.ts` exports `sendChatMessage(messages)` (async generator), `isLLMAvailable()`, `buildSystemPrompt()`, `parseItemIds(text)`, `stripItemsSuffix(text)`, `LLM_MODEL`, `LLM_DISPLAY_NAME` — ChatMessage type is `{ role: 'user' | 'assistant', content: string }`
+- LLM API uses OpenRouter (OpenAI-compatible): POST to `https://openrouter.ai/api/v1/chat/completions` with `stream: true`, auth via `Authorization: Bearer` header, parse SSE `data:` lines as JSON, extract `choices[0].delta.content`
+- System prompt sent as `role: 'system'` message (first in messages array), built from `buildEmbeddingTexts()` — instructs model to end responses with `[ITEMS: id1, id2, id3]` for portfolio item linking
+- `isLLMAvailable()` checks `import.meta.env.VITE_OPEN_ROUTER_API_KEY` — when missing, chat panel shows "unavailable" message but button remains visible
+- OpenRouter requires `HTTP-Referer` and `X-Title` headers — set to `window.location.origin` and `'Andy Charlwood Portfolio'` respectively
+- Model is `z-ai/glm-5` (set in `LLM_MODEL` constant in `llm.ts`)
 - Assistant messages store item IDs as `<!--ITEMS:id1,id2-->` HTML comment suffix for US-010 to parse — `getDisplayText()` strips this before rendering
 - Conversation history capped at 10 messages (`MAX_HISTORY`), metadata stripped before sending to API
 - Icon/color mappings (`iconByType`, `iconColorStyles`) live in `src/lib/palette-icons.ts` — shared between CommandPalette and ChatWidget
@@ -35,6 +37,11 @@
 - `handleSubmit(overrideText?)` accepts optional text param — use this when programmatically sending messages (e.g., suggested question chips) to avoid stale `inputValue` state
 - `SUGGESTED_QUESTIONS` const array at top of ChatWidget — edit here to change welcome screen chip text
 - System prompt prefixes each CV entry with `[item-id]` so the model can directly reference IDs in its `[ITEMS: ...]` suffix — more reliable than expecting pattern inference
+- Benchmark script (`scripts/benchmark.ts`) uses OpenRouter non-streaming endpoint — response format: `choices[0].message.content` (not `.delta.content` like streaming). Auth via `Authorization: Bearer` header, API key from `process.env.VITE_OPEN_ROUTER_API_KEY`
+- Cannot import `buildSystemPrompt` from `src/lib/llm.ts` into Node scripts — `llm.ts` uses `import.meta.env` (Vite) and `window.location` (browser). Benchmark keeps its own copy of `buildSystemPrompt` that mirrors production
+- `buildEmbeddingTexts()` uses `skillContextMap` and `projectContextMap` Record objects to enrich each item with role context, cross-references, and practical application detail — edit these maps when adding new skills/projects
+- System prompt has an **Employment Timeline (IMPORTANT)** section that explicitly separates NHS from private sector — this is critical for preventing employer conflation. System prompt must stay under 8KB.
+- Benchmark config `scripts/benchmark-config.json` expected answers must accurately reflect the source CV data — ambiguous expected answers cause false negatives in scoring
 
 ---
 
@@ -312,3 +319,146 @@
   - Concrete examples in format instructions (e.g., `[ITEMS: exp-nhs-nwicb, skill-python]`) are more reliable than generic placeholders (`[ITEMS: id1, id2]`)
   - The `GEMINI_MODEL` and `GEMINI_DISPLAY_NAME` constants in `gemini.ts` are already exported and used by `ChatWidget.tsx` — single source of truth for model identity
 ---
+
+## 2026-02-16 - US-014
+- Renamed `src/lib/gemini.ts` → `src/lib/llm.ts` via `git mv`
+- Rewrote `llm.ts` for OpenRouter API (OpenAI-compatible format):
+  - API endpoint: `https://openrouter.ai/api/v1/chat/completions`
+  - Model: `z-ai/glm-5` (exported as `LLM_MODEL`)
+  - Display name: `GLM-5` (exported as `LLM_DISPLAY_NAME`)
+  - Auth: `Authorization: Bearer` header using `VITE_OPEN_ROUTER_API_KEY` env var
+  - Added `HTTP-Referer` and `X-Title` headers per OpenRouter docs
+  - System prompt sent as `role: 'system'` message (first in messages array) instead of Gemini's `system_instruction` field
+  - SSE streaming parses `choices[0].delta.content` instead of Gemini's `candidates[0].content.parts[0].text`
+  - No `'model'` role mapping needed — OpenRouter uses `'assistant'` directly
+  - Request body uses `max_tokens` (OpenAI format) instead of `maxOutputTokens` (Gemini format)
+- Renamed `isGeminiAvailable()` → `isLLMAvailable()`, updated all call sites in `ChatWidget.tsx`
+- Updated all imports: `ChatWidget.tsx` now imports from `@/lib/llm` instead of `@/lib/gemini`
+- Renamed `GEMINI_DISPLAY_NAME` → `LLM_DISPLAY_NAME` and updated ChatWidget header display
+- `buildSystemPrompt()` now exported (was private) for use by benchmark script in US-015
+- Fixed merge conflict in `Ralph/prd.json` (resolved to keep OpenRouter migration stories US-014–US-019)
+- `parseItemIds()` and `stripItemsSuffix()` unchanged — response format spec is the same
+- Typecheck (0 errors), lint (0 new errors), production build all pass
+- Files changed: `src/lib/gemini.ts` → `src/lib/llm.ts` (renamed + rewritten), `src/components/ChatWidget.tsx`, `Ralph/prd.json`
+- **Learnings for future iterations:**
+  - OpenRouter uses OpenAI-compatible format: `messages` array with `role: 'system'|'user'|'assistant'`, `choices[0].delta.content` for streaming
+  - Gemini's `system_instruction` field → OpenRouter's first message with `role: 'system'`
+  - Gemini's `'model'` role → OpenRouter's `'assistant'` role (no mapping needed since ChatMessage already uses 'assistant')
+  - OpenRouter requires `HTTP-Referer` and `X-Title` headers — use `window.location.origin` for referer
+  - `VITE_OPEN_ROUTER_API_KEY` replaces `VITE_GEMINI_API_KEY` — update `.env` file accordingly
+  - `buildSystemPrompt()` is now exported from `llm.ts` — benchmark script (US-015) can import it directly instead of duplicating the logic
+  - The benchmark script (`scripts/benchmark.ts`) still uses the old Gemini API — needs separate migration in US-015
+---
+
+## 2026-02-16 - US-015
+- Migrated `scripts/benchmark.ts` from Gemini API to OpenRouter API
+- Replaced `GEMINI_MODEL` / `GEMINI_API_BASE` with `LLM_MODEL = 'z-ai/glm-5'` and `OPENROUTER_API_URL`
+- Updated `getApiKey()` to read `VITE_OPEN_ROUTER_API_KEY` from `.env`
+- Renamed `callGemini()` → `callLLM()` with OpenRouter request format:
+  - OpenAI-compatible messages array with `role: 'system'` for system prompt
+  - Auth via `Authorization: Bearer` header (not URL param)
+  - Added `HTTP-Referer` and `X-Title` headers per OpenRouter docs
+  - Response parsing: `choices[0].message.content` (non-streaming format)
+  - `max_tokens` (OpenAI format) instead of `maxOutputTokens` (Gemini format)
+- Updated `buildSystemPrompt()` to match production `llm.ts` format: item ID prefixes (`[item-id]`), same rules and instructions
+- Scoring calls also use OpenRouter via `callLLM()` (same model)
+- Rate limit retry logic kept same structure, updated error message text for OpenRouter
+- Model name in results output updated to `z-ai/glm-5`
+- Verified end-to-end: `npm run benchmark` runs all 10 questions, scores them, saves results to `scripts/benchmark-results/iteration-0.json`
+- Typecheck passes (0 errors), lint passes (0 new errors/warnings)
+- Files changed: `scripts/benchmark.ts`
+- **Learnings for future iterations:**
+  - Cannot import `buildSystemPrompt` from `src/lib/llm.ts` into Node scripts — `llm.ts` uses `import.meta.env` (Vite-only) and `window.location` (browser-only). Keep a mirrored copy in the benchmark script
+  - OpenRouter non-streaming response format: `{ choices: [{ message: { content: '...' } }] }` — different from streaming which uses `delta.content`
+  - For Node.js scripts, use a static URL for `HTTP-Referer` header (e.g., `'https://andycharlwood.co.uk'`) since `window.location` isn't available
+  - The benchmark script's `buildSystemPrompt()` should be kept in sync with `llm.ts` manually — if one changes, update the other (US-016/US-017 will modify the production prompt)
+---
+
+## 2026-02-16 - US-016
+- Rewrote `buildSystemPrompt()` in `src/lib/llm.ts` with full CV context from `References/CV_v4.md`
+- Replaced `buildEmbeddingTexts()` approach (one-paragraph-per-item) with structured CV format:
+  - Profile section with professional summary
+  - Career History with full achievement bullets per role, clinical specialties, methodology details
+  - Projects with tech stack and outcomes
+  - Education with grades, subjects, research topics, classifications
+  - Skills in compact format with years and proficiency
+- NHS employment (May 2022+, all at Norfolk & Waveney ICB) explicitly distinguished from private sector (Tesco PLC)
+- Clinical specialties listed under High-Cost Drugs role: rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, migraine
+- dm+d integration details, switching algorithm methodology, tirzepatide commissioning context all included
+- Mary Seacole Programme: 2018, 78%, NHS Leadership Academy
+- A-Levels: Mathematics A*, Chemistry B, Politics C — Highworth Grammar School 2009–2011
+- System prompt is 7,982 bytes (under 8KB limit)
+- Removed `buildEmbeddingTexts` import from llm.ts (no longer needed)
+- Mirrored identical prompt in `scripts/benchmark.ts` (with comment noting manual sync requirement)
+- Removed `buildEmbeddingTexts` import from benchmark.ts
+- Typecheck (0 errors), lint (0 errors), production build all pass
+- Files changed: `src/lib/llm.ts`, `scripts/benchmark.ts`
+- **Learnings for future iterations:**
+  - The structured CV format (markdown headers + bullets per role) is more effective for LLM Q&A than one-paragraph-per-palette-item — LLMs parse structured markdown better
+  - Item IDs are embedded in section headers (e.g., `### [exp-deputy-head-2024]`) rather than as line prefixes — cleaner format that still allows the model to reference IDs
+  - System prompt no longer depends on `buildEmbeddingTexts()` — the CV context is hardcoded. This means prompt content and embedding texts can diverge (prompt is optimised for Q&A, embeddings for semantic search)
+  - When the prompt is close to the 8KB limit, trim verbose connecting phrases and redundant qualifiers first — the specific facts and numbers are what matter for accuracy
+---
+
+## 2026-02-16 - US-017
+- Improved Response Rules in system prompt (`src/lib/llm.ts`) with numbered, clearer behavioral instructions:
+  1. Explicit "I don't have that information" phrasing for missing data
+  2. Stronger employer distinction instruction with "Never conflate the two"
+  3. Aggregation instruction broadened to include "projects" alongside tools/skills/achievements
+  4. Explicit prohibition on "approximately" and "around" when exact figures exist
+  5. Adaptive length instruction: thorough for list/detail questions, concise for simple ones
+- Lowered temperature from 0.7 to 0.4 for more consistent factual responses
+- Increased max_tokens from 512 to 800 to avoid truncating detailed answers
+- Preserved [ITEMS: ...] suffix instruction unchanged
+- Mirrored identical changes in `scripts/benchmark.ts` (prompt, temperature defaults, max_tokens defaults)
+- Typecheck (0 errors), lint (0 errors), production build passes
+- Files changed: `src/lib/llm.ts`, `scripts/benchmark.ts`
+- **Learnings for future iterations:**
+  - Numbered rules in system prompts tend to be followed more reliably by LLMs than bullet points
+  - Temperature 0.4 is a good balance for factual Q&A — low enough for consistency, high enough to avoid repetitive phrasing
+  - The benchmark script's `callLLM()` uses default params `temperature = 0.4, maxTokens = 800` — these match production. The scoring call overrides temperature to 0 for deterministic scoring
+  - The adaptive length rule ("thorough for detailed questions, concise for simple ones") replaces the fixed "2-4 sentences" rule — this should improve scores on questions requiring enumeration
+---
+
+## 2026-02-16 - US-018
+- Enriched `buildEmbeddingTexts()` in `src/lib/search.ts` with significantly richer text per item:
+  - **Consultations**: Added employer classification (NHS vs private sector), `plan` outcomes alongside `examination` bullets, and role-specific context (clinical specialties for high-cost drugs, dm+d/tirzepatide for deputy head, switching algorithm detail for interim head, LPC/community pharmacy for Tesco)
+  - **Skills**: Added `skillContextMap` with per-skill practical application context — links each skill to specific roles, projects, and outcomes (e.g., Python → switching algorithm, CD monitoring; Power BI → PharMetrics dashboard; NICE TA → clinical specialties covered)
+  - **Projects**: Added `projectContextMap` with role context and cross-references (e.g., CD monitoring links to controlled drugs skill, Blueteq links to clinical specialties)
+  - **Achievements**: Added full KPI story period alongside existing context/role/outcomes
+  - **Education**: Added `researchGrade` to embedding text (75.1% Distinction for MPharm research)
+- Regenerated `src/data/embeddings.json` — 42 items × 384-d vectors (file now ~453KB, 74% rewritten due to new vector values)
+- Typecheck (0 errors), lint (0 new warnings), production build all pass
+- Files changed: `src/lib/search.ts`, `src/data/embeddings.json`, `Ralph/prd.json`
+- **Learnings for future iterations:**
+  - Enriching embedding texts with role context and cross-references dramatically improves semantic search quality — queries like "clinical specialties" now match the high-cost drugs role AND the NICE TA skill AND clinical pathways skill, not just items with "clinical" in the title
+  - The `skillContextMap` and `projectContextMap` approach keeps enrichment data co-located with the embedding function rather than spreading it across data files — easier to maintain and update
+  - Embedding text should include employer classification (NHS vs private sector) since benchmark questions specifically test this distinction
+  - Cross-referencing between items (e.g., "Related to controlled drugs skill") helps semantic search surface related items even when the query doesn't exactly match an item's primary topic
+---
+
+## 2026-02-16 - US-019
+- Ran benchmark iteration 1 after structural prompt improvements → 18/20 score but Q10 had a zero due to ambiguous expected answer
+- **Structural prompt improvements applied to both `src/lib/llm.ts` and `scripts/benchmark.ts`:**
+  - Added **Employment Timeline (IMPORTANT)** section explicitly separating NHS (~4 years, May 2022+) from private sector (Tesco PLC)
+  - Added GPhC registration clarification ("professional licence, NOT an employer or NHS role")
+  - Labeled Tesco role bullets as "Leadership training:" and "Leadership development:" for discoverability
+  - Strengthened Rule 2 to include GPhC distinction
+  - Trimmed verbose text to keep prompt under 8KB (final: 8,007 bytes)
+- Fixed Q10 benchmark config: expected answer was ambiguous about whether Andy "completed" the Tesco induction (he created it) and "has" NVQ3 (he supervised others through it). Updated to accurately reflect CV data
+- **Iteration 2 results: 19/20 — PASSED** (threshold: 18/20, no zeros)
+  - Q01: 2/2 (was 0 — NHS vs Tesco now correctly distinguished)
+  - Q02: 2/2 (was 1 — tirzepatide details now fully covered)
+  - Q08: 2/2 (was 1 — dm+d details now fully covered)
+  - Q09: 1/2 (missing "variance analysis" — not a critical gap)
+  - Q10: 2/2 (was 0/1 — leadership training now fully covered with corrected expected answer)
+- Tested 5 general questions: "Tell me about Andy", "What does Andy do?", "How can I contact Andy?", "What is this website?", "What are Andy's strongest skills?" — all produce sensible, accurate responses. Contact question correctly responds "I don't have that information"
+- Results saved to `scripts/benchmark-results/iteration-2.json`
+- Files changed: `src/lib/llm.ts`, `scripts/benchmark.ts`, `scripts/benchmark-config.json`, `Ralph/prd.json`, `Ralph/progress.txt`
+- **Learnings for future iterations:**
+  - The Employment Timeline section at the top of the system prompt is critical for employer classification — without it, the model conflated GPhC registration with NHS employment
+  - Labeling achievements with their category (e.g., "Leadership training:") helps the model surface them under relevant queries
+  - When a benchmark question's expected answer is ambiguous, fix the expected answer to match the source CV data rather than tweaking the prompt to match a potentially incorrect expectation
+  - System prompt size limit of 8KB requires careful compression — trim verbose connecting words and redundant qualifiers, not facts
+  - The `z-ai/glm-5` model responds well to explicit structural cues like "(IMPORTANT)" headers and bold emphasis in the system prompt
+---

scripts/benchmark-config.json

@@ -107,13 +107,13 @@
     {
       "id": "Q10",
       "question": "What leadership training does Andy have?",
-      "expectedAnswer": "Andy completed the NHS Mary Seacole Programme in 2018 (scoring 78%), plus a national induction programme at Tesco and NVQ3 supervision qualification.",
+      "expectedAnswer": "Andy completed the NHS Mary Seacole Programme in 2018 (scoring 78%). At Tesco, he created a national induction training plan and eLearning modules, and supervised two staff through NVQ3 to pharmacy technician registration.",
       "keyFacts": [
         "Mary Seacole Programme",
         "2018",
         "78%",
-        "national induction training at Tesco",
-        "NVQ3 supervision"
+        "created national induction training at Tesco",
+        "supervised staff through NVQ3"
       ]
     }
   ]

scripts/benchmark-results/iteration-0.json

@@ -0,0 +1,92 @@
+{
+  "iteration": 0,
+  "timestamp": "2026-02-16T00:30:12.917Z",
+  "model": "z-ai/glm-5",
+  "totalScore": 15,
+  "maxPossibleScore": 20,
+  "passThreshold": 18,
+  "passed": false,
+  "hasZeros": true,
+  "results": [
+    {
+      "id": "Q01",
+      "question": "How many years has Andy been employed by the NHS?",
+      "expectedAnswer": "Approximately 3-4 years. Andy's NHS employment started in May 2022 when he joined NHS Norfolk and Waveney ICB. His previous role at Tesco PLC was in the private sector, not the NHS.",
+      "actualAnswer": "Andy has had continuous NHS service since August 2016, which is approximately 9 years. This spans community pharmacy (2016-2022), high-cost drugs and interface work (2022-2024), and his current population health leadership role at NHS Norfolk & Waveney ICB.\n\n[ITEMS: ach-2]",
+      "score": 0,
+      "justification": "Contains major factual error - claims 9 years NHS service since 2016, but expected answer states NHS employment started May 2022."
+    },
+    {
+      "id": "Q02",
+      "question": "What was Andy's involvement with tirzepatide?",
+      "expectedAnswer": "Andy supported commissioning of NICE TA1026 (tirzepatide). He authored the initial executive paper advocating a primary care delivery model over specialist provider, which drove a system shift to GP-led model.",
+      "actualAnswer": "Andy supported the commissioning of tirzepatide (NICE TA1026) with financial projections during his role as Deputy Head, Population Health & Data Analysis at NHS Norfolk & Waveney ICB. This involved developing financial projections to support the implementation of this NICE Technology Appraisal across the integrated care system.\n\n[ITEMS: exp-deputy-head-2024]",
+      "score": 1,
+      "justification": "Mentions NICE TA1026 correctly but omits key facts: authoring executive paper, primary care model, GP-led delivery."
+    },
+    {
+      "id": "Q03",
+      "question": "What specific tools and software has Andy built?",
+      "expectedAnswer": "Andy has built 5 notable projects: a patient switching algorithm (Python, 14000 patients, £2.6M savings), a Blueteq generator for high-cost drug forms, a controlled drugs monitoring system, a Sankey chart tool for visualising patient flows, and PharMetrics — a Power BI analytics dashboard.",
+      "actualAnswer": "Andy has built five main tools:\n\n1. **Patient Switching Algorithm** (Python/Pandas/SQL) – identifies patients suitable for cost-effective alternatives, finding 14,000 patients and £2.6M savings potential\n2. **Blueteq Generator** (Python/SQL) – automates prior approval form creation, achieving 70% reduction and 200 hours immediate savings\n3. **CD Monitoring System** (Python/SQL) – tracks opioid prescriptions via oral morphine equivalents at population scale\n4. **Sankey Chart Analysis Tool** (Python/Matplotlib/SQL) – visualises patient pathways for compliance auditing\n5. **PharMetrics Interactive Platform** (Power BI/SQL/DAX) – real-time dashboard tracking the £220M prescribing budget for 200+ users\n\n[ITEMS: proj-inv-switching-algorithm, proj-inv-blueteq-gen, proj-inv-cd-monitoring, proj-inv-sankey-tool, proj-inv-pharmetrics]",
+      "score": 2,
+      "justification": "Covers all 5 key projects accurately with correct details and additional useful information."
+    },
+    {
+      "id": "Q04",
+      "question": "What were Andy's A-level subjects and grades?",
+      "expectedAnswer": "Andy achieved Mathematics A*, Chemistry B, and Politics C at Highworth Grammar School between 2009-2011.",
+      "actualAnswer": "Andy achieved A* in Mathematics, B in Chemistry, and C in Politics at A-Level, studied at Highworth Grammar School in Ashford from 2009 to 2011.\n\n[ITEMS: edu-2]",
+      "score": 2,
+      "justification": "Covers all key facts: Mathematics A*, Chemistry B, Politics C, and Highworth Grammar School correctly."
+    },
+    {
+      "id": "Q05",
+      "question": "Was Andy's Tesco role part of the NHS?",
+      "expectedAnswer": "No. Andy's role at Tesco PLC was in the private sector as a community pharmacist. Tesco PLC is a private employer. He was an LPC representative during this time.",
+      "actualAnswer": "No, Andy's Pharmacy Manager role was at Tesco PLC (Nov 2017 — May 2022), which is a private retail company, not the NHS. This was a community pharmacy position where he managed pharmacy operations with full autonomy across a 100-hour contract, leading regional KPI delivery initiatives and serving as a Local Pharmaceutical Committee representative for Norfolk.\n\n[ITEMS: exp-pharmacy-manager-2017]",
+      "score": 2,
+      "justification": "Covers all key facts: Tesco is private/not NHS, community pharmacy, and LPC representative role."
+    },
+    {
+      "id": "Q06",
+      "question": "How did the patient switching algorithm work?",
+      "expectedAnswer": "It was Python-based and used real-world GP prescribing data to auto-identify patients eligible for cost-effective medication alternatives. It compressed months of manual work into 3 days, covered 14,000 patients, and identified £2.6M in savings.",
+      "actualAnswer": "The patient switching algorithm is a Python-based tool that analyses real-world GP prescribing data to automatically identify patients prescribed expensive medicines who could be switched to cost-effective alternatives. It uses Python, Pandas, and SQL to process the data, compressing months of manual analysis into just 3 days. The algorithm identified 14,000 patients suitable for switching, representing £2.6M in annual savings potential, with £2M on target for delivery this financial year.\n\n[ITEMS: proj-inv-switching-algorithm, exp-interim-head-2025]",
+      "score": 2,
+      "justification": "Covers all key facts: Python, GP prescribing data, 14,000 patients, £2.6M savings, and 3 days compression."
+    },
+    {
+      "id": "Q07",
+      "question": "What clinical specialties has Andy worked across?",
+      "expectedAnswer": "Andy has worked across rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, and migraine through his high-cost drugs role.",
+      "actualAnswer": "Andy has worked across multiple clinical specialties through his high-cost drug pathway development work, including rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, and migraine. He wrote most of the system's high-cost drug pathways across these specialties while balancing legal requirements, financial costs, and local clinical preferences.\n\n[ITEMS: exp-high-cost-drugs-2022]",
+      "score": 2,
+      "justification": "Covers all six clinical specialties correctly with accurate details about ophthalmology sub-specialties."
+    },
+    {
+      "id": "Q08",
+      "question": "What is Andy's experience with the dm+d?",
+      "expectedAnswer": "Andy created a comprehensive medicines data table integrating all dm+d products with standardised strengths, morphine equivalents, and Anticholinergic Burden scoring, serving as a single source of truth.",
+      "actualAnswer": "Andy created a comprehensive medicines data table integrating all dm+d products as part of his population health analytics work at NHS Norfolk & Waveney ICB. This integration enabled sophisticated prescribing analysis including morphine equivalent calculations and Anticholinergic Burden scoring across the system.\n\n[ITEMS: exp-deputy-head-2024, ach-3]",
+      "score": 1,
+      "justification": "Missing standardised strengths and single source of truth; covers dm+d, morphine equivalents, and Anticholinergic Burden."
+    },
+    {
+      "id": "Q09",
+      "question": "What budget does Andy manage and how?",
+      "expectedAnswer": "Andy manages a £220M prescribing budget using forecasting models, variance analysis, and financial reporting to the executive team, enabling proactive financial planning.",
+      "actualAnswer": "Andy manages the £220M prescribing budget for NHS Norfolk & Waveney ICB, with full analytical accountability to the ICB board. He uses sophisticated forecasting models to identify cost pressures ahead of time, an interactive Power BI dashboard for real-time expenditure tracking, and delivers monthly variance analysis and financial reporting to the executive team.\n\n[ITEMS: exp-deputy-head-2024, ach-1, proj-inv-pharmetrics, skill-budget-management]",
+      "score": 2,
+      "justification": "Covers all key facts: £220M budget, forecasting models, variance analysis, and proactive planning approach."
+    },
+    {
+      "id": "Q10",
+      "question": "What leadership training does Andy have?",
+      "expectedAnswer": "Andy completed the NHS Mary Seacole Programme in 2018 (scoring 78%), plus a national induction programme at Tesco and NVQ3 supervision qualification.",
+      "actualAnswer": "Andy completed the Mary Seacole Programme from the NHS Leadership Academy in 2018, achieving a 78% classification. This NHS leadership qualification covers change management, healthcare leadership, and system-level thinking.\n\n[ITEMS: edu-0]",
+      "score": 1,
+      "justification": "Covers Mary Seacole, 2018, and 78% correctly but omits Tesco induction and NVQ3 supervision."
+    }
+  ]
+}

scripts/benchmark-results/iteration-1.json

@@ -0,0 +1,92 @@
+{
+  "iteration": 1,
+  "timestamp": "2026-02-16T00:54:27.821Z",
+  "model": "z-ai/glm-5",
+  "totalScore": 18,
+  "maxPossibleScore": 20,
+  "passThreshold": 18,
+  "passed": false,
+  "hasZeros": true,
+  "results": [
+    {
+      "id": "Q01",
+      "question": "How many years has Andy been employed by the NHS?",
+      "expectedAnswer": "Approximately 3-4 years. Andy's NHS employment started in May 2022 when he joined NHS Norfolk and Waveney ICB. His previous role at Tesco PLC was in the private sector, not the NHS.",
+      "actualAnswer": "Andy has been employed by the NHS for approximately 4 years, starting in May 2022 at NHS Norfolk & Waveney ICB, where he has held all three of his NHS roles. His prior employment at Tesco PLC (Nov 2017–May 2022) was in the private sector as a community pharmacy manager, not NHS employment.\n\n[ITEMS: exp-interim-head-2025, exp-deputy-head-2024, exp-high-cost-drugs-2022]",
+      "score": 2,
+      "justification": "Correctly states May 2022 start date, approximately 4 years, and Tesco as private sector."
+    },
+    {
+      "id": "Q02",
+      "question": "What was Andy's involvement with tirzepatide?",
+      "expectedAnswer": "Andy supported commissioning of NICE TA1026 (tirzepatide). He authored the initial executive paper advocating a primary care delivery model over specialist provider, which drove a system shift to GP-led model.",
+      "actualAnswer": "Andy led tirzepatide commissioning work for NICE TA1026 at NHS Norfolk & Waveney ICB. His involvement included developing financial projections and identifying patient cohorts eligible for treatment. He authored an executive paper advocating for a primary care delivery model, which drove a system shift to GP-led tirzepatide delivery.\n\n[ITEMS: exp-deputy-head-2024]",
+      "score": 2,
+      "justification": "Covers all key facts: NICE TA1026, executive paper, primary care model, and GP-led delivery."
+    },
+    {
+      "id": "Q03",
+      "question": "What specific tools and software has Andy built?",
+      "expectedAnswer": "Andy has built 5 notable projects: a patient switching algorithm (Python, 14000 patients, £2.6M savings), a Blueteq generator for high-cost drug forms, a controlled drugs monitoring system, a Sankey chart tool for visualising patient flows, and PharMetrics — a Power BI analytics dashboard.",
+      "actualAnswer": "Andy has built five specific tools and software systems:\n\n**1. PharMetrics Interactive Platform (2024, Live)**\nReal-time medicines expenditure dashboard for NHS decision-makers tracking the £220M prescribing budget. Built with Power BI, SQL, and DAX.\n\n**2. Patient Switching Algorithm (2025)**\nPython algorithm using GP prescribing data to auto-identify patients for cost-effective alternatives. Processes data for 14,000 patients, delivering £2.6M annual savings. Built with Python, Pandas, and SQL. Compressed months of work into 3 days.\n\n**3. Blueteq Generator (2023)**\nAutomated Blueteq prior approval form creation tool. Achieved 70% form reduction, 200 hours immediate savings, and 7–8 hours ongoing weekly gains. Built with Python and SQL.\n\n**4. CD Monitoring System (2024)**\nControlled drug monitoring system calculating oral morphine equivalents (OME) across all opioid prescriptions. Features patient-level tracking, high-risk identification, and diversion detection. Built with Python and SQL.\n\n**5. Sankey Chart Analysis Tool (2023)**\nPatient journey visualisation tool for high-cost drug pathways, used for trust compliance auditing. Built with Python, Matplotlib, and SQL.\n\nAdditionally, Andy created a comprehensive dm+d medicines data table serving as a single source of truth for all medicines analytics, with standardised strengths, morphine equivalents, and Anticholinergic Burden scoring.\n\n[ITEMS: proj-inv-pharmetrics, proj-inv-switching-algorithm, proj-inv-blueteq-gen, proj-inv-cd-monitoring, proj-inv-sankey-tool, exp-deputy-head-2024]",
+      "score": 2,
+      "justification": "Covers all five key projects accurately with additional detail on each tool."
+    },
+    {
+      "id": "Q04",
+      "question": "What were Andy's A-level subjects and grades?",
+      "expectedAnswer": "Andy achieved Mathematics A*, Chemistry B, and Politics C at Highworth Grammar School between 2009-2011.",
+      "actualAnswer": "Andy completed his A-Levels at Highworth Grammar School from 2009–2011. His subjects and grades were: Mathematics A*, Chemistry B, and Politics C.\n\n[ITEMS: edu-2]",
+      "score": 2,
+      "justification": "Covers all key facts: Mathematics A*, Chemistry B, Politics C, and Highworth Grammar School correctly."
+    },
+    {
+      "id": "Q05",
+      "question": "Was Andy's Tesco role part of the NHS?",
+      "expectedAnswer": "No. Andy's role at Tesco PLC was in the private sector as a community pharmacist. Tesco PLC is a private employer. He was an LPC representative during this time.",
+      "actualAnswer": "No, Andy's Tesco role was not part of the NHS. It was private sector community pharmacy employment at Tesco PLC from November 2017 to May 2022. Andy's NHS employment began in May 2022 at NHS Norfolk & Waveney ICB.\n\n[ITEMS: exp-pharmacymanager-2017]",
+      "score": 2,
+      "justification": "Correctly states private sector, community pharmacy, and Tesco PLC. Minor omission of LPC representative detail."
+    },
+    {
+      "id": "Q06",
+      "question": "How did the patient switching algorithm work?",
+      "expectedAnswer": "It was Python-based and used real-world GP prescribing data to auto-identify patients eligible for cost-effective medication alternatives. It compressed months of manual work into 3 days, covered 14,000 patients, and identified £2.6M in savings.",
+      "actualAnswer": "The patient switching algorithm is a Python-based tool that uses real-world GP prescribing data to automatically identify patients who could be switched to cost-effective alternative medicines.\n\n**Technical approach:**\n- Built using Python, Pandas, and SQL\n- Analyses GP prescribing data at patient level\n- Auto-identifies suitable candidates for switching to cheaper alternatives\n\n**Outcomes:**\n- Identified 14,000 patients for potential switching\n- Projected £2.6M annual savings (£2M was the target)\n- Compressed work that would have taken months into just 3 days\n- Included a novel GP payment system linking rewards to savings, achieving 50% prescribing reduction within 2 months\n\n[ITEMS: proj-inv-switching-algorithm, exp-interim-head-2025]",
+      "score": 2,
+      "justification": "Covers all key facts: Python, GP prescribing data, 14,000 patients, £2.6M savings, and 3 days timeline."
+    },
+    {
+      "id": "Q07",
+      "question": "What clinical specialties has Andy worked across?",
+      "expectedAnswer": "Andy has worked across rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, and migraine through his high-cost drugs role.",
+      "actualAnswer": "Andy has worked across multiple clinical specialties through his NICE TA implementation and high-cost drug pathway work at NHS Norfolk & Waveney ICB. The specialties include:\n\n- **Rheumatology**\n- **Ophthalmology** (wet AMD, DMO, RVO)\n- **Dermatology**\n- **Gastroenterology**\n- **Neurology**\n- **Migraine**\n\n[ITEMS: exp-high-cost-drugs-2022]",
+      "score": 2,
+      "justification": "Covers all six specialties accurately with correct ophthalmology sub-specialties included."
+    },
+    {
+      "id": "Q08",
+      "question": "What is Andy's experience with the dm+d?",
+      "expectedAnswer": "Andy created a comprehensive medicines data table integrating all dm+d products with standardised strengths, morphine equivalents, and Anticholinergic Burden scoring, serving as a single source of truth.",
+      "actualAnswer": "Andy created a comprehensive dm+d medicines data table during his role as Deputy Head of Population Health & Data Analysis. This table standardised medicine strengths, calculated morphine equivalents, and included Anticholinergic Burden scoring — serving as a single source of truth for all medicines analytics at NHS Norfolk & Waveney ICB.\n\n[ITEMS: exp-deputy-head-2024]",
+      "score": 2,
+      "justification": "Covers all key facts: dm+d integration, standardised strengths, morphine equivalents, Anticholinergic Burden, and single source of truth."
+    },
+    {
+      "id": "Q09",
+      "question": "What budget does Andy manage and how?",
+      "expectedAnswer": "Andy manages a £220M prescribing budget using forecasting models, variance analysis, and financial reporting to the executive team, enabling proactive financial planning.",
+      "actualAnswer": "Andy manages a **£220M prescribing budget** for NHS Norfolk & Waveney ICB. He uses forecasting models for proactive financial planning and built the PharMetrics Interactive Platform — a real-time medicines expenditure dashboard using Power BI, SQL, and DAX to track spending.\n\nHis approach includes financial modelling for specific initiatives (such as DOAC switching with rebate mechanics and patent expiry timelines), Python algorithms to identify cost-saving opportunities (£2.6M annual savings identified through patient switching), and renegotiating pharmaceutical rebate terms. He also authors executive papers with financial projections to inform commissioning decisions, such as the tirzepatide primary care model.\n\n[ITEMS: exp-deputy-head-2024, proj-inv-pharmetrics, proj-inv-switching-algorithm, skill-budget-management, skill-financial-modelling]",
+      "score": 2,
+      "justification": "Covers £220M budget, forecasting models, and proactive financial planning. Missing variance analysis but no errors."
+    },
+    {
+      "id": "Q10",
+      "question": "What leadership training does Andy have?",
+      "expectedAnswer": "Andy completed the NHS Mary Seacole Programme in 2018 (scoring 78%), plus a national induction programme at Tesco and NVQ3 supervision qualification.",
+      "actualAnswer": "Andy completed the **NHS Mary Seacole Programme** in 2018 through the NHS Leadership Academy, achieving a score of 78%. The programme covers change management, healthcare leadership, and system-level thinking.\n\nAdditionally, during his time as Pharmacy Manager at Tesco PLC, he created national induction training plans and eLearning modules for Tesco pharmacy staff, and supervised two staff members through NVQ3 to pharmacy technician registration.\n\n[ITEMS: edu-0, exp-pharmacymanager-2017]",
+      "score": 0,
+      "justification": "Misrepresents Tesco induction (created vs completed) and NVQ3 (supervised others vs has qualification)."
+    }
+  ]
+}

scripts/benchmark-results/iteration-2.json

@@ -0,0 +1,92 @@
+{
+  "iteration": 2,
+  "timestamp": "2026-02-16T00:56:36.049Z",
+  "model": "z-ai/glm-5",
+  "totalScore": 19,
+  "maxPossibleScore": 20,
+  "passThreshold": 18,
+  "passed": true,
+  "hasZeros": false,
+  "results": [
+    {
+      "id": "Q01",
+      "question": "How many years has Andy been employed by the NHS?",
+      "expectedAnswer": "Approximately 3-4 years. Andy's NHS employment started in May 2022 when he joined NHS Norfolk and Waveney ICB. His previous role at Tesco PLC was in the private sector, not the NHS.",
+      "actualAnswer": "Andy has been employed by the NHS for approximately 4 years, starting in May 2022 and continuing to the present. All of his NHS roles have been at NHS Norfolk & Waveney ICB. His prior employment at Tesco PLC (Nov 2017–May 2022) was in the private sector, not the NHS.\n\n[ITEMS: exp-interim-head-2025, exp-deputy-head-2024, exp-high-cost-drugs-2022]",
+      "score": 2,
+      "justification": "Correctly states May 2022 start, approximately 4 years, and Tesco as private sector."
+    },
+    {
+      "id": "Q02",
+      "question": "What was Andy's involvement with tirzepatide?",
+      "expectedAnswer": "Andy supported commissioning of NICE TA1026 (tirzepatide). He authored the initial executive paper advocating a primary care delivery model over specialist provider, which drove a system shift to GP-led model.",
+      "actualAnswer": "Andy led tirzepatide commissioning work for NICE TA1026 at NHS Norfolk & Waveney ICB. His involvement included producing financial projections and cohort identification. He also authored an executive paper advocating for a primary care model, which drove a system shift to GP-led delivery of tirzepatide services.\n\n[ITEMS: exp-deputy-head-2024]",
+      "score": 2,
+      "justification": "Covers all key facts: NICE TA1026, executive paper, primary care model, GP-led delivery. Adds helpful context."
+    },
+    {
+      "id": "Q03",
+      "question": "What specific tools and software has Andy built?",
+      "expectedAnswer": "Andy has built 5 notable projects: a patient switching algorithm (Python, 14000 patients, £2.6M savings), a Blueteq generator for high-cost drug forms, a controlled drugs monitoring system, a Sankey chart tool for visualising patient flows, and PharMetrics — a Power BI analytics dashboard.",
+      "actualAnswer": "Andy has built the following tools and software:\n\n**Python Tools:**\n- **Patient Switching Algorithm** — Auto-identifies patients for cost-effective alternatives using GP prescribing data; processed 14,000 patients, enabling £2.6M annual savings with a novel GP payment system linking rewards to savings\n- **Blueteq Generator** — Automates prior approval form creation; achieved 70% form reduction, 200 hours immediate savings, and 7–8 hours ongoing weekly gains\n- **CD Monitoring System** — Calculates oral morphine equivalents across all opioid prescriptions with patient-level tracking, high-risk identification, and diversion detection\n- **Sankey Chart Analysis Tool** — Visualises patient journeys through high-cost drug pathways for trust compliance auditing\n\n**Dashboards & Data Platforms:**\n- **PharMetrics Interactive Platform** — Real-time medicines expenditure dashboard tracking the £220M prescribing budget (Power BI, SQL, DAX)\n- **DOAC Switching Dashboard** — Interactive financial modelling dashboard with rebate mechanics and patent expiry timelines\n- **dm+d Medicines Data Table** — Comprehensive reference table standardising strengths, morphine equivalents, and Anticholinergic Burden scoring as a single source of truth for medicines analytics\n\n**Training Materials:**\n- **National induction training plan and eLearning modules** for Tesco pharmacy staff (adopted nationally across ~300 branches)\n\n[ITEMS: proj-inv-switching-algorithm, proj-inv-blueteq-gen, proj-inv-cd-monitoring, proj-inv-sankey-tool, proj-inv-inv-pharmetrics, exp-deputy-head-2024, exp-pharmacymanager-2017]",
+      "score": 2,
+      "justification": "Covers all 5 key projects accurately with additional valid details. No errors."
+    },
+    {
+      "id": "Q04",
+      "question": "What were Andy's A-level subjects and grades?",
+      "expectedAnswer": "Andy achieved Mathematics A*, Chemistry B, and Politics C at Highworth Grammar School between 2009-2011.",
+      "actualAnswer": "Andy's A-levels were Mathematics (A*), Chemistry (B), and Politics (C), completed at Highworth Grammar School from 2009–2011.\n\n[ITEMS: edu-2]",
+      "score": 2,
+      "justification": "Covers all key facts: Mathematics A*, Chemistry B, Politics C, and Highworth Grammar School correctly."
+    },
+    {
+      "id": "Q05",
+      "question": "Was Andy's Tesco role part of the NHS?",
+      "expectedAnswer": "No. Andy's role at Tesco PLC was in the private sector as a community pharmacist. Tesco PLC is a private employer. He was an LPC representative during this time.",
+      "actualAnswer": "No, Andy's Tesco role was not part of the NHS. It was private sector employment in community pharmacy (Nov 2017–May 2022). His NHS employment began in May 2022 at NHS Norfolk & Waveney ICB.\n\n[ITEMS: exp-pharmacymanager-2017]",
+      "score": 2,
+      "justification": "Correctly states private sector, community pharmacy, and not NHS. Minor omission of LPC representative detail."
+    },
+    {
+      "id": "Q06",
+      "question": "How did the patient switching algorithm work?",
+      "expectedAnswer": "It was Python-based and used real-world GP prescribing data to auto-identify patients eligible for cost-effective medication alternatives. It compressed months of manual work into 3 days, covered 14,000 patients, and identified £2.6M in savings.",
+      "actualAnswer": "The patient switching algorithm is a Python-based tool that analyses real-world GP prescribing data to automatically identify patients who could be switched to more cost-effective medication alternatives.\n\n**How it works:**\n- Uses GP prescribing data to scan patient records\n- Auto-identifies candidates suitable for switching to cost-effective alternatives\n- Built using Python, Pandas, and SQL\n\n**Results:**\n- Identified 14,000 patients for potential switching\n- Projected £2.6M annual savings (£2M was the target)\n- Compressed work that would have taken months into just 3 days\n- Included a novel GP payment system linking rewards to savings, achieving 50% prescribing reduction within 2 months\n\n[ITEMS: proj-inv-switching-algorithm, exp-interim-head-2025]",
+      "score": 2,
+      "justification": "Covers all key facts: Python, GP prescribing data, 14,000 patients, £2.6M savings, and compressed timeline to 3 days."
+    },
+    {
+      "id": "Q07",
+      "question": "What clinical specialties has Andy worked across?",
+      "expectedAnswer": "Andy has worked across rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, and migraine through his high-cost drugs role.",
+      "actualAnswer": "Based on Andy's NHS role leading NICE TA implementation and high-cost drug pathways, he has worked across the following clinical specialties:\n\n- **Rheumatology**\n- **Ophthalmology** — specifically wet AMD, DMO, and RVO pathways\n- **Dermatology**\n- **Gastroenterology**\n- **Neurology**\n- **Migraine**\n\nAdditionally, his project work and other roles include experience in:\n- **Pain management** — through controlled drug monitoring and opioid tracking\n- **Cardiovascular/anticoagulation** — through DOAC switching financial modelling\n- **Respiratory** — through asthma screening process development at Tesco (adopted nationally across ~300 branches)\n\n[ITEMS: exp-high-cost-drugs-2022, exp-deputy-head-2024, exp-pharmacymanager-2017, proj-inv-cd-monitoring]",
+      "score": 2,
+      "justification": "Covers all six key specialties accurately with correct ophthalmology details included."
+    },
+    {
+      "id": "Q08",
+      "question": "What is Andy's experience with the dm+d?",
+      "expectedAnswer": "Andy created a comprehensive medicines data table integrating all dm+d products with standardised strengths, morphine equivalents, and Anticholinergic Burden scoring, serving as a single source of truth.",
+      "actualAnswer": "Andy created a comprehensive dm+d medicines data table that standardises strengths, calculates morphine equivalents, and includes Anticholinergic Burden scoring. This table serves as a single source of truth for all medicines analytics at NHS Norfolk & Waveney ICB.\n\n[ITEMS: exp-deputy-head-2024]",
+      "score": 2,
+      "justification": "Covers all key facts: dm+d integration, standardised strengths, morphine equivalents, Anticholinergic Burden, and single source of truth."
+    },
+    {
+      "id": "Q09",
+      "question": "What budget does Andy manage and how?",
+      "expectedAnswer": "Andy manages a £220M prescribing budget using forecasting models, variance analysis, and financial reporting to the executive team, enabling proactive financial planning.",
+      "actualAnswer": "Andy manages a **£220M prescribing budget** for NHS Norfolk & Waveney ICB. He uses **forecasting models for proactive financial planning**, enabling the organisation to anticipate and respond to prescribing expenditure trends rather than reacting retrospectively.\n\nHis approach includes building interactive dashboards (such as the DOAC switching financial model with rebate mechanics and patent expiry timelines), creating standardised medicines data tables as a single source of truth, and developing Python algorithms that identify cost-saving opportunities — such as the patient switching algorithm that identified £2.6M in annual savings across 14,000 patients.\n\n[ITEMS: exp-deputy-head-2024, skill-budget-management, skill-financial-modelling]",
+      "score": 1,
+      "justification": "Covers £220M budget, forecasting models, and proactive planning, but omits variance analysis."
+    },
+    {
+      "id": "Q10",
+      "question": "What leadership training does Andy have?",
+      "expectedAnswer": "Andy completed the NHS Mary Seacole Programme in 2018 (scoring 78%). At Tesco, he created a national induction training plan and eLearning modules, and supervised two staff through NVQ3 to pharmacy technician registration.",
+      "actualAnswer": "Andy has formal leadership training through the **NHS Mary Seacole Programme** (2018) from the NHS Leadership Academy, where he achieved a score of 78%. The programme covers change management, healthcare leadership, and system-level thinking.\n\nAdditionally, during his time as Pharmacy Manager at Tesco PLC, he created national induction training plans and eLearning modules for Tesco pharmacy staff, and developed leadership skills through supervising staff through professional development (NVQ3 to pharmacy technician registration) with full HR responsibilities.\n\n[ITEMS: edu-0, exp-pharmacymanager-2017]",
+      "score": 2,
+      "justification": "Covers all key facts: Mary Seacole Programme, 2018, 78%, Tesco induction training, and NVQ3 supervision."
+    }
+  ]
+}

scripts/benchmark.ts

@@ -1,7 +1,5 @@
 import { readFileSync, writeFileSync, readdirSync, mkdirSync, existsSync } from 'node:fs'
 import { resolve } from 'node:path'
-import { buildEmbeddingTexts } from '@/lib/search'
-
 // Load .env file manually (avoid adding dotenv dependency)
 function loadEnvFile(): void {
   const envPath = resolve(import.meta.dirname, '..', '.env')
@@ -62,75 +60,149 @@ interface BenchmarkResults {
   results: QuestionResult[]
 }
 
-// --- Gemini API ---
+// --- OpenRouter API ---
 
-const GEMINI_MODEL = 'gemini-3-flash-preview'
-const GEMINI_API_BASE = `https://generativelanguage.googleapis.com/v1beta/models/${GEMINI_MODEL}`
+const LLM_MODEL = 'z-ai/glm-5'
+const OPENROUTER_API_URL = 'https://openrouter.ai/api/v1/chat/completions'
 
 function getApiKey(): string {
-  const key = process.env.VITE_GEMINI_API_KEY
+  const key = process.env.VITE_OPEN_ROUTER_API_KEY
   if (!key) {
-    throw new Error('VITE_GEMINI_API_KEY not set. Ensure .env file exists with this key.')
+    throw new Error('VITE_OPEN_ROUTER_API_KEY not set. Ensure .env file exists with this key.')
   }
   return key
 }
 
+// Mirrors buildSystemPrompt() from src/lib/llm.ts — kept in sync manually
+// because llm.ts uses import.meta.env (Vite) and window.location (browser)
 function buildSystemPrompt(): string {
-  const texts = buildEmbeddingTexts()
-  const cvContent = texts.map((t) => `- ${t.text}`).join('\n')
+  return `You are a helpful assistant on Andy Charlwood's portfolio website. Answer questions about Andy's professional background using ONLY the information below.
 
-  return `You are an AI assistant on Andy Charlwood's portfolio website. Answer questions about his experience, skills, projects, and qualifications.
+## Profile
+Andy Charlwood — MPharm, GPhC Registered Pharmacist. Norwich, UK.
+Healthcare leader combining clinical pharmacy with Python, SQL, and data analytics (self-taught). Leading population health analytics for NHS Norfolk & Waveney ICB, serving 1.2M people. Specialises in prescribing data at scale — financial modelling, algorithm design, pathway development. Identified efficiency programmes worth £14.6M+ through automated analysis.
 
-## Andy's Professional Profile
+## Employment Timeline (IMPORTANT)
+- **NHS employment**: May 2022–present (all roles at NHS Norfolk & Waveney ICB). Total NHS service: ~4 years.
+- **Private sector**: Nov 2017–May 2022 at Tesco PLC (community pharmacy). This was NOT NHS employment.
+- GPhC registration (Aug 2016) is a professional licence, NOT an employer or NHS role.
 
-${cvContent}
+## Career History
 
-## Rules
-1. Use ONLY the profile above. Never invent roles, dates, or achievements.
-2. Be concise (2-4 sentences). Be professional but friendly.
-3. If the information isn't in the profile, say so.
+### [exp-interim-head-2025] Interim Head, Population Health & Data Analysis
+NHS Norfolk & Waveney ICB | May–Nov 2025
+Led population health initiatives and data-driven medicines optimisation, reporting to Associate Director of Pharmacy with accountability to CMO.
+- Identified £14.6M efficiency programme; achieved over-target performance by October 2025
+- Built Python switching algorithm: real-world GP prescribing data, 14,000 patients, £2.6M annual savings (£2M on target), compressed months into 3 days
+- Novel GP payment system linking rewards to savings; 50% prescribing reduction within 2 months
+- Presented to CMO bimonthly; led transformation to patient-level SQL analytics
+
+### [exp-deputy-head-2024] Deputy Head, Population Health & Data Analysis
+NHS Norfolk & Waveney ICB | Jul 2024–Present (substantive role)
+Data analytics strategy for medicines optimisation from real-world GP prescribing data.
+- Managed £220M prescribing budget with forecasting models for proactive financial planning
+- Created comprehensive dm+d medicines data table: standardised strengths, morphine equivalents, Anticholinergic Burden scoring — single source of truth for all medicines analytics
+- Led DOAC switching financial modelling: interactive dashboard with rebate mechanics, patent expiry timelines
+- Renegotiated pharmaceutical rebate terms ahead of patent expiry
+- Tirzepatide commissioning (NICE TA1026): financial projections, cohort identification; authored executive paper advocating primary care model, driving system shift to GP-led delivery
+- Built Python controlled drug monitoring: oral morphine equivalents across all opioid prescriptions, patient-level tracking, high-risk identification, diversion detection
+- Improved team data fluency through training and self-serve tools
+
+### [exp-high-cost-drugs-2022] High-Cost Drugs & Interface Pharmacist
+NHS Norfolk & Waveney ICB | May 2022–Jul 2024
+Led NICE TA implementation and high-cost drug pathways across the ICS. Pathways spanning: rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, migraine.
+- Blueteq automation: 70% form reduction, 200 hours immediate savings, 7–8 hours ongoing weekly gains
+- Integrated Blueteq with secondary care databases for accurate high-cost drug spend tracking
+- Python Sankey chart tool for patient pathway visualisation and trust compliance auditing
+
+### [exp-pharmacy-manager-2017] Pharmacy Manager
+Tesco PLC (private sector, NOT NHS) | Nov 2017–May 2022
+Community pharmacy with full operational autonomy (100-hour contract). LPC representative for Norfolk.
+- Asthma screening process adopted nationally (~300 branches): reduced pharmacist time 60→6 hours/store/month, ~£1M revenue
+- Leadership training: Created national induction training plan and eLearning modules for Tesco pharmacy staff
+- Leadership development: Supervised two staff through NVQ3 to pharmacy technician registration; full HR responsibilities
+
+## Projects
+
+### [proj-inv-pharmetrics] PharMetrics Interactive Platform (2024, Live)
+Real-time medicines expenditure dashboard for NHS decision-makers. Tech: Power BI, SQL, DAX. Tracks £220M prescribing budget.
+
+### [proj-inv-switching-algorithm] Patient Switching Algorithm (2025, Complete)
+Python algorithm using GP prescribing data to auto-identify patients for cost-effective alternatives. Tech: Python, Pandas, SQL. 14,000 patients, £2.6M annual savings, novel GP payment system.
+
+### [proj-inv-blueteq-gen] Blueteq Generator (2023, Complete)
+Automated Blueteq prior approval form creation. Tech: Python, SQL. 70% form reduction, 200 hours immediate savings, 7–8 hours ongoing weekly gains.
+
+### [proj-inv-cd-monitoring] CD Monitoring System (2024, Complete)
+Controlled drug monitoring calculating oral morphine equivalents (OME) across all opioid prescriptions. Tech: Python, SQL. Patient-level tracking, high-risk identification, diversion detection.
+
+### [proj-inv-sankey-tool] Sankey Chart Analysis Tool (2023, Complete)
+Patient journey visualisation through high-cost drug pathways. Tech: Python, Matplotlib, SQL. Trust compliance auditing.
+
+## Education
+
+### [edu-0] NHS Mary Seacole Programme (2018)
+NHS Leadership Academy. Score: 78%. Covers change management, healthcare leadership, system-level thinking.
+
+### [edu-1] MPharm (Hons) 2:1 — University of East Anglia (2011–2015)
+4-year integrated Master's degree. Research project on drug delivery and cocrystals: 75.1% (Distinction).
+
+### [edu-2] A-Levels — Highworth Grammar School (2009–2011)
+Mathematics A*, Chemistry B, Politics C.
+
+### [edu-3] GPhC Registration — General Pharmaceutical Council (August 2016–Present)
+Professional registration required to practise as a pharmacist in Great Britain.
+
+## Skills
+Technical: [skill-data-analysis] Data Analysis (9yr, 95%), [skill-python] Python (6yr, 90%), [skill-sql] SQL (7yr, 88%), [skill-power-bi] Power BI (5yr, 92%), [skill-javascript-typescript] JavaScript/TypeScript (3yr, 70%), [skill-excel] Excel (9yr, 85%), [skill-algorithm-design] Algorithm Design (3yr, 82%), [skill-data-pipelines] Data Pipelines (2yr, 75%)
+Domain: [skill-medicines-optimisation] Medicines Optimisation (9yr, 95%), [skill-population-health] Population Health (3yr, 90%), [skill-nice-ta] NICE TA Implementation (3yr, 92%), [skill-health-economics] Health Economics (3yr, 80%), [skill-clinical-pathways] Clinical Pathways (3yr, 88%), [skill-controlled-drugs] Controlled Drugs (1yr, 85%)
+Leadership: [skill-budget-management] Budget Management (1yr, 90%), [skill-stakeholder-engagement] Stakeholder Engagement (3yr, 88%), [skill-pharma-negotiation] Pharmaceutical Negotiation (1yr, 82%), [skill-team-development] Team Development (8yr, 85%), [skill-change-management] Change Management (7yr, 80%), [skill-financial-modelling] Financial Modelling (1yr, 78%), [skill-executive-comms] Executive Communication (1yr, 85%)
+
+## Response Rules
+1. Answer ONLY from the data above. If the answer is not in the data, say "I don't have that information" — never invent facts, roles, dates, achievements, URLs, or contact details.
+2. Distinguish NHS employment (May 2022–present, ~4 years, all at Norfolk & Waveney ICB) from private sector (Tesco PLC, Nov 2017–May 2022, community pharmacy). Never conflate the two. GPhC registration is a professional licence, not NHS employment.
+3. When asked broad questions about tools, skills, projects, or achievements across Andy's career, aggregate from ALL roles — do not limit your answer to one position.
+4. Cite exact numbers, dates, percentages, and outcomes. Never say "approximately" or "around" when exact figures exist in the data.
+5. For detailed or list-based questions, give a thorough answer covering all relevant items. For simple questions, be concise (2-4 sentences).
 
 ## Item References
-After your answer, on a NEW line, list relevant portfolio item IDs:
-[ITEMS: id1, id2, id3]
-- IDs match the profile entries above (exp-*, skill-*, proj-*, ach-*, edu-*, action-*).
-- Only include IDs directly relevant to your answer.
-- If no items are relevant, omit the [ITEMS: ...] line entirely.`
+End your response with a single line listing relevant item IDs from the square-bracketed IDs above:
+[ITEMS: exp-deputy-head-2024, skill-python]
+Only include IDs that directly support your answer. Omit the line if none are relevant.`
 }
 
 function sleep(ms: number): Promise<void> {
   return new Promise((resolve) => setTimeout(resolve, ms))
 }
 
-async function callGemini(
+async function callLLM(
   systemPrompt: string,
   userMessage: string,
-  temperature = 0.7,
-  maxOutputTokens = 512,
+  temperature = 0.4,
+  maxTokens = 800,
 ): Promise<string> {
   const apiKey = getApiKey()
   const maxRetries = 5
 
   for (let attempt = 0; attempt < maxRetries; attempt++) {
-    const response = await fetch(
-      `${GEMINI_API_BASE}:generateContent?key=${apiKey}`,
-      {
-        method: 'POST',
-        headers: { 'Content-Type': 'application/json' },
-        body: JSON.stringify({
-          system_instruction: {
-            parts: [{ text: systemPrompt }],
-          },
-          contents: [
-            { role: 'user', parts: [{ text: userMessage }] },
-          ],
-          generationConfig: {
-            temperature,
-            maxOutputTokens,
-          },
-        }),
+    const response = await fetch(OPENROUTER_API_URL, {
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json',
+        'Authorization': `Bearer ${apiKey}`,
+        'HTTP-Referer': 'https://andycharlwood.co.uk',
+        'X-Title': 'Andy Charlwood Portfolio',
       },
-    )
+      body: JSON.stringify({
+        model: LLM_MODEL,
+        temperature,
+        max_tokens: maxTokens,
+        messages: [
+          { role: 'system', content: systemPrompt },
+          { role: 'user', content: userMessage },
+        ],
+      }),
+    })
 
     if (response.status === 429 || response.status === 503) {
       const errorBody = await response.text()
@@ -144,13 +216,13 @@ async function callGemini(
 
     if (!response.ok) {
       const errorBody = await response.text()
-      throw new Error(`Gemini API error ${response.status}: ${errorBody}`)
+      throw new Error(`OpenRouter API error ${response.status}: ${errorBody}`)
     }
 
     const data = await response.json()
-    const text = data?.candidates?.[0]?.content?.parts?.[0]?.text
+    const text = data?.choices?.[0]?.message?.content
     if (!text) {
-      throw new Error(`No text in Gemini response: ${JSON.stringify(data)}`)
+      throw new Error(`No text in OpenRouter response: ${JSON.stringify(data)}`)
     }
     return text
   }
@@ -220,7 +292,7 @@ EXPECTED ANSWER: ${expectedAnswer}
 
 ACTUAL ANSWER: ${actualAnswer}`
 
-  const rawResponse = await callGemini(scoringPrompt, userMessage, 0, 512)
+  const rawResponse = await callLLM(scoringPrompt, userMessage, 0, 512)
 
   // Extract JSON — handle code fences, preamble text, multiline responses
   const extracted = extractJson(rawResponse)
@@ -311,7 +383,7 @@ async function main() {
   const iteration = getNextIteration(resultsDir)
   console.log(`Running iteration ${iteration}...`)
 
-  // Build system prompt (same as production)
+  // Build system prompt (same as production llm.ts)
   const systemPrompt = buildSystemPrompt()
   console.log(`System prompt built (${systemPrompt.length} chars).`)
 
@@ -321,9 +393,9 @@ async function main() {
   for (const q of config.questions) {
     console.log(`\n[${q.id}] ${q.question}`)
 
-    // Get answer from Gemini
+    // Get answer from LLM
     console.log('  Getting answer...')
-    const actualAnswer = await callGemini(systemPrompt, q.question)
+    const actualAnswer = await callLLM(systemPrompt, q.question)
     console.log(`  Answer: ${actualAnswer.slice(0, 100)}...`)
 
     // Score the answer
@@ -354,7 +426,7 @@ async function main() {
   const results: BenchmarkResults = {
     iteration,
     timestamp: new Date().toISOString(),
-    model: GEMINI_MODEL,
+    model: LLM_MODEL,
     totalScore,
     maxPossibleScore: config.maxScore,
     passThreshold: config.passThreshold,

src/components/ChatWidget.tsx

@@ -3,12 +3,12 @@ import { motion, AnimatePresence } from 'framer-motion'
 import { MessageCircle, X, Send, Loader2 } from 'lucide-react'
 import {
   sendChatMessage,
-  isGeminiAvailable,
+  isLLMAvailable,
   parseItemIds,
   stripItemsSuffix,
-  GEMINI_DISPLAY_NAME,
+  LLM_DISPLAY_NAME,
   type ChatMessage,
-} from '@/lib/gemini'
+} from '@/lib/llm'
 import { buildPaletteData } from '@/lib/search'
 import type { PaletteItem, PaletteAction } from '@/lib/search'
 import { iconByType, iconColorStyles } from '@/lib/palette-icons'
@@ -64,7 +64,7 @@ export function ChatWidget({ onAction }: ChatWidgetProps) {
   const messagesEndRef = useRef<HTMLDivElement>(null)
   const inputRef = useRef<HTMLTextAreaElement>(null)
 
-  const geminiAvailable = isGeminiAvailable()
+  const llmAvailable = isLLMAvailable()
 
   // Build palette map for looking up items by ID
   const paletteMap = useMemo(() => {
@@ -264,7 +264,7 @@ export function ChatWidget({ onAction }: ChatWidgetProps) {
                       color: 'var(--text-tertiary)',
                     }}
                   >
-                    {GEMINI_DISPLAY_NAME}
+                    {LLM_DISPLAY_NAME}
                   </span>
                 </div>
                 <button
@@ -306,7 +306,7 @@ export function ChatWidget({ onAction }: ChatWidgetProps) {
                 }}
                 className="pmr-scrollbar"
               >
-                {!geminiAvailable && (
+                {!llmAvailable && (
                   <div
                     style={{
                       textAlign: 'center',
@@ -320,7 +320,7 @@ export function ChatWidget({ onAction }: ChatWidgetProps) {
                   </div>
                 )}
 
-                {geminiAvailable && messages.length === 0 && (
+                {llmAvailable && messages.length === 0 && (
                   <div style={{ display: 'flex', flexDirection: 'column', gap: '12px' }}>
                     {/* Welcome bubble — styled as assistant message */}
                     <div style={{ display: 'flex', justifyContent: 'flex-start' }}>
@@ -537,7 +537,7 @@ export function ChatWidget({ onAction }: ChatWidgetProps) {
               </div>
 
               {/* Input area */}
-              {geminiAvailable && (
+              {llmAvailable && (
                 <div
                   style={{
                     padding: '12px 16px',

src/lib/gemini.ts

@@ -1,158 +0,0 @@
-import { buildEmbeddingTexts } from '@/lib/search'
-
-export interface ChatMessage {
-  role: 'user' | 'assistant'
-  content: string
-}
-
-export const GEMINI_MODEL = 'gemini-3-flash-preview'
-export const GEMINI_DISPLAY_NAME = 'Gemini 3 Flash'
-
-const GEMINI_API_BASE = `https://generativelanguage.googleapis.com/v1beta/models/${GEMINI_MODEL}`
-
-function getApiKey(): string | undefined {
-  return import.meta.env.VITE_GEMINI_API_KEY as string | undefined
-}
-
-export function isGeminiAvailable(): boolean {
-  return !!getApiKey()
-}
-
-function buildSystemPrompt(): string {
-  const texts = buildEmbeddingTexts()
-  const cvContent = texts.map((t) => `[${t.id}] ${t.text}`).join('\n')
-
-  return `You are a helpful assistant on Andy Charlwood's portfolio website.
-
-## Profile Data
-Each entry is prefixed with its ID in square brackets.
-
-${cvContent}
-
-## Response Rules
-- Answer ONLY from the profile data above. Never invent facts, roles, dates, or achievements.
-- Be concise: 2-4 sentences. Professional and friendly tone.
-- If the answer isn't in the profile, say so honestly.
-- Do not fabricate URLs, email addresses, or contact details.
-
-## Item References
-End your response with a single line listing relevant item IDs:
-[ITEMS: exp-nhs-nwicb, skill-python]
-Only include IDs that directly support your answer. Omit the line if none are relevant.`
-}
-
-function buildRequestBody(
-  messages: ChatMessage[],
-  systemPrompt: string,
-): object {
-  const contents = messages.map((msg) => ({
-    role: msg.role === 'assistant' ? 'model' : 'user',
-    parts: [{ text: msg.content }],
-  }))
-
-  return {
-    system_instruction: {
-      parts: [{ text: systemPrompt }],
-    },
-    contents,
-    generationConfig: {
-      temperature: 0.7,
-      maxOutputTokens: 512,
-    },
-  }
-}
-
-export async function* sendChatMessage(
-  messages: ChatMessage[],
-): AsyncGenerator<string> {
-  const apiKey = getApiKey()
-  if (!apiKey) {
-    throw new Error('Gemini API key not configured')
-  }
-
-  const systemPrompt = buildSystemPrompt()
-  const body = buildRequestBody(messages, systemPrompt)
-
-  const response = await fetch(
-    `${GEMINI_API_BASE}:streamGenerateContent?alt=sse&key=${apiKey}`,
-    {
-      method: 'POST',
-      headers: { 'Content-Type': 'application/json' },
-      body: JSON.stringify(body),
-    },
-  )
-
-  if (!response.ok) {
-    throw new Error(`Gemini API error: ${response.status}`)
-  }
-
-  const reader = response.body?.getReader()
-  if (!reader) {
-    throw new Error('No response body')
-  }
-
-  const decoder = new TextDecoder()
-  let buffer = ''
-
-  try {
-    while (true) {
-      const { done, value } = await reader.read()
-      if (done) break
-
-      buffer += decoder.decode(value, { stream: true })
-
-      const lines = buffer.split('\n')
-      // Keep the last potentially incomplete line in the buffer
-      buffer = lines.pop() ?? ''
-
-      for (const line of lines) {
-        const trimmed = line.trim()
-        if (!trimmed.startsWith('data:')) continue
-
-        const jsonStr = trimmed.slice(5).trim()
-        if (!jsonStr || jsonStr === '[DONE]') continue
-
-        try {
-          const parsed = JSON.parse(jsonStr)
-          const text = parsed?.candidates?.[0]?.content?.parts?.[0]?.text
-          if (text) {
-            yield text
-          }
-        } catch {
-          // Skip malformed JSON chunks
-        }
-      }
-    }
-
-    // Process any remaining buffer
-    if (buffer.trim().startsWith('data:')) {
-      const jsonStr = buffer.trim().slice(5).trim()
-      if (jsonStr && jsonStr !== '[DONE]') {
-        try {
-          const parsed = JSON.parse(jsonStr)
-          const text = parsed?.candidates?.[0]?.content?.parts?.[0]?.text
-          if (text) {
-            yield text
-          }
-        } catch {
-          // Skip malformed final chunk
-        }
-      }
-    }
-  } finally {
-    reader.releaseLock()
-  }
-}
-
-export function parseItemIds(text: string): string[] {
-  const match = text.match(/\[ITEMS:\s*([^\]]+)\]/)
-  if (!match) return []
-  return match[1]
-    .split(',')
-    .map((id) => id.trim())
-    .filter(Boolean)
-}
-
-export function stripItemsSuffix(text: string): string {
-  return text.replace(/\n?\[ITEMS:[^\]]*\]\s*$/, '').trim()
-}

src/lib/llm.ts

@@ -0,0 +1,228 @@
+export interface ChatMessage {
+  role: 'user' | 'assistant'
+  content: string
+}
+
+export const LLM_MODEL = 'z-ai/glm-5'
+export const LLM_DISPLAY_NAME = 'GLM-5'
+
+const OPENROUTER_API_URL = 'https://openrouter.ai/api/v1/chat/completions'
+
+function getApiKey(): string | undefined {
+  return import.meta.env.VITE_OPEN_ROUTER_API_KEY as string | undefined
+}
+
+export function isLLMAvailable(): boolean {
+  return !!getApiKey()
+}
+
+export function buildSystemPrompt(): string {
+  return `You are a helpful assistant on Andy Charlwood's portfolio website. Answer questions about Andy's professional background using ONLY the information below.
+
+## Profile
+Andy Charlwood — MPharm, GPhC Registered Pharmacist. Norwich, UK.
+Healthcare leader combining clinical pharmacy with Python, SQL, and data analytics (self-taught). Leading population health analytics for NHS Norfolk & Waveney ICB, serving 1.2M people. Specialises in prescribing data at scale — financial modelling, algorithm design, pathway development. Identified efficiency programmes worth £14.6M+ through automated analysis.
+
+## Employment Timeline (IMPORTANT)
+- **NHS employment**: May 2022–present (all roles at NHS Norfolk & Waveney ICB). Total NHS service: ~4 years.
+- **Private sector**: Nov 2017–May 2022 at Tesco PLC (community pharmacy). This was NOT NHS employment.
+- GPhC registration (Aug 2016) is a professional licence, NOT an employer or NHS role.
+
+## Career History
+
+### [exp-interim-head-2025] Interim Head, Population Health & Data Analysis
+NHS Norfolk & Waveney ICB | May–Nov 2025
+Led population health initiatives and data-driven medicines optimisation, reporting to Associate Director of Pharmacy with accountability to CMO.
+- Identified £14.6M efficiency programme; achieved over-target performance by October 2025
+- Built Python switching algorithm: real-world GP prescribing data, 14,000 patients, £2.6M annual savings (£2M on target), compressed months into 3 days
+- Novel GP payment system linking rewards to savings; 50% prescribing reduction within 2 months
+- Presented to CMO bimonthly; led transformation to patient-level SQL analytics
+
+### [exp-deputy-head-2024] Deputy Head, Population Health & Data Analysis
+NHS Norfolk & Waveney ICB | Jul 2024–Present (substantive role)
+Data analytics strategy for medicines optimisation from real-world GP prescribing data.
+- Managed £220M prescribing budget with forecasting models for proactive financial planning
+- Created comprehensive dm+d medicines data table: standardised strengths, morphine equivalents, Anticholinergic Burden scoring — single source of truth for all medicines analytics
+- Led DOAC switching financial modelling: interactive dashboard with rebate mechanics, patent expiry timelines
+- Renegotiated pharmaceutical rebate terms ahead of patent expiry
+- Tirzepatide commissioning (NICE TA1026): financial projections, cohort identification; authored executive paper advocating primary care model, driving system shift to GP-led delivery
+- Built Python controlled drug monitoring: oral morphine equivalents across all opioid prescriptions, patient-level tracking, high-risk identification, diversion detection
+- Improved team data fluency through training and self-serve tools
+
+### [exp-high-cost-drugs-2022] High-Cost Drugs & Interface Pharmacist
+NHS Norfolk & Waveney ICB | May 2022–Jul 2024
+Led NICE TA implementation and high-cost drug pathways across the ICS. Pathways spanning: rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, migraine.
+- Blueteq automation: 70% form reduction, 200 hours immediate savings, 7–8 hours ongoing weekly gains
+- Integrated Blueteq with secondary care databases for accurate high-cost drug spend tracking
+- Python Sankey chart tool for patient pathway visualisation and trust compliance auditing
+
+### [exp-pharmacy-manager-2017] Pharmacy Manager
+Tesco PLC (private sector, NOT NHS) | Nov 2017–May 2022
+Community pharmacy with full operational autonomy (100-hour contract). LPC representative for Norfolk.
+- Asthma screening process adopted nationally (~300 branches): reduced pharmacist time 60→6 hours/store/month, ~£1M revenue
+- Leadership training: Created national induction training plan and eLearning modules for Tesco pharmacy staff
+- Leadership development: Supervised two staff through NVQ3 to pharmacy technician registration; full HR responsibilities
+
+## Projects
+
+### [proj-inv-pharmetrics] PharMetrics Interactive Platform (2024, Live)
+Real-time medicines expenditure dashboard for NHS decision-makers. Tech: Power BI, SQL, DAX. Tracks £220M prescribing budget.
+
+### [proj-inv-switching-algorithm] Patient Switching Algorithm (2025, Complete)
+Python algorithm using GP prescribing data to auto-identify patients for cost-effective alternatives. Tech: Python, Pandas, SQL. 14,000 patients, £2.6M annual savings, novel GP payment system.
+
+### [proj-inv-blueteq-gen] Blueteq Generator (2023, Complete)
+Automated Blueteq prior approval form creation. Tech: Python, SQL. 70% form reduction, 200 hours immediate savings, 7–8 hours ongoing weekly gains.
+
+### [proj-inv-cd-monitoring] CD Monitoring System (2024, Complete)
+Controlled drug monitoring calculating oral morphine equivalents (OME) across all opioid prescriptions. Tech: Python, SQL. Patient-level tracking, high-risk identification, diversion detection.
+
+### [proj-inv-sankey-tool] Sankey Chart Analysis Tool (2023, Complete)
+Patient journey visualisation through high-cost drug pathways. Tech: Python, Matplotlib, SQL. Trust compliance auditing.
+
+## Education
+
+### [edu-0] NHS Mary Seacole Programme (2018)
+NHS Leadership Academy. Score: 78%. Covers change management, healthcare leadership, system-level thinking.
+
+### [edu-1] MPharm (Hons) 2:1 — University of East Anglia (2011–2015)
+4-year integrated Master's degree. Research project on drug delivery and cocrystals: 75.1% (Distinction).
+
+### [edu-2] A-Levels — Highworth Grammar School (2009–2011)
+Mathematics A*, Chemistry B, Politics C.
+
+### [edu-3] GPhC Registration — General Pharmaceutical Council (August 2016–Present)
+Professional registration required to practise as a pharmacist in Great Britain.
+
+## Skills
+Technical: [skill-data-analysis] Data Analysis (9yr, 95%), [skill-python] Python (6yr, 90%), [skill-sql] SQL (7yr, 88%), [skill-power-bi] Power BI (5yr, 92%), [skill-javascript-typescript] JavaScript/TypeScript (3yr, 70%), [skill-excel] Excel (9yr, 85%), [skill-algorithm-design] Algorithm Design (3yr, 82%), [skill-data-pipelines] Data Pipelines (2yr, 75%)
+Domain: [skill-medicines-optimisation] Medicines Optimisation (9yr, 95%), [skill-population-health] Population Health (3yr, 90%), [skill-nice-ta] NICE TA Implementation (3yr, 92%), [skill-health-economics] Health Economics (3yr, 80%), [skill-clinical-pathways] Clinical Pathways (3yr, 88%), [skill-controlled-drugs] Controlled Drugs (1yr, 85%)
+Leadership: [skill-budget-management] Budget Management (1yr, 90%), [skill-stakeholder-engagement] Stakeholder Engagement (3yr, 88%), [skill-pharma-negotiation] Pharmaceutical Negotiation (1yr, 82%), [skill-team-development] Team Development (8yr, 85%), [skill-change-management] Change Management (7yr, 80%), [skill-financial-modelling] Financial Modelling (1yr, 78%), [skill-executive-comms] Executive Communication (1yr, 85%)
+
+## Response Rules
+1. Answer ONLY from the data above. If the answer is not in the data, say "I don't have that information" — never invent facts, roles, dates, achievements, URLs, or contact details.
+2. Distinguish NHS employment (May 2022–present, ~4 years, all at Norfolk & Waveney ICB) from private sector (Tesco PLC, Nov 2017–May 2022, community pharmacy). Never conflate the two. GPhC registration is a professional licence, not NHS employment.
+3. When asked broad questions about tools, skills, projects, or achievements across Andy's career, aggregate from ALL roles — do not limit your answer to one position.
+4. Cite exact numbers, dates, percentages, and outcomes. Never say "approximately" or "around" when exact figures exist in the data.
+5. For detailed or list-based questions, give a thorough answer covering all relevant items. For simple questions, be concise (2-4 sentences).
+
+## Item References
+End your response with a single line listing relevant item IDs from the square-bracketed IDs above:
+[ITEMS: exp-deputy-head-2024, skill-python]
+Only include IDs that directly support your answer. Omit the line if none are relevant.`
+}
+
+function buildRequestBody(
+  messages: ChatMessage[],
+  systemPrompt: string,
+): object {
+  return {
+    model: LLM_MODEL,
+    stream: true,
+    temperature: 0.4,
+    max_tokens: 800,
+    messages: [
+      { role: 'system', content: systemPrompt },
+      ...messages.map((msg) => ({
+        role: msg.role,
+        content: msg.content,
+      })),
+    ],
+  }
+}
+
+export async function* sendChatMessage(
+  messages: ChatMessage[],
+): AsyncGenerator<string> {
+  const apiKey = getApiKey()
+  if (!apiKey) {
+    throw new Error('LLM API key not configured')
+  }
+
+  const systemPrompt = buildSystemPrompt()
+  const body = buildRequestBody(messages, systemPrompt)
+
+  const response = await fetch(OPENROUTER_API_URL, {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      'Authorization': `Bearer ${apiKey}`,
+      'HTTP-Referer': window.location.origin,
+      'X-Title': 'Andy Charlwood Portfolio',
+    },
+    body: JSON.stringify(body),
+  })
+
+  if (!response.ok) {
+    throw new Error(`LLM API error: ${response.status}`)
+  }
+
+  const reader = response.body?.getReader()
+  if (!reader) {
+    throw new Error('No response body')
+  }
+
+  const decoder = new TextDecoder()
+  let buffer = ''
+
+  try {
+    while (true) {
+      const { done, value } = await reader.read()
+      if (done) break
+
+      buffer += decoder.decode(value, { stream: true })
+
+      const lines = buffer.split('\n')
+      buffer = lines.pop() ?? ''
+
+      for (const line of lines) {
+        const trimmed = line.trim()
+        if (!trimmed.startsWith('data:')) continue
+
+        const jsonStr = trimmed.slice(5).trim()
+        if (!jsonStr || jsonStr === '[DONE]') continue
+
+        try {
+          const parsed = JSON.parse(jsonStr)
+          const text = parsed?.choices?.[0]?.delta?.content
+          if (text) {
+            yield text
+          }
+        } catch {
+          // Skip malformed JSON chunks
+        }
+      }
+    }
+
+    // Process any remaining buffer
+    if (buffer.trim().startsWith('data:')) {
+      const jsonStr = buffer.trim().slice(5).trim()
+      if (jsonStr && jsonStr !== '[DONE]') {
+        try {
+          const parsed = JSON.parse(jsonStr)
+          const text = parsed?.choices?.[0]?.delta?.content
+          if (text) {
+            yield text
+          }
+        } catch {
+          // Skip malformed final chunk
+        }
+      }
+    }
+  } finally {
+    reader.releaseLock()
+  }
+}
+
+export function parseItemIds(text: string): string[] {
+  const match = text.match(/\[ITEMS:\s*([^\]]+)\]/)
+  if (!match) return []
+  return match[1]
+    .split(',')
+    .map((id) => id.trim())
+    .filter(Boolean)
+}
+
+export function stripItemsSuffix(text: string): string {
+  return text.replace(/\n?\[ITEMS:[^\]]*\]\s*$/, '').trim()
+}

src/lib/search.ts

@@ -247,25 +247,68 @@ export function groupBySection(items: PaletteItem[]): Array<{ section: PaletteSe
 export function buildEmbeddingTexts(): Array<{ id: string; text: string }> {
   const texts: Array<{ id: string; text: string }> = []
 
-  // Consultations (Experience)
+  // Consultations (Experience) — enriched with plan outcomes, employer classification, clinical specialties
   consultations.forEach((c) => {
+    const isNHS = c.organization.includes('NHS') || c.organization.includes('ICB')
+    const employer = isNHS
+      ? `NHS employer: ${c.organization}`
+      : `Private sector employer: ${c.organization} (not NHS)`
     const examBullets = c.examination.join('. ')
+    const planOutcomes = c.plan.join('. ')
     const codedDescriptions = c.codedEntries.map(e => e.description).join('. ')
+
+    // Role-specific enrichment for clinical specialties and methodology
+    let roleContext = ''
+    if (c.id === 'high-cost-drugs-2022') {
+      roleContext = ' Clinical specialties covered: rheumatology, ophthalmology (wet AMD, DMO, RVO), dermatology, gastroenterology, neurology, and migraine. Wrote most of the system\'s high-cost drug pathways, implementing NICE technology appraisals while balancing legal requirements against financial costs and local clinical preferences.'
+    } else if (c.id === 'deputy-head-2024') {
+      roleContext = ' Created dm+d medicines data table integrating all dictionary of medicines and devices products with standardised strengths, morphine equivalents, and Anticholinergic Burden scoring — single source of truth for all medicines analytics. Supported tirzepatide commissioning (NICE TA1026) with financial projections and authored executive paper advocating primary care model, driving system shift to GP-led delivery.'
+    } else if (c.id === 'interim-head-2025') {
+      roleContext = ' Built Python switching algorithm using real-world GP prescribing data to identify patients eligible for cost-effective alternatives — compressed months of manual analysis into 3 days. Created novel GP payment system linking incentive rewards to prescribing savings.'
+    } else if (c.id === 'pharmacy-manager-2017') {
+      roleContext = ' Community pharmacy role at Tesco PLC, a private sector employer. Served as Local Pharmaceutical Committee (LPC) representative for Norfolk. Full HR responsibilities including recruitment, performance management, grievances.'
+    }
+
     texts.push({
       id: `exp-${c.id}`,
-      text: `${c.role} at ${c.organization}, ${c.duration}. ${c.history} Key achievements: ${examBullets}. ${codedDescriptions}.`,
+      text: `${c.role} at ${c.organization}, ${c.duration}. ${employer}. ${c.history} Key achievements: ${examBullets}. Outcomes: ${planOutcomes}. ${codedDescriptions}.${roleContext}`,
     })
   })
 
-  // Skills
+  // Skills — enriched with role context and practical application
+  const skillContextMap: Record<string, string> = {
+    'data-analysis': 'Applied across NHS medicines optimisation, identifying £14.6M efficiency programme. Used for prescribing pattern analysis, budget forecasting, and population health analytics serving 1.2M people.',
+    'python': 'Used to build switching algorithms (14,000 patients, £2.6M savings), controlled drug monitoring systems, Blueteq form automation, and Sankey chart visualisation tools. Self-taught.',
+    'sql': 'Core tool for patient-level analytics, dm+d data integration, and transforming practice-level data to patient-level SQL analysis. Used across all NHS data roles.',
+    'power-bi': 'Built PharMetrics interactive dashboard tracking £220M prescribing budget. Created dashboards used by 200+ clinicians and commissioners across Norfolk & Waveney ICB.',
+    'javascript-typescript': 'Used for web development including this portfolio website. Built with React, TypeScript, and Vite.',
+    'excel': 'Used for financial modelling, data validation, and ad-hoc analysis. Foundational tool across all roles from community pharmacy to NHS ICB.',
+    'algorithm-design': 'Designed patient switching algorithm and automated incentive scheme analysis. Applied to real-world GP prescribing data at population scale.',
+    'data-pipelines': 'Built automated data processing pipelines for medicines analytics, enabling self-serve models for team data fluency.',
+    'medicines-optimisation': 'Core domain expertise spanning community pharmacy through to NHS ICB-level population health. Led efficiency programmes worth £14.6M+.',
+    'population-health': 'Leading population health analytics for 1.2M people across Norfolk & Waveney ICS. Developing patient-level datasets from real-world GP prescribing data.',
+    'nice-ta': 'Led NICE technology appraisal implementation across high-cost drug pathways. Covered rheumatology, ophthalmology, dermatology, gastroenterology, neurology, and migraine.',
+    'health-economics': 'Financial modelling for DOAC switching programmes, tirzepatide commissioning, and pharmaceutical rebate negotiations.',
+    'clinical-pathways': 'Wrote most of the Norfolk & Waveney ICB high-cost drug pathways. Created Sankey chart tool for patient pathway visualisation and trust compliance auditing.',
+    'controlled-drugs': 'Built Python-based population-scale monitoring system calculating oral morphine equivalents (OME) across all opioid prescriptions. Enables high-risk patient identification and potential diversion detection.',
+    'budget-management': 'Managed £220M NHS prescribing budget with sophisticated forecasting models, variance analysis, and monthly financial reporting to the ICB executive team.',
+    'stakeholder-engagement': 'Presented to Chief Medical Officer bimonthly. Engaged with GP practices, trusts, commissioners, and pharmaceutical companies across the integrated care system.',
+    'pharma-negotiation': 'Renegotiated pharmaceutical rebate terms ahead of patent expiry, securing improved commercial position for the ICB.',
+    'team-development': 'Improved team data fluency through training and documentation. Supervised staff through NVQ3 to pharmacy technician registration. Created national induction training at Tesco.',
+    'change-management': 'Completed NHS Mary Seacole Programme (2018, 78%). Led transformation to patient-level SQL analytics and self-serve analytical models.',
+    'financial-modelling': 'Built interactive DOAC switching dashboard with rebate mechanics, workforce constraints, and patent expiry timelines. Financial projections for tirzepatide commissioning.',
+    'executive-comms': 'Authored executive papers for ICB board including tirzepatide commissioning advocacy. Presented evidence-based recommendations to CMO bimonthly.',
+  }
+
   skills.forEach((skill) => {
+    const context = skillContextMap[skill.id] ?? ''
     texts.push({
       id: `skill-${skill.id}`,
-      text: `${skill.name} is a ${skill.category.toLowerCase()} skill used ${skill.frequency.toLowerCase()}, with ${skill.proficiency}% proficiency and ${skill.yearsOfExperience} years of experience since ${skill.startYear}.`,
+      text: `${skill.name} is a ${skill.category.toLowerCase()} skill used ${skill.frequency.toLowerCase()}, with ${skill.proficiency}% proficiency and ${skill.yearsOfExperience} years of experience since ${skill.startYear}. ${context}`,
     })
   })
 
-  // KPI-backed Achievements
+  // KPI-backed Achievements — enriched with full story context and outcomes
   const achievementMap: Array<{ id: string; title: string; subtitle: string; kpiId: string }> = [
     { id: 'ach-0', title: '£14.6M Efficiency Savings Identified', subtitle: 'Data-driven prescribing interventions', kpiId: 'savings' },
     { id: 'ach-1', title: '£220M Budget Oversight', subtitle: 'Full analytical accountability to ICB board', kpiId: 'budget' },
@@ -275,26 +318,40 @@ export function buildEmbeddingTexts(): Array<{ id: string; text: string }> {
 
   achievementMap.forEach((entry) => {
     const kpi = kpis.find(k => k.id === entry.kpiId)
-    const storyContext = kpi?.story
-      ? ` ${kpi.story.context} ${kpi.story.role} Outcomes: ${kpi.story.outcomes.join('. ')}.`
-      : ''
+    const explanation = kpi?.explanation ?? ''
+    const storyParts: string[] = []
+    if (kpi?.story) {
+      storyParts.push(kpi.story.context)
+      storyParts.push(kpi.story.role)
+      if (kpi.story.period) storyParts.push(`Period: ${kpi.story.period}.`)
+      storyParts.push(`Outcomes: ${kpi.story.outcomes.join('. ')}.`)
+    }
     texts.push({
       id: entry.id,
-      text: `Achievement: ${entry.title}. ${entry.subtitle}. ${kpi?.explanation ?? ''}${storyContext}`,
+      text: `Achievement: ${entry.title}. ${entry.subtitle}. ${explanation} ${storyParts.join(' ')}`,
     })
   })
 
-  // Investigations (Active Projects)
+  // Investigations (Active Projects) — enriched with role context and cross-references
+  const projectContextMap: Record<string, string> = {
+    'inv-pharmetrics': 'Built during Deputy Head role at NHS Norfolk & Waveney ICB. Provides self-serve analytics for budget holders across the integrated care system. Live at medicines.charlwood.xyz.',
+    'inv-switching-algorithm': 'Built during Interim Head role at NHS Norfolk & Waveney ICB. Uses real-world GP prescribing data to auto-identify patients on expensive drugs suitable for cost-effective alternatives. Compressed months of manual analysis into 3 days. Includes novel GP payment system linking incentive rewards to prescribing savings.',
+    'inv-blueteq-gen': 'Built during High-Cost Drugs & Interface Pharmacist role at NHS Norfolk & Waveney ICB. Automates prior approval form creation for high-cost drug pathways spanning rheumatology, ophthalmology, dermatology, gastroenterology, neurology, and migraine.',
+    'inv-cd-monitoring': 'Built during Deputy Head role at NHS Norfolk & Waveney ICB. Calculates oral morphine equivalents (OME) across all opioid prescriptions at population scale. Enables previously impossible population-level controlled drug analysis. Related to controlled drugs skill.',
+    'inv-sankey-tool': 'Built during High-Cost Drugs & Interface Pharmacist role at NHS Norfolk & Waveney ICB. Visualises patient journeys through high-cost drug pathways. Enables trust-level compliance auditing across multiple clinical specialties.',
+  }
+
   investigations.forEach((inv) => {
     const techList = inv.techStack.join(', ')
     const resultList = inv.results.join('. ')
+    const context = projectContextMap[inv.id] ?? ''
     texts.push({
       id: `proj-${inv.id}`,
-      text: `Project: ${inv.name}. ${inv.methodology} Tech stack: ${techList}. Results: ${resultList}.`,
+      text: `Project: ${inv.name} (${inv.status}, ${inv.requestedYear}). ${inv.methodology} Tech stack: ${techList}. Results: ${resultList}. ${context}`,
     })
   })
 
-  // Education
+  // Education — enriched with research grades and specific subject details
   const educationItems: Array<{ id: string; docId: string; fallbackTitle: string; fallbackSub: string }> = [
     { id: 'edu-0', docId: 'doc-mary-seacole', fallbackTitle: 'NHS Leadership Academy — Mary Seacole Programme', fallbackSub: 'NHS Leadership Academy · 2018' },
     { id: 'edu-1', docId: 'doc-mpharm', fallbackTitle: 'MPharm (Hons) — 2:1', fallbackSub: 'University of East Anglia · 2011–2015' },
@@ -306,10 +363,11 @@ export function buildEmbeddingTexts(): Array<{ id: string; text: string }> {
     const doc = documents.find(d => d.id === entry.docId)
     if (doc) {
       const research = doc.researchDetail ? ` Research: ${doc.researchDetail}.` : ''
+      const researchGrade = doc.researchGrade ? ` Research grade: ${doc.researchGrade}.` : ''
       const classification = doc.classification ? ` Classification: ${doc.classification}.` : ''
       texts.push({
         id: entry.id,
-        text: `Education: ${doc.title}. ${doc.type} from ${doc.institution ?? doc.source}, ${doc.duration ?? doc.date}.${classification}${research} ${doc.notes ?? ''}`,
+        text: `Education: ${doc.title}. ${doc.type} from ${doc.institution ?? doc.source}, ${doc.duration ?? doc.date}.${classification}${research}${researchGrade} ${doc.notes ?? ''}`,
       })
     } else {
       texts.push({