HighCostDrugsDemo/progress.txt

# Progress Log — Reflex → Dash Migration

## Project Context

Migrating the HCD Analysis frontend from Reflex to Dash (Plotly) + Dash Mantine Components. Pipeline/analysis logic in `src/` is untouched, but shared utilities (data queries, figure construction) should be added TO `src/` so Dash callbacks call into them rather than duplicating code.

**Previous state**: Fully working Reflex app with pre-computed pathway architecture (SQLite), dual chart types (directory + indication), drug-aware indication matching. All pipeline work is done.

**New goal**: Replace Reflex with Dash for better control over layout, CSS, and component behavior. Add a dmc.Drawer-based "card browser" for drug/indication selection organized by clinical directorate.

## Key Data Patterns

### SQLite pathway_nodes table
- ~3,600 rows across 12 datasets (6 date filters × 2 chart types)
- Key columns: `parents, ids, labels, level, value, cost, costpp, cost_pp_pa, colour, first_seen, last_seen, first_seen_parent, last_seen_parent, average_spacing, trust_name, directory, drug_sequence, chart_type, date_filter_id`
- Level 0 = Root, Level 1 = Trust, Level 2 = Directory/Indication, Level 3 = Drug, Level 4+ = Pathway
- `chart_type`: "directory" or "indication"
- `date_filter_id`: "all_6mo" (default), "all_12mo", "1yr_6mo", "1yr_12mo", "2yr_6mo", "2yr_12mo"
- UNIQUE constraint: (date_filter_id, chart_type, ids)

### DimSearchTerm.csv (for card browser)
- Located at `data/DimSearchTerm.csv`
- Columns: Search_Term, CleanedDrugName (pipe-separated drug fragments), PrimaryDirectorate
- ~165 rows; some Search_Terms appear twice (e.g., "diabetes" under DIABETIC MEDICINE and OPHTHALMOLOGY)
- Drug fragments are UPPERCASE substrings matched against standardized drug names
- SEARCH_TERM_MERGE_MAP in `src/data_processing/diagnosis_lookup.py` merges asthma variants: {"allergic asthma": "asthma", "severe persistent allergic asthma": "asthma"}

### Data loading logic to extract
- `pathways_app/pathways_app.py` lines 407-488: `load_data()` — loads available drugs, directorates, indications, total records, last updated from SQLite
- `pathways_app/pathways_app.py` lines 490-642: `load_pathway_data()` — queries pathway_nodes with date_filter_id + chart_type + optional drug/directory filters
- `pathways_app/pathways_app.py` lines 769-920: `icicle_figure` — builds go.Icicle with 10-field customdata, NHS colorscale, texttemplate, hovertemplate

### CSS from 01_nhs_classic.html
- Lines 8-314 contain the full CSS (copy to dash_app/assets/nhs.css)
- Google Fonts: `Source Sans 3` weights 300,400,600,700,900
- CSS variables: `--nhs-blue: #005EB8`, `--nhs-dark-blue: #003087`, `--nhs-light-blue: #41B6E6`, etc.
- Key classes: `.top-header`, `.sidebar`, `.main`, `.kpi-row`, `.kpi-card`, `.filter-bar`, `.toggle-pill`, `.chart-card`, `.chart-tab`, `.page-footer`
- Remove `.icicle`, `.icicle__row`, `.icicle__cell`, `.lvl-*` classes — those are mock chart CSS, Plotly handles the real chart

### Dash-specific patterns
- State via `dcc.Store`: 3 stores (app-state, chart-data, reference-data)
- Callbacks: unidirectional flow (filter change → app-state → chart-data → UI components)
- DMC components: `dmc.MantineProvider` wraps everything, `dmc.Drawer` for card browser
- Pattern-matching callbacks: `{"type": "drug-chip", "index": drug_name}` for dynamic drug chip selection
- Assets auto-served from `dash_app/assets/` directory

### Database path from dash_app/
- From `dash_app/data/queries.py`: `Path(__file__).resolve().parents[2] / "data" / "pathways.db"`
- From `dash_app/data/card_browser.py`: same pattern for `data/DimSearchTerm.csv`

### Existing src/ code to build on (not duplicate)
- `src/visualization/plotly_generator.py` already has `create_icicle_figure(ice_df, title)` that takes a DataFrame with columns like `"First seen"`, `"Last seen (Parent)"` (with spaces). The Reflex AppState `icicle_figure` (pathways_app.py:769) takes list-of-dicts with keys like `first_seen`, `last_seen_parent` (underscores). For Dash, add a NEW function `create_icicle_from_nodes(nodes, title)` that accepts list-of-dicts. Don't modify the existing DataFrame-based function.
- `src/data_processing/database.py` has `DatabaseManager` class. Add standalone query functions here (or a new `pathway_queries.py`) so Dash and Reflex share the same SQL.
- The existing `create_icicle_figure` uses Viridis colorscale; the Reflex version uses NHS blue gradient. The Dash version should use the NHS blue gradient from pathways_app.py.

### Architecture decision: shared code in src/
- Pipeline/analysis logic is OFF LIMITS: pathway_pipeline.py, transforms.py, diagnosis_lookup.py (matching), pathway_analyzer.py, statistics.py, refresh_pathways.py
- Shared utilities are ENCOURAGED to add to src/: plotly_generator.py (new dict-based function), database.py (query functions)
- dash_app/data/queries.py should be a thin wrapper that calls into src/

## Iteration Log

## Iteration 1 — 2026-02-06
### Task: Phase 0 — Tasks 0.1 + 0.2 (Project Scaffolding + CSS Extraction)
### Why this task:
- This is the first iteration. Phase 0 scaffolding is the foundation everything else depends on.
- Tasks 0.1 and 0.2 are tightly coupled (CSS needs the assets dir from 0.1), so both done together.
### Status: COMPLETE
### What was done:
- Created `dash_app/` directory with subdirectories: `assets/`, `data/`, `components/`, `callbacks/`, `utils/`
- Created `__init__.py` in all packages
- Created `run_dash.py` entry point at project root
- Updated `pyproject.toml` with `dash>=2.14.0` and `dash-mantine-components>=0.14.0`
- Ran `uv sync` — installed Dash 4.0.0 and DMC 2.5.1 (newer than plan expected)
- Created `dash_app/app.py` with `MantineProvider` wrapper, 3 `dcc.Store` components (app-state, chart-data, reference-data), and placeholder layout
- Extracted CSS from `01_nhs_classic.html` into `dash_app/assets/nhs.css` (7.5KB)
  - Added Google Fonts `@import` for Source Sans 3
  - Removed mock icicle chart CSS (`.icicle`, `.icicle__row`, `.icicle__cell`, `.lvl-*`)
  - Kept all real component CSS: header, sidebar, KPI, filter bar, chart card, footer, responsive
### Validation results:
- Tier 1 (Code): `python -c "from dash_app.app import app"` — OK, layout type is MantineProvider
- Tier 1 (App starts): `python run_dash.py` — serves at http://127.0.0.1:8050/
- CSS: nhs.css (7,491 bytes) found in Dash assets folder, auto-served
### Files changed:
- `pyproject.toml` — added dash + dmc dependencies
- `run_dash.py` — new entry point
- `dash_app/__init__.py` — new package
- `dash_app/app.py` — new app with MantineProvider + stores + placeholder
- `dash_app/assets/nhs.css` — extracted from 01_nhs_classic.html
- `dash_app/data/__init__.py` — empty package
- `dash_app/components/__init__.py` — empty package
- `dash_app/callbacks/__init__.py` — empty package
- `dash_app/utils/__init__.py` — empty package
### Committed: 1c3ece6 "feat: create dash_app skeleton with nhs.css and MantineProvider (Phase 0)"
### Patterns discovered:
- Dash 4.0.0 and DMC 2.5.1 installed (plan said 2.x and 0.14.x). The API is compatible — `MantineProvider`, `Drawer`, `Accordion`, `Chip` all available. DMC 2.x is based on Mantine v7.
- `dmc.MantineProvider(children=[...])` works for wrapping the layout in both DMC versions.
- `suppress_callback_exceptions=True` needed in Dash app since callbacks reference IDs not yet in layout during phased development.
### Next iteration should:
- Start Phase 1: Task 1.1 — Create shared data access functions
- Read `pathways_app/pathways_app.py` lines 407-642 to extract `load_data()` and `load_pathway_data()` logic
- Create shared functions in `src/data_processing/pathway_queries.py` (database.py is already used for DatabaseManager)
- Create thin wrapper `dash_app/data/queries.py` that calls the shared functions
- Verify with `python -c "from dash_app.data.queries import load_initial_data; print(load_initial_data())"`
### Blocked items:
- None

## Iteration 2 — 2026-02-06
### Task: Phase 1 — Task 1.1 (Create shared data access functions)
### Why this task:
- Phase 0 complete in iteration 1; Phase 1 is next in dependency order
- progress.txt from iteration 1 explicitly recommended this task
- All UI components and callbacks (Phases 2-5) depend on having data access working
### Status: COMPLETE
### What was done:
- Created `src/data_processing/pathway_queries.py` with two shared functions:
  - `load_initial_data(db_path)` — returns available drugs (42), directorates (14), indications (32), total_records, last_updated
  - `load_pathway_nodes(db_path, filter_id, chart_type, selected_drugs, selected_directorates)` — returns nodes list, unique_patients, total_drugs, total_cost, last_updated
- Both functions extracted directly from AppState methods in pathways_app.py (lines 407-642), with Reflex `self.*` references replaced by function parameters
- All return values are plain dicts/lists — JSON-serializable for dcc.Store
- Created thin wrapper `dash_app/data/queries.py` that resolves DB_PATH and delegates to shared functions
- Used separate file (pathway_queries.py) rather than adding to database.py because database.py is connection management (240 lines), queries are a distinct concern
### Validation results:
- Tier 1 (Code): `python -c "from dash_app.data.queries import load_initial_data"` — OK (requires uv run for .pth file)
- Tier 1 (App starts): `from dash_app.app import app` — OK, layout type is MantineProvider
- Tier 3 (Functional):
  - `load_initial_data()`: 42 drugs, 14 directorates, 32 indications, last_updated=2026-02-06T00:08:55
  - `load_pathway_data("all_6mo", "directory")`: 293 nodes, 11,118 patients, 39 drugs, £130.5M cost
  - `load_pathway_data("all_6mo", "indication")`: 438 nodes, 11,252 patients
  - `load_pathway_data("all_6mo", "directory", selected_drugs=["ADALIMUMAB"])`: 70 nodes (drug filter works)
### Files changed:
- `src/data_processing/pathway_queries.py` — NEW: shared query functions
- `dash_app/data/queries.py` — NEW: thin Dash wrapper with DB_PATH resolution
- `IMPLEMENTATION_PLAN.md` — Task 1.1 marked [x]
### Committed: b71748f "feat: add shared pathway query functions for Dash data access (Task 1.1)"
### Patterns discovered:
- `src/` is on sys.path only when using `uv run` (via .pth file created by setup_dev.py). Running `python` directly won't find `data_processing` module. Always use `uv run python` for testing.
- `total_records` from `pathway_refresh_log` returns 0 — the refresh log's `source_row_count` field appears empty despite `completed_at` having a value. This is cosmetic — the KPI can use `unique_patients` from chart-data instead.
- Drug filtering correctly includes nodes with NULL drug_sequence (root, trust, directory levels) alongside matching drug nodes. Root node patient count becomes 0 when drug filter is active — this matches Reflex behavior.
### Next iteration should:
- Start Task 1.2 — Build directorate card tree from DimSearchTerm.csv
- Create `dash_app/data/card_browser.py` with `build_directorate_tree()` and `get_all_drugs()`
- Read `data/DimSearchTerm.csv` to understand the data format
- Import SEARCH_TERM_MERGE_MAP from `data_processing.diagnosis_lookup` for asthma normalization
- Remember: drug fragments in CleanedDrugName are UPPERCASE substrings, not exact matches
### Blocked items:
- None

## Iteration 3 — 2026-02-06
### Task: Phase 1 — Task 1.2 (Build directorate card tree from DimSearchTerm.csv)
### Why this task:
- Phase 1 data access layer needs both query functions (done in 1.1) AND the card browser data (this task)
- Progress.txt from iteration 2 explicitly recommended this task
- The drawer component (Phase 4) depends on this data structure
### Status: COMPLETE
### What was done:
- Created `dash_app/data/card_browser.py` with two functions:
  - `build_directorate_tree()` — reads DimSearchTerm.csv, applies SEARCH_TERM_MERGE_MAP, returns nested dict: {PrimaryDirectorate: {Search_Term: [drug_fragment, ...]}}
  - `get_all_drugs()` — delegates to `load_initial_data()` from queries.py (no SQL duplication)
- SEARCH_TERM_MERGE_MAP correctly imported from `data_processing.diagnosis_lookup`
- Asthma variants ("allergic asthma", "severe persistent allergic asthma") merge into single "asthma" entry with combined/deduplicated drug fragments
- Tree uses sets during construction to deduplicate drug fragments from merged terms, then converts to sorted lists
### Validation results:
- Tier 1 (Code): `from dash_app.data.card_browser import build_directorate_tree` — OK
- Tier 1 (App starts): `from dash_app.app import app` — OK, layout type is MantineProvider
- Tier 3 (Functional):
  - `build_directorate_tree()`: 19 directorates, correctly nested dict
  - CARDIOLOGY has 10 indications, CLINICAL HAEMATOLOGY has many more
  - Asthma merge verified: only "asthma" present (not "allergic asthma" or "severe persistent allergic asthma"), OMALIZUMAB included in merged drug list
  - `get_all_drugs()`: 42 drugs from pathway_nodes level 3
### Files changed:
- `dash_app/data/card_browser.py` — NEW: directorate tree builder + drug list accessor
- `IMPLEMENTATION_PLAN.md` — Task 1.2 marked [x]
### Committed: 7654942 "feat: add directorate card tree builder for drug browser drawer (Task 1.2)"
### Patterns discovered:
- `get_all_drugs()` uses a lazy import (`from dash_app.data.queries import load_initial_data`) to avoid circular imports since both modules are in `dash_app/data/`
- Drug fragments in DimSearchTerm.csv are already UPPERCASE in the CSV, but `.upper()` is applied defensively
- "diabetes" appears twice in DimSearchTerm.csv (DIABETIC MEDICINE and OPHTHALMOLOGY) — this is handled naturally by the directorate grouping (same search_term under two different directorates)
### Next iteration should:
- Start Phase 2: Task 2.1 — Header + sidebar components
- Read `01_nhs_classic.html` to get the exact HTML structure, CSS classes, and SVG icons
- Create `dash_app/components/header.py` with `make_header()` function
- Create `dash_app/components/sidebar.py` with `make_sidebar()` function
- Use CSS classes from nhs.css (`.top-header`, `.sidebar`, etc.) — NOT inline styles
- Header needs placeholder `html.Span` with IDs for callback updates: `id="header-record-count"`, `id="header-last-updated"`
- Sidebar needs click targets for drawer open (Drug Selection, Indications items)
### Blocked items:
- None

## Iteration 4 — 2026-02-06
### Task: Phase 2 — Task 2.1 (Header + sidebar components)
### Why this task:
- Phases 0 and 1 complete. Phase 2 is the static layout phase, and 2.1 is the first task.
- Previous iteration (3) explicitly recommended this as the next task.
- Header and sidebar are foundational layout components — everything else renders inside this shell.
### Status: COMPLETE
### What was done:
- Created `dash_app/components/header.py` with `make_header()` function:
  - Top bar with `.top-header` class, NHS logo badge, "HCD Analysis" title, breadcrumb
  - Data freshness indicators: green status dot + `html.Span(id="header-record-count")` + `html.Span(id="header-last-updated")` for callback updates
  - Structure matches `01_nhs_classic.html` lines 319-333 exactly
- Created `dash_app/components/sidebar.py` with `make_sidebar()` function:
  - 7 navigation items across 2 sections (Analysis: 5 items, Reports: 2 items)
  - SVG icons from the HTML concept, embedded via data URI `html.Img` elements (Dash lacks inline SVG support)
  - Added `.sidebar__icon` CSS class matching `.sidebar__item svg` sizing (18x18px)
  - "Drug Selection" (`id="sidebar-drug-selection"`) and "Indications" (`id="sidebar-indications"`) have IDs + `n_clicks=0` for drawer callbacks in Phase 4
  - Footer: "NHS Norfolk & Waveney ICB / High Cost Drugs Programme"
- Updated `dash_app/app.py`:
  - Imports and uses `make_header()` and `make_sidebar()`
  - Layout: MantineProvider → [3 stores, Header, Nav(sidebar), Main(placeholder)]
  - Main content area uses `html.Main(className="main")` — ready for KPIs/filter bar/chart card in Task 2.2
### Validation results:
- Tier 1 (Code): All imports OK — `from dash_app.components.header import make_header`, sidebar, app all pass
- Tier 1 (App starts): `python run_dash.py` → "Dash is running on http://127.0.0.1:8050/" — no errors
- Tier 2 (Layout): Programmatic check confirms structure: MantineProvider with 6 children (3 stores + Header(top-header) + Nav(sidebar) + Main(main))
### Files changed:
- `dash_app/components/header.py` — NEW: header component
- `dash_app/components/sidebar.py` — NEW: sidebar component with SVG icons
- `dash_app/app.py` — Updated to import and assemble header + sidebar
- `dash_app/assets/nhs.css` — Added `.sidebar__icon` rule
- `IMPLEMENTATION_PLAN.md` — Task 2.1 marked [x]
### Committed: bdc1690 "feat: add header and sidebar components for Dash layout (Task 2.1)"
### Patterns discovered:
- Dash doesn't support inline SVG elements natively (no `html.Svg` or similar). Workaround: embed SVG as data URI in `html.Img` elements using `urllib.parse.quote()`. Limitation: `stroke="currentColor"` doesn't inherit from parent CSS, so icons appear in their default SVG color (black) rather than inheriting the sidebar item color. This is cosmetic and acceptable for now.
- Dash serves a minimal HTML shell; all React components render client-side. Don't test for CSS classes in the HTTP response — use programmatic layout inspection instead.
- `html.A` elements in Dash accept `n_clicks=0` property, making them callback-compatible for click events (needed for sidebar drawer triggers).
### Next iteration should:
- Start Task 2.2 — Main content area: KPI row + filter bar + chart card
- Create three component files:
  - `dash_app/components/kpi_row.py` with `make_kpi_row()` — 4 KPI cards with IDs for callback updates
  - `dash_app/components/filter_bar.py` with `make_filter_bar()` — chart type toggle pills + date dropdowns
  - `dash_app/components/chart_card.py` with `make_chart_card()` — chart area with tabs + `dcc.Graph(id="pathway-chart")`
- Read `01_nhs_classic.html` lines 377-516 for the exact structure
- Use CSS classes: `.kpi-row`, `.kpi-card`, `.filter-bar`, `.toggle-pill`, `.chart-card`, `.chart-tab`
- Filter bar needs: chart type toggle pills (By Directory / By Indication), Initiated dropdown, Last seen dropdown
- Chart card needs: title, dynamic subtitle (hierarchy label), tab row (Icicle active, Sankey/Timeline disabled), `dcc.Graph`
- KPI card IDs: `id="kpi-patients"`, `id="kpi-drugs"`, `id="kpi-cost"`, `id="kpi-match"`
- Toggle pill IDs: `id="chart-type-directory"`, `id="chart-type-indication"`
- Filter IDs: `id="filter-initiated"`, `id="filter-last-seen"`
### Blocked items:
- None

## Iteration 5 — 2026-02-06
### Task: Phase 2 — Task 2.2 (Main content area: KPI row + filter bar + chart card)
### Why this task:
- Phase 2.1 (header + sidebar) complete. Task 2.2 fills the main content area with actual components.
- Previous iteration (4) explicitly recommended this as the next task.
- The main area was a placeholder — this task adds the three core content sections.
### Status: COMPLETE
### What was done:
- Created `dash_app/components/kpi_row.py` with `make_kpi_row()`:
  - 4 KPI cards in a grid: Unique Patients, Drug Types, Total Cost, Indication Match
  - Fourth card has `.kpi-card--green` modifier for green top border
  - Each value has an ID for callback updates: `kpi-patients`, `kpi-drugs`, `kpi-cost`, `kpi-match`
  - Default value is "—" (em dash) — populated by callbacks in Phase 3
  - Helper `_kpi_card()` builds individual cards with CSS classes from nhs.css
- Created `dash_app/components/filter_bar.py` with `make_filter_bar()`:
  - Chart type toggle pills: "By Directory" (active) / "By Indication" with `.toggle-pill` CSS
  - Button IDs: `chart-type-directory`, `chart-type-indication` with `n_clicks=0` for callbacks
  - Initiated dropdown via `dcc.Dropdown`: All years (default), Last 2 years, Last 1 year
  - Last seen dropdown via `dcc.Dropdown`: Last 6 months (default), Last 12 months
  - Used `dcc.Dropdown` (not `html.Select`) for native Dash callback compatibility
  - Drug/directorate dropdowns omitted — those go in the drawer (Phase 4)
  - Added `.filter-dropdown` CSS to nhs.css for dcc.Dropdown sizing within filter bar
- Created `dash_app/components/chart_card.py` with `make_chart_card()`:
  - Card header with title "Patient Pathway Visualization" and dynamic subtitle (`id="chart-subtitle"`)
  - Default subtitle: "Trust → Directorate → Drug → Patient Pathway"
  - Tab row: Icicle (active), Sankey (disabled), Timeline (disabled)
  - `dcc.Graph(id="pathway-chart")` with min-height 500px and display config
  - Sankey/Timeline tabs are `disabled=True` placeholders for future expansion
- Updated `dash_app/app.py`:
  - Added imports for `make_kpi_row`, `make_filter_bar`, `make_chart_card`
  - Main content area now: `[make_kpi_row(), make_filter_bar(), make_chart_card()]`
  - Removed placeholder text
### Validation results:
- Tier 1 (Code): All imports pass — kpi_row, filter_bar, chart_card, app all import OK
- Tier 1 (App starts): `python run_dash.py` → "Dash is running on http://127.0.0.1:8050/" — no errors
- Tier 2 (Layout): Verified programmatically:
  - MantineProvider has 6 children (3 stores + header + sidebar + main)
  - Main has 3 children: Section(kpi-row), Section(filter-bar), Section(chart-card)
  - All 13 expected IDs found: app-state, chart-data, reference-data, kpi-patients, kpi-drugs, kpi-cost, kpi-match, chart-type-directory, chart-type-indication, filter-initiated, filter-last-seen, pathway-chart, chart-subtitle
### Files changed:
- `dash_app/components/kpi_row.py` — NEW: 4 KPI cards with callback-updatable values
- `dash_app/components/filter_bar.py` — NEW: toggle pills + date filter dropdowns
- `dash_app/components/chart_card.py` — NEW: chart header + tabs + dcc.Graph
- `dash_app/app.py` — Updated: imports and assembles new components in main area
- `dash_app/assets/nhs.css` — Added: `.filter-dropdown` CSS for dcc.Dropdown styling
- `IMPLEMENTATION_PLAN.md` — Task 2.2 items marked [x]
### Committed: 307563b "feat: add KPI row, filter bar, and chart card components (Task 2.2)"
### Patterns discovered:
- `dcc.Dropdown` must be used instead of `html.Select` for Dash callback compatibility. `html.Select` doesn't have a `value` property that triggers callbacks. `dcc.Dropdown` renders its own React-Select widget that needs CSS customization to match the `.filter-select` styling from the HTML concept.
- `html.Button` with `n_clicks=0` works for toggle pills — callbacks can use `callback_context.triggered_id` to determine which button was clicked.
- `dcc.Graph` accepts a `config` dict for mode bar customization (removing lasso/select, hiding logo).
- Unicode arrow `\u2192` (→) used in chart subtitle for cross-platform compatibility.
### Next iteration should:
- Start Task 2.3 — Footer + full page assembly
- Create `dash_app/components/footer.py` with `make_footer()` using `.page-footer` CSS class
- Add footer to the main content area in `app.py` (below chart card)
- Verify full page layout matches `01_nhs_classic.html` visually
- The 3 dcc.Store components are already in place (added in Phase 0), so 2.3's store bullet is already done
- After 2.3, Phase 2 will be complete — Phase 3 (Core Callbacks) is next
- Phase 3.1 will need: reference data loading callback (on page load), filter state management callback (toggle pills + dropdowns → app-state), and `register_callbacks(app)` wiring
### Blocked items:
- None

## Iteration 6 — 2026-02-06
### Task: Phase 2 — Task 2.3 (Footer + full page assembly)
### Why this task:
- Tasks 2.1 and 2.2 complete. Task 2.3 is the final piece of Phase 2.
- Previous iteration (5) explicitly recommended this as the next task.
- Completing this marks Phase 2 done, enabling Phase 3 (Core Callbacks).
### Status: COMPLETE
### What was done:
- Created `dash_app/components/footer.py` with `make_footer()`:
  - Returns `html.Footer(className="page-footer")` matching 01_nhs_classic.html line 520
  - Text: "NHS Norfolk and Waveney ICB — High Cost Drugs Analysis"
- Updated `dash_app/app.py`:
  - Added `make_footer` import
  - Added `make_footer()` as 4th child in main content area (after chart card)
- Full page layout now complete: MantineProvider → [3 stores, Header, Sidebar, Main(KPIs, Filter bar, Chart card, Footer)]
- Note: dcc.Store components and page assembly were already done in earlier iterations. Task 2.3 effectively just needed the footer component.
### Validation results:
- Tier 1 (Code): `from dash_app.components.footer import make_footer` — OK, returns Footer with className="page-footer"
- Tier 1 (App starts): `from dash_app.app import app` — OK, no errors
- Tier 2 (Layout): Verified programmatically:
  - MantineProvider: 6 children (3 stores + Header(top-header) + Nav(sidebar) + Main(main))
  - Main: 4 children (Section(kpi-row), Section(filter-bar), Section(chart-card), Footer(page-footer))
  - All component IDs present and correct
### Files changed:
- `dash_app/components/footer.py` — NEW: footer component
- `dash_app/app.py` — Updated: import + added footer to main area
- `IMPLEMENTATION_PLAN.md` — Task 2.3 marked [x]
### Committed: 3568e03 "feat: add footer component and complete Phase 2 static layout (Task 2.3)"
### Patterns discovered:
- Phase 2 was incrementally built across iterations 4-6. The layout assembly (app.py) evolved naturally — each iteration added its components. No "big bang" assembly step was needed.
### Next iteration should:
- Start Phase 3: Task 3.1 — Reference data loading + filter state management
- Create `dash_app/callbacks/filters.py` with two callbacks:
  1. `load_reference_data`: fires on page load, calls `queries.load_initial_data()`, populates `reference-data` store + header record count + header last updated
  2. `update_app_state`: fires when chart-type toggle buttons or date dropdowns change, computes `date_filter_id` from initiated+last_seen combo, updates `app-state` store, manages toggle pill active classes
- Create `dash_app/callbacks/__init__.py` with `register_callbacks(app)` function
- Wire `register_callbacks(app)` call in `app.py` after layout definition
- Key IDs to use as inputs: `chart-type-directory`, `chart-type-indication` (n_clicks), `filter-initiated` (value), `filter-last-seen` (value)
- Key IDs to use as outputs: `app-state` (data), `reference-data` (data), `header-record-count` (children), `header-last-updated` (children)
- Date filter ID mapping: initiated "all"/"2yr"/"1yr" + last_seen "6mo"/"12mo" → e.g. "all_6mo", "1yr_12mo"
- For toggle pills, use Output for className to toggle `toggle-pill--active` class
- Read pathways_app/pathways_app.py lines 407-488 to understand the reference data structure
### Blocked items:
- None

## Iteration 7 — 2026-02-06
### Task: Phase 3 — Task 3.1 (Reference data loading + filter state management)
### Why this task:
- Phases 0-2 complete. Phase 3 is next in dependency order.
- Previous iteration (6) explicitly recommended this as the next task.
- All subsequent callbacks (3.2 chart data, 3.3 KPIs, 3.4 icicle chart) depend on filter state management from 3.1.
### Status: COMPLETE
### What was done:
- Created `dash_app/callbacks/filters.py` with two callbacks:
  1. `load_reference_data`: fires on page load via `dcc.Location` pathname Input, calls `queries.load_initial_data()`, populates `reference-data` store + header record count ("Data loaded" or "N records") + header last updated (date string)
  2. `update_app_state`: fires on chart-type toggle clicks (n_clicks) + date dropdown changes (value), uses `ctx.triggered_id` to determine chart type, computes `date_filter_id` from `{initiated}_{last_seen}`, updates `app-state` store, toggles `toggle-pill--active` CSS class on pills
- Created `dash_app/callbacks/__init__.py` with `register_callbacks(app)` function that imports and registers all callback modules
- Wired `register_callbacks(app)` in `app.py` after layout definition
- Added `dcc.Location(id="url", refresh=False)` to layout for page-load trigger
- Initially used `app-state` as Input for `load_reference_data` but changed to `dcc.Location` to avoid re-firing on every filter change (would cause circular updates)
### Validation results:
- Tier 1 (Code): All imports pass — `filters.py`, `callbacks/__init__.py`, `app.py` all OK
- Tier 1 (App starts): `python run_dash.py` → "Dash is running on http://127.0.0.1:8050/" — no errors
- Tier 3 (Functional):
  - Both callbacks registered in `app.callback_map`: reference-data.data, app-state.data
  - Reference data loads correctly: 42 drugs, 14 directorates, 32 indications, last updated 2026-02-06
  - Date filter ID computation: all 6 combinations verified correct
  - Toggle pill class toggling: active = "toggle-pill toggle-pill--active", inactive = "toggle-pill"
### Files changed:
- `dash_app/callbacks/filters.py` — NEW: load_reference_data + update_app_state callbacks
- `dash_app/callbacks/__init__.py` — Updated: register_callbacks(app) with filter callback registration
- `dash_app/app.py` — Updated: added dcc.Location, import + call register_callbacks(app)
- `IMPLEMENTATION_PLAN.md` — Task 3.1 items marked [x]
### Committed: eda35c7 "feat: add reference data loading and filter state callbacks (Task 3.1)"
### Patterns discovered:
- Dash 4.0.0 uses `ctx.triggered_id` (direct string) rather than the older `callback_context.triggered[0]["prop_id"]` parsing pattern. Simpler and more reliable.
- Using `app-state` as Input for reference data loading creates circular updates: `update_app_state` writes app-state → `load_reference_data` fires → queries DB unnecessarily. Use `dcc.Location(id="url")` pathname as the page-load trigger instead.
- `dcc.Location(refresh=False)` fires once on page load with `pathname="/"` — perfect for initialization callbacks.
- `total_records` from `pathway_refresh_log.source_row_count` is still 0 (known since iteration 2). The header shows "Data loaded" as fallback text instead of "0 records".
### Next iteration should:
- Start Task 3.2 — Pathway data loading callback
- Create the callback in `dash_app/callbacks/chart.py`:
  - Input: `app-state` store (fires when filters change)
  - Output: `chart-data` store
  - Calls `queries.load_pathway_data(filter_id, chart_type, selected_drugs, selected_directorates)`
  - Must handle initial page load (when app-state has default values) AND subsequent filter changes
- Register in `dash_app/callbacks/__init__.py` alongside filter callbacks
- Verify: changing date filter dropdown → app-state updates → chart-data store gets new pathway nodes
- This is the bridge between filter state and chart rendering — 3.3 (KPIs) and 3.4 (icicle chart) both depend on chart-data being populated
### Blocked items:
- None

## Iteration 8 — 2026-02-06
### Task: Phase 3 — Task 3.2 (Pathway data loading callback)
### Why this task:
- Phase 3.1 (filter state management) is complete. Task 3.2 is the next dependency in the callback chain.
- Previous iteration (7) explicitly recommended this task.
- Tasks 3.3 (KPIs) and 3.4 (icicle chart) both depend on chart-data being populated by this callback.
### Status: COMPLETE
### What was done:
- Created `dash_app/callbacks/chart.py` with `register_chart_callbacks(app)`:
  - `load_pathway_data` callback: Input=`app-state` store, Output=`chart-data` store
  - Extracts `date_filter_id`, `chart_type`, `selected_drugs`, `selected_directorates` from app-state
  - Converts empty lists to None (so query function skips WHERE clause for unfiltered state)
  - Lazy imports `queries.load_pathway_data` to avoid circular import at module level
  - Returns `no_update` when app-state is empty (prevents query on initial empty state)
- Registered in `dash_app/callbacks/__init__.py` alongside filter callbacks
- Callback chain now complete: filter change → app-state → chart-data
### Validation results:
- Tier 1 (Code): `from dash_app.callbacks.chart import register_chart_callbacks` — OK
- Tier 1 (App starts): `from dash_app.app import app` — OK, all 3 callbacks registered
- Tier 3 (Functional): Verified callback chain with real data:
  - Default (all_6mo, directory): 293 nodes, 11,118 patients, 39 drugs, £130.6M
  - Indication chart: 438 nodes, 11,252 patients
  - Drug filter (ADALIMUMAB): 70 nodes
  - Date filter (2yr_12mo): 147 nodes, 5,104 patients
  - JSON serialization verified — dcc.Store compatible
### Files changed:
- `dash_app/callbacks/chart.py` — NEW: pathway data loading callback
- `dash_app/callbacks/__init__.py` — Updated: registers chart callback
- `IMPLEMENTATION_PLAN.md` — Task 3.2 marked [x]
### Committed: ad9fa1c "feat: add pathway data loading callback bridging filters to chart-data (Task 3.2)"
### Patterns discovered:
- Empty list `[]` from app-state needs conversion to `None` before passing to `load_pathway_nodes()`. The query function treats `None` as "no filter" (skips WHERE clause) but `[]` would create an empty IN clause returning zero results.
- `no_update` is the correct return for when app-state is not yet populated (rather than returning an empty dict), since it prevents downstream callbacks from firing unnecessarily.
### Next iteration should:
- Start Task 3.3 — KPI update callback
- Create `dash_app/callbacks/kpi.py`:
  - Input: `chart-data` store
  - Output: 4 KPI card values (`kpi-patients`, `kpi-drugs`, `kpi-cost`, `kpi-match`)
  - Extract `unique_patients`, `total_drugs`, `total_cost` from chart-data
  - Format: patients with commas (e.g., "11,118"), cost as "£130.6M", drugs as plain number
  - Indication match rate: could be "~93%" (from CLAUDE.md) or computed from data — check if chart-data has match info
- Register in callbacks/__init__.py
- After 3.3, move to Task 3.4 (icicle chart rendering) — the final core callback
- Task 3.4 requires reading `pathways_app/pathways_app.py` lines 769-920 for the icicle figure construction, then adding `create_icicle_from_nodes()` to `src/visualization/plotly_generator.py`
### Blocked items:
- None

## Iteration 9 — 2026-02-06
### Task: Phase 3 — Task 3.3 (KPI update callback)
### Why this task:
- Phase 3.2 (pathway data loading) is complete. Task 3.3 is next in the callback chain.
- Previous iteration (8) explicitly recommended this task.
- KPI cards currently show "—" placeholders — this callback populates them with real data.
### Status: COMPLETE
### What was done:
- Created `dash_app/callbacks/kpi.py` with `register_kpi_callbacks(app)`:
  - `update_kpis` callback: Input=`chart-data` + `app-state`, Output=4 KPI card values
  - Formats: patients with commas (11,118), drugs as plain number (39), cost as £130.6M
  - `_format_cost()` helper handles M/K/plain thresholds
  - 4th KPI (Indication Match): shows "~93%" when chart_type is "indication", "—" otherwise
  - Returns `no_update` when chart-data is empty (prevents clearing KPIs during load)
- Registered in `dash_app/callbacks/__init__.py` — now 4 callbacks total
- Decision: Indication match rate is static "~93%" because:
  - Reflex app's `indication_match_rate` was never dynamically computed (always 0.0 → "—")
  - Computing from nodes would require detecting "(no GP dx)" labels, adding complexity for marginal value
  - ~93% is the documented rate from the SNOMED cluster matching pipeline
### Validation results:
- Tier 1 (Code): `from dash_app.callbacks.kpi import register_kpi_callbacks` — OK
- Tier 1 (App starts): `from dash_app.app import app` — OK, 4 callbacks registered
- Tier 3 (Functional):
  - Cost formatting: 130.5M → £130.5M, 50K → £50.0K, 999 → £999
  - Directory chart: 11,118 patients, 39 drugs, £130.6M, Match = —
  - Indication chart: Match = ~93%
  - All 4 KPI output IDs registered: kpi-patients, kpi-drugs, kpi-cost, kpi-match
### Files changed:
- `dash_app/callbacks/kpi.py` — NEW: KPI update callback with formatting
- `dash_app/callbacks/__init__.py` — Updated: registers kpi callback
- `IMPLEMENTATION_PLAN.md` — Task 3.3 marked [x]
### Committed: 9c971c0 "feat: add KPI update callback with formatted patient/drug/cost display (Task 3.3)"
### Patterns discovered:
- `\u00a3` (£) is the safe way to embed the pound sign in Python strings — avoids encoding issues with source files
- `no_update` tuple must match the number of outputs: `return no_update, no_update, no_update, no_update` for 4 outputs
- The `app-state` is used as a secondary Input (not just `chart-data`) to get `chart_type` for the indication match KPI — this is fine since `app-state` changes trigger `chart-data` changes which also trigger this callback
### Next iteration should:
- Start Task 3.4 — Icicle chart rendering callback (the final core callback)
- Two sub-tasks:
  1. Add `create_icicle_from_nodes(nodes, title)` to `src/visualization/plotly_generator.py`
     - Read `pathways_app/pathways_app.py` lines 769-920 for the exact Plotly go.Icicle construction
     - Extract the 10-field customdata, NHS blue colorscale, texttemplate, hovertemplate
     - Keep existing `create_icicle_figure(ice_df)` untouched — add new function alongside
  2. Add `update_chart` callback to `dash_app/callbacks/chart.py`
     - Input: `chart-data` store, Output: `pathway-chart` figure
     - Calls `create_icicle_from_nodes(chart_data["nodes"], title)`
     - Dynamic title based on chart type
- IMPORTANT: Read the guardrails about the icicle figure — must use NHS blue gradient, not Viridis
- After 3.4, Phase 3 is complete — Phase 4 (Drawer) is next
### Blocked items:
- None

## Iteration 10 — 2026-02-06
### Task: Phase 3 — Task 3.4 (Icicle chart rendering callback)
### Why this task:
- Tasks 3.1-3.3 complete. Task 3.4 is the final task in Phase 3.
- Previous iteration (9) explicitly recommended this task.
- This is the most critical callback — it renders the actual Plotly icicle chart from pathway data.
### Status: COMPLETE
### What was done:
- Added `create_icicle_from_nodes(nodes, title)` to `src/visualization/plotly_generator.py`:
  - Accepts list-of-dicts (the JSON format from dcc.Store / load_pathway_nodes)
  - Same 10-field customdata structure as Reflex icicle_figure (value, colour, cost, costpp, first_seen, last_seen, first_seen_parent, last_seen_parent, average_spacing, cost_pp_pa)
  - NHS blue gradient colorscale: Heritage Blue #003087 → Pale Blue #E3F2FD (5 stops)
  - Same texttemplate and hovertemplate strings as Reflex version
  - maxdepth=3, branchvalues="total", sort=False
  - Layout: transparent background, reduced margins (t:40, l:8, r:8, b:24), autosize
  - Font: Source Sans 3 (matching nhs.css) instead of Inter (Reflex used Inter)
  - Existing `create_icicle_figure(ice_df)` left completely untouched
- Added `update_chart` callback to `dash_app/callbacks/chart.py`:
  - Input: chart-data store + app-state store
  - Output: pathway-chart figure + chart-subtitle children
  - Calls `create_icicle_from_nodes(chart_data["nodes"], title)`
  - Dynamic subtitle: "Trust → Directorate → Drug → Pathway" or "Trust → Indication → Drug → Pathway"
- Added `_generate_chart_title()` helper mirroring Reflex `_generate_pathway_chart_title()`:
  - Formats: "By Directory | All years / Last 6 months" (default)
  - Handles drug/directorate selections in title
### Validation results:
- Tier 1 (Code): `from visualization.plotly_generator import create_icicle_from_nodes` — OK
- Tier 1 (Code): `from dash_app.callbacks.chart import register_chart_callbacks` — OK
- Tier 1 (App starts): `from dash_app.app import app` — OK, 5 callbacks registered
- Tier 3 (Functional):
  - Directory chart: 293 nodes → Figure with 1 icicle trace, 293 labels, 10-field customdata
  - Indication chart: 438 nodes → Figure renders correctly
  - Colorscale starts at (0.0, '#003087') — NHS Heritage Blue confirmed
  - Title: "Patient Pathways — By Directory | All years / Last 6 months"
  - All 14 expected layout IDs present
### Files changed:
- `src/visualization/plotly_generator.py` — Added: create_icicle_from_nodes() function (115 lines)
- `dash_app/callbacks/chart.py` — Rewritten: added update_chart callback + _generate_chart_title helper
- `IMPLEMENTATION_PLAN.md` — Task 3.4 items marked [x]
### Committed: 40ce7fc "feat: add icicle chart rendering with NHS colorscale and dynamic titles (Task 3.4)"
### Patterns discovered:
- The Reflex version uses "Inter" font; switched to "Source Sans 3" to match the nhs.css design reference. This is the only intentional deviation from the Reflex code.
- `\u00a3` (£) works in Plotly template strings for the pound sign — no encoding issues.
- The `update_chart` callback uses both `chart-data` (for nodes) and `app-state` (for chart_type to determine subtitle) as Inputs. Both fire when filters change, but Dash deduplicates — the callback runs once with both updated values.
- `_generate_chart_title()` is a module-level function (not inside `register_chart_callbacks`) so it can be tested independently.
### Next iteration should:
- Start Phase 4: Task 4.1 — dmc.Drawer layout
- Create `dash_app/components/drawer.py` with `make_drawer()` function:
  - `dmc.Drawer(id="drug-drawer", position="right", size="480px")`
  - Top section: "All Drugs" card with flat alphabetical drug list from pathway_nodes level 3
  - Below: one card per PrimaryDirectorate from DimSearchTerm.csv with dmc.Accordion per Search_Term
  - Drug chips with pattern-matching IDs: `{"type": "drug-chip", "index": drug_name}`
  - Bottom: "Clear Filters" button
- Read `01_nhs_classic.html` for drawer styling if applicable (though drawer is mostly DMC-styled)
- Use `build_directorate_tree()` from `dash_app/data/card_browser.py` (already built in Task 1.2)
- Use `get_all_drugs()` from `dash_app/data/card_browser.py` for the flat drug list
- Add drawer to the app layout in `app.py`
- IMPORTANT: DMC version is 2.5.1 (Mantine v7 based). Check DMC 2.x API for Drawer, Accordion, Chip — prop names may differ from 0.14.x. E.g., `dmc.Drawer` may use `opened` instead of `open`, `onClose` instead of `on_close`.
### Blocked items:
- None

## Iteration 11 — 2026-02-06
### Task: Phase 4 — Task 4.1 (dmc.Drawer layout)
### Why this task:
- Phases 0-3 are complete. Phase 4 (Directorate Card Browser) is next in dependency order.
- Iteration 10 explicitly recommended this as the next task.
- The drawer layout is needed before drawer callbacks (Task 4.2) can be built.
### Status: COMPLETE
### What was done:
- Created `dash_app/components/drawer.py` with `make_drawer()` function:
  - `dmc.Drawer(id="drug-drawer", position="right", size="480px", opened=False)`
  - Title: "Drug & Indication Browser" via `dmc.Text`
  - ScrollArea wrapping all content for vertical overflow
  - **All Drugs section**: `dmc.ChipGroup(id="all-drugs-chips", multiple=True)` with 42 drug chips from pathway_nodes level 3
  - **Directorate section**: Nested `dmc.Accordion` with 19 directorates, each containing an inner accordion of indications (163 total). Drug fragments shown as `dmc.Badge` with pattern-match IDs `{"type": "drug-fragment", "index": "DIRECTORATE|FRAGMENT"}`
  - **Clear button**: `dmc.Button("Clear All Filters", id="clear-drug-filters", variant="outline", color="red")`
- Added drawer CSS to `dash_app/assets/nhs.css`:
  - `.drawer-section`, `.drawer-section-title`, `.drawer-chips-wrap`, `.drawer-drug-badge`, `.drawer-directorate-accordion`, `.drawer-clear-btn`
  - Chip font-family override to Source Sans 3
  - Badge hover effect for drug fragments
- Updated `dash_app/app.py`:
  - Added `make_drawer()` import and placed drawer in MantineProvider children (after sidebar, before main)
### Validation results:
- Tier 1 (Code): `from dash_app.components.drawer import make_drawer` — OK
- Tier 1 (App starts): `python run_dash.py` → "Dash is running on http://127.0.0.1:8050/" — no errors
- Tier 2 (Layout): Verified programmatically:
  - MantineProvider: 8 children (4 stores/location + Header + Nav + Drawer + Main)
  - Drawer: position=right, size=480px, opened=False
  - All Drugs ChipGroup: 42 chips, multiple=True
  - Directorate Accordion: 19 directorates, 163 indications
  - 433 IDs in layout (mostly drug-fragment pattern-match IDs)
  - Key IDs present: drug-drawer, all-drugs-chips, clear-drug-filters
  - App has 5 callbacks registered (same as before — no new callbacks in this layout-only task)
### Files changed:
- `dash_app/components/drawer.py` — NEW: drawer component with drug chips and directorate cards
- `dash_app/app.py` — Updated: imports and adds drawer to layout
- `dash_app/assets/nhs.css` — Added: drawer-specific CSS classes
- `IMPLEMENTATION_PLAN.md` — Task 4.1 marked [x]
### Committed: 5dc552f "feat: add dmc.Drawer drug browser with directorate cards and drug chips (Task 4.1)"
### Patterns discovered:
- DMC 2.5.1 `dmc.Chip` doesn't expose `n_clicks`. It's a controlled component that works through `dmc.ChipGroup` with `value` prop (list of selected values when `multiple=True`). This is simpler than pattern-matching callbacks for drug selection — a single callback on `all-drugs-chips.value` replaces N individual chip click handlers.
- DMC 2.5.1 `dmc.Drawer` uses `opened` (not `open`), consistent with Mantine v7 API.
- `dmc.AccordionControl` + `dmc.AccordionPanel` inside `dmc.AccordionItem` is the DMC 2.x pattern. Nesting accordions works — directorate accordion → indication accordion inside.
- Drug fragments use `dmc.Badge` (not `dmc.Chip`) because they're informational/action triggers, not multi-select toggles. Badge IDs use pattern-matching format `{"type": "drug-fragment", "index": "DIR|FRAG"}` for callback handling in 4.2.
- The drawer data is loaded at layout time (module-level `make_drawer()` call during app import), not lazily. This is fine since the data is small (~165 CSV rows + 42 drugs from SQLite) and only runs once at app startup.
### Next iteration should:
- Start Task 4.2 — Drawer callbacks
- Create `dash_app/callbacks/drawer.py` with `register_drawer_callbacks(app)`:
  1. **Open/close drawer**: sidebar "Drug Selection" (`id="sidebar-drug-selection"`, n_clicks) or "Indications" (`id="sidebar-indications"`, n_clicks) click → set `drug-drawer.opened=True`. Drawer's built-in close (X button, overlay click) handles closing via `opened` prop.
  2. **Drug chip selection**: `all-drugs-chips.value` (ChipGroup value change) → update `app-state.selected_drugs` → triggers chain: app-state → chart-data → KPIs + chart
  3. **Drug fragment click**: `{"type": "drug-fragment", "index": ALL}.n_clicks` → match fragment against full drug names → select matching drugs in `all-drugs-chips.value` + `app-state.selected_drugs`
  4. **Clear filters**: `clear-drug-filters.n_clicks` → reset `app-state.selected_drugs=[]` + `all-drugs-chips.value=[]`
- Register in `dash_app/callbacks/__init__.py`
- Key consideration: The `all-drugs-chips` ChipGroup value must stay in sync with `app-state.selected_drugs`. Consider whether to use `all-drugs-chips.value` as the source of truth (simpler) or `app-state.selected_drugs` (needed for other callbacks). May need a two-way sync or just derive one from the other.
- IMPORTANT: Clicking a drug fragment badge should filter to drugs whose FULL names contain that fragment (substring match). The fragment "PEGYLATED" would match "PEGYLATED INTERFERON ALFA-2A" for example. The matching logic should use `drug_name.upper().find(fragment) >= 0`.
### Blocked items:
- None

## Iteration 12 — 2026-02-06
### Task: Phase 4 — Task 4.2 (Drawer callbacks)
### Why this task:
- Phase 4.1 (drawer layout) complete. Task 4.2 adds the interactivity to the drawer.
- Iteration 11 explicitly recommended this as the next task.
- Without callbacks, the drawer can't open or affect the chart — it's just a static component.
### Status: COMPLETE
### What was done:
- Created `dash_app/callbacks/drawer.py` with `register_drawer_callbacks(app)`:
  1. `open_drawer`: Input = sidebar-drug-selection + sidebar-indications n_clicks → Output = drug-drawer.opened = True. Drawer's built-in close (X / overlay) handles closing.
  2. `handle_fragment_or_clear`: Input = drug-fragment badges (pattern-matching ALL) n_clicks + clear-drug-filters n_clicks → Output = all-drugs-chips.value
     - Fragment click: extracts fragment from `{"type": "drug-fragment", "index": "DIR|FRAG"}`, finds all drugs containing the fragment (substring match), toggles them in chip selection
     - Clear click: sets chip value to empty list `[]`
     - Toggle behavior: if all matching drugs already selected, deselects them; otherwise adds them
- Modified `dash_app/callbacks/filters.py` `update_app_state`:
  - Added `all-drugs-chips.value` as 5th Input (after date filter dropdowns)
  - When chip selection changes, `selected_drugs` in app-state is updated
  - This completes the callback chain: chip change → app-state → chart-data → chart + KPIs
- Registered drawer callbacks in `dash_app/callbacks/__init__.py` — now 7 callbacks total
- Design decision: `all-drugs-chips.value` (ChipGroup) is the visual source of truth for drug selection. The drawer callbacks write to it, and `update_app_state` reads from it to sync into `app-state.selected_drugs`. No `allow_duplicate` needed — clean unidirectional flow.
### Validation results:
- Tier 1 (Code): All imports pass — drawer.py, filters.py, callbacks/__init__.py, app.py
- Tier 1 (App starts): MantineProvider layout, 7 callbacks registered, app ready to serve
- Tier 3 (Functional):
  - Callback chain verified: sidebar clicks → drawer opens (drug-drawer.opened Output)
  - Fragment matching: "ADALIMUMAB" → matches ["ADALIMUMAB"], "RITUXI" → matches ["RITUXIMAB"]
  - Some fragments (PEGYLATED, INHALED) match 0 drugs in pathway_nodes — returns no_update (correct behavior: those fragments may reference drugs not in current data)
  - Clear button → empty chip list → app-state.selected_drugs = [] → full chart reload
  - update_app_state Input list confirmed: 5 Inputs (2 toggle pills + 2 dropdowns + 1 ChipGroup)
### Files changed:
- `dash_app/callbacks/drawer.py` — NEW: open_drawer + handle_fragment_or_clear callbacks
- `dash_app/callbacks/__init__.py` — Updated: registers drawer callbacks
- `dash_app/callbacks/filters.py` — Updated: added all-drugs-chips.value as Input to update_app_state
- `IMPLEMENTATION_PLAN.md` — Task 4.2 marked [x]
### Committed: fe76e5a "feat: add drawer callbacks for drug selection, fragment matching, and clear (Task 4.2)"
### Patterns discovered:
- `dmc.ChipGroup.value` is the cleanest way to manage multi-select drug state. The ChipGroup handles visual selection state internally; callbacks only need to update the `value` prop (list of selected values). No per-chip n_clicks tracking needed.
- Fragment badge IDs use format `{"type": "drug-fragment", "index": "DIRECTORATE|FRAGMENT"}`. The directorate prefix is stripped with `split("|", 1)[-1]` to get the pure fragment for substring matching.
- Pattern-matching `Input({"type": "drug-fragment", "index": ALL}, "n_clicks")` with `prevent_initial_call=True` is essential — without it, the callback fires on page load with all n_clicks as None/0, which would trigger `no_update`.
- The toggle behavior (click fragment → select matching drugs; click again → deselect) is implemented by checking if all matching drugs are already in the current selection.
- `allow_duplicate=True` was avoided by routing all drug selection through `all-drugs-chips.value` → `update_app_state`. Only one callback writes to each Output.
### Next iteration should:
- Start Phase 5: Task 5.1 — Trust selection
- Options for trust selection:
  - Add a "Trusts" section to the drawer (preferred — keeps all filters together)
  - OR add trust checkboxes in the sidebar
- Implementation: add `dmc.ChipGroup(id="trust-chips", multiple=True)` to the drawer with available trusts from reference-data
- Wire trust selection similarly to drug selection: trust-chips.value → update_app_state → app-state.selected_trusts → chart reload
- Check `src/data_processing/pathway_queries.py` `load_pathway_nodes()` — does it support a `selected_trusts` parameter? If not, add trust filtering to the query.
- Read `pathways_app/pathways_app.py` for how trust filtering was handled in Reflex
- ALTERNATIVE: Skip 5.1 for now and do 5.2 (loading/error/empty states) which is more impactful for UX polish. Trust selection is less critical since there are few trusts.
### Blocked items:
- None