HighCostDrugsDemo/IMPLEMENTATION_PLAN.md

# Implementation Plan - HCD Analysis UI Redesign

## Project Overview

Complete frontend redesign of the Patient Pathway Analysis tool. Replace the current multi-page sidebar layout with a modern, single-page dashboard featuring:
- Instant reactive filtering with debounce
- Interactive Plotly icicle chart that updates in real-time
- NHS-inspired but bold, modern visual design
- KPI metrics that respond to filter changes

**Design Reference:** See `DESIGN_SYSTEM.md` for color palette, typography, spacing, and component specs.

**Source Code:** The existing `pathways_app/pathways_app.py` contains the current implementation. Create a new `pathways_app/app_v2.py` for the redesign, leaving the original intact until verification.

## Quality Checks

Run after each task:

```bash
# Syntax check
python -m py_compile pathways_app/app_v2.py

# Import verification
python -c "from pathways_app.app_v2 import app"

# Reflex compilation test
cd pathways_app && timeout 60 python -m reflex run 2>&1 | head -30

# If compilation shows errors, fix before marking task complete
```

## Phase 1: Foundation

### 1.1 Design Tokens Module
- [x] Create `pathways_app/styles.py` with design token classes:
  - `Colors` class with all palette colors as constants
  - `Typography` class with font sizes, weights
  - `Spacing` class with spacing scale
  - `Shadows` class with shadow values
  - `Radii` class with border radius values
- [x] Create helper functions for common style patterns (e.g., `card_style()`, `button_primary_style()`)
- [x] Verify imports work: `from pathways_app.styles import Colors, Spacing`

### 1.2 App Skeleton
- [x] Create `pathways_app/app_v2.py` with basic Reflex app structure
- [x] Define new `AppState` class with minimal state (placeholder for now)
- [x] Create single-page layout structure matching DESIGN_SYSTEM.md
- [x] Verify `reflex run` compiles and shows blank page with correct structure
- [x] Configure Reflex theme with design system colors

## Phase 2: Layout Components

### 2.1 Top Navigation Bar
- [x] Create `top_bar()` component:
  - Logo (use existing NHS person logo from assets)
  - App title "HCD Analysis"
  - Chart type tabs/pills (Icicle active, placeholders for future charts)
  - Data freshness indicator (right side): "12,450 records (2d ago)"
- [x] Style with Heritage Blue accents, clean typography
- [x] Fixed height: 64px
- [x] Verify renders correctly

### 2.2 Filter Section
- [x] Create `filter_section()` component with card styling
- [x] Add date range pickers:
  - "Initiated" range with enable/disable checkbox (default: disabled)
  - "Last Seen" range with enable/disable checkbox (default: enabled, last 6 months)
  - "To" date defaults to latest date in dataset (placeholder — actual data integration in Phase 3)
- [x] Add searchable multi-select dropdowns:
  - Drugs dropdown with search, select all, count display
  - Indications dropdown with search, select all, count display
  - Directorates dropdown with search, select all, count display
- [ ] Implement debounced filter change handlers (300ms) — deferred to Phase 3.3
- [x] Style according to design system

### 2.3 KPI Row
- [x] Create `kpi_card()` component:
  - Large mono number (32-48px)
  - Label below (caption style)
  - Subtle background tint
- [x] Create `kpi_row()` component with responsive grid
- [x] Initially show: Unique Patients count
- [x] Leave space for future metrics (Drugs count, Total cost, Match rate)
- [x] KPIs should be reactive to filter state

### 2.4 Chart Container
- [x] Create `chart_section()` component
- [x] Full-width card with appropriate padding
- [x] Placeholder for Plotly chart (integrate in Phase 4)
- [x] Loading state with skeleton/spinner
- [x] Error state with friendly message

## Phase 3: State Management

### 3.1 Core State Variables
- [x] Define filter state variables in `AppState`:
  - `initiated_filter_enabled: bool = False`
  - `initiated_from_date: str = ""`  (ISO date string)
  - `initiated_to_date: str = ""`
  - `last_seen_filter_enabled: bool = True`
  - `last_seen_from_date: str` (default: 6 months ago, computed at class definition)
  - `last_seen_to_date: str` (default: today, updated on data load)
  - `selected_drugs: list[str] = []` (empty = all)
  - `selected_indications: list[str] = []` (empty = all)
  - `selected_directorates: list[str] = []` (empty = all)
- [x] Define data state variables:
  - `data_loaded: bool = False`
  - `total_records: int = 0`
  - `last_updated: str = ""` (ISO timestamp)
  - `raw_data: list[dict[str, Any]] = []` (list of dicts, Reflex-friendly)
  - `latest_date_in_data: str = ""` (for "to" date defaults)
- [x] Define UI state variables:
  - `chart_loading: bool = False`
  - `error_message: str = ""`
  - `current_chart: str = "icicle"`

### 3.2 Data Loading
- [x] Create `load_data()` method that reads from SQLite
- [x] Populate available options for dropdowns (drugs, indications, directorates)
- [x] Detect latest date in dataset for "to" date defaults
- [x] Calculate total records and last updated timestamp
- [x] Call on app initialization

### 3.3 Filter Logic
- [x] Create `apply_filters()` computed method that filters the data based on current state
- [x] Handle initiated date filter (when enabled)
- [x] Handle last seen date filter (when enabled)
- [x] Handle drug/indication/directorate multi-select filters
- [x] Return filtered DataFrame

### 3.4 KPI Calculations
- [x] Create computed properties for KPI values:
  - `unique_patients: int` — COUNT(DISTINCT patient_id) from filtered data
  - `total_drugs: int` — COUNT(DISTINCT drug_name_std) from filtered data
  - `total_cost: float` — SUM(price_actual) from filtered data
  - (Blocked: indication_match_rate requires Snowflake GP data)
- [x] Ensure KPIs update reactively when filters change
- Note: KPIs implemented in apply_filters() method, called by all filter event handlers

## Phase 4: Interactive Chart

### 4.1 Chart Data Preparation
- [x] Create `prepare_chart_data()` method that transforms filtered data for Plotly icicle
- [x] Reuse/adapt logic from existing `pathway_analyzer.py` (simplified hierarchy: Trust → Directory → Drug)
- [x] Return data structure compatible with `go.Icicle()` (list of dicts with parents, ids, labels, value, cost, colour)
- [x] Generate chart_title based on current filter state
- [x] Call prepare_chart_data() from apply_filters() for reactive updates

### 4.2 Reactive Plotly Integration
- [x] Create `generate_icicle_chart()` computed property that returns Plotly figure
- [x] Configure chart colors using design system palette
- [x] Configure chart interactivity (zoom, pan, click, hover)
- [x] Set responsive sizing

### 4.3 Chart Component
- [x] Integrate `rx.plotly()` component in chart_section
- [x] Pass reactive figure from state
- [x] Handle loading states (show skeleton while computing)
- [x] Handle empty data state (friendly message)
- [x] Verify chart updates when filters change

## Phase 5: Polish & Verification

### 5.1 Visual Polish
- [x] Review all components against DESIGN_SYSTEM.md
- [x] Ensure consistent spacing throughout
- [x] Ensure consistent typography throughout
- [x] Add hover states and transitions to interactive elements
- [x] Test responsive behavior (resize browser)

### 5.2 Performance Optimization
- [ ] Profile filter + chart update cycle
- [ ] Ensure debounce is working correctly (not triggering on every keystroke)
- [ ] Optimize any slow computed properties
- [ ] Verify smooth 60fps interactions

### 5.3 Error Handling
- [ ] Handle no data loaded state gracefully
- [ ] Handle filter resulting in zero records
- [ ] Handle any data loading errors
- [ ] User-friendly error messages

### 5.4 Final Verification
- [x] Load real data from SQLite (440K records, 552 drugs, 29 directorates, 32 indications)
- [x] Test all filter combinations (no filter, Last Seen, Drug, Directorate, Combined - all working)
- [x] Verify KPIs update correctly (patients, drugs, cost all compute correctly)
- [x] Verify chart updates correctly (1,887 hierarchical nodes generated correctly)
- [ ] Compare key metrics with original app to ensure correctness
- [ ] Test with large dataset for performance

### 5.5 Cleanup
- [ ] Remove or comment out old `pathways_app.py` code paths
- [x] Update any imports/references to use new app (updated __init__.py to re-export from app_v2)
- [ ] Update README with new run instructions
- [ ] Document any breaking changes

## Completion Criteria

All tasks marked `[x]` AND:
- [x] App compiles without errors (`reflex run` succeeds)
- [ ] All filters work with instant (debounced) updates
- [x] KPIs display correct numbers matching filter state (verified via SQL queries)
- [x] Icicle chart renders and updates reactively (1,887 nodes generated correctly)
- [ ] Visual design matches DESIGN_SYSTEM.md
- [ ] No console errors during normal operation
- [x] Verified with real patient data from SQLite (440K records tested)