admin/HighCostDrugsDemo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: update design tokens for SaaS redesign (Task 5.1)

Browse Source 
- Typography: Reduce sizes (Display 32→28, H1 24→18, H2 20→16, Caption 12→11)
- Spacing: Tighten scale by ~25% (SM 8→6, MD 12→8, LG 16→12, etc.)
- Shadows: Lighter values for modern feel
- Colors: Modernize semantic colors (#10B981 success, #EF4444 error)
- Layout: TOP_BAR_HEIGHT 64→48px, new FILTER_STRIP_HEIGHT 48px

New style helpers added:
- compact_kpi_card_style/value/label - 50% smaller KPI cards
- kpi_badge_style - inline pill variant for zero-height KPIs
- filter_strip_style - horizontal single-row container
- compact_dropdown_trigger_style - 32px height triggers
- chart_container_style/wrapper - full-width flex-grow
- top_bar_style/tab/logo - compact 48px top bar

All tokens verified via import and Reflex compile.
This commit is contained in:
Andrew Charlwood
2026-02-05 01:46:58 +00:00
parent 27d2d603c3
commit 0a68c2a5a5
2 changed files with 493 additions and 296 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION_PLAN.md
+132 -208
View File
@@ -1,22 +1,23 @@
# Implementation Plan - Pathway Data Architecture
# Implementation Plan - UI Redesign Phase
## Project Overview
Pre-compute patient treatment pathways from Snowflake and store in SQLite for fast Reflex filtering. This replaces the current simplified `prepare_chart_data()` with full pathway hierarchy support.
Redesign the HCD Analysis application with a modern SaaS-style interface. The current NHS-dashboard style is functional but dated. This phase focuses on:
**Architecture**: Snowflake → Pathway Processing → SQLite (pre-computed) → Reflex (filter & view)
1. **Modern SaaS aesthetic** - Clean, ambitious design that looks like a premium product
2. **Maximized chart space** - The icicle chart is the hero; everything else supports it
3. **Compact controls** - Filters and KPIs should be efficient, not sprawling
4. **Full-width layout** - Use the entire viewport width
**Key Benefits**:
- Performance: Pathway calculation done once during data refresh, not on every filter
- Simplicity: Reflex filters pre-computed data with simple SQL WHERE clauses
- Full Pathways: Sequential treatment pathways (drug_0 → drug_1 → drug_2...) with statistics
**Design Philosophy**:
- Thematically aligned with blue color scheme but NOT constrained by NHS branding
- Think Stripe, Linear, Vercel - clean, spacious, confident
- Data visualization is the star; chrome should be minimal
**Design Reference**: See `PATHWAY_DATA_ARCHITECTURE_PLAN.md` for detailed architecture, schema, and data flow.
**Source Code**:
- Existing analysis: `analysis/pathway_analyzer.py`
- Existing visualization: `visualization/plotly_generator.py`
- Existing Reflex app: `pathways_app/app_v2.py`
**Source Files**:
- `pathways_app/pathways_app.py` - Main Reflex application
- `pathways_app/styles.py` - Design tokens and style helpers
- `DESIGN_SYSTEM.md` - Design specifications
## Quality Checks
@@ -24,222 +25,145 @@ Run after each task:
```bash
# Syntax check for Python files
python -m py_compile <file.py>
python -m py_compile pathways_app/pathways_app.py
# Import verification
python -c "from <module> import <class>"
python -c "from pathways_app.pathways_app import app"
# For Reflex changes
cd pathways_app && timeout 60 python -m reflex run 2>&1 | head -30
# Reflex compile
python -m reflex compile
```
## Phase 1: Schema & Data Pipeline Foundation
## Phase 5: UI Redesign
### 1.1 Extend Database Schema
- [x] Add `pathway_date_filters` table with 6 pre-defined combinations:
- `all_6mo`, `all_12mo`, `1yr_6mo`, `1yr_12mo`, `2yr_6mo`, `2yr_12mo`
- [x] Add `pathway_nodes` table with:
- Hierarchy structure (parents, ids, labels, level)
- Patient counts and costs (value, cost, costpp, cost_pp_pa)
- Date ranges (first_seen, last_seen, first_seen_parent, last_seen_parent)
- Treatment statistics (average_spacing, average_administered, avg_days)
- Denormalized filter columns (trust_name, directory, drug_sequence)
- Foreign key to date_filter_id
- [x] Add `pathway_refresh_log` table for tracking refresh status
- [x] Create indexes for efficient filtering
- [x] Verify schema with: `python -c "from data_processing.schema import *"`
### 5.1 Update Design System for Modern SaaS
- [x] Update `DESIGN_SYSTEM.md` with new specifications:
- Reduce top bar height from 64px to 48px
- Define compact filter row (single horizontal strip)
- Define compact KPI card dimensions (reduce padding, font sizes)
- Add full-width chart container specs
- [x] Update `pathways_app/styles.py` tokens to match:
- Typography: DISPLAY 32→28px, H1 24→18px, H2 20→16px, CAPTION 12→11px
- Spacing: SM 8→6px, MD 12→8px, LG 16→12px, XL 24→16px, XXL 32→24px, XXXL 48→32px
- Shadows: Lighter values (0.04, 0.06, 0.08 opacity)
- Colors: Modernized semantic colors (SUCCESS #10B981, etc.)
- Layout: TOP_BAR_HEIGHT 64→48px, FILTER_STRIP_HEIGHT 48px
- [x] Add new style helpers:
- `compact_kpi_card_style()` - 12px padding, 24px value font
- `compact_kpi_value_style()` / `compact_kpi_label_style()` - matching text styles
- `kpi_badge_style()` - inline pill/badge variant (zero height impact)
- `filter_strip_style()` - 48px horizontal container
- `compact_dropdown_trigger_style()` - 32px height triggers
- `searchable_dropdown_panel_style()` / `searchable_dropdown_item_style()` - compact items
- `chart_container_style()` / `chart_wrapper_style()` - full-width, flex-grow
- `top_bar_style()` / `top_bar_tab_style()` / `logo_style()` - 48px top bar
- [x] Verify: `python -c "from pathways_app.styles import *"` - PASSED
### 1.2 Create Pathway Pipeline Module
- [x] Create `data_processing/pathway_pipeline.py` with:
- `fetch_and_transform_data()` - Snowflake fetch + UPID/drug/directory transformations
- `process_pathway_for_date_filter(df, date_filter_config)` - Single filter processing
- `extract_denormalized_fields(ice_df)` - Extract trust, directory, drug_sequence from ids
- `convert_to_records(ice_df, date_filter_id)` - Convert ice_df to list of dicts for SQLite
- [x] Integrate with existing `analysis/pathway_analyzer.py` functions
- [x] Verify: `python -c "from data_processing.pathway_pipeline import *"`
### 5.2 Compact Filter Section (50-67% height reduction)
- [ ] Redesign filter_section() as a single horizontal strip:
- All filters in ONE row: Date dropdowns | Drugs | Indications | Directorates
- Remove "Filters" header (saves vertical space)
- Use smaller dropdown triggers (height: 32px instead of 40px)
- Use icon-only labels where possible
- [ ] Reduce searchable_dropdown() panel heights:
- Max item list height: 150px (was 200px)
- Smaller search input
- Tighter spacing (4px gaps instead of 8px)
- [ ] Make filter dropdowns collapsible/expandable (optional advanced feature)
- [ ] Verify: Filter section height ≤ 60px when collapsed
### 1.3 Create Migration Script
- [x] Create script to set up new tables in existing `data/pathways.db`
- Note: Existing `python -m data_processing.migrate` handles this (updated in Task 1.1)
- [x] Pre-populate `pathway_date_filters` with 6 combinations
- Note: Auto-populated via INSERT OR REPLACE in PATHWAY_DATE_FILTERS_SCHEMA
- [x] Verify migration runs cleanly on fresh database
- Verified: All 3 pathway tables created, 6 date filters populated correctly
### 5.3 Compact KPI Cards (50% reduction)
- [ ] Reduce KPI card dimensions:
- Padding: 12px (was 24px)
- Value font size: 24px (was 32px)
- Label font size: 11px (was 12px)
- [ ] Make KPIs a single compact row:
- All 4 KPIs in horizontal strip
- Minimal vertical footprint
- Consider inline layout: "12,345 patients | £45.2M cost | 89 drugs | 7 trusts"
- [ ] Alternative: KPI badges/pills instead of cards
- [ ] Verify: KPI row height ≤ 48px
## Phase 2: CLI Refresh Command
### 5.4 Full-Width Chart Layout
- [ ] Remove PAGE_MAX_WIDTH constraint for chart container
- [ ] Chart should stretch to viewport width (with small padding: 16px each side)
- [ ] Update chart height calculation:
- Use CSS calc() or flex-grow to fill remaining viewport height
- Minimum height: 500px
- Target: viewport height minus (top bar + filters + KPIs + padding)
- [ ] Update Plotly layout:
- Remove fixed height=600, use responsive sizing
- Reduce margins further for maximum chart area
- [ ] Verify: Chart fills available space on 1920x1080 display
### 2.1 Create Refresh Command
- [x] Create `cli/refresh_pathways.py` with:
- Uses DATE_FILTER_CONFIGS and compute_date_ranges from pathway_pipeline.py
- `refresh_pathways(minimum_patients, provider_codes, ...)` main function
- `insert_pathway_records()` for SQLite insertion
- `log_refresh_start/complete/failed()` for refresh tracking
- [x] Implement refresh flow:
1. Fetch ALL data from Snowflake (full date range) via fetch_and_transform_data()
2. Apply transformations (UPID, drug names, directory) - handled by pipeline
3. Clear existing pathway_nodes via clear_pathway_nodes()
4. For each of 6 date filter configs: filter → process → insert
5. Update pathway_refresh_log
- [x] Add CLI argument parsing (--minimum-patients, --provider-codes, --dry-run, --verbose)
- [x] Verify: `python -m cli.refresh_pathways --help`
### 5.5 Top Bar Refinement
- [ ] Reduce top bar height to 48px (was 64px)
- [ ] Simplify chart tabs - smaller pills or just text links
- [ ] Consider moving data freshness indicator inline with filters
- [ ] Make logo smaller (28px instead of 36px)
- [ ] Verify: Top bar is minimal but functional
### 2.2 Test Refresh Pipeline
- [x] Run refresh with Snowflake data
- Successfully fetched 656,695 records from Snowflake in ~7s
- Transformed to 519,848 records after UPID/drug/directory processing
- [x] Verify all 6 date_filter_ids populated in pathway_nodes
- Note: Only `all_6mo` has data (293 nodes) due to test data freshness
- Other filters (all_12mo, 1yr_*, 2yr_*) have no matching data in current Snowflake snapshot
- This is expected — the pipeline works, data just doesn't match date filters
- [x] Verify pathway structure matches original `generate_icicle_chart()` output
- Structure verified: N&WICS - TRUST - DIRECTORY - DRUG - PATHWAY levels
- 8 trusts, 14 directories represented correctly
- [x] Verify patient counts are correct (compare with original app)
- Sample: QEH RHEUMATOLOGY has 591 patients — consistent with expected volumes
- [x] Document estimated processing time (expect 6-12 minutes for 440K records)
- Actual: ~6.2 minutes (371.7s) for 656K → 519K → 293 nodes
- Breakdown: Snowflake fetch 7s, Transformations ~6min, Pathway processing ~30s
## Phase 3: Reflex Integration
### 3.1 Update AppState
- [x] Replace date picker state with dropdown state:
- `selected_initiated: str = "all"` ("all", "1yr", "2yr")
- `selected_last_seen: str = "6mo"` ("6mo", "12mo")
- Added `initiated_options` and `last_seen_options` for dropdown rendering
- Added `set_initiated_filter()` and `set_last_seen_filter()` event handlers
- [x] Add `date_filter_id` computed property: `f"{selected_initiated}_{selected_last_seen}"`
- [x] Rewrite `load_pathway_data()` to query `pathway_nodes` table:
- Base filter: `WHERE date_filter_id = ?`
- Trust/directory/drug filters on denormalized columns
- Updated all filter handlers to call `load_pathway_data()` instead of `apply_filters()`
- [x] Add `recalculate_parent_totals()` for filtered hierarchies
- [x] Update KPI calculations from root node data
- KPIs now extracted from root node (level 0) in pathway_nodes
- `unique_patients`, `total_cost`, `total_drugs` updated from query results
### 3.2 Update Icicle Figure
- [x] Update `icicle_figure` computed property to use all pathway_nodes columns
- [x] Match original 10-field customdata structure:
- values, colours, costs, costpp
- first_seen, last_seen, first_seen_parent, last_seen_parent
- average_spacing, cost_pp_pa
- [x] Restore full hover/text templates from `visualization/plotly_generator.py`
- [x] Verify chart renders correctly with treatment statistics
- Note: Structure validated via code inspection, visual verification pending Task 3.3 UI completion
### 3.3 Update UI Components
- [x] Replace date pickers with select dropdowns:
- Initiated: "All years", "Last 2 years", "Last 1 year"
- Last Seen: "Last 6 months", "Last 12 months"
- Note: Created `initiated_filter_dropdown()` and `last_seen_filter_dropdown()` components using `rx.select.root` pattern
- [x] Add "Data refreshed: X ago" indicator from pathway_refresh_log
- Note: Already implemented in top_bar() using `last_updated_display` computed property
- Uses pathway_refresh_log.completed_at via `load_pathway_data()`
- [x] Update filter section layout
- Replaced `date_range_picker` calls with new dropdown components
- Simplified filter section layout with cleaner structure
- [x] Verify UI compiles and renders correctly
- python -m py_compile: PASS
- Import check: PASS
- python -m reflex compile: PASS (11.095 seconds)
## Phase 4: Testing & Validation
### 4.1 End-to-End Validation
- [x] **Pathway hierarchy matches original**: Compare specific pathway ids structure
- Verified: 6 levels (Root → Trust → Directory → Drug → Pathway steps)
- 293 nodes total for all_6mo filter
- [x] **Patient counts match**: Compare root patient count for same date range
- Root: 11,118 patients, £130.5M total cost
- ~32% of fact_interventions patients (filtered by last 6 months)
- [x] **Treatment statistics display correctly**: Verify "Average treatment duration" hover data
- average_spacing, cost_pp_pa, first_seen, last_seen populated for drug nodes
- Sample: ADALIMUMAB shows 35.6 treatments, £3,384/patient/annum
- [x] **Drug filtering works**: Filter to FARICIMAB, verify correct pathways shown
- drug_sequence column populated for LIKE pattern matching
- Sample sequences: OMALIZUMAB, ADALIMUMAB, INFLIXIMAB, ETANERCEPT
- [x] **Chart renders with all tooltip data**: Verify 10-field customdata structure
- All 10 fields present: value, colour, cost, costpp, first_seen, last_seen,
first_seen_parent, last_seen_parent, average_spacing, cost_pp_pa
### 4.2 Performance Testing
- [x] Measure filter change response time (target: <500ms)
- Actual: ~51ms (10% of budget) — queries 2-4ms + chart 47ms
- [x] Measure initial page load (target: <2s including data load)
- Actual: ~51ms (2.5% of budget)
- [x] Verify chart interaction (zoom, hover) is smooth with no lag
- 293 nodes well within Plotly's 10K+ capability
- [x] Test with full dataset
- 440K fact_interventions → 293 pathway_nodes (pre-computed)
- Database queries: all <5ms (100x under target)
- Chart generation: ~48ms average
### 4.3 Documentation
- [x] Update CLAUDE.md with new architecture
- Added Pathway Data Architecture section with date filter table
- Updated package structure with cli/ and pathway_pipeline.py
- Added CLI module documentation
- Added pathway pipeline documentation
- Updated data flow diagrams (pre-computed vs legacy)
- Added pathway tables to database schema
- [x] Document CLI usage for `refresh_pathways`
- Added CLI commands section with examples
- Documented refresh workflow (fetch → transform → process → insert)
- [x] Update README with new run instructions
- Note: No separate README exists — CLAUDE.md serves as primary documentation
- Added database migration command to run instructions
- Added CLI refresh command to run instructions
- [x] Document any breaking changes from original app
- Added "Breaking Changes from Original App" section
- Documented date filter changes (pickers → dropdowns)
- Documented data refresh model changes
- Documented state variable changes
- Documented icicle chart enhancements
### 5.6 Visual Polish
- [ ] Add subtle hover states to interactive elements
- [ ] Ensure consistent focus rings for accessibility
- [ ] Test responsive behavior at common breakpoints (1366, 1920, 2560px widths)
- [ ] Remove any unused styles from styles.py
- [ ] Verify: No visual regressions, app looks cohesive
## Completion Criteria
All tasks marked `[x]` AND:
- [x] App compiles without errors (`reflex run` succeeds)
- Verified: `python -m reflex compile` succeeds in 2.8s
- [x] All 6 date filter combinations work correctly
- Verified: Code handles all 6 filters (all_6mo, all_12mo, 1yr_6mo, 1yr_12mo, 2yr_6mo, 2yr_12mo)
- Note: Only `all_6mo` has data currently (other filters have no matching records in Snowflake)
- This is a data freshness issue, not a code issue — pipeline correctly processes all filters
- [x] Drug/directory/trust filters work with instant updates
- Verified: Query time <5ms for all filter combinations
- [x] KPIs display correct numbers matching filter state
- Verified: unique_patients=11,118, total_cost=£130.5M from root node
- [x] Icicle chart renders with full pathway data and statistics
- Verified: 10-field customdata structure, all fields populated
- [x] Treatment duration and dosing information displays in tooltips
- Verified: average_spacing contains full dosing info string
- [x] No console errors during normal operation
- Verified: python -m py_compile passes, imports successful, reflex compile succeeds
- Note: Interactive browser testing requires manual verification
- [x] Verified with real patient data from Snowflake
- Verified: 656K records fetched, 293 pathway nodes generated
- [ ] App compiles without errors (`reflex compile` succeeds)
- [ ] Filter section height ≤ 60px (measured visually)
- [ ] KPI row height ≤ 48px (measured visually)
- [ ] Top bar height = 48px
- [ ] Chart stretches to full viewport width (minus 32px total padding)
- [ ] Chart fills remaining vertical space (min 500px)
- [ ] Design feels like modern SaaS, not NHS dashboard
- [ ] All interactive elements have appropriate hover/focus states
## Reference
### Date Filter Combinations
### Current Layout (to be improved)
```
┌─────────────────────────────────────────────────────────────────┐
│ Top Bar (64px) - Logo, Tabs, Freshness │
├─────────────────────────────────────────────────────────────────┤
│ Filter Section (~200px) - Headers, Date dropdowns, Multi-select│
├─────────────────────────────────────────────────────────────────┤
│ KPI Cards (~100px) - 4 large cards in row │
├─────────────────────────────────────────────────────────────────┤
│ Chart (~600px fixed) - Constrained width │
└─────────────────────────────────────────────────────────────────┘
```
| ID | Initiated | Last Seen | Default |
|----|-----------|-----------|---------|
| `all_6mo` | All years | Last 6 months | Yes |
| `all_12mo` | All years | Last 12 months | No |
| `1yr_6mo` | Last 1 year | Last 6 months | No |
| `1yr_12mo` | Last 1 year | Last 12 months | No |
| `2yr_6mo` | Last 2 years | Last 6 months | No |
| `2yr_12mo` | Last 2 years | Last 12 months | No |
### Target Layout
```
┌─────────────────────────────────────────────────────────────────┐
│ Logo │ Tabs │ │ Freshness │ 48px
├─────────────────────────────────────────────────────────────────┤
│ [Date▾] [Date▾] [Drugs▾] [Indications▾] [Directories▾] │ KPIs │ 48-60px
├─────────────────────────────────────────────────────────────────┤
│ │
│ CHART │ flex-grow
│ (full width) │
│ │
└─────────────────────────────────────────────────────────────────┘
```
### Key Measurements
| Element | Current | Target |
|---------|---------|--------|
| Top bar | 64px | 48px |
| Filter section | ~200px | ≤60px |
| KPI row | ~100px | ≤48px (or merged with filters) |
| Chart | 600px fixed, constrained width | flex-grow, full width |
| Total overhead | ~364px | ~156px (57% reduction) |
### Key Files
| File | Purpose |
|------|---------|
| `data_processing/schema.py` | Database schema definitions |
| `data_processing/pathway_pipeline.py` | New pathway processing pipeline |
| `cli/refresh_pathways.py` | CLI refresh command |
| `analysis/pathway_analyzer.py` | Existing pathway analysis logic |
| `visualization/plotly_generator.py` | Existing chart generation |
| `pathways_app/app_v2.py` | Reflex application |
| `pathways_app/pathways_app.py` | Main Reflex application |
| `pathways_app/styles.py` | Design tokens and style helpers |
| `DESIGN_SYSTEM.md` | Design specifications |