Commit Graph

164 Commits

Author SHA1 Message Date
Andrew Charlwood 139a71b752 docs: update progress.txt with iteration 17 completion (Task 5.6) 2026-02-05 02:16:28 +00:00
Andrew Charlwood 9b466b4e6c feat: add hover/focus states and clean up unused styles (Task 5.6)
- Add subtle hover states to KPI badges, dropdown triggers, tabs
- Add consistent focus rings for accessibility (2px Pale Blue)
- Update button styles with focus/active states
- Clean up unused styles: compact_kpi_* (Option B), unused imports
- All interactive elements now have appropriate hover/focus feedback
2026-02-05 02:16:01 +00:00
Andrew Charlwood 731db2d85f docs: update progress.txt with iteration 16 completion (Task 5.5) 2026-02-05 02:08:41 +00:00
Andrew Charlwood 754e98dbe5 feat: refine top bar with style helpers (Task 5.5)
- Use top_bar_style() for 48px height container
- Use logo_style() for 28px height logo (was 36px)
- Use top_bar_tab_style() for 28px height pills
- Simplify data freshness to single line
- Remove max_width constraint for full-width bar
- Lighter shadow (SM instead of MD)
2026-02-05 02:08:01 +00:00
Andrew Charlwood fc03e44ce2 docs: update progress.txt with iteration 15 completion (Task 5.4) 2026-02-05 02:04:42 +00:00
Andrew Charlwood ef2a109528 feat: full-width responsive chart layout (Task 5.4)
- Remove PAGE_MAX_WIDTH constraint from main_content()
- Update chart_display() with calc(100vh - 152px) height
- Update icicle_figure with autosize=True and reduced margins
- Update chart_section() with flex layout for height fill
- Update page_layout() with 100vh height
2026-02-05 02:03:55 +00:00
Andrew Charlwood 390328f2b4 docs: update progress.txt with iteration 14 completion (Task 5.3) 2026-02-05 01:59:35 +00:00
Andrew Charlwood 826dd1c022 feat: compact KPI badges integrated into filter strip (Task 5.3)
- Add kpi_badge() and kpi_badges() functions for inline pill-style KPIs
- Integrate KPI badges into filter_section() on the right side
- Remove separate kpi_row() from main_content() layout
- Zero extra vertical height - KPIs now share the filter strip row

Design: Follows Option A from DESIGN_SYSTEM.md (preferred approach)
2026-02-05 01:59:00 +00:00
Andrew Charlwood 645fe0ab6c docs: update progress.txt with iteration 13 completion (Task 5.2) 2026-02-05 01:54:13 +00:00
Andrew Charlwood d2bed71078 feat: compact filter section as single horizontal strip (Task 5.2)
- Redesign filter_section() as 48px horizontal strip
- Remove "Filters" header (saves vertical space)
- Compact initiated_filter_dropdown() and last_seen_filter_dropdown()
  - 32px height triggers via compact_dropdown_trigger_style()
  - Labels moved inside dropdown panels
- Compact searchable_dropdown() component
  - 32px trigger height, no external label
  - Reduced panel item height (150px max, was 200px)
  - Smaller search input (size="1"), tighter spacing
- All filters now in ONE row with divider separator

Target: filter section height ≤ 60px (from ~200px)
2026-02-05 01:53:38 +00:00
Andrew Charlwood c9654905be docs: update progress.txt with iteration 12 completion (Task 5.1) 2026-02-05 01:47:40 +00:00
Andrew Charlwood 0a68c2a5a5 feat: update design tokens for SaaS redesign (Task 5.1)
- Typography: Reduce sizes (Display 32→28, H1 24→18, H2 20→16, Caption 12→11)
- Spacing: Tighten scale by ~25% (SM 8→6, MD 12→8, LG 16→12, etc.)
- Shadows: Lighter values for modern feel
- Colors: Modernize semantic colors (#10B981 success, #EF4444 error)
- Layout: TOP_BAR_HEIGHT 64→48px, new FILTER_STRIP_HEIGHT 48px

New style helpers added:
- compact_kpi_card_style/value/label - 50% smaller KPI cards
- kpi_badge_style - inline pill variant for zero-height KPIs
- filter_strip_style - horizontal single-row container
- compact_dropdown_trigger_style - 32px height triggers
- chart_container_style/wrapper - full-width flex-grow
- top_bar_style/tab/logo - compact 48px top bar

All tokens verified via import and Reflex compile.
2026-02-05 01:46:58 +00:00
Andrew Charlwood 27d2d603c3 docs: update progress.txt with iteration 11 completion (Task 4.3 Documentation) 2026-02-05 00:57:48 +00:00
Andrew Charlwood 76e0d64820 docs: complete Task 4.3 Documentation
Update CLAUDE.md with new pathway data architecture:
- Add Pathway Data Architecture section with date filter table
- Update package structure with cli/ and pathway_pipeline.py
- Add CLI module and pathway pipeline documentation
- Update data flow diagrams (pre-computed vs legacy)
- Add pathway tables to database schema section
- Add CLI commands section with usage examples
- Add Breaking Changes section documenting:
  - Date filter changes (pickers -> dropdowns)
  - Data refresh model (real-time -> pre-computed)
  - State variable changes
  - Icicle chart enhancements

Mark all Task 4.3 subtasks complete in IMPLEMENTATION_PLAN.md
Update completion criteria status
2026-02-05 00:56:34 +00:00
Andrew Charlwood 49bf4cdf1b docs: update progress.txt with iteration 10 completion (Task 4.2 Performance) 2026-02-05 00:50:57 +00:00
Andrew Charlwood 870d2e6e0e feat: complete Task 4.2 Performance Testing - all targets met 2026-02-05 00:50:14 +00:00
Andrew Charlwood 58450d78fa docs: update progress.txt with iteration 9 completion (Task 4.1 E2E Validation) 2026-02-05 00:44:39 +00:00
Andrew Charlwood cabaa72e9d feat: complete Task 4.1 End-to-End Validation
All 5 validation tests pass:
- Hierarchy structure: 6 levels (Root→Trust→Directory→Drug→Pathway)
- Patient counts: 11,118 patients, £130.5M from root node
- Treatment statistics: average_spacing, cost_pp_pa populated
- Drug filtering: drug_sequence column for LIKE patterns
- Customdata: All 10 fields present and populated
2026-02-05 00:43:55 +00:00
Andrew Charlwood 7628c5fa20 docs: update progress.txt with iteration 8 completion (Task 3.3 UI Components) 2026-02-05 00:38:16 +00:00
Andrew Charlwood a6f1d8b30e feat: replace date pickers with select dropdowns (Task 3.3)
- Created initiated_filter_dropdown() and last_seen_filter_dropdown() components
- Uses rx.select.root pattern with static options for Reflex compatibility
- Updated filter_section() to use new dropdown components
- Removed old date_range_picker() function (replaced by new dropdowns)
- Data freshness indicator already working in top_bar via load_pathway_data()
- Verified: py_compile PASS, imports PASS, reflex compile PASS (11.1s)
2026-02-05 00:37:18 +00:00
Andrew Charlwood 8f2425a9ae docs: update progress.txt with iteration 7 completion (Task 3.2 Icicle Figure) 2026-02-05 00:31:10 +00:00
Andrew Charlwood ced994f93f feat: update icicle_figure with full 10-field customdata (Task 3.2)
Updated the icicle_figure computed property in AppState to use the full
10-field customdata structure matching visualization/plotly_generator.py:

- value (patient count)
- colour (proportion of parent)
- cost (total cost)
- costpp (cost per patient)
- first_seen (first intervention date)
- last_seen (last intervention date)
- first_seen_parent (earliest date in parent)
- last_seen_parent (latest date in parent)
- average_spacing (dosing information)
- cost_pp_pa (cost per patient per annum)

Updated texttemplate and hovertemplate to display treatment statistics
including duration, dosing, and full cost breakdown.
2026-02-05 00:30:22 +00:00
Andrew Charlwood fc3b3525c6 docs: update progress.txt with iteration 6 completion (Task 3.1 AppState) 2026-02-05 00:26:59 +00:00
Andrew Charlwood 7948ca7da3 feat: update AppState to query pre-computed pathway_nodes (Task 3.1)
- Add dropdown state for date filters (selected_initiated, selected_last_seen)
- Add date_filter_id computed property combining the two selections
- Add load_pathway_data() method to query pathway_nodes table
- Add recalculate_parent_totals() for filtered hierarchies
- Update all filter handlers to call load_pathway_data()
- Update KPI calculations from root node data

Phase 3 Reflex integration: Task 3.1 complete
2026-02-05 00:26:21 +00:00
Andrew Charlwood 0a13ba550e docs: update progress.txt with iteration 5 completion (Task 2.2 Pipeline Test) 2026-02-05 00:21:08 +00:00
Andrew Charlwood adc1dbfc58 feat: complete Task 2.2 - test refresh pipeline with Snowflake data
Tested full refresh pipeline end-to-end with real Snowflake data:
- Fixed trust filter to read Name column from defaultTrusts.csv
- Fixed Decimal type handling in calculate_cost_per_patient_per_annum
- Fixed array handling in convert_to_records for average_administered
- Added required reference CSV files to data/ directory
- Configured Snowflake connection (account, warehouse, user)

Results:
- Snowflake fetch: 656,695 records in ~7s
- Transformations: 519,848 records after UPID/drug/directory
- Pathway nodes: 293 for all_6mo (8 trusts, 14 directories)
- Total processing time: ~6.2 minutes
2026-02-05 00:20:12 +00:00
Andrew Charlwood 8b65dfd9a8 docs: update progress.txt with iteration 4 completion (Task 2.1 CLI) 2026-02-04 23:30:50 +00:00
Andrew Charlwood 092fdbba5a feat: add CLI refresh command for pathway data (Task 2.1)
Add cli/refresh_pathways.py with:
- refresh_pathways() main function for full pipeline orchestration
- insert_pathway_records() for SQLite insertion
- log_refresh_start/complete/failed() for refresh tracking
- CLI with --minimum-patients, --provider-codes, --dry-run, --verbose

Uses existing pipeline functions:
- fetch_and_transform_data() from pathway_pipeline.py
- process_all_date_filters() for 6 date filter combinations
- Schema helpers from data_processing/schema.py
2026-02-04 23:30:11 +00:00
Andrew Charlwood 9bb4748588 docs: mark Task 1.3 complete (migration already handled by schema)
Task 1.3 (Create Migration Script) is satisfied by existing code:
- python -m data_processing.migrate creates all pathway tables
- pathway_date_filters auto-populated via INSERT OR REPLACE in schema
- Verified: fresh database creates all 3 tables with 6 date filters
2026-02-04 23:25:14 +00:00
Andrew Charlwood b48dbbc96a docs: update progress.txt with iteration 2 completion (Task 1.2 Pipeline) 2026-02-04 23:21:50 +00:00
Andrew Charlwood 5945649ae3 feat: add pathway pipeline module (Task 1.2)
Create data_processing/pathway_pipeline.py with:
- DateFilterConfig dataclass for date filter configuration
- DATE_FILTER_CONFIGS with 6 pre-defined combinations
- compute_date_ranges() for computing actual dates from config
- fetch_and_transform_data() for Snowflake fetch + transformations
- process_pathway_for_date_filter() using existing generate_icicle_chart()
- extract_denormalized_fields() to parse trust/directory/drugs from ids
- convert_to_records() for SQLite insertion
- process_all_date_filters() convenience function
2026-02-04 23:21:39 +00:00
Andrew Charlwood f2717a2219 docs: update progress.txt with iteration 1 completion (Task 1.1 Schema) 2026-02-04 23:17:52 +00:00
Andrew Charlwood 34396fef5e feat: add pathway data architecture schema (Task 1.1)
Add three new tables to support pre-computed pathway data:
- pathway_date_filters: 6 pre-defined date filter combinations
- pathway_nodes: pre-computed pathway hierarchy with all visualization data
- pathway_refresh_log: tracks data refresh status

Includes:
- 8 indexes for efficient filtering by date_filter_id, trust, directory, drug
- Helper functions: create/drop/verify/get_counts for pathway tables
- clear_pathway_nodes() for selective or full data clearing
- get_pathway_refresh_status() for checking last refresh
- Integration with existing ALL_TABLES_SCHEMA and combined helpers
2026-02-04 23:17:27 +00:00
Andrew Charlwood 85b3c20341 docs: update progress.txt with iteration 17 completion (Task 5.2 Debounce) 2026-02-04 19:26:26 +00:00
Andrew Charlwood b93af40ac4 feat: implement debounced inputs for filter performance (Task 5.2)
- Wrap date inputs with rx.debounce_input() (300ms timeout)
- Wrap dropdown search inputs with rx.debounce_input() (300ms timeout)
- Prevents excessive filter/chart updates during user input
- Satisfies "instant (debounced) updates" completion criterion
2026-02-04 19:25:57 +00:00
Andrew Charlwood f863e79299 docs: update progress.txt with iteration 16 completion (Task 5.3 Error Handling) 2026-02-04 19:21:31 +00:00
Andrew Charlwood 6c3b3abf03 feat: improve error handling with user-friendly messages (Task 5.3)
- Updated all error messages to be more descriptive and actionable
- Verified error states render correctly in UI (loading, error, empty)
- Confirmed filter handlers check data_loaded before applying filters
- Error handling covers: missing DB, empty DB, SQLite errors, filter errors
2026-02-04 19:20:57 +00:00
Andrew Charlwood 8c04e65ced docs: update progress.txt with correct commit hash 2026-02-04 19:16:58 +00:00
Andrew Charlwood 0dd99e6a42 docs: complete visual polish audit against DESIGN_SYSTEM.md (Task 5.1)
- Verified all design tokens match spec exactly
- Confirmed responsive behavior via flex_wrap patterns
- Audited hover states and transitions
- Validated chart colorscale uses design system palette
2026-02-04 19:16:47 +00:00
Andrew Charlwood b5afb5deb2 docs: update progress.txt with iteration 14 verification results 2026-02-04 19:09:02 +00:00
Andrew Charlwood 307094eca3 docs: verify data loading, filter logic, and chart generation (Task 5.4 partial) 2026-02-04 19:08:09 +00:00
Andrew Charlwood fbf046eb8a docs: update progress.txt with iteration 13 completion 2026-02-04 18:59:38 +00:00
Andrew Charlwood e099538ea9 feat: enable app_v2 via package init, verify reflex compilation
- Update pathways_app/__init__.py to re-export app from app_v2
- Verified reflex run compiles 33/33 components successfully
- App runs on localhost:3003 (frontend) and :8002 (backend)
- Mark completion criteria "App compiles" as verified
2026-02-04 18:59:08 +00:00
Andrew Charlwood 5267a9f4ef docs: update progress.txt with iteration 12 completion 2026-02-04 18:49:44 +00:00
Andrew Charlwood 158d873750 feat: integrate rx.plotly() chart component (Task 4.3)
- Replace chart_ready_placeholder() with chart_display() function
- chart_display() wraps rx.plotly() with AppState.icicle_figure
- Chart updates reactively when filters change via computed property
- Loading, error, and empty states already handled in chart_section()
2026-02-04 18:49:13 +00:00
Andrew Charlwood a749514889 docs: update progress.txt with iteration 11 completion 2026-02-04 18:45:59 +00:00
Andrew Charlwood ec8f8dc198 feat: implement reactive Plotly icicle chart generation (Task 4.2)
- Add plotly.graph_objects import
- Create icicle_figure computed property (@rx.var)
- NHS-inspired blue gradient colorscale (Heritage Blue → Pale Blue)
- Custom hover template with patient count, percentage, and cost
- Responsive height (600px), transparent background
- Maintain hierarchy order with sort=False
2026-02-04 18:45:22 +00:00
Andrew Charlwood 14f970d37b feat: implement chart data preparation (Task 4.1)
- Add prepare_chart_data() method for hierarchical chart data
- Build Trust → Directory → Drug hierarchy from filtered SQLite data
- Calculate patient counts and costs at each hierarchy level
- Compute color values (proportions) for visualization
- Generate dynamic chart title based on filter state
- Call prepare_chart_data() from apply_filters() for reactivity
- Mark Task 3.4 complete (KPIs implemented in apply_filters)
2026-02-04 18:41:37 +00:00
Andrew Charlwood cd15ab6cdf feat: implement filter logic with reactive KPI updates (Task 3.3)
- Add apply_filters() method that queries SQLite with current filter state
- Handle initiated date filter (first intervention date range)
- Handle last seen date filter (last intervention date range)
- Handle drug and directorate multi-select filters
- Use CTE pattern for efficient patient-level date filtering
- Update KPI values (unique_patients, total_drugs, total_cost) on filter change
- Call apply_filters() from all filter event handlers
- Call apply_filters() after initial data load
2026-02-04 14:17:27 +00:00
Andrew Charlwood f38ccfc128 feat: implement data loading from SQLite (Task 3.2)
- Add load_data() method to AppState that connects to SQLite database
- Populate available_drugs, available_directorates, available_indications from DB
- Detect latest date in dataset and set filter defaults accordingly
- Load KPI values: total_records, unique_patients, total_drugs, total_cost
- Add on_load handler to trigger data loading on page initialization
- Handle database errors gracefully with meaningful error messages
2026-02-04 14:11:03 +00:00