Commit Graph

109 Commits

Author SHA1 Message Date
Andrew Charlwood 8f1bb488df chore: reset Ralph loop for visualization improvements phase
- IMPLEMENTATION_PLAN.md: new plan with Phases A-D (bug fixes, polish, new analytics, backend analytics)
- RALPH_PROMPT.md: updated focus to chart improvements
- progress.txt: reset with preserved architecture patterns
- guardrails.md: trimmed to relevant rules, added chart-specific guardrails
- ralph.ps1: updated banner text
2026-02-07 02:17:48 +00:00
Andrew Charlwood caf7ada287 style: improve filter modal aesthetics for large screens (Task 11.4)
- Drug modal: size 70%, sm chips in 4-col SimpleGrid layout
- Trust modal: size lg, md chips in vertical Stack layout
- Directorate modal: size 70%, larger badges and text
- CSS: modal title full-width, chip nowrap, scroll padding
2026-02-07 00:37:47 +00:00
Andrew Charlwood 10d47d616c style: make Patient Pathways chart fill available viewport height (Task 11.3)
- Changed .main from min-height to height with overflow-y auto
- Added flex chain: #view-container → #patient-pathways-view → .chart-card
  → dcc.Loading wrapper → #chart-container → #pathway-chart
- All flex parents get min-height: 0 for proper flex shrinking
- dcc.Graph set to responsive=True with flex: 1 and minHeight: 0
- Trust Comparison view scrolls naturally via .main overflow-y auto
2026-02-07 00:31:47 +00:00
Andrew Charlwood a4059b5829 style: increase Trust Comparison chart height from 320px to 500px (Task 11.2)
Charts were overspilling 320px containers. Increased to 500px for
readability. Added overflow: hidden + min-height: 0 to .tc-chart-cell
to prevent any content leaking. Dashboard scrolls vertically, which
is acceptable for 6 readable charts.
2026-02-07 00:25:08 +00:00
Andrew Charlwood 99ccddbb98 style: move Clear All button to left side next to filter buttons (Task 11.1)
Moved Clear All from right-aligned separate sibling into the
pathway-filters__buttons group. Removed justify-content: space-between
from .pathway-filters. Added thin vertical separator between
Directorates and Clear All for visual distinction.
2026-02-07 00:20:11 +00:00
Andrew Charlwood 5f890f2d93 docs: final integration verification + CLAUDE.md update for two-view architecture (Task 10.11) 2026-02-06 22:55:22 +00:00
Andrew Charlwood c5b6191a5b style: clean up dead CSS, add CSS variables, improve responsive rules (Task 10.10)
- Remove dead .kpi-row/.kpi-card CSS (KPI row replaced by header fraction KPIs)
- Remove dead .top-header__breadcrumb CSS (breadcrumb removed in 10.3)
- Remove dead .filter-select CSS (replaced by dcc.Dropdown)
- Remove dead .sidebar__item--disabled CSS (disabled items removed in restructure)
- Delete unused dash_app/components/kpi_row.py
- Add --sub-header-h and --header-total-h CSS variables
- Update .main and .sub-header to use CSS variables instead of hardcoded values
- Add responsive rules: header KPIs collapse at 1200px, hide at 768px
- Add responsive pathway-filters wrap at 768px
- Remove dead .kpi-row responsive rules
2026-02-06 22:39:48 +00:00
Andrew Charlwood f1eecf88bf feat: scope pathway filters to Patient Pathways view with design-spec CSS (Task 10.9) 2026-02-06 22:32:02 +00:00
Andrew Charlwood ea6b9065bf feat: Trust Comparison 6-chart dashboard with real data (Task 10.8)
- Add 3 new visualization functions to plotly_generator.py:
  create_trust_market_share_figure, create_trust_heatmap_figure,
  create_trust_duration_figure
- Replace 6 placeholder callbacks in trust_comparison.py with real
  implementations using trust-comparison queries + figure builders
- Cost Waterfall reuses existing figure function via key mapping
- Dosing reuses existing create_dosing_figure with group_by="trust"
- Cost Effectiveness reuses existing function scoped to directorate
- All 6 charts respond to date filter and chart type toggle
- Validated with both directory (RHEUMATOLOGY) and indication (asthma)
2026-02-06 22:23:47 +00:00
Andrew Charlwood 10739ca84d feat: Trust Comparison landing page + directorate selector (Task 10.7)
- Add get_directorate_summary() query for per-directorate patient/drug counts
- Create trust_comparison.py with landing grid and 6-chart dashboard layout
- Wire directorate card clicks and back button through app-state callbacks
- Add TC landing and dashboard CSS per Phase 10 design spec
- Placeholder charts for 6 dashboard graphs (filled in Task 10.8)
- Chart type toggle clears selected directorate when switching modes
2026-02-06 22:15:10 +00:00
Andrew Charlwood 9d4e32910d feat: add 5 trust-comparison query functions for Phase 10.6
New per-trust-within-directorate queries:
- get_trust_market_share: drugs by trust within a directorate
- get_trust_cost_waterfall: cost per patient by trust
- get_trust_dosing: drug dosing intervals by trust
- get_trust_heatmap: trust x drug matrix for one directorate
- get_trust_durations: drug durations by trust

Also verified existing get_pathway_costs(directory=X) works for
directorate-scoped Cost Effectiveness (no new function needed).

Thin wrappers added in dash_app/data/queries.py.
2026-02-06 22:04:43 +00:00
Andrew Charlwood 979b79794c feat: reduce Patient Pathways to Icicle + Sankey tabs only (Task 10.5)
Removed 6 chart tabs (Market Share, Cost Effectiveness, Cost Waterfall,
Dosing, Heatmap, Duration) from the Patient Pathways tab bar. These will
reappear in the Trust Comparison dashboard (Task 10.8).

All _render_* helper functions preserved in chart.py for reuse.
2026-02-06 21:57:06 +00:00
Andrew Charlwood 7e0c851063 feat: add global filter sub-header bar (Task 10.4)
Extract chart type toggle and date filter dropdowns from filter_bar.py
into a new sub-header component. Sub-header is fixed-position below the
main header, visible across both views. Filter bar now contains only
drug/trust/directorate buttons for Patient Pathways view.
2026-02-06 21:51:56 +00:00
Andrew Charlwood 11b5cc5b81 feat: header fraction KPIs replacing KPI row (Task 10.3)
Remove KPI card row, add 3 inline fraction KPIs to header bar:
filtered/total patients, drugs, cost. Breadcrumb removed.
KPI callback refactored for 6 output IDs (3 filtered + 3 total).
total_cost added to load_initial_data() reference data.
2026-02-06 21:45:13 +00:00
Andrew Charlwood 7d51efc25e feat: add two-view architecture with sidebar navigation (Task 10.2)
- Add active_view and selected_comparison_directorate to app-state
- Sidebar: rename to Patient Pathways + add Trust Comparison nav item
- View container pattern: two view divs toggled by active_view
- Navigation callback: sidebar clicks switch views + update active state
- Trust Comparison placeholder landing page with tc-landing structure
2026-02-06 21:38:12 +00:00
Andrew Charlwood 94b1ac640a docs: design specification for Phase 10 two-view architecture (Task 10.1) 2026-02-06 21:33:43 +00:00
Andrew Charlwood 547bc7c867 docs: complete Phase 9 final integration (Task 9.10)
All 8 chart tabs verified — queries, figures, and filter dispatch
tested in both directory and indication modes. CLAUDE.md updated
with new chart types, query functions, and parsing utilities.
Phase 9 completion criteria all satisfied.
2026-02-06 20:22:21 +00:00
Andrew Charlwood 965fc8c3d2 feat: add Treatment Duration bar chart (Task 9.9) 2026-02-06 20:12:01 +00:00
Andrew Charlwood 0af76e68e0 feat: add Directorate × Drug Heatmap chart (Task 9.8) 2026-02-06 20:04:19 +00:00
Andrew Charlwood 02fe4b4e28 feat: add Dosing Interval Comparison chart (Task 9.7) 2026-02-06 19:58:28 +00:00
Andrew Charlwood 4ffcdf4268 feat: add Drug Switching Sankey diagram (Task 9.6) 2026-02-06 19:50:43 +00:00
Andrew Charlwood 73a8d1a49f feat: add Cost Waterfall bar chart (Task 9.5) 2026-02-06 19:44:37 +00:00
Andrew Charlwood 4ef7239eed feat: add Pathway Cost Effectiveness lollipop chart (Task 9.4)
- Create create_cost_effectiveness_figure() in plotly_generator.py
  Horizontal lollipop chart with dot size by patient count,
  colour gradient green→amber→red by cost, retention annotations
- Fix calculate_retention_rate() to accept both 'value' and 'patients' keys
- Add _render_cost_effectiveness() dispatch in chart.py callbacks
- Wire into tab switching for active_tab='cost-effectiveness'
2026-02-06 19:38:54 +00:00
Andrew Charlwood f8960a3064 feat: add First-Line Market Share chart (Task 9.3)
- create_market_share_figure() in src/visualization/plotly_generator.py
- Horizontal stacked bar chart: directorates × drugs with patient %
- Wire into tab dispatch via _render_market_share() helper in chart.py
- Responds to date, chart type, trust, and directorate filters
2026-02-06 19:28:20 +00:00
Andrew Charlwood d98cd4fd69 feat: add 7 analytics chart query functions (Task 9.2)
New query functions in src/data_processing/pathway_queries.py:
- get_drug_market_share: Level 3 drug nodes grouped by directory
- get_pathway_costs: Level 4+ pathway nodes with cost_pp_pa
- get_cost_waterfall: Directorate cost per patient from level 3 aggregation
- get_drug_transitions: Sankey source/target drug transitions with ordinal line labels
- get_dosing_intervals: Parsed average_spacing by trust/directory
- get_drug_directory_matrix: Directory x drug pivot with patient/cost metrics
- get_treatment_durations: Weighted avg_days by drug within directorates

Thin wrappers added in dash_app/data/queries.py for all 7 functions.
2026-02-06 19:21:10 +00:00
Andrew Charlwood fe2d048a21 feat: add parsing utilities and 8-tab chart infrastructure (Task 9.1)
- Create src/data_processing/parsing.py with parse_average_spacing(),
  parse_pathway_drugs(), and calculate_retention_rate()
- Add 8-tab bar to chart_card.py (Icicle, Market Share, Cost Effectiveness,
  Cost Waterfall, Sankey, Dosing, Heatmap, Duration)
- Add active-tab dcc.Store and tab switching callback in chart.py
- Remove Chart Views section from sidebar (now in tab bar)
- Lazy rendering: only active tab's chart is computed
2026-02-06 19:13:19 +00:00
Andrew Charlwood de08d4b520 fix: prune empty ancestor nodes and update KPIs for filtered views (Section 8)
- Add _prune_empty_ancestors() to remove directorate/trust nodes with no
  matching children when drug or directorate filters are active (e.g.,
  filtering by Immunoglobulin no longer shows empty Ophthalmology box)
- Sum level-3 drug nodes for KPI values when entity filters are active
  instead of using the root node's pre-computed unfiltered totals
2026-02-06 16:25:56 +00:00
Andrew Charlwood f2c5b2645e refactor: replace dmc.Drawer with dmc.Modal for filter selection (Task 7.4 + 7.5)
- Created 3 separate modals: Drug Selection (lg), Trust Selection (sm),
  Directorate Browser (xl) with centered overlay
- Added filter trigger buttons to filter bar with count badges
- Added "Clear All" button in filter bar for global filter reset
- Per-modal clear buttons for drugs and trusts
- Preserved all existing selection logic (same component IDs)
- Deleted drawer.py component and callbacks (replaced by modals.py)
- Updated CSS: filter-btn styles, modal chip/badge styles
2026-02-06 15:42:48 +00:00
Andrew Charlwood 7aa49b0d6b refactor: restructure sidebar with chart views, remove placeholder items (Task 7.3)
- Remove non-functional sidebar items: Cost Analysis, Export Data
- Remove filter trigger items: Drug/Trust/Directory Selection, Indications
- Add Chart Views section: Icicle Chart (active), Sankey Diagram (disabled), Timeline (disabled)
- Remove tab row from chart_card.py (chart view selection now in sidebar)
- Remove open_drawer callback (sidebar no longer has filter triggers)
- Add .sidebar__item--disabled CSS class
2026-02-06 15:29:53 +00:00
Andrew Charlwood 00627a7299 fix: preserve ancestor nodes in drug/directorate filters to prevent broken icicle hierarchy (Task 7.2)
Drug filter WHERE clause used `drug_sequence IS NULL` to keep ancestor nodes,
but levels 0-2 have empty string '' not NULL. Changed to level-based gating:
- Drug filter: `(level < 3 OR drug_sequence LIKE ...)`
- Directorate filter: `(level < 2 OR directory IN (...) OR directory IS NULL OR directory = '')`
- Trust filter was already correct (had `OR trust_name = ''`)
2026-02-06 15:24:09 +00:00
Andrew Charlwood 7be136ac87 fix: resolve DuplicateIdError by including search_term in drug-fragment badge IDs (Task 7.1)
Badge IDs changed from f"{directorate}|{frag}" to f"{directorate}|{search_term}|{frag}"
to handle fragments appearing under multiple indications within the same directorate.
Callback parsing updated to use rsplit("|", 1)[-1] for the 3-part key.
2026-02-06 15:19:18 +00:00
Andrew Charlwood 54b4a0f743 docs: update all documentation for Dash migration (Phase 6)
Rewrote README.md, USER_GUIDE.md, and DEPLOYMENT.md to reflect
the Dash application. Updated RALPH_PROMPT.md, guardrails.md, and
DESIGN_SYSTEM.md to remove Reflex references. All non-archive
documentation now reflects the current Dash + DMC architecture.
2026-02-06 14:54:12 +00:00
Andrew Charlwood fe8642dfaf feat: remove Reflex, archive old app, update docs for Dash migration (Task 5.4)
- Remove reflex dependency from pyproject.toml
- Move pathways_app/ and rxconfig.py to archive/
- Update CLAUDE.md: Dash app structure, callback chain, run command
- All completion criteria validated (10/10 pass)
2026-02-06 14:35:43 +00:00
Andrew Charlwood e877268805 feat: add data freshness indicator with relative time and patient count (Task 5.3) 2026-02-06 14:21:45 +00:00
Andrew Charlwood 5593d08062 feat: add loading spinner, empty state, and error handling to chart area (Task 5.2) 2026-02-06 14:16:26 +00:00
Andrew Charlwood f0505ee43e feat: add trust selection to drawer with filter wiring (Task 5.1) 2026-02-06 14:09:36 +00:00
Andrew Charlwood fe76e5a313 feat: add drawer callbacks for drug selection, fragment matching, and clear (Task 4.2) 2026-02-06 13:59:00 +00:00
Andrew Charlwood 5dc552f8c5 feat: add dmc.Drawer drug browser with directorate cards and drug chips (Task 4.1) 2026-02-06 13:51:24 +00:00
Andrew Charlwood 40ce7fc5f9 feat: add icicle chart rendering with NHS colorscale and dynamic titles (Task 3.4)
- Add create_icicle_from_nodes() to src/visualization/plotly_generator.py
  accepting list-of-dicts from dcc.Store with NHS blue gradient colorscale,
  10-field customdata, and matching text/hover templates from Reflex version
- Add update_chart callback to dash_app/callbacks/chart.py rendering
  go.Icicle figure from chart-data store with dynamic subtitle
- Title generation helper mirrors Reflex _generate_pathway_chart_title()
2026-02-06 13:44:13 +00:00
Andrew Charlwood 9c971c083b feat: add KPI update callback with formatted patient/drug/cost display (Task 3.3) 2026-02-06 13:38:11 +00:00
Andrew Charlwood ad9fa1cfec feat: add pathway data loading callback bridging filters to chart-data (Task 3.2) 2026-02-06 13:33:31 +00:00
Andrew Charlwood eda35c7168 feat: add reference data loading and filter state callbacks (Task 3.1) 2026-02-06 13:29:30 +00:00
Andrew Charlwood 3568e03fc2 feat: add footer component and complete Phase 2 static layout (Task 2.3) 2026-02-06 13:24:33 +00:00
Andrew Charlwood 307563bb31 feat: add KPI row, filter bar, and chart card components (Task 2.2) 2026-02-06 13:20:42 +00:00
Andrew Charlwood bdc1690f0f feat: add header and sidebar components for Dash layout (Task 2.1)
- header.py: NHS branded top bar with logo, title, breadcrumb,
  data freshness indicators (record count + last updated with IDs
  for callback updates)
- sidebar.py: Navigation with 7 items across Analysis/Reports
  sections, SVG icons via data URI, Drug Selection and Indications
  items have IDs for drawer open callbacks (Phase 4)
- app.py: Assembles header + sidebar + main content placeholder
- nhs.css: Added .sidebar__icon rule for img-based SVG icons
2026-02-06 13:13:03 +00:00
Andrew Charlwood 76549420a0 feat: add directorate card tree builder for drug browser drawer (Task 1.2) 2026-02-06 13:06:29 +00:00
Andrew Charlwood b71748fa7d feat: add shared pathway query functions for Dash data access (Task 1.1)
Extract load_data() and load_pathway_data() logic from Reflex AppState
into standalone functions in src/data_processing/pathway_queries.py.
Create thin dash_app/data/queries.py wrapper with DB_PATH resolution.
2026-02-06 13:02:34 +00:00
Andrew Charlwood 1c3ece6480 feat: create dash_app skeleton with nhs.css and MantineProvider (Phase 0)
- dash_app/ directory structure: app.py, assets/, data/, components/, callbacks/, utils/
- run_dash.py entry point at project root
- Added dash>=2.14.0 and dash-mantine-components>=0.14.0 to pyproject.toml
- app.py: Dash app with MantineProvider wrapper and 3 dcc.Store components
- nhs.css: extracted from 01_nhs_classic.html (sans mock icicle CSS)
- Validated: app starts cleanly at localhost:8050
2026-02-06 12:57:47 +00:00
Andrew Charlwood f3bba6dfab docs: complete Phase 4 validation — full refresh and data verification (Task 4.1-4.3)
Full refresh: 2,947 nodes (1,101 directory + 1,846 indication) in 738s.
Validation: RA/asthma drugs correctly grouped, fallback labels present,
directory charts unchanged, Reflex compiles. All completion criteria met.
2026-02-06 00:12:53 +00:00
Andrew Charlwood c6e426e36c fix: increase network timeout and batch size for GP lookup queries (Task 3.2)
Dry run test revealed GP lookup queries timing out at 30s (connection_timeout
in snowflake.toml). Increased to 600s. Also increased batch_size from 500 to
5000 — query time is ~40s regardless of batch size (CTE compilation overhead),
so larger batches reduce total time from ~50min to ~6min for 36K patients.

Dry run results: 91.8% GP match rate, 49.3% drug-indication match rate,
42,072 modified UPIDs, 1,846 pathway nodes across 6 date filters.
2026-02-05 23:55:12 +00:00