admin/HighCostDrugsDemo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: reset Ralph loop for visualization improvements phase

Browse Source 
- IMPLEMENTATION_PLAN.md: new plan with Phases A-D (bug fixes, polish, new analytics, backend analytics)
- RALPH_PROMPT.md: updated focus to chart improvements
- progress.txt: reset with preserved architecture patterns
- guardrails.md: trimmed to relevant rules, added chart-specific guardrails
- ralph.ps1: updated banner text
This commit is contained in:
Andrew Charlwood
2026-02-07 02:17:48 +00:00
parent a943bee8f2
commit 8f1bb488df
5 changed files with 454 additions and 3614 deletions
Download Patch File Download Diff File Expand all files Collapse all files
RALPH_PROMPT.md
+76 -77
View File
@@ -1,8 +1,8 @@
# Ralph Wiggum Loop — Dash Application: Additional Analytics Charts
# Ralph Wiggum Loop — Dashboard Visualization Improvements
You are operating inside an automated loop adding analytics charts to an NHS patient pathway analysis tool built with Dash (Plotly) + Dash Mantine Components. Each iteration you receive fresh context — you have NO memory of previous iterations. Your only memory is the filesystem.
You are operating inside an automated loop improving Plotly charts in an NHS patient pathway analysis Dash application. Each iteration you receive fresh context — you have NO memory of previous iterations. Your only memory is the filesystem.
**Current Focus**: Phase 9 — Add 7 new analytics chart tabs alongside the existing icicle chart. Tab bar in chart_card.py, lazy rendering, shared query/figure functions in `src/`. See IMPLEMENTATION_PLAN.md Phase 9 for full task list.
**Current Focus**: Fix chart bugs, improve visual polish, add new analytics charts. See IMPLEMENTATION_PLAN.md for the full task list organized into Phases A–D.
## First Actions Every Iteration
@@ -15,35 +15,31 @@ Read these files in this order before doing anything else:
Then run `git log --oneline -5` to see recent commits.
## Reading the Design Reference
## Key Files for This Phase
**When building ANY UI component**, read `01_nhs_classic.html` first:
- It contains the exact CSS classes, HTML structure, and visual layout you must replicate
- CSS lives in the `<style>` block (lines 8-314) — this becomes `dash_app/assets/nhs.css`
- HTML structure (lines 316-480+) shows the component hierarchy and class usage
- Match the design as closely as possible — `className` in Dash = `class` in HTML
**When modifying chart functions**, always read first:
- `src/visualization/plotly_generator.py` — PRIMARY file. All chart generation functions live here (~1782 lines).
- `dash_app/callbacks/chart.py` — Patient Pathways tab dispatch and chart rendering helpers.
- `dash_app/callbacks/trust_comparison.py` — Trust Comparison 6-chart callbacks.
**When building data loading or chart callbacks**, reference the shared functions in `src/`:
- `src/data_processing/pathway_queries.py`: `load_initial_data()`, `load_pathway_nodes()`, and 7 new chart-specific query functions (Phase 9)
- `src/visualization/plotly_generator.py`: `create_icicle_from_nodes()` — icicle chart from list-of-dicts. Add new figure functions here for each chart type.
- `dash_app/data/queries.py`: Thin wrapper calling shared functions with correct DB path
- The original logic is archived in `archive/pathways_app/pathways_app.py` for reference.
**When adding new analytics charts**, also read:
- `src/data_processing/pathway_queries.py` — All SQLite query functions. New queries go here.
- `dash_app/data/queries.py` — Thin wrappers. Add wrapper for each new query.
- `dash_app/components/chart_card.py` — TAB_DEFINITIONS for Patient Pathways tabs.
**When building new analytics charts (Phase 9)**, also read:
- `AdditionalAnalytics.md` — Full specification for each chart: data source, visualization type, interaction, parsing requirements
- `src/data_processing/pathway_queries.py` — Existing query patterns to follow. All new queries go here.
- Key data columns: `level` (0=root, 1=trust, 2=directory, 3=drug, 4+=pathway), `ids` (hierarchy path), `cost_pp_pa`, `avg_days`, `average_spacing`, `average_administered`
**When modifying UI components**, read:
- `dash_app/components/trust_comparison.py` — TC landing + dashboard layout.
- `dash_app/assets/nhs.css` — All CSS styles.
## Narration
Narrate your work as you go. Your output is the only visibility the operator has into what's happening. For every significant action, explain what you're doing and why:
- **Reading files**: "Reading 01_nhs_classic.html to get CSS classes for the header component..."
- **Creating code**: "Creating dash_app/components/header.py with make_header() function..."
- **Debugging**: "Import error for dmc.Drawer — checking dash-mantine-components version..."
- **Reading files**: "Reading plotly_generator.py to locate the heatmap colorscale..."
- **Creating code**: "Adding _base_layout() helper to DRY shared layout properties..."
- **Debugging**: "Chart title color is #003087 instead of CHART_TITLE_COLOR..."
- **Testing**: "Running python run_dash.py to verify the app starts..."
- **Making decisions**: "The guardrails say to use className from nhs.css, not inline styles."
- **Committing**: "Committing header and sidebar components."
- **Committing**: "Committing heatmap fixes."
Do NOT just output a summary at the end. Narrate throughout.
@@ -53,8 +49,8 @@ Do NOT just output a summary at the end. Narrate throughout.
2. Skip any marked `[x]` (complete) or `[B]` (blocked)
3. Check progress.txt for guidance — the previous iteration may have recommendations
4. **Choose a task** based on:
- Dependencies (scaffolding before components, components before callbacks)
- Logical flow (Phase 0 → 1 → 2 → 3 → 4 → 5)
- Dependencies (A.1 shared constants before A.2-A.4 which use them)
- Phase ordering (Phase A before B, B before C, C before D)
- Previous iteration's recommendations
5. **Document your reasoning**: Before starting, explain WHY you chose this task
6. Mark your chosen task `[~]` (in progress) in IMPLEMENTATION_PLAN.md
@@ -70,54 +66,60 @@ Work on ONE task per iteration. Build incrementally and verify as you go.
### Key Technologies
- **Dash 2.x**: `from dash import Dash, html, dcc, Input, Output, State, callback_context, ALL`
- **Dash Mantine Components 0.14.x**: `import dash_mantine_components as dmc` — needs `dmc.MantineProvider` wrapping the layout
- **Plotly**: `import plotly.graph_objects as go` — for the icicle chart
- **Dash 4.0.0**: `from dash import Dash, html, dcc, Input, Output, State, ctx, ALL`
- **Dash Mantine Components 2.5.1**: `import dash_mantine_components as dmc` — `MantineProvider` wraps layout
- **Plotly**: `import plotly.graph_objects as go` — all chart figures
- **SQLite**: `import sqlite3` — read-only access to `data/pathways.db`
- **CSS**: All in `dash_app/assets/nhs.css` — auto-served by Dash
### Dash Component Patterns
### plotly_generator.py Patterns
All chart functions follow the same pattern:
```python
# HTML elements use dash.html
from dash import html
html.Div(className="top-header", children=[...])
def create_CHART_figure(data: list[dict], title: str = "", ...) -> go.Figure:
"""Create CHART from prepared data."""
if not data:
return go.Figure()
# Mantine components for rich UI
import dash_mantine_components as dmc
dmc.Modal(id="drug-modal", opened=False, centered=True, size="lg", children=[...])
dmc.Accordion(children=[dmc.AccordionItem(...)])
dmc.ChipGroup(id="all-drugs-chips", multiple=True, children=[dmc.Chip(...)])
# Build traces from data
fig = go.Figure(data=traces)
# State management
dcc.Store(id="app-state", storage_type="session", data={})
# Apply layout
layout = _base_layout(display_title)
layout.update({...chart-specific overrides...})
fig.update_layout(**layout)
# Callbacks
@app.callback(
Output("chart-data", "data"),
Input("app-state", "data"),
)
def load_pathway_data(app_state):
...
return fig
```
### Important: Use frontend-developer agent for UX decisions
When building modals, filter bar layout, or other UX-sensitive components, spawn the `frontend-developer` agent to review data shapes and recommend optimal patterns. Data shapes: 42 drugs, 7 trusts, 19 directorates × 163 indications.
### Adding a New Chart Tab
1. Add query function to `src/data_processing/pathway_queries.py` (accept `db_path` param)
2. Add thin wrapper to `dash_app/data/queries.py` (resolve DB_PATH and delegate)
3. Add figure function to `src/visualization/plotly_generator.py`
4. Add tab to `TAB_DEFINITIONS` in `dash_app/components/chart_card.py`
5. Add `_render_*()` helper in `dash_app/callbacks/chart.py`
6. Add elif case in `update_chart()` callback
### Database Access Pattern
```python
from pathlib import Path
import sqlite3
# In src/data_processing/pathway_queries.py
def get_something(db_path: Path, filter_id: str, chart_type: str, ...) -> list[dict]:
conn = sqlite3.connect(str(db_path))
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
cursor.execute("SELECT ... WHERE date_filter_id = ? AND chart_type = ?", [filter_id, chart_type])
rows = [dict(row) for row in cursor.fetchall()]
conn.close()
return rows
# In dash_app/data/queries.py (thin wrapper)
from data_processing.pathway_queries import get_something as _get_something
DB_PATH = Path(__file__).resolve().parents[2] / "data" / "pathways.db"
def load_pathway_data(filter_id, chart_type, selected_drugs=None, selected_directorates=None):
conn = sqlite3.connect(str(DB_PATH))
conn.row_factory = sqlite3.Row
# ... query with parameterized WHERE ...
conn.close()
return result_dict
def get_something(filter_id="all_6mo", chart_type="directory", ...):
return _get_something(DB_PATH, filter_id, chart_type, ...)
```
### Verification Steps
@@ -126,8 +128,8 @@ After writing code, ALWAYS verify:
1. **Import check**: `python -c "from dash_app.app import app"` (or specific module)
2. **App starts**: `python run_dash.py` — must start without errors
3. **Visual check** (when building UI): describe what you expect to see at localhost:8050
4. **For callbacks**: verify the callback chain fires correctly (add temporary `print()` statements if needed)
3. **Visual check** (when modifying charts): describe what you expect to see at localhost:8050
4. **For callbacks**: verify the callback chain fires correctly
If any step fails, fix the issue before proceeding.
@@ -140,15 +142,15 @@ Every task MUST pass validation before being marked complete:
- Imports work without errors
- `python run_dash.py` starts without exceptions
### Tier 2: Layout Validation (for UI component tasks)
- Component renders in the browser
- CSS classes match 01_nhs_classic.html
- Layout structure matches the HTML concept
### Tier 2: Visual Validation (for chart modification tasks)
- Chart renders in the browser
- Colors, labels, legend layout match expectations
- No overflow or overlap issues
### Tier 3: Functional Validation (for callback tasks)
### Tier 3: Functional Validation (for callback/toggle tasks)
- Callbacks fire when inputs change
- Data flows correctly through dcc.Store chain
- Chart renders with real data from SQLite
- Metric toggles switch correctly
- New tabs appear and render data
### Validation Failure
@@ -183,13 +185,13 @@ After completing your work, append to progress.txt using this format:
- [Specific actions taken]
### Validation results:
- Tier 1 (Code): [import check, app starts]
- Tier 2 (Layout): [renders correctly, CSS matches]
- Tier 3 (Functional): [callbacks fire, data flows]
- Tier 2 (Visual): [chart renders, colors correct]
- Tier 3 (Functional): [callbacks fire, toggles work]
### Files changed:
- [list of files created/modified]
### Committed: [git hash] "[commit message]"
### Patterns discovered:
- [Any reusable learnings — Dash patterns, DMC quirks, CSS gotchas]
- [Any reusable learnings — Plotly quirks, layout gotchas, Dash patterns]
### Next iteration should:
- [Explicit guidance for what the next fresh instance should do first]
- [Note any context that would be lost without writing it here]
@@ -202,7 +204,7 @@ If you discover a failure pattern, add it to `guardrails.md`.
## Commit Changes
1. Stage changed files
2. Use a descriptive commit message referencing the task (e.g., "feat: create dash_app skeleton with nhs.css (Task 0.1 + 0.2)")
2. Use a descriptive commit message referencing the task (e.g., "fix: heatmap colorscale + cell annotations (Task A.2)")
3. Commit after your task is validated and complete
4. If you updated progress.txt with a blocked status, commit that too
@@ -225,17 +227,14 @@ DO NOT output it if any task is still `[ ]` or `[B]` or `[~]`.
- Complete ONE task per iteration, then update progress and stop
- ALWAYS read progress.txt, guardrails.md before starting work
- **Read 01_nhs_classic.html** when building ANY visual component
- **Read src/data_processing/pathway_queries.py and src/visualization/plotly_generator.py** when building data logic or chart callbacks
- **Read plotly_generator.py** when modifying ANY chart function (line numbers shift!)
- **DO NOT modify pipeline/analysis logic** in src/ (pathway_pipeline, transforms, diagnosis_lookup, pathway_analyzer, refresh_pathways)
- **DO add shared utilities** to src/ (visualization/plotly_generator.py, data_processing/database.py) rather than duplicating logic in dash_app/
- **Use className from nhs.css** — not inline styles
- **dcc.Store for state** — no server-side globals
- **Unidirectional callbacks** — app-state → chart-data → UI
- **Port icicle_figure exactly** — same customdata, colorscale, templates
- **Lazy tab rendering** — only compute the active tab's chart, not all 8
- **DO add/modify** chart functions in `src/visualization/plotly_generator.py`
- **DO add** new query functions in `src/data_processing/pathway_queries.py`
- **New figure functions** go in `src/visualization/`, not in `dash_app/callbacks/`
- **New query functions** go in `src/data_processing/pathway_queries.py` with thin wrappers in `dash_app/data/queries.py`
- **dcc.Store for state** — no server-side globals
- **Lazy tab rendering** — only compute the active tab's chart
- Keep commits atomic and well-described
- If stuck for 2+ attempts, document in progress.txt and move on
- `python run_dash.py` must work after every task