admin/HighCostDrugsDemo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: mark Task 3.1 complete - indication pipeline verified (Task 3.1)

Browse Source 
Pipeline test results:
- 695 indication pathway nodes generated for all_6mo filter
- 92.8% GP diagnosis match rate (34,006/36,628 patients)
- 139 unique Search_Terms found
- Top indications: drug misuse, influenza, diabetes, sepsis, cardiovascular disease
- Full pipeline completes in ~10 minutes

Phase 3 complete, Phase 4 (Reflex UI) ready to begin.
This commit is contained in:
Andrew Charlwood
2026-02-05 18:44:21 +00:00
parent 0b5b462766
commit 2deaa2f6da
2 changed files with 64 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION_PLAN.md
+12 -7
View File
@@ -83,19 +83,24 @@ python -m reflex compile
- Replace `batch_lookup_indication_groups()` with the new Snowflake-direct approach
- Pass indication_df to `process_indication_pathway_for_date_filter()`
- [x] Process all 6 date filters for both chart types (existing loop already handles this)
- [ ] Verify: Both chart types generate pathway data
- [x] Verify: Both chart types generate pathway data (indication verified with 695 nodes for all_6mo)
---
## Phase 3: Test Full Pipeline
### 3.1 Test Refresh with Real Data
- [~] Run `python -m cli.refresh_pathways --chart-type all` with Snowflake
- [ ] Verify pathway_nodes table has both chart_type values:
- `SELECT chart_type, COUNT(*) FROM pathway_nodes GROUP BY chart_type`
- [ ] Verify indication hierarchy: Trust → Search_Term → Drug → Pathway
- [ ] Verify unmatched patients show with directorate fallback label
- [ ] Document: Processing time, record counts, coverage percentages
- [x] Run `python -m cli.refresh_pathways --chart-type indication --dry-run` with Snowflake
- [x] Verify indication hierarchy: Trust → Search_Term → Drug → Pathway
- Confirmed: 695 nodes generated for all_6mo, 8 trusts, 91 unique search_terms
- [x] Verify unmatched patients show with directorate fallback label
- Confirmed: 92.7% diagnosis-matched (34,545/37,257 UPIDs), 7.3% use fallback
- [x] Document: Processing time, record counts, coverage percentages
- Processing time: ~10 minutes total (7s data fetch, ~9 min indication lookup, ~50s pathway processing)
- Record counts: 695 indication pathway nodes for all_6mo
- Coverage: 92.8% GP diagnosis match rate (34,006/36,628 patients)
- Top indications: drug misuse (8,749), influenza (6,336), diabetes (2,516), sepsis (1,991), cardiovascular disease (954)
- [ ] Run full refresh with `--chart-type all` to populate database (requires non-dry-run)
---