Files
HighCostDrugsDemo/data/directory_list.csv
T
Andrew Charlwood adc1dbfc58 feat: complete Task 2.2 - test refresh pipeline with Snowflake data
Tested full refresh pipeline end-to-end with real Snowflake data:
- Fixed trust filter to read Name column from defaultTrusts.csv
- Fixed Decimal type handling in calculate_cost_per_patient_per_annum
- Fixed array handling in convert_to_records for average_administered
- Added required reference CSV files to data/ directory
- Configured Snowflake connection (account, warehouse, user)

Results:
- Snowflake fetch: 656,695 records in ~7s
- Transformations: 519,848 records after UPID/drug/directory
- Pathway nodes: 293 for all_6mo (8 trusts, 14 directories)
- Total processing time: ~6.2 minutes
2026-02-05 00:20:12 +00:00

59 lines
1.0 KiB
CSV

directory
ACCIDENT & EMERGENCY
ANAESTHETICS
ANTICOAGULANT SERVICE
AUDIOLOGY
BLOOD AND MARROW TRANSPLANTATION
BREAST SURGERY
CARDIOLOGY
CARDIOTHORACIC TRANSPLANTATION
CHEMICAL PATHOLOGY
CLINICAL HAEMATOLOGY
CLINICAL IMMUNOLOGY
CLINICAL NEUROPHYSIOLOGY
CLINICAL ONCOLOGY (PREVIOUSLY RADIOTHERAPY)
COLORECTAL SURGERY
CRITICAL CARE MEDICINE
DERMATOLOGY
DIABETIC MEDICINE
DIETETICS
ENDOCRINOLOGY
ENT
GASTROENTEROLOGY
GENERAL MEDICINE
GENERAL SURGERY
GERIATRIC MEDICINE
GYNAECOLOGICAL ONCOLOGY
GYNAECOLOGY
INFECTIOUS DISEASES
MEDICAL ONCOLOGY
MEDICAL OPHTHALMOLOGY
MIDWIFE EPISODE
NEONATOLOGY
NEPHROLOGY
NEUROLOGY
NEUROSURGERY
OBSTETRICS
OPHTHALMOLOGY
ORAL SURGERY
ORTHOPTICS
ORTHOTICS
PAEDIATRIC CARDIOLOGY
PAEDIATRIC DIABETIC MEDICINE
PAEDIATRIC GASTROENTEROLOGY
PAEDIATRIC UROLOGY
PAEDIATRICS
PAIN MANAGEMENT
PLASTIC SURGERY
REHABILITATION
RHEUMATOLOGY
STROKE MEDICINE
THORACIC MEDICINE
THORACIC SURGERY
TRANSIENT ISCHAEMIC ATTACK
TRANSPLANTATION SURGERY
TRAUMA & ORTHOPAEDICS
UPPER GASTROINTESTINAL SURGERY
UROLOGY
VASCULAR SURGERY