Files
HighCostDrugsDemo/data/treatment_function_codes.csv
Andrew Charlwood adc1dbfc58 feat: complete Task 2.2 - test refresh pipeline with Snowflake data
Tested full refresh pipeline end-to-end with real Snowflake data:
- Fixed trust filter to read Name column from defaultTrusts.csv
- Fixed Decimal type handling in calculate_cost_per_patient_per_annum
- Fixed array handling in convert_to_records for average_administered
- Added required reference CSV files to data/ directory
- Configured Snowflake connection (account, warehouse, user)

Results:
- Snowflake fetch: 656,695 records in ~7s
- Transformations: 519,848 records after UPID/drug/directory
- Pathway nodes: 293 for all_6mo (8 trusts, 14 directories)
- Total processing time: ~6.2 minutes
2026-02-05 00:20:12 +00:00

6.1 KiB

1CodeService
2100GENERAL SURGERY SERVICE
3101UROLOGY SERVICE
4102TRANSPLANT SURGERY SERVICE
5103BREAST SURGERY SERVICE
6104COLORECTAL SURGERY SERVICE
7105HEPATOBILIARY AND PANCREATIC SURGERY SERVICE
8106UPPER GASTROINTESTINAL SURGERY SERVICE
9107VASCULAR SURGERY SERVICE
10108SPINAL SURGERY SERVICE
11109BARIATRIC SURGERY SERVICE
12110TRAUMA AND ORTHOPAEDIC SERVICE
13111ORTHOPAEDIC SERVICE
14113ENDOCRINE SURGERY SERVICE
15115TRAUMA SURGERY SERVICE
16120EAR NOSE AND THROAT SERVICE
17130OPHTHALMOLOGY SERVICE
18140ORAL SURGERY SERVICE
19141RESTORATIVE DENTISTRY SERVICE
20143ORTHODONTIC SERVICE
21144MAXILLOFACIAL SURGERY SERVICE
22145ORAL AND MAXILLOFACIAL SURGERY SERVICE
23150NEUROSURGICAL SERVICE
24160PLASTIC SURGERY SERVICE
25161BURNS CARE SERVICE
26170CARDIOTHORACIC SURGERY SERVICE
27172CARDIAC SURGERY SERVICE
28173THORACIC SURGERY SERVICE
29174CARDIOTHORACIC TRANSPLANTATION SERVICE
30191PAIN MANAGEMENT SERVICE
31142PAEDIATRIC DENTISTRY SERVICE
32171PAEDIATRIC SURGERY SERVICE
33211PAEDIATRIC UROLOGY SERVICE
34212PAEDIATRIC TRANSPLANTATION SURGERY SERVICE
35213PAEDIATRIC GASTROINTESTINAL SURGERY SERVICE
36214PAEDIATRIC TRAUMA AND ORTHOPAEDIC SERVICE
37215PAEDIATRIC EAR NOSE AND THROAT SERVICE
38216PAEDIATRIC OPHTHALMOLOGY SERVICE
39217PAEDIATRIC ORAL AND MAXILLOFACIAL SURGERY SERVICE
40218PAEDIATRIC NEUROSURGERY SERVICE
41219PAEDIATRIC PLASTIC SURGERY SERVICE
42220PAEDIATRIC BURNS CARE SERVICE
43221PAEDIATRIC CARDIAC SURGERY SERVICE
44222PAEDIATRIC THORACIC SURGERY SERVICE
45223PAEDIATRIC EPILEPSY SERVICE
46230PAEDIATRIC CLINICAL PHARMACOLOGY SERVICE
47240PAEDIATRIC PALLIATIVE MEDICINE SERVICE
48241PAEDIATRIC PAIN MANAGEMENT SERVICE
49242PAEDIATRIC INTENSIVE CARE SERVICE
50250PAEDIATRIC HEPATOLOGY SERVICE
51251PAEDIATRIC GASTROENTEROLOGY SERVICE
52252PAEDIATRIC ENDOCRINOLOGY SERVICE
53253PAEDIATRIC CLINICAL HAEMATOLOGY SERVICE
54254PAEDIATRIC AUDIO VESTIBULAR MEDICINE SERVICE
55255PAEDIATRIC CLINICAL IMMUNOLOGY AND ALLERGY SERVICE
56256PAEDIATRIC INFECTIOUS DISEASES SERVICE
57257PAEDIATRIC DERMATOLOGY SERVICE
58258PAEDIATRIC RESPIRATORY MEDICINE SERVICE
59259PAEDIATRIC NEPHROLOGY SERVICE
60260PAEDIATRIC MEDICAL ONCOLOGY SERVICE
61261PAEDIATRIC INHERITED METABOLIC MEDICINE SERVICE
62262PAEDIATRIC RHEUMATOLOGY SERVICE
63263PAEDIATRIC DIABETES SERVICE
64264PAEDIATRIC CYSTIC FIBROSIS SERVICE
65270PAEDIATRIC EMERGENCY MEDICINE SERVICE
66280PAEDIATRIC INTERVENTIONAL RADIOLOGY SERVICE
67290COMMUNITY PAEDIATRIC SERVICE
68291PAEDIATRIC NEURODISABILITY SERVICE
69321PAEDIATRIC CARDIOLOGY SERVICE
70421PAEDIATRIC NEUROLOGY SERVICE
71180EMERGENCY MEDICINE SERVICE
72190ANAESTHETIC SERVICE
73192INTENSIVE CARE MEDICINE SERVICE
74200AVIATION AND SPACE MEDICINE SERVICE
75300GENERAL INTERNAL MEDICINE SERVICE
76301GASTROENTEROLOGY SERVICE
77302ENDOCRINOLOGY SERVICE
78303CLINICAL HAEMATOLOGY SERVICE
79304CLINICAL PHYSIOLOGY SERVICE
80305CLINICAL PHARMACOLOGY SERVICE
81306HEPATOLOGY SERVICE
82307DIABETES SERVICE
83308BLOOD AND MARROW TRANSPLANTATION SERVICE
84309HAEMOPHILIA SERVICE
85310AUDIO VESTIBULAR MEDICINE SERVICE
86311CLINICAL GENETICS SERVICE
87313CLINICAL IMMUNOLOGY AND ALLERGY SERVICE
88314REHABILITATION MEDICINE SERVICE
89315PALLIATIVE MEDICINE SERVICE
90316CLINICAL IMMUNOLOGY SERVICE
91317ALLERGY SERVICE
92318INTERMEDIATE CARE SERVICE
93319RESPITE CARE SERVICE
94320CARDIOLOGY SERVICE
95322CLINICAL MICROBIOLOGY SERVICE
96323SPINAL INJURIES SERVICE
97324ANTICOAGULANT SERVICE
98325SPORT AND EXERCISE MEDICINE SERVICE
99326ACUTE INTERNAL MEDICINE SERVICE
100327CARDIAC REHABILITATION SERVICE
101328STROKE MEDICINE SERVICE
102329TRANSIENT ISCHAEMIC ATTACK SERVICE
103330DERMATOLOGY SERVICE
104331CONGENITAL HEART DISEASE SERVICE
105333RARE DISEASE SERVICE
106335INHERITED METABOLIC MEDICINE SERVICE
107340RESPIRATORY MEDICINE SERVICE
108341RESPIRATORY PHYSIOLOGY SERVICE
109342PULMONARY REHABILITATION SERVICE
110343ADULT CYSTIC FIBROSIS SERVICE
111344COMPLEX SPECIALISED REHABILITATION SERVICE
112345SPECIALIST REHABILITATION SERVICE
113346LOCAL SPECIALIST REHABILITATION SERVICE
114347SLEEP MEDICINE SERVICE
115348POST-COVID-19 SYNDROME SERVICE
116350INFECTIOUS DISEASES SERVICE
117352TROPICAL MEDICINE SERVICE
118360GENITOURINARY MEDICINE SERVICE
119361RENAL MEDICINE SERVICE
120370MEDICAL ONCOLOGY SERVICE
121371NUCLEAR MEDICINE SERVICE
122400NEUROLOGY SERVICE
123401CLINICAL NEUROPHYSIOLOGY SERVICE
124410RHEUMATOLOGY SERVICE
125420PAEDIATRIC SERVICE
126422NEONATAL CRITICAL CARE SERVICE
127424WELL BABY SERVICE
128430ELDERLY MEDICINE SERVICE
129431ORTHOGERIATRIC MEDICINE SERVICE
130450DENTAL MEDICINE SERVICE
131451SPECIAL CARE DENTISTRY SERVICE
132460MEDICAL OPHTHALMOLOGY SERVICE
133461OPHTHALMIC AND VISION SCIENCE SERVICE
134501OBSTETRICS SERVICE
135502GYNAECOLOGY SERVICE
136503GYNAECOLOGICAL ONCOLOGY SERVICE
137504COMMUNITY SEXUAL AND REPRODUCTIVE HEALTH SERVICE
138505FETAL MEDICINE SERVICE
139510RETIRED BUT RETAINED FOR HISTORICAL PURPOSES
140520RETIRED BUT RETAINED FOR HISTORICAL PURPOSES
141560MIDWIFERY SERVICE
142610RETIRED BUT RETAINED FOR HISTORICAL PURPOSES
143620RETIRED BUT RETAINED FOR HISTORICAL PURPOSES
144656CLINICAL PSYCHOLOGY SERVICE
145700LEARNING DISABILITY SERVICE
146710ADULT MENTAL HEALTH SERVICE
147711CHILD AND ADOLESCENT PSYCHIATRY SERVICE
148712FORENSIC PSYCHIATRY SERVICE
149713MEDICAL PSYCHOTHERAPY SERVICE
150715OLD AGE PSYCHIATRY SERVICE
151720EATING DISORDERS SERVICE
152721ADDICTION SERVICE
153722LIAISON PSYCHIATRY SERVICE
154723PSYCHIATRIC INTENSIVE CARE SERVICE
155724PERINATAL MENTAL HEALTH SERVICE
156725MENTAL HEALTH RECOVERY AND REHABILITATION SERVICE
157726MENTAL HEALTH DUAL DIAGNOSIS SERVICE
158727DEMENTIA ASSESSMENT SERVICE
159730NEUROPSYCHIATRY SERVICE
160800CLINICAL ONCOLOGY SERVICE
161811INTERVENTIONAL RADIOLOGY SERVICE
162812DIAGNOSTIC IMAGING SERVICE
163822CHEMICAL PATHOLOGY SERVICE
164832RETIRED BUT RETAINED FOR HISTORICAL PURPOSES
165834MEDICAL VIROLOGY SERVICE
166650PHYSIOTHERAPY SERVICE
167651OCCUPATIONAL THERAPY SERVICE
168652SPEECH AND LANGUAGE THERAPY SERVICE
169653PODIATRY SERVICE
170654DIETETICS SERVICE
171655ORTHOPTICS SERVICE
172657PROSTHETICS SERVICE
173658ORTHOTICS SERVICE
174659DRAMATHERAPY SERVICE
175660ART THERAPY SERVICE
176661MUSIC THERAPY SERVICE
177662OPTOMETRY SERVICE
178663PODIATRIC SURGERY SERVICE
179670UROLOGICAL PHYSIOLOGY SERVICE
180673VASCULAR PHYSIOLOGY SERVICE
181675CARDIAC PHYSIOLOGY SERVICE
182677GASTROINTESTINAL PHYSIOLOGY SERVICE
183840AUDIOLOGY SERVICE
184920DIABETIC EDUCATION SERVICE
185990RETIRED BUT RETAINED FOR HISTORICAL PURPOSES