Files
Andrew Charlwood adc1dbfc58 feat: complete Task 2.2 - test refresh pipeline with Snowflake data
Tested full refresh pipeline end-to-end with real Snowflake data:
- Fixed trust filter to read Name column from defaultTrusts.csv
- Fixed Decimal type handling in calculate_cost_per_patient_per_annum
- Fixed array handling in convert_to_records for average_administered
- Added required reference CSV files to data/ directory
- Configured Snowflake connection (account, warehouse, user)

Results:
- Snowflake fetch: 656,695 records in ~7s
- Transformations: 519,848 records after UPID/drug/directory
- Pathway nodes: 293 for all_6mo (8 trusts, 14 directories)
- Total processing time: ~6.2 minutes
2026-02-05 00:20:12 +00:00

10 KiB

1NameCode
2MANCHESTER UNIVERSITY NHS FOUNDATION TRUSTR0A
3SOUTH TYNESIDE AND SUNDERLAND NHS FOUNDATION TRUSTR0B
4UNIVERSITY HOSPITALS DORSET NHS FOUNDATION TRUSTR0D
5HEREFORDSHIRE AND WORCESTERSHIRE HEALTH AND CARE NHS TRUSTR1A
6SOLENT NHS TRUSTR1C
7SHROPSHIRE COMMUNITY HEALTH NHS TRUSTR1D
8ISLE OF WIGHT NHS TRUSTR1F
9BARTS HEALTH NHS TRUSTR1H
10LONDON NORTH WEST UNIVERSITY HEALTHCARE NHS TRUSTR1K
11ESSEX PARTNERSHIP UNIVERSITY NHS FOUNDATION TRUSTR1L
12ROYAL SURREY COUNTY HOSPITAL NHS FOUNDATION TRUSTRA2
13YEOVIL DISTRICT HOSPITAL NHS FOUNDATION TRUSTRA4
14UNIVERSITY HOSPITALS BRISTOL AND WESTON NHS FOUNDATION TRUSTRA7
15TORBAY AND SOUTH DEVON NHS FOUNDATION TRUSTRA9
16BRADFORD TEACHING HOSPITALS NHS FOUNDATION TRUSTRAE
17MID AND SOUTH ESSEX NHS FOUNDATION TRUSTRAJ
18ROYAL FREE LONDON NHS FOUNDATION TRUSTRAL
19ROYAL NATIONAL ORTHOPAEDIC HOSPITAL NHS TRUSTRAN
20NORTH MIDDLESEX UNIVERSITY HOSPITAL NHS TRUSTRAP
21THE HILLINGDON HOSPITALS NHS FOUNDATION TRUSTRAS
22NORTH EAST LONDON NHS FOUNDATION TRUSTRAT
23KINGSTON HOSPITAL NHS FOUNDATION TRUSTRAX
24DORSET COUNTY HOSPITAL NHS FOUNDATION TRUSTRBD
25WALSALL HEALTHCARE NHS TRUSTRBK
26WIRRAL UNIVERSITY TEACHING HOSPITAL NHS FOUNDATION TRUSTRBL
27ST HELENS AND KNOWSLEY TEACHING HOSPITALS NHS TRUSTRBN
28LIVERPOOL HEART AND CHEST HOSPITAL NHS FOUNDATION TRUSTRBQ
29ALDER HEY CHILDREN'S NHS FOUNDATION TRUSTRBS
30MID CHESHIRE HOSPITALS NHS FOUNDATION TRUSTRBT
31THE CHRISTIE NHS FOUNDATION TRUSTRBV
32NORTHERN DEVON HEALTHCARE NHS TRUSTRBZ
33BEDFORDSHIRE HOSPITALS NHS FOUNDATION TRUSTRC9
34YORK AND SCARBOROUGH TEACHING HOSPITALS NHS FOUNDATION TRUSTRCB
35HARROGATE AND DISTRICT NHS FOUNDATION TRUSTRCD
36AIREDALE NHS FOUNDATION TRUSTRCF
37SHEFFIELD CHILDREN'S NHS FOUNDATION TRUSTRCU
38THE QUEEN ELIZABETH HOSPITALRCX
39ROYAL UNITED HOSPITALS BATH NHS FOUNDATION TRUSTRD1
40MILTON KEYNES UNIVERSITY HOSPITAL NHS FOUNDATION TRUSTRD8
41EAST SUFFOLK AND NORTH ESSEX NHS FOUNDATION TRUSTRDE
42SUSSEX COMMUNITY NHS FOUNDATION TRUSTRDR
43FRIMLEY HEALTH NHS FOUNDATION TRUSTRDU
44DORSET HEALTHCARE UNIVERSITY NHS FOUNDATION TRUSTRDY
45ROYAL CORNWALL HOSPITALS NHS TRUSTREF
46LIVERPOOL UNIVERSITY HOSPITALS NHS FOUNDATION TRUSTREM
47THE CLATTERBRIDGE CANCER CENTRE NHS FOUNDATION TRUSTREN
48LIVERPOOL WOMEN'S NHS FOUNDATION TRUSTREP
49THE WALTON CENTRE NHS FOUNDATION TRUSTRET
50BARKINGRF4
51BARNSLEY HOSPITAL NHS FOUNDATION TRUSTRFF
52THE ROTHERHAM NHS FOUNDATION TRUSTRFR
53CHESTERFIELD ROYAL HOSPITAL NHS FOUNDATION TRUSTRFS
54LEEDS AND YORK PARTNERSHIP NHS FOUNDATION TRUSTRGD
55ROYAL PAPWORTH HOSPITAL NHS FOUNDATION TRUSTRGM
56NORTH WEST ANGLIA NHS FOUNDATION TRUSTRGN
57JAMES PAGET UNIVERSITY HOSPITALS NHS FOUNDATION TRUSTRGP
58WEST SUFFOLK NHS FOUNDATION TRUSTRGR
59CAMBRIDGE UNIVERSITY HOSPITALS NHS FOUNDATION TRUSTRGT
60SOMERSET NHS FOUNDATION TRUSTRH5
61ROYAL DEVON UNIVERSITY HEALTHCARE NHS FOUNDATION TRUSTRH8
62NOTTINGHAMSHIRE HEALTHCARE NHS FOUNDATION TRUSTRHA
63UNIVERSITY HOSPITAL SOUTHAMPTON NHS FOUNDATION TRUSTRHM
64SHEFFIELD TEACHING HOSPITALS NHS FOUNDATION TRUSTRHQ
65PORTSMOUTH HOSPITALS UNIVERSITY NATIONAL HEALTH SERVICE TRUSTRHU
66ROYAL BERKSHIRE NHS FOUNDATION TRUSTRHW
67GUY'S AND ST THOMAS' NHS FOUNDATION TRUSTRJ1
68LEWISHAM AND GREENWICH NHS TRUSTRJ2
69CROYDON HEALTH SERVICES NHS TRUSTRJ6
70ST GEORGE'S UNIVERSITY HOSPITALS NHS FOUNDATION TRUSTRJ7
71CORNWALL PARTNERSHIP NHS FOUNDATION TRUSTRJ8
72SOUTH WARWICKSHIRE NHS FOUNDATION TRUSTRJC
73UNIVERSITY HOSPITALS OF NORTH MIDLANDS NHS TRUSTRJE
74NORTHERN LINCOLNSHIRE AND GOOLE NHS FOUNDATION TRUSTRJL
75EAST CHESHIRE NHS TRUSTRJN
76COUNTESS OF CHESTER HOSPITAL NHS FOUNDATION TRUSTRJR
77KING'S COLLEGE HOSPITAL NHS FOUNDATION TRUSTRJZ
78SHERWOOD FOREST HOSPITALS NHS FOUNDATION TRUSTRK5
79UNIVERSITY HOSPITALS PLYMOUTH NHS TRUSTRK9
80UNIVERSITY HOSPITALS COVENTRY AND WARWICKSHIRE NHS TRUSTRKB
81WHITTINGTON HEALTH NHS TRUSTRKE
82WEST LONDON NHS TRUSTRKL
83THE ROBERT JONES AND AGNES HUNT ORTHOPAEDIC HOSPITAL NHS FOUNDATION TRUSTRL1
84THE ROYAL WOLVERHAMPTON NHS TRUSTRL4
85WYE VALLEY NHS TRUSTRLQ
86GEORGE ELIOT HOSPITAL NHS TRUSTRLT
87NORTH STAFFORDSHIRE COMBINED HEALTHCARE NHS TRUSTRLY
88NORFOLK AND NORWICH UNIVERSITY HOSPITALS NHS FOUNDATION TRUSTRM1
89NORTHERN CARE ALLIANCE NHS FOUNDATION TRUSTRM3
90BOLTON NHS FOUNDATION TRUSTRMC
91TAMESIDE AND GLOSSOP INTEGRATED CARE NHS FOUNDATION TRUSTRMP
92NORFOLK AND SUFFOLK NHS FOUNDATION TRUSTRMY
93GREAT WESTERN HOSPITALS NHS FOUNDATION TRUSTRN3
94HAMPSHIRE HOSPITALS NHS FOUNDATION TRUSTRN5
95DARTFORD AND GRAVESHAM NHS TRUSTRN7
96THE DUDLEY GROUP NHS FOUNDATION TRUSTRNA
97TAVISTOCK AND PORTMAN NHS FOUNDATION TRUSTRNK
98NORTH CUMBRIA INTEGRATED CARE NHS FOUNDATION TRUSTRNN
99KETTERING GENERAL HOSPITAL NHS FOUNDATION TRUSTRNQ
100NORTHAMPTON GENERAL HOSPITAL NHS TRUSTRNS
101OXFORD HEALTH NHS FOUNDATION TRUSTRNU
102SALISBURY NHS FOUNDATION TRUSTRNZ
103NORTHAMPTONSHIRE HEALTHCARE NHS FOUNDATION TRUSTRP1
104GREAT ORMOND STREET HOSPITAL FOR CHILDREN NHS FOUNDATION TRUSTRP4
105DONCASTER AND BASSETLAW TEACHING HOSPITALS NHS FOUNDATION TRUSTRP5
106MOORFIELDS EYE HOSPITAL NHS FOUNDATION TRUSTRP6
107LINCOLNSHIRE PARTNERSHIP NHS FOUNDATION TRUSTRP7
108MEDWAY NHS FOUNDATION TRUSTRPA
109QUEEN VICTORIA HOSPITAL NHS FOUNDATION TRUSTRPC
110OXLEAS NHS FOUNDATION TRUSTRPG
111THE ROYAL MARSDEN NHS FOUNDATION TRUSTRPY
112BIRMINGHAM WOMEN'S AND CHILDREN'S NHS FOUNDATION TRUSTRQ3
113VELINDRE NHS TRUSTRQF
114CHELSEA AND WESTMINSTER HOSPITAL NHS FOUNDATION TRUSTRQM
115THE PRINCESS ALEXANDRA HOSPITAL NHS TRUSTRQW
116HOMERTON HEALTHCARE NHS FOUNDATION TRUSTRQX
117SOUTH WEST LONDON AND ST GEORGE'S MENTAL HEALTH NHS TRUSTRQY
118GATESHEAD HEALTH NHS FOUNDATION TRUSTRR7
119LEEDS TEACHING HOSPITALS NHS TRUSTRR8
120MIDLANDS PARTNERSHIP NHS FOUNDATION TRUSTRRE
121WRIGHTINGTONRRF
122THE ROYAL ORTHOPAEDIC HOSPITAL NHS FOUNDATION TRUSTRRJ
123UNIVERSITY HOSPITALS BIRMINGHAM NHS FOUNDATION TRUSTRRK
124BARNETRRP
125LONDON AMBULANCE SERVICE NHS TRUSTRRU
126UNIVERSITY COLLEGE LONDON HOSPITALS NHS FOUNDATION TRUSTRRV
127CAMBRIDGESHIRE AND PETERBOROUGH NHS FOUNDATION TRUSTRT1
128PENNINE CARE NHS FOUNDATION TRUSTRT2
129WELSH AMBULANCE SERVICES NHS TRUSTRT4
130LEICESTERSHIRE PARTNERSHIP NHS TRUSTRT5
131THE NEWCASTLE UPON TYNE HOSPITALS NHS FOUNDATION TRUSTRTD
132GLOUCESTERSHIRE HOSPITALS NHS FOUNDATION TRUSTRTE
133NORTHUMBRIA HEALTHCARE NHS FOUNDATION TRUSTRTF
134UNIVERSITY HOSPITALS OF DERBY AND BURTON NHS FOUNDATION TRUSTRTG
135OXFORD UNIVERSITY HOSPITALS NHS FOUNDATION TRUSTRTH
136ASHFORD AND ST PETER'S HOSPITALS NHS FOUNDATION TRUSTRTK
137SURREY AND SUSSEX HEALTHCARE NHS TRUSTRTP
138GLOUCESTERSHIRE HEALTH AND CARE NHS FOUNDATION TRUSTRTQ
139SOUTH TEES HOSPITALS NHS FOUNDATION TRUSTRTR
140NORTH WEST BOROUGHS HEALTHCARE NHS FOUNDATION TRUSTRTV
141UNIVERSITY HOSPITALS OF MORECAMBE BAY NHS FOUNDATION TRUSTRTX
142CENTRAL AND NORTH WEST LONDON NHS FOUNDATION TRUSTRV3
143SOUTH LONDON AND MAUDSLEY NHS FOUNDATION TRUSTRV5
144HUMBER TEACHING NHS FOUNDATION TRUSTRV9
145NORTH BRISTOL NHS TRUSTRVJ
146AVON AND WILTSHIRE MENTAL HEALTH PARTNERSHIP NHS TRUSTRVN
147EPSOM AND ST HELIER UNIVERSITY HOSPITALS NHS TRUSTRVR
148EAST KENT HOSPITALS UNIVERSITY NHS FOUNDATION TRUSTRVV
149NORTH TEES AND HARTLEPOOL NHS FOUNDATION TRUSTRVW
150SOUTHPORT AND ORMSKIRK HOSPITAL NHS TRUSTRVY
151SOUTHERN HEALTH NHS FOUNDATION TRUSTRW1
152MERSEY CARE NHS FOUNDATION TRUSTRW4
153LANCASHIRE & SOUTH CUMBRIA NHS FOUNDATION TRUSTRW5
154PENNINE ACUTE HOSPITALS NHS TRUSTRW6
155HULL UNIVERSITY TEACHING HOSPITALS NHS TRUSTRWA
156UNITED LINCOLNSHIRE HOSPITALS NHS TRUSTRWD
157UNIVERSITY HOSPITALS OF LEICESTER NHS TRUSTRWE
158MAIDSTONE AND TUNBRIDGE WELLS NHS TRUSTRWF
159WEST HERTFORDSHIRE TEACHING HOSPITALS NHS TRUSTRWG
160EAST AND NORTH HERTFORDSHIRE NHS TRUSTRWH
161STOCKPORT NHS FOUNDATION TRUSTRWJ
162EAST LONDON NHS FOUNDATION TRUSTRWK
163WORCESTERSHIRE ACUTE HOSPITALS NHS TRUSTRWP
164HERTFORDSHIRE PARTNERSHIP UNIVERSITY NHS FOUNDATION TRUSTRWR
165DEVON PARTNERSHIP NHS TRUSTRWV
166WARRINGTON AND HALTON TEACHING HOSPITALS NHS FOUNDATION TRUSTRWW
167BERKSHIRE HEALTHCARE NHS FOUNDATION TRUSTRWX
168CALDERDALE AND HUDDERSFIELD NHS FOUNDATION TRUSTRWY
169NOTTINGHAM UNIVERSITY HOSPITALS NHS TRUSTRX1
170SUSSEX PARTNERSHIP NHS FOUNDATION TRUSTRX2
171TEESRX3
172CUMBRIARX4
173NORTH EAST AMBULANCE SERVICE NHS FOUNDATION TRUSTRX6
174NORTH WEST AMBULANCE SERVICE NHS TRUSTRX7
175YORKSHIRE AMBULANCE SERVICE NHS TRUSTRX8
176EAST MIDLANDS AMBULANCE SERVICE NHS TRUSTRX9
177CHESHIRE AND WIRRAL PARTNERSHIP NHS FOUNDATION TRUSTRXA
178EAST SUSSEX HEALTHCARE NHS TRUSTRXC
179ROTHERHAM DONCASTER AND SOUTH HUMBER NHS FOUNDATION TRUSTRXE
180MID YORKSHIRE HOSPITALS NHS TRUSTRXF
181SOUTH WEST YORKSHIRE PARTNERSHIP NHS FOUNDATION TRUSTRXG
182SANDWELL AND WEST BIRMINGHAM HOSPITALS NHS TRUSTRXK
183BLACKPOOL TEACHING HOSPITALS NHS FOUNDATION TRUSTRXL
184DERBYSHIRE HEALTHCARE NHS FOUNDATION TRUSTRXM
185LANCASHIRE TEACHING HOSPITALS NHS FOUNDATION TRUSTRXN
186COUNTY DURHAM AND DARLINGTON NHS FOUNDATION TRUSTRXP
187BUCKINGHAMSHIRE HEALTHCARE NHS TRUSTRXQ
188EAST LANCASHIRE HOSPITALS NHS TRUSTRXR
189BIRMINGHAM AND SOLIHULL MENTAL HEALTH NHS FOUNDATION TRUSTRXT
190GREATER MANCHESTER MENTAL HEALTH NHS FOUNDATION TRUSTRXV
191THE SHREWSBURY AND TELFORD HOSPITAL NHS TRUSTRXW
192SURREY AND BORDERS PARTNERSHIP NHS FOUNDATION TRUSTRXX
193KENT AND MEDWAY NHS AND SOCIAL CARE PARTNERSHIP TRUSTRXY
194BRIDGEWATER COMMUNITY HEALTHCARE NHS FOUNDATION TRUSTRY2
195NORFOLK COMMUNITY HEALTH AND CARE NHS TRUSTRY3
196HERTFORDSHIRE COMMUNITY NHS TRUSTRY4
197LINCOLNSHIRE COMMUNITY HEALTH SERVICES NHS TRUSTRY5
198LEEDS COMMUNITY HEALTHCARE NHS TRUSTRY6
199WIRRAL COMMUNITY HEALTH AND CARE NHS FOUNDATION TRUSTRY7
200DERBYSHIRE COMMUNITY HEALTH SERVICES NHS FOUNDATION TRUSTRY8
201HOUNSLOW AND RICHMOND COMMUNITY HEALTHCARE NHS TRUSTRY9
202WEST MIDLANDS AMBULANCE SERVICE UNIVERSITY NHS FOUNDATION TRUSTRYA
203EAST OF ENGLAND AMBULANCE SERVICE NHS TRUSTRYC
204SOUTH EAST COAST AMBULANCE SERVICE NHS FOUNDATION TRUSTRYD
205SOUTH CENTRAL AMBULANCE SERVICE NHS FOUNDATION TRUSTRYE
206SOUTH WESTERN AMBULANCE SERVICE NHS FOUNDATION TRUSTRYF
207COVENTRY AND WARWICKSHIRE PARTNERSHIP NHS TRUSTRYG
208IMPERIAL COLLEGE HEALTHCARE NHS TRUSTRYJ
209DUDLEY INTEGRATED HEALTH AND CARE NHS TRUSTRYK
210UNIVERSITY HOSPITALS SUSSEX NHS FOUNDATION TRUSTRYR
211PUBLIC HEALTH WALES NHS TRUSTRYT
212CAMBRIDGESHIRE COMMUNITY SERVICES NHS TRUSTRYV
213BIRMINGHAM COMMUNITY HEALTHCARE NHS FOUNDATION TRUSTRYW
214CENTRAL LONDON COMMUNITY HEALTHCARE NHS TRUSTRYX
215KENT COMMUNITY HEALTH NHS FOUNDATION TRUSTRYY