feat: complete Task 2.2 - test refresh pipeline with Snowflake data

Tested full refresh pipeline end-to-end with real Snowflake data:
- Fixed trust filter to read Name column from defaultTrusts.csv
- Fixed Decimal type handling in calculate_cost_per_patient_per_annum
- Fixed array handling in convert_to_records for average_administered
- Added required reference CSV files to data/ directory
- Configured Snowflake connection (account, warehouse, user)

Results:
- Snowflake fetch: 656,695 records in ~7s
- Transformations: 519,848 records after UPID/drug/directory
- Pathway nodes: 293 for all_6mo (8 trusts, 14 directories)
- Total processing time: ~6.2 minutes

data/org_codes.csv

@@ -0,0 +1,215 @@
+Name,Code
+MANCHESTER UNIVERSITY NHS FOUNDATION TRUST,R0A
+SOUTH TYNESIDE AND SUNDERLAND NHS FOUNDATION TRUST,R0B
+UNIVERSITY HOSPITALS DORSET NHS FOUNDATION TRUST,R0D
+HEREFORDSHIRE AND WORCESTERSHIRE HEALTH AND CARE NHS TRUST,R1A
+SOLENT NHS TRUST,R1C
+SHROPSHIRE COMMUNITY HEALTH NHS TRUST,R1D
+ISLE OF WIGHT NHS TRUST,R1F
+BARTS HEALTH NHS TRUST,R1H
+LONDON NORTH WEST UNIVERSITY HEALTHCARE NHS TRUST,R1K
+ESSEX PARTNERSHIP UNIVERSITY NHS FOUNDATION TRUST,R1L
+ROYAL SURREY COUNTY HOSPITAL NHS FOUNDATION TRUST,RA2
+YEOVIL DISTRICT HOSPITAL NHS FOUNDATION TRUST,RA4
+UNIVERSITY HOSPITALS BRISTOL AND WESTON NHS FOUNDATION TRUST,RA7
+TORBAY AND SOUTH DEVON NHS FOUNDATION TRUST,RA9
+BRADFORD TEACHING HOSPITALS NHS FOUNDATION TRUST,RAE
+MID AND SOUTH ESSEX NHS FOUNDATION TRUST,RAJ
+ROYAL FREE LONDON NHS FOUNDATION TRUST,RAL
+ROYAL NATIONAL ORTHOPAEDIC HOSPITAL NHS TRUST,RAN
+NORTH MIDDLESEX UNIVERSITY HOSPITAL NHS TRUST,RAP
+THE HILLINGDON HOSPITALS NHS FOUNDATION TRUST,RAS
+NORTH EAST LONDON NHS FOUNDATION TRUST,RAT
+KINGSTON HOSPITAL NHS FOUNDATION TRUST,RAX
+DORSET COUNTY HOSPITAL NHS FOUNDATION TRUST,RBD
+WALSALL HEALTHCARE NHS TRUST,RBK
+WIRRAL UNIVERSITY TEACHING HOSPITAL NHS FOUNDATION TRUST,RBL
+ST HELENS AND KNOWSLEY TEACHING HOSPITALS NHS TRUST,RBN
+LIVERPOOL HEART AND CHEST HOSPITAL NHS FOUNDATION TRUST,RBQ
+ALDER HEY CHILDREN'S NHS FOUNDATION TRUST,RBS
+MID CHESHIRE HOSPITALS NHS FOUNDATION TRUST,RBT
+THE CHRISTIE NHS FOUNDATION TRUST,RBV
+NORTHERN DEVON HEALTHCARE NHS TRUST,RBZ
+BEDFORDSHIRE HOSPITALS NHS FOUNDATION TRUST,RC9
+YORK AND SCARBOROUGH TEACHING HOSPITALS NHS FOUNDATION TRUST,RCB
+HARROGATE AND DISTRICT NHS FOUNDATION TRUST,RCD
+AIREDALE NHS FOUNDATION TRUST,RCF
+SHEFFIELD CHILDREN'S NHS FOUNDATION TRUST,RCU
+THE QUEEN ELIZABETH HOSPITAL,RCX
+ROYAL UNITED HOSPITALS BATH NHS FOUNDATION TRUST,RD1
+MILTON KEYNES UNIVERSITY HOSPITAL NHS FOUNDATION TRUST,RD8
+EAST SUFFOLK AND NORTH ESSEX NHS FOUNDATION TRUST,RDE
+SUSSEX COMMUNITY NHS FOUNDATION TRUST,RDR
+FRIMLEY HEALTH NHS FOUNDATION TRUST,RDU
+DORSET HEALTHCARE UNIVERSITY NHS FOUNDATION TRUST,RDY
+ROYAL CORNWALL HOSPITALS NHS TRUST,REF
+LIVERPOOL UNIVERSITY HOSPITALS NHS FOUNDATION TRUST,REM
+THE CLATTERBRIDGE CANCER CENTRE NHS FOUNDATION TRUST,REN
+LIVERPOOL WOMEN'S NHS FOUNDATION TRUST,REP
+THE WALTON CENTRE NHS FOUNDATION TRUST,RET
+BARKING,RF4
+BARNSLEY HOSPITAL NHS FOUNDATION TRUST,RFF
+THE ROTHERHAM NHS FOUNDATION TRUST,RFR
+CHESTERFIELD ROYAL HOSPITAL NHS FOUNDATION TRUST,RFS
+LEEDS AND YORK PARTNERSHIP NHS FOUNDATION TRUST,RGD
+ROYAL PAPWORTH HOSPITAL NHS FOUNDATION TRUST,RGM
+NORTH WEST ANGLIA NHS FOUNDATION TRUST,RGN
+JAMES PAGET UNIVERSITY HOSPITALS NHS FOUNDATION TRUST,RGP
+WEST SUFFOLK NHS FOUNDATION TRUST,RGR
+CAMBRIDGE UNIVERSITY HOSPITALS NHS FOUNDATION TRUST,RGT
+SOMERSET NHS FOUNDATION TRUST,RH5
+ROYAL DEVON UNIVERSITY HEALTHCARE NHS FOUNDATION TRUST,RH8
+NOTTINGHAMSHIRE HEALTHCARE NHS FOUNDATION TRUST,RHA
+UNIVERSITY HOSPITAL SOUTHAMPTON NHS FOUNDATION TRUST,RHM
+SHEFFIELD TEACHING HOSPITALS NHS FOUNDATION TRUST,RHQ
+PORTSMOUTH HOSPITALS UNIVERSITY NATIONAL HEALTH SERVICE TRUST,RHU
+ROYAL BERKSHIRE NHS FOUNDATION TRUST,RHW
+GUY'S AND ST THOMAS' NHS FOUNDATION TRUST,RJ1
+LEWISHAM AND GREENWICH NHS TRUST,RJ2
+CROYDON HEALTH SERVICES NHS TRUST,RJ6
+ST GEORGE'S UNIVERSITY HOSPITALS NHS FOUNDATION TRUST,RJ7
+CORNWALL PARTNERSHIP NHS FOUNDATION TRUST,RJ8
+SOUTH WARWICKSHIRE NHS FOUNDATION TRUST,RJC
+UNIVERSITY HOSPITALS OF NORTH MIDLANDS NHS TRUST,RJE
+NORTHERN LINCOLNSHIRE AND GOOLE NHS FOUNDATION TRUST,RJL
+EAST CHESHIRE NHS TRUST,RJN
+COUNTESS OF CHESTER HOSPITAL NHS FOUNDATION TRUST,RJR
+KING'S COLLEGE HOSPITAL NHS FOUNDATION TRUST,RJZ
+SHERWOOD FOREST HOSPITALS NHS FOUNDATION TRUST,RK5
+UNIVERSITY HOSPITALS PLYMOUTH NHS TRUST,RK9
+UNIVERSITY HOSPITALS COVENTRY AND WARWICKSHIRE NHS TRUST,RKB
+WHITTINGTON HEALTH NHS TRUST,RKE
+WEST LONDON NHS TRUST,RKL
+THE ROBERT JONES AND AGNES HUNT ORTHOPAEDIC HOSPITAL NHS FOUNDATION TRUST,RL1
+THE ROYAL WOLVERHAMPTON NHS TRUST,RL4
+WYE VALLEY NHS TRUST,RLQ
+GEORGE ELIOT HOSPITAL NHS TRUST,RLT
+NORTH STAFFORDSHIRE COMBINED HEALTHCARE NHS TRUST,RLY
+NORFOLK AND NORWICH UNIVERSITY HOSPITALS NHS FOUNDATION TRUST,RM1
+NORTHERN CARE ALLIANCE NHS FOUNDATION TRUST,RM3
+BOLTON NHS FOUNDATION TRUST,RMC
+TAMESIDE AND GLOSSOP INTEGRATED CARE NHS FOUNDATION TRUST,RMP
+NORFOLK AND SUFFOLK NHS FOUNDATION TRUST,RMY
+GREAT WESTERN HOSPITALS NHS FOUNDATION TRUST,RN3
+HAMPSHIRE HOSPITALS NHS FOUNDATION TRUST,RN5
+DARTFORD AND GRAVESHAM NHS TRUST,RN7
+THE DUDLEY GROUP NHS FOUNDATION TRUST,RNA
+TAVISTOCK AND PORTMAN NHS FOUNDATION TRUST,RNK
+NORTH CUMBRIA INTEGRATED CARE NHS FOUNDATION TRUST,RNN
+KETTERING GENERAL HOSPITAL NHS FOUNDATION TRUST,RNQ
+NORTHAMPTON GENERAL HOSPITAL NHS TRUST,RNS
+OXFORD HEALTH NHS FOUNDATION TRUST,RNU
+SALISBURY NHS FOUNDATION TRUST,RNZ
+NORTHAMPTONSHIRE HEALTHCARE NHS FOUNDATION TRUST,RP1
+GREAT ORMOND STREET HOSPITAL FOR CHILDREN NHS FOUNDATION TRUST,RP4
+DONCASTER AND BASSETLAW TEACHING HOSPITALS NHS FOUNDATION TRUST,RP5
+MOORFIELDS EYE HOSPITAL NHS FOUNDATION TRUST,RP6
+LINCOLNSHIRE PARTNERSHIP NHS FOUNDATION TRUST,RP7
+MEDWAY NHS FOUNDATION TRUST,RPA
+QUEEN VICTORIA HOSPITAL NHS FOUNDATION TRUST,RPC
+OXLEAS NHS FOUNDATION TRUST,RPG
+THE ROYAL MARSDEN NHS FOUNDATION TRUST,RPY
+BIRMINGHAM WOMEN'S AND CHILDREN'S NHS FOUNDATION TRUST,RQ3
+VELINDRE NHS TRUST,RQF
+CHELSEA AND WESTMINSTER HOSPITAL NHS FOUNDATION TRUST,RQM
+THE PRINCESS ALEXANDRA HOSPITAL NHS TRUST,RQW
+HOMERTON HEALTHCARE NHS FOUNDATION TRUST,RQX
+SOUTH WEST LONDON AND ST GEORGE'S MENTAL HEALTH NHS TRUST,RQY
+GATESHEAD HEALTH NHS FOUNDATION TRUST,RR7
+LEEDS TEACHING HOSPITALS NHS TRUST,RR8
+MIDLANDS PARTNERSHIP NHS FOUNDATION TRUST,RRE
+WRIGHTINGTON,RRF
+THE ROYAL ORTHOPAEDIC HOSPITAL NHS FOUNDATION TRUST,RRJ
+UNIVERSITY HOSPITALS BIRMINGHAM NHS FOUNDATION TRUST,RRK
+BARNET,RRP
+LONDON AMBULANCE SERVICE NHS TRUST,RRU
+UNIVERSITY COLLEGE LONDON HOSPITALS NHS FOUNDATION TRUST,RRV
+CAMBRIDGESHIRE AND PETERBOROUGH NHS FOUNDATION TRUST,RT1
+PENNINE CARE NHS FOUNDATION TRUST,RT2
+WELSH AMBULANCE SERVICES NHS TRUST,RT4
+LEICESTERSHIRE PARTNERSHIP NHS TRUST,RT5
+THE NEWCASTLE UPON TYNE HOSPITALS NHS FOUNDATION TRUST,RTD
+GLOUCESTERSHIRE HOSPITALS NHS FOUNDATION TRUST,RTE
+NORTHUMBRIA HEALTHCARE NHS FOUNDATION TRUST,RTF
+UNIVERSITY HOSPITALS OF DERBY AND BURTON NHS FOUNDATION TRUST,RTG
+OXFORD UNIVERSITY HOSPITALS NHS FOUNDATION TRUST,RTH
+ASHFORD AND ST PETER'S HOSPITALS NHS FOUNDATION TRUST,RTK
+SURREY AND SUSSEX HEALTHCARE NHS TRUST,RTP
+GLOUCESTERSHIRE HEALTH AND CARE NHS FOUNDATION TRUST,RTQ
+SOUTH TEES HOSPITALS NHS FOUNDATION TRUST,RTR
+NORTH WEST BOROUGHS HEALTHCARE NHS FOUNDATION TRUST,RTV
+UNIVERSITY HOSPITALS OF MORECAMBE BAY NHS FOUNDATION TRUST,RTX
+CENTRAL AND NORTH WEST LONDON NHS FOUNDATION TRUST,RV3
+SOUTH LONDON AND MAUDSLEY NHS FOUNDATION TRUST,RV5
+HUMBER TEACHING NHS FOUNDATION TRUST,RV9
+NORTH BRISTOL NHS TRUST,RVJ
+AVON AND WILTSHIRE MENTAL HEALTH PARTNERSHIP NHS TRUST,RVN
+EPSOM AND ST HELIER UNIVERSITY HOSPITALS NHS TRUST,RVR
+EAST KENT HOSPITALS UNIVERSITY NHS FOUNDATION TRUST,RVV
+NORTH TEES AND HARTLEPOOL NHS FOUNDATION TRUST,RVW
+SOUTHPORT AND ORMSKIRK HOSPITAL NHS TRUST,RVY
+SOUTHERN HEALTH NHS FOUNDATION TRUST,RW1
+MERSEY CARE NHS FOUNDATION TRUST,RW4
+LANCASHIRE & SOUTH CUMBRIA NHS FOUNDATION TRUST,RW5
+PENNINE ACUTE HOSPITALS NHS TRUST,RW6
+HULL UNIVERSITY TEACHING HOSPITALS NHS TRUST,RWA
+UNITED LINCOLNSHIRE HOSPITALS NHS TRUST,RWD
+UNIVERSITY HOSPITALS OF LEICESTER NHS TRUST,RWE
+MAIDSTONE AND TUNBRIDGE WELLS NHS TRUST,RWF
+WEST HERTFORDSHIRE TEACHING HOSPITALS NHS TRUST,RWG
+EAST AND NORTH HERTFORDSHIRE NHS TRUST,RWH
+STOCKPORT NHS FOUNDATION TRUST,RWJ
+EAST LONDON NHS FOUNDATION TRUST,RWK
+WORCESTERSHIRE ACUTE HOSPITALS NHS TRUST,RWP
+HERTFORDSHIRE PARTNERSHIP UNIVERSITY NHS FOUNDATION TRUST,RWR
+DEVON PARTNERSHIP NHS TRUST,RWV
+WARRINGTON AND HALTON TEACHING HOSPITALS NHS FOUNDATION TRUST,RWW
+BERKSHIRE HEALTHCARE NHS FOUNDATION TRUST,RWX
+CALDERDALE AND HUDDERSFIELD NHS FOUNDATION TRUST,RWY
+NOTTINGHAM UNIVERSITY HOSPITALS NHS TRUST,RX1
+SUSSEX PARTNERSHIP NHS FOUNDATION TRUST,RX2
+TEES,RX3
+CUMBRIA,RX4
+NORTH EAST AMBULANCE SERVICE NHS FOUNDATION TRUST,RX6
+NORTH WEST AMBULANCE SERVICE NHS TRUST,RX7
+YORKSHIRE AMBULANCE SERVICE NHS TRUST,RX8
+EAST MIDLANDS AMBULANCE SERVICE NHS TRUST,RX9
+CHESHIRE AND WIRRAL PARTNERSHIP NHS FOUNDATION TRUST,RXA
+EAST SUSSEX HEALTHCARE NHS TRUST,RXC
+ROTHERHAM DONCASTER AND SOUTH HUMBER NHS FOUNDATION TRUST,RXE
+MID YORKSHIRE HOSPITALS NHS TRUST,RXF
+SOUTH WEST YORKSHIRE PARTNERSHIP NHS FOUNDATION TRUST,RXG
+SANDWELL AND WEST BIRMINGHAM HOSPITALS NHS TRUST,RXK
+BLACKPOOL TEACHING HOSPITALS NHS FOUNDATION TRUST,RXL
+DERBYSHIRE HEALTHCARE NHS FOUNDATION TRUST,RXM
+LANCASHIRE TEACHING HOSPITALS NHS FOUNDATION TRUST,RXN
+COUNTY DURHAM AND DARLINGTON NHS FOUNDATION TRUST,RXP
+BUCKINGHAMSHIRE HEALTHCARE NHS TRUST,RXQ
+EAST LANCASHIRE HOSPITALS NHS TRUST,RXR
+BIRMINGHAM AND SOLIHULL MENTAL HEALTH NHS FOUNDATION TRUST,RXT
+GREATER MANCHESTER MENTAL HEALTH NHS FOUNDATION TRUST,RXV
+THE SHREWSBURY AND TELFORD HOSPITAL NHS TRUST,RXW
+SURREY AND BORDERS PARTNERSHIP NHS FOUNDATION TRUST,RXX
+KENT AND MEDWAY NHS AND SOCIAL CARE PARTNERSHIP TRUST,RXY
+BRIDGEWATER COMMUNITY HEALTHCARE NHS FOUNDATION TRUST,RY2
+NORFOLK COMMUNITY HEALTH AND CARE NHS TRUST,RY3
+HERTFORDSHIRE COMMUNITY NHS TRUST,RY4
+LINCOLNSHIRE COMMUNITY HEALTH SERVICES NHS TRUST,RY5
+LEEDS COMMUNITY HEALTHCARE NHS TRUST,RY6
+WIRRAL COMMUNITY HEALTH AND CARE NHS FOUNDATION TRUST,RY7
+DERBYSHIRE COMMUNITY HEALTH SERVICES NHS FOUNDATION TRUST,RY8
+HOUNSLOW AND RICHMOND COMMUNITY HEALTHCARE NHS TRUST,RY9
+WEST MIDLANDS AMBULANCE SERVICE UNIVERSITY NHS FOUNDATION TRUST,RYA
+EAST OF ENGLAND AMBULANCE SERVICE NHS TRUST,RYC
+SOUTH EAST COAST AMBULANCE SERVICE NHS FOUNDATION TRUST,RYD
+SOUTH CENTRAL AMBULANCE SERVICE NHS FOUNDATION TRUST,RYE
+SOUTH WESTERN AMBULANCE SERVICE NHS FOUNDATION TRUST,RYF
+COVENTRY AND WARWICKSHIRE PARTNERSHIP NHS TRUST,RYG
+IMPERIAL COLLEGE HEALTHCARE NHS TRUST,RYJ
+DUDLEY INTEGRATED HEALTH AND CARE NHS TRUST,RYK
+UNIVERSITY HOSPITALS SUSSEX NHS FOUNDATION TRUST,RYR
+PUBLIC HEALTH WALES NHS TRUST,RYT
+CAMBRIDGESHIRE COMMUNITY SERVICES NHS TRUST,RYV
+BIRMINGHAM COMMUNITY HEALTHCARE NHS FOUNDATION TRUST,RYW
+CENTRAL LONDON COMMUNITY HEALTHCARE NHS TRUST,RYX
+KENT COMMUNITY HEALTH NHS FOUNDATION TRUST,RYY