keybindings-help debug frontend-design sc:analyze sc:build sc:cleanup sc:design sc:document sc:estimate sc:explain sc:git sc:implement sc:improve sc:index sc:load sc:spawn sc:task sc:test sc:troubleshoot sc:workflow compact context cost init pr-comments release-notes review security-review insights

Bash general-purpose statusline-setup Explore Plan code-reviewer documentation-specialist performance-optimizer medicines-intelligence-analyst project-analyst team-configurator tech-lead-orchestrator data-scientist postgres-expert sql-pro django-api-developer django-backend-expert django-orm-expert laravel-backend-expert laravel-eloquent-expert actix-expert android-expert angular-expert angularjs-expert ansible-expert aspnet-core-expert astro-expert auth0-expert ava-expert braintree-expert bullmq-expert bun-expert c-expert cassandra-expert celery-expert circleci-expert clojure-expert cockroachdb-expert cpp-expert csharp-expert css-expert cypress-expert dart-expert deno-expert django-expert docker-expert dynamodb-expert elasticsearch-expert electron-expert elixir-expert elk-expert erlang-expert expo-expert express-expert fastapi-expert fastify-expert fiber-expert flask-expert flutter-expert flyway-expert gin-expert github-actions-expert gitlab-ci-expert go-expert grafana-expert graphql-expert grpc-expert haskell-expert html-expert ios-expert jasmine-expert java-expert javascript-expert jenkins-expert jest-expert jquery-expert jwt-expert kafka-expert keycloak-expert knex-expert kotlin-expert kubernetes-expert langchain-expert laravel-expert liquibase-expert loki-expert lua-expert mariadb-expert mocha-expert mongodb-expert mongoose-expert mqtt-expert mssql-expert mysql-expert nats-expert neo4j-expert nestjs-expert nextjs-expert nodejs-expert numpy-expert oauth-oidc-expert ocaml-expert openai-api-expert openapi-expert opensearch-expert opentelemetry-expert owasp-top10-expert pandas-expert perl-expert phoenix-expert php-expert playwright-expert prisma-expert prometheus-expert pulumi-expert puppeteer-expert python-expert pytorch-expert rabbitmq-expert rails-expert react-expert react-native-expert redis-expert remix-expert rest-expert rollup-expert ruby-expert rust-expert scala-expert scikit-learn-expert selenium-expert sequelize-expert sidekiq-expert sns-expert solidjs-expert spring-boot-expert sql-expert sqlite-expert sqs-expert stripe-expert svelte-expert swift-expert swiftui-expert tailwind-expert tauri-expert tensorflow-expert terraform-expert testcafe-expert trpc-expert typeorm-expert typescript-expert vector-db-expert vitest-expert vue-expert webpack-expert websocket-expert rails-activerecord-expert rails-api-developer react-component-architect react-nextjs-expert vue-component-architect vue-nuxt-expert api-architect backend-developer frontend-developer tailwind-frontend-expert


I'll start by reading the required files in order to understand the current state of the project.


     1ÔåÆ# Progress Log - Drug-Aware Indication Matching
     2ÔåÆ
     3ÔåÆ## Project Context
     4ÔåÆ
     5ÔåÆThis project extends the indication-based pathway charts (Phase 1-5 complete) with drug-aware matching.
     6ÔåÆ
     7ÔåÆ**Previous state**: Patients get ONE indication based on their most recent GP diagnosis match (SNOMED cluster codes). This ignores which drugs the patient is taking.
     8ÔåÆ
     9ÔåÆ**New goal**: Match each drug to an indication by cross-referencing the patient's GP diagnoses AND the drug's Search_Term mapping from DimSearchTerm.csv.
    10ÔåÆ
    11ÔåÆ## Key Data/Patterns
    12ÔåÆ
    13ÔåÆ### DimSearchTerm.csv
    14ÔåÆ- Located at `data/DimSearchTerm.csv`
    15ÔåÆ- Columns: Search_Term, CleanedDrugName (pipe-separated), PrimaryDirectorate
    16ÔåÆ- ~165 rows mapping clinical conditions to drug name fragments
    17ÔåÆ- Drug fragments are substrings that match standardized drug names from HCD data
    18ÔåÆ- Some entries have generic fragments: INHALED, CONTINUOUS, STANDARD-DOSE, PEGYLATED
    19ÔåÆ
    20ÔåÆ### Current get_patient_indication_groups() in diagnosis_lookup.py
    21ÔåÆ- Uses CLUSTER_MAPPING_SQL as CTE in Snowflake query
    22ÔåÆ- Returns ONLY the most recent match per patient (QUALIFY ROW_NUMBER() = 1)
    23ÔåÆ- Needs to return ALL matching Search_Terms per patient (remove QUALIFY)
    24ÔåÆ- Batches 500 patients per query
    25ÔåÆ
    26ÔåÆ### Modified UPID approach
    27ÔåÆ- Current: UPID = Provider Code[:3] + PersonKey (e.g., "RMV12345")
    28ÔåÆ- New: UPID = original + "|" + search_term (e.g., "RMV12345|rheumatoid arthritis")
    29ÔåÆ- The pipe delimiter "|" is safe because existing UPIDs are alphanumeric
    30ÔåÆ- generate_icicle_chart_indication() treats UPID as an opaque identifier ÔÇö modified UPIDs work transparently
    31ÔåÆ- The " - " delimiter in pathway ids is used for hierarchy levels, not within UPIDs
    32ÔåÆ
    33ÔåÆ### PseudoNHSNoLinked mapping
    34ÔåÆ- HCD data has PseudoNHSNoLinked column that matches PatientPseudonym in GP records
    35ÔåÆ- PersonKey is provider-specific local ID ÔÇö do NOT use for GP matching
    36ÔåÆ- One PseudoNHSNoLinked can map to multiple UPIDs (multi-provider patients)
    37ÔåÆ- GP match lookup: PseudoNHSNoLinked ÔåÆ list of matched Search_Terms
    38ÔåÆ
    39ÔåÆ### Drug matching logic
    40ÔåÆ- For each HCD row (UPID + Drug Name):
    41ÔåÆ  1. Get patient's GP-matched Search_Terms with code_frequency (via PseudoNHSNoLinked)
    42ÔåÆ  2. Get which Search_Terms list this drug (from DimSearchTerm.csv)
    43ÔåÆ  3. Intersection = valid indications
    44ÔåÆ  4. If 1: use it. If multiple: pick highest code_frequency (most GP coding = most likely indication). If 0: fallback to directory.
    45ÔåÆ- Modified UPID groups drugs under same indication together naturally
    46ÔåÆ- code_frequency = COUNT(*) of matching SNOMED codes per Search_Term per patient in GP records
    47ÔåÆ- GP code time range: only count codes from MIN(Intervention Date) onwards (the HCD data window)
    48ÔåÆ  - Reduces noise from old/irrelevant diagnoses, makes frequency more meaningful
    49ÔåÆ  - Pass earliest_hcd_date as parameter to get_patient_indication_groups()
    50ÔåÆ- Tiebreaker rationale: 47 RA codes vs 2 crohn's codes ÔåÆ RA is clearly the active condition
    51ÔåÆ
    52ÔåÆ### Known edge cases
    53ÔåÆ- Some DimSearchTerm drug fragments are generic (INHALED, ORAL, CONTINUOUS)
    54ÔåÆ  - These could match broadly but are constrained by GP diagnosis requirement
    55ÔåÆ- A patient visiting multiple providers has multiple UPIDs
    56ÔåÆ  - Each UPID gets its own drug-indication matching independently
    57ÔåÆ- Same Search_Term appears twice in DimSearchTerm.csv with different directorates
    58ÔåÆ  - e.g., "diabetes" ÔåÆ DIABETIC MEDICINE and OPHTHALMOLOGY
    59ÔåÆ  - For indication charts, we use Search_Term not directorate, so this is fine
    60ÔåÆ
    61ÔåÆ## Iteration Log
    62ÔåÆ
    63ÔåÆ## Iteration 1 ÔÇö 2026-02-05
    64ÔåÆ### Task: 1.3 ÔÇö Build drug-to-Search_Term lookup from DimSearchTerm.csv
    65ÔåÆ### Why this task:
    66ÔåÆ- First iteration, chose Phase 1 foundations. Task 1.2 (CSV loading) is self-contained and testable locally without Snowflake.
    67ÔåÆ- Task 1.1 (Snowflake query update) can't be verified without a live connection ÔÇö better to do 1.2 first.
    68ÔåÆ- Both 1.1 and 1.2 are independent, so order doesn't matter for dependencies.
    69ÔåÆ### Status: COMPLETE
    70ÔåÆ### What was done:
    71ÔåÆ- Added `load_drug_indication_mapping()` to `diagnosis_lookup.py`:
    72ÔåÆ  - Loads `data/DimSearchTerm.csv`, builds two dicts:
    73ÔåÆ    - `fragment_to_search_terms`: drug fragment (UPPER) ÔåÆ list of Search_Terms
    74ÔåÆ    - `search_term_to_fragments`: search_term ÔåÆ list of drug fragments (UPPER)
    75ÔåÆ  - Handles duplicate Search_Terms (e.g., "diabetes" rows combined)
    76ÔåÆ  - Result: 164 Search_Terms, 346 drug fragments
    77ÔåÆ- Added `get_search_terms_for_drug()` to `diagnosis_lookup.py`:
    78ÔåÆ  - Returns all Search_Terms whose drug fragments are substrings of the drug name (case-insensitive)
    79ÔåÆ  - Named differently from plan's `drug_matches_search_term()` ÔÇö returns all matches at once rather than single boolean, more practical for Phase 2
    80ÔåÆ- Updated `__all__` exports
    81ÔåÆ### Validation results:
    82ÔåÆ- Tier 1 (Code): py_compile passed, import check passed
    83ÔåÆ- Tier 2 (Data): ADALIMUMAB ÔåÆ 7 indications (including axial spondyloarthritis, rheumatoid arthritis), OMALIZUMAB ÔåÆ 4 indications (asthma, allergic asthma, etc.), PEGYLATED LIPOSOMAL DOXORUBICIN ÔåÆ 4 matches via substring, "ADALIMUMAB 40MG" matches correctly with dosage info, diabetes fragments combined from 2 CSV rows
    84ÔåÆ- Tier 3 (Functional): N/A (no UI changes)
    85ÔåÆ### Files changed:
    86ÔåÆ- data_processing/diagnosis_lookup.py (added load_drug_indication_mapping, get_search_terms_for_drug)
    87ÔåÆ- IMPLEMENTATION_PLAN.md (marked 1.2 subtasks [x])
    88ÔåÆ### Committed: 0779df7 "feat: add drug-to-indication mapping from DimSearchTerm.csv (Task 1.3)"
    89ÔåÆ### Patterns discovered:
    90ÔåÆ- DimSearchTerm.csv has 164 unique Search_Terms (not 165 as noted) because diabetes appears twice with different directorates but same Search_Term
    91ÔåÆ- Some drug fragments are very generic: INHALED, CONTINUOUS, ORAL, STANDARD-DOSE, INTRAVENOUS, PEGYLATED, ROUTINE, INDUCTION ÔÇö these will match broadly but are constrained by the GP diagnosis requirement in Phase 2
    92ÔåÆ- Function signatures for Phase 2: `get_search_terms_for_drug(drug_name, search_term_to_fragments)` returns list[str] ÔÇö use this to get candidate indications per drug
    93ÔåÆ### Next iteration should:
    94ÔåÆ- Work on Task 1.2: Merge asthma Search_Terms in CLUSTER_MAPPING_SQL and load_drug_indication_mapping()
    95ÔåÆ  - Merge "allergic asthma", "asthma", "severe persistent allergic asthma" ÔåÆ "asthma"
    96ÔåÆ  - Keep "urticaria" separate
    97ÔåÆ  - This is self-contained and testable locally
    98ÔåÆ- OR work on Task 1.1: Update `get_patient_indication_groups()` to return ALL matches with code_frequency
    99ÔåÆ  - The current query at line ~1352 of diagnosis_lookup.py uses `QUALIFY ROW_NUMBER() OVER (PARTITION BY pc."PatientPseudonym" ORDER BY pc."EventDateTime" DESC) = 1` ÔÇö this must be replaced with GROUP BY + COUNT(*)
   100ÔåÆ  - Add `earliest_hcd_date` parameter to restrict GP codes to HCD data window
   101ÔåÆ  - Return columns: PatientPseudonym, Search_Term, code_frequency (not EventDateTime)
   102ÔåÆ- OR if Snowflake isn't available to test 1.1, skip to Task 2.1 (assign_drug_indications function) which can be built and tested with mock data
   103ÔåÆ### Blocked items:
   104ÔåÆ- None
   105ÔåÆ

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# Progress Log - Drug-Aware Indication Matching

## Project Context

This project extends the indication-based pathway charts (Phase 1-5 complete) with drug-aware matching.

**Previous state**: Patients get ONE indication based on their most recent GP diagnosis match (SNOMED cluster codes). This ignores which drugs the patient is taking.

**New goal**: Match each drug to an indication by cross-referencing the patient's GP diagnoses AND the drug's Search_Term mapping from DimSearchTerm.csv.

## Key Data/Patterns

### DimSearchTerm.csv
- Located at `data/DimSearchTerm.csv`
- Columns: Search_Term, CleanedDrugName (pipe-separated), PrimaryDirectorate
- ~165 rows mapping clinical conditions to drug name fragments
- Drug fragments are substrings that match standardized drug names from HCD data
- Some entries have generic fragments: INHALED, CONTINUOUS, STANDARD-DOSE, PEGYLATED

### Current get_patient_indication_groups() in diagnosis_lookup.py
- Uses CLUSTER_MAPPING_SQL as CTE in Snowflake query
- Returns ONLY the most recent match per patient (QUALIFY ROW_NUMBER() = 1)
- Needs to return ALL matching Search_Terms per patient (remove QUALIFY)
- Batches 500 patients per query

### Modified UPID approach
- Current: UPID = Provider Code[:3] + PersonKey (e.g., "RMV12345")
- New: UPID = original + "|" + search_term (e.g., "RMV12345|rheumatoid arthritis")
- The pipe delimiter "|" is safe because existing UPIDs are alphanumeric
- generate_icicle_chart_indication() treats UPID as an opaque identifier ÔÇö modified UPIDs work transparently
- The " - " delimiter in pathway ids is used for hierarchy levels, not within UPIDs

### PseudoNHSNoLinked mapping
- HCD data has PseudoNHSNoLinked column that matches PatientPseudonym in GP records
- PersonKey is provider-specific local ID ÔÇö do NOT use for GP matching
- One PseudoNHSNoLinked can map to multiple UPIDs (multi-provider patients)
- GP match lookup: PseudoNHSNoLinked ÔåÆ list of matched Search_Terms

### Drug matching logic
- For each HCD row (UPID + Drug Name):
  1. Get patient's GP-matched Search_Terms with code_frequency (via PseudoNHSNoLinked)
  2. Get which Search_Terms list this drug (from DimSearchTerm.csv)
  3. Intersection = valid indications
  4. If 1: use it. If multiple: pick highest code_frequency (most GP coding = most likely indication). If 0: fallback to directory.
- Modified UPID groups drugs under same indication together naturally
- code_frequency = COUNT(*) of matching SNOMED codes per Search_Term per patient in GP records
- GP code time range: only count codes from MIN(Intervention Date) onwards (the HCD data window)
  - Reduces noise from old/irrelevant diagnoses, makes frequency more meaningful
  - Pass earliest_hcd_date as parameter to get_patient_indication_groups()
- Tiebreaker rationale: 47 RA codes vs 2 crohn's codes ÔåÆ RA is clearly the active condition

### Known edge cases
- Some DimSearchTerm drug fragments are generic (INHALED, ORAL, CONTINUOUS)
  - These could match broadly but are constrained by GP diagnosis requirement
- A patient visiting multiple providers has multiple UPIDs
  - Each UPID gets its own drug-indication matching independently
- Same Search_Term appears twice in DimSearchTerm.csv with different directorates
  - e.g., "diabetes" ÔåÆ DIABETIC MEDICINE and OPHTHALMOLOGY
  - For indication charts, we use Search_Term not directorate, so this is fine

## Iteration Log

## Iteration 1 ÔÇö 2026-02-05
### Task: 1.3 ÔÇö Build drug-to-Search_Term lookup from DimSearchTerm.csv
### Why this task:
- First iteration, chose Phase 1 foundations. Task 1.2 (CSV loading) is self-contained and testable locally without Snowflake.
- Task 1.1 (Snowflake query update) can't be verified without a live connection ÔÇö better to do 1.2 first.
- Both 1.1 and 1.2 are independent, so order doesn't matter for dependencies.
### Status: COMPLETE
### What was done:
- Added `load_drug_indication_mapping()` to `diagnosis_lookup.py`:
  - Loads `data/DimSearchTerm.csv`, builds two dicts:
    - `fragment_to_search_terms`: drug fragment (UPPER) ÔåÆ list of Search_Terms
    - `search_term_to_fragments`: search_term ÔåÆ list of drug fragments (UPPER)
  - Handles duplicate Search_Terms (e.g., "diabetes" rows combined)
  - Result: 164 Search_Terms, 346 drug fragments
- Added `get_search_terms_for_drug()` to `diagnosis_lookup.py`:
  - Returns all Search_Terms whose drug fragments are substrings of the drug name (case-insensitive)
  - Named differently from plan's `drug_matches_search_term()` ÔÇö returns all matches at once rather than single boolean, more practical for Phase 2
- Updated `__all__` exports
### Validation results:
- Tier 1 (Code): py_compile passed, import check passed
- Tier 2 (Data): ADALIMUMAB ÔåÆ 7 indications (including axial spondyloarthritis, rheumatoid arthritis), OMALIZUMAB ÔåÆ 4 indications (asthma, allergic asthma, etc.), PEGYLATED LIPOSOMAL DOXORUBICIN ÔåÆ 4 matches via substring, "ADALIMUMAB 40MG" matches correctly with dosage info, diabetes fragments combined from 2 CSV rows
- Tier 3 (Functional): N/A (no UI changes)
### Files changed:
- data_processing/diagnosis_lookup.py (added load_drug_indication_mapping, get_search_terms_for_drug)
- IMPLEMENTATION_PLAN.md (marked 1.2 subtasks [x])
### Committed: 0779df7 "feat: add drug-to-indication mapping from DimSearchTerm.csv (Task 1.3)"
### Patterns discovered:
- DimSearchTerm.csv has 164 unique Search_Terms (not 165 as noted) because diabetes appears twice with different directorates but same Search_Term
- Some drug fragments are very generic: INHALED, CONTINUOUS, ORAL, STANDARD-DOSE, INTRAVENOUS, PEGYLATED, ROUTINE, INDUCTION ÔÇö these will match broadly but are constrained by the GP diagnosis requirement in Phase 2
- Function signatures for Phase 2: `get_search_terms_for_drug(drug_name, search_term_to_fragments)` returns list[str] ÔÇö use this to get candidate indications per drug
### Next iteration should:
- Work on Task 1.2: Merge asthma Search_Terms in CLUSTER_MAPPING_SQL and load_drug_indication_mapping()
  - Merge "allergic asthma", "asthma", "severe persistent allergic asthma" ÔåÆ "asthma"
  - Keep "urticaria" separate
  - This is self-contained and testable locally
- OR work on Task 1.1: Update `get_patient_indication_groups()` to return ALL matches with code_frequency
  - The current query at line ~1352 of diagnosis_lookup.py uses `QUALIFY ROW_NUMBER() OVER (PARTITION BY pc."PatientPseudonym" ORDER BY pc."EventDateTime" DESC) = 1` ÔÇö this must be replaced with GROUP BY + COUNT(*)
  - Add `earliest_hcd_date` parameter to restrict GP codes to HCD data window
  - Return columns: PatientPseudonym, Search_Term, code_frequency (not EventDateTime)
- OR if Snowflake isn't available to test 1.1, skip to Task 2.1 (assign_drug_indications function) which can be built and tested with mock data
### Blocked items:
- None


     1ÔåÆ# Implementation Plan - Drug-Aware Indication Matching
     2ÔåÆ
     3ÔåÆ## Project Overview
     4ÔåÆ
     5ÔåÆUpdate the indication-based pathway charts so that patient indications are matched **per drug**, not just per patient. Currently, each patient gets ONE indication (most recent GP diagnosis match). This ignores which drugs the patient is actually taking.
     6ÔåÆ
     7ÔåÆ### The Problem
     8ÔåÆ
     9ÔåÆA patient on ADALIMUMAB + OMALIZUMAB currently gets assigned a single indication (e.g., "rheumatoid arthritis" ÔÇö the most recent GP match). But:
    10ÔåÆ- ADALIMUMAB is used for rheumatoid arthritis, axial spondyloarthritis, crohn's disease, etc.
    11ÔåÆ- OMALIZUMAB is used for asthma, allergic asthma, urticaria
    12ÔåÆ
    13ÔåÆThese are different clinical pathways and should be treated as separate treatment journeys.
    14ÔåÆ
    15ÔåÆ### The Solution
    16ÔåÆ
    17ÔåÆMatch each drug to an indication by cross-referencing:
    18ÔåÆ1. **GP diagnosis** ÔÇö which Search_Terms the patient has matching SNOMED codes for
    19ÔåÆ2. **Drug mapping** ÔÇö which Search_Terms list each drug (from `DimSearchTerm.csv`)
    20ÔåÆ
    21ÔåÆOnly assign a drug to an indication if BOTH conditions are met. If a patient's drugs map to different indications, they become separate pathways (via modified UPID).
    22ÔåÆ
    23ÔåÆ### Key Design Decisions
    24ÔåÆ
    25ÔåÆ| Aspect | Decision |
    26ÔåÆ|--------|----------|
    27ÔåÆ| Drug-indication source | `data/DimSearchTerm.csv` ÔÇö Search_Term ÔåÆ CleanedDrugName mapping |
    28ÔåÆ| UPID modification | `{original_UPID}\|{search_term}` for drugs with matched indication |
    29ÔåÆ| GP diagnosis matching | Return ALL matches per patient (not just most recent) |
    30ÔåÆ| Drug matching | Substring match: HCD drug name contains DimSearchTerm fragment |
    31ÔåÆ| Multiple indication matches per drug | Use highest GP code frequency as tiebreaker (COUNT of matching SNOMED codes per Search_Term) |
    32ÔåÆ| GP code time range | Only codes from MIN(Intervention Date) onwards ÔÇö restricts to HCD data window |
    33ÔåÆ| No indication match | Fallback to directory (same as current behavior) |
    34ÔåÆ| Same patient, different indications | Separate pathways via different modified UPIDs |
    35ÔåÆ
    36ÔåÆ### Examples
    37ÔåÆ
    38ÔåÆ**Patient on ADALIMUMAB + GOLIMUMAB, GP dx: axial spondyloarthritis + asthma**
    39ÔåÆ- axial spondyloarthritis drug list includes both ADALIMUMAB and GOLIMUMAB
    40ÔåÆ- ÔåÆ Both drugs grouped under "axial spondyloarthritis", single pathway
    41ÔåÆ- Modified UPID: `RMV12345|axial spondyloarthritis`
    42ÔåÆ
    43ÔåÆ**Patient on ADALIMUMAB + OMALIZUMAB, GP dx: axial spondyloarthritis + asthma**
    44ÔåÆ- axial spondyloarthritis lists ADALIMUMAB but not OMALIZUMAB
    45ÔåÆ- asthma lists OMALIZUMAB but not ADALIMUMAB
    46ÔåÆ- ÔåÆ Two separate pathways:
    47ÔåÆ  - `RMV12345|axial spondyloarthritis` with ADALIMUMAB
    48ÔåÆ  - `RMV12345|asthma` with OMALIZUMAB
    49ÔåÆ
    50ÔåÆ**Patient on ADALIMUMAB, GP dx: rheumatoid arthritis (47 codes) + crohn's disease (2 codes)**
    51ÔåÆ- Both Search_Terms list ADALIMUMAB AND patient has GP dx for both
    52ÔåÆ- ÔåÆ Tiebreaker: highest code frequency ÔÇö rheumatoid arthritis has 47 matching SNOMED codes vs 2 for crohn's
    53ÔåÆ- ÔåÆ Single pathway under rheumatoid arthritis (more clinical activity = more likely the treatment indication)
    54ÔåÆ
    55ÔåÆ---
    56ÔåÆ
    57ÔåÆ## Phase 1: Update Snowflake Query & Drug Mapping
    58ÔåÆ
    59ÔåÆ### 1.1 Update `get_patient_indication_groups()` to return ALL matches with frequency
    60ÔåÆ- [ ] Modify the Snowflake query in `get_patient_indication_groups()` (diagnosis_lookup.py):
    61ÔåÆ  - Remove `QUALIFY ROW_NUMBER() OVER (PARTITION BY ... ORDER BY EventDateTime DESC) = 1`
    62ÔåÆ  - Return ALL matching Search_Terms per patient with code frequency:
    63ÔåÆ    ```sql
    64ÔåÆ    SELECT pc."PatientPseudonym" AS "PatientPseudonym",
    65ÔåÆ           aic.Search_Term AS "Search_Term",
    66ÔåÆ           COUNT(*) AS "code_frequency"
    67ÔåÆ    FROM PrimaryCareClinicalCoding pc
    68ÔåÆ    JOIN AllIndicationCodes aic ON pc."SNOMEDCode" = aic.SNOMEDCode
    69ÔåÆ    WHERE pc."PatientPseudonym" IN (...)
    70ÔåÆ      AND pc."EventDateTime" >= :earliest_hcd_date
    71ÔåÆ    GROUP BY pc."PatientPseudonym", aic.Search_Term
    72ÔåÆ    ```
    73ÔåÆ  - `code_frequency` = number of matching SNOMED codes per Search_Term per patient
    74ÔåÆ  - Higher frequency = more clinical activity = stronger signal for tiebreaker
    75ÔåÆ  - `earliest_hcd_date` = `MIN(Intervention Date)` from the HCD DataFrame ÔÇö restricts GP codes to the HCD data window, reducing noise from old/irrelevant diagnoses
    76ÔåÆ- [ ] Accept `earliest_hcd_date` parameter in `get_patient_indication_groups()` and pass to query
    77ÔåÆ- [ ] Keep batch processing (500 patients per query)
    78ÔåÆ- [ ] Update return type: DataFrame now has multiple rows per patient (PatientPseudonym, Search_Term, code_frequency)
    79ÔåÆ- [ ] Verify: Query returns more rows than before (patients with multiple matching diagnoses)
    80ÔåÆ
    81ÔåÆ### 1.2 Merge related asthma Search_Terms in CLUSTER_MAPPING_SQL
    82ÔåÆ- [ ] In `CLUSTER_MAPPING_SQL` (diagnosis_lookup.py), merge these 3 Search_Terms into one `"asthma"` entry:
    83ÔåÆ  - `allergic asthma` (Cluster: OMALIZUMAB only)
    84ÔåÆ  - `asthma` (Cluster: BENRALIZUMAB, DUPILUMAB, INHALED, MEPOLIZUMAB, OMALIZUMAB, RESLIZUMAB)
    85ÔåÆ  - `severe persistent allergic asthma` (Cluster: OMALIZUMAB only)
    86ÔåÆ- [ ] Map all 3 Cluster_IDs to `Search_Term = 'asthma'` in the CTE VALUES
    87ÔåÆ- [ ] `urticaria` (OMALIZUMAB, DERMATOLOGY) stays SEPARATE ÔÇö do NOT merge with asthma
    88ÔåÆ- [ ] Also update `load_drug_indication_mapping()` to apply the same merge when loading DimSearchTerm.csv:
    89ÔåÆ  - Combine drug lists from all 3 entries under a single `"asthma"` key
    90ÔåÆ  - Deduplicate drug fragments (OMALIZUMAB appears in all 3)
    91ÔåÆ- [ ] Verify: GP code lookup returns `"asthma"` (not `"allergic asthma"` or `"severe persistent allergic asthma"`)
    92ÔåÆ- [ ] Verify: Drug mapping for `"asthma"` includes full combined drug list: BENRALIZUMAB, DUPILUMAB, INHALED, MEPOLIZUMAB, OMALIZUMAB, RESLIZUMAB
    93ÔåÆ
    94ÔåÆ### 1.3 Build drug-to-Search_Term lookup from DimSearchTerm.csv
    95ÔåÆ- [x] Add function `load_drug_indication_mapping()` to `diagnosis_lookup.py`:
    96ÔåÆ  - Loads `data/DimSearchTerm.csv`
    97ÔåÆ  - Builds dict: `drug_fragment (uppercase) ÔåÆ list[Search_Term]`
    98ÔåÆ  - Also builds reverse: `search_term ÔåÆ list[drug_fragments]`
    99ÔåÆ  - CleanedDrugName is pipe-separated (e.g., "ADALIMUMAB|GOLIMUMAB|IXEKIZUMAB")
   100ÔåÆ- [x] Add function `get_search_terms_for_drug(drug_name, search_term_to_fragments) -> list[str]`:
   101ÔåÆ  - Returns all Search_Terms whose drug fragments are substrings of the drug name (case-insensitive)
   102ÔåÆ  - More practical than per-term boolean check ÔÇö returns all matches at once for Phase 2 use
   103ÔåÆ- [x] Verify: ADALIMUMAB matches "axial spondyloarthritis", OMALIZUMAB matches "asthma"
   104ÔåÆ
   105ÔåÆ---
   106ÔåÆ
   107ÔåÆ## Phase 2: Drug-Aware Indication Matching Logic
   108ÔåÆ
   109ÔåÆ### 2.1 Create `assign_drug_indications()` function
   110ÔåÆ- [ ] Add to `diagnosis_lookup.py` or `pathway_pipeline.py`:
   111ÔåÆ  ```
   112ÔåÆ  def assign_drug_indications(
   113ÔåÆ      df: pd.DataFrame,              # HCD data with UPID, Drug Name columns
   114ÔåÆ      gp_matches_df: pd.DataFrame,   # PatientPseudonym ÔåÆ list of matched Search_Terms
   115ÔåÆ      drug_mapping: dict,             # From load_drug_indication_mapping()
   116ÔåÆ  ) -> tuple[pd.DataFrame, pd.DataFrame]:
   117ÔåÆ      Returns: (modified_df, indication_df)
   118ÔåÆ      - modified_df: HCD data with UPID replaced by {UPID}|{indication}
   119ÔåÆ      - indication_df: mapping modified_UPID ÔåÆ Search_Term
   120ÔåÆ  ```
   121ÔåÆ- [ ] Logic per UPID + Drug Name pair:
   122ÔåÆ  1. Get patient's GP-matched Search_Terms with code_frequency (from gp_matches_df via PseudoNHSNoLinked)
   123ÔåÆ  2. Get which Search_Terms include this drug (from drug_mapping)
   124ÔåÆ  3. Intersection = valid indications for this drug-patient pair
   125ÔåÆ  4. If 1 match: use it
   126ÔåÆ  5. If multiple matches: use highest code_frequency as tiebreaker (most GP coding activity = most likely treatment indication)
   127ÔåÆ  6. If 0 matches: use fallback directory
   128ÔåÆ- [ ] Modify UPID in df rows: `{original_UPID}|{matched_search_term}`
   129ÔåÆ- [ ] Build indication_df: `{modified_UPID}` ÔåÆ `Search_Term` (or fallback label)
   130ÔåÆ- [ ] Verify: Function compiles, handles edge cases (no GP match, no drug match)
   131ÔåÆ
   132ÔåÆ### 2.2 Handle tiebreaker for multiple indication matches
   133ÔåÆ- [ ] When a drug matches multiple Search_Terms AND patient has GP dx for multiple:
   134ÔåÆ  - Use `code_frequency` from the GP query (COUNT of matching SNOMED codes per Search_Term)
   135ÔåÆ  - Higher code_frequency = more clinical activity for that condition = more likely treatment indication
   136ÔåÆ  - E.g., patient with 47 RA codes and 2 crohn's codes ÔåÆ ADALIMUMAB assigned to RA
   137ÔåÆ  - code_frequency is already returned by the updated query in Task 1.1
   138ÔåÆ- [ ] Verify: Tiebreaker logic correctly picks highest-frequency diagnosis
   139ÔåÆ- [ ] Verify: Tie on frequency (rare but possible) falls back to alphabetical Search_Term for determinism
   140ÔåÆ
   141ÔåÆ---
   142ÔåÆ
   143ÔåÆ## Phase 3: Pipeline Integration
   144ÔåÆ
   145ÔåÆ### 3.1 Update `refresh_pathways.py` indication processing
   146ÔåÆ- [ ] In the `elif current_chart_type == "indication":` block:
   147ÔåÆ  1. Call `get_patient_indication_groups()` as before (but now returns ALL matches)
   148ÔåÆ  2. Load drug mapping: `drug_mapping = load_drug_indication_mapping()`
   149ÔåÆ  3. Call `assign_drug_indications(df, gp_matches_df, drug_mapping)`
   150ÔåÆ  4. Use modified_df (with indication-aware UPIDs) for pathway processing
   151ÔåÆ  5. Use indication_df for the indication mapping
   152ÔåÆ- [ ] Pass modified_df (not original df) to `process_indication_pathway_for_date_filter()`
   153ÔåÆ- [ ] Verify: Pipeline compiles, `python -m py_compile cli/refresh_pathways.py`
   154ÔåÆ
   155ÔåÆ### 3.2 Test with dry run
   156ÔåÆ- [ ] Run `python -m cli.refresh_pathways --chart-type indication --dry-run -v`
   157ÔåÆ- [ ] Verify:
   158ÔåÆ  - Modified UPIDs appear in pipeline log (e.g., `RMV12345|rheumatoid arthritis`)
   159ÔåÆ  - Patient counts are reasonable (will be higher than before since same patient can appear under multiple indications)
   160ÔåÆ  - Drug-indication matching is logged (match rate, fallback rate)
   161ÔåÆ  - Pathway hierarchy shows drug-specific grouping under correct indications
   162ÔåÆ
   163ÔåÆ---
   164ÔåÆ
   165ÔåÆ## Phase 4: Full Refresh & Validation
   166ÔåÆ
   167ÔåÆ### 4.1 Full refresh with both chart types
   168ÔåÆ- [ ] Run `python -m cli.refresh_pathways --chart-type all`
   169ÔåÆ- [ ] Verify:
   170ÔåÆ  - Both chart types generate data
   171ÔåÆ  - Directory charts unchanged (no modified UPIDs)
   172ÔåÆ  - Indication charts reflect drug-aware matching
   173ÔåÆ
   174ÔåÆ### 4.2 Validate indication chart correctness
   175ÔåÆ- [ ] Check that drugs under an indication all appear in that Search_Term's drug list
   176ÔåÆ- [ ] Verify that a patient on drugs for different indications creates separate pathway branches
   177ÔåÆ- [ ] Verify that drugs sharing an indication are grouped in the same pathway
   178ÔåÆ- [ ] Log: patient count comparison (old vs new approach)
   179ÔåÆ
   180ÔåÆ### 4.3 Validate Reflex UI
   181ÔåÆ- [ ] Run `python -m reflex compile` to verify app compiles
   182ÔåÆ- [ ] Verify chart type toggle still works
   183ÔåÆ- [ ] Verify indication chart shows correct hierarchy
   184ÔåÆ
   185ÔåÆ---
   186ÔåÆ
   187ÔåÆ## Completion Criteria
   188ÔåÆ
   189ÔåÆAll tasks marked `[x]` AND:
   190ÔåÆ- [ ] App compiles without errors (`reflex compile` succeeds)
   191ÔåÆ- [ ] Both chart types generate pathway data
   192ÔåÆ- [ ] Indication charts show drug-specific indication matching
   193ÔåÆ- [ ] Drugs under the same indication for the same patient are in one pathway
   194ÔåÆ- [ ] Drugs under different indications for the same patient create separate pathways
   195ÔåÆ- [ ] Fallback works for drugs with no indication match
   196ÔåÆ- [ ] Full refresh completes successfully
   197ÔåÆ- [ ] Existing directory charts are unaffected
   198ÔåÆ
   199ÔåÆ---
   200ÔåÆ
   201ÔåÆ## Reference
   202ÔåÆ
   203ÔåÆ### DimSearchTerm.csv Structure
   204ÔåÆ```
   205ÔåÆSearch_Term,CleanedDrugName,PrimaryDirectorate
   206ÔåÆrheumatoid arthritis,ABATACEPT|ADALIMUMAB|ANAKINRA|BARICITINIB|...,RHEUMATOLOGY
   207ÔåÆasthma,BENRALIZUMAB|DUPILUMAB|INHALED|MEPOLIZUMAB|OMALIZUMAB|RESLIZUMAB,THORACIC MEDICINE
   208ÔåÆ```
   209ÔåÆ
   210ÔåÆ### Modified UPID Format
   211ÔåÆ```
   212ÔåÆOriginal:  RMV12345
   213ÔåÆModified:  RMV12345|rheumatoid arthritis
   214ÔåÆFallback:  RMV12345|RHEUMATOLOGY (no GP dx)
   215ÔåÆ```
   216ÔåÆ
   217ÔåÆ### Current vs New Indication Flow
   218ÔåÆ```
   219ÔåÆCURRENT:
   220ÔåÆ  Patient ÔåÆ GP dx (most recent) ÔåÆ single Search_Term ÔåÆ one pathway
   221ÔåÆ
   222ÔåÆNEW:
   223ÔåÆ  Patient + Drug A ÔåÆ GP dx matching Drug A ÔåÆ Search_Term X
   224ÔåÆ  Patient + Drug B ÔåÆ GP dx matching Drug B ÔåÆ Search_Term Y
   225ÔåÆ  ÔåÆ If X == Y: one pathway under X
   226ÔåÆ  ÔåÆ If X != Y: two pathways (modified UPIDs)
   227ÔåÆ```
   228ÔåÆ
   229ÔåÆ### Key Files
   230ÔåÆ
   231ÔåÆ| File | Changes |
   232ÔåÆ|------|---------|
   233ÔåÆ| `data_processing/diagnosis_lookup.py` | Update query, add drug mapping functions |
   234ÔåÆ| `data_processing/pathway_pipeline.py` | Possibly minor changes for modified UPIDs |
   235ÔåÆ| `cli/refresh_pathways.py` | Integrate drug-aware matching into pipeline |
   236ÔåÆ| `data/DimSearchTerm.csv` | Reference data (read-only) |
   237ÔåÆ| `analysis/pathway_analyzer.py` | No changes expected (UPID changes are transparent) |
   238ÔåÆ| `pathways_app/pathways_app.py` | No changes expected |
   239ÔåÆ

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# Implementation Plan - Drug-Aware Indication Matching

## Project Overview

Update the indication-based pathway charts so that patient indications are matched **per drug**, not just per patient. Currently, each patient gets ONE indication (most recent GP diagnosis match). This ignores which drugs the patient is actually taking.

### The Problem

A patient on ADALIMUMAB + OMALIZUMAB currently gets assigned a single indication (e.g., "rheumatoid arthritis" ÔÇö the most recent GP match). But:
- ADALIMUMAB is used for rheumatoid arthritis, axial spondyloarthritis, crohn's disease, etc.
- OMALIZUMAB is used for asthma, allergic asthma, urticaria

These are different clinical pathways and should be treated as separate treatment journeys.

### The Solution

Match each drug to an indication by cross-referencing:
1. **GP diagnosis** ÔÇö which Search_Terms the patient has matching SNOMED codes for
2. **Drug mapping** ÔÇö which Search_Terms list each drug (from `DimSearchTerm.csv`)

Only assign a drug to an indication if BOTH conditions are met. If a patient's drugs map to different indications, they become separate pathways (via modified UPID).

### Key Design Decisions

| Aspect | Decision |
|--------|----------|
| Drug-indication source | `data/DimSearchTerm.csv` ÔÇö Search_Term ÔåÆ CleanedDrugName mapping |
| UPID modification | `{original_UPID}\|{search_term}` for drugs with matched indication |
| GP diagnosis matching | Return ALL matches per patient (not just most recent) |
| Drug matching | Substring match: HCD drug name contains DimSearchTerm fragment |
| Multiple indication matches per drug | Use highest GP code frequency as tiebreaker (COUNT of matching SNOMED codes per Search_Term) |
| GP code time range | Only codes from MIN(Intervention Date) onwards ÔÇö restricts to HCD data window |
| No indication match | Fallback to directory (same as current behavior) |
| Same patient, different indications | Separate pathways via different modified UPIDs |

### Examples

**Patient on ADALIMUMAB + GOLIMUMAB, GP dx: axial spondyloarthritis + asthma**
- axial spondyloarthritis drug list includes both ADALIMUMAB and GOLIMUMAB
- ÔåÆ Both drugs grouped under "axial spondyloarthritis", single pathway
- Modified UPID: `RMV12345|axial spondyloarthritis`

**Patient on ADALIMUMAB + OMALIZUMAB, GP dx: axial spondyloarthritis + asthma**
- axial spondyloarthritis lists ADALIMUMAB but not OMALIZUMAB
- asthma lists OMALIZUMAB but not ADALIMUMAB
- ÔåÆ Two separate pathways:
  - `RMV12345|axial spondyloarthritis` with ADALIMUMAB
  - `RMV12345|asthma` with OMALIZUMAB

**Patient on ADALIMUMAB, GP dx: rheumatoid arthritis (47 codes) + crohn's disease (2 codes)**
- Both Search_Terms list ADALIMUMAB AND patient has GP dx for both
- ÔåÆ Tiebreaker: highest code frequency ÔÇö rheumatoid arthritis has 47 matching SNOMED codes vs 2 for crohn's
- ÔåÆ Single pathway under rheumatoid arthritis (more clinical activity = more likely the treatment indication)

---

## Phase 1: Update Snowflake Query & Drug Mapping

### 1.1 Update `get_patient_indication_groups()` to return ALL matches with frequency
- [ ] Modify the Snowflake query in `get_patient_indication_groups()` (diagnosis_lookup.py):
  - Remove `QUALIFY ROW_NUMBER() OVER (PARTITION BY ... ORDER BY EventDateTime DESC) = 1`
  - Return ALL matching Search_Terms per patient with code frequency:
    ```sql
    SELECT pc."PatientPseudonym" AS "PatientPseudonym",
           aic.Search_Term AS "Search_Term",
           COUNT(*) AS "code_frequency"
    FROM PrimaryCareClinicalCoding pc
    JOIN AllIndicationCodes aic ON pc."SNOMEDCode" = aic.SNOMEDCode
    WHERE pc."PatientPseudonym" IN (...)
      AND pc."EventDateTime" >= :earliest_hcd_date
    GROUP BY pc."PatientPseudonym", aic.Search_Term
    ```
  - `code_frequency` = number of matching SNOMED codes per Search_Term per patient
  - Higher frequency = more clinical activity = stronger signal for tiebreaker
  - `earliest_hcd_date` = `MIN(Intervention Date)` from the HCD DataFrame ÔÇö restricts GP codes to the HCD data window, reducing noise from old/irrelevant diagnoses
- [ ] Accept `earliest_hcd_date` parameter in `get_patient_indication_groups()` and pass to query
- [ ] Keep batch processing (500 patients per query)
- [ ] Update return type: DataFrame now has multiple rows per patient (PatientPseudonym, Search_Term, code_frequency)
- [ ] Verify: Query returns more rows than before (patients with multiple matching diagnoses)

### 1.2 Merge related asthma Search_Terms in CLUSTER_MAPPING_SQL
- [ ] In `CLUSTER_MAPPING_SQL` (diagnosis_lookup.py), merge these 3 Search_Terms into one `"asthma"` entry:
  - `allergic asthma` (Cluster: OMALIZUMAB only)
  - `asthma` (Cluster: BENRALIZUMAB, DUPILUMAB, INHALED, MEPOLIZUMAB, OMALIZUMAB, RESLIZUMAB)
  - `severe persistent allergic asthma` (Cluster: OMALIZUMAB only)
- [ ] Map all 3 Cluster_IDs to `Search_Term = 'asthma'` in the CTE VALUES
- [ ] `urticaria` (OMALIZUMAB, DERMATOLOGY) stays SEPARATE ÔÇö do NOT merge with asthma
- [ ] Also update `load_drug_indication_mapping()` to apply the same merge when loading DimSearchTerm.csv:
  - Combine drug lists from all 3 entries under a single `"asthma"` key
  - Deduplicate drug fragments (OMALIZUMAB appears in all 3)
- [ ] Verify: GP code lookup returns `"asthma"` (not `"allergic asthma"` or `"severe persistent allergic asthma"`)
- [ ] Verify: Drug mapping for `"asthma"` includes full combined drug list: BENRALIZUMAB, DUPILUMAB, INHALED, MEPOLIZUMAB, OMALIZUMAB, RESLIZUMAB

### 1.3 Build drug-to-Search_Term lookup from DimSearchTerm.csv
- [x] Add function `load_drug_indication_mapping()` to `diagnosis_lookup.py`:
  - Loads `data/DimSearchTerm.csv`
  - Builds dict: `drug_fragment (uppercase) ÔåÆ list[Search_Term]`
  - Also builds reverse: `search_term ÔåÆ list[drug_fragments]`
  - CleanedDrugName is pipe-separated (e.g., "ADALIMUMAB|GOLIMUMAB|IXEKIZUMAB")
- [x] Add function `get_search_terms_for_drug(drug_name, search_term_to_fragments) -> list[str]`:
  - Returns all Search_Terms whose drug fragments are substrings of the drug name (case-insensitive)
  - More practical than per-term boolean check ÔÇö returns all matches at once for Phase 2 use
- [x] Verify: ADALIMUMAB matches "axial spondyloarthritis", OMALIZUMAB matches "asthma"

---

## Phase 2: Drug-Aware Indication Matching Logic

### 2.1 Create `assign_drug_indications()` function
- [ ] Add to `diagnosis_lookup.py` or `pathway_pipeline.py`:
  ```
  def assign_drug_indications(
      df: pd.DataFrame,              # HCD data with UPID, Drug Name columns
      gp_matches_df: pd.DataFrame,   # PatientPseudonym ÔåÆ list of matched Search_Terms
      drug_mapping: dict,             # From load_drug_indication_mapping()
  ) -> tuple[pd.DataFrame, pd.DataFrame]:
      Returns: (modified_df, indication_df)
      - modified_df: HCD data with UPID replaced by {UPID}|{indication}
      - indication_df: mapping modified_UPID ÔåÆ Search_Term
  ```
- [ ] Logic per UPID + Drug Name pair:
  1. Get patient's GP-matched Search_Terms with code_frequency (from gp_matches_df via PseudoNHSNoLinked)
  2. Get which Search_Terms include this drug (from drug_mapping)
  3. Intersection = valid indications for this drug-patient pair
  4. If 1 match: use it
  5. If multiple matches: use highest code_frequency as tiebreaker (most GP coding activity = most likely treatment indication)
  6. If 0 matches: use fallback directory
- [ ] Modify UPID in df rows: `{original_UPID}|{matched_search_term}`
- [ ] Build indication_df: `{modified_UPID}` ÔåÆ `Search_Term` (or fallback label)
- [ ] Verify: Function compiles, handles edge cases (no GP match, no drug match)

### 2.2 Handle tiebreaker for multiple indication matches
- [ ] When a drug matches multiple Search_Terms AND patient has GP dx for multiple:
  - Use `code_frequency` from the GP query (COUNT of matching SNOMED codes per Search_Term)
  - Higher code_frequency = more clinical activity for that condition = more likely treatment indication
  - E.g., patient with 47 RA codes and 2 crohn's codes ÔåÆ ADALIMUMAB assigned to RA
  - code_frequency is already returned by the updated query in Task 1.1
- [ ] Verify: Tiebreaker logic correctly picks highest-frequency diagnosis
- [ ] Verify: Tie on frequency (rare but possible) falls back to alphabetical Search_Term for determinism

---

## Phase 3: Pipeline Integration

### 3.1 Update `refresh_pathways.py` indication processing
- [ ] In the `elif current_chart_type == "indication":` block:
  1. Call `get_patient_indication_groups()` as before (but now returns ALL matches)
  2. Load drug mapping: `drug_mapping = load_drug_indication_mapping()`
  3. Call `assign_drug_indications(df, gp_matches_df, drug_mapping)`
  4. Use modified_df (with indication-aware UPIDs) for pathway processing
  5. Use indication_df for the indication mapping
- [ ] Pass modified_df (not original df) to `process_indication_pathway_for_date_filter()`
- [ ] Verify: Pipeline compiles, `python -m py_compile cli/refresh_pathways.py`

### 3.2 Test with dry run
- [ ] Run `python -m cli.refresh_pathways --chart-type indication --dry-run -v`
- [ ] Verify:
  - Modified UPIDs appear in pipeline log (e.g., `RMV12345|rheumatoid arthritis`)
  - Patient counts are reasonable (will be higher than before since same patient can appear under multiple indications)
  - Drug-indication matching is logged (match rate, fallback rate)
  - Pathway hierarchy shows drug-specific grouping under correct indications

---

## Phase 4: Full Refresh & Validation

### 4.1 Full refresh with both chart types
- [ ] Run `python -m cli.refresh_pathways --chart-type all`
- [ ] Verify:
  - Both chart types generate data
  - Directory charts unchanged (no modified UPIDs)
  - Indication charts reflect drug-aware matching

### 4.2 Validate indication chart correctness
- [ ] Check that drugs under an indication all appear in that Search_Term's drug list
- [ ] Verify that a patient on drugs for different indications creates separate pathway branches
- [ ] Verify that drugs sharing an indication are grouped in the same pathway
- [ ] Log: patient count comparison (old vs new approach)

### 4.3 Validate Reflex UI
- [ ] Run `python -m reflex compile` to verify app compiles
- [ ] Verify chart type toggle still works
- [ ] Verify indication chart shows correct hierarchy

---

## Completion Criteria

All tasks marked `[x]` AND:
- [ ] App compiles without errors (`reflex compile` succeeds)
- [ ] Both chart types generate pathway data
- [ ] Indication charts show drug-specific indication matching
- [ ] Drugs under the same indication for the same patient are in one pathway
- [ ] Drugs under different indications for the same patient create separate pathways
- [ ] Fallback works for drugs with no indication match
- [ ] Full refresh completes successfully
- [ ] Existing directory charts are unaffected

---

## Reference

### DimSearchTerm.csv Structure
```
Search_Term,CleanedDrugName,PrimaryDirectorate
rheumatoid arthritis,ABATACEPT|ADALIMUMAB|ANAKINRA|BARICITINIB|...,RHEUMATOLOGY
asthma,BENRALIZUMAB|DUPILUMAB|INHALED|MEPOLIZUMAB|OMALIZUMAB|RESLIZUMAB,THORACIC MEDICINE
```

### Modified UPID Format
```
Original:  RMV12345
Modified:  RMV12345|rheumatoid arthritis
Fallback:  RMV12345|RHEUMATOLOGY (no GP dx)
```

### Current vs New Indication Flow
```
CURRENT:
  Patient ÔåÆ GP dx (most recent) ÔåÆ single Search_Term ÔåÆ one pathway

NEW:
  Patient + Drug A ÔåÆ GP dx matching Drug A ÔåÆ Search_Term X
  Patient + Drug B ÔåÆ GP dx matching Drug B ÔåÆ Search_Term Y
  ÔåÆ If X == Y: one pathway under X
  ÔåÆ If X != Y: two pathways (modified UPIDs)
```

### Key Files

| File | Changes |
|------|---------|
| `data_processing/diagnosis_lookup.py` | Update query, add drug mapping functions |
| `data_processing/pathway_pipeline.py` | Possibly minor changes for modified UPIDs |
| `cli/refresh_pathways.py` | Integrate drug-aware matching into pipeline |
| `data/DimSearchTerm.csv` | Reference data (read-only) |
| `analysis/pathway_analyzer.py` | No changes expected (UPID changes are transparent) |
| `pathways_app/pathways_app.py` | No changes expected |


     1ÔåÆ# Guardrails
     2ÔåÆ
     3ÔåÆKnown failure patterns. Read EVERY iteration. Follow ALL of these rules.
     4ÔåÆIf you discover a new failure pattern during your work, add it to this file.
     5ÔåÆ
     6ÔåÆ---
     7ÔåÆ
     8ÔåÆ## Drug-Indication Matching Guardrails
     9ÔåÆ
    10ÔåÆ### Match drugs to indications, not just patients to indications
    11ÔåÆ- **When**: Building the indication mapping for pathway charts
    12ÔåÆ- **Rule**: Each drug must be validated against BOTH the patient's GP diagnoses AND the drug-to-indication mapping from DimSearchTerm.csv. A patient being diagnosed with rheumatoid arthritis does NOT mean all their drugs are for rheumatoid arthritis.
    13ÔåÆ- **Why**: The previous approach assigned ONE indication per patient (most recent GP dx), ignoring which drugs actually treat which conditions. This produced misleading pathways.
    14ÔåÆ
    15ÔåÆ### Use DimSearchTerm.csv for drug-to-Search_Term mapping
    16ÔåÆ- **When**: Determining which Search_Term a drug belongs to
    17ÔåÆ- **Rule**: Load `data/DimSearchTerm.csv`. The `CleanedDrugName` column has pipe-separated drug name fragments. Match HCD drug names against these fragments using substring matching (case-insensitive).
    18ÔåÆ- **Why**: This CSV is the authoritative mapping of which drugs are used for which clinical indications.
    19ÔåÆ
    20ÔåÆ### Use substring matching for drug fragments
    21ÔåÆ- **When**: Matching HCD drug names against DimSearchTerm CleanedDrugName fragments
    22ÔåÆ- **Rule**: Check if any fragment from DimSearchTerm is a SUBSTRING of the HCD drug name (case-insensitive). E.g., "PEGYLATED" should match "PEGYLATED LIPOSOMAL DOXORUBICIN".
    23ÔåÆ- **Why**: DimSearchTerm contains both full drug names (ADALIMUMAB) and partial fragments (PEGYLATED, INHALED). Exact match would miss the partial ones.
    24ÔåÆ
    25ÔåÆ### Modified UPID uses pipe delimiter
    26ÔåÆ- **When**: Creating indication-aware UPIDs
    27ÔåÆ- **Rule**: Format is `{original_UPID}|{search_term}`. Use pipe `|` as delimiter. Do NOT use ` - ` (hyphen with spaces) as that's used for pathway hierarchy levels in the `ids` column.
    28ÔåÆ- **Why**: The `ids` column uses " - " to separate hierarchy levels (e.g., "N&WICS - NNUH - rheumatoid arthritis - ADALIMUMAB"). Using the same delimiter in UPIDs would break hierarchy parsing.
    29ÔåÆ
    30ÔåÆ### Return ALL GP matches per patient, not just most recent
    31ÔåÆ- **When**: Querying Snowflake for patient GP diagnoses
    32ÔåÆ- **Rule**: Remove `QUALIFY ROW_NUMBER() OVER (PARTITION BY ... ORDER BY EventDateTime DESC) = 1`. Return ALL matching Search_Terms per patient with `GROUP BY + COUNT(*)` for code_frequency.
    33ÔåÆ- **Why**: A patient may have GP diagnoses for both rheumatoid arthritis AND asthma. We need ALL matches to cross-reference with their drugs.
    34ÔåÆ
    35ÔåÆ### Restrict GP code lookup to HCD data window
    36ÔåÆ- **When**: Building the WHERE clause for the GP record query
    37ÔåÆ- **Rule**: Add `AND pc."EventDateTime" >= :earliest_hcd_date` where `earliest_hcd_date` is `MIN(Intervention Date)` from the HCD DataFrame. Pass this as a parameter to `get_patient_indication_groups()`.
    38ÔåÆ- **Why**: Old GP codes from years before treatment started add noise. A diagnosis coded 10 years ago may no longer be relevant. Restricting to the HCD window ensures code_frequency reflects recent clinical activity for the conditions being actively treated.
    39ÔåÆ
    40ÔåÆ### Tiebreaker: highest GP code frequency when a drug matches multiple indications
    41ÔåÆ- **When**: A single drug maps to multiple Search_Terms AND the patient has GP dx for multiple
    42ÔåÆ- **Rule**: Use `code_frequency` (COUNT of matching SNOMED codes per Search_Term per patient) from the GP query. The Search_Term with the most matching codes in the patient's GP record wins. If tied, use alphabetical Search_Term for determinism.
    43ÔåÆ- **Why**: E.g., ADALIMUMAB is listed under rheumatoid arthritis, crohn's disease, psoriatic arthritis, etc. A patient with 47 RA codes and 2 crohn's codes is almost certainly on ADALIMUMAB for RA. Frequency of GP coding is a much stronger signal of clinical intent than recency ÔÇö a recent one-off asthma check doesn't mean ADALIMUMAB is for asthma.
    44ÔåÆ
    45ÔåÆ### Same patient, different indications = separate modified UPIDs
    46ÔåÆ- **When**: A patient's drugs map to different Search_Terms
    47ÔåÆ- **Rule**: Create separate modified UPIDs for each indication. E.g., `RMV12345|rheumatoid arthritis` and `RMV12345|asthma`. These are treated as separate "patients" by the pathway analyzer.
    48ÔåÆ- **Why**: This is the core design ÔÇö drugs for different indications should create separate treatment pathways, even for the same physical patient.
    49ÔåÆ
    50ÔåÆ### Fallback to directory for unmatched drugs
    51ÔåÆ- **When**: A drug doesn't match any Search_Term OR the patient has no GP dx for any of the drug's Search_Terms
    52ÔåÆ- **Rule**: Use fallback format: `{UPID}|{Directory} (no GP dx)`. The indication_df maps this to `"{Directory} (no GP dx)"`.
    53ÔåÆ- **Why**: Maintains consistent behavior with the previous approach for patients/drugs without GP diagnosis matches.
    54ÔåÆ
    55ÔåÆ### Merge asthma Search_Terms but keep urticaria separate
    56ÔåÆ- **When**: Working with asthma-related Search_Terms from CLUSTER_MAPPING_SQL or DimSearchTerm.csv
    57ÔåÆ- **Rule**: Merge "allergic asthma", "asthma", and "severe persistent allergic asthma" into a single "asthma" Search_Term. Keep "urticaria" as a separate Search_Term ÔÇö do NOT merge it with asthma.
    58ÔåÆ- **Why**: These are clinically the same condition at different severity levels. Splitting them fragments the data. Urticaria is a distinct dermatological condition that happens to share OMALIZUMAB.
    59ÔåÆ
    60ÔåÆ### Don't modify directory chart processing
    61ÔåÆ- **When**: Making changes to the indication matching logic
    62ÔåÆ- **Rule**: Only modify the indication chart path (`elif current_chart_type == "indication":`). Directory charts use unmodified UPIDs and directory-based grouping.
    63ÔåÆ- **Why**: Directory charts work correctly and should not be affected by indication matching changes.
    64ÔåÆ
    65ÔåÆ---
    66ÔåÆ
    67ÔåÆ## Snowflake Query Guardrails
    68ÔåÆ
    69ÔåÆ### Use PseudoNHSNoLinked for GP record matching
    70ÔåÆ- **When**: Querying GP records (PrimaryCareClinicalCoding) for patient diagnoses
    71ÔåÆ- **Rule**: Use `PseudoNHSNoLinked` column from HCD data, NOT `PersonKey` (LocalPatientID)
    72ÔåÆ- **Why**: PersonKey is provider-specific local ID. Only PseudoNHSNoLinked matches PatientPseudonym in GP records.
    73ÔåÆ
    74ÔåÆ### Embed cluster query as CTE in Snowflake
    75ÔåÆ- **When**: Looking up patient indications during data refresh
    76ÔåÆ- **Rule**: Use the `CLUSTER_MAPPING_SQL` content as a WITH clause in the patient lookup query
    77ÔåÆ- **Why**: This ensures we always use the complete cluster mapping and don't need local storage
    78ÔåÆ
    79ÔåÆ### Quote mixed-case column aliases in Snowflake SQL
    80ÔåÆ- **When**: Writing SELECT queries that return results to Python code
    81ÔåÆ- **Rule**: Use `AS "ColumnName"` (quoted) for any column alias you'll access by name in Python
    82ÔåÆ- **Why**: Snowflake uppercases unquoted identifiers. `SELECT foo AS Search_Term` returns `SEARCH_TERM`, so `row.get('Search_Term')` returns None. Fix: `SELECT foo AS "Search_Term"`
    83ÔåÆ
    84ÔåÆ### Build indication_df from all unique UPIDs, not PseudoNHSNoLinked
    85ÔåÆ- **When**: Creating the indication mapping DataFrame for pathway processing
    86ÔåÆ- **Rule**: Use `df.drop_duplicates(subset=['UPID'])` not `drop_duplicates(subset=['PseudoNHSNoLinked'])`
    87ÔåÆ- **Why**: A patient visiting multiple providers has multiple UPIDs. Using unique PseudoNHSNoLinked only maps one UPID per patient, leaving others as NaN.
    88ÔåÆ
    89ÔåÆ---
    90ÔåÆ
    91ÔåÆ## Data Processing Guardrails
    92ÔåÆ
    93ÔåÆ### Copy DataFrames in functions that modify columns
    94ÔåÆ- **When**: Writing functions like `prepare_data()` that modify DataFrame columns
    95ÔåÆ- **Rule**: Always `df = df.copy()` at the start of any function that modifies column values on the input DataFrame
    96ÔåÆ- **Why**: `prepare_data()` mapped Provider Code ÔåÆ Name in-place. When called multiple times on the same DataFrame, only the first call worked. The fix: `df.copy()` prevents destructive mutation.
    97ÔåÆ
    98ÔåÆ### Include chart_type in UNIQUE constraints for pathway_nodes
    99ÔåÆ- **When**: Creating or modifying the pathway_nodes table schema
   100ÔåÆ- **Rule**: The UNIQUE constraint MUST include `chart_type`: `UNIQUE(date_filter_id, chart_type, ids)`
   101ÔåÆ- **Why**: Without `chart_type`, `INSERT OR REPLACE` silently overwrites directory chart nodes when indication chart nodes are inserted.
   102ÔåÆ
   103ÔåÆ### Handle NaN in Directory when building fallback labels
   104ÔåÆ- **When**: Creating fallback indication labels for patients without GP diagnosis match
   105ÔåÆ- **Rule**: Check `pd.notna(directory)` before concatenating to string. Use `"UNKNOWN (no GP dx)"` for NaN cases.
   106ÔåÆ- **Why**: NaN handling prevents TypeError and ensures meaningful fallback labels.
   107ÔåÆ
   108ÔåÆ### Use parameterized queries for SQLite
   109ÔåÆ- **When**: Building WHERE clauses with user-selected filters
   110ÔåÆ- **Rule**: Use `?` placeholders and pass params tuple ÔÇö never string interpolation
   111ÔåÆ- **Why**: Prevents SQL injection and handles special characters in drug/directory names
   112ÔåÆ
   113ÔåÆ### Use existing pathway_analyzer functions
   114ÔåÆ- **When**: Processing pathway data for the icicle chart
   115ÔåÆ- **Rule**: Reuse functions from `analysis/pathway_analyzer.py` ÔÇö don't reinvent
   116ÔåÆ- **Why**: The existing code handles edge cases (empty groups, statistics calculation, color mapping)
   117ÔåÆ
   118ÔåÆ---
   119ÔåÆ
   120ÔåÆ## Reflex Guardrails
   121ÔåÆ
   122ÔåÆ### Use .to() methods for Var operations in rx.foreach
   123ÔåÆ- **When**: Working with items inside `rx.foreach` render functions
   124ÔåÆ- **Rule**: Use `item.to(int)` for numeric comparisons, `item.to_string()` for text operations
   125ÔåÆ- **Why**: Items from rx.foreach are Var objects, not plain Python values.
   126ÔåÆ
   127ÔåÆ### Use rx.cond for conditional rendering, not Python if
   128ÔåÆ- **When**: Conditionally showing/hiding components or changing styles based on state
   129ÔåÆ- **Rule**: Use `rx.cond(condition, true_component, false_component)` ÔÇö not Python `if`
   130ÔåÆ- **Why**: Python `if` evaluates at definition time; `rx.cond` evaluates reactively at render time
   131ÔåÆ
   132ÔåÆ---
   133ÔåÆ
   134ÔåÆ## Process Guardrails
   135ÔåÆ
   136ÔåÆ### One task per iteration
   137ÔåÆ- **When**: Temptation to do additional tasks after completing the current one
   138ÔåÆ- **Rule**: Complete ONE task, validate it, commit it, update progress, then stop
   139ÔåÆ- **Why**: Multiple tasks increase error risk and make failures harder to diagnose
   140ÔåÆ
   141ÔåÆ### Never mark complete without validation
   142ÔåÆ- **When**: Task feels "done" but hasn't been tested
   143ÔåÆ- **Rule**: All validation tiers must pass before marking `[x]`
   144ÔåÆ- **Why**: "Feels done" is not "is done"
   145ÔåÆ
   146ÔåÆ### Write explicit handoff notes
   147ÔåÆ- **When**: Every iteration, before stopping
   148ÔåÆ- **Rule**: The "Next iteration should" section must contain specific, actionable guidance
   149ÔåÆ- **Why**: The next iteration has zero memory. If you don't write it down, it's lost.
   150ÔåÆ
   151ÔåÆ### Check existing code for patterns
   152ÔåÆ- **When**: Unsure how to implement something
   153ÔåÆ- **Rule**: Look at `pathways_app/pathways_app.py`, `analysis/pathway_analyzer.py`, `cli/refresh_pathways.py`
   154ÔåÆ- **Why**: The existing codebase has solved many quirks already
   155ÔåÆ
   156ÔåÆ<!--
   157ÔåÆADD NEW GUARDRAILS BELOW as failures are observed during the loop.
   158ÔåÆ
   159ÔåÆFormat:
   160ÔåÆ### [Short descriptive name]
   161ÔåÆ- **When**: What situation triggers this guardrail?
   162ÔåÆ- **Rule**: What must you do (or not do)?
   163ÔåÆ- **Why**: What failure prompted adding this guardrail?
   164ÔåÆ-->
   165ÔåÆ

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# Guardrails

Known failure patterns. Read EVERY iteration. Follow ALL of these rules.
If you discover a new failure pattern during your work, add it to this file.

---

## Drug-Indication Matching Guardrails

### Match drugs to indications, not just patients to indications
- **When**: Building the indication mapping for pathway charts
- **Rule**: Each drug must be validated against BOTH the patient's GP diagnoses AND the drug-to-indication mapping from DimSearchTerm.csv. A patient being diagnosed with rheumatoid arthritis does NOT mean all their drugs are for rheumatoid arthritis.
- **Why**: The previous approach assigned ONE indication per patient (most recent GP dx), ignoring which drugs actually treat which conditions. This produced misleading pathways.

### Use DimSearchTerm.csv for drug-to-Search_Term mapping
- **When**: Determining which Search_Term a drug belongs to
- **Rule**: Load `data/DimSearchTerm.csv`. The `CleanedDrugName` column has pipe-separated drug name fragments. Match HCD drug names against these fragments using substring matching (case-insensitive).
- **Why**: This CSV is the authoritative mapping of which drugs are used for which clinical indications.

### Use substring matching for drug fragments
- **When**: Matching HCD drug names against DimSearchTerm CleanedDrugName fragments
- **Rule**: Check if any fragment from DimSearchTerm is a SUBSTRING of the HCD drug name (case-insensitive). E.g., "PEGYLATED" should match "PEGYLATED LIPOSOMAL DOXORUBICIN".
- **Why**: DimSearchTerm contains both full drug names (ADALIMUMAB) and partial fragments (PEGYLATED, INHALED). Exact match would miss the partial ones.

### Modified UPID uses pipe delimiter
- **When**: Creating indication-aware UPIDs
- **Rule**: Format is `{original_UPID}|{search_term}`. Use pipe `|` as delimiter. Do NOT use ` - ` (hyphen with spaces) as that's used for pathway hierarchy levels in the `ids` column.
- **Why**: The `ids` column uses " - " to separate hierarchy levels (e.g., "N&WICS - NNUH - rheumatoid arthritis - ADALIMUMAB"). Using the same delimiter in UPIDs would break hierarchy parsing.

### Return ALL GP matches per patient, not just most recent
- **When**: Querying Snowflake for patient GP diagnoses
- **Rule**: Remove `QUALIFY ROW_NUMBER() OVER (PARTITION BY ... ORDER BY EventDateTime DESC) = 1`. Return ALL matching Search_Terms per patient with `GROUP BY + COUNT(*)` for code_frequency.
- **Why**: A patient may have GP diagnoses for both rheumatoid arthritis AND asthma. We need ALL matches to cross-reference with their drugs.

### Restrict GP code lookup to HCD data window
- **When**: Building the WHERE clause for the GP record query
- **Rule**: Add `AND pc."EventDateTime" >= :earliest_hcd_date` where `earliest_hcd_date` is `MIN(Intervention Date)` from the HCD DataFrame. Pass this as a parameter to `get_patient_indication_groups()`.
- **Why**: Old GP codes from years before treatment started add noise. A diagnosis coded 10 years ago may no longer be relevant. Restricting to the HCD window ensures code_frequency reflects recent clinical activity for the conditions being actively treated.

### Tiebreaker: highest GP code frequency when a drug matches multiple indications
- **When**: A single drug maps to multiple Search_Terms AND the patient has GP dx for multiple
- **Rule**: Use `code_frequency` (COUNT of matching SNOMED codes per Search_Term per patient) from the GP query. The Search_Term with the most matching codes in the patient's GP record wins. If tied, use alphabetical Search_Term for determinism.
- **Why**: E.g., ADALIMUMAB is listed under rheumatoid arthritis, crohn's disease, psoriatic arthritis, etc. A patient with 47 RA codes and 2 crohn's codes is almost certainly on ADALIMUMAB for RA. Frequency of GP coding is a much stronger signal of clinical intent than recency ÔÇö a recent one-off asthma check doesn't mean ADALIMUMAB is for asthma.

### Same patient, different indications = separate modified UPIDs
- **When**: A patient's drugs map to different Search_Terms
- **Rule**: Create separate modified UPIDs for each indication. E.g., `RMV12345|rheumatoid arthritis` and `RMV12345|asthma`. These are treated as separate "patients" by the pathway analyzer.
- **Why**: This is the core design ÔÇö drugs for different indications should create separate treatment pathways, even for the same physical patient.

### Fallback to directory for unmatched drugs
- **When**: A drug doesn't match any Search_Term OR the patient has no GP dx for any of the drug's Search_Terms
- **Rule**: Use fallback format: `{UPID}|{Directory} (no GP dx)`. The indication_df maps this to `"{Directory} (no GP dx)"`.
- **Why**: Maintains consistent behavior with the previous approach for patients/drugs without GP diagnosis matches.

### Merge asthma Search_Terms but keep urticaria separate
- **When**: Working with asthma-related Search_Terms from CLUSTER_MAPPING_SQL or DimSearchTerm.csv
- **Rule**: Merge "allergic asthma", "asthma", and "severe persistent allergic asthma" into a single "asthma" Search_Term. Keep "urticaria" as a separate Search_Term ÔÇö do NOT merge it with asthma.
- **Why**: These are clinically the same condition at different severity levels. Splitting them fragments the data. Urticaria is a distinct dermatological condition that happens to share OMALIZUMAB.

### Don't modify directory chart processing
- **When**: Making changes to the indication matching logic
- **Rule**: Only modify the indication chart path (`elif current_chart_type == "indication":`). Directory charts use unmodified UPIDs and directory-based grouping.
- **Why**: Directory charts work correctly and should not be affected by indication matching changes.

---

## Snowflake Query Guardrails

### Use PseudoNHSNoLinked for GP record matching
- **When**: Querying GP records (PrimaryCareClinicalCoding) for patient diagnoses
- **Rule**: Use `PseudoNHSNoLinked` column from HCD data, NOT `PersonKey` (LocalPatientID)
- **Why**: PersonKey is provider-specific local ID. Only PseudoNHSNoLinked matches PatientPseudonym in GP records.

### Embed cluster query as CTE in Snowflake
- **When**: Looking up patient indications during data refresh
- **Rule**: Use the `CLUSTER_MAPPING_SQL` content as a WITH clause in the patient lookup query
- **Why**: This ensures we always use the complete cluster mapping and don't need local storage

### Quote mixed-case column aliases in Snowflake SQL
- **When**: Writing SELECT queries that return results to Python code
- **Rule**: Use `AS "ColumnName"` (quoted) for any column alias you'll access by name in Python
- **Why**: Snowflake uppercases unquoted identifiers. `SELECT foo AS Search_Term` returns `SEARCH_TERM`, so `row.get('Search_Term')` returns None. Fix: `SELECT foo AS "Search_Term"`

### Build indication_df from all unique UPIDs, not PseudoNHSNoLinked
- **When**: Creating the indication mapping DataFrame for pathway processing
- **Rule**: Use `df.drop_duplicates(subset=['UPID'])` not `drop_duplicates(subset=['PseudoNHSNoLinked'])`
- **Why**: A patient visiting multiple providers has multiple UPIDs. Using unique PseudoNHSNoLinked only maps one UPID per patient, leaving others as NaN.

---

## Data Processing Guardrails

### Copy DataFrames in functions that modify columns
- **When**: Writing functions like `prepare_data()` that modify DataFrame columns
- **Rule**: Always `df = df.copy()` at the start of any function that modifies column values on the input DataFrame
- **Why**: `prepare_data()` mapped Provider Code ÔåÆ Name in-place. When called multiple times on the same DataFrame, only the first call worked. The fix: `df.copy()` prevents destructive mutation.

### Include chart_type in UNIQUE constraints for pathway_nodes
- **When**: Creating or modifying the pathway_nodes table schema
- **Rule**: The UNIQUE constraint MUST include `chart_type`: `UNIQUE(date_filter_id, chart_type, ids)`
- **Why**: Without `chart_type`, `INSERT OR REPLACE` silently overwrites directory chart nodes when indication chart nodes are inserted.

### Handle NaN in Directory when building fallback labels
- **When**: Creating fallback indication labels for patients without GP diagnosis match
- **Rule**: Check `pd.notna(directory)` before concatenating to string. Use `"UNKNOWN (no GP dx)"` for NaN cases.
- **Why**: NaN handling prevents TypeError and ensures meaningful fallback labels.

### Use parameterized queries for SQLite
- **When**: Building WHERE clauses with user-selected filters
- **Rule**: Use `?` placeholders and pass params tuple ÔÇö never string interpolation
- **Why**: Prevents SQL injection and handles special characters in drug/directory names

### Use existing pathway_analyzer functions
- **When**: Processing pathway data for the icicle chart
- **Rule**: Reuse functions from `analysis/pathway_analyzer.py` ÔÇö don't reinvent
- **Why**: The existing code handles edge cases (empty groups, statistics calculation, color mapping)

---

## Reflex Guardrails

### Use .to() methods for Var operations in rx.foreach
- **When**: Working with items inside `rx.foreach` render functions
- **Rule**: Use `item.to(int)` for numeric comparisons, `item.to_string()` for text operations
- **Why**: Items from rx.foreach are Var objects, not plain Python values.

### Use rx.cond for conditional rendering, not Python if
- **When**: Conditionally showing/hiding components or changing styles based on state
- **Rule**: Use `rx.cond(condition, true_component, false_component)` ÔÇö not Python `if`
- **Why**: Python `if` evaluates at definition time; `rx.cond` evaluates reactively at render time

---

## Process Guardrails

### One task per iteration
- **When**: Temptation to do additional tasks after completing the current one
- **Rule**: Complete ONE task, validate it, commit it, update progress, then stop
- **Why**: Multiple tasks increase error risk and make failures harder to diagnose

### Never mark complete without validation
- **When**: Task feels "done" but hasn't been tested
- **Rule**: All validation tiers must pass before marking `[x]`
- **Why**: "Feels done" is not "is done"

### Write explicit handoff notes
- **When**: Every iteration, before stopping
- **Rule**: The "Next iteration should" section must contain specific, actionable guidance
- **Why**: The next iteration has zero memory. If you don't write it down, it's lost.

### Check existing code for patterns
- **When**: Unsure how to implement something
- **Rule**: Look at `pathways_app/pathways_app.py`, `analysis/pathway_analyzer.py`, `cli/refresh_pathways.py`
- **Why**: The existing codebase has solved many quirks already

<!--
ADD NEW GUARDRAILS BELOW as failures are observed during the loop.

Format:
### [Short descriptive name]
- **When**: What situation triggers this guardrail?
- **Rule**: What must you do (or not do)?
- **Why**: What failure prompted adding this guardrail?
-->


Now let me check recent git history and read the current state of the key file I'll need to modify.


Bash

cd "C:\Users\charlwoodand\Ralph local\Tasks\UI Redesign" && git log --oneline -5

Show recent git commits


c85aae4 docs: update progress.txt with Iteration 1 results (Task 1.2)
0779df7 feat: add drug-to-indication mapping from DimSearchTerm.csv (Task 1.2)
1c4d2c0 docs: mark project complete - all tasks done, viewport testing blocked by env (Iteration 9)
fed9094 docs: update CLAUDE.md with indication chart architecture and CLI docs (Task 5.2)
4884e0a fix: recreate pathway_nodes with correct UNIQUE constraint and validate end-to-end (Task 5.1)