Problem
Hospitals and GP systems record the same care in different ways, so a question that crosses organisations is hard to answer. Two standards address this. FHIR is how health systems exchange records, one patient and one event at a time. OMOP is how researchers analyse them, in fixed tables where everyone uses the same codes. The NHS Research Secure Data Environment network plans to use OMOP where it helps with work across sites, so moving data from one to the other is a real and growing job.
There is a second problem behind it. Trusts often can’t pool patient records, so each one trains on far fewer patients than it could. Federated learning sends the model to the data instead of the data to the model. Each site trains on its own patients, and only the learned weights travel back to be averaged. It needs every site to store data the same way, which is what OMOP provides.
I built both as one independent project, with no client. Every patient is synthetic, generated by Synthea, so no real patient data is used anywhere. NHS England had no part in it. The questions were simple. Can every record be accounted for on the way into OMOP? And how much does a trust gain from joining a federated model, compared with working alone or pooling everything?
Data
Synthea generated 1,133 patients as FHIR R4 bundles, giving 1,400,153 records in 4.18 GB of JSON. It is rule-based, so its patients are useful for testing a pipeline and useless for clinical findings.
Measuring a first sample of 55 patients changed the plan in three ways.
Nearly half the bytes were US billing. Claim and ExplanationOfBenefit records were 48% of the JSON. OMOP has no use for them here, so they stay in the raw table and are marked as left out, with the reason written down.
About three in ten prescriptions don’t name their drug. They point to a separate Medication record. Reading only the obvious field would have silently lost them, so staging resolves both shapes into the same columns, and a test fails if any prescription ends up without a drug.
Readmission was not a usable target. That was the first idea for the model. In the sample there were 37 admissions spread over 75 years, and only 1 was followed by another within 30 days. Emergency visits happen to about 13% of patients a year, which gave a target worth predicting.
Architecture
- SyntheaFHIR R4 bundles
- Rawone row per record, as published
- VocabulariesAthena, read from the zip
- Stagingtyped tables, dbt
- Code mapsource code to standard concept
- OMOP 5.4routed by concept domain
- FeaturesOMOP tables only
- Flowerfive simulated trusts
Raw stays exactly as published. Each bundle entry becomes one row of JSON. A file is replaced in one transaction, so a rerun gives the same table, and a file only counts as loaded when the row count in Postgres matches the count in the file. All 1,400,153 records arrived.
Read the vocabularies from the zip. The Athena download is 4.9 GB unzipped, and the machine had 2 GB of disk free. Most of the download covers vocabularies this data never uses, such as NDC and SPL. The loader streams straight from the zip and keeps only what is needed, which is 3.9 million concepts and 3.1 million mappings, so nothing is ever unpacked to disk. Rows read and rows kept are both recorded, so the filtering can be checked and isn’t just assumed.
Everything is rebuilt by dbt. Models run from staging through a code map to the OMOP tables. A test compares each output table with the official OMOP 5.4 definition, column by column, in order, and checks that required columns have no gaps.
Translating the records
Each source code becomes the standard concept it maps to. A code with no standard concept isn’t dropped. It takes concept 0 and keeps its original code, so the fact survives.
Show as table
| Item | Value |
|---|---|
| Observation | 99.96% |
| Procedure | 99.46% |
| MedicationRequest | 100.00% |
| Condition | 99.58% |
| Immunization | 100.00% |
| AllergyIntolerance | 100.00% |
The harder part is that OMOP places a record by what it is, not by what FHIR called it. A record’s concept has a domain, and the domain decides the table.
| FHIR called it | OMOP table | Rows |
|---|---|---|
| AllergyIntolerance | observation | 1,212 |
| Condition | observation | 21,523 |
| Condition | condition_occurrence | 19,259 |
| Immunization | drug_exposure | 16,523 |
| MedicationRequest | drug_exposure | 49,554 |
| Observation | measurement | 505,452 |
| Observation | observation | 277,999 |
| Procedure | procedure_occurrence | 99,041 |
| Procedure | measurement | 41,924 |
| Procedure | observation | 37,143 |
| Procedure | drug_exposure | 302 |
| Procedure | device_exposure | 87 |
| Procedure | condition_occurrence | 16 |
So 21,523 “conditions” are really observations, such as employment status or “medication review due”. Keeping them as conditions would count a full-time job as a diagnosis. And 302 “procedures” are drugs given, mostly vaccines.
Accounting for every record. Each FHIR resource type is given a decision. It is loaded, used to read another record, or left out with a reason. A new type with no decision fails the build, and so does a loaded type that loses rows.
| Outcome | Records |
|---|---|
| Loaded into OMOP | 898,876 |
| Used to read another record | 16,351 |
| Left out, with a written reason | 484,926 |
| Total, matching the source files | 1,400,153 |
Federated learning
The task. Predict whether a patient will have an emergency visit in the next year, from 7,626 patient-years across 1,015 people. All 34 inputs come from the OMOP tables and from before the prediction date, so the same query would run at any OMOP site. A test checks that last year’s outcome and this year’s look-back count the same visits, because a window that is one day off is how future data leaks in.
Five simulated trusts. Cities were grouped by location into five catchments, keeping each city whole. They range from 60 patients to 434. A second split gives each trust a different age band on purpose, because synthetic regions look more alike than real ones.
One model, three ways. Each method is the same logistic regression with the same training code, so the only difference is where the data sits. Inputs use fixed scaling rather than learned means, because a shared mean is itself information leaving a trust. In the federated runs each trust opens only its own file, using Flower, the framework behind FLIP, the NHS London platform.
| AUROC by how trusts were split | One trust alone | Federated | All data pooled |
|---|---|---|---|
| By region | 0.650 | 0.676 | 0.681 |
| By age, so they differ | 0.650 | 0.681 | 0.681 |
A trust does better by joining. The gain is 0.013 to 0.041 by region and 0.016 to 0.046 by age, with 95% confidence, and all five trusts gained in both splits. It also gives up very little. Federated trails pooling by 0.002 to 0.009 by region, and in the age split the gap is 0.002 or less either way.
Show as table
| Item | Value | Note |
|---|---|---|
| Trust A, 141 patients | 0.698 | alone 0.664 |
| Trust B, 60 patients | 0.788 | alone 0.689 |
| Trust C, 168 patients | 0.619 | alone 0.558 |
| Trust D, 212 patients | 0.648 | alone 0.626 |
| Trust E, 434 patients | 0.681 | alone 0.670 |
The score itself is modest, and it should be. Synthea’s emergency visits are partly random injuries, and a score of 0.68 says nothing clinical. The comparison is the finding.
Choosing how long to train. The first run made federated look clearly worse, at 0.663 by region and 0.651 by age. That came from my stopping at 40 rounds. I tuned all three methods on patients held back from training, and federated needed 320 rounds to catch up, because each round averages small steps from five sites.
Show as table
| Item | Value |
|---|---|
| 10 rounds | 0.573 |
| 20 rounds | 0.633 |
| 40 rounds | 0.663 |
| 80 rounds | 0.673 |
| 160 rounds | 0.675 |
| 320 rounds | 0.676 |
The live console
The FHIR to OMOP Desk has two working parts. One lets you switch the five trusts on and off and see what each gains. Training doesn’t fit on a small server, so all 31 possible groups were trained in advance and the console says so. The other translates a FHIR record into OMOP rows live, with the routing explained. Its code map holds only the codes in this project’s data, not the full vocabularies.
What didn’t work
Allergies became doses. Routing by concept is right almost everywhere, and my first version applied it to allergies too. An allergy’s code names the substance, so 168 allergies landed in drug_exposure, which says the patient took the drug they are allergic to. I found it while checking my own routing table, not from a test. Every allergy now goes to the observation table with the substance as its value, and a test blocks it from coming back.
Stated reasons that weren’t true. I wrote each reason for leaving a record out from what I expected. Checking them against the data showed two were wrong. Half of the DiagnosticReport records are free text notes, not groups of results, and SupplyDelivery records are tied to patients. Both reasons were rewritten. The others held, including all 16,351 MedicationAdministration records matching a prescription for the same drug at the same visit.
Visits after death. 118 visits fell after a patient’s death, which usually means records joined to the wrong person. These are genuine paperwork visits for the death certificate, so they stay, and a test blocks anything else from appearing after death.
Limitations
- Synthea patients follow written rules, so the model learns those rules and not real medicine. Only the comparison between methods carries over, and real trusts may differ more than these.
- The trusts are simulated on one machine, so network failures, slow sites and governance aren’t tested.
- Model weights can still reveal something about training data, and this project adds no protection against that.
- Synthea writes US data, so drugs use RxNorm and vaccines use CVX.
- FHIR gives no end date for a prescription here, so
drug_exposureshows that a drug was prescribed, not how long it was taken. - Provider and care site tables aren’t built, and US billing records and free text notes are left out.
- The test folds were seen once at 40 rounds, before tuning. Training length was then chosen only on held-back patients.
Stack
- FHIR R4
- OMOP CDM 5.4
- dbt
- PostgreSQL
- Python
- Flower
- Docker
- Synthea