How it works
And, more usefully, where it fails.
The idea
Trial eligibility criteria are prose written for a human to read one trial at a time. Most systems that automate this hand the whole sentence to a language model and ask for a verdict — which means the model is doing the arithmetic, the unit conversion and the calendar maths.
Find studies starts where ClinicalTrials.gov does not: children, women, and men as a first cut, taken from the registry's own sex and age fields, not a keyword search. A trial can sit in more than one lane when the protocol admits more than one. Matching a note still evaluates the compiled predicates.
This does the opposite. Criteria are compiled once, offline, into typed executable predicates. At query time the comparison is ordinary Python: no model runs, so the same note always produces the same answer, and every number can be traced to the sentence it came from.
What it produces
- A date, not a category. A trial blocked only by a washout is not "ineligible" — it opens on a specific day, computed as calendar arithmetic from the last-dose date in the note.
- A citation for every verdict. Click any criterion and the exact sentence it was computed from highlights in the note.
- An abstention when the record is silent. A field that was never recorded is unknown, never negative. The tool says which test would resolve it.
Measured, including where it loses
| Measurement | Result |
|---|---|
| Eligibility criteria that compile to an executable predicate | 60.3% of 13,202 criteria |
| Errors on 32 held-out numeric criteria, written after the system was frozen | 0 |
| Language-model calls per query | 0 |
| Determinism: distinct outputs over 20 identical runs | 1 |
| Verdicts that resolve to a highlightable source sentence | 75.6% |
| Ranking on the TREC 2022 benchmark (NDCG@10) | 0.36, against TrialGPT's published 0.73 |
That last row is the honest weak point and it is here on purpose. Those benchmark notes are roughly 600 characters with no lab values and no dates, so not one washout or lab predicate can fire — the benchmark does not contain the inputs this system runs on. That explains the gap; it does not erase it.
Limits worth knowing before you trust it
- Forty percent of criteria are not evaluated. They stay free text. "Eligible" means "eligible on the criteria that compiled".
- Extraction errors propagate. Mitigated by showing the source span for every field and abstaining rather than guessing, but not eliminated.
- The corpus is a snapshot, taken 14 August 2026. Recruiting status ages badly; every trial links to its live registry record.
- Lung cancer only. Trial density varies several-fold across cancer types, so none of these numbers transfer.
Data source
Recruiting interventional trials for NCI’s 13 common cancers, including the original 500 NSCLC records, from the ClinicalTrials.gov API v2, a service of the US National Library of Medicine, which does not endorse this tool.
Source
The code that computes every verdict on this site, the evaluation harness,
and the hand-read gold set are all public:
github.com/science182/trial-eligibility-compiler.
Every measurement on this page is reproducible with
eval/compile_rate.py and eval/differentiation.py.