The Hidden Variable in Matching Performance
When EHR-based patient matching underperforms — generating candidate lists with elevated screen failure rates or missing eligible patients who surface later through manual chart review — the standard diagnostic response focuses on the matching algorithm. Is the NLP model handling negation correctly? Are the temporal logic rules properly specified? Is criteria parsing mapping to the right ontology terms?
These are legitimate questions. But in a substantial fraction of matching performance problems, the algorithm is not the primary failure point. The EHR data being fed into it is. An algorithm trained on clean, well-structured data will underperform when deployed against data that is fragmented, inconsistently coded, or missing critical fields — not because the algorithm is wrong, but because the information needed to evaluate eligibility simply isn't there in usable form.
Data quality is the input variable that determines matching output quality, and it varies far more across sites and health systems than most sponsors appreciate when they're designing EHR-based recruitment programs. The therapeutic area, site type, EHR platform, and documentation culture at each site collectively determine data quality profiles that are site-specific and protocol-specific — not generalizable.
Three Dimensions of EHR Data Quality That Affect Matching
Completeness: Is the Relevant Field Populated?
Completeness is the most fundamental quality dimension for matching purposes. If a key eligibility criterion requires a lab value that is not consistently entered in structured fields — because results return via a reference lab that scans PDFs rather than transmitting HL7, or because the relevant specialty uses a separate system not integrated with the main EHR — then matching against that criterion produces unreliable results regardless of algorithm quality.
The structured completeness gap is highly therapeutic-area-specific. Metabolic disease and cardiovascular trials relying on standard lab panels (HbA1c, lipid panels, renal function) generally encounter high completeness at sites with active primary care programs. Oncology trials requiring molecular pathology data — genomic mutation status, biomarker expression levels from immunohistochemistry or FISH — encounter much lower structured completeness. Pathology reports are frequently stored as unstructured PDFs, and structured extraction of genomic data requires custom integration with pathology information systems that most EHR platforms do not natively support.
Accuracy: Are Coded Fields Correctly Coded?
A field that is populated is not necessarily accurate. Systematic coding inaccuracies are well-documented in clinical informatics research and affect matching in predictable ways. Diagnosis codes entered at intake for billing purposes may not be updated when a diagnosis is refined or reversed. Medication records may carry historical entries with "active" status flags that were never deactivated when the medication was discontinued. Problem list entries persist long after the condition resolves. Each creates a category of matching errors: patients who appear eligible based on a current code that actually reflects a historical state, or who appear ineligible because a medication exclusion applies to a drug they stopped taking 18 months ago.
Consistency: Are Similar States Documented the Same Way?
Consistency across providers within a site and across sites within a network is the most insidious dimension, because it produces matching errors that are difficult to detect without auditing. If two subspecialty groups at the same academic medical center document the same clinical state using different ICD-10 codes — one using the specific manifestation code, one using the category code — a query optimized for the specific code will miss patients seen by the second group. If "active treatment" is defined differently by the oncology and hematology practices at the same institution, a matching algorithm that treats both sources as equivalent produces inconsistent eligibility results that appear to be algorithm errors but are documentation culture differences.
What Poor Data Quality Costs: A Quantified View
The cost of data quality failures in matching appears in two places simultaneously. The more visible is elevated screen failure rate. Candidate lists derived from incomplete or inaccurate source data include patients who fail formal screening on criteria that would have excluded them if the data had been complete — each failed screen consuming coordinator time and, in protocols with investigational screening procedures, direct per-screen costs that can reach $3,000-$6,000 per screen in therapeutic areas requiring imaging, biopsies, or specialized laboratory panels.
The less visible cost is missed eligible patients — false negatives where an eligible patient's qualifying signal is buried in unstructured data or absent from structured fields. Missed patients don't appear in screen failure statistics because they were never presented to coordinators. They represent enrollment capacity that wasn't realized, translating directly into timeline extension without any operational signal that the capacity was available.
Consider a scenario at a mid-size academic medical center running a Phase III RCC trial requiring documented PD-L1 expression status. PD-L1 expression was recorded in pathology reports as narrative text, not in any structured field. An EHR query relying on structured fields identified 12 candidate patients over six months. Subsequent unstructured note analysis surfaced 19 additional patients whose PD-L1 expression was documented in pathology reports that the structured query had not accessed. The effective false-negative rate from that single data quality gap was over 60% of the eligible population at that site over that period.
Pre-Deployment Data Quality Assessment: A Practical Framework
Given how materially data quality affects matching performance, pre-deployment assessment should be a standard component of any EHR-based recruitment program — not a post-hoc investigation after screen failure rates come in higher than projected.
A practical pre-deployment assessment covers four areas:
- Structured completeness rate for key eligibility criteria: What percentage of patients with the target diagnosis have the relevant lab values, vital signs, and medication fields populated in structured form? A completeness rate below 60% for a criterion that will drive a significant fraction of eligibility determinations is a signal to design around rather than accept as a baseline.
- Coding practice audit: For the primary inclusion and exclusion diagnoses, what ICD-10 coding practices are in use at this site? Are there common variations or category code substitutions that the matching query needs to accommodate?
- Unstructured data accessibility: Are pathology reports, radiology reports, and subspecialty consult notes accessible via the FHIR API or data extract pathway, and in what format? A site where key clinical information lives in inaccessible unstructured documents requires an NLP-based matching strategy supplementing or replacing the structured field approach.
- Historical data retention depth: How many years of patient history are accessible via the API? Some eligibility criteria require multi-year lookback — if the accessible history only covers three years but a criterion requires "no prior malignancy in 5 years," the effective lookback window creates a systematic eligibility assessment gap.
Sites with poor completeness or accuracy are not automatically disqualified from an EHR matching program, but they require a different matching strategy: more emphasis on NLP extraction from unstructured sources, modified query logic to accommodate coding variations, and explicit handling of missing data (treating absence of a lab value differently from presence of an out-of-range value).
Building a Feedback Loop Between Matching Outcomes and Data Quality
Data quality is not static. EHR documentation practices improve when specific, actionable feedback is provided to site coordinators and clinical informatics teams. A matching system that identifies "lab value X is missing in 72% of candidate records at Site A" is providing information that is directly actionable through coordinator training, structured documentation workflow prompts, or targeted retrospective data entry.
A well-designed recruitment platform closes the loop between matching outcomes and data quality signals. When candidates fail formal screening, the analysis should distinguish between: failures that were predictable from data available in the EHR at the time of matching (suggesting a matching algorithm error), failures where the relevant exclusion data was genuinely absent or inaccurate in the EHR (suggesting a data quality issue), and failures due to criteria that genuinely cannot be evaluated from EHR data alone (suggesting a pre-screening limitation that should be communicated to sites rather than treated as a matching failure).
Sites that show the most improvement on enrollment metrics over the course of a deployment typically are not the sites with the best baseline data quality — they are the sites that respond to data quality feedback and improve their documentation practices. The improvement in data quality translates directly into improvement in matching accuracy, which translates into lower screen failure rates and faster enrollment velocity. The feedback loop is where the operational compounding of an EHR-based matching program actually lives.
