Chapter 12: Go and find out why the data are missing

In this chapter you will learn to:

• Profile missing data by field, participant, and characteristic.
• Sort gaps into technical, process, design, clinical documentation, and participant choice.
• Draft a missing data plan that ends in reporting.
• Set a strategy for handling missing data for each use, and write it into the analysis plan.

Case study: The Records That Never Arrived

I was asked to analyze the electronic health record data linked to a registry, using Registry Forge, the free, open-source pipeline I developed for turning patient-directed health record data into research-ready outputs, published in JAMIA Open (Boyce et al., 2026). The registry used two consents: one for the registry and its surveys, and a second that authorized a vendor to retrieve each participant's health records. The registry's contact table listed everyone who had given that second consent. When I compared it against the records that had arrived, only about 60 percent of the people who had authorized the transfer had any records at all.

For participants treated at U.S. medical centers, a gap that size seemed too large to be ordinary, so I assumed I had misunderstood something. I described what I saw to the vendor, copied the study team, and asked whether I was reading it correctly. The vendor's answer was that there was no technical problem: those participants had not followed through. Faced with a complicated system and a confident vendor, the natural response is to accept that explanation.

Some time later, at a research meeting, I heard a participant mention how excited they were to have shared their health records with the study. They believed the transfer had happened. I did not act on who that person was; I took the remark as a reason to look at the whole linkage process more closely. Querying the intermediate tables, beyond the final extract, showed that many transfers had started and then stopped. Most of the failures traced to participants who could not find their health system or their doctor in the vendor's search. Some hospitals were outside the vendor's network, and some searches may have used terms the system did not recognize. These were addressable problems. At least another 20 to 30 percent of the consented participants, probably more, were recoverable, which would take record linkage from about 60 percent toward 95 percent.

The investigation took about a day and a half of working through tables. I brought the evidence to the vendor, and together we made the changes that resolved it. I also checked whether the participants without records differed systematically from those with them, because a gap concentrated in particular hospitals or regions would bias any analysis built on the records that did arrive.

Most failed transfers traced to health systems and doctors that participants could not find in the vendor's search, and at least another 20 to 30 percent of consented participants were recoverable.

Lessons for registry leaders

A gap between what participants authorized and what arrived is a finding to investigate, not a fact about the participants. The vendor's explanation was plausible and incomplete, and the evidence sat in tables beyond the final extract and in one participant's passing remark. Finding it took a person who knew what the data should look like, time set aside to look, and the standing to bring the evidence back to the vendor.

When a blank is not a no

Registry survey data often arrive with a yes and nothing else. A symptom checklist records the boxes a participant ticked, and an unticked box could mean the answer was no, the question did not apply, or the participant never reached it. Clinical notes often handle this better than surveys: a note saying the patient denies seizures, or denies cognitive decline, records that the question was asked and the answer was no. A checklist that keeps only the yeses loses the denominator, the number of people who were asked.

In rare neurodevelopmental conditions, the no is as informative as the yes. Seizures together with intellectual disability can narrow the list of possible genetic causes considerably compared with seizures alone, and a clinician or geneticist reasoning about a syndrome uses both. If I see seizures recorded and the field for intellectual disability is blank, I cannot tell whether the child has intellectual disability, does not, or was never asked, and every conclusion I draw about the underlying condition, or about how this group compares with others, is less certain.

Kinds of missing data

Kind	What it looks like	Where to look
Technical	Consented participants with no records; data that stop at a date; a site or vendor with far less data than the others; transfers started and not finished	Consent and contact tables against what arrived; intermediate and staging tables; the vendor, with your evidence in hand
Process	Participants who could not complete a step: a health system or doctor they could not find in a search, a reminder that never reached them, a survey in a language they do not read, contact details out of date	Where participants stop in the enrollment or linkage flow; the coordinators; the participants themselves
Design	A checklist that keeps only the yeses; skip logic that hides a question; a question added after some participants had enrolled	The data dictionary, the form versions, and the skip logic
Clinical documentation	Something the clinic never recorded, so its absence says nothing; a note that says "denies seizures" is a recorded no, while silence is unknown	Representative records (Chapter 4)
Participant choice	A declined item, a skipped sensitive question, a withdrawal	Response codes for declined and withdrawn

A missing data plan

The plan is a page in the Slow Registry Workbook. It follows the order in which the work is done.

1. Design for it. Before collection, give every item explicit responses for no, not applicable, declined, and unknown. Ask each phenotype as yes, no, or unknown rather than as a checklist. Version every form and record when each question was added.
2. Reconcile expected against received. At every handoff, compare what should exist with what arrived: consents for record linkage against records received, enrollments against completed surveys, samples sent against results returned. Break the comparison down by site, vendor, health system, and month.
3. Profile what is missing. By field, by participant, over time, and by the characteristics you do have, such as country, language, enrollment wave, and form version.
4. Investigate the cause. Query the intermediate tables, not only the final extract. Ask the vendor with your evidence in hand. Ask coordinators and participants where the process broke down. Assign a reason code to each gap.
5. Recover what can be recovered. Fix the technical fault, help participants complete the step they could not, re-send, call, or check an earlier form.
6. Check for bias. Compare participants with and without the data on what you know about both groups. If they differ, the data are not missing at random, and the analysis has to account for it.
7. Decide how missing data will be handled, and write it into the analysis plan. The strategy can differ case by case, because it depends on the use, who is using the data, what they are trying to show, and the research question, and the people who will use the results decide it with a statistician. Complete-case analysis, multiple imputation, and sensitivity analyses each rest on assumptions about why data are missing, which the statistical literature describes as missing completely at random, missing at random, and missing not at random. The investigation in the steps above is what tells you which assumption is defensible. Stef van Buuren's Flexible Imputation of Missing Data is a standard reference on the methods.
8. Report it. State how many participants are missing each variable, why, and how the analysis handled it. The STROBE guidelines (Strengthening the Reporting of Observational Studies in Epidemiology) ask authors to "explain how missing data were addressed" and to "indicate the number of participants with missing data for each variable of interest" (STROBE checklist).

The rules

1. Treat every blank as having a cause, and record the cause when you learn it: declined, unreachable, not applicable, not yet due, not asked, transfer failed.
2. Treat any explanation for a gap as a hypothesis until the data support it, including an explanation from a vendor. Vendors know their systems and the registry knows its participants; the cause is usually found with both.
3. Design every phenotype question so that a no can be recorded. A checklist that keeps only the yeses cannot distinguish absent from unasked.
4. Look for structure before analyzing around it. A pattern by site, vendor, language, country, or enrollment wave is a process problem, not a modeling problem.
5. Fix the process that produced the gap, not only the gap.
6. Budget time for the investigation. Tracing one gap through the tables can take a day or more, and it can recover a large share of a cohort.
7. Report how much is missing, where, why, and how it was handled, in every summary.
8. Find the cause of each gap, and also set a strategy for handling missing data for each use. The strategy belongs in the analysis plan and can differ with the research question, who is using the data, and what they are trying to say.

What AI can do, and what only a person can do

What AI can do

Profile missingness by field, participant, site, and time; draft reconciliation queries once someone has defined what should match; draft outreach messages for you to check.

What only a person can do

Notice that data which should exist are absent; hear what a participant says about their own experience; bring evidence to a vendor and press for a fix; decide whether the reason for a gap changes the question; decide, for each use, how missing data will be handled.

Take-home

Missing data are a relationship and a process; the cause is usually somewhere a person can go and look, and registries that look recover data that others would have written off.

Practice

Exercise

Using the synthetic Lantern Registry export:

1. List every blank or non-answer in the export and the field it sits in.
2. Look for structure. Where do the blanks cluster?
3. For each cluster, write a hypothesis about its cause, the kind of missingness it suggests, and what you would ask the coordinators or the vendor.
4. Draft the first entries of your missing data plan.

Check your work

The follow-up date and seizure count are blank for all three participants in India and one of two in Brazil, and no other country has a blank follow-up. That clustering points to a process cause, such as a follow-up survey offered only in English, a reminder that never reached those families, or a vendor or time zone problem, rather than to chance. Ask the coordinators how follow-up was offered in each country before analyzing around the gap.

Other gaps: weight is blank for LR-028; the confirmation field is blank and the variant pending for LR-011; the variant is blank for LR-027, who has a clinician letter only; developmental delay is "N/A" for LR-015, which could mean not applicable, unknown, or declined. Each needs a reason code, and the plan should record whether each was recovered.

Your workbook page

The Missing data plan page of the Slow Registry Workbook applies this chapter to your own registry. Fill it in online or print it and work by hand. The AI prompt library has a prompt for this chapter's topic, paired with the step only a person can do.

Check your understanding

A vendor says the participants with no health records simply did not follow through. What do you do?

Treat the explanation as a hypothesis. Compare consents against records received, query the intermediate tables, talk to participants and coordinators, and bring the vendor the evidence.

Should every analysis of the registry handle missing data the same way?

Not necessarily. The strategy depends on the use, who is using the data, what they are trying to show, and the research question. Find the cause of each gap first, then decide the handling with a statistician and the people who will use the results, and record it in the analysis plan.

Why ask phenotype questions as yes, no, or unknown rather than as a checklist?

A checklist that keeps only the yeses cannot tell absent from unasked. In rare neurodevelopmental conditions a recorded no is informative, such as seizures without intellectual disability.

Up next: The surest way to have fewer gaps is to be in regular contact before they open. Continue to Chapter 13.