Chapter 2: Start with the analysis plan¶
In this chapter you will learn to:
- Name the registry's first product: a descriptive paper, a trial-ready cohort, a data set an industry partner can evaluate, or something else you can name.
- Work backward from the product to the fields, sources, and methods it requires.
- Separate what is central to families from what researchers say they need, and tie each to its user.
- Use AI to organize and to find, and a conversation to learn what is not yet published.
Case study: A Caregiver's List and a Researcher's List¶
As Charlie's mother, I know what is central to me: sleep and behavior, and the seizures themselves, documented over time. I have noticed a great many curiosities over those years, patterns and novelties that interest me as a caregiver, and I am interested in what it is like to care for someone with his condition, because sleep and behavior affect me personally every day. If I had built a registry from what was central to me, those would have been its first modules.
When I talk with the research community, it is clear that these are not always the things they need in order to solve the problems that stand between them and a treatment for Charlie. They ask about his diet. They ask about the effect of interventions we have tried. They ask whether he would wear a device so that they could observe him over time instead of having me fill out a questionnaire about him. More than anything, they ask about biological samples. I know researchers who would be glad to have a blood draw from him once a year, so that they could run the omics on it, together with a small amount of information about his phenotype, and nothing else. For some research questions, that is all they would need.
Neither list is wrong. It is fine, and in my view important, to collect what is central to families; a registry that ignores caregiving, sleep, and behavior ignores the people it serves. The mistake I have seen most often in registries built by families is to assume that what is central to a parent is equally central to the researchers who will use the data, and to find out otherwise after the data have been collected. The way to find out beforehand is to decide what the registry's first product will be and to work backward from it. The first product is not always a paper. For many of us it is a cohort that a sponsor could recruit from, or a partnership with a biotech or a drug company, because developing a therapy for our children is why we got into this. Whatever it is, naming it is what turns a list of interesting questions into an analysis plan.
Lessons for registry leaders¶
What is central to a caregiver and what a researcher needs in order to move toward a treatment overlap less than a new registry leader expects, and both belong in the registry once each is tied to the user it serves. Deciding the first product, and planning the data backward from it, is how the two lists get sorted before collection begins rather than after.
Plan backward from the first product¶
Think of the coffee shop that wants to add a pastry case. Serving coffee and knowing the regulars is one business; selling baked goods is a different one, with its own decisions. Will the pastries be made in house or brought in from a supplier? What display, refrigeration, and licensing does the case need? Which suppliers have to be in place, and by when? If the case is to open on a chosen date, the months before it are spent lining up the equipment, the suppliers, and the recipes that get the shop to that date. Many patient advocacy groups are at the coffee stage: they serve their community well, they provide services, and they know their families. A registry is the pastry case. It is a different kind of work, and it is planned the way any business plans a new line: name the product, set the date, and work backward through everything the product requires.
In research, the product is what the registry will produce for someone else to use. The most common first products are a publication that describes the population, a cohort that is ready to recruit from when a trial opens, and a data set that an industry partner can evaluate. Each one has requirements that can be set out before the first form exists. A descriptive paper needs the rows of its Table 1 (Chapter 1), the outcome it will report, and a defensible denominator. A trial-ready cohort needs a confirmed diagnosis at the tier a sponsor will accept (Chapter 8), current contact details, consent to recontact, and the eligibility fields the likely protocols will screen on. An industry partnership needs the data dictionary, the governance, and the sample and consent provisions that a company's legal and scientific reviewers will ask about before they engage. The Following the data guide follows each type of registry data, from health records and surveys to EEG, imaging, biological samples, and wearables, through consent, collection, standardization, storage, linkage, analysis, and sharing; once the product is named, it shows what each data type the product needs will require along the way.
You do not have to know the answers, because collecting the data is how the answers arrive. You do have to know the questions. If sleep is central to you, the plan says that the registry will report on sleep in children with the condition, and then asks how sleep would need to be collected for that report to hold up: with which instrument, at what frequency, answered by whom, and against what the literature already reports. That question often sends you into the literature to see how others have measured it, and sometimes to a researcher who has. The evidence question from Chapter 8 applies here in advance: what will each measure need to stand on, given the use you have named for it?
Some registries and repositories are meant to be broad on purpose, serving many studies rather than one. Breadth is not a reason to skip the plan. The programs that do breadth well plan for the broadest application they can foresee, with standard elements, broad consent, and modular design, and the plan is more important when the ambition is to serve many users, because it is the only way to avoid serving none of them well. A registry can also decide not to be all things to all people, and that decision belongs in the plan as well.
Talk to the researchers before you finish the plan¶
The plan is drafted in conversation. An AI tool can organize your thoughts about what different kinds of researchers may need, suggest which fields they tend to use, and find researchers publishing on your condition or on a related mechanism. That is a good use of it. What it cannot do is tell you what a particular researcher is hoping to study next, because that hypothesis is not published yet. AI reports what is shared openly; the questions that would shape your registry are often still forming in a researcher's mind, or in a parent's, and reach you only in conversation. Sit down with the researchers who might one day use the data and ask what they would need to see, what they would be glad to have, and what they would not need at all. Record the answers next to what is central to families, and let the plan hold both, each tied to the user it serves.
The rules¶
- Decide what the first product is before you choose a field: a descriptive paper, a trial-ready cohort, a data set an industry partner can evaluate, or something else you can name. If the registry is meant to be broad, name the broadest applications it is planned for.
- Work backward from the product to the plan. List what the product requires: the population, the Table 1 rows, the outcomes, the comparisons, the samples, the consent, and the governance. Each requirement becomes a line in the analysis plan, and then a field, a source, and a method in the registry.
- Keep two lists and tie each to its user: what is central to families and caregivers, and what researchers say they need in order to move toward a treatment. Collect both where you can, and do not assume the lists are the same.
- Ask the researchers. Use AI to organize what different kinds of researchers may need and to find them, then have the conversation, because the hypothesis that would shape your registry is often not published yet.
- For each measure the product needs, decide how it will be collected so that the product holds up: the instrument, the frequency, who answers, and the standard the field expects, found by reading how others have reported it.
- Set a date for the first product and plan the months before it, as a business plans a new line, so that the forms, the consent, the suppliers (labs, vendors, platforms), and the people are in place in the right order.
- Keep the analysis plan as a versioned document and revisit it as the registry grows. The plan will change, and the point is to have one to change.
- Let the product settle design conflicts. When two users want different things, the user of the first product decides, as the Table 1 test in Chapter 1 says, and the other user's needs go into the plan for the next product.
What AI can do, and what only a person can do¶
What AI can do¶
Organize what different kinds of researchers may need from a registry like yours; find researchers, and published measures for an outcome you care about; draft the analysis plan skeleton from your notes.
What only a person can do¶
Name the first product; sit down with a researcher and learn what is not yet published; decide what is central to families and what is central to the science, and keep both.
Take-home¶
Decide what the registry will produce first, a paper, a trial-ready cohort, or a partnership, and plan the data backward from it, because what is central to a parent and what a researcher needs are not the same lists, and only a conversation shows where they differ.
Practice¶
Exercise¶
Assume the Lantern Registry's community has decided that its first product is a cohort a sponsor could recruit from when a trial opens. Open the first product and analysis plan page and the synthetic Lantern Registry export.
- List what a trial-ready cohort requires: the confirmed diagnosis at the tier a sponsor would accept, consent to recontact, current contact details, and the eligibility fields a protocol is likely to screen on (age, seizure frequency, current medication, and a recent follow-up).
- For each requirement, mark whether the export holds it, holds it at a standard of evidence that would not satisfy a sponsor, or does not hold it at all.
- Name one measure the families of the Lantern Registry might consider central, such as sleep, and decide where it sits in the plan: in the first product, in a later product, or in a family-facing module collected alongside the product.
Check your work
| Requirement | In the export? | What the plan needs |
|---|---|---|
| Confirmed diagnosis at a sponsor's tier | Held at a lower standard: an upload flag and a variant string, with no record of review by a qualified person | Adjudication by a named, qualified reviewer, recorded with the date |
| Consent to recontact | Not held: the export has no consent fields | A consent version and a recontact flag in the registry, and the consent language itself |
| Current contact details | Not held, and they should not be in an analysis export; they live in the registry under access control | A separate, access-controlled contact table that the coordinator keeps current |
| Age | Held, once impossible dates of birth are queried | A validated date of birth, or age in months at enrollment |
| Seizure frequency | Held for 15 evaluable participants; placeholders, text, and a contradiction set the rest aside | A numeric field with explicit codes for unknown and none, at a stated recall period |
| Current medication | Held as free text with brand names and abbreviations | Ingredient coded to RxNorm, with the dose in its own fields |
| Recent follow-up | Held for most; several are missing, stale, or before enrollment | A follow-up schedule and a reason code for every gap |
Sleep is a reasonable family-facing module, collected with a named instrument at a stated frequency, and it belongs in the plan as its own line tied to the families who asked for it, rather than in the trial-ready product a sponsor will evaluate.
Your workbook page¶
The First product and analysis plan page of the Slow Registry Workbook applies this chapter to your own registry. Fill it in online or print it and work by hand. The AI prompt library has a prompt for this chapter's topic, paired with the step only a person can do.
Check your understanding¶
The registry's families say sleep is the most important thing to study. Should the registry collect sleep data?
Yes, if the registry can collect it well, and the plan says which user it serves. If the first product is a trial-ready cohort, sleep is a family-facing module with its own instrument and frequency, not the spine of the registry. Collecting what is central to families and what researchers need are both legitimate; the mistake is to assume they are the same list.
Why not ask an AI tool to draft the analysis plan from the published literature?
Ask it to draft the skeleton, organize what different kinds of researchers tend to need, and find researchers working on the condition or a related mechanism. The plan still needs the conversation, because the hypothesis that would shape the registry is often not published yet, and a tool that reads what is shared openly cannot report what a researcher is hoping to study next.
A repository is designed to serve many studies rather than one. Does it still need a first product?
It needs a plan for the broadest application it can foresee: standard elements, broad consent, and modular design. Naming the applications it is planned for is what keeps a broad registry from serving none of its users well, and it can also decide which users it will not serve.
Up next: Before you build anything, find out who is already in the space, because the registry you are imagining may already exist. Continue to Chapter 3.