Chapter 6: Know who belongs on the data team¶
In this chapter you will learn to:
- Name the core data roles and the specialties a registry pulls in as needed.
- Match an advisor to the question being asked.
- Check a denominator before a percentage leaves the team.
Case study: The Right Expert for a Different Question¶
I once worked with a registry team whose gene was not newly discovered. It had been studied for years in another body system, in research that had nothing to do with epilepsy, and then it was found to cause an epilepsy as well. The parents who started the registry did what seems reasonable: they looked for the expert on the gene, and found the person with the most publications on it. He was not an epilepsy researcher and not a clinician, so he could not tell them which questions were clinically relevant for a child with seizures. They had not had good experiences with neurologists up to that point; that was part of why they wanted a registry, because they felt the existing science was insufficient. So they had no neurologist, no neurological researcher, no one from pediatric or clinical research of any kind.
Their children were very little, and it had been months, perhaps a year, since the diagnosis. Before that they had no background in the kinds of specialists there are, or the difference between basic and clinical research. They did not know what they did not know. A basic scientist can inform your biorepository and point you toward the preclinical questions that would interest the field, but may not be a data analyst, may not know what is clinically relevant, may not be up on the clinical literature, and may know nothing about patient registries. So the group made up its own questions. By the time I got to it, the questionnaires were a huge, messy, unstandardized, uninformed set, and it was out of control. The problem was never effort. It was that nobody on the team could see which expertise was missing. This happens in many conditions, not only this one.
Lessons for registry leaders¶
A publication record on a gene measures expertise in that gene, not in the clinical questions a registry has to answer. A family-led group months into a diagnosis cannot be expected to know which expertise is missing, and identifying it is work an experienced registry advisor or a disease-area consortium can do with the group.
The rules¶
- Name the core data roles and fill them: someone who can download the data, someone who can do basic analytics and present them to the team regularly, and the data manager who owns the dictionary. These people work with the data every week.
- Match the advisor to the question. The person with the most publications on your gene may be a basic scientist who can shape your biorepository and your preclinical questions, and cannot tell you what is clinically relevant, what a registry needs, or how to analyze clinical data. A clinical registry needs a clinical researcher in the condition, and a pediatric registry needs someone from pediatric research.
- A bad experience with clinicians is a reason to find a better clinician advisor, not a reason to have none. Distrust of the existing science is often why a group starts a registry; it cannot be why the registry has no clinical expertise.
- Know the specialties you will pull in and when: statistician or biostatistician, data engineer, software architect, computational biologist, epidemiologist, and an environmental epidemiologist if you are assessing risk factors. Within biostatistics, the people who analyze omics data and the people who analyze clinical data are usually different people. Someone trained only on clinical data is not the person to run your omics analysis, though they can supervise it with the right context, and the reverse is also true.
- Keep advisors on call for what the core team cannot cover, and keep a list of who they are and what they advise on. If you cannot name the expertise you are missing, ask a consortium in your disease area to tell you; not knowing what you do not know is the normal starting condition for a family-led group, and there are people whose job is to close that gap.
- Plan publications ahead: which question, which analyst, which advisor, which cut of the data, and when.
- Whoever reviews a summary checks the denominator. If 100 people completed the survey, 4 answered the section on pregnancy, and 2 of those reported a complication, the complication rate is not 50 percent of the cohort. This is easy to miss without professional training in reading these reports, which is why the reviewer has to be someone familiar with the data.
- Software can produce the summary and so can AI, but neither does the review. If AI drafts the report, a person with the relevant expertise reads it before anyone else does.
- Present the data to the team on a schedule. The people who look at the data every week are the ones who catch what is wrong with them.
What AI can do, and what only a person can do¶
What AI can do¶
Draft the summary report, profile the data, and propose analyses for review.
What only a person can do¶
Know which specialty a question needs; check the denominator; decide what gets published and by whom.
Take-home¶
A registry needs a core group that works with the data every week and a named list of the expertise it will pull in, and no summary leaves the team without someone qualified reading it.
Practice¶
Exercise¶
Use the synthetic Lantern Registry export and the data team roles worksheet.
- Report the percentage of participants with developmental delay (
dev_delay) in two ways: first, count only the cells that say exactly "Yes" and divide by every record; then harmonize the codes that mean yes, set aside the records with no usable answer, and divide by the records with a known answer. - Explain which number you would report, what you would say about the records you set aside, and what the denominator would be among the participants with a qualifying genetic report, whom you will identify in Chapter 8.
- Fill in the roles worksheet for your own registry: who downloads the data, who runs basic analytics, who owns the dictionary, and which specialties you would pull in for which questions.
Check your work
Counting only the exact word "Yes" gives 26 of 32 records, or 81 percent. Harmonizing "Yes," "yes," "1," and "TRUE" gives 29 records with yes, 2 with no, and 1 with "N/A," which is not an answer; that is 29 of 31 records with a known answer, or 94 percent. The 32 records also include a duplicate enrollment and participants without a qualifying genetic report, so neither number describes the cohort yet. Among the 25 participants with a qualifying report on file in Chapter 8, every participant with a known answer has developmental delay, and one has an unknown answer that is reported separately.
Whichever number is reported, it rests on decisions someone has to make and record before the percentage is shared.
Your workbook page¶
The Data team roles page of the Slow Registry Workbook applies this chapter to your own registry. Fill it in online or print it and work by hand. The AI prompt library has a prompt for this chapter's topic, paired with the step only a person can do.
Check your understanding¶
A survey went to 100 participants. Four answered the pregnancy section, and two of them reported a complication. What can you report?
That 2 of the 4 participants who answered the pregnancy section reported a complication. You cannot report that 50 percent of the cohort had a complication, and with 4 respondents the finding supports very little.
The most-published expert on your gene is a basic scientist. What can that person help with, and what not?
They can shape your biorepository and the preclinical questions that would interest the field. They are usually not the right advisor on which questions are clinically relevant, how to design a clinical registry, or how to analyze clinical data.
Up next: Once the questions, the coordinator, and the data team are in place, settle the protocol, the dictionary, and the consent before anyone enrolls. Continue to Chapter 7.