Our evaluation was designed using common program evaluation frameworks (10, 11). To gather evidence about the outcomes of the HRSA-funded programs, we conducted semi-structured interviews or focus groups with representatives of the NBS system. Representatives included HRSA-funded NBS program grantees, state NBS program staff, parents of children with a condition identified through NBS, patient advocates, healthcare professionals, and NBS researchers and subject matter experts. Interview and focus group questions were organized around the six HRSA-supported program activities outlined in the federal legislation as well as health equity and outcomes, and tailored to each group. For example, parent interviews focused more on NBS education, whereas subject matter experts were probed about topics such as the timeliness of NBS. We have included the state public health department guide in Supplementary material as example. All evaluation activities were determined to be Not Human Subjects Research by the (blinded for review) IRB. Nonetheless, written informed consent was obtained from all participants prior to data collection and was reviewed verbally at the start the interview or focus group.

Prospective participants were identified through a combination of review of the NBS literature, patient advocacy organizations’ websites for conditions on the RUSP, known subject matter experts, and input from HRSA. Although we based recruitment on a convenience sample, we wanted to recruit a diverse group of participants. For example, we recruited state NBS program staff from different regions of the country, including smaller and larger states, as well as laboratory and follow-up staff. Healthcare providers represented a variety of specialty areas, such as genetics, pediatrics or primary care, and neurology. An assortment of RUSP conditions were represented in parent and patient advocate interviewees, and parents were from different states. Because recruitment was based on familiarity with and expertise in NBS, we did not collect demographic information on participants, apart from state of residence when applicable, or use demographic information to guide sampling.

We initially invited 54 individuals to participate, with a goal of including 9 participants per group. If an invited individual did not respond after the first email, we sent up to two follow-up emails. If the individual did not respond or declined participation, we invited another individual representing the same group. In total, we invited 67 individuals and 52 consented to participate, including: (a) NBS program grantees (n = 9), (b) state NBS program staff (n = 8), (c), parents of children with a condition identified through NBS (n = 9), (d) patient advocates (n = 8), (e) healthcare professionals (n = 9), and (f) NBS researchers and subject matter experts (n = 9). In total, participants represented 3 research or professional organizations, 8 national patient advocacy organizations, and 24 NBS staff, parents, clinicians, and NBS researchers or subject matter experts from 24 states (AK, AL, AZ, CA, CO, CT, FL, GA, IA, IL, KS, MD, MI, MN, NC, NY, OH, OK, PR, RI, TN, UT, VA, and WI).

Program grantees, state public health departments, and NBS researchers and subject matter experts were asked to participate in interviews. Parents, patient advocates, and healthcare providers were asked to participate in focus groups. Each focus group was organized and conducted by type of NBS representative. However, if an individual was unavailable to participate in a focus group, we offered an interview instead. In total, 21 individuals participated in one of six focus groups lasting 90 min, and the remaining individuals participated in interviews lasting from 60 to 90 min. Participants were offered a gift card in appreciation for their time.

With permission from participants, each interview or focus group was recorded and transcribed. We used rapid turnaround analysis, which is commonly used in health services and evaluation research when there is a need to triangulate findings from multiple qualitative and quantitative data sources and quickly obtain insights needed to make timely decisions regarding strategy and practice (12). Rapid turnaround analysis is ideal for studies in which a semi-structured guide is used to obtain qualitative data on pre-determined constructs of interest (13), as was the case in this evaluation.

A pair of analysts was assigned to conduct all data collection and analysis for a given participant group. Notes were taken into a structured template that aligned with the interview or moderator guide. Transcripts were reviewed to fill in gaps in notes and extract illustrative quotations. Analyst pairs then manually coded notes to tag segments that addressed the HRSA-supported program activities. We used a collaborative qualitative analysis approach (14); for each set of notes, one analyst conducted initial coding, and the second analyst reviewed the proposed coding. The pair then met to discuss any questions or disagreements and come to consensus on the final coding. Coded segments of notes were transferred to corresponding sections of a matrix for each group. Analyst pairs then created summaries across participants within their group for each program activity. The matrices helped analysts quickly and systematically investigate similarities and differences in responses across participants and data sources, (i.e., facilitating comparison within and across groups) (15, 16). Themes from the matrices were integrated into an overall summary for each program activity.

Participants agreed that progress has been made to enhance, improve, or expand the ability of state NBS programs to screen newborns and conduct follow-up. Various federally funded programs have helped create a more efficient and proficient NBS system, with most emphasis placed on supporting labs and screening techniques. In particular, HRSA funding has facilitated states’ expansion of their ability to screen by supporting the implementation of new RUSP disorders; providing training from other states with experience implementing new conditions; outsourcing second-tier testing to regional testing centers; providing technical assistance on new conditions through calls, webinars, and web-based resources; providing educational materials for healthcare professionals about new RUSP conditions; and creating a readiness tool for states to support the addition of new conditions to their panels. As one state NBS staff noted, “I don’t think we could do what we do [without these programs]. They’re the ones that help us move it forward. They connect us with other states. They provide us with screening algorithms…I would say these programs are our go-to on screening.”

Although progress has been made under this activity, gaps remain including insufficient federal guidance and funding for NBS implementation. This has resulted in a duplication of efforts across states and inequities in how differently states fund their NBS programs. Some participants wished for stronger guidance and consistent policies about adding RUSP conditions, as many were concerned by the variability in how quickly screening for new conditions are implemented across states. Participants also wished for more funding to help bolster NBS programs that may be at risk in some states and to help states add new conditions. Some worried about the security of federal public health funding. Additionally, participants mentioned the need for more collaboration among federal agencies and funded programs.

Participants also expressed concern over the NBS system’s capacity to handle new treatments and screening technology and the lack of infrastructure to enable data sharing, data standards, and harmonization across states. This includes data collected in the hospitals at the time of specimen collection, screening results stored in a Laboratory Information Management System (LIMS) database, short-term follow-up data located in electronic health records, and long-term follow-up data from registries and vital records. A few participants were familiar with recent federal investments in interoperability and data modernization and were hopeful about their potential impact.

Participants noted that there is strong support for education and training opportunities for lab and follow-up staff, particularly around timeliness, adding new conditions, and a data repository on quality indicators. One state NBS staff commented that efforts to provide training, education and TA have been ongoing, saying, “It’s unprecedented the amount of time and energy spent, even during this past challenging year, to provide opportunities for people to enhance their skills and move forward with different projects, whether in the laboratory or with education and training.” In addition, funded programs have been particularly effective at providing education and TA to state NBS programs and labs and engaging NBS programs of all sizes. TA was provided in a variety of ways including a web site which hosted forums and a listserv, workgroups for states NBS staff on different topics, and individualized TA (both in-person and virtual).

However, additional training and TA is needed for state lab and follow-up staff around topics such as data analytics, adding new conditions, long-term follow-up, and communication between lab and follow-up staff. Participants noted that smaller states often need more TA than larger states, which tend to have more internal resources for training. These differences in resources dedicated to training and program improvement can deepen existing disparities between states. Participants suggested possible strategies to address this disparity, including funding to attend in-person training, provision of TA from contractors, and direct financial support for labs.

Participants noted that although trainings and education about NBS are available for healthcare professionals, there is not widespread awareness and engagement with these offerings. Participants noted the need for educating hospital staff who collect NBS specimens and communicating with families about the specimen collection process. Similarly, primary care providers, who are often tasked with delivering out-of-range NBS results to families, need education on understanding the NBS system, including what a positive screen means as well as how to communicate results to a family.

Short-term follow-up in the NBS system has been effective at connecting families of children with a positive screen to clinicians for confirmatory testing and diagnosis. Participants spoke to the helpfulness of a range of activities provided by federally funded program activities, including webinars and trainings on short-term follow-up, peer-to-peer learning across a variety of topics, and a data repository that collects data on quality indicators related to timeliness of short-term follow-up.

Despite successes with short-term follow-up, several participants emphasized the lack of focus on long-term follow-up. One of the key needs identified was a national long-term follow-up system that would enable states to track data on children with different conditions starting with confirmatory diagnosis and onward. This is especially important as new conditions are screened that may have less long-term evidence of treatment efficacy or variable phenotypes, such as late-onset conditions. As one subject matter expert said, “Sometimes you put a test on the panel and don’t have the data to know if you made a good choice or not and whether kids benefit… [What is needed is] a more robust national system to conduct research on these screening possibilities and develop registries and long-term follow-up so we can evaluate different treatment modalities and if kids are receiving benefit from the NBS.”

Additional concerns were expressed about the inconsistency in short-term follow-up with families after a positive screen. Follow-up processes vary among states and by condition, and often primary care providers who are contacting families are not familiar with the condition or that an out-of-range screen requires confirmatory testing. According to participants, providers need basic anticipatory training related to out-of-range screens and next steps. Participants also shared gaps related to connecting families with psychosocial supports after an out-of-range screen or diagnosis. More consistent systems are needed to connect families with appropriate resources, including educational materials, access to formula or other treatments, and social support. These psychosocial barriers are exacerbated when families do not have sufficient insurance to cover many of the services that are necessary to manage their child’s diagnosis. Collectively, these issues can result in inequitable access to confirmatory testing and treatment.

Nearly all participants agreed that great strides have been made in improving the timeliness of NBS from specimen collection through diagnosis. It was noted that much of this progress was due to HRSA’s investments in establishing quality indicators and support to collect data and set a standard for states. One NBS program grantee said, “Sometimes just stating a goal makes people aim for that goal which before, they were just doing the best they could.”

Recommendations for future efforts included maintaining timeliness standards through ongoing education, quality improvement efforts, and funding. Participants indicated the timeliness needs to be consistently addressed due to staff turnover. Additionally, although the timeliness data includes quality indicators on reporting of time critical and non-time-critical screen-positive results, concerns were expressed about short-term follow-up quality indicators, including the amount of time from reporting screen-positive results to the receipt of medical intervention or a confirmed clinical diagnosis. This was likely due to the need to rely on healthcare providers, rather than NBS program staff, to collect the data. Several participants noted that timeliness may not need to be a focus for all conditions, and, in some cases, it may be more important to value precision in reporting data over speed.

Several participants listed multiple programs that have increased the amount, quality, and availability of educational resources for families about staff. For example, several referenced a website which included easy-to-understand information on screening procedures, screening outcomes, and how to respond to screening results; a list of which conditions for which each state screens; disorder-specific information outlining early signs parents should look for, treatment options, and expected outcomes; and options for support services and how to access care. Participants appreciated the diversity of high-quality educational materials designed to reach a broad spectrum of families.

Participants stressed that reaching and educating families is a complex, multifaceted endeavor and requires a variety of educational strategies. As one healthcare provider noted, “Education isn’t one size fits all.” In general, there is the perception that there are a lot of good educational resources available to families, but that it is a very patchworked system with no centralized location for all materials and little collaboration among groups. The reach of educational materials often depends on partners, infrastructure, and funding, which is not consistent among states or conditions.

Other areas for improvement included that some educational materials are not updated as needed and often do not reach their intended audiences. Several participants noted that families turn to the internet and social media for disease-specific education and support, and there was some concern about the quality and accuracy of information available through general internet searches. To better reach families, participants suggested focusing on the timing of the dissemination of materials and providing more accessible and clear education in the prenatal period as a positive screen can be traumatic to families who lack awareness and may not understand the context. Additionally, participants noted the need for increased visibility for existing educational resources that may not currently be returned as the top results within Google or other search engines. Finally, parents stressed the need to target content appropriately. Many participants noted that families need education about a variety of topics in NBS, such as the difference between screening and a diagnosis.

Improvements in health equity and outcomes have been made through the support, guidance, and direction provided to the NBS system. The provision of education, training, and TA to enhance state capacity and support the implementation of new conditions has led to earlier diagnosis and treatment for all children. HRSA’s investment in a data repository that provides states with a system to track timeliness data and evaluate progress has significantly helped to improve the speed of receipt and screening of specimens and reporting results. In addition, the inclusion of medically underserved and diverse families in a needs assessment to determine the design and development of various educational materials has resulted in more inclusive resources for all families.

Participants shared diverse perspectives on the gaps in the NBS system related to health equity and outcomes. Many participants noted that the NBS system itself is a driver of equity since each baby receives the same screening, yet inequities emerge when states screen for different conditions, when timeliness becomes an issue, when families have challenges accessing confirmatory testing, and when families transition into clinical care and have unequal access to resources (e.g., specialized treatments, transportation, social and emotional support). These gaps in equity all are amplified by loss to follow-up, as access to care is diminished over time. Long-term follow-up is needed both to prevent loss to follow-up and to assess whether there are disparities in equity. These inequities are correlated with and driven by other well-known inequities in access to care such as systemic racism and implicit bias.

This evaluation provided important information on HRSA-supported federal investments in NBS. State programs have made several enhancements, with federal support to expand the number of conditions on state NBS panels and funding to provide technical assistance on the implementation of RUSP conditions among the most significant. Participants indicated that despite this success, there was a need to increase funding to states, especially those with fewer resources, to reduce the state-to-state variability in the types of conditions screened. In addition, participants called for more coordination among federal agencies. A strategic plan could be used to guide the future of NBS, with an emphasis on outlining how each agency aligned with the components of the plan. MCHB recently released a Blueprint for Change for all children and youth with special health care needs, which identified four key areas (equity, quality of life, access to services, and financing of services) and sets forth an agenda for the next 15–20 years (17). An NBS-specific plan would supplement this work. This would complement the National Academies of Sciences, Engineering, and Medicine consensus study report on NBS (18). This work may set the stage for a roadmap, similar to one that has been conducted for sickle cell disease (19).

Although state NBS programs have access to education and training resources on a variety of topics through federal programs, there are still gaps. In particular, participants mentioned several data-related needs, including support for data analysis and data management. With the likelihood of genomic sequencing coming to NBS, whether in the form of whole genome sequencing or targeted panels, these challenges will be exacerbated given the effort needed to process sequencing data, interpret variants, and store information long-term (20, 21). HRSA has addressed some of these needs with recent investments in interoperability (22). However, additional infrastructure support in informatics and clinical expertise to understand gene-disease pairs will be needed to move state-based NBS programs towards the genomic era.

Although many participants reported success resulting from efforts to improve the timeliness of NBS, most agreed that continued federal support is needed. Recent work provided data on quality improvement projects conducted by states to address various aspects of timeliness, from specimen collection, testing, or reporting out of results (23). States implemented different approaches depending on which timeliness indicator needed improvement, such as educational campaigns with hospital staff and birthing facilities, expanding operating hours or adding screening on weekends, or using health information technology to assist with data transfer. NBS programs will continue to need support to make improvements or maintain timeliness goals.

Participants also highlighted the need to educate healthcare providers (in particular, primary care providers), hospital staff, and families on the NBS system and specific conditions. Materials and trainings are often available and of high-quality, but dissemination has been challenging. Educational efforts have been hindered further by the workforce challenges in genetics (24). Earlier work with parents found a preference for communicating about NBS at multiple timepoints, beginning at preconception and continuing through the postnatal period (25, 26). A survey of prenatal providers indicated that although most agreed that NBS was important they thought either pediatricians or hospital staff would discuss it with women (27). Multimedia resources presented shortly after birth have been shown to increase parents’ understanding of NBS (28). Websites that provide information on NBS are variable in their quality (29). However, parents of children with an out-of-range screen often turn to the internet for information (30). Ensuring providers and families have timely access to high-quality NBS materials is essential for the continued success of this important public health program. Strong partnerships between NBS programs and a diverse group of system representatives can assist with successful educational efforts (31–33).

The lack of long-term follow-up data individuals diagnosed with conditions through NBS continued to be a noted gap. This echoes recommendations from other NBS experts who have long called for a need to focus on long-term follow-up data (34–36). HRSA has invested in long-term follow-up activities (37), however these were recently initiated programs that were not included in this evaluation. Work from these programs has resulted in further defining minimal data to collect on identified infants as part of clinical or public health long-term follow-up (38). Likewise, families have stressed the need for long-term follow-up data, especially related to communication and relationships with providers, roles of the care team and parents, and care access and utilization (39). These data would expand our understanding of who is receiving care or has been loss to follow up. This information is crucial to addressing health equity issues in access to clinical care and reducing disparities (40, 41).

There are three limitations of our study that should be noted. First, although we included a diverse group, our participants consisted of a convenience sample. Further, we did not gather extensive demographic information on participants. Our findings, therefore, may not have been fully representative of all NBS participants. Second, we focused on specific initiatives and activities supported by HRSA. However, there are other federal agencies that support NBS programs that we did not cover in our evaluation. In particular, investments by the National Institutes for Health (e.g., rare disease research, pilot studies in NBS) and Centers for Disease Control and Prevention (e.g., support for implementation of new NBS conditions, NBS quality assurance program) were not included. Thus, we may not have captured all successes and gaps. Finally, although our use of the rapid turnaround analysis approach was appropriate for the evaluation and based on recommended practice, a traditional qualitative coding approach may have provided slightly different results.