Electronic health records were accessed from 134,090 children (0–18 years) through AudGenDB. This publicly available, online database is HIPAA compliant, and all data are de-identified. AudGenDB contains audiologic testing, radiology reports, clinical genetic results, and medical health record data for patients who received audiological care at Children’s Hospital of Philadelphia, Vanderbilt University Medical Center, or Boston Children’s Hospital.

To construct the dataset for the present study, we first searched the database to identify all encounters that contained International Classification of Diseases 9th and 10th revision (ICD-9 and ICD-10) diagnosis codes from the encounter. To determine available encounters with audiogram data, the encounter was required to obtain at least one air conduction threshold with insert earphones, supra-aural headphones, or soundfield. Aided testing — pure-tone thresholds while wearing a listening device — and bone conduction thresholds were excluded. After filtering, the number of encounters that met these inclusion criteria was 226,580. Encounters were from a total of 100,960 children, with 81.2% receiving care at Children’s Hospital of Philadelphia, 10.8% at Boston Children’s Hospital, and 7.9% at Vanderbilt University Medical Center.

Using the ICD-9 and ICD-10 diagnosis codes, we identified 8,810 children who had a diagnosis of ID, ASD, DS, or CP, which hereafter we refer to collectively as children who have developmental disabilities. Specifically, children who have developmental disabilities had at least one of the ICD-9 or ICD-10 listed in Supplemental Table 1 in the child’s diagnosis or problem list record. Table 1 reports the number of children included for each developmental disability condition. If a child was identified as having a developmental disability, this flag was applied to all encounters for the child. Children who did not meet the criteria for developmental disabilities were classified as not having a developmental disability diagnosis — referred to as the comparison group in the present study.

Table 1 provides an overview of the demographics for children included in the dataset stratified by developmental disability status. The number (n) and proportion of children for each demographic category are provided for the children examined in this study who had encounters with audiogram data. Children were classified based on their age, race, and ethnicity based on their disability status. Race was categorized into 5 groups: white, Black or African American, Asian, Indigenous (including Indian, Native Alaskan, American Indian, Native Hawaiian, and Pacific Islander), and other (including “other,” unknown, missing, or not revealed). Ethnicity was categorized into 3 groups: Hispanic, non-Hispanic, and other (including “other,” unknown, missing, or not revealed). Race and ethnicity data should be interpreted cautiously as there were differences in available categories across sites and over time.

For each encounter with audiogram data, hearing status was classified into one of three mutually exclusive categories: insufficient data, typical hearing, or reduced hearing. This determination was based on the recorded air conduction thresholds at 500, 1,000, 2,000, and 4,000 Hz for each ear. Soundfield thresholds were only used if ear-specific data were not available. No attempts were made to classify the type or severity of the reduced hearing. An encounter was flagged as “reduced hearing” if either ear reached criteria for that method.

As detailed in Table 2, we built five methods to classify hearing status. In the naming convention used for these methods, T is the number of thresholds required, E is the number of encounters considered, and C is the threshold criteria for classifying reduced hearing. The subscript then further defines each of these variables for the method. Possible T values are 1, 8, or 3–4; possible E values are single encounter (S) or multiple encounters (M); possible C values are a single elevated threshold (T) or an elevated PTA.

Method T₁ E_S C_T, the least restrictive method, results in a reduced hearing identification if a single threshold is elevated and does not have a requirement for the number of thresholds obtained during the encounter. This is the method that has been used by other AudGenDB studies to determine hearing status (e.g., Kreicher et al., 2018b; Close et al., 2020). Method T₈ E_S C_T aligns with best-practice clinical recommendations for determining hearing status (American Speech-Language-Hearing Association, 2004): requiring threshold data for all four frequencies (500, 1,000, 2,000, and 4,000 Hz) from both ears. Based on the approach commonly used in epidemiological studies (e.g., Mehra et al., 2009; Van Naarden Braun et al., 2015), Method T_3-4 E_S C_PTA identifies reduced hearing based on an elevated four frequency PTA (PTA-4): the average of 500, 1,000, 2000 and 4,000 Hz. If data at 4000 Hz are unavailable, a PTA-3 (500, 1,000, and 2000 Hz) is permitted. Additionally, Method T_3-4 E_S C_PTA allows for the use of soundfield data if ear-specific thresholds are unavailable. The flexibility for using a PTA-3 and soundfield data was included to replicate the methods used by other studies using data from AudGenDB (e.g., Kreicher et al., 2018b; Close et al., 2020) and may be particularly useful to promote the inclusion of encounters from children who are young and/or have developmental disabilities.

In contrast to the other methods, Methods T₈ E_M C_T and T_3-4 E_M C_PTA allow threshold data to build up over multiple encounters. The rationale for this approach is that audiologists may obtain an audiogram for a child over multiple encounters. Except for the use of multiple encounters, Methods T₈ E_M C_T and T_3-4 E_M C_PTA are identical to Methods T₈ E_S C_T and T_3-4 E_S C_PTA, respectively. The algorithm for Methods T₈ E_M C_T and T_3-4 E_M C_PTA searches through audiograms in chronological order to obtain missing threshold values. Once a particular threshold is obtained, it is not replaced by subsequent data. No time limitation was placed on the delay between encounters.

For each method, reduced hearing was classified based on a liberal or conservative criterion (> 15 or > 25 dB HL). Replicating the methodology used by other AudGenDB studies, in the liberal approach reduced hearing was defined as a threshold or PTA of >15 dB HL if obtained under insert earphones or headphones; > 20 dB HL if obtained in the soundfield, or > 25 dB HL for infants (< 1 year). In the conservative approach, reduced hearing was defined as a threshold or PTA of >25 dB HL based on World Health Organization guidelines (2016), regardless of transducer or child age. Although the conservative criterion excludes cases classified in the slight severity range (16 to 25 dB HL), it is expected to reduce the risk of over-identifying reduced hearing in children who were clinically managed as having typical hearing sensitivity.

Prior to analysis, we used data cleaning techniques based on SQL-like code in R (Wickham et al., 2019; R Core Team, 2021) to manipulate the multiple comma separated value (CSV) files to extract relevant information about patients’ demographics, encounter history, and behavioral data. We conducted exploratory data analyses of potential inconsistencies in variable entry between the three clinical sites. Except for differences in billing practices related to diagnoses codes for reduced hearing, we did not detect relevant site differences. As a result, we did not control for the site in our data sampling techniques or analyses. We performed statistical analyses with R (version 3.5.2) and Python (version 3). Consistent with the purpose of this study, we focused on descriptive statistics to describe the method in relevant and meaningful terms to guide researchers in developing classification systems. We limited the use of hypothesis testing given the large number of samples and the likelihood of significant results for small effects. All hypothesis tests were prespecified to be conducted at the 0.05 significance level. In this paper, N is used to denote the number of encounters and n is used to denote the number of children. In analyses that consider differences across the four developmental disabilities, encounters from children who have multiple conditions are included under all diagnosed disabilities.

Results are provided in Table 3 and are grouped by criterion for all encounters with audiogram data (N = 226,580) without consideration for developmental disability status. For the full dataset, the number of encounters in which hearing status could be determined varied across methods. Because the inclusion criteria for the study required encounters to have at least one threshold in the record, all encounters could be classified for Method T₁ E_S C_T. The proportion of encounters where hearing status could be determined was 0.806 for Method T_3-4 E_S C_PTA and 0.867 for Method T_3-4 E_M C_PTA. The lowest proportion of classified encounters was seen for Methods T₈ E_S C_T and T₈ E_M C_T at 0.622 and 0.658, respectively. This pattern of results supports our hypothesis that the proportion of classifiable encounters decreases with increased threshold requirements. Additionally, one-tailed t-tests for differences in proportions of identifiable encounters confirm that there is a significant benefit for allowing data to build up over multiple encounters: T₈ E_M C_T > T₈ E_S C_T (t = 25.78, p < 0.001), and T_3-4 E_M C_PTA > T_3-4 E_S C_PTA (t = 56.31, p < 0.001). Allowing information to build up over time resulted in the inclusion of 299 patients with Method T₈ E_M C_T and 541 patients with Method T_3-4 E_M C_PTA who had insufficient data to be classified with the companion, single-encounter method.

Age at hearing status determination is the mean child age for the first encounter in which hearing status could be determined for each patient. Mean child age for hearing status determination was younger for Method T₁ E_S C_T than all other methods. Relative to Method T₁ E_S C_T, the delay in age at hearing status determination was 0.3 years for Methods T_3-4 E_S C_PTA and T_3-4 E_M C_PTA, and 1.8 years for Methods T₈ E_S C_T and T₈ E_M C_T. These results suggest that child age was affected by the number of required thresholds. Inconsistent with our hypothesis, there was no clinically meaningful reduction of child age if threshold data were allowed to build up over multiple encounters. Collectively, these results reveal a cost associated with methods that require more threshold data for big data applications: a delay in age for being able to determine hearing status that cannot be offset by allowing data to build up over multiple encounters.

To evaluate the effects of developmental disability status on the ability to classify hearing status, Figure 1 shows the proportion of encounters that could be classified for each method. Each panel represents one method for classifying hearing status. Within a panel, the first two columns provide data for children who have any of the developmental disability (DD) diagnosis (lighter scaling), then data for each developmental disability diagnosis are provided (ID, ASD, DS, and CP). For each disability type, encounters were dichotomized into two age groups: < 4 years (purple, solid) or ≥ 4 years (green, hashed). The rationale for this age split is because by 4 years children can complete a full audiogram in a single clinical encounter (Barr, 1955; Nielsen and Olsen, 1997). The horizontal lines in each panel provide the proportion of encounters that could be classified from children in the comparison group. The purple (solid) line is an age of encounter of <4 years, whereas the green (dashed) line is an age of encounter of ≥4 years.

Figure 1 shows clear differences in the proportion of classifiable encounters based on developmental disability status and age at the time of encounter. Consistent with clinical literature (Barr, 1955; Nielsen and Olsen, 1997), for children ≥4 years without a developmental diagnosis, the proportion of classified encounters was 0.909 to 0.961 across the methods. In contrast to this group, differences across the classification methods were seen for both age groups in children who have developmental disabilities. However, the pattern of results across methods largely mirrors those reported above for the full dataset.

There are three trends that warrant further considerations. First, for children who have developmental disabilities, Methods T_3-4 E_S C_PTA and T_3-4 E_M C_PTA result in a higher proportion of classifiable encounters than Methods T₈ E_S C_T and T₈ E_M C_T. This difference was particularly sizable for encounters from children <4 years. This pattern indicates that audiograms from young children or children who have developmental disabilities are likely to contain limited thresholds and/or soundfield results.

Second, there appears to be a greater benefit for allowing data to build up over encounters for children who have developmental disabilities than for children in the comparison group. However, the pattern of benefit for allowing data to build up over encounters appears to be relatively similar across the four different developmental disabilities. The largest benefit for allowing information to build up over time was seen for Method T_3-4 E_M C_PTA. Method T_3-4 E_M C_PTA results in the inclusion of an additional 101 children who have developmental disabilities that could not be classified with Method T_3-4 E_S C_PTA.

The final observation is that, across all methods and regardless of child age, encounters from children who have developmental disabilities had lower rates of classifiable encounters than the comparison group. To evaluate this trend, the average proportion of classified encounters across all five methods was calculated based on developmental disability status. Encounters were not disaggregated by child age for this analysis. The proportion of classified encounters was 0.817 for children who have developmental disabilities (N = 22,621) and 0.873 for children in the comparison group (N = 203,959). This difference indicates that encounters are more likely to result in an incomplete audiogram for children who have developmental disabilities than for children in the comparison group. To further explore potential differences across the four developmental disability diagnoses, the proportion of classifiable encounters was computed for each subgroup. The proportion of classified encounters was 0.817 for ID (N = 4,352); 0.812 for ASD (N = 12,712); 0.834 for DS (N = 6,980); and 0.769 for CP (N = 2,991). This finding suggests that encounters from children who have CP have a lower classification rate than seen for encounters from children who have ID, ASD, or DS.

Table 4 provides the mean and 95% confidence intervals for child age (in years) at the time of the first encounter with audiogram data at which hearing status could be determined for each method based on developmental disability status, as well as for each developmental diagnosis. The average age provided for determining hearing status for Method T₁ E_S C_T reflects the first encounter in the clinic where behavioral threshold (s) were obtained. The mean age of first encounter in which behavioral testing is attempted is 1.9 months later for children who have developmental disabilities than for children in the comparison group. Moreover, mean age of hearing status determination is different across the four subgroups: ASD < CP < DS < ID. As shown in Table 4, the 95% confidence intervals for mean child age for Method T₁ E_S C_T do not overlap with the mean child age for any of the subgroups. Consistent with these findings, hearing status was determined at older ages for the remaining methods for children who have developmental disabilities than for children in the comparison group. The one exception is children who have ASD. These results should be interpreted cautiously because it may be that this developmental profile affects the timeline for the referral to audiology as well as the ability to collect behavioral data. Moreover, the true age of onset of reduced hearing is likely different across the developmental disability subgroups.

To further explore the difference in child age across the methods, while considering differences for when behavioral testing was first attempted, the gap in age was calculated between the three single-encounter methods. The age gap for Method T₈ E_S C_T and Method T_3-4 E_S C_PTA was calculated relative to Method T₁ E_S C_T. The calculated age gaps were larger for children who have developmental disabilities than for children in the comparison group. For example, the difference between Method T₁ E_S C_T and Method T₈ E_S C_T was 2.5 years for children who have developmental disabilities and 1.7 years for children in the comparison group. With the group of children who have developmental disabilities, this difference was 2.0 years for children who have ID, 2.3 years for children who have ASD, 2.9 years for children who have DS, and 2.3 years for children who have CP. These values should be interpreted cautiously because only 53% of children who have developmental disabilities have sufficient data in their record to allow classification for the two methods: 62, 48, 60, and 52% for children who have ID, ASD, DS, and CP, respectively. In contrast, 66% of children in the comparison group were able to be classified for both methods.

The final comparison of interest was to examine the difference between single- and multiple-encounter method pairs (T₈ E_S C_T and T₈ E_M C_T; T_3-4 E_S C_PTA and T_3-4 E_M C_PTA) to see if allowing data to build up resulted in an earlier age at the time of hearing status determination. Across all subgroups of children, the mean age of children at the time of classification is younger for multiple-encounter methods than single-encounter methods. However, as shown in Table 4, the value for the multiple-encounter method falls well within the 95% confidence interval estimate for the single-encounter, companion method for all subgroups of children. Thus, allowing data to build up over multiple test sessions does not result in a meaningful decrease in child age at the time of hearing status determination for children who have developmental disabilities.

One potential concern is that stability of hearing status may be variable across classification methods and developmental disability status. To evaluate the stability of each method, children were identified in the dataset who had at least two encounters with audiogram data. Multiple encounters were available from 47,893 patients in the dataset, of which 9.6% (n = 4,589) had a developmental disability. For each classification method, we compared the patient’s hearing status from the first classifiable encounter to a randomly selected classifiable encounter from a later time point. For each method, the proportion of classification change was computed: a change in the hearing status (i.e., “typical hearing” or “reduced hearing”) between the two encounters, without consideration for directionality. For each classification method, Supplemental Table 2 (liberal criterion) and Supplemental Table 3 (conservative criterion) provide the proportion of classification change, proportion of children who have reduced hearing, and sample size (n) by developmental disability status. Results from the build-up methods should be interpreted with caution because it is possible that the same thresholds could have been used to determine hearing status for the two encounters. Across all methods and criteria, the rate of change in classification was lower for children who had developmental disabilities than for children in the comparison group.

To further examine the stability of hearing classification by disability type, the combination of the classification from the two encounters was compared. Patients were assigned to one of four categories: “reduced hearing” classification for both encounters (“+/+”); “typical hearing” classification for both encounters (“−/−”); “reduced hearing” classification at the first encounter and “typical hearing” classification at the second encounter (“+/−”); or “typical hearing” classification at the first encounter and “reduced hearing” classification at the second encounter (“−/+”). Collapsing across the classification methods and criteria, Figure 2 provides the percentage of occurrence for each category based on developmental status. The colormap provides a visual representation for the percentage value in each box. Across all membership groups, only 5 to 8% of patients had instability because their classification shifted from “typical hearing” to “reduced hearing.” In contrast, most changes in classification were in the opposite direction, although children who have developmental disabilities were less likely to have this shift than children in the comparison group. Specifically, whereas 20% of patients in the comparison group had a shift from “reduced hearing” to “typical hearing” (+/−) across the two encounters, this category only described 10.6% (CP) to 15.6% (ASD) of encounters for the developmental disability subgroups. The prevalence of a stable “reduced hearing” classification (+/+) was substantially higher in children who have developmental disabilities than for children in the comparison group (DS > CP > ID > ASD). Thus, children who have developmental disabilities – especially those who have ID, DS, and CP – are at high risk for having reduced hearing that persists over both encounters.

Leveraging data in electronic health records provides a unique opportunity to accelerate the pace of pediatric research in the areas of audiology and developmental disabilities. This approach can be used to determine disparities and inequities in pediatric hearing health care, as well as determine prevalence of reduced hearing in childhood for specific developmental profiles. However, because hearing-related billing codes may not reflect actual hearing status, care must be used in the extraction and interpretation of raw clinical data to determine hearing status. Furthermore, as discussed below, the audiogram parameters used to identify cases of reduced hearing can disproportionately exclude children who have developmental disabilities in the final dataset. Results from this study can guide researchers in determining reduced hearing criteria which is inclusive of children who have developmental disabilities, and therefore, allowing representation of this population when conducting research to guide clinical decisions. These considerations will become important to guard against bias if precision medicine algorithms are incorporated into standard hearing health care in the future. These considerations are also important during the creation of large-scale repositories in response to open science initiatives to accelerate biomedical discoveries. To date, publicly available datasets or data hubs of individuals who have developmental disabilities often contain no hearing data; those that do are limited to hearing-related ICD codes. Results from the current study indicate there is need to capture raw clinical data to accurately understand the co-occurrence of reduced hearing in children who have developmental disabilities.

In addition to extending our knowledge of the appropriateness of different methods for identifying reduced hearing cases in applications using big data analytics, results from this study have important implications for clinical care of children who have developmental disabilities. Results in the present study provide clear evidence that pediatric audiologists have a difficult time obtaining a complete audiogram with current clinical methods in children who have developmental disabilities. About 28% of children who have developmental disabilities in AudGenDB had no audiogram data, whereas 23% of children in the comparison group had no audiogram data. When behavioral thresholds were recorded, they were more likely to be in the soundfield, rather than ear-specific thresholds, for children who have developmental disabilities than for the comparison group. Over reliance on soundfield testing means that children who have developmental disabilities are at risk for undiagnosed reduced hearing that is unilateral. Results from the present study also question the common clinical perception that an audiogram is successfully obtained over multiple test sessions. Collectively, these observations suggest that children who have developmental disabilities experience disparities in their access to the gold-standard assessment of hearing. Consequently, certain hearing profiles are likely at risk for being undiagnosed in children who have developmental disabilities (e.g., slight or mild, high frequency, and unilateral hearing profiles). This issue may be somewhat mitigated by the use of physiologic measures (e.g., auditory brainstem response and otoacoustic emissions), but these procedures are prone to missing some of the above hearing profiles and may require sedation. Challenges in adhering to recommended pediatric audiology diagnostic practices seen here and for other studies (Findlen and Schuller, 2020; Nightengale et al., 2020; Trudeau et al., 2021), suggests that there is a need for future research and clinical guidance to ensure equitable access for children who have developmental disabilities to a comprehensive hearing assessment in a timely manner.

Results from the present study suggest that certain developmental profiles are at more risk than others for providing limited behavioral threshold data. Encounters from children diagnosed with ASD generally had a higher rate of classification than those encounters from children who have ID, DS, or CP. Children who have CP were more likely to have encounters that could not be classified than the other disability subgroups. Additionally, children who have DS had larger age gaps across the different classification methods than children who have other developmental disabilities. These observations may reflect difficulty to collect a complete audiogram because of developmental misalignment between the child and task. Alternatively, audiologists may be prioritizing other types of clinical data in the management of children based on their developmental disability diagnosis. However, Trudeau et al. (2021) recently reported significant delays for making a referral for auditory brainstem response testing in children who have developmental disabilities when behavioral thresholds were not successfully obtained. The average delay was 9 months for children who have ASD; 22 months for children who have DS; and 20 months for children who have CP. Further research is planned to better understand the developmental profiles of children at greatest risk for having no or limited pure-tone threshold data. Of particular interest is to consider other diagnostic assessments in addition to audiogram data to determine the test battery components that are used in clinical practice for specific developmental profiles. Understanding barriers in determining a child’s hearing profile in a timely manner is critical to ensure appropriate audiological management for children who have ID, DS, or CP given their high likelihood of having reduced hearing. Moreover, future research is needed to evaluate how delays in diagnosing reduced hearing in these populations affects access to hearing technology or other deafness-specific interventions.

One limitation of this study is that the data were retrospective clinical data from three sites. Errors in our data may be like those commonly seen in clinical data. Our study is reliant on data from children seen for hearing health care and does not reflect the general population. Testing protocols differ across clinical sites and may also differ across audiologists within a clinic. Moreover, because data from two of the sites were obtained over a long period there were likely changes in testing protocols and/or handling of the electronic health records over time (e.g., data storage or extraction procedures to AudGenDB).

External validity of this study may be limited to academic medical settings with specialization in pediatric care. The three clinical sites used in the study have large teams of pediatric audiologists, diagnostic protocols for children, and access to hospital resources tailored to children and their families. Additionally, these clinics have specialty clinics and regularly receive referrals for children who have complex medical or developmental profiles. The gaps in clinical care for children who have developmental disabilities that were identified in the present study may be greater for audiology clinics that do not have a similar level of audiologist expertise and/or hospital resources as these sites.

We used a narrow definition of developmental disabilities by limiting to codes to those associated with ID, ASD, DS, and CP. These four conditions are some of the more common developmental conditions and children who have ID, DS, or CP and known to be at higher risk for reduced hearing compared to the general population (e.g., Hild et al., 2008; Reid et al., 2011; Van Naarden Braun et al., 2015; Weir et al., 2018; Kreicher et al., 2018b; Trudeau et al., 2021). This definition may mean that the reported occurrence of reduced hearing was higher than if this group had been constructed differently. It also means children who have other developmental disabilities were included in the comparison group. Our reliance on the ICD diagnosis codes to determine developmental disability status, can result in group membership errors because of billing errors or emissions, or that children received a developmental disability diagnosis at school or a different medical facility.

Another limitation of this study is that we did not consider the type of reduced hearing. Some encounters flagged as “reduced hearing” are in fact due to a transient conductive component that will resolve over time. In the present study, cases of reduced conductive hearing are likely the primary driver of instability for hearing classification across two encounters. Limited availability of bone conduction thresholds in the database makes it challenging to confirm the type of reduced hearing. Although previous AudGenDB studies have used abnormal tympanometry as a marker of a possible conductive component (e.g., Kreicher et al., 2018b), it is relatively poor at identifying ears with a significant air-bone gap based on threshold data (Keefe and Simmons, 2003; Prieve et al., 2013). In addition to the challenges associated with identifying cases of reduced conductive hearing, another compelling argument for not excluding these cases is that children who have developmental disabilities are often at risk for persistent or recurring reduced conductive hearing (Rosenhall et al., 1999; Herer, 2012; Weir et al., 2018; Kreicher et al., 2018b; Hamberis et al., 2020). Moreover, based on language outcome data from children who have DS (Laws and Hall, 2014), it may be that this type of reduced hearing is particularly detrimental for developmental outcomes in children who have developmental disabilities.