A total sample of 8,793 self-initiated EMAs, provided by 2,301 hearing aid wearers as part of their hearing care, were analyzed retrospectively. The de-identified EMA data was collected through a smartphone mobile application compatible with commercially available hearing aids. No demographic data was collected, other than country in which the hearing aids were adjusted, as well as brand and type of the hearing aids – from which the technology level of the devices could be deducted. All listeners were lawfully informed that their de-identified data could be analyzed for clinical and research purposes upon acknowledging the data privacy notice of the mobile application. Ethical approval was obtained from the Institutional Review Board at Lamar University, Beaumont, Texas, United States (IRB-FY21-252) prior to analyzing the data.

The smartphone mobile application through which the data was collected, was freely available for download in the app store. The particular feature of the app through which the data was collected, was accessible for hearing aid wearers when it was activated by a clinician in the fitting software. Clinicians could choose to activate the feature whenever they believed that the listener and their hearing rehabilitation would benefit from a real-time feedback system, e.g., to highlight situations relevant for further counselling or fine-tuning. Listeners were required to open the mobile application on their own initiative, when they decided that an event of interest took place (self-initiated EMA), and (1) indicate whether they were having a positive or negative experience (multiple choice, further referred to as self-reported satisfaction rating); (2) respond to one, sometimes two (depending on the answers provided by the listener) multiple choice questions related to the momentary listening environment and problems encountered in these environments (not considered for this report); (3) optionally, add additional feedback through an open-text field, by describing their experiences in their own words.

An initial sample of 30,127 self-initiated EMAs on real-world hearing aid experiences was collected worldwide between May 2018 and June 2021 and extracted from cloud-based data logging of the smartphone mobile application. The data was preprocessed using R statistical software (19). As visualized in Figure 1, only EMAs containing an optional open text statement (18,170 EMAs, or 60% of the original sample) and EMAs collected from hearing aids fitted in English-speaking countries (i.e., Australia, Canada, England, Ireland, New Zealand, United States) were considered for further analysis. Text statements shorter than 20 characters were removed to eliminate entries with little content, after which a further manual data cleaning ensured that spelling mistakes were corrected, nonsense text, and a couple more non-English entries were removed. This last process resulted in a final dataset of 8,793 EMAs considered for further analysis.

The final sample of 8,793 self-initiated EMAs was analyzed using R statistical software (19) and Iramuteq [version 0.7, alpha 2] (20), an interface of R statistical software (19) for automated text analysis.

To explore how listeners describe their experiences with hearing technology, a cluster analysis was performed on the open text data, in order to identify emerging themes in the reports. The cluster analysis method applied here was a Descending Hierarchical Classification using the Reinert Method (21, 22). This method was chosen as it is an automated text analysis method that allows to process large amounts of text data in a time-efficient way, while at the same time aiming for similar goals as classical (manual) qualitative text analysis methods (23). The method considers segments of text as textual units and is based on the principle that words that occur together are related to each other, in terms of semantic meaning (reflecting topics or themes) or context (reflective of a certain period in time, geographic location, or socio-demographic traits of the speaker or writer) (24). Thereby, the Reinert Method (21, 22) not only allows identification of themes in a text corpus, but also reveals what themes are more or less related. The method has been used in a variety of research fields, such as sociolinguistics (24, 25), political sciences (26, 27), medicine (28, 29), and recently, also in audiology (30–34).

While the Reinert Method (21, 22) is a predominantly unsupervised classification algorithm (24), it requires some input from the researcher, such as the final number of clusters to be obtained (also see next paragraph). By default, the algorithm segments the text corpus considered into smaller textual units, which typically contain 35 to 40 words. The algorithm takes into account punctuation to obtain the text segments. When one considers a book as a text corpus, the textual units would approximately correspond to sentences. For a corpus consisting of naturally short texts statements such as considered here, i.e., one text corresponding to one written feedback from one listener, further segmentation is not required. The algorithm, then, lemmatizes all word forms considered, i.e., the word forms are transformed to their most basic, dictionary form. Afterwards, the text corpus is converted to a matrix. The matrix consists of one row for every textual unit that makes up the total corpus and one column for every adjective, noun, adverb, and verb that is present in the corpus. Each cell contains a value “1” (when the textual unit indicated by the row contains the word form indicated by the column) or “0” (when the textual unit indicated by the row does not contain the word form indicated by the column). For a simplified example, see Ratinaud (22). The algorithm then splits the matrix into smaller matrices, each split relying on a factorial correspondence analysis (22), a statistical method that summarizes a dataset along two dimensions based on the Pearson Chi-square (χ²) statistic. Hereby, the χ²-value reflects the association between the rows and columns of a contingency table and χ²/n (with n the grand total of the contingency table) the association strength between two categorical variables (for a review on factorial correspondence analysis, see (35) – note that χ²/n applies only to the simple use case of a two-way contingency table). For the first split, the algorithm divides the initial matrix into two sub-matrices, which are as lexically different from each other as possible. This is done by minimizing the number of word forms that the two sub-matrices have in common (23), or in terms of the factorial correspondence analysis underlying it, by maximizing χ²/n for the contingency table calculated by summing the two resulting sub-matrices (22). The split is then iteratively optimized by moving each row from the main matrix to the other sub-matrix than the one it ended up in and recalculating χ²/n. The change is kept, whenever it increases χ²/n, and reverted when it decreases χ²/n. This process is repeated until no more changes increase χ²/n. In this first step, the algorithm has identified the two most distinctive clusters in the corpus, i.e., two lexical themes that are as different from each other as possible (23). Afterwards, the algorithm relies on recursive bipartitions, i.e., each of the two lexical themes is again split into two new clusters based on a similar approach as outlined above, after which the largest of the remaining clusters is split into two clusters. This last process of further splitting the largest remaining cluster is repeated until a predefined number of final clusters is obtained (22). The result of the descending hierarchical classification is a number of lexical themes or clusters, each characterized by a list of words that have the strongest association, i.e., the highest χ² values, with each respective cluster. For an example of how χ² is calculated for individual word forms, see Pélissier (36, p. 35). The Descending Hierarchical Classification will further be referred to as cluster analysis.

Concretely, the data set considered here was first pre-processed to match the required data format for Iramuteq [version 0.7, alpha 2] (20), as per the available instructions (36, 37). In the second step, textual units were defined. For the current dataset, one text statement, reflecting one written feedback from one hearing aid wearer, was considered as one textual unit. As the statements were short in nature, the texts were not further segmented into smaller units. To avoid further segmenting while running the analysis, the “make text segments” option can be deactivated when importing the text corpus. Further, the “simple on texts” option can be chosen when running the clustering in Iramuteq (20). A challenge with any automated clustering is deciding how many topics will be considered (23). For this analysis, the “number of terminal clusters on phase 1” was increased from 10 (default setting) to 20. It is important to stress the exploratory nature of the analysis. Clustering is an iterative approach: when there is too much overlap between two resulting clusters and they cannot be distinguished from each other by the researcher or domain expert, the number of clusters can be decreased. When a certain cluster captures too much content according to the researcher or domain expert, the number of clusters can be increased to explore whether multiple themes were captured by the same cluster. Also, it is important to note that the algorithm creates the number of clusters requested, but only keeps clusters of a certain size. Under the default settings, the minimum size required to keep a cluster is determined by the number of text (segments) in the total text corpus, divided by the number of clusters requested. Also, the “maximum number of analyzed forms” was increased from 3,000 (default setting) to 30,000. As we had a reasonably large data set, it ensures that the algorithm considered all individual word forms (other than those with a frequency < 3) available in the text corpus. Based on the output of the algorithm, it is the researcher's task to interpret the clusters and name them, based on a list of characteristic words and text statements that are most associated with each cluster. For this report, five researchers (i.e., CV, IO, VM, SL, DWS) with expert domain knowledge interpreted the output of the algorithm separately to afterwards agree on the naming of the clusters.

Based on the output of Iramuteq [version 0.7, alpha 2] (20), a χ² test of Independence was performed using R statistical software (19), to explore the association between the identified clusters and self-reported satisfaction ratings, i.e., positive vs. negative. A positive or negative hearing experience could be specified by the listener as part of the self-initiated EMAs, in the first screen of the survey.

The 8,793 self-initiated EMAs were logged through a mobile application by 2,301 individual hearing aid wearers. The data was skewed: the sample contained between 1 and 4 EMAs for 78% of listeners (median (IQR) = 2 (1–4) EMAs per listener) and between 4 and 90 EMAs for 22% of listeners (min = 1; max = 90 EMAs per listener). Most EMAs included in the sample were logged for hearing aids fitted in the United States of America (5,577; 63%) and Canada (1,349; 15%), followed by Australia (945; 11%), England (893; 10%), New Zealand (19; 0.2%), and Ireland (10; 0.1%). Almost half of the EMAs (4,226; 48%) were logged for hearing aids with premium technology levels (see Figure 2). Overall, listeners indicated 3,462 positive experiences with hearing technology (39%) and 5,331 negative experiences (61%).

Across the 8,793 text statements considered in this analysis, with one text segment collected as part of one listener self-initiated EMA, a total number of 145,926 word forms and 6,245 different word forms were recorded. About 44% of all individual word forms (2,769 out of 6,245), corresponding to 2% of the total number of words (2,769 out of 145,926) were recorded only once.

Cluster analysis revealed that 97.9% (8,606 out of 8,793) of the available text data could be classified into the resulting clusters, indicating that the results are representative of the corpus. The clustering revealed two branches (see Figure 3), one consisting of four clusters and the other consisting of three clusters. The branches and clusters were named by the researchers based on their expert domain knowledge, while interpreting the characteristic words (see Figure 4) and characteristic texts (see Table 1) derived for each cluster, based on a ranking of χ²-values.

The largest of the two branches, containing 58.1% of clustered texts, was named sound quality and speech intelligibility in challenging situations. This branch consisted of four clusters, reported here from largest to smallest: 1.

Communication and speech intelligibility in challenging situations, containing 21.0% of clustered texts, describing experiences related to conversations and speech understanding in social or acoustically challenging settings, such as restaurants, bars, office spaces, and cars – very often portraying multiple speakers in different locations.

The smallest of the two branches, containing 42% of clustered texts, was named hearing aid management. This branch consisted of three clusters, reported from largest to smallest: 1.

Connectivity, containing 20.8% of clustered texts, describing experiences related to different types of connectivity, e.g., pairing hearing aids with phones and mobile applications, or with other devices, such as TVs or tablets.

A two-sided χ² Test of Independence revealed that there was no significant relationship between the seven clusters and the ratio of positive vs. negative self-reported satisfaction ratings (X² (6) = 4.79, p = .57). The signs of the χ² test's standardized residuals inform about the direction of the outcomes, with positive values suggesting a positive relationship, and negative values a negative relationship. As visualized in Figure 5 (first three columns, bottom row), the positive residuals suggested a trend for text statements related to hearing aid management to be associated with negative satisfaction ratings (z_{physical fit & comfort}= .75, z_battery = .92, z_connectivity = 1.25). In turn, and as visualized in Figure 5 (last four columns, top row), text statements related to sound quality and speech intelligibility in challenging situations tended to be associated with positive satisfaction ratings (z_{noise & naturalness of sounds} = .09, z_{music experience and sound quality}= .49, z_{TV experience and loudness comfort} = .88, z_{communication & speech intelligibility in challenging situations}= 1.34). Note that a positive residual, suggesting a positive relationship between a cluster and positive satisfaction ratings, is accompanied by a negative residual of the same size, suggesting a negative relationship between the same cluster and negative satisfaction ratings.

To our knowledge, this is the first study to report on a large data set of self-initiated EMAs, containing open-text statements provided by hearing aid wearers. It is important to acknowledge that the data explored was not collected for research purposes. The data was logged to support hearing aid wearers and clinicians during the hearing rehabilitation process, i.e., for purposes of counselling and fine-tuning by collecting in-the-moment feedback. Using clinical data for research purposes offers real-world insights that can support continuous improvements in health care (55) and personalized care (56), but comes with limitations. Verheij et al. (57) and Dillard et al. (58), for instance, describe the use of electronic health records for research purposes and how each step in the process, from a clinically relevant event (e.g., decision of an individual to seek help, or decision of a clinician to perform a diagnostic test) to data cleaning can impact the quality of such data sets and generalizability of the results. Thus, it is important to note that the feature through which the present data set was collected was available only to listeners who actively managed their hearing loss through hearing aids, who chose a particular brand of hearing aids, who were comfortable using a mobile application, for whom clinicians had activated the feature (e.g., when they thought the person could benefit from or would be motivated to use a real-time feedback system), and who described their experiences in English. Further, participants self-selected to open the app to provide an optional open-text statement. As a result, the data presented here is likely representative of a tech savvy and motivated subgroup of hearing aid wearers. When considering the initial sample of 30,127 self-initiated EMAs prior to data cleaning, it is indeed noteworthy that 60% (18,170 EMAs) included an optional open-text statement – which can be considered a high response rate for an optional assessment. Still, the majority of listeners (78%) only used the real-time feedback system between one and four times. While the results suggest that self-initiated EMAs can provide valuable audiological insights into the experiences of, at least a subgroup, of hearing aid wearers, EMA is prone to poor compliance when participants are asked to determine themselves when an event of interest takes place to, in turn, self-initiate a response (16). Therefore, it would be of interest for future research to investigate compliance to EMA in a large, clinical population of hearing aid wearers when, e.g., time- or event-based triggering is introduced.

A further limitation of this study is that we cannot rule out that listeners self-initiated EMAs retrospectively, once the hearing event was over, thereby still being prone to recall bias. As is apparent from anecdotal examples described in the past tense, listeners might especially describe events retrospectively when they occurred in a setting where phone use is less appropriate, e.g., “saw a singer in the pub last night (…)”, “I attended an important work meeting with 8 client representatives (…)”. At the same time, it is encouraging that listeners still provided feedback shortly after the event, e.g., when they had time or felt comfortable doing so, as the information could still be valuable to clinicians. Finally, no demographic or audiological information was available for the hearing aid wearers, beyond the hearing aid technology level.