Autism spectrum disorder (ASD) is a childhood-onset neurodevelopmental disorder with a rapidly increasing prevalence. ASD is now thought to affect as many as 1 in 68 children, and over 3.5 million people, in the United States (1). The economic impact of ASD is substantial with an aggregated total cost in the United States of approximately $236 billion (2). This figure includes direct and indirect costs from a variety of sources, including medical care, special education, and lost parental productivity (2, 3). Many children with ASD benefit from intensive behavioral interventions, although the costs are often prohibitive, ranging between an additional $40,000 and $60,000 per child per year in the United States (4).

The rapid rise in the number of individuals diagnosed with ASD has strained the limited behavioral therapy resources, and many children face considerable delays in obtaining these much-needed interventions (5). These interventions are crucial given that there are no medical treatments to improve the deficits seen in ASD. ASD is principally characterized by impairments in social communication, and the presence of repetitive and restrictive behaviors (6). Evidence-based behavioral therapies have sought to address these deficits by focusing on improving communication, social interactions, and reducing challenging behaviors (7). Vital socially salient information is transmitted through facial emotions, expressions, eye gaze, and other nuanced social cues. Unfortunately, many people with ASD have been found to demonstrate deficits in facial processing, potentially accounting for some of their difficulties in social communication (8).

The high demand for therapeutic autism services combined with recent technological advances have led to a rapid increase in attempts to deliver such interventions through digital modalities (9). Compared to traditional behavioral therapies, evidence for these digital interventions are limited, but there are some research studies to suggest that digital interventions may improve specific skills and behaviors in people with ASD (Table 1).

Furthermore, children with ASD may be especially receptive to such digital interventions (17, 18). Children with ASD have been shown to have a preference for electronic media (19), game-like elements (20), computer-generated speech (21), and those with particular visual strengths may be especially adept at engaging with digital modalities (22). Current diagnostic and therapeutic approaches seldom rely on quantitative data measures of symptomatology, severity, or condition trajectory, and it is hoped that digital tools may help to provide such data. Emerging technology and mobile sensor-rich devices may help to deepen and refine new paradigms for understanding social communication, such as those outlined within the social processes construct of the National Institute of Mental Health’s Research Domain Criteria (RDoC) (23) and to undertake digital phenotyping and subtyping of behavioral conditions (24).

Some of the most promising technologies include augmented reality, and some reports have highlighted the early use of augmented reality interventions in children with ASD (25–27). The Brain Power System (BPS) is a smartglasses-based augmented reality system for children and adults with ASD to teach themselves life skills that may facilitate or enhance self-sufficiency. The BPS consists of a number of gamified applications; the social communication module includes applications designed to teach users to recognize facial emotional expressions and to attend to the faces of others. The BPS collects quantitative data about the user’s environment and interactions through the use of an array of inbuilt sensors and analyzes these data through the use of artificial intelligence (AI), including the use of Affdex emotion AI (Affectiva, Boston, MA, USA) (28). This report will explore the feasibility and usability of the BPS through caregiver interview, staff observation, and structured validation with autism rating scales.

The BPS was developed in response to several areas of need highlighted by people with ASD and their families. One of the most frequently encountered concerns by parents was a feeling of “disconnection” from their children during conversations. This sense of disconnection is likely related to several previously outlined social communication deficits in ASD, including deficits in social-emotional reciprocity, non-verbal communicative behaviors used for social interaction, and deficits in developing, maintaining, and understanding relationships (6).

In this report, the feasibility of the BPS was assessed when it was used to provide a single session of a behavioral coaching intervention to two males with ASD. The tolerability, usability, and behavioral changes associated with the intervention were assessed via a validated autism inventory, and subjective caregiver and user report. The aberrant behavior checklist (ABC) was the validated autism inventory used in this study (29).

Both of the users and their respective caregivers completed the intervention session without any adverse effects noted by the study staff or reported by the participant and caregiver.

Augmented reality interventions provide an opportunity to enhance social communication in ASD, and our report demonstrates preliminary evidence of usability and feasibility of one combination of smartglasses and augmented reality technology. Both users managed to use the device without any observable adverse effects, and the BPS was found to be well tolerated, easy to use, and highly engaging according to the caregivers. Caregivers also noted that the users had improved social interactions, through improvements in non-verbal communication, social engagement, and eye contact while using the BPS. It was reassuring to see that ABC scores 24 h post-intervention did not get worse, when compared to the pre-intervention baseline. In fact, it was noted that both children had considerable reductions in symptoms across all five ABC subscales, including irritability/agitation, lethargy/social withdrawal, stereotypic behavior, hyperactivity/non-compliance, and inappropriate speech. While these findings highlight promising feasibility, and usability, and may also suggest some improvement to subjective and objective behavior, there are a number of notable limitations. First, the intervention was tested in two young males, and our findings cannot be expected to generalize to the broader ASD population, which is highly demographically and clinically heterogeneous. Additionally, in this report, the primary aim was to assess feasibility through the use of a single intervention session. To further understand the improvements in ASD symptoms as measured by the ABC, larger studies will be required. Our ongoing research efforts are investigating the outcomes of repeated coaching sessions, in addition to longitudinal monitoring of the ABC to see if there are sustained changes to the ABC based on the use of these interventions. We will also need explore how females with ASD may respond to the BPS, as sex and gender may influence symptoms presentations, and we should be mindful of the range of sexually dimorphic differences in social communication, memory, and cognitive flexibility (42).

When using technologies like the BPS, a vast amount of user data is collected through the inbuilt smartglasses sensors, and analyzed through the applications. These quantitative datasets, encompassing visual, audio, physiologic, and movement data around social interactions, may allow for deeper insights into baseline social communication deficits, and improvements in interventions. Through the use of machine learning we may be able to adopt a data-driven approach to identifying optimal therapeutic and educational approaches for each individual user.

Smartglasses may offer a number of distinct advantages compared to applications delivered via smartphones. With smartglasses, users are heads-up as opposed to immersed in a screen, and they remain hands-free, thus able to use their hands to engage in both non-verbal social communication and undertake educational/occupational tasks. This type of mobile and lightweight technology allows users to coach themselves in the privacy of their own home, and whenever is most convenient. Finally, such technology can be rapidly scaled to meet demands.

The ASD community has considerable difficulty accessing effective and timely therapeutic interventions. This situation may potentially be addressed through the use of digital interventions, such as augmented reality. Augmented reality may be especially effective as it may deliver visual and auditory cues while the user is simultaneously engaged in natural or structured social interactions. Additionally, wearable technologies contain sensors that can be used to record and quantitatively monitor a user’s interaction. This report highlights the need for further research into the use of augmented reality technologies as a therapeutic tool for people with ASD. To the authors’ knowledge, this is the first published report of the use of augmented reality smartglasses, such as Google Glass, as a behavioral aid in a pediatric population. In the two reported cases, augmented reality smartglasses demonstrated high feasibility, usability, and tolerability. Overall, the results are encouraging but should be considered in the context of the limitations outlined.

This report includes data from human participants who used the Brain Power System. The use of the Brain Power System by children and adults with autism is under AIRB 2015-405A; “Brain Empowerment – Youth Oriented Unconstrained Reality for Social Engagement and Lifeskills Formation,” IRB-approved by Asentral INC IRB, who is affiliated with the Commonwealth of Massachusetts’ Department of Public Health.

NS is the inventor of the Brain Power System. JS, AV, and NS helped to design and create the intervention. AV was the lead on the writing of this technology report, with additional review and contributions by JS, RL, and NS.

This report was supported by Brain Power, a neurotechnology company developing a range of augmented reality technologies for smartglasses.