Our literature search was designed to identify reports of clinical RCTs testing aVNS as an intervention. Two databases were searched systematically: PubMed and Scopus (includes MEDLINE and Embase databases). Additionally, two search strategies were used. The first strategy combined search terms related to aVNS and RCT. The second search strategy focused on search terms related to commercial aVNS devices and their manufacturers. Complete search strings for both strategies are available in Supplementary Material 1. The search was last updated in July 2020. In addition, citations of all selected studies were searched to identify additional studies that met the inclusion criteria. The citations of relevant reviews (Murray et al., 2016; Yap et al., 2020) were also searched. Duplicate records were removed, and the remaining records were screened at a title and abstract level to check if a clinical RCT on auricular stimulation was reported.

As our primary goal was to assess the effects of auricular stimulation, studies using any stimulation modality from any field, including acupuncture and electroacupuncture were initially included as long as the intervention was at the auricle. When it became evident that a meta-analysis would not be possible due to incomplete reporting of information, we decided to exclude traditional Chinese medicine (TCM) studies, which typically used acupuncture, electroacupuncture, or acupuncture beads. Studies were considered TCM studies if acupoints were used to justify location of stimulation or if they were published in a TCM journal. This is captured in Figure 2 adapted from PRISMA (Moher et al., 2009). All studies excluded at the end of the search after full-text review are listed in Supplementary Material 2.

In papers where more than one clinical trial was reported, each trial that was randomized and controlled was included in the systematic review; non-RCT portions of included publications were not analyzed. Only publications 1991 and after were included, with the cutoff marking the first time autonomic activity biomarkers were reportedly measured during auricular stimulation (Johnson et al., 1991).

Included studies had to report measurements of direct clinical significance. This exclusion partially relied on whether the study claimed direct clinical implications of their findings. Additionally, studies were excluded if the measurements did not have a well-established link to clinically significant outcomes. For instance, pupil size, functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and somatosensory evoked potentials (SSEPs) are secondary physiological measures of target engagement (Burger et al., 2020). Although they may be useful to study the mechanisms of aVNS, they do not have well-established links to clinically significant outcomes. In comparison, heart rate variability (HRV), a measure of sympathovagal tone, is considered a measurement of direct clinical significance as sympathovagal imbalance is related to several disease states (Bootsma et al., 2003). Similarly, studies focused on cognitive neuroscience topics, such as behavior, learning, fear extinction, or executive functions were excluded. In contrast, psychological studies addressing addiction, depression, pain, and stress were included in the final qualitative review as they have had direct clinical significance.

We used the Cochrane risk of bias 2.0 tool (RoB), an established tool to assess bias in clinical RCTs (Sterne et al., 2019), which has been cited over 40,000 times in Google Scholar, to evaluate the quality of evidence in each study. The RoB tool assesses bias in five subsections intended to capture the most common sources of possible bias in clinical studies. It is important to note a rating of “some” or “high” risk of bias does not mean that researchers conducting the study were themselves biased, or that the results they found are inaccurate. Deviations from ideal practice frequently occur due to a variety of potentially uncontrollable reasons. These deviations from the ideal just increase the chance that any stated result is a “false negative” or a “false positive” beyond the stated statistical convention used in the study.

For each study, the tool provides a suggested algorithm to rate bias through a series of guiding questions across the following five sections. Each section ends with a bias assignment of “low,” “some concerns,” or “high.” Template rubrics provided by Cochrane with the answers to these guiding questions have been included for every study evaluated in Supplementary Material 9. At several instances, the suggested algorithm was overridden by the reviewer with justification annotated on the individual rubric found in Supplementary Material 9. Below is an explanation of how each subsection was evaluated with respect to aVNS, see (Higgins et al., 2019) for more information on the recommended implementation of the Cochrane assessment tool.

Each paper was read in its entirety and a summary table was completed capturing study motivation, study design, study results, and critical review. Study motivation outlined the study hypothesis and hypothesized therapeutic mechanism of action if mentioned in the paper. We also noted whether implantable VNS had achieved the hypothesized effect in humans. Study design encapsulated subject enrollment information (diseased or healthy, the power of the study, and the inclusion and exclusion criteria), type of control and blinding, group design (crossover or parallel), stimulation parameters, randomization, baseline comparison, and washout periods. Lastly, study results included primary and secondary endpoints, adverse effects, excluded and missing data, and statistical analysis details (pre-registered, handling of missing and incomplete data, multiple group comparison, etc.). Study results also analyzed if the effect was due to a few responders or improvements across the broad group, worsening of any subjects, control group effect size, and clinical relevance and significance of findings. Where sufficient information was reported, standardized effect size was calculated using Hedges' g (Turner and Bernard, 2006).

Each publication had a primary reviewer, and an additional secondary reviewer went through all papers. Any concerns raised by either reviewer were discussed as a group. If crucial basic information (e.g., which ear was stimulated, electrode used, etc.) was not reported (NR), an attempt was made to reach out to the author and if unsuccessful, to infer the information from similar studies by the group. Inferred or requested information is annotated as such. This effort helped highlight incomplete reporting of work while maximizing available information for the review to conduct an informed analysis.

A sortable table summarizing the design and result features of every reviewed study has been included as an excel file in Supplementary Material 3 to allow viewing based on specific features of interest. Design and result features have been reduced to common keywords in this spreadsheet to facilitate sorting; however, this means specific details of outcome measures have been reduced to general categories in some cases.

Upon initial review, it was immediately evident that implementation of both active and sham varied greatly across studies. Table 1 details the electrode design, configuration (monopolar or bipolar), target location, and stimulation parameters for the active and control arms of the study.

Table 1 is organized by indication type, then primary endpoints, then RoB score. Data is organized in the following columns:

The way in which studies were designed also varied greatly. Of the 41 RCTs reviewed, 20 used a crossover design while 21 opted for a parallel design. In terms of control group design, 19 studies used a sham, 17 a placebo, 2 used both sham and placebo, and 3 had no intervention as control. Studies also varied in duration: 12 were chronic and 29 were acute. The differences between these study design methods are important to emphasize and explored in section Long Term Solutions—Improvement in Control Design and Blinding.

Table 2 summarizes study designs and is organized by indication type, then primary endpoints, then RoB score. Data is organized in the following columns:

The Cochrane 2.0 Risk of Bias (RoB) assessment subscores for each study are summarized in Table 3. Explanations for each RoB subscore assignment [L = low (green), S = some concerns (orange), and H = high (red)] can be found generally explained in section Cochrane Risk of Bias 2.0 Tool to Assess Quality of Evidence and specifically explained for each study reviewed in Supplementary Material 9. The RoB tool provides a suggested algorithm to determine overall score based on the subscores of all sections. In several instances, the suggested algorithm was overridden by the reviewer with justification annotated on the individual rubric found in Supplementary Material 9.

Only two studies (Bauer et al., 2016; Maharjan et al., 2018) were assigned an overall “low” risk of bias. This is unsurprising, as the risk of bias assessment is rigorous and aVNS studies are in the less rigorous exploratory stages of investigation. Subsection and overall score percentages are illustrated in Figure 4.

The subsection “randomization process” was one of the best-scoring sections, in part because we assumed randomization was concealed from study investigators and subjects, even if the methodology to do so was not explicit. The studies that scored poorly in this section did not check for baseline imbalances between randomized groups or had baseline imbalances suggesting possible issues with the randomization method.

Notably, the “deviations from intended interventions” subsection tended to have the highest risk of bias. This was mainly due to issues with potential subject unblinding as a result of easily perceptible differences between active and control groups. For example, in a crossover design, the subject experiences both the paresthesia-inducing active intervention and the non-paresthesia-inducing placebo control. The difference in paresthesia may unblind the subject to the identity of the active vs. control interventions.

The subsection “missing outcome data” was generally scored as “low” risk of bias across studies. Studies scoring “high” or “some concerns” had unreported outcome data with a non-trivial difference in the proportion of missing data between interventions.

The “measurement of the outcome” subsection also generally scored a “low” risk of bias across studies. In order to score “some concerns” or worse, outcome assessor blinding to subject intervention had to be compromised. For a “high” RoB score in this section, studies measured endpoints that could be influenced by investigator unblinding, such as investigator-assessed disease evaluation questionnaires. Empirical measurements such as heart rate and blood pressure were less susceptible to this kind of bias and hence scored better.

The subsection with the highest risk for bias (“high” or “some concerns” scores) was “selection of the reported results.” This was primarily due to a lack of pre-registration in most studies. Suggestions to improve study reporting are listed in section Short Term Solutions—Guide to Reporting.

The RoB analysis revealed potential for bias in the outcomes of the studies reviewed. Here, the studies are grouped by outcome measures—cardiac, inflammatory, epilepsy, and pain. The outcome measures of the studies—regardless of indication—are qualitatively synthesized and couched in the findings of the RoB assessment. Findings from the RoB assessment are used to point out instances where trial design or reporting may have influenced interpretation of the trial outcomes. For reviews comprehensively synthesizing pre-clinical and both non-randomized and randomized clinical studies by indication, without emphasis on RoB, see Yap et al. (2020) and Jiang et al. (2020).

Several steps can be implemented immediately to increase the quality and consistency of reporting in aVNS studies, enabling comparison of results across studies.

Across the studies, the greatest risk of bias came from the Cochrane section “selection of reported results.” A comprehensive guide to clinical trial reporting is found published by the CONSORT group along with detailed elaborations (Moher et al., 2010). See Kovacic et al. (2017) for an aVNS study that followed the CONSORT reporting recommendations. Pre-registration of trials, use of appropriate statistical analysis, justifying clinical relevance of outcome measures, and contextualizing clinical significance of results are discussed here. These ideas are summarized in Table 8. If followed across aVNS studies, these suggestions would reduce the risk of bias identified in the RoB section reporting of results and enable the synthesis of knowledge by making reporting more comparable across studies (Farmer et al., 2020).

As the translation of drugs into clinical use is more established than neuromodulation therapies, it is instructive to review the translation of drugs for pitfalls in moving toward FDA market approved therapies. Less than 12% of drugs that received an FDA Investigational New Drug approval to begin human studies (the current stage of development of many aVNS based therapies) reached market approval (Paul et al., 2010; DiMasi et al., 2016). Gupta et al. (2011) identified several factors that hindered the successful translation of drug therapies from early-stage results to market approval, which are also relevant to aVNS therapies.

A method to directly measure local neural target engagement will provide an immediate biomarker of on- and off-target activity and forestall some of the hurdles encountered in drug therapy development. A minimally invasive method to measure target engagement percutaneously could be deployed across preclinical models and early clinical studies (Ottaviani et al., 2020). Doing so would increase the translatability of findings by providing data to titrate electrode design, placement, and stimulation waveform parameters to optimize for on-target engagement.

On- and off-target nerve activation is especially relevant in the case of the auricle that is innervated by several nerves with uncertainty on the specific areas of innervation in literature and across subjects. Data from direct measures of on- and off-target engagement could be used (1) to titrate the therapy by adjusting electrode design, placement, and stimulation waveform parameters to optimize on-target engagement, (2) to scale pre-clinical animal doses to humans by preserving the fiber types activated, and (3) to help investigate fundamental mechanisms of action by isolating local neural pathways.

aVNS is commonly delivered at the cymba concha with the assumption that the cymba concha is innervated only by the auricular vagus. This is based on two pieces of evidence. Firstly, Peuker and Filler (2002) showed the cymba concha is innervated only by the auricular vagus in 7 of 7 cadavers. Notwithstanding, there may be variation in innervation or spread of the electric field, which could activate the neighboring auriculotemporal branch of the trigeminal nerve and even the great auricular nerve. Variations in peripheral nerve innervation has been well-studied for other regions of the body, such as the hand (Bas and Kleinert, 1999; Guru et al., 2015). The reliance on the Peuker and Filler study is concerning due to its small sample size. Given the importance of the claims in Peuker and Filler, a further dissection study with a larger sample size is called for to investigate whether the results hold over a larger population of ethnically diverse individuals. Secondly, functional magnetic resonance imaging (fMRI) evidence is also used to suggest vagal innervation of the conchae (Frangos et al., 2015). However, fMRI is a surrogate measure of target engagement and is especially problematic when imaging deep in the brainstem, as described below.

Target engagement is commonly established using secondary surrogates such as fMRI, somatosensory evoked potentials (SSEPs), and cardiac measures. Secondary surrogates of target engagement are often contaminated with physical and biological noise, leading to potential confounds. For example, Botvinik-Nezer et al. (2020) and Becq et al. (2020) showed that results of fMRI studies were highly dependent on data processing techniques applied. In addition, pathways starting from the trigeminal nerve in the auricle also connect to NTS (Chiluwal et al., 2017), and activate the same region in the brain when in fact the auricular vagus might not be recruited during stimulation. fMRI of the brainstem is further complicated (Napadow et al., 2019) as distance from the measurement coils increases, the effective resolution decreases (Gruber et al., 2018). Still further, novelty, such as being stimulated in the ear or being in an MRI scanner, activates the locus coeruleus (LC) (Wagatsuma et al., 2018), which has connections with NTS. Thereby confounding potential aVNS effects on NTS with LC induced activity due to novelty effects. Lastly, fMRI has non-standard results between subjects requiring individual calibration and making subject to subject comparisons challenging. Additionally, SSEP recordings are sometimes contaminated and misinterpreted due to EMG leakage (Usami et al., 2013). The common measures of target engagement are secondary surrogates and prone to confounds—creating a need for direct measures of local target engagement at the nerve trunks innervating the auricle.

Given the lack of direct measures of local target engagement, aVNS studies rely largely on stimulation parameters that are similar to those used for implantable VNS. The assumption that these stimulation parameters will result in similar target engagement and therapeutic effects may not hold due to the differences in target fiber type, fiber orientation, and electrode design and contact area—all of which affect neural recruitment. Cardiac effects of implantable VNS are thought to be mediated by activation of parasympathetic efferent B fibers innervating the heart (Sabbah et al., 2011) or aortic baroreceptor afferents depending on the stimulation parameters used. Studies of the baroreceptors at the aorta and carotid sinus bulb identified fiber types consistent with Aδ and C fibers (Seagard et al., 1990; Reynolds et al., 2006). Strikingly, consensus workshops have suggested that aVNS is mediated by activation of Aβ fibers (Kaniusas et al., 2019). There is also no evidence of baroreceptors identified in the ear. These difference between the auricular and cervical vagus suggests a direct porting of stimulation parameters developed for implantable VNS would be insufficient to invoke cardiac responses, unless another yet unidentified mechanism for cardiac responses mediated through NTS is responsible, which can be targeted by Aβ fiber input that then indirectly modulates sympathetic or parasympathetic input to the heart. Unlike stimulation of the cervical vagus nerve trunk, where the electrode contacts are oriented parallel to the target axons, electrode contacts for aVNS do not have consistent orientation with respect to the target axons, which exist as a web of axons in the auricle. Orientation of fibers relative to the stimulation electrode have a large effect on fiber recruitment (Grill, 1999) and could potentially lead to preferential activation of nerve pathways oriented in a particular direction to the stimulation contacts, as well as inconsistent activation of specific fiber types across the auricle. Additionally, target fibers in the auricle transition to unmyelinated fibers as they approach sensory receptor cells (Provitera et al., 2007). For these reasons, while cathodic leading stimulation might have lower recruitment thresholds in implantable VNS, the principle may not hold for aVNS (Anderson et al., 2019). In aVNS, target fiber type, electrode design, electrode size, transcutaneous placement, and orientation of the target fiber relative to the electrode are different both compared to implantable VNS and across aVNS studies. Therefore, it is unsurprising that stimulation parameters ported from implantable VNS may not replicate the physiological effects or recruitment of fiber types that have been observed during implantable VNS.

In relation to electrode design, injected charge density, as opposed to current or voltage, is the most relevant metric of neural activation. This is because stimulation evoked action potentials occur in regions of the neural cell membrane where there is an elevated charge density (McNeal, 1976; Rattay, 1999). For effective comparison across studies using different electrodes, it is imperative to report on the electrode area, especially on the area as it makes contact with tissue, along with stimulation current.

The above discussion stresses a general lack of confidence in ascertaining which of several nerve trunks innervating the auricle are being activated during aVNS that may be generating the on- or off-target effects. Cakmak et al. (2017) further proposed that the therapeutic effects they reported for motor symptoms of Parkinson's diseases came from direct recruitment of the intrinsic auricular muscles instead of the auricular vagus nerve. The fundamentals of neural stimulation do not support directly porting stimulation parameters from implantable VNS. Therefore, it is important to understand which fiber types on the auricular vagus are being activated, if at all. This knowledge requires direct measures of local target engagement from the nerves innervating the auricle.

To further the development of aVNS, it will be essential to understand local target engagement of the nerve trunks innervating the ear. Ultrasound guided (Ritchie et al., 2016) percutaneous microelectrode recordings (Ottaviani et al., 2020) from the major nerve trunks innervating the ear, similar to the technique to measure muscle sympathetic nerve activity (MSNA), provides a way to directly measure local neural recruitment. Real-time data on neural target engagement would enable optimization of stimulation parameters, electrode, and control designs (Chang et al., 2020). These data would also improve translation of stimulation dosages from animal models to humans. Tsaava et al. (2020) showed that the anti-inflammatory effects of implantable VNS are stimulation dose dependent and could lead to the worsening of inflammation at certain dosages. Since target engagement in humans is currently unknown, there is no consistent means of determining dosage accurately—raising potential safety concerns. This minimally invasive method to record neural target engagement is already used clinically and could be rapidly translated to the clinic for use in titrating neuromodulation therapies.

Understanding primary target engagement at the ear will also further our understanding of aVNS mechanisms. For example, large animal recordings of evoked compound action potentials from the major nerve trunks innervating the ear may help in understanding the relationship between on- and off-target nerve engagement and corresponding physiological effects. Simultaneous recordings at the cervical vagus may allow differentiation of direct efferent vagal effects vs. NTS mediated effects, which would appear with a longer latency due to synaptic delay and longer conduction path length. Measuring neural target engagement at the auricle provides a first step to systematically studying aVNS mechanisms and optimizing clinical effects.

The design of an indistinguishable yet non-therapeutic control is central to maintaining the blinding in a clinical trial. Stemming from limited understanding of local target engagement and mechanism of action of aVNS, it is difficult to implement an active control (i.e., sham) that has similar perception to the therapeutic group but will not unknowingly engage a therapeutic pathway. This uncertainty in the therapeutic inertness of the control violates some of the basic premises for a RCT and makes it difficult to evaluate aVNS RCTs on the Oxford Scale for clinical evidence. Systematic effort must be made to design controls, which are key to maintaining blinding in aVNS RCTs.

Common control designs used in aVNS studies are summarized in Figure 5. A placebo is defined when the electrode placement and device are similar to the active intervention, but no stimulation is delivered. A waveform sham is defined when a different—non-therapeutic—waveform is delivered at the same location as active intervention. In a location sham, the same waveform as active intervention is delivered at a different location on the auricle and should not engage a therapeutic nerve. Lastly, no intervention or a pharmacological control may be used. Location sham was the most common control used in 16 of 41 RCTs reviewed. These different control designs are evaluated at length in Supplementary Material 4 along with recommendations on appropriate control types depending on trial design.

Inappropriate implementation of the control group resulted in compromised blinding in many studies. Subject unblinding occurred when subjects were able to feel a paresthesia in the active intervention but not in the control. For example, in cross-over trials, where subjects undergo both the active and control intervention, post-hoc assessment of blinding becomes critical given the perception of sensation being unequal could easily break the blind. Investigator unblinding occurred when investigators were able to see differences in electrode placement or device operation. Unblinding due to inappropriate control design is the main contributor leading to risk of bias in the Cochrane section “deviation from intended intervention.” The design of appropriate controls is difficult for trials testing non-pharmacological interventions—especially for paresthesia-inducing neuromodulation trials (Robbins and Lipton, 2017; Translating neuromodulation, 2019)—but is essential to establish a double blind.

To aid in the maintenance of the double blind, appropriate design of controls and consistent evaluation of blinding are required. Measurement of target engagement via microneurography of the major nerve trunks innervating the ear (Ottaviani et al., 2020) will enable understanding of neural recruitment occurring during active and sham stimulation and guide appropriate design of controls. Appropriate design of controls for non-invasive neuromodulation studies are discussed at length in Supplementary Material 4. In addition, post-hoc evaluation of blinding in subjects and investigators will gather knowledge on the blind quality setup by respective control designs. Methods to assess the quality of blinding in non-invasive neuromodulation studies are discussed in Supplementary Material 5. Three-armed trials (no treatment, placebo/sham, and treatment) will also generate data on extent of the placebo effect and quality of blinding (Howick et al., 2013). Over time, the consistent use of control designs and evaluation of blinding will grow our understanding of the concealability and therapeutic inertness of various control methods.

The discussion on lack of direct measures of target engagement and unknowns surrounding implementation of perceptually similar yet therapeutically inert controls to maintain the double blind are applicable to other neuromodulation therapies—especially paresthesia inducing therapies such as implantable VNS and spinal cord stimulation (SCS).

Study of local target engagement in neuromodulation therapies such as implantable VNS and SCS will inform stimulation parameters, possible mechanisms of action, and electrode design and placement to maximize on-target nerve recruitment and minimize therapy limiting off-target effects such as muscle contractions (Yoo et al., 2013; Nicolai et al., 2020). To investigate the central mechanisms of action we first have to establish local target engagement to determine which on- and off-target nerves are recruited during stimulation and which fiber types are recruited at therapeutically relevant levels of stimulation. Measurement of local target engagement in preclinical and early clinical studies provides a bottom-up approach to systematically develop and deploy neuromodulation therapies.

A systematic review and meta-analysis of SCS RCTs for pain showed that the quality of control used had an impact on the effect size of the outcome, (Duarte et al., 2020b). They concluded that thorough consideration of control design and consequent subject and investigator blinding is essential to improve the quality of evidence on SCS therapy for pain (Duarte et al., 2020a). A systematic review of dorsal root ganglion (DRG) stimulation for pain also showed serious concern for bias across all studies reviewed due to compromised subject and investigator blinding (Deer et al., 2020). The suggestions laid forth on measuring local target engagement and consistent post-hoc evaluation of blinding in subjects and investigators will expand our knowledge of effective control design for paresthesia inducing neuromodulation therapies such as aVNS, SCS, and implantable VNS.

This review is based on experience in other neuromodulation clinical trials and pre-clinical studies, existing frameworks for analysis such as the Cochrane Risk of Bias and Oxford clinical scale, and literature review motivated by an interest in conducting future aVNS clinical trials. However, at the writing of this article, none of the authors have conducted an aVNS clinical trial. Secondly, this is a systematic review but not a meta-analysis. Due to insufficient reporting in trials, a meta-analysis could not be conducted. The analysis was more qualitative with the intention of summarizing the quality of evidence in the field and making recommendations to improve clinical translatability. Thirdly, this review was not pre-registered, blinded, or formally randomized. Additionally, while the RoB tool provides a consistent method to evaluate trials where the shortcoming is stated explicitly, the ability to identify confounds is often reliant on the critical reading of the reviewer. This made it possible for a secondary reviewer to find additional risk of bias in several instances, which were initially missed by the primary reviewer but included upon identification and consensus. Lastly, numerous instances of missing information in trial reporting were identified and attempts were made to reach out to the authors for that information. These attempts were not always successful. Overall, the points made in the review are robust and withstand the limitations.