<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Archiving and Interchange DTD v2.3 20070202//EN" "archivearticle.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="systematic-review">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Sleep</journal-id>
<journal-title>Frontiers in Sleep</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Sleep</abbrev-journal-title>
<issn pub-type="epub">2813-2890</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/frsle.2023.1082253</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Sleep</subject>
<subj-group>
<subject>Systematic Review</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Memory retention following acoustic stimulation in slow-wave sleep: a meta-analytic review of replicability and measurement quality</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes" equal-contrib="yes">
<name><surname>Harlow</surname> <given-names>Tylor J.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<xref ref-type="author-notes" rid="fn001"><sup>&#x02020;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/2072993/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes" equal-contrib="yes">
<name><surname>Jan&#x000E9;</surname> <given-names>Matthew B.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="corresp" rid="c002"><sup>&#x0002A;</sup></xref>
<xref ref-type="author-notes" rid="fn001"><sup>&#x02020;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/2090748/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Read</surname> <given-names>Heather L.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/47984/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Chrobak</surname> <given-names>James J.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/79443/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Psychological Sciences, University of Connecticut</institution>, <addr-line>Storrs, CT</addr-line>, <country>United States</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Biomedical Engineering, University of Connecticut</institution>, <addr-line>Storrs, CT</addr-line>, <country>United States</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Dan Denis, University of York, United Kingdom</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Marina Wunderlin, Universit&#x000E4;re Psychiatrische Dienste Bern, Switzerland; Miguel Navarrete, St. Jude Children&#x00027;s Research Hospital, United States</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Tylor J. Harlow <email>tylor.harlow&#x00040;uconn.edu</email></corresp>
<corresp id="c002">Matthew B. Jan&#x000E9; <email>matthew.jane&#x00040;uconn.edu</email></corresp>
<fn fn-type="equal" id="fn001"><p>&#x02020;These authors share first authorship</p></fn></author-notes>
<pub-date pub-type="epub">
<day>11</day>
<month>05</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="collection">
<year>2023</year>
</pub-date>
<volume>2</volume>
<elocation-id>1082253</elocation-id>
<history>
<date date-type="received">
<day>28</day>
<month>10</month>
<year>2022</year>
</date>
<date date-type="accepted">
<day>11</day>
<month>04</month>
<year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2023 Harlow, Jan&#x000E9;, Read and Chrobak.</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Harlow, Jan&#x000E9;, Read and Chrobak</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license> </permissions>
<abstract>
<p>The role of slow oscillations and spindles during sleep on memory retention has become an area of great interest in the recent decade. Accordingly, there are multiple studies that examine the efficacy of acoustic stimulation during sleep to facilitate slow oscillations and associated memory retention. Here, we run meta-analyses on a current set of 14 studies that use audible noise-burst sound stimulation to modulate overnight retention of word pairs (<italic>k</italic><sub><italic>S</italic></sub> = 12 studies, <italic>k</italic><sub><italic>ES</italic></sub> = 14 effect sizes, <italic>n</italic> = 206 subjects). Our meta-analyses demonstrate a steady, yearly decline in effect size that accounts for 91.8% of the heterogeneity between studies. We find that the predicted effect on memory retention in 2013 favored the acoustic stimulation condition at <italic>d</italic><sub>&#x003B4;</sub> = 0.99 (95% CI [0.49, 1.49]), while the predicted effect in 2021 declined to a moderate and significant effect favoring no acoustic stimulation at <italic>d</italic><sub>&#x003B4;</sub> = &#x02212;0.39 (95% CI [&#x02212;0.73, &#x02212;0.05]). Our meta-regression model finds no coded study-level characteristics could account for the decline in effect sizes over time other than the publication date alone. Using available data, we estimate that 34% of subjects are not actually blind to the acoustic stimulation condition due to hearing acoustic stimulation during sleep. In addition, we find that the test-retest reliability of memory retention scores is nearly zero (<italic>&#x003C1;</italic><sub><italic>d</italic></sub> = 0.01, 95% CI [&#x02212;0.18, 0.21]), and through simulation demonstrate the impact this has on statistical power and observed effect sizes. Based on our analyses, we discuss the need for larger sample sizes, true placebo controls, age range restrictions, open-data sharing, and improvements in the reliability of memory retention tasks.</p></abstract>
<kwd-group>
<kwd>meta-analysis</kwd>
<kwd>slow-wave sleep</kwd>
<kwd>slow oscillations</kwd>
<kwd>acoustic stimulation</kwd>
<kwd>memory</kwd>
<kwd>reliability</kwd>
<kwd>decline effect</kwd>
</kwd-group>
<counts>
<fig-count count="3"/>
<table-count count="5"/>
<equation-count count="13"/>
<ref-count count="86"/>
<page-count count="18"/>
<word-count count="12695"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Sleep, Behavior and Mental Health</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="s1">
<title>1. Introduction</title>
<p>Converging lines of research support the theory that synchronous slow waves, spindles, and ripple oscillations and their coupling reflect the memory consolidation process that happens during sleep. In general, performance on declarative (verbal, event, and place) and non-declarative (sensory and motor skill) memory tasks improves following sleep (Barrett and Ekstrand, <xref ref-type="bibr" rid="B3">1972</xref>; Plihal and Born, <xref ref-type="bibr" rid="B59">1997</xref>; Mednick et al., <xref ref-type="bibr" rid="B43">2003</xref>, <xref ref-type="bibr" rid="B44">2011</xref>; Ellenbogen et al., <xref ref-type="bibr" rid="B15">2007</xref>; Nishida and Walker, <xref ref-type="bibr" rid="B54">2007</xref>; Miyamoto et al., <xref ref-type="bibr" rid="B46">2017</xref>). An increase in the density of synchronous slow cortical oscillations (0.5&#x02013;1.0 Hz) and thalamic spindles (12&#x02013;15 Hz) during sleep is correlated with performance on various memory tasks following sleep (De Gennaro and Ferrara, <xref ref-type="bibr" rid="B9">2003</xref>; Fogel and Smith, <xref ref-type="bibr" rid="B17">2011</xref>; Niknazar et al., <xref ref-type="bibr" rid="B53">2015</xref>; Cowan et al., <xref ref-type="bibr" rid="B7">2020</xref>). The density of slow waves is highest during stages 2 and 3 of non-rapid eye movement (NREM) sleep (Stokes et al., <xref ref-type="bibr" rid="B75">2022</xref>). In contrast, the density of fast spindle frequency oscillations is highest during stage 2 sleep when spindles are phase coupled with slow waves creating temporally coherent k-complex events (De Gennaro and Ferrara, <xref ref-type="bibr" rid="B9">2003</xref>; Fogel and Smith, <xref ref-type="bibr" rid="B17">2011</xref>; Stokes et al., <xref ref-type="bibr" rid="B75">2022</xref>). Intracranially, slow waves arise from alternating phases of high and low spike-rate output from cortical neurons (Massimini et al., <xref ref-type="bibr" rid="B42">2004</xref>; Steriade, <xref ref-type="bibr" rid="B74">2006</xref>; Sanchez-Vives, <xref ref-type="bibr" rid="B67">2020</xref>). In contrast, spindles are driven by thalamocortical circuits (Steriade, <xref ref-type="bibr" rid="B74">2006</xref>). Thalamocortical spindles are phase coupled to the positive phase of cortical slow waves (Steriade, <xref ref-type="bibr" rid="B74">2006</xref>; Diekelmann and Born, <xref ref-type="bibr" rid="B11">2010</xref>). This coupling between slow waves and spindles positively correlates with improved sleep-dependent memory over the course of development and in adulthood (Hahn et al., <xref ref-type="bibr" rid="B18">2020</xref>; Kurz et al., <xref ref-type="bibr" rid="B29">2020</xref>). Computational models support a physiological mechanism where cortical output during the negative phase of the slow wave suppresses thalamic spindles and release from suppression in transition to the positive phase of slow waves promotes synchronous spindles (Mak-McCully et al., <xref ref-type="bibr" rid="B37">2014</xref>). At the synaptic level, slow waves and spindles are thought to have complementary effects on synaptic plasticity which ultimately improves the signal-to-noise ratio of synapses and networks mediating memory (Tononi and Cirelli, <xref ref-type="bibr" rid="B77">2006</xref>; Miyamoto et al., <xref ref-type="bibr" rid="B46">2017</xref>). Spindles in turn are coupled with faster synchronous ripple oscillations (80&#x02013;250 Hz) generated by hippocampal networks during non-declarative memory acquisition and during sleep (Chrobak and Buzs&#x000E1;ki, <xref ref-type="bibr" rid="B6">1998</xref>; Diekelmann and Born, <xref ref-type="bibr" rid="B11">2010</xref>; Miyamoto et al., <xref ref-type="bibr" rid="B46">2017</xref>) The increase in coupled activity between recurrently connected hippocampal and neocortical brain networks during sleep is thought to mediate the transfer or consolidation of memories. Accordingly, phase-coupling of slow wave, spindle, and ripple oscillations across cortical, thalamic, and hippocampal networks all are considered biomarkers of memory consolidation (Isomura et al., <xref ref-type="bibr" rid="B25">2006</xref>; Staresina et al., <xref ref-type="bibr" rid="B73">2015</xref>; Rothschild et al., <xref ref-type="bibr" rid="B66">2017</xref>; Navarrete et al., <xref ref-type="bibr" rid="B49">2020</xref>).</p>
<p>There is a growing interest in developing effective approaches to enhance slow oscillations and their coupling with spindles to improve memory consolidation in various clinical populations. Populations with altered spindle activity include schizophrenia (Wamsley et al., <xref ref-type="bibr" rid="B80">2012</xref>; Stokes et al., <xref ref-type="bibr" rid="B75">2022</xref>), Alzheimer&#x00027;s disease (De Gennaro and Ferrara, <xref ref-type="bibr" rid="B9">2003</xref>; Rauchs et al., <xref ref-type="bibr" rid="B63">2008</xref>; Weng et al., <xref ref-type="bibr" rid="B82">2020</xref>), Autism Spectrum Disorders (Limoges et al., <xref ref-type="bibr" rid="B33">2005</xref>; Tessier et al., <xref ref-type="bibr" rid="B76">2015</xref>), sleep disorders (Leong et al., <xref ref-type="bibr" rid="B32">2022</xref>) as well as natural aging (Martin et al., <xref ref-type="bibr" rid="B41">2013</xref>; Mander et al., <xref ref-type="bibr" rid="B39">2017</xref>; Purcell et al., <xref ref-type="bibr" rid="B61">2017</xref>; Helfrich et al., <xref ref-type="bibr" rid="B21">2018</xref>; Djonlagic et al., <xref ref-type="bibr" rid="B14">2021</xref>). Historically, drugs have been developed to facilitate sleep physiology and memory. Multiple studies find the sleep aid, zolpidem, increases the density and power of sleep spindles (Dijk et al., <xref ref-type="bibr" rid="B13">2010</xref>; Lundahl et al., <xref ref-type="bibr" rid="B35">2012</xref>; Mednick et al., <xref ref-type="bibr" rid="B45">2013</xref>; Niknazar et al., <xref ref-type="bibr" rid="B53">2015</xref>; Zhang et al., <xref ref-type="bibr" rid="B86">2020</xref>) and the coupling of slow wave and spindle oscillations (Niknazar et al., <xref ref-type="bibr" rid="B53">2015</xref>; Zhang et al., <xref ref-type="bibr" rid="B86">2020</xref>; Leong et al., <xref ref-type="bibr" rid="B32">2022</xref>). Moreover, zolpidem strengthens hippocampal-prefrontal network coupling (Kersant&#x000E9; et al., <xref ref-type="bibr" rid="B27">2023</xref>) and is correlated with improved memory task performance on the following day (Niknazar et al., <xref ref-type="bibr" rid="B53">2015</xref>; Zhang et al., <xref ref-type="bibr" rid="B86">2020</xref>; Leong et al., <xref ref-type="bibr" rid="B32">2022</xref>). Unfortunately, zolpidem also enhances negative emotional memory consolidation and a meta-study finds zolpidem use is associated with increased suicide rates (Simon et al., <xref ref-type="bibr" rid="B71">2021</xref>; Khan et al., <xref ref-type="bibr" rid="B28">2022</xref>). Thus, zolpidem may only be an effective approach for enhancing slow waves, spindles, and memory in certain subpopulations. Multiple non-pharmacological approaches are under investigation including non-phase-locked or feedback-controlled transcranial electrical stimulation to increase slow wave oscillation amplitudes and improve overnight memory consolidation (Marshall et al., <xref ref-type="bibr" rid="B40">2006</xref>; Lustenberger et al., <xref ref-type="bibr" rid="B36">2016</xref>). Another approach is acoustic stimulation phase-locked to slow oscillations which enhances slow wave and spindle oscillations during sleep and can improve memory consolidation (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>). Finally, a variation of the latter approach is to play sound as a contextual cue while people are learning and then play the same sound for &#x0201C;targeted memory reactivation&#x0201D; and consolidation during sleep (Cairney et al., <xref ref-type="bibr" rid="B4">2016</xref>; Hu et al., <xref ref-type="bibr" rid="B24">2020</xref>).</p>
<p>Over the past 10 years, fourteen studies have examined the efficacy of using acoustic stimulation during sleep to enhance slow wave and spindle oscillations and memory. One of the first studies to examine both the physiological and behavioral effects of overnight acoustic stimulation is a seminal study by Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>). This study plays sequences of two pink noise sound bursts phase-locked to the positive phase of an ongoing slow oscillation (SO) during the first 2 h of overnight NREM (stage 2, 3) sleep and measures how this impacts performance on a declarative word-pair memory task the next day (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>). Here, the inter-stimulus interval between two noise bursts is set to each individual subject&#x00027;s average slow wave interval. Two outcome measures include the physiologic metric of SO amplitude and the behavioral metric of performance on a memory task. Positive outcomes include a significant increase in SO and spindle amplitudes and improved performance on a word-pair memory task with acoustic stimulation vs. control condition without stimulation (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>). In this initial study, there is a substantial effect size (<italic>d</italic> = 1.08) supporting this non-pharmacological approach for improving verbal memory performance. In the past 10 years, seven clinical studies in total have examined the effects of phase-locked acoustic stimulation during sleep on word-pair memory in young adults or children (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>, <xref ref-type="bibr" rid="B52">2015</xref>; Ong et al., <xref ref-type="bibr" rid="B55">2016</xref>; Leminen et al., <xref ref-type="bibr" rid="B31">2017</xref>; Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>). Five out of these seven studies find phase-locked acoustic stimulation during sleep increases SO amplitudes and improves performance on word-pair memory tasks in children (Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>) and young adults that do not have diagnosed clinical conditions (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>, <xref ref-type="bibr" rid="B52">2015</xref>; Ong et al., <xref ref-type="bibr" rid="B55">2016</xref>; Leminen et al., <xref ref-type="bibr" rid="B31">2017</xref>). Two of the seven studies examining phase-locked acoustic stimulation effects on adult populations do not find significant improvements in word-pair memory (Henin et al., <xref ref-type="bibr" rid="B22">2019</xref>; Schneider et al., <xref ref-type="bibr" rid="B69">2020</xref>; Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>). Similar to some prior studies (Henin et al., <xref ref-type="bibr" rid="B22">2019</xref>; Diep et al., <xref ref-type="bibr" rid="B12">2020</xref>; Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>), the 2021 study by Harrington and colleagues uses a different type of word-pair memory test which requires subjects to remember unfamiliar word pairs. Based on this, they suggest that overnight SO phase-locked acoustic stimulation is more effective for consolidation (or reconsolidation) of familiar word pairs (Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>). Similarly, other studies find phase-locked acoustic stimulation that significantly increases slow wave and spindle amplitudes does not improve performance on spatial memory (Henin et al., <xref ref-type="bibr" rid="B22">2019</xref>), visual object and facial memory, or non-declarative finger tapping procedural memory tasks (Leminen et al., <xref ref-type="bibr" rid="B31">2017</xref>). Given that different types of memory engage distinct brain areas and neuronal networks, it is possible that the current acoustic stimulation design is optimal for certain types of memory consolidation. To explore this possibility, Henin et al. (<xref ref-type="bibr" rid="B22">2019</xref>) set out to compare how phase-locked acoustic stimulation impacts both word-pair and spatial memory tasks using the same word-pair memory task employed by prior studies including that of Ngo and colleagues. Surprisingly, the Henin et al. (<xref ref-type="bibr" rid="B22">2019</xref>) study did not find a significant increase in performance with congruent or incongruent word-pairs in spite of observing significant increases in SO and spindle amplitudes for both. Collectively, there are five studies that support a phase-locked acoustic stimulation approach for enhancing declarative verbal memory but there remains considerable variability across all the studies completed to date.</p>
<p>Two recent meta-analytic studies (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>) examine potential study design moderators that could account for variability in effects across studies using the acoustic stimulation approach to enhance memory consolidations. Collectively, these meta-analytic studies consider age, phase-locking, and the type of memory task as potential contributors to variations in the efficacy of acoustic stimulation on memory. It is challenging to attribute the low effect size across studies to age alone because there are only three out of 14 current studies using older cohorts. For example, Papalambros et al. (<xref ref-type="bibr" rid="B56">2017</xref>) find a cohort of older adults (mean age 75.2 years) benefits from phase-locked acoustic stimulation showing improved verbal memory and substantial effect size (<italic>d</italic> = 0.63) on the order of that found with younger cohorts (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>). In contrast, Schneider et al. (<xref ref-type="bibr" rid="B69">2020</xref>) find a cohort of older adults (mean age = 54.6 years) does not show enhanced memory performance following phase-locked acoustic stimulation. A <italic>post-hoc</italic> comparison by Schneider and colleagues finds that with or without acoustic stimulation SO amplitudes are smaller for older (mean age = 54.6 years) vs. younger (age 24.2 years) cohorts. Based on this result, Schneider and colleagues suggest the physiological capacity to generate and augment slow-wave oscillations in older populations may be reduced. The use of phase-locked vs. non-phase-locked acoustic stimulation is another potential contributor. However, there are not many studies probing memory effects with non-phase-locked stimulation or with older age cohorts. Indeed, Wunderlin and colleagues consider the combination of age and phase-locking as potential moderators for study outcomes. Accordingly, when combining studies using phase-locked acoustic stimulation and younger cohorts, the memory performance effect size increases almost two-fold larger than the overall mean (<italic>d</italic> = 0.25, 95% CI [&#x02212;0.02, 0.53 ] to <italic>d</italic> = 0.44, 95% CI [0.09, 0.79]). This suggests that phase-locked acoustic stimulation in younger adults could be an effective tool to modulate memory retention. Toward this end, Wunderlin and colleagues compare portable systems that allow for multiple nights of electroencephalogram (EEG) recordings and phase-locked acoustic stimulation delivery with the goal of collecting quality data over multiple nights to strengthen future studies (Zeller et al., <xref ref-type="bibr" rid="B85">2023</xref>). A meta-analysis by Stanyer and colleagues reports a memory performance effect size (<italic>d</italic> = 0.68, 95% CI [0.06, 1.30]) that is almost three-fold larger than reported for a similar set of studies in the meta-analysis by Wunderlin and colleagues. Nevertheless, Stanyer and colleagues also recommend future studies should include repeated measures and larger sample sizes to reduce between-study heterogeneity and help the future development of this novel approach. Collectively, these recent meta-analyses point toward potential ways to optimize future acoustic stimulation studies designed to improve memory and other cognitive operations.</p>
<p>In this review, we perform a meta-analysis of fourteen clinical studies completed between 2013 and 2023 that deliver acoustic stimulation during NREM sleep and measure the effects on slow wave amplitude and memory task performance. Specifically, we aimed to address the following four research questions:</p>
<list list-type="simple">
<list-item><p>R1. What is the average effect of acoustic stimulation during slow-wave sleep on sleep-dependent changes in word-pair retention?</p></list-item>
<list-item><p>R2. Does the effect size change with subsequent replications?</p></list-item>
<list-item><p>R3. If there are changes in the reported effects in subsequent replications, is this attributable to any codable study characteristics (e.g., mean age of sample, semantic congruence of word pairs, blinding procedures, change in SO power)?</p></list-item>
<list-item><p>R4. Are word-pair cued-recall tasks reliable enough to demonstrate changes in memory retention if they are present?</p></list-item>
</list>
<p>In order to address these questions, first we reproduce and extend prior meta-analyses by including three additional datasets from two new publications. We confirm the positive memory effects reported previously by Wunderlin et al. (<xref ref-type="bibr" rid="B84">2021</xref>) when including the same subset of studies using phase-locked acoustic stimulation and non-elderly adults. Upon including two new studies, one of which includes children and a second that includes an incongruent word-pair task, we find a significant downward trend in the effects of phase-locked stimulation on memory performance between 2013 and 2023. We run a meta-regression model examining eight potential moderators driving this downward trend including age, gender, phase-locked condition, properties of memory tasks (word count and word congruence), sleep condition (overnight vs. nap), double-blind procedures, and physiological increase in slow waves. None of these study-level characteristics account for this trend. In addition, difference scores are known to be highly unreliable in cognitive performance assessments, including the Stroop test (Hedge et al., <xref ref-type="bibr" rid="B20">2018</xref>). Here, we examine the test-retest reliability of the difference scores (pre-sleep vs. post-sleep) used in all these studies to measure changes in memory retention. We find that these difference scores have low reliability and a strong potential to decrease statistical power. Accordingly, we hypothesize that unreliable memory task performance difference scores likely contribute a large degree of heterogeneity across studies testing acoustic stimulus effects on memory. Finally, we discuss how the memory task score, statistical approach, sample size, and other study design features may be improved to strengthen future research investigating the efficacy of acoustic stimulation to boost slow waves, spindle oscillations, and improve memory consolidation.</p></sec>
<sec id="s2">
<title>2. Methods and materials</title>
<sec>
<title>2.1. Study selection</title>
<p>The goal of the current meta-analysis is to confirm and extend the work of two previous meta-analytic reviews (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>). Here, we combine studies included in these two meta-analyses (<xref ref-type="fig" rid="F1">Figure 1A</xref>). Additionally, we compare the statistical measures across our study and the two prior meta-analyses and confirm a high correspondence with Wunderlin and colleagues (<xref ref-type="fig" rid="F1">Figure 1B</xref>) but less so with Stanyer and colleagues (<xref ref-type="table" rid="T1">Table 1</xref> and <xref ref-type="fig" rid="F1">Figure 1C</xref>). To expand on these prior studies, we conducted a supplemental google scholar search of the last 3 years (2020&#x02013;2023), using the following key-word search: <italic>(&#x0201C;acoustic stimulation&#x0201D; OR &#x0201C;auditory stimulation&#x0201D;) AND (&#x0201C;slow wave sleep&#x0201D; OR &#x0201C;slow oscillations&#x0201D;) AND (&#x0201C;memory&#x0201D; OR &#x0201C;consolidation&#x0201D;)</italic>. The first 50 most relevant results are given a full-text screening for inclusion. This results in two relevant articles (Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>; Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>). Harrington et al. (<xref ref-type="bibr" rid="B19">2021</xref>) is likely excluded from previous meta-analyses (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>) as a consequence of being published after literature review. Prehn-Kristensen et al. (<xref ref-type="bibr" rid="B60">2020</xref>) may have been excluded as it compares subjects with and without attention deficit hyperactivity disorder (ADHD). Here, we extract the necessary effect size for the healthy control subjects for both the rewarded and non-rewarded conditions, which are included in the present meta-analysis. Additionally, we searched for studies that had cited the already-included articles and found no additional studies. In sum, we obtained <italic>k</italic><sub><italic>S</italic></sub> = 12 studies (<italic>k</italic><sub><italic>ES</italic></sub> = 14 effect sizes), and <italic>N</italic> = 206 healthy subjects of variable age (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 1</xref>).</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Standardized mean differences in the current meta-analysis and previous meta-analyses. <bold>(A)</bold> All effect sizes in the current meta-analysis estimated using Glass&#x00027; estimator. <bold>(B, C)</bold> Standardized mean differences (SMD) reported from previous meta-analyses (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>) plotted against reproduced estimates using their respective methodologies. Purple lines indicate the mean of the reproduced and reported SMD where the intersection demonstrates how they diverge from equality.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="frsle-02-1082253-g0001.tif"/>
</fig>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Standardized mean difference estimates and reproduced estimates from previous meta-analyses.</p></caption> 
<table frame="box" rules="all">
<thead><tr style="background-color:#919497;color:#ffffff">
<th/>
<th valign="top" align="left"><bold>Current (<italic>d</italic><sub>&#x003B4;</sub>)</bold></th>
<th valign="top" align="center" colspan="3"><bold>Wunderlin (</bold><italic><bold>d</bold></italic><sub><bold><italic><bold>rm</bold></italic></bold></sub><bold>)</bold></th>
<th valign="top" align="center" colspan="3"><bold>Stanyer (</bold><italic><bold>d</bold></italic><sub><bold><italic><bold>av</bold></italic></bold></sub><bold>)</bold></th>
</tr>
<tr style="background-color:#919497;color:#ffffff">
<td valign="top" align="left"><bold>References</bold></td>
<td valign="top" align="center"><bold>Estimate</bold></td>
<td valign="top" align="center"><bold>Reported</bold></td>
<td valign="top" align="center"><bold>Reproduced</bold></td>
<td valign="top" align="center"><bold>Diff</bold></td>
<td valign="top" align="center"><bold>Reported</bold></td>
<td valign="top" align="center"><bold>Reproduced</bold></td>
<td valign="top" align="center"><bold>Diff</bold></td>
</tr> 
</thead>
<tbody>
<tr>
<td valign="top" align="left">Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>)</td>
<td valign="top" align="center">1.03</td>
<td valign="top" align="center">1.07</td>
<td valign="top" align="center">1.07</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">3.68</td>
<td valign="top" align="center">1.07</td>
<td valign="top" align="center">2.61</td>
</tr> <tr>
<td valign="top" align="left">Ngo et al. (<xref ref-type="bibr" rid="B52">2015</xref>)</td>
<td valign="top" align="center">0.71</td>
<td valign="top" align="center">0.64</td>
<td valign="top" align="center">0.65</td>
<td valign="top" align="center">0.01</td>
<td valign="top" align="center">2.66</td>
<td valign="top" align="center">0.65</td>
<td valign="top" align="center">2.01</td>
</tr> <tr>
<td valign="top" align="left">Ong et al. (<xref ref-type="bibr" rid="B55">2016</xref>)</td>
<td valign="top" align="center">0.39</td>
<td valign="top" align="center">0.41</td>
<td valign="top" align="center">0.41</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.98</td>
<td valign="top" align="center">0.41</td>
<td valign="top" align="center">0.47</td>
</tr> <tr>
<td valign="top" align="left">Weigenand et al. (<xref ref-type="bibr" rid="B81">2016</xref>)</td>
<td valign="top" align="center">0.03</td>
<td valign="top" align="center">0.03</td>
<td valign="top" align="center">0.03</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.03</td>
<td valign="top" align="center">0.03</td>
<td valign="top" align="center">0.00</td>
</tr> <tr>
<td valign="top" align="left">Papalambros et al. (<xref ref-type="bibr" rid="B56">2017</xref>)</td>
<td valign="top" align="center">1.12</td>
<td valign="top" align="center">0.77</td>
<td valign="top" align="center">0.69</td>
<td valign="top" align="center">0.08</td>
<td valign="top" align="center">&#x02212;0.35</td>
<td valign="top" align="center">0.76</td>
<td valign="top" align="center">1.11</td>
</tr> <tr>
<td valign="top" align="left">Leminen et al. (<xref ref-type="bibr" rid="B31">2017</xref>)</td>
<td valign="top" align="center">0.64</td>
<td valign="top" align="center">0.66</td>
<td valign="top" align="center">0.66</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.68</td>
<td valign="top" align="center">0.66</td>
<td valign="top" align="center">0.02</td>
</tr> <tr>
<td valign="top" align="left">Henin et al. (<xref ref-type="bibr" rid="B22">2019</xref>) [1]</td>
<td valign="top" align="center">&#x02212;0.12</td>
<td valign="top" align="center">&#x02212;0.15</td>
<td valign="top" align="center">&#x02212;0.15</td>
<td valign="top" align="center">0.00</td>
<td/>
<td valign="top" align="center">&#x02212;0.15</td>
<td/>
</tr> <tr>
<td valign="top" align="left">Henin et al. (<xref ref-type="bibr" rid="B22">2019</xref>) [2]</td>
<td valign="top" align="center">&#x02212;0.04</td>
<td valign="top" align="center">&#x02212;0.04</td>
<td valign="top" align="center">&#x02212;0.04</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">1.16</td>
<td valign="top" align="center">&#x02212;0.04</td>
<td valign="top" align="center">1.20</td>
</tr> <tr>
<td valign="top" align="left">Choi et al. (<xref ref-type="bibr" rid="B5">2019</xref>)</td>
<td valign="top" align="center">&#x02212;0.48</td>
<td valign="top" align="center">&#x02212;0.29</td>
<td valign="top" align="center">&#x02212;0.37</td>
<td valign="top" align="center">0.08</td>
<td valign="top" align="center">&#x02212;0.01</td>
<td valign="top" align="center">&#x02212;0.38</td>
<td valign="top" align="center">0.37</td>
</tr> <tr>
<td valign="top" align="left">Prehn-Kristensen et al. (<xref ref-type="bibr" rid="B60">2020</xref>) [1]</td>
<td valign="top" align="center">0.34</td>
<td/>
<td valign="top" align="center">0.23</td>
<td/>
<td/>
<td valign="top" align="center">0.24</td>
<td/>
</tr> <tr>
<td valign="top" align="left">Prehn-Kristensen et al. (<xref ref-type="bibr" rid="B60">2020</xref>) [2]</td>
<td valign="top" align="center">&#x02212;0.45</td>
<td/>
<td valign="top" align="center">&#x02212;0.41</td>
<td/>
<td/>
<td valign="top" align="center">&#x02212;0.41</td>
<td/>
</tr> <tr>
<td valign="top" align="left">Schneider et al. (<xref ref-type="bibr" rid="B69">2020</xref>)</td>
<td valign="top" align="center">&#x02212;0.59</td>
<td valign="top" align="center">&#x02212;0.51</td>
<td valign="top" align="center">&#x02212;0.51</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.61</td>
<td valign="top" align="center">&#x02212;0.51</td>
<td valign="top" align="center">0.10</td>
</tr> <tr>
<td valign="top" align="left">Diep et al. (<xref ref-type="bibr" rid="B12">2020</xref>)</td>
<td valign="top" align="center">0.13</td>
<td valign="top" align="center">0.12</td>
<td valign="top" align="center">0.12</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.60</td>
<td valign="top" align="center">0.12</td>
<td valign="top" align="center">0.48</td>
</tr> <tr>
<td valign="top" align="left">Harrington et al. (<xref ref-type="bibr" rid="B19">2021</xref>)</td>
<td valign="top" align="center">&#x02212;0.58</td>
<td/>
<td valign="top" align="center">&#x02212;0.57</td>
<td/>
<td/>
<td valign="top" align="center">&#x02212;0.57</td>
<td/>
</tr> <tr>
<td valign="top" align="left">Mean</td>
<td valign="top" align="center">0.15</td>
<td valign="top" align="center">0.25</td>
<td valign="top" align="center">0.13</td>
<td valign="top" align="center">0.02</td>
<td valign="top" align="center">1.00</td>
<td valign="top" align="center">0.13</td>
<td valign="top" align="center">0.84</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p>Effect size estimates in the current meta-analysis are compared against the reported effect sizes with previous meta-analyses. The reported effect sizes from Wunderlin et al. (<xref ref-type="bibr" rid="B84">2021</xref>) and Stanyer et al. (<xref ref-type="bibr" rid="B72">2022</xref>) are compared against our replication of their own calculations. The first column corresponds to our current use of Glass&#x00027;s estimator (<italic>d</italic><sub>&#x003B4;</sub>). The absolute differences between our reproduced estimates and the reported estimates from the original meta-analyses are listed under &#x0201C;Diff.&#x0201D; The mean of each column is an unweighted average of all elements.</p>
</table-wrap-foot>
</table-wrap>
</sec>
<sec>
<title>2.2. Experimental study designs</title>
<p>Key study design features were aligned across all studies included. First, all studies included examined the efficacy of noise-burst acoustic stimulation during slow-wave sleep on overnight memory retention in the form of a word-pair cued recall task. Two experimental sessions consisted of a SHAM condition (control, no acoustic stimulation) and a STIM condition (with acoustic stimulation) during sleep (nap or overnight). The studies all had repeated measures and cross-over designs. That is, the same subjects participated in the control (SHAM, no audio) and acoustic stimulation (STIM) conditions. Ordering of STIM and SHAM conditions was appropriately counterbalanced in all studies. Word pairs were shown to participants, and they were subsequently assessed on their immediate recall prior to the sleep session. After waking, memory was assessed on the same word pairs. Memory retention scores were the difference in recall accuracy between pre-sleep and post-sleep assessments in all studies included. Memory retention scores were then compared between STIM and SHAM conditions.</p></sec>
<sec>
<title>2.3. Data collection</title>
<p>Data included in this meta-analysis was obtained from each publication or by contacting the corresponding authors. Relevant study characteristics were recorded (see <xref ref-type="table" rid="T2">Table 2</xref>) alongside effect size information. Fortunately, we were able to collect all characteristics of each moderator of each study. Most coded characteristics were directly reported in the respective manuscripts. However, slow oscillation power was not, therefore we calculated the standardized mean difference (SMD) in slow oscillation power/amplitude between STIM and SHAM. To quantify the effect of acoustic stimulation on memory retention, the standardized mean differences in word-pair retention scores between STIM and SHAM conditions were recorded. To obtain repeated measures effect sizes, correlation coefficients must be calculated between pre-sleep and post-sleep measurements. Since most studies did not report pre-sleep vs. post-sleep correlations, corresponding authors were contacted in order to acquire raw data, or data was extracted from figures containing sufficient information using WebPlotDigitizer. We were able to obtain raw data sets for seven independent samples from five studies (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>, <xref ref-type="bibr" rid="B52">2015</xref>; Weigenand et al., <xref ref-type="bibr" rid="B81">2016</xref>; Henin et al., <xref ref-type="bibr" rid="B22">2019</xref>; Schneider et al., <xref ref-type="bibr" rid="B69">2020</xref>). Two samples were included from Ngo et al. (<xref ref-type="bibr" rid="B52">2015</xref>), but the second experiment had no SHAM condition therefore this sample is not used in the effect size calculations, however it is included in the reliability section of the meta-analysis. Since each study implemented a crossover design, we were able to estimate reliability coefficients for each of the seven raw data sets. Additionally, five studies reported the proportion of subjects that were aware of the stimulus at some point during the STIM condition (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>, <xref ref-type="bibr" rid="B52">2015</xref>; Weigenand et al., <xref ref-type="bibr" rid="B81">2016</xref>; Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>; Schneider et al., <xref ref-type="bibr" rid="B69">2020</xref>). This proportion of subjects aware of the acoustic stimulus at any point was extracted from these five studies to examine the possibility that subjects were potentially not entirely blind to study conditions due to the no audio vs. acoustic stimulation designs employed by all (<xref ref-type="fig" rid="F2">Figure 2C</xref>).</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Description of relevant moderators.</p></caption> 
<table frame="box" rules="all">
<thead><tr style="background-color:#919497;color:#ffffff">
<th valign="top" align="left"><bold>Moderator</bold></th>
<th valign="top" align="left"><bold>Description</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Publication date</td>
<td valign="top" align="left">Date of publication</td>
</tr> <tr>
<td valign="top" align="left">Age</td>
<td valign="top" align="left">Mean age of participants</td>
</tr> <tr>
<td valign="top" align="left">Proportion female</td>
<td valign="top" align="left">Proportion of participants that are female</td>
</tr> <tr>
<td valign="top" align="left">Phase-locking</td>
<td valign="top" align="left">Phase-locked (1) or non-phase-locked (0) stimulation</td>
</tr> <tr>
<td valign="top" align="left">Word count</td>
<td valign="top" align="left">Number of words during word-pair task</td>
</tr> <tr>
<td valign="top" align="left">Congruent words</td>
<td valign="top" align="left">Semantically congruent (1) or incongruent (0) word-pairs</td>
</tr> <tr>
<td valign="top" align="left">Overnight</td>
<td valign="top" align="left">Whole-night (1) or nap period (0)</td>
</tr> <tr>
<td valign="top" align="left">&#x00394; SO power</td>
<td valign="top" align="left">Change in slow oscillation amplitude/power</td>
</tr> <tr>
<td valign="top" align="left">Double-blind</td>
<td valign="top" align="left">Experimenter or Investigator blinding to experimental condition</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p>Provides a brief description of each study-level moderator used in the meta-regression and subgroup analyses. A full description of all moderators for each study can be found at: <ext-link ext-link-type="uri" xlink:href="https://osf.io/8b49k/">https://osf.io/8b49k/</ext-link>.</p>
</table-wrap-foot>
</table-wrap>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>Effect decline and heterogeneity. <bold>(A)</bold> Meta-regression for publication date to observed effects across studies. Data-labels indicate first author of respective publication. <bold>(B)</bold> Results of a leave-one-out cross-validation (leave-one-out-cross-validation) where each regression coefficient is the fitted slope estimate when the corresponding study is removed. Note the resulting regression coefficients are similar indicating that no one study has major influence on the regression slope parameter. <bold>(C)</bold> The proportion of people reported hearing the auditory stimuli during the night when it was truly the STIM condition.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="frsle-02-1082253-g0002.tif"/>
</fig></sec>
<sec>
<title>2.4. Effect-size calculation</title>
<p>Previous meta-analyses reported different formulations of standardized mean differences (SMD) in word-pair retention scores between STIM and SHAM conditions. Wunderlin et al. (<xref ref-type="bibr" rid="B84">2021</xref>) used a repeated measures estimator (<italic>d</italic><sub><italic>rm</italic></sub>), whereas Stanyer et al. (<xref ref-type="bibr" rid="B72">2022</xref>) presumably used an average variance estimator (<italic>d</italic><sub><italic>av</italic></sub>). Our meta-analysis finds that variances in STIM and SHAM conditions violate the assumption of equal variances. This is evidenced by the average (<italic>n</italic>-weighted) variance ratio of 1.38 (<inline-formula><mml:math id="M1"><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>stim</mml:mtext></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>/</mml:mo><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>sham</mml:mtext></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:math></inline-formula>). This indicates that the STIM condition has a 38% higher variance than the SHAM condition. Therefore, we use the Glass&#x00027; estimator (<italic>d</italic><sub>&#x003B4;</sub>) to calculate standardized mean difference (SMD) because it does not assume equal variances. An in-depth description of various effect-size calculations can be seen in Lakens (<xref ref-type="bibr" rid="B30">2013</xref>). All the estimators, <italic>d</italic><sub><italic>rm</italic></sub>, <italic>d</italic><sub><italic>av</italic></sub>, and <italic>d</italic><sub>&#x003B4;</sub> only differ in how they are standardized. Thus, they all take the form of:</p>
<disp-formula id="E1"><label>(1)</label><mml:math id="M2"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>d</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>M</mml:mi></mml:mrow><mml:mrow><mml:mtext>stim</mml:mtext></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>M</mml:mi></mml:mrow><mml:mrow><mml:mtext>sham</mml:mtext></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mo>*</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:mo>&#x000B7;</mml:mo><mml:mi>J</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Where <italic>J</italic>(<italic>n</italic>) is the small sample correction factor, <inline-formula><mml:math id="M3"><mml:mi>J</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:mfrac><mml:mrow><mml:mn>3</mml:mn></mml:mrow><mml:mrow><mml:mn>4</mml:mn><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>5</mml:mn></mml:mrow></mml:mfrac></mml:math></inline-formula>, and <italic>M</italic><sub>sham</sub> and <italic>M</italic><sub>stim</sub> correspond to the mean of the SHAM and STIM condition, respectively. The differences between estimators are in their calculation of the standardizer, <italic>S</italic><sup>&#x0002A;</sup>, such that:</p>
<disp-formula id="E2"><label>(2)</label><mml:math id="M4"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mtext>for&#x000A0;</mml:mtext><mml:msub><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msub><mml:mtext>:&#x000A0;</mml:mtext><mml:msup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mo>*</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>stim</mml:mtext></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>&#x0002B;</mml:mo><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>sham</mml:mtext></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>-</mml:mo><mml:mn>2</mml:mn><mml:msub><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>stim</mml:mtext></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>sham</mml:mtext></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E3"><label>(3)</label><mml:math id="M5"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mtext>for&#x000A0;</mml:mtext><mml:msub><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>a</mml:mi><mml:mi>v</mml:mi></mml:mrow></mml:msub><mml:mtext>:&#x000A0;</mml:mtext><mml:msup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mo>*</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>sham</mml:mtext></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>&#x0002B;</mml:mo><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>stim</mml:mtext></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:mfrac></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E4"><label>(4)</label><mml:math id="M6"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mtext>for&#x000A0;</mml:mtext><mml:msub><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi></mml:mrow></mml:msub><mml:mtext>:&#x000A0;</mml:mtext><mml:msup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mo>*</mml:mo></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mtext>sham</mml:mtext></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Where <italic>S</italic><sub>stim</sub> and <italic>S</italic><sub>sham</sub> indicates the standard deviation for STIM and SHAM, respectively. The associated standard errors (<italic>se</italic>) for each SMD estimator are as follows:</p>
<disp-formula id="E5"><label>(5)</label><mml:math id="M7"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>s</mml:mi><mml:msub><mml:mrow><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>J</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x000B7;</mml:mo><mml:msqrt><mml:mrow><mml:mrow><mml:mo stretchy="true">(</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>n</mml:mi></mml:mrow></mml:mfrac><mml:mo>-</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mi>n</mml:mi></mml:mrow></mml:mfrac></mml:mrow><mml:mo stretchy="true">)</mml:mo></mml:mrow><mml:mo>&#x000B7;</mml:mo><mml:mn>2</mml:mn><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E6"><label>(6)</label><mml:math id="M8"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>s</mml:mi><mml:msub><mml:mrow><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>a</mml:mi><mml:mi>v</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>J</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x000B7;</mml:mo><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mi>n</mml:mi></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>a</mml:mi><mml:mi>v</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:mn>4</mml:mn><mml:mi>n</mml:mi></mml:mrow></mml:mfrac></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<disp-formula id="E7"><label>(7)</label><mml:math id="M9"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>s</mml:mi><mml:msub><mml:mrow><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>J</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x000B7;</mml:mo><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mi>n</mml:mi></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Where <italic>n</italic> indicates the sample size. The point estimate and standard error of the repeated measures estimator (<italic>d</italic><sub><italic>rm</italic></sub>) requires the correlation between STIM and SHAM retention scores (<italic>r</italic><sub><italic>d</italic></sub>) which are rarely reported in repeated measure studies. Therefore, based on acquired raw data sets and plot digitizing (extracting data from figures), we calculated all available correlations. For the remaining studies where correlations were not available, the weighted mean correlation (i.e., random effects) is used to populate missing values.</p></sec>
<sec>
<title>2.5. Random-effects modeling</title>
<p>To perform our new meta-analysis and confirm prior meta-analyses and assess potential sources of variation in our regression model we estimate the true effect size using a Random-effects model approach (<xref ref-type="fig" rid="F1">Figure 1A</xref>). Random-effects modeling is used to estimate the mean of true effect sizes. Random-effects models allow for variation between true effect sizes (heterogeneity) across studies as should be the case when studies vary in their population, methodology, or design. We use the restricted maximum likelihood estimator to calculate between-study heterogeneity, that is, the standard deviation of true effect sizes (&#x003C4;). All meta-analytic modeling is completed in R using the <italic>metafor</italic> package (Viechtbauer, <xref ref-type="bibr" rid="B79">2010</xref>).</p></sec>
<sec>
<title>2.6. Mixed-effects (meta-regression) modeling</title>
<p>A meta-regression model was implemented to quantify the potential moderating effects of multiple study-level characteristics on the effect size (<italic>d</italic><sub>&#x003B4;</sub>). Because our initial analysis identified a downward trend in effect size with publication date (<xref ref-type="fig" rid="F2">Figure 2A</xref>), the regression model was logistically set up to examine whether independent moderators could account for this decline. Thus, we built nine different regression models to investigate whether the effect size was conditionally dependent on the study&#x00027;s publication date or other moderating variables (e.g., age; see <xref ref-type="table" rid="T2">Tables 2</xref>, <xref ref-type="table" rid="T3">3</xref>). For interpretability, we also report the effects for various subgroups based off of categorical moderators (<xref ref-type="table" rid="T4">Table 4</xref>). The first model consists of a single moderator (publication date) while the other models consist of both publication date and one other moderating variable. As detailed above, the restricted maximum likelihood estimation method was used for all meta-analytic models. Additionally, we utilized the moving constant technique (Johnson and Huedo-Medina, <xref ref-type="bibr" rid="B26">2011</xref>) by setting the initial moderator value to zero, this allows the intercept to be interpreted as the predicted effect size (<italic>d</italic><sub>&#x003B4;</sub>) at the date of the original publication (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>). To compare the nine regression models, each model is re-fit with maximum likelihood (as opposed to restricted maximum likelihood) which is necessary to conduct a likelihood ratio test between models. The likelihood ratio test assesses whether the addition of a given model parameter is warranted.</p>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Meta-regression models and summary statistics.</p></caption> 
<table frame="box" rules="all">
<thead><tr style="background-color:#919497;color:#ffffff">
<th valign="top" align="left"><bold>Model</bold></th>
<th valign="top" align="center"><bold>1</bold></th>
<th valign="top" align="center"><bold>2</bold></th>
<th valign="top" align="center"><bold>3</bold></th>
<th valign="top" align="center"><bold>4</bold></th>
<th valign="top" align="center"><bold>5</bold></th>
<th valign="top" align="center"><bold>6</bold></th>
<th valign="top" align="center"><bold>7</bold></th>
<th valign="top" align="center"><bold>8</bold></th>
<th valign="top" align="center"><bold>9</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Intercept</td>
<td valign="top" align="center">0.989<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[0.492, 1.485]</td>
<td valign="top" align="center">0.758&#x0002A;<break/>[0.164, 1.352]</td>
<td valign="top" align="center">1.072&#x0002A;&#x0002A;<break/>[0.358, 1.786]</td>
<td valign="top" align="center">0.875&#x0002A;&#x0002A;<break/>[0.220, 1.531]</td>
<td valign="top" align="center">0.970&#x0002A;<break/>[0.020, 1.919]</td>
<td valign="top" align="center">1.415&#x0002A;&#x0002A;<break/>[0.537, 2.294]</td>
<td valign="top" align="center">0.928&#x0002A;&#x0002A;<break/>[0.244, 1.612]</td>
<td valign="top" align="center">0.915&#x0002A;<break/>[0.070, 1.560]</td>
<td valign="top" align="center">1.471<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[0.696, 2.247]</td>
</tr> <tr>
<td valign="top" align="left">Publication date</td>
<td valign="top" align="center">-0.166<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.254, -0.078]</td>
<td valign="top" align="center">-0.171<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.257, -0.084]</td>
<td valign="top" align="center">-0.174<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.273, -0.074]</td>
<td valign="top" align="center">-0.164<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.257, -0.072]</td>
<td valign="top" align="center">-0.167<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.261, -0.072]</td>
<td valign="top" align="center">-0.207<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.318, -0.096]</td>
<td valign="top" align="center">-0.169<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.261, -0.077]</td>
<td valign="top" align="center">-0.155<xref ref-type="table-fn" rid="TN2"><sup>&#x0002A;&#x0002A;</sup></xref><break/>[-0.252, -0.058]</td>
<td valign="top" align="center">-0.206<xref ref-type="table-fn" rid="TN3"><sup>&#x0002A;&#x0002A;&#x0002A;</sup></xref><break/>[-0.305, -0.107]</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="10"><bold>Additional moderators</bold></td>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Age</td>
<td/>
<td valign="top" align="center">-0.393 [-0.880, 0.094]</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Proportion female</td>
<td/>
<td/>
<td valign="top" align="center">0.009 [-0.004, 0.022]</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Phase-locking</td>
<td/>
<td/>
<td/>
<td valign="top" align="center">-0.113 [-0.844, 0.618]</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Word count</td>
<td/>
<td/>
<td/>
<td/>
<td valign="top" align="center">0.145 [-0.325, 0.615]</td>
<td/>
<td/>
<td/>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Congruent words</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td valign="top" align="center">0.0003 [-0.007, 0.008]</td>
<td/>
<td/>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Overnight</td>
<td/>
<td/>
<td/>
<td/>	
<td/>
<td/>
<td valign="top" align="center">-0.318 [-0.854, 0.219]</td>
<td/>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; &#x00394; SO power</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td valign="top" align="center">0.091 [-0.500, 0.681]</td>
<td/>
</tr> <tr>
<td valign="top" align="left">&#x0002B; Double-blind</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td valign="top" align="center">0.092 [-0.190, 0.374]</td>
</tr> <tr>
<td valign="top" align="left"><italic>R</italic><sup>2</sup>(%)</td>
<td valign="top" align="center">91.8</td>
<td valign="top" align="center">96.0</td>
<td valign="top" align="center">86.5</td>
<td valign="top" align="center">83.0</td>
<td valign="top" align="center">83.8</td>
<td valign="top" align="center">98.4</td>
<td valign="top" align="center">84.6</td>
<td valign="top" align="center">87.7</td>
<td valign="top" align="center">100</td>
</tr> <tr>
<td valign="top" align="left">&#x003C4;<sub>resid</sub></td>
<td valign="top" align="center">0.110</td>
<td valign="top" align="center">0.077</td>
<td valign="top" align="center">0.142</td>
<td valign="top" align="center">0.159</td>
<td valign="top" align="center">0.155</td>
<td valign="top" align="center">0.049</td>
<td valign="top" align="center">0.151</td>
<td valign="top" align="center">0.135</td>
<td valign="top" align="center">0.000</td>
</tr> <tr>
<td valign="top" align="left"><italic>Q</italic><sub><italic>m</italic></sub>(<italic>df</italic>)</td>
<td valign="top" align="center">13.72 (1)</td>
<td valign="top" align="center">15.85 (2)</td>
<td valign="top" align="center">13.27 (2)</td>
<td valign="top" align="center">13.21 (2)</td>
<td valign="top" align="center">12.92 (2)</td>
<td valign="top" align="center">15.79 (2)</td>
<td valign="top" align="center">13.09 (2)</td>
<td valign="top" align="center">13.72 (2)</td>
<td valign="top" align="center">17.14 (2)</td>
</tr> <tr>
<td valign="top" align="left">AIC</td>
<td valign="top" align="center">16.56</td>
<td valign="top" align="center">16.82</td>
<td valign="top" align="center">18.41</td>
<td valign="top" align="center">18.19</td>
<td valign="top" align="center">18.56</td>
<td valign="top" align="center">17.18</td>
<td valign="top" align="center">18.46</td>
<td valign="top" align="center">18.12</td>
<td valign="top" align="center">16.06</td>
</tr> <tr>
<td valign="top" align="left">BIC</td>
<td valign="top" align="center">18.48</td>
<td valign="top" align="center">19.38</td>
<td valign="top" align="center">20.97</td>
<td valign="top" align="center">20.75</td>
<td valign="top" align="center">21.12</td>
<td valign="top" align="center">19.73</td>
<td valign="top" align="center">21.02</td>
<td valign="top" align="center">20.67</td>
<td valign="top" align="center">18.62</td>
</tr> <tr>
<td valign="top" align="left">AICc</td>
<td valign="top" align="center">18.96</td>
<td valign="top" align="center">21.27</td>
<td valign="top" align="center">22.85</td>
<td valign="top" align="center">22.64</td>
<td valign="top" align="center">23.00</td>
<td valign="top" align="center">21.62</td>
<td valign="top" align="center">22.91</td>
<td valign="top" align="center">22.56</td>
<td valign="top" align="center">20.50</td>
</tr> <tr>
<td valign="top" align="left">Likelihood ratio</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">1.740</td>
<td valign="top" align="center">0.152</td>
<td valign="top" align="center">0.369</td>
<td valign="top" align="center">0.001</td>
<td valign="top" align="center">1.385</td>
<td valign="top" align="center">0.100</td>
<td valign="top" align="center">0.444</td>
<td valign="top" align="center">2.502</td>
</tr> <tr>
<td valign="top" align="left">p (LRT from model 1)</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">0.187</td>
<td valign="top" align="center">0.697</td>
<td valign="top" align="center">0.543</td>
<td valign="top" align="center">0.977</td>
<td valign="top" align="center">0.239</td>
<td valign="top" align="center">0.752</td>
<td valign="top" align="center">0.505</td>
<td valign="top" align="center">0.114</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p>All models contain 14 effect sizes (<italic>n</italic> = 206 subjects). The regression coefficients with corresponding 95% confidence intervals are displayed alongside respective moderator variable. Each of the eight models contain publication date, with each successive model containing a single additional moderator one at a time. The final rows denote summary statistics of the meta regression models (<italic>Q</italic>, <italic>Q</italic><sub><italic>m</italic></sub>-statistic for moderators; <italic>R</italic><sup>2</sup>, percent of heterogeneity accounted for; &#x003C4;<sub><italic>resid</italic></sub>, residual heterogeneity; AIC, Akaike information criterion; BIC, Bayesian information criterion; AICc, corrected Akaike information criterion). Bold AIC and BIC indicate lowest values (best model). Each column denotes the respective model&#x00027;s coefficients and summary statistics.</p>
<fn id="TN1"><label>&#x0002A;</label><p><italic>p</italic> &#x0003C; 0.05,</p></fn> 
<fn id="TN2"><label>&#x0002A;&#x0002A;</label><p><italic>p</italic> &#x0003C; 0.01,</p></fn> 
<fn id="TN3"><label>&#x0002A;&#x0002A;&#x0002A;</label><p><italic>p</italic> &#x0003C; 0.001.</p></fn>
</table-wrap-foot>
</table-wrap>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Subgroup analyses.</p></caption> 
<table frame="box" rules="all">
<thead><tr style="background-color:#919497;color:#ffffff">
<th/>
<th/>
<th/>
<th/>
<th valign="top" align="center" colspan="2"><bold>95% CI</bold></th>
<th/>
<th/>
</tr>
<tr style="background-color:#919497;color:#ffffff">
<td valign="top" align="left"><bold>Study</bold></td>
<td valign="top" align="center"><italic>k</italic><sub><italic>ES</italic></sub></td>
<td valign="top" align="center"><italic>n</italic></td>
<td valign="top" align="center"><inline-formula><mml:math id="M10"><mml:msub><mml:mrow><mml:mover accent="true"><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mo>&#x00304;</mml:mo></mml:mover></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula></td>
<td valign="top" align="center"><bold>Lower</bold></td>
<td valign="top" align="center"><bold>Upper</bold></td>
<td valign="top" align="center">&#x003C4;</td>
<td valign="top" align="center"><italic>I</italic><sup>2</sup></td>
</tr> 
</thead>
<tbody>
<tr>
<td valign="top" align="left">Overall</td>
<td valign="top" align="center">14</td>
<td valign="top" align="center">206</td>
<td valign="top" align="center">0.14</td>
<td valign="top" align="center">&#x02212;0.14</td>
<td valign="top" align="center">0.42</td>
<td valign="top" align="center">0.385</td>
<td valign="top" align="center">52.8</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="8"><bold>Age</bold></td>
</tr> <tr>
<td valign="top" align="left">Young</td>
<td valign="top" align="center">11</td>
<td valign="top" align="center">152</td>
<td valign="top" align="center">0.14</td>
<td valign="top" align="center">&#x02212;0.16</td>
<td valign="top" align="center">0.43</td>
<td valign="top" align="center">0.319</td>
<td valign="top" align="center">42.4</td>
</tr> <tr>
<td valign="top" align="left">Middle-Age/Elderly</td>
<td valign="top" align="center">3</td>
<td valign="top" align="center">54</td>
<td valign="top" align="center">0.19</td>
<td valign="top" align="center">&#x02212;0.74</td>
<td valign="top" align="center">1.12</td>
<td valign="top" align="center">0.738</td>
<td valign="top" align="center">82.0</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="8"><bold>Blinding</bold></td>
</tr> <tr>
<td valign="top" align="left">Single blind</td>
<td valign="top" align="center">10</td>
<td valign="top" align="center">148</td>
<td valign="top" align="center">0.20</td>
<td valign="top" align="center">&#x02212;0.19</td>
<td valign="top" align="center">0.58</td>
<td valign="top" align="center">0.503</td>
<td valign="top" align="center">65.3</td>
</tr> <tr>
<td valign="top" align="left">Double blind</td>
<td valign="top" align="center">4</td>
<td valign="top" align="center">58</td>
<td valign="top" align="center">0.03</td>
<td valign="top" align="center">&#x02212;0.32</td>
<td valign="top" align="center">0.37</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="8"><bold>Semantic congruence of word-pairs</bold></td>
</tr> <tr>
<td valign="top" align="left">Congruent</td>
<td valign="top" align="center">9</td>
<td valign="top" align="center">143</td>
<td valign="top" align="center">0.28</td>
<td valign="top" align="center">&#x02212;0.11</td>
<td valign="top" align="center">0.67</td>
<td valign="top" align="center">0.466</td>
<td valign="top" align="center">63.3</td>
</tr> <tr>
<td valign="top" align="left">Incongruent</td>
<td valign="top" align="center">5</td>
<td valign="top" align="center">63</td>
<td valign="top" align="center">&#x02212;0.09</td>
<td valign="top" align="center">&#x02212;0.42</td>
<td valign="top" align="center">0.24</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="8"><bold>Sleep type</bold></td>
</tr> <tr>
<td valign="top" align="left">Whole night</td>
<td valign="top" align="center">12</td>
<td valign="top" align="center">178</td>
<td valign="top" align="center">0.14</td>
<td valign="top" align="center">&#x02212;0.19</td>
<td valign="top" align="center">0.47</td>
<td valign="top" align="center">0.442</td>
<td valign="top" align="center">59.6</td>
</tr> <tr>
<td valign="top" align="left">Nap</td>
<td valign="top" align="center">2</td>
<td valign="top" align="center">28</td>
<td valign="top" align="center">0.16</td>
<td valign="top" align="center">&#x02212;0.34</td>
<td valign="top" align="center">0.66</td>
<td valign="top" align="center">0.007</td>
<td valign="top" align="center">0.04</td>
</tr> <tr style="background-color:#dee1e1">
<td valign="top" align="left" colspan="8"><bold>Phase locking</bold></td>
</tr> <tr>
<td valign="top" align="left">Phase locked</td>
<td valign="top" align="center">11</td>
<td valign="top" align="center">148</td>
<td valign="top" align="center">0.21</td>
<td valign="top" align="center">&#x02212;0.14</td>
<td valign="top" align="center">0.57</td>
<td valign="top" align="center">0.461</td>
<td valign="top" align="center">59.2</td>
</tr> <tr>
<td valign="top" align="left">Non-phase locked</td>
<td valign="top" align="center">3</td>
<td valign="top" align="center">58</td>
<td valign="top" align="center">&#x02212;0.04</td>
<td valign="top" align="center">&#x02212;0.39</td>
<td valign="top" align="center">0.31</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p><italic>k</italic><sub><italic>ES</italic></sub>, number of effect sizes; <italic>n</italic>, total sample size; <inline-formula><mml:math id="M11"><mml:msub><mml:mrow><mml:mover accent="true"><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mo>&#x00304;</mml:mo></mml:mover></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>, mean standardized mean difference; &#x003C4;, standard deviation of true effect sizes (i.e., heterogeneity); <italic>I</italic><sup>2</sup>, Percentage of total variation in effect sizes due to variation in true effect sizes.</p>
</table-wrap-foot>
</table-wrap>
<p>In addition, the sparsity of study-level characteristics could potentially drive our analyses to be insensitive to additive contributions of study-level characteristics to the observed heterogeneity in effect sizes. Many studies differed from Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>) methodologically, and some of those studies differed in multiple facets. To address this, we have constructed a &#x0201C;Methodological Deviation&#x0201D; factor consisting of the sum score of four other coded study design characteristics. Specifically, the sum score is taken as the sum of how study-level characteristics diverge from the seminal publication (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>). The four characteristics used are age, semantic congruence, word count, and phase-locking. For example, if a study included subjects with similar mean age as Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>) (young adults), that component of general risk &#x0201C;Methodological Deviation&#x0201D; was coded as zero. The maximum &#x0201C;Methodological Deviation&#x0201D; is four, being relatively methodologically inconsistent with Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>), and the minimum zero being highly methodologically consistent with Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>). Subsequently, the &#x0201C;Methodological Deviation&#x0201D; factor was run with the omission of Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>). A major limitation to this index is that, since it is simply a sum score, it assumes that each methodological deviation from Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>) has identical weights.</p></sec>
<sec>
<title>2.7. Sensitivity analysis</title>
<p>Outliers can cause spurious results in small sample meta-regressions, therefore a leave-one-out cross-validation procedure was used to assess the robustness of the meta-regression model. For each iteration, one effect size is removed from the data set and the model is fit to the remaining effect sizes (<xref ref-type="fig" rid="F2">Figure 2B</xref>). If the meta-regression model is to be considered robust, then each iteration of the leave-one-out-cross-validation should not greatly impact the model parameters. Specifically, each iteration of the leave-one-out-cross-validation should yield a significant slope coefficient, and no slope coefficient should significantly differ from the full model.</p></sec>
<sec>
<title>2.8. Meta-analysis of reliability coefficients</title>
<p>Reliability is a psychometric property that indexes the precision of a measurement instrument. A test-retest reliability coefficient is a method of estimating the reliability of a task by administering the same task on two separate occasions and then calculated the correlation between time points (see a diagrammatic representation in <xref ref-type="fig" rid="F3">Figures 3A</xref>, <xref ref-type="fig" rid="F3">B</xref>). In all the experiments within this meta-analysis, subjects underwent a cross-over design, that is, the same participants were in the control (SHAM) and treatment (STIM) conditions. Since there are multiple measurements for each participant, this allows for the opportunity to assess the test-retest reliability of the word-pair recall task used to measure memory retention. The word-pair cued recall task is characterized by three phases: (1) A learning phase where participants are presented with a list of word pairs, (2) a pre-sleep (immediate) recall phase where participants are assessed on their word-pair recall accuracy and scored on the number of word-pairs they correctly recall, (3) a post-sleep recall phase where participants are assessed on their word-pair recall accuracy and scored on the number of word-pairs they correctly recall. Memory retention is scored by the number of word pairs correctly recalled post-sleep subtracted from the number of word pairs correctly recalled pre-sleep (see <xref ref-type="fig" rid="F3">Figures 3A</xref>, <xref ref-type="fig" rid="F3">B</xref>):</p>
<disp-formula id="E8"><label>(8)</label><mml:math id="M12"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mtext>post</mml:mtext></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mtext>pre</mml:mtext></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p>The effect of reliability on significance tests and observed effect sizes. <bold>(A)</bold> Measurement model of experimental paradigm indicate that true pre and post-sleep scores (unobserved scores uncontaminated by measurement error) underlies the observed pre-sleep and post-sleep scores. Differences in true scores between delayed and immediate recall are indicative of memory retention. <bold>(B)</bold> Venn diagram representation of how pre-post test correlation effects the reliability of difference scores. <bold>(C)</bold> For variable true effect sizes with a median sample size of &#x000F1; = 15, the statistical power increases with the reliability of the measure. The dashed lines indicate the estimated reliability of the word-pair recall difference scores. <bold>(D)</bold> The biasing effect of low reliability on observed SMDs (<italic>d</italic><sub>&#x003B4;</sub>) demonstrates that decreases in reliability bias SMDs toward zero. The effect of this bias is stronger in higher SMDs. The dashed lines indicate the estimated reliability of the word-pair recall difference scores.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="frsle-02-1082253-g0003.tif"/>
</fig>
<p>Where the difference score, <italic>X</italic><sub><italic>d</italic></sub> is used as the final memory retention score. However, word-pair recall is an imperfect measure of memory retention, therefore we can calculate the observed difference scores of <italic>X</italic><sub><italic>d</italic></sub> in terms of true scores (<italic>T</italic>; scores indicative of actual memory retention) and error variance (<italic>E</italic>; scores indicative of random error):</p>
<disp-formula id="E9"><label>(9)</label><mml:math id="M13"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mtext>post</mml:mtext></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mtext>pre</mml:mtext></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mtext>post</mml:mtext></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mtext>post</mml:mtext></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>-</mml:mo><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mtext>pre</mml:mtext></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mtext>pre</mml:mtext></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>In classical test theory, the reliability of a measure is a ratio of true variance to observed variance such that:</p>
<disp-formula id="E10"><label>(10)</label><mml:math id="M14"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>&#x003C1;</mml:mi><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mi>X</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mi>T</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo>&#x0002B;</mml:mo><mml:msubsup><mml:mrow><mml:mi>S</mml:mi></mml:mrow><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>To compute the reliability of a difference score we need the reliability of pre-sleep scores (<italic>&#x003C1;</italic><sub>pre</sub>), reliability of post-sleep scores (<italic>&#x003C1;</italic><sub>post</sub>) and the correlation between pre and post-sleep scores (<italic>r</italic><sub><italic>pp</italic></sub>). The reliability of pre-sleep (<italic>&#x003C1;</italic><sub>pre</sub>) is the test-retest correlation between the SHAM condition&#x00027;s pre-sleep score and the STIM condition&#x00027;s pre-sleep scores. The reliability of post-sleep (<italic>&#x003C1;</italic><sub>post</sub>) is the test-retest correlation between the SHAM condition&#x00027;s post-sleep score and the STIM condition&#x00027;s post-sleep scores. Lastly, the pre/post correlation (<italic>r</italic><sub>pp</sub>) is estimated from the correlation between pre and post-sleep scores of the SHAM condition:</p>
<disp-formula id="E11"><label>(11)</label><mml:math id="M15"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>&#x003C1;</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:msub><mml:mrow><mml:mi>&#x003C1;</mml:mi></mml:mrow><mml:mrow><mml:mtext>pre</mml:mtext></mml:mrow></mml:msub><mml:mo>&#x0002B;</mml:mo><mml:msub><mml:mrow><mml:mi>&#x003C1;</mml:mi></mml:mrow><mml:mrow><mml:mtext>post</mml:mtext></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:mn>2</mml:mn><mml:msub><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mtext>pp</mml:mtext></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mn>1</mml:mn><mml:mo>-</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mtext>pp</mml:mtext></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>The estimates from each study are pooled to estimate the reliability of word-pair recall task as an indicator of memory retention (<xref ref-type="table" rid="T5">Table 5</xref>). A convenient property of Glass&#x00027; estimator (<italic>d</italic><sub>&#x003B4;</sub>) is the simplicity of the correction for task reliability. For a discussion on corrections for repeated measures effect sizes see the blog post by Pustejovsky (<xref ref-type="bibr" rid="B62">2023</xref>). For <italic>d</italic><sub>&#x003B4;</sub>, the unreliability of the task introduces error variance into the total observed variance, thus inflating the observed standard deviation and subsequently reducing the observed standardized mean difference (<italic>d</italic><sub>&#x003B4;, <italic>obs</italic></sub>). This attenuation can be formally defined as:</p>
<disp-formula id="E12"><label>(12)</label><mml:math id="M18"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:msub><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mo>,</mml:mo><mml:mi>o</mml:mi><mml:mi>b</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi></mml:mrow></mml:msub><mml:mo>&#x000B7;</mml:mo><mml:msqrt><mml:mrow><mml:msub><mml:mrow><mml:mi>&#x003C1;</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<table-wrap position="float" id="T5">
<label>Table 5</label>
<caption><p>Meta-analysis of test-retest reliability of word-pair cued recall task.</p></caption> 
<table frame="box" rules="all">
<thead><tr style="background-color:#919497;color:#ffffff">
<th/>
<th/>
<th/>
<th/>
<th valign="top" align="center" colspan="2"><bold>95% CI</bold></th>
<th/>
<th/>
</tr>
<tr style="background-color:#919497;color:#ffffff">
<td valign="top" align="left"><bold>Coefficient</bold></td>
<td valign="top" align="center"><italic>k</italic><sub><italic>ES</italic></sub></td>
<td valign="top" align="center"><italic>n</italic></td>
<td valign="top" align="center"><bold>Est</bold></td>
<td valign="top" align="center"><bold>Lower</bold></td>
<td valign="top" align="center"><bold>Upper</bold></td>
<td valign="top" align="center">&#x003C4;</td>
<td valign="top" align="center"><italic>I</italic><sup>2</sup></td>
</tr> 
</thead>
<tbody>
<tr>
<td valign="top" align="left">Reliability of pre-sleep scores (<italic>&#x003C1;</italic><sub>pre</sub>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.71</td>
<td valign="top" align="center">0.62</td>
<td valign="top" align="center">0.81</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr>
<td valign="top" align="left">Reliability of post-sleep scores (<italic>&#x003C1;</italic><sub>post</sub>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.84</td>
<td valign="top" align="center">0.77</td>
<td valign="top" align="center">0.90</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr>
<td valign="top" align="left">Pre-post correlation (<italic>r</italic><sub>pp</sub>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.95</td>
<td valign="top" align="center">0.92</td>
<td valign="top" align="center">0.98</td>
<td valign="top" align="center">0.030</td>
<td valign="top" align="center">64.1</td>
</tr> <tr>
<td valign="top" align="left">Pre-post correlation&#x02014;delayed (<inline-formula><mml:math id="M16"><mml:msubsup><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mtext>p</mml:mtext></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>d</mml:mi><mml:mi>e</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.74</td>
<td valign="top" align="center">0.65</td>
<td valign="top" align="center">0.83</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr>
<td valign="top" align="left">Reliability of difference scores (<italic>&#x003C1;</italic><sub><italic>d</italic></sub>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.01</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.20</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr>
<td valign="top" align="left">Reliability of difference scores&#x02014;liberal est (<inline-formula><mml:math id="M17"><mml:msubsup><mml:mrow><mml:mi>&#x003C1;</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>l</mml:mi><mml:mi>i</mml:mi><mml:mi>b</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.01</td>
<td valign="top" align="center">0.00</td>
<td valign="top" align="center">0.21</td>
<td valign="top" align="center">0.000</td>
<td valign="top" align="center">0</td>
</tr> <tr>
<td valign="top" align="left">Observed correlation of difference scores (<italic>r</italic><sub><italic>d</italic></sub>)</td>
<td valign="top" align="center">7</td>
<td valign="top" align="center">110</td>
<td valign="top" align="center">0.46</td>
<td valign="top" align="center">0.25</td>
<td valign="top" align="center">0.68</td>
<td valign="top" align="center">0.221</td>
<td valign="top" align="center">60.3</td>
</tr></tbody>
</table>
<table-wrap-foot>
<p><italic>k</italic><sub><italic>ES</italic></sub>, number of effect sizes; <italic>n</italic>, total sample size; Est, parameter estimate; &#x003C4;, standard deviation of true effect sizes (i.e., heterogeneity); <italic>I</italic><sup>2</sup>, Percentage of total variation in effect sizes due to variation in true effect sizes.</p>
</table-wrap-foot>
</table-wrap>
<p>Where <italic>d</italic><sub>&#x003B4;, <italic>true</italic></sub> is the true standardized mean difference (see <xref ref-type="fig" rid="F3">Figure 3C</xref>). This attenuation is also carried over to the standard error such that:</p>
<disp-formula id="E13"><label>(13)</label><mml:math id="M19"><mml:mtable class="eqnarray" columnalign="left"><mml:mtr><mml:mtd><mml:mi>s</mml:mi><mml:msub><mml:mrow><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mo>,</mml:mo><mml:mi>o</mml:mi><mml:mi>b</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>J</mml:mi><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo>&#x000B7;</mml:mo><mml:msqrt><mml:mrow><mml:mfrac><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mi>n</mml:mi></mml:mrow></mml:mfrac><mml:mo>&#x0002B;</mml:mo><mml:mfrac><mml:mrow><mml:msubsup><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x003B4;</mml:mi><mml:mo>,</mml:mo><mml:mi>o</mml:mi><mml:mi>b</mml:mi><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:mrow><mml:mrow><mml:mn>2</mml:mn><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:mrow></mml:msqrt></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>The attenuation of the observed effect size (<italic>d</italic><sub>&#x003B4;, <italic>obs</italic></sub>) and the corresponding change in the standard error (<italic>se</italic><sub>&#x003B4;, <italic>obs</italic></sub>) altogether results in decreased statistical power with lower reliability (see <xref ref-type="fig" rid="F3">Figure 3C</xref>).</p></sec></sec>
<sec sec-type="results" id="s3">
<title>3. Results</title>
<p>The aim of this meta-analysis is to compare and extend two previous meta-analyses (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>) with the goal of identifying ways to improve future studies on acoustic stimulation for memory enhancement. Our full analysis includes fourteen effect sizes from 12 studies that test how acoustic stimulation during sleep impacts post-sleep memory task performance (see Section 2, <xref ref-type="fig" rid="F1">Figure 1</xref>). Our full analysis includes three additional effect sizes from two publications that followed the initial meta-analyses by Wunderlin and colleagues (Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>; Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>) (<xref ref-type="fig" rid="F1">Figure 1A</xref>, boxes indicate distinct studies included for each meta-analysis). As a starting point, we first analyze a subset of 11 studies and confirm that our standardized mean difference effects are virtually identical to effect sizes reported by Wunderlin and colleagues (mean absolute difference in SMD estimates = 0.02; <xref ref-type="table" rid="T1">Table 1</xref> and <xref ref-type="fig" rid="F1">Figure 1B</xref>). Moreover, we confirm that the corresponding pooled effects for these eleven studies are small (<italic>d</italic><sub><italic>rm</italic></sub> = 0.25, 95% CI [&#x02212;0.02, 0.53], <italic>p</italic> = 0.211), consistent with the prior report by Wunderlin and colleagues (<italic>d</italic><sub><italic>rm</italic></sub> = 0.25, 95% CI [&#x02212;0.02, 0.53]). When analyzing a subset of six studies that employ phase-locked acoustic stimulation and include non-elderly adults, Wunderlin and colleagues report almost a two-fold increase in effect size (<italic>d</italic><sub><italic>rm</italic></sub> = 0.44, 95% CI [0.09, 0.79]). Accordingly, we confirm that phase-locked acoustic stimulation protocol and age have a significant positive result for this same subset of combined studies (<italic>d</italic><sub><italic>rm</italic></sub> = 0.44, 95% CI [0.07, 0.80]). However, this effect is no longer significant (<italic>d</italic><sub>&#x003B4;</sub> = 0.23, 95% CI [&#x02212;0.12, 0.58], <italic>p</italic> = 0.206) when we include three additional effect sizes for the current meta-analysis (Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>; Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>). As detailed below, our regression models find a progressive decline in effects over the past 10 years. This trend also explains away the subgroup effect previously reported by Wunderlin and colleagues for phase-locked stimulation in young adults (Wunderlin studies: <italic>p</italic> = 0.512; All current studies: <italic>p</italic> = 0.642), while the effect of publication date remains significant (Wunderlin studies: <italic>p</italic> = 0.002, All current studies: <italic>p</italic> &#x0003C; 0.0001). The Stanyer meta-analysis (Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>) examines the possible differences in effects for studies testing declarative vs. non-declarative memory. Stanyer and colleagues find no significant differences across memory task type and suggest that future studies should include more subjects and potentially more overnight measures which we concur. However, we find our standardized mean difference effects are not strongly correlated with those reported by Stanyer and colleagues (mean absolute difference in SMD estimates = 0.84; <xref ref-type="fig" rid="F1">Figure 1C</xref> and <xref ref-type="table" rid="T1">Table 1</xref>). For our full meta-analysis, we examine the possibility that the inclusion of several recent studies that use double-blind procedures could account for the progressive decline in effect size (see <xref ref-type="fig" rid="F2">Figure 2A</xref>, blue symbols). As observed for other moderators, double-blind procedures did not account for the decline in effect size (<xref ref-type="table" rid="T3">Table 3</xref>). Finally, we calculated the percentage of subjects that reported hearing acoustic stimuli in five independent studies (Section 2, <xref ref-type="fig" rid="F2">Figure 2C</xref>). We estimate that 34% of participants report hearing acoustic stimuli at some point during their sleep session (<italic>k</italic><sub><italic>S</italic></sub> = 5 studies, 27 of 80 participants). This raises the possibility that many subjects in these studies are indeed not blinded to the experimental condition which could create placebo, or nocebo, effects (Petersen et al., <xref ref-type="bibr" rid="B57">2014</xref>; Lindheimer et al., <xref ref-type="bibr" rid="B34">2015</xref>; Faraone et al., <xref ref-type="bibr" rid="B16">2022</xref>). This leads us to suggest that future studies should include a true placebo acoustic stimulation condition, as has been done for previous physiological studies (Ngo et al., <xref ref-type="bibr" rid="B50">2013a</xref>).</p>
<sec>
<title>3.1. Decline in observed effect sizes over time</title>
<p>Our meta-analysis examining a total of 14 studies finds a progressive annual decline in reported memory effects with acoustic stimulation during sleep. Accordingly, there is a decline in reported effects (standardized mean difference) between 2013 and 2023 (<xref ref-type="fig" rid="F2">Figure 2A</xref>). Our meta-regression models examine multiple potential sources of variability contributing to this decline (Section 2) including publication date, age, gender, phase-locking, word count, and congruence of words in memory tasks, overnight sleep condition, double-blind procedures, and the physiological index of slow oscillation power (see <xref ref-type="table" rid="T3">Table 3</xref>). For effect sizes within categorical moderators, see <xref ref-type="table" rid="T4">Table 4</xref>. Using a meta-regression model (Section 2, Model 1) where publication date is the sole moderator, we find that the date of publication demonstrates a negative trend that accounts for 91.8% of the heterogeneity between studies (B = &#x02212;0.166 [&#x02212;0.254, &#x02212;0.078], <italic>Q</italic><sub><italic>m</italic></sub> = 13.72, <italic>p</italic> = 0.0002, <italic>k</italic><sub><italic>ES</italic></sub> = 14 effect sizes, <italic>n</italic> = 206, &#x003C4;<sub><italic>resid</italic></sub> = 0.110, see <xref ref-type="fig" rid="F2">Figure 2A</xref> and <xref ref-type="table" rid="T3">Table 3</xref>). Given the small number of effect sizes in this literature, we looked to evaluate the robustness of this trend with leave-one-out cross-validation (Section 2.7, <xref ref-type="fig" rid="F2">Figure 2A</xref>) to ensure highly influential studies do not artifactually produce this trend. Our leave-one-out cross-validation (Section 2) analysis finds that the regression coefficient (beta) in each iteration remains significant and unchanged from the full model coefficient (<xref ref-type="fig" rid="F2">Figure 2B</xref>). This indicates that the decline in effect size is not likely driven by outliers.</p>
<p>When adding one variable at a time using a step-wise procedure (Section 2), we find that no other study-level characteristic accounts for the annual decline in study effects (see <xref ref-type="table" rid="T3">Table 3</xref>, Models 2&#x02013;9). Corrected Akaike and Bayesian Information Criterion (AICc and BIC, respectively) tests confirm that the publication date (Model 1) is the most parsimonious (according to BIC), explanatory model of the set (according to AICc). Additionally, log-likelihood test demonstrates that the addition of any alternative moderators does not improve the model fit. When we look at the predicted effect size (based on Model 1) across time, the first study (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>) publication date has a large effect size <italic>d</italic><sub>&#x003B4;</sub> = 0.99 (95% CI [0.49, 1.49]) that declines to a net negative effect of <italic>d</italic><sub>&#x003B4;</sub> = &#x02212;0.39 (95% CI [&#x02212;0.73, &#x02212;0.05]) in the year of the most recent publication (see Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>, <xref ref-type="fig" rid="F2">Figure 2A</xref>). This indicates that recent studies show a markedly reduced effect of acoustic stimulation on memory task performance, in spite of having similar cohort sizes. Next, we consider the possibility that there are cumulative methodological deviations accounting for the publication date effect. The relatively low number of studies deviating from Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>) in certain study characteristics (age, phase-locking, whole night) could lead to such analyses being underpowered for observing unique moderator effects. Likewise, combinations of study characteristics could drive heterogeneity in observed effect sizes. However, our &#x0201C;Methodological Deviation&#x0201D; moderator model (Section 2) indicates that cumulative methodological deviation does not account for any of the heterogeneity in effect sizes (B = &#x02212;0.034 [&#x02212;0.257, 0.190], <italic>p</italic> = 0.768). This indicates that a cumulative methodological deviation from Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>) does not explain the decline in effect sizes across studies or years.</p></sec>
<sec>
<title>3.2. Reliability of word-pair association retention task</title>
<p>In neuroscience, psychometric evaluations and measurement errors for assessing cognitive processes such as working memory and memory consolidation are rarely investigated. A potential shortcoming of standard test vs. retest memory task difference scores used in all the studies included in our meta-analysis is the loss of shared variability (<xref ref-type="fig" rid="F3">Figure 3</xref>). For example, when there is no correlation between a first test (e.g., pre-sleep) vs. a second test (e.g., post-sleep) condition there is no shared variability resulting in a highly reliable difference score (<xref ref-type="fig" rid="F3">Figure 3B</xref>, top row). In contrast, if subjects have similar variability across test and retest conditions there can be highly overlapped variability that is lost in the difference score measure (<xref ref-type="fig" rid="F3">Figure 3B</xref>, rows 2 and 3). We estimate the reliability of the memory task difference scores as the ratio of true variance to total observed variance (Section 2, true plus error variance, Equation 10). If there is a loss in total observed variance due to overlapped variability for test vs. retest conditions this will result in low reliability for the memory performance difference score. The reliability of the word-pair association task used to measure memory retention (and consolidation) in this literature has yet to be evaluated. Thankfully, we can analyze this as investigators have provided seven raw data sets from a subset of five studies in the current meta-analysis (Section 2, <xref ref-type="table" rid="T5">Table 5</xref>). Each of these studies includes independent measures of memory task performance before and after sleep that can be used to calculate pre-sleep and post-sleep test-retest reliability (Section 2, <xref ref-type="fig" rid="F3">Figure 3A</xref>). Using this model, we find that the pre-sleep memory task performance has fair reliability with a <italic>&#x003C1;</italic><sub>pre</sub> = 0.711 (95% CI [0.615, 0.807]). The post-sleep memory task performance shows a slightly better reliability of <italic>&#x003C1;</italic><sub>post</sub> = 0.835 (95% CI [0.775, 0.896]). However, the correlation between pre-sleep and post-sleep scores from the control sleep condition (SHAM acoustic stimulation) is notably strong at <italic>r</italic><sub>pp</sub> = 0.949 (95% CI [0.920, 0.979]). This high correlation between test-retest scores reduces the reliability of the difference score, as described above (<xref ref-type="fig" rid="F3">Figure 3B</xref>). Additionally, we compute correlations for pre-sleep vs. post-sleep tests which are delayed by one week by calculating the correlation between pre-sleep scores from the STIM condition and post-sleep scores for the SHAM condition (Section 2). This delayed test-retest correlation is <inline-formula><mml:math id="M20"><mml:msubsup><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mtext>pp</mml:mtext></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>d</mml:mi><mml:mi>e</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> = 0.742 (95% CI [0.655, 0.830]). Based on classical test theory (Section 2, Equation 7) our estimates indicate a near zero reliability for difference scores (<italic>&#x003C1;</italic><sub>d</sub> = 0.006, 95% CI [&#x02212;0.188, 0.199]). Even if the delayed pre-sleep vs. post-sleep correlation is used (a more liberal estimate), the reliability of the difference score is still close to zero, <inline-formula><mml:math id="M21"><mml:msubsup><mml:mrow><mml:mi>&#x003C1;</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>d</mml:mi><mml:mi>e</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mrow></mml:msubsup></mml:math></inline-formula> = 0.015 (95% CI [&#x02212;0.178, 0.208]). This indicates that the observed changes in word-pair retention may be attributable to noise. This is surprising considering that the correlation between difference scores for STIM vs. SHAM conditions is <italic>r</italic><sub><italic>d</italic></sub> = 0.465 (95% CI [0.247, 0.682]), as the difference score correlation should be bounded by the reliability estimate. However, even if we treat this correlation as an estimate of the reliability of difference scores, the reliability would still be considered very poor. In summary, measurement error in the task used to measure memory retention is expected to bias effect sizes and decrease statistical power (<xref ref-type="fig" rid="F3">Figures 3C</xref>, <xref ref-type="fig" rid="F3">D</xref>). It is possible that the poor reliability of the memory task difference scores in conjunction with low sample sizes and variable study-level characteristics could drive the observed heterogeneity in effect sizes.</p></sec></sec>
<sec sec-type="discussion" id="s4">
<title>4. Discussion</title>
<p>The use of acoustic simulation during slow-wave sleep to modulate slow waves and overnight word pair memory retention has been the focus of several studies, meta-analyses, and reviews within the last decade. Within this body of literature, there exists a variety of reported effects on memory performance, with meta-analyses suggesting small-to-moderate overall effect sizes and high degrees of variability across studies (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>). Previously, Wunderlin et al. (<xref ref-type="bibr" rid="B84">2021</xref>) found a larger memory enhancement effect when pooling studies that had included phase-locked acoustic stimulation and non-elderly participant age. For this same pool of studies, we confirm this result. Additionally, we confirm their report of a low overall effect and a large degree of heterogeneity even when controlling for acoustic stimulation and age. In addition, we extend the Wunderlin study by analyzing three additional recent studies and find there is no longer a significant pooled effect. Moreover, our meta-regression models examine multiple potential sources of variability including publication date, age, gender, phase-locking, word count and congruence of words in memory tasks, overnight sleep, double-blind procedures, and the physiological index of slow oscillation power (see <xref ref-type="table" rid="T2">Tables 2</xref>, <xref ref-type="table" rid="T3">3</xref>). We find that none of these variables, except for publication date, accounts for the cumulative decline of memory effects with acoustic stimulation (see <xref ref-type="table" rid="T3">Table 3</xref>).</p>
<sec>
<title>4.1. Publication date</title>
<p>Previous meta-analyses (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>; Stanyer et al., <xref ref-type="bibr" rid="B72">2022</xref>) reported small-to-moderate effects of acoustic stimulation on sleep-dependent changes in word-pair memory retention as well as a large degree of heterogeneity unexplained. We demonstrate a yearly decline in reported effects, explaining 91.8% of the heterogeneity. In light of these findings, we also presented updated analyses of moderators examined in previous meta-analyses such as age and semantic congruence in word-pair memory tasks (<xref ref-type="table" rid="T3">Table 3</xref>). As expected, the addition of all other coded study characteristics did not explain heterogeneity above solely publication date. These trends describe a yearly decline in the reported efficacy of acoustic stimulation to modulate sleep-related word-pair memory consolidation (or retention). We also report that studies consistently demonstrate significant modulation of slow oscillation power ranging from <italic>d</italic><sub>&#x003B4;</sub> = 0.48 to <italic>d</italic><sub>&#x003B4;</sub> = 2.91. Heterogeneity in observed effects on memory performance was not attributable to that of observed effects of acoustic stimulation on slow oscillation power.</p>
<p>Decline effects are a common phenomenon in psychological sciences (Schooler, <xref ref-type="bibr" rid="B70">2011</xref>; Pietschnig et al., <xref ref-type="bibr" rid="B58">2019</xref>; Schimmack, <xref ref-type="bibr" rid="B68">2020</xref>) and other disciplines (Munaf&#x000F2; et al., <xref ref-type="bibr" rid="B48">2007</xref>). The decline effect describes the phenomena where effect sizes decrease upon subsequent replications of the initial study. The yearly decline presented here likewise explains previously reported significant findings (Wunderlin et al., <xref ref-type="bibr" rid="B84">2021</xref>) for acoustic stimulation overnight memory retention. The inclusion of publication date as a covariate in Model 6 by Wunderlin et al. (<xref ref-type="bibr" rid="B84">2021</xref>) diminished the effect of phase-locked stimulation in non-elderly where the regression coefficient is no longer significant. A finding which, is consistent still with the addition of recently published works with phase-locked acoustic stimulation in non-elderly (children and adults; Prehn-Kristensen et al., <xref ref-type="bibr" rid="B60">2020</xref>; Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>). This is further demonstrated to be robust to the contribution of outliers via our leave-one-out-cross-validation analysis (<xref ref-type="fig" rid="F2">Figure 2B</xref>). Highlighting the consistency of the significant effect of publication date on the heterogeneity in observed effect sizes across studies.</p>
<p>There are many reported mechanisms for the Decline Effect. Recent work (Pietschnig et al., <xref ref-type="bibr" rid="B58">2019</xref>) has demonstrated that underpowered designs of initial studies drive observed effect declines in intelligence literature. Contrary to this, as demonstrated through our leave-one-out-cross-validation analysis of the effect of publication date on the observed effect sizes across studies, we see that no single study drives this trend. A relevant factor here is that the presently analyzed studies do not vary greatly in their sample size (<italic>n</italic> range = 11&#x02013;24) and therefore statistical power. In addition, other work has argued that unpublished findings contribute to effect declines across studies (Schooler, <xref ref-type="bibr" rid="B70">2011</xref>), as well as for selective reporting (Schimmack, <xref ref-type="bibr" rid="B68">2020</xref>). However, while we can not confirm any previously purported mechanisms driving effect declines here, we do suspect that the poor reliability of the task being used in the present literature does decrease statistical power and bias the effect size estimates (<xref ref-type="fig" rid="F3">Figures 3C</xref>, <xref ref-type="fig" rid="F3">D</xref>).</p></sec>
<sec>
<title>4.2. Task reliability</title>
<p>The measurement properties of behavioral and cognitive tests are rarely investigated in behavioral and cognitive neuroscience. In this literature, memory retention is measured using difference scores of word-pair recall tests (post-sleep scores minus pre-sleep scores). Difference scores are sometimes necessary to isolate cognitive processes such as inhibitory control which is often measured using a color-word congruence or Stroop test where the reaction times of congruent trials are subtracted from the reaction time of the incongruent word pairs to create a difference score. However, difference scores are notoriously unreliable (Rogosa and Willett, <xref ref-type="bibr" rid="B65">1983</xref>; Hedge et al., <xref ref-type="bibr" rid="B20">2018</xref>), especially when the pre-test and post-test is highly correlated, which is the case in this literature. Here, we find that the word-pair task expressed as a difference score is not a reliable index of memory retention. When calculating statistical power at different &#x0201C;true&#x0201D; effect sizes, we find that low difference score reliability is associated with lower power (<xref ref-type="fig" rid="F3">Figure 3B</xref>, vertical dotted line). Additionally, this low difference score reliability can result in variable and biased observed effect size estimates as illustrated for a range of simulated true effect sizes (<xref ref-type="fig" rid="F3">Figure 3D</xref>). With lower reliability, effect size estimates for <italic>d</italic><sub>&#x003B4;</sub> and <italic>d</italic><sub><italic>av</italic></sub> are consistently biased toward zero. Taken together, it is evident that the reliability of overnight changes in word-pair retention is inadequate, and drives biased effect size estimates and decreases statistical power. Given we are only able to obtain reliability estimates for a small portion of the presently analyzed study, we can not with confidence evaluate study-level differences in reliability as a potential moderator. However, low task reliability could potentially account for the large heterogeneity in effects across studies.</p></sec>
<sec>
<title>4.3. Limitations</title>
<p>While the above analyses substantially explain a large degree of heterogeneity found in this body of literature, there remain other factors that could be of interest. There is great evidence suggesting that the rhythmic coordination of oscillations during slow-wave sleep is an essential component of memory processes during slow-wave sleep (Isomura et al., <xref ref-type="bibr" rid="B25">2006</xref>; Staresina et al., <xref ref-type="bibr" rid="B73">2015</xref>; Rothschild et al., <xref ref-type="bibr" rid="B66">2017</xref>; Navarrete et al., <xref ref-type="bibr" rid="B49">2020</xref>). It is our view that this is the theoretical and physiological basis for all the studies analyzed in the present study: Ngo et al. (<xref ref-type="bibr" rid="B51">2013b</xref>), Ngo et al. (<xref ref-type="bibr" rid="B52">2015</xref>), Ong et al. (<xref ref-type="bibr" rid="B55">2016</xref>), Leminen et al. (<xref ref-type="bibr" rid="B31">2017</xref>), Henin et al. (<xref ref-type="bibr" rid="B22">2019</xref>), Choi et al. (<xref ref-type="bibr" rid="B5">2019</xref>), Prehn-Kristensen et al. (<xref ref-type="bibr" rid="B60">2020</xref>), Schneider et al. (<xref ref-type="bibr" rid="B69">2020</xref>), Diep et al. (<xref ref-type="bibr" rid="B12">2020</xref>), and Harrington et al. (<xref ref-type="bibr" rid="B19">2021</xref>). Accordingly, the degree to which acoustic stimulation enhances slow-oscillation power and its temporal coupling to spindles could determine the corresponding memory consolidation. The frequency distribution of spindle oscillations and coupling of slow waves and spindle oscillations change during development through adulthood (Purcell et al., <xref ref-type="bibr" rid="B61">2017</xref>; Hahn et al., <xref ref-type="bibr" rid="B18">2020</xref>; Kurz et al., <xref ref-type="bibr" rid="B29">2020</xref>). Additionally, slow wave activity declines with aging in adults (Mander et al., <xref ref-type="bibr" rid="B38">2013</xref>, <xref ref-type="bibr" rid="B39">2017</xref>). In theory, the efficacy of acoustic and other forms of stimulation to enhance slow waves, spindles, and memory consolidation could vary across development and aging. Hence, there is a strong rationale to constrain age to strengthen initial future studies validating an approach. Studies such as that of Schneider and colleagues find older adults can have reduced pre-stimulus slow wave oscillations compared with young adults and from a physiological standpoint this may impact the ability to enhance the slow waves (Schneider et al., <xref ref-type="bibr" rid="B69">2020</xref>). However, the vast majority of studies fail to report the phase-locking estimates and their standard deviations for the arrival of acoustic stimuli relative to slow-oscillations, with the exception of three studies (Ong et al., <xref ref-type="bibr" rid="B55">2016</xref>; Leminen et al., <xref ref-type="bibr" rid="B31">2017</xref>; Papalambros et al., <xref ref-type="bibr" rid="B56">2017</xref>). Similarly, sleep spindles are known to be coupled with slow oscillations and spindle density correlates with memory retention and general cognitive ability (Reynolds et al., <xref ref-type="bibr" rid="B64">2018</xref>; Ujma, <xref ref-type="bibr" rid="B78">2018</xref>) and spindles are considered biomarkers of the memory reconsolidation process (Hennies et al., <xref ref-type="bibr" rid="B23">2016</xref>; Denis et al., <xref ref-type="bibr" rid="B10">2021</xref>). Additionally, studies indicate slow and fast spindles play distinct roles in memory consolidation (Barakat et al., <xref ref-type="bibr" rid="B2">2011</xref>; M&#x000F6;lle et al., <xref ref-type="bibr" rid="B47">2011</xref>; Ayoub et al., <xref ref-type="bibr" rid="B1">2013</xref>; Cox et al., <xref ref-type="bibr" rid="B8">2017</xref>). In line with this, several studies included in this meta-analysis find distinct time scales of effects on slow vs. fast spindles (Ngo et al., <xref ref-type="bibr" rid="B51">2013b</xref>; Ong et al., <xref ref-type="bibr" rid="B55">2016</xref>; Weigenand et al., <xref ref-type="bibr" rid="B81">2016</xref>; Henin et al., <xref ref-type="bibr" rid="B22">2019</xref>) while other studies simply reported effects on fast spindles (Ngo et al., <xref ref-type="bibr" rid="B52">2015</xref>; Leminen et al., <xref ref-type="bibr" rid="B31">2017</xref>; Schneider et al., <xref ref-type="bibr" rid="B69">2020</xref>; Harrington et al., <xref ref-type="bibr" rid="B19">2021</xref>). However, several studies also reported reductions in sleep spindle power during periods of slow-wave sleep without acoustic stimulation (Weigenand et al., <xref ref-type="bibr" rid="B81">2016</xref>; Diep et al., <xref ref-type="bibr" rid="B12">2020</xref>; Schneider et al., <xref ref-type="bibr" rid="B69">2020</xref>). Taken together, studies varied greatly in the extent to which they broadly characterized changes in these features during sleep. Inconsistent reporting of characteristic changes in slow wave and spindle oscillations and their relationships to one another and behavioral measures makes meta-analytically addressing these problems infeasible with such small sample sizes. In line with this, the sparse distribution of study-level characteristics in combination with these small sample sizes could lead to a lack of clear variability between study subgroups, and moderator analyses being insensitive to contributions of study-level characteristics to the heterogeneity. Thus, better tracking and reporting of these metrics in future studies with larger sample sizes would greatly facilitate future studies and meta-analyses.</p></sec>
<sec>
<title>4.4. Conclusions and recommendations for future research</title>
<p>In conclusion, our results demonstrate a yearly decline in the reported efficacy of acoustic stimulation to modulate overnight word-pair retention. The presence of such a trend in small bodies of studies investigating novel interventions is unsurprising (Schimmack, <xref ref-type="bibr" rid="B68">2020</xref>). For the studies presented here, this finding is accompanied by another highly relevant characteristic across studies: the memory tasks used are largely unfit for the present study designs. While several study characteristics were unable to be accounted for in this meta-analysis, and the present literature analyzed is small, we find the results presented here suggestive of the following five recommendations:</p>
<list list-type="bullet">
<list-item><p>Current studies are underpowered due to small sample sizes. Future research should utilize larger samples to increase statistical power.</p></list-item>
<list-item><p>Reliability of the memory task should be strengthened. Increasing the number of sleep and memory test sessions will provide repeated measurements that will enhance reliability, as reliability increases with the number of retests. Additionally, future studies could utilize latent change score modeling using data from three or more sessions, as this affords the ability to dissociate variance and to estimate the change in &#x0201C;true scores&#x0201D; between intervention and control conditions, thereby mitigating the statistical unreliability of memory task difference scores (Hedge et al., <xref ref-type="bibr" rid="B20">2018</xref>).</p></list-item>
<list-item><p>Provided steps are taken to ensure statistically robust measures and reliability, multiple forms of memory tasks may be suitable. For example, a meta-analysis finds acoustic, cue-related targeted memory reactivation enhances memory performance across 91 experiments that employ a variety of declarative and skill acquisition memory tasks. This approach likely taps into similar physiological circuits, given that targeted acoustic cues enhance memory when played during stages 2 and 3 of non-rapid-eye-movement (NREM) sleep but not during other stages of sleep (Hu et al., <xref ref-type="bibr" rid="B24">2020</xref>).</p></list-item>
<list-item><p>Our estimates suggest &#x0007E;34% of subjects are not blind to the study conditions as they report hearing acoustic stimulation during the STIM condition. Many studies reported varying proportions of the participants becoming aware of acoustic stimuli during the evening. This raises the risk, as each study consisted of only a single STIM and SHAM session, that at least some subjects were cognoscente of when they were undergoing the experimental condition. The incorporation of a random-phase stimulation condition, similar to Ngo et al. (<xref ref-type="bibr" rid="B50">2013a</xref>), would be a simple yet effective way to provide a placebo-control in this regard.</p></list-item>
<list-item><p>Mixed-reporting of any behavioral data or EEG features and characteristics hinders accurate accounting for variation in many characteristics (e.g., spindle features) across studies. All of which are research trajectories that could be greatly informative to a variety of disciplines. Future work should prioritize transparency by making their code and data (physiological and behavioral) publicly available (Wilkinson et al., <xref ref-type="bibr" rid="B83">2016</xref>).</p></list-item>
</list>
<p>The application of these recommendations to future studies would greatly reduce the risk of small-sample bias, low statistical power, and confounding psychological effects. They would facilitate future research-synthesis efforts by providing the greater statistical power necessary for analyses of behavioral and physiological effects of acoustic stimulation of slow oscillations.</p></sec></sec>
<sec sec-type="data-availability" id="s5">
<title>Data availability statement</title>
<p>The original contributions presented in the study are publicly available. This data can be found here: <ext-link ext-link-type="uri" xlink:href="https://osf.io/8b49k/">https://osf.io/8b49k/</ext-link>.</p></sec>
<sec sec-type="author-contributions" id="s6">
<title>Author contributions</title>
<p>TH: conceptualization, data curation, methodology, formal analysis, writing&#x02014;original draft, and writing&#x02014;reviewing and editing. MJ: conceptualization, data curation, visualization, methodology, formal analysis, software, and writing&#x02014;reviewing and editing. HR: funding, senior-authorship, and writing&#x02014;reviewing and editing. JC: funding, senior-authorship, supervision, and writing&#x02014;reviewing and editing. All authors contributed to the article and approved the submitted version.</p></sec>
</body>
<back>
<sec sec-type="funding-information" id="s7">
<title>Funding</title>
<p>This study was supported by Brain-Computer Interface Core, State of Connecticut (grant 2806580), HR (PI). The University of Connecticut Institute for Brain and Cognitive Science (IBACS grant 2541260), JC (PI), HR (Co-PI), TH (Graduate Fellow), and MJ (Graduate Fellow). NIH, Collaborative Research and Computational Neuroscience (CRCNS), (grant 1R01DC020097-01) HR (Co-PI). The University of Connecticut College of Liberal Arts and Sciences (CLAS), (grant 2539120), HR (PI) and TH (Graduate Fellow).</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of interest</title>
<p>HR has ownership interest in Elemind Technologies, Inc. and this private company did not sponsor this research. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s8">
<title>Publisher&#x00027;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<sec sec-type="supplementary-material" id="s9">
<title>Supplementary material</title>
<p>The Supplementary Material for this article can be found online at: <ext-link ext-link-type="uri" xlink:href="https://www.frontiersin.org/articles/10.3389/frsle.2023.1082253/full#supplementary-material">https://www.frontiersin.org/articles/10.3389/frsle.2023.1082253/full#supplementary-material</ext-link></p>
<supplementary-material xlink:href="Image_1.JPEG" id="SM1" mimetype="image/jpeg" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</sec>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ayoub</surname> <given-names>A.</given-names></name> <name><surname>Aumann</surname> <given-names>D.</given-names></name> <name><surname>H&#x000F6;rschelmann</surname> <given-names>A.</given-names></name> <name><surname>Kouchekmanesch</surname> <given-names>A.</given-names></name> <name><surname>Paul</surname> <given-names>P.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Differential effects on fast and slow spindle activity, and the sleep slow oscillation in humans with carbamazepine and flunarizine to antagonize voltage-dependent Na&#x0002B; and Ca2&#x0002B; channel activity</article-title>. <source>Sleep</source> <volume>36</volume>, <fpage>905</fpage>&#x02013;<lpage>911</lpage>. <pub-id pub-id-type="doi">10.5665/sleep.2722</pub-id><pub-id pub-id-type="pmid">23729934</pub-id></citation></ref>
<ref id="B2">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barakat</surname> <given-names>M.</given-names></name> <name><surname>Doyon</surname> <given-names>J.</given-names></name> <name><surname>Debas</surname> <given-names>K.</given-names></name> <name><surname>Vandewalle</surname> <given-names>G.</given-names></name> <name><surname>Morin</surname> <given-names>A.</given-names></name> <name><surname>Poirier</surname> <given-names>G.</given-names></name> <etal/></person-group>. (<year>2011</year>). <article-title>Fast and slow spindle involvement in the consolidation of a new motor sequence</article-title>. <source>Behav. Brain Res</source>. <volume>217</volume>, <fpage>117</fpage>&#x02013;<lpage>121</lpage>. <pub-id pub-id-type="doi">10.1016/j.bbr.2010.10.019</pub-id><pub-id pub-id-type="pmid">20974183</pub-id></citation></ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barrett</surname> <given-names>T. R.</given-names></name> <name><surname>Ekstrand</surname> <given-names>B. R.</given-names></name></person-group> (<year>1972</year>). <article-title>Effect of sleep on memory: III. Controlling for time-of-day effects</article-title>. <source>J. Exp. Psychol</source>. <volume>96</volume>, <fpage>321</fpage>&#x02013;<lpage>327</lpage>. <pub-id pub-id-type="doi">10.1037/h0033625</pub-id><pub-id pub-id-type="pmid">4345763</pub-id></citation></ref>
<ref id="B4">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cairney</surname> <given-names>S. A.</given-names></name> <name><surname>Lindsay</surname> <given-names>S.</given-names></name> <name><surname>Sobczak</surname> <given-names>J. M.</given-names></name> <name><surname>Paller</surname> <given-names>K. A.</given-names></name> <name><surname>Gaskell</surname> <given-names>M. G.</given-names></name></person-group> (<year>2016</year>). <article-title>The benefits of targeted memory reactivation for consolidation in sleep are contingent on memory accuracy and direct cue-memory associations</article-title>. <source>Sleep</source> <volume>39</volume>, <fpage>1139</fpage>&#x02013;<lpage>1150</lpage>. <pub-id pub-id-type="doi">10.5665/sleep.5772</pub-id><pub-id pub-id-type="pmid">26856905</pub-id></citation></ref>
<ref id="B5">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Choi</surname> <given-names>J.</given-names></name> <name><surname>Won</surname> <given-names>K.</given-names></name> <name><surname>Jun</surname> <given-names>S. C.</given-names></name></person-group> (<year>2019</year>). <article-title>Acoustic stimulation following sleep spindle activity may enhance procedural memory consolidation during a nap</article-title>. <source>IEEE Access</source> <volume>7</volume>, <fpage>56297</fpage>&#x02013;<lpage>56307</lpage>. <pub-id pub-id-type="doi">10.1109/ACCESS.2019.2913457</pub-id></citation>
</ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chrobak</surname> <given-names>J. J.</given-names></name> <name><surname>Buzs&#x000E1;ki</surname> <given-names>G.</given-names></name></person-group> (<year>1998</year>). <article-title>Operational dynamics in the hippocampal-entorhinal axis</article-title>. <source>Neurosci. Biobehav. Rev</source>. <volume>22</volume>, <fpage>303</fpage>&#x02013;<lpage>310</lpage>. <pub-id pub-id-type="doi">10.1016/S0149-7634(97)00016-X</pub-id><pub-id pub-id-type="pmid">9579320</pub-id></citation></ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cowan</surname> <given-names>E.</given-names></name> <name><surname>Liu</surname> <given-names>A.</given-names></name> <name><surname>Henin</surname> <given-names>S.</given-names></name> <name><surname>Kothare</surname> <given-names>S.</given-names></name> <name><surname>Devinsky</surname> <given-names>O.</given-names></name> <name><surname>Davachi</surname> <given-names>L.</given-names></name></person-group> (<year>2020</year>). <article-title>Sleep spindles promote the restructuring of memory representations in ventromedial prefrontal cortex through enhanced hippocampal-cortical functional connectivity</article-title>. <source>J. Neurosci</source>. <volume>40</volume>, <fpage>1909</fpage>&#x02013;<lpage>1919</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.1946-19.2020</pub-id><pub-id pub-id-type="pmid">31959699</pub-id></citation></ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cox</surname> <given-names>R.</given-names></name> <name><surname>Schapiro</surname> <given-names>A. C.</given-names></name> <name><surname>Manoach</surname> <given-names>D. S.</given-names></name> <name><surname>Stickgold</surname> <given-names>R.</given-names></name></person-group> (<year>2017</year>). <article-title>Individual differences in frequency and topography of slow and fast sleep spindles</article-title>. <source>Front. Hum. Neurosci</source>. <volume>11</volume>:<fpage>433</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2017.00433</pub-id><pub-id pub-id-type="pmid">28928647</pub-id></citation></ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>De Gennaro</surname> <given-names>L.</given-names></name> <name><surname>Ferrara</surname> <given-names>M.</given-names></name></person-group> (<year>2003</year>). <article-title>Sleep spindles: an overview</article-title>. <source>Sleep Med. Rev</source>. <volume>7</volume>, <fpage>423</fpage>&#x02013;<lpage>440</lpage>. <pub-id pub-id-type="doi">10.1053/smrv.2002.0252</pub-id><pub-id pub-id-type="pmid">14573378</pub-id></citation></ref>
<ref id="B10">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Denis</surname> <given-names>D.</given-names></name> <name><surname>Mylonas</surname> <given-names>D.</given-names></name> <name><surname>Poskanzer</surname> <given-names>C.</given-names></name> <name><surname>Bursal</surname> <given-names>V.</given-names></name> <name><surname>Payne</surname> <given-names>J. D.</given-names></name> <name><surname>Stickgold</surname> <given-names>R.</given-names></name></person-group> (<year>2021</year>). <article-title>Sleep spindles preferentially consolidate weakly encoded memories</article-title>. <source>J. Neurosci</source>. <volume>41</volume>, <fpage>4088</fpage>&#x02013;<lpage>4099</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.0818-20.2021</pub-id><pub-id pub-id-type="pmid">33741722</pub-id></citation></ref>
<ref id="B11">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Diekelmann</surname> <given-names>S.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name></person-group> (<year>2010</year>). <article-title>The memory function of sleep</article-title>. <source>Nat. Rev. Neurosci</source>. <volume>11</volume>, <fpage>114</fpage>&#x02013;<lpage>126</lpage>. <pub-id pub-id-type="doi">10.1038/nrn2762</pub-id><pub-id pub-id-type="pmid">20046194</pub-id></citation></ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Diep</surname> <given-names>C.</given-names></name> <name><surname>Ftouni</surname> <given-names>S.</given-names></name> <name><surname>Manousakis</surname> <given-names>J. E.</given-names></name> <name><surname>Nicholas</surname> <given-names>C. L.</given-names></name> <name><surname>Drummond</surname> <given-names>S. P. A.</given-names></name> <name><surname>Anderson</surname> <given-names>C.</given-names></name></person-group> (<year>2020</year>). <article-title>Acoustic slow wave sleep enhancement via a novel, automated device improves executive function in middle-aged men</article-title>. <source>Sleep</source> <volume>43</volume>:<fpage>zsz197</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsz197</pub-id><pub-id pub-id-type="pmid">31691831</pub-id></citation></ref>
<ref id="B13">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dijk</surname> <given-names>D. J.</given-names></name> <name><surname>James</surname> <given-names>L. M.</given-names></name> <name><surname>Peters</surname> <given-names>S.</given-names></name> <name><surname>Walsh</surname> <given-names>J. K.</given-names></name> <name><surname>Deacon</surname> <given-names>S.</given-names></name></person-group> (<year>2010</year>). <article-title>Sex differences and the effect of gaboxadol and zolpidem on EEG power spectra in NREM and REM sleep</article-title>. <source>J. Psychopharmacol</source>. <volume>24</volume>, <fpage>1613</fpage>&#x02013;<lpage>1618</lpage>. <pub-id pub-id-type="doi">10.1177/0269881109105788</pub-id><pub-id pub-id-type="pmid">19487320</pub-id></citation></ref>
<ref id="B14">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Djonlagic</surname> <given-names>I.</given-names></name> <name><surname>Mariani</surname> <given-names>S.</given-names></name> <name><surname>Fitzpatrick</surname> <given-names>A. L.</given-names></name> <name><surname>Van Der Klei</surname> <given-names>V. M. G. T. H.</given-names></name> <name><surname>Johnson</surname> <given-names>D. A.</given-names></name> <name><surname>Wood</surname> <given-names>A. C.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>Macro and micro sleep architecture and cognitive performance in older adults</article-title>. <source>Nat. Hum. Behav</source>. <volume>5</volume>, <fpage>123</fpage>&#x02013;<lpage>145</lpage>. <pub-id pub-id-type="doi">10.1038/s41562-020-00964-y</pub-id><pub-id pub-id-type="pmid">33328625</pub-id></citation></ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ellenbogen</surname> <given-names>J. M.</given-names></name> <name><surname>Hu</surname> <given-names>P. T.</given-names></name> <name><surname>Payne</surname> <given-names>J. D.</given-names></name> <name><surname>Titone</surname> <given-names>D.</given-names></name> <name><surname>Walker</surname> <given-names>M. P.</given-names></name></person-group> (<year>2007</year>). <article-title>Human relational memory requires time and sleep</article-title>. <source>Proc. Natl. Acad. Sci. U.S.A</source>. <volume>104</volume>, <fpage>7723</fpage>&#x02013;<lpage>7728</lpage>. <pub-id pub-id-type="doi">10.1073/pnas.0700094104</pub-id><pub-id pub-id-type="pmid">17449637</pub-id></citation></ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Faraone</surname> <given-names>S. V.</given-names></name> <name><surname>Newcorn</surname> <given-names>J. H.</given-names></name> <name><surname>Cipriani</surname> <given-names>A.</given-names></name> <name><surname>Brandeis</surname> <given-names>D.</given-names></name> <name><surname>Kaiser</surname> <given-names>A.</given-names></name> <name><surname>Hohmann</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Placebo and nocebo responses in randomised, controlled trials of medications for ADHD: a systematic review and meta-analysis</article-title>. <source>Mol. Psychiatry</source> <volume>27</volume>, <fpage>212</fpage>&#x02013;<lpage>219</lpage>. <pub-id pub-id-type="doi">10.1038/s41380-021-01134-w</pub-id><pub-id pub-id-type="pmid">33972692</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fogel</surname> <given-names>S. M.</given-names></name> <name><surname>Smith</surname> <given-names>C. T.</given-names></name></person-group> (<year>2011</year>). <article-title>The function of the sleep spindle: a physiological index of intelligence and a mechanism for sleep-dependent memory consolidation</article-title>. <source>Neurosci. Biobehav. Rev</source>. <volume>35</volume>, <fpage>1154</fpage>&#x02013;<lpage>1165</lpage>. <pub-id pub-id-type="doi">10.1016/j.neubiorev.2010.12.003</pub-id><pub-id pub-id-type="pmid">21167865</pub-id></citation></ref>
<ref id="B18">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hahn</surname> <given-names>M. A.</given-names></name> <name><surname>Heib</surname> <given-names>D.</given-names></name> <name><surname>Schabus</surname> <given-names>M.</given-names></name> <name><surname>Hoedlmoser</surname> <given-names>K.</given-names></name> <name><surname>Helfrich</surname> <given-names>R. F.</given-names></name></person-group> (<year>2020</year>). <article-title>Slow oscillation-spindle coupling predicts enhanced memory formation from childhood to adolescence</article-title>. <source>eLife</source> <volume>9</volume>:<fpage>e53730</fpage>. <pub-id pub-id-type="doi">10.7554/eLife.53730</pub-id><pub-id pub-id-type="pmid">32579108</pub-id></citation></ref>
<ref id="B19">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Harrington</surname> <given-names>M. O.</given-names></name> <name><surname>Ngo</surname> <given-names>H.-V. V.</given-names></name> <name><surname>Cairney</surname> <given-names>S. A.</given-names></name></person-group> (<year>2021</year>). <article-title>No benefit of auditory closed-loop stimulation on memory for semantically-incongruent associations</article-title>. <source>Neurobiol. Learn. Mem</source>. <volume>183</volume>:<fpage>107482</fpage>. <pub-id pub-id-type="doi">10.1016/j.nlm.2021.107482</pub-id><pub-id pub-id-type="pmid">34182134</pub-id></citation></ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hedge</surname> <given-names>C.</given-names></name> <name><surname>Powell</surname> <given-names>G.</given-names></name> <name><surname>Sumner</surname> <given-names>P.</given-names></name></person-group> (<year>2018</year>). <article-title>The reliability paradox: why robust cognitive tasks do not produce reliable individual differences</article-title>. <source>Behav. Res. Methods</source> <volume>50</volume>, <fpage>1166</fpage>&#x02013;<lpage>1186</lpage>. <pub-id pub-id-type="doi">10.3758/s13428-017-0935-1</pub-id><pub-id pub-id-type="pmid">28726177</pub-id></citation></ref>
<ref id="B21">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Helfrich</surname> <given-names>R. F.</given-names></name> <name><surname>Mander</surname> <given-names>B. A.</given-names></name> <name><surname>Jagust</surname> <given-names>W. J.</given-names></name> <name><surname>Knight</surname> <given-names>R. T.</given-names></name> <name><surname>Walker</surname> <given-names>M. P.</given-names></name></person-group> (<year>2018</year>). <article-title>Old brains come uncoupled in sleep: slow wave-spindle synchrony, brain atrophy, and forgetting</article-title>. <source>Neuron</source> <volume>97</volume>, <fpage>221</fpage>&#x02013;<lpage>230.e4</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuron.2017.11.020</pub-id><pub-id pub-id-type="pmid">29249289</pub-id></citation></ref>
<ref id="B22">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Henin</surname> <given-names>S.</given-names></name> <name><surname>Borges</surname> <given-names>H.</given-names></name> <name><surname>Shankar</surname> <given-names>A.</given-names></name> <name><surname>Sarac</surname> <given-names>C.</given-names></name> <name><surname>Melloni</surname> <given-names>L.</given-names></name> <name><surname>Friedman</surname> <given-names>D.</given-names></name> <etal/></person-group>. (<year>2019</year>). <article-title>Closed-loop acoustic stimulation enhances sleep oscillations but not memory performance</article-title>. <source>eNeuro</source> 6. <pub-id pub-id-type="doi">10.1523/ENEURO.0306-19.2019</pub-id><pub-id pub-id-type="pmid">31604814</pub-id></citation></ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hennies</surname> <given-names>N.</given-names></name> <name><surname>Ralph</surname> <given-names>M. A. L.</given-names></name> <name><surname>Kempkes</surname> <given-names>M.</given-names></name> <name><surname>Cousins</surname> <given-names>J. N.</given-names></name> <name><surname>Lewis</surname> <given-names>P. A.</given-names></name></person-group> (<year>2016</year>). <article-title>Sleep spindle density predicts the effect of prior knowledge on memory consolidation</article-title>. <source>J. Neurosci</source>. <volume>36</volume>, <fpage>3799</fpage>&#x02013;<lpage>3810</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.3162-15.2016</pub-id><pub-id pub-id-type="pmid">27030764</pub-id></citation></ref>
<ref id="B24">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hu</surname> <given-names>X.</given-names></name> <name><surname>Cheng</surname> <given-names>L. Y.</given-names></name> <name><surname>Chiu</surname> <given-names>M. H.</given-names></name> <name><surname>Paller</surname> <given-names>K. A.</given-names></name></person-group> (<year>2020</year>). <article-title>Promoting memory consolidation during sleep: a meta-analysis of targeted memory reactivation</article-title>. <source>Psychol. Bull</source>. <volume>146</volume>, <fpage>218</fpage>&#x02013;<lpage>244</lpage>. <pub-id pub-id-type="doi">10.1037/bul0000223</pub-id><pub-id pub-id-type="pmid">32027149</pub-id></citation></ref>
<ref id="B25">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Isomura</surname> <given-names>Y.</given-names></name> <name><surname>Sirota</surname> <given-names>A.</given-names></name> <name><surname>&#x000D6;zen</surname> <given-names>S.</given-names></name> <name><surname>Montgomery</surname> <given-names>S.</given-names></name> <name><surname>Mizuseki</surname> <given-names>K.</given-names></name> <name><surname>Henze</surname> <given-names>D. A.</given-names></name> <etal/></person-group>. (<year>2006</year>). <article-title>Integration and segregation of activity in entorhinal-hippocampal subregions by neocortical slow oscillations</article-title>. <source>Neuron</source> <volume>52</volume>, <fpage>871</fpage>&#x02013;<lpage>882</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuron.2006.10.023</pub-id><pub-id pub-id-type="pmid">17145507</pub-id></citation></ref>
<ref id="B26">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Johnson</surname> <given-names>B. T.</given-names></name> <name><surname>Huedo-Medina</surname> <given-names>T. B.</given-names></name></person-group> (<year>2011</year>). <article-title>Depicting estimates using the intercept in meta-regression models: the moving constant technique</article-title>. <source>Res. Synth. Methods</source> <volume>2</volume>, <fpage>204</fpage>&#x02013;<lpage>220</lpage>. <pub-id pub-id-type="doi">10.1002/jrsm.49</pub-id><pub-id pub-id-type="pmid">24920964</pub-id></citation></ref>
<ref id="B27">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kersant&#x000E9;</surname> <given-names>F.</given-names></name> <name><surname>Purple</surname> <given-names>R. J.</given-names></name> <name><surname>Jones</surname> <given-names>M. W.</given-names></name></person-group> (<year>2023</year>). <article-title>The GABAA receptor modulator zolpidem augments hippocampal-prefrontal coupling during non-REM sleep</article-title>. <source>Neuropsychopharmacology</source> <volume>48</volume>, <fpage>594</fpage>&#x02013;<lpage>604</lpage>. <pub-id pub-id-type="doi">10.1038/s41386-022-01355-9</pub-id><pub-id pub-id-type="pmid">35717464</pub-id></citation></ref>
<ref id="B28">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Khan</surname> <given-names>H.</given-names></name> <name><surname>Garg</surname> <given-names>A.</given-names></name> <name><surname>Yasmeen</surname> <given-names>N.</given-names></name> <name><surname>Agarwal</surname> <given-names>N. B.</given-names></name> <name><surname>Yadav</surname> <given-names>D. K.</given-names></name> <name><surname>Ashif Khan</surname> <given-names>M.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Zolpidem use and risk of suicide: a systematic review and meta-analysis</article-title>. <source>Psychiatry Res</source>. <volume>316</volume>:<fpage>114777</fpage>. <pub-id pub-id-type="doi">10.1016/j.psychres.2022.114777</pub-id><pub-id pub-id-type="pmid">35985088</pub-id></citation></ref>
<ref id="B29">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kurz</surname> <given-names>E.-M.</given-names></name> <name><surname>Conzelmann</surname> <given-names>A.</given-names></name> <name><surname>Barth</surname> <given-names>G. M.</given-names></name> <name><surname>Renner</surname> <given-names>T. J.</given-names></name> <name><surname>Zinke</surname> <given-names>K.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name></person-group> (<year>2020</year>). <article-title>How do children with autism spectrum disorder form gist memory during sleep? A study of slow oscillation-spindle coupling</article-title>. <source>Sleep</source> <volume>44</volume>:<fpage>zsaa290</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsaa290</pub-id><pub-id pub-id-type="pmid">33367905</pub-id></citation></ref>
<ref id="B30">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lakens</surname> <given-names>D.</given-names></name></person-group> (<year>2013</year>). <article-title>Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and anovas</article-title>. <source>Front. Psychol</source>. <volume>4</volume>:<fpage>863</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2013.00863</pub-id><pub-id pub-id-type="pmid">24324449</pub-id></citation></ref>
<ref id="B31">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Leminen</surname> <given-names>M. M.</given-names></name> <name><surname>Virkkala</surname> <given-names>J.</given-names></name> <name><surname>Saure</surname> <given-names>E.</given-names></name> <name><surname>Paajanen</surname> <given-names>T.</given-names></name> <name><surname>Zee</surname> <given-names>P. C.</given-names></name> <name><surname>Santostasi</surname> <given-names>G.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Enhanced memory consolidation via automatic sound stimulation during non-REM sleep</article-title>. <source>Sleep</source> <volume>40</volume>:<fpage>zsx003</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsx003</pub-id><pub-id pub-id-type="pmid">28364428</pub-id></citation></ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Leong</surname> <given-names>C. W. Y.</given-names></name> <name><surname>Leow</surname> <given-names>J. W. S.</given-names></name> <name><surname>Grunstein</surname> <given-names>R. R.</given-names></name> <name><surname>Naismith</surname> <given-names>S. L.</given-names></name> <name><surname>Teh</surname> <given-names>J. Z.</given-names></name> <name><surname>D&#x00027;Rozario</surname> <given-names>A. L.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>A systematic scoping review of the effects of central nervous system active drugs on sleep spindles and sleep-dependent memory consolidation</article-title>. <source>Sleep Med. Rev</source>. <volume>62</volume>:<fpage>101605</fpage>. <pub-id pub-id-type="doi">10.1016/j.smrv.2022.101605</pub-id><pub-id pub-id-type="pmid">35313262</pub-id></citation></ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Limoges</surname> <given-names>l.</given-names></name> <name><surname>Mottron</surname> <given-names>L.</given-names></name> <name><surname>Bolduc</surname> <given-names>C.</given-names></name> <name><surname>Berthiaume</surname> <given-names>C.</given-names></name> <name><surname>Godbout</surname> <given-names>R.</given-names></name></person-group> (<year>2005</year>). <article-title>Atypical sleep architecture and the autism phenotype</article-title>. <source>Brain</source> <volume>128</volume>, <fpage>1049</fpage>&#x02013;<lpage>1061</lpage>. <pub-id pub-id-type="doi">10.1093/brain/awh425</pub-id><pub-id pub-id-type="pmid">15705609</pub-id></citation></ref>
<ref id="B34">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lindheimer</surname> <given-names>J. B.</given-names></name> <name><surname>O&#x00027;Connor</surname> <given-names>P. J.</given-names></name> <name><surname>Dishman</surname> <given-names>R. K.</given-names></name></person-group> (<year>2015</year>). <article-title>Quantifying the placebo effect in psychological outcomes of exercise training: a meta-analysis of randomized trials</article-title>. <source>Sports Med</source>. <volume>45</volume>, <fpage>693</fpage>&#x02013;<lpage>711</lpage>. <pub-id pub-id-type="doi">10.1007/s40279-015-0303-1</pub-id><pub-id pub-id-type="pmid">25762083</pub-id></citation></ref>
<ref id="B35">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lundahl</surname> <given-names>J.</given-names></name> <name><surname>Deacon</surname> <given-names>S.</given-names></name> <name><surname>Maurice</surname> <given-names>D.</given-names></name> <name><surname>Staner</surname> <given-names>L.</given-names></name></person-group> (<year>2012</year>). <article-title>EEG spectral power density profiles during NREM sleep for gaboxadol and zolpidem in patients with primary insomnia</article-title>. <source>J. Psychopharmacol</source>. <volume>26</volume>, <fpage>1081</fpage>&#x02013;<lpage>1087</lpage>. <pub-id pub-id-type="doi">10.1177/0269881111424457</pub-id><pub-id pub-id-type="pmid">22057018</pub-id></citation></ref>
<ref id="B36">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lustenberger</surname> <given-names>C.</given-names></name> <name><surname>Boyle</surname> <given-names>M. R.</given-names></name> <name><surname>Alagapan</surname> <given-names>S.</given-names></name> <name><surname>Mellin</surname> <given-names>J. M.</given-names></name> <name><surname>Vaughn</surname> <given-names>B. V.</given-names></name> <name><surname>Fr&#x000F6;hlich</surname> <given-names>F.</given-names></name></person-group> (<year>2016</year>). <article-title>Feedback-controlled transcranial alternating current stimulation reveals a functional role of sleep spindles in motor memory consolidation</article-title>. <source>Curr. Biol</source>. <volume>26</volume>, <fpage>2127</fpage>&#x02013;<lpage>2136</lpage>. <pub-id pub-id-type="doi">10.1016/j.cub.2016.06.044</pub-id><pub-id pub-id-type="pmid">27476602</pub-id></citation></ref>
<ref id="B37">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mak-McCully</surname> <given-names>R. A.</given-names></name> <name><surname>Deiss</surname> <given-names>S. R.</given-names></name> <name><surname>Rosen</surname> <given-names>B. Q.</given-names></name> <name><surname>Jung</surname> <given-names>K.-Y.</given-names></name> <name><surname>Sejnowski</surname> <given-names>T. J.</given-names></name> <name><surname>Bastuji</surname> <given-names>H.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>Synchronization of isolated downstates (K-complexes) may be caused by cortically-induced disruption of thalamic spindling</article-title>. <source>PLoS Comput. Biol</source>. <volume>10</volume>:<fpage>e1003855</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pcbi.1003855</pub-id><pub-id pub-id-type="pmid">25255217</pub-id></citation></ref>
<ref id="B38">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mander</surname> <given-names>B. A.</given-names></name> <name><surname>Rao</surname> <given-names>V.</given-names></name> <name><surname>Lu</surname> <given-names>B.</given-names></name> <name><surname>Saletin</surname> <given-names>J. M.</given-names></name> <name><surname>Lindquist</surname> <given-names>J. R.</given-names></name> <name><surname>Ancoli-Israel</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Prefrontal atrophy, disrupted NREM slow waves and impaired hippocampal-dependent memory in aging</article-title>. <source>Nat. Neurosci</source>. <volume>16</volume>, <fpage>357</fpage>&#x02013;<lpage>364</lpage>. <pub-id pub-id-type="doi">10.1038/nn.3324</pub-id><pub-id pub-id-type="pmid">23354332</pub-id></citation></ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mander</surname> <given-names>B. A.</given-names></name> <name><surname>Winer</surname> <given-names>J. R.</given-names></name> <name><surname>Walker</surname> <given-names>M. P.</given-names></name></person-group> (<year>2017</year>). <article-title>Sleep and human aging</article-title>. <source>Neuron</source> <volume>94</volume>, <fpage>19</fpage>&#x02013;<lpage>36</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuron.2017.02.004</pub-id><pub-id pub-id-type="pmid">28384471</pub-id></citation></ref>
<ref id="B40">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marshall</surname> <given-names>L.</given-names></name> <name><surname>Helgad&#x000F3;ttir</surname> <given-names>H.</given-names></name> <name><surname>M&#x000F6;lle</surname> <given-names>M.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name></person-group> (<year>2006</year>). <article-title>Boosting slow oscillations during sleep potentiates memory</article-title>. <source>Nature</source> <volume>444</volume>, <fpage>610</fpage>&#x02013;<lpage>613</lpage>. <pub-id pub-id-type="doi">10.1038/nature05278</pub-id><pub-id pub-id-type="pmid">17086200</pub-id></citation></ref>
<ref id="B41">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Martin</surname> <given-names>N.</given-names></name> <name><surname>Lafortune</surname> <given-names>M.</given-names></name> <name><surname>Godbout</surname> <given-names>J.</given-names></name> <name><surname>Barakat</surname> <given-names>M.</given-names></name> <name><surname>Robillard</surname> <given-names>R.</given-names></name> <name><surname>Poirier</surname> <given-names>G.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Topography of age-related changes in sleep spindles</article-title>. <source>Neurobiol. Aging</source> <volume>34</volume>, <fpage>468</fpage>&#x02013;<lpage>476</lpage>. <pub-id pub-id-type="doi">10.1016/j.neurobiolaging.2012.05.020</pub-id><pub-id pub-id-type="pmid">22809452</pub-id></citation></ref>
<ref id="B42">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Massimini</surname> <given-names>M.</given-names></name> <name><surname>Huber</surname> <given-names>R.</given-names></name> <name><surname>Ferrarelli</surname> <given-names>F.</given-names></name> <name><surname>Hill</surname> <given-names>S.</given-names></name> <name><surname>Tononi</surname> <given-names>G.</given-names></name></person-group> (<year>2004</year>). <article-title>The sleep slow oscillation as a traveling wave</article-title>. <source>J. Neurosci</source>. <volume>24</volume>, <fpage>6862</fpage>&#x02013;<lpage>6870</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.1318-04.2004</pub-id><pub-id pub-id-type="pmid">15295020</pub-id></citation></ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mednick</surname> <given-names>S.</given-names></name> <name><surname>Nakayama</surname> <given-names>K.</given-names></name> <name><surname>Stickgold</surname> <given-names>R.</given-names></name></person-group> (<year>2003</year>). <article-title>Sleep-dependent learning: a nap is as good as a night</article-title>. <source>Nat. Neurosci</source>. <volume>6</volume>, <fpage>697</fpage>&#x02013;<lpage>698</lpage>. <pub-id pub-id-type="doi">10.1038/nn1078</pub-id><pub-id pub-id-type="pmid">12819785</pub-id></citation></ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mednick</surname> <given-names>S. C.</given-names></name> <name><surname>Cai</surname> <given-names>D. J.</given-names></name> <name><surname>Shuman</surname> <given-names>T.</given-names></name> <name><surname>Anagnostaras</surname> <given-names>S.</given-names></name> <name><surname>Wixted</surname> <given-names>J.</given-names></name></person-group> (<year>2011</year>). <article-title>An opportunistic theory of cellular and systems consolidation</article-title>. <source>Trends Neurosci</source>. <volume>34</volume>, <fpage>504</fpage>&#x02013;<lpage>514</lpage>. <pub-id pub-id-type="doi">10.1016/j.tins.2011.06.003</pub-id><pub-id pub-id-type="pmid">21742389</pub-id></citation></ref>
<ref id="B45">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mednick</surname> <given-names>S. C.</given-names></name> <name><surname>McDevitt</surname> <given-names>E. A.</given-names></name> <name><surname>Walsh</surname> <given-names>J. K.</given-names></name> <name><surname>Wamsley</surname> <given-names>E.</given-names></name> <name><surname>Paulus</surname> <given-names>M.</given-names></name> <name><surname>Kanady</surname> <given-names>J. C.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>The critical role of sleep spindles in hippocampal-dependent memory: a pharmacology study</article-title>. <source>J. Neurosci</source>. <volume>33</volume>, <fpage>4494</fpage>&#x02013;<lpage>4504</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.3127-12.2013</pub-id><pub-id pub-id-type="pmid">23467365</pub-id></citation></ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Miyamoto</surname> <given-names>D.</given-names></name> <name><surname>Hirai</surname> <given-names>D.</given-names></name> <name><surname>Murayama</surname> <given-names>M.</given-names></name></person-group> (<year>2017</year>). <article-title>The roles of cortical slow waves in synaptic plasticity and memory consolidation</article-title>. <source>Front. Neural Circuits</source> <volume>11</volume>:<fpage>92</fpage>. <pub-id pub-id-type="doi">10.3389/fncir.2017.00092</pub-id><pub-id pub-id-type="pmid">29213231</pub-id></citation></ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>M&#x000F6;lle</surname> <given-names>M.</given-names></name> <name><surname>Bergmann</surname> <given-names>T. O.</given-names></name> <name><surname>Marshall</surname> <given-names>L.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name></person-group> (<year>2011</year>). <article-title>Fast and slow spindles during the sleep slow oscillation: disparate coalescence and engagement in memory processing</article-title>. <source>Sleep</source> <volume>34</volume>, <fpage>1411</fpage>&#x02013;<lpage>1421</lpage>. <pub-id pub-id-type="doi">10.5665/SLEEP.1290</pub-id><pub-id pub-id-type="pmid">21966073</pub-id></citation></ref>
<ref id="B48">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Munaf&#x000F2;</surname> <given-names>M. R.</given-names></name> <name><surname>Matheson</surname> <given-names>I. J.</given-names></name> <name><surname>Flint</surname> <given-names>J.</given-names></name></person-group> (<year>2007</year>). <article-title>Association of the DRD2 gene Taq1A polymorphism and alcoholism: a meta-analysis of case-control studies and evidence of publication bias</article-title>. <source>Mol. Psychiatry</source> <volume>12</volume>, <fpage>454</fpage>&#x02013;<lpage>461</lpage>. <pub-id pub-id-type="doi">10.1038/sj.mp.4001938</pub-id><pub-id pub-id-type="pmid">17453061</pub-id></citation></ref>
<ref id="B49">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Navarrete</surname> <given-names>M.</given-names></name> <name><surname>Valderrama</surname> <given-names>M.</given-names></name> <name><surname>Lewis</surname> <given-names>P. A.</given-names></name></person-group> (<year>2020</year>). <article-title>The role of slow-wave sleep rhythms in the cortical-hippocampal loop for memory consolidation</article-title>. <source>Curr. Opin. Behav. Sci</source>. <volume>32</volume>, <fpage>102</fpage>&#x02013;<lpage>110</lpage>. <pub-id pub-id-type="doi">10.1016/j.cobeha.2020.02.006</pub-id></citation>
</ref>
<ref id="B50">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ngo</surname> <given-names>H.-V. V.</given-names></name> <name><surname>Claussen</surname> <given-names>J. C.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name> <name><surname>M&#x000F6;lle</surname> <given-names>M.</given-names></name></person-group> (<year>2013a</year>). <article-title>Induction of slow oscillations by rhythmic acoustic stimulation</article-title>. <source>J. Sleep Res</source>. <volume>22</volume>, <fpage>22</fpage>&#x02013;<lpage>31</lpage>. <pub-id pub-id-type="doi">10.1111/j.1365-2869.2012.01039.x</pub-id><pub-id pub-id-type="pmid">22913273</pub-id></citation></ref>
<ref id="B51">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ngo</surname> <given-names>H.-V. V.</given-names></name> <name><surname>Martinetz</surname> <given-names>T.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name> <name><surname>M&#x000F6;lle</surname> <given-names>M.</given-names></name></person-group> (<year>2013b</year>). <article-title>Auditory closed-loop stimulation of the sleep slow oscillation enhances memory</article-title>. <source>Neuron</source> <volume>78</volume>, <fpage>545</fpage>&#x02013;<lpage>553</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuron.2013.03.006</pub-id><pub-id pub-id-type="pmid">23583623</pub-id></citation></ref>
<ref id="B52">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ngo</surname> <given-names>H.-V. V.</given-names></name> <name><surname>Miedema</surname> <given-names>A.</given-names></name> <name><surname>Faude</surname> <given-names>I.</given-names></name> <name><surname>Martinetz</surname> <given-names>T.</given-names></name> <name><surname>M&#x000F6;lle</surname> <given-names>M.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name></person-group> (<year>2015</year>). <article-title>Driving sleep slow oscillations by auditory closed-loop stimulation-a self-limiting process</article-title>. <source>J. Neurosci</source>. <volume>35</volume>, <fpage>6630</fpage>&#x02013;<lpage>6638</lpage>. <pub-id pub-id-type="doi">10.1523/JNEUROSCI.3133-14.2015</pub-id><pub-id pub-id-type="pmid">25926443</pub-id></citation></ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Niknazar</surname> <given-names>M.</given-names></name> <name><surname>Krishnan</surname> <given-names>G. P.</given-names></name> <name><surname>Bazhenov</surname> <given-names>M.</given-names></name> <name><surname>Mednick</surname> <given-names>S. C.</given-names></name></person-group> (<year>2015</year>). <article-title>Coupling of thalamocortical sleep oscillations are important for memory consolidation in humans</article-title>. <source>PLoS ONE</source> <volume>10</volume>:<fpage>e0144720</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0144720</pub-id><pub-id pub-id-type="pmid">26671283</pub-id></citation></ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nishida</surname> <given-names>M.</given-names></name> <name><surname>Walker</surname> <given-names>M. P.</given-names></name></person-group> (<year>2007</year>). <article-title>Daytime naps, motor memory consolidation and regionally specific sleep spindles</article-title>. <source>PLoS ONE</source> <volume>2</volume>:<fpage>e341</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0000341</pub-id><pub-id pub-id-type="pmid">17406665</pub-id></citation></ref>
<ref id="B55">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ong</surname> <given-names>J. L.</given-names></name> <name><surname>Lo</surname> <given-names>J. C.</given-names></name> <name><surname>Chee</surname> <given-names>N. I. Y. N.</given-names></name> <name><surname>Santostasi</surname> <given-names>G.</given-names></name> <name><surname>Paller</surname> <given-names>K. A.</given-names></name> <name><surname>Zee</surname> <given-names>P. C.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>Effects of phase-locked acoustic stimulation during a nap on EEG spectra and declarative memory consolidation</article-title>. <source>Sleep Med</source>. <volume>20</volume>, <fpage>88</fpage>&#x02013;<lpage>97</lpage>. <pub-id pub-id-type="doi">10.1016/j.sleep.2015.10.016</pub-id><pub-id pub-id-type="pmid">27318231</pub-id></citation></ref>
<ref id="B56">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Papalambros</surname> <given-names>N. A.</given-names></name> <name><surname>Santostasi</surname> <given-names>G.</given-names></name> <name><surname>Malkani</surname> <given-names>R. G.</given-names></name> <name><surname>Braun</surname> <given-names>R.</given-names></name> <name><surname>Weintraub</surname> <given-names>S.</given-names></name> <name><surname>Paller</surname> <given-names>K. A.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Acoustic enhancement of sleep slow oscillations and concomitant memory improvement in older adults</article-title>. <source>Front. Hum. Neurosci</source>. <volume>11</volume>:<fpage>109</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2017.00109</pub-id><pub-id pub-id-type="pmid">28337134</pub-id></citation></ref>
<ref id="B57">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Petersen</surname> <given-names>G. L.</given-names></name> <name><surname>Finnerup</surname> <given-names>N. B.</given-names></name> <name><surname>Colloca</surname> <given-names>L.</given-names></name> <name><surname>Amanzio</surname> <given-names>M.</given-names></name> <name><surname>Price</surname> <given-names>D. D.</given-names></name> <name><surname>Jensen</surname> <given-names>T. S.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>The magnitude of nocebo effects in pain: a meta-analysis</article-title>. <source>Pain</source> <volume>155</volume>, <fpage>1426</fpage>&#x02013;<lpage>1434</lpage>. <pub-id pub-id-type="doi">10.1016/j.pain.2014.04.016</pub-id><pub-id pub-id-type="pmid">24780622</pub-id></citation></ref>
<ref id="B58">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pietschnig</surname> <given-names>J.</given-names></name> <name><surname>Siegel</surname> <given-names>M.</given-names></name> <name><surname>Eder</surname> <given-names>J. S. N.</given-names></name> <name><surname>Gittler</surname> <given-names>G.</given-names></name></person-group> (<year>2019</year>). <article-title>Effect declines are systematic, strong, and ubiquitous: a meta-meta-analysis of the decline effect in intelligence research</article-title>. <source>Front. Psychol</source>. <volume>10</volume>:<fpage>2874</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2019.02874</pub-id><pub-id pub-id-type="pmid">31920891</pub-id></citation></ref>
<ref id="B59">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Plihal</surname> <given-names>W.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name></person-group> (<year>1997</year>). <article-title>Effects of early and late nocturnal sleep on declarative and procedural memory</article-title>. <source>J. Cogn. Neurosci</source>. <volume>9</volume>, <fpage>534</fpage>&#x02013;<lpage>547</lpage>. <pub-id pub-id-type="doi">10.1162/jocn.1997.9.4.534</pub-id><pub-id pub-id-type="pmid">23968216</pub-id></citation></ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Prehn-Kristensen</surname> <given-names>A.</given-names></name> <name><surname>Ngo</surname> <given-names>H.-V. V.</given-names></name> <name><surname>Lentfer</surname> <given-names>L.</given-names></name> <name><surname>Bergh&#x000E4;user</surname> <given-names>J.</given-names></name> <name><surname>Brandes</surname> <given-names>L.</given-names></name> <name><surname>Schulze</surname> <given-names>L.</given-names></name> <etal/></person-group>. (<year>2020</year>). <article-title>Acoustic closed-loop stimulation during sleep improves consolidation of reward-related memory information in healthy children but not in children with attention-deficit hyperactivity disorder</article-title>. <source>Sleep</source> <volume>43</volume>:<fpage>zsaa017</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsaa017</pub-id><pub-id pub-id-type="pmid">32034912</pub-id></citation></ref>
<ref id="B61">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Purcell</surname> <given-names>S. M.</given-names></name> <name><surname>Manoach</surname> <given-names>D. S.</given-names></name> <name><surname>Demanuele</surname> <given-names>C.</given-names></name> <name><surname>Cade</surname> <given-names>B. E.</given-names></name> <name><surname>Mariani</surname> <given-names>S.</given-names></name> <name><surname>Cox</surname> <given-names>R.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Characterizing sleep spindles in 11,630 individuals from the National Sleep Research Resource</article-title>. <source>Nat. Commun</source>. <volume>8</volume>:<fpage>15930</fpage>. <pub-id pub-id-type="doi">10.1038/ncomms15930</pub-id><pub-id pub-id-type="pmid">28649997</pub-id></citation></ref>
<ref id="B62">
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Pustejovsky</surname> <given-names>J. E.</given-names></name></person-group> (<year>2023</year>). <source>Cohen&#x00027;s DZ Makes Me Dizzy When Considering Measurement Error</source>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://jepusto.com">jepusto.com</ext-link></citation>
</ref>
<ref id="B63">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rauchs</surname> <given-names>G.</given-names></name> <name><surname>Schabus</surname> <given-names>M.</given-names></name> <name><surname>Parapatics</surname> <given-names>S.</given-names></name> <name><surname>Bertran</surname> <given-names>F.</given-names></name> <name><surname>Clochon</surname> <given-names>P.</given-names></name> <name><surname>Hot</surname> <given-names>P.</given-names></name> <etal/></person-group>. (<year>2008</year>). <article-title>Is there a link between sleep changes and memory in Alzheimer&#x00027;s disease?</article-title> <source>Neuroreport</source> <volume>19</volume>, <fpage>1159</fpage>&#x02013;<lpage>1162</lpage>. <pub-id pub-id-type="doi">10.1097/WNR.0b013e32830867c4</pub-id><pub-id pub-id-type="pmid">18596620</pub-id></citation></ref>
<ref id="B64">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Reynolds</surname> <given-names>C. M.</given-names></name> <name><surname>Short</surname> <given-names>M. A.</given-names></name> <name><surname>Gradisar</surname> <given-names>M.</given-names></name></person-group> (<year>2018</year>). <article-title>Sleep spindles and cognitive performance across adolescence: a meta-analytic review</article-title>. <source>J. Adolesc</source>. <volume>66</volume>, <fpage>55</fpage>&#x02013;<lpage>70</lpage>. <pub-id pub-id-type="doi">10.1016/j.adolescence.2018.04.003</pub-id><pub-id pub-id-type="pmid">29783103</pub-id></citation></ref>
<ref id="B65">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rogosa</surname> <given-names>D. R.</given-names></name> <name><surname>Willett</surname> <given-names>J. B.</given-names></name></person-group> (<year>1983</year>). <article-title>Demonstrating the reliability of the difference score in the measurement of change</article-title>. <source>J. Educ. Measure</source>. <volume>20</volume>, <fpage>335</fpage>&#x02013;<lpage>343</lpage>. <pub-id pub-id-type="doi">10.1111/j.1745-3984.1983.tb00211.x</pub-id></citation>
</ref>
<ref id="B66">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rothschild</surname> <given-names>G.</given-names></name> <name><surname>Eban</surname> <given-names>E.</given-names></name> <name><surname>Frank</surname> <given-names>L. M.</given-names></name></person-group> (<year>2017</year>). <article-title>A cortical-hippocampal-cortical loop of information processing during memory consolidation</article-title>. <source>Nat. Neurosci</source>. <volume>20</volume>, <fpage>251</fpage>&#x02013;<lpage>259</lpage>. <pub-id pub-id-type="doi">10.1038/nn.4457</pub-id><pub-id pub-id-type="pmid">27941790</pub-id></citation></ref>
<ref id="B67">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sanchez-Vives</surname> <given-names>M. V.</given-names></name></person-group> (<year>2020</year>). <article-title>Origin and dynamics of cortical slow oscillations</article-title>. <source>Curr. Opin. Physiol</source>. <volume>15</volume>, <fpage>217</fpage>&#x02013;<lpage>223</lpage>. <pub-id pub-id-type="doi">10.1016/j.cophys.2020.04.005</pub-id></citation>
</ref>
<ref id="B68">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schimmack</surname> <given-names>U.</given-names></name></person-group> (<year>2020</year>). <article-title>A meta-psychological perspective on the decade of replication failures in social psychology</article-title>. <source>Can. Psychol</source>. <volume>61</volume>, <fpage>364</fpage>&#x02013;<lpage>376</lpage>. <pub-id pub-id-type="doi">10.1037/cap0000246</pub-id></citation>
</ref>
<ref id="B69">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schneider</surname> <given-names>J.</given-names></name> <name><surname>Lewis</surname> <given-names>P. A.</given-names></name> <name><surname>Koester</surname> <given-names>D.</given-names></name> <name><surname>Born</surname> <given-names>J.</given-names></name> <name><surname>Ngo</surname> <given-names>H.-V. V.</given-names></name></person-group> (<year>2020</year>). <article-title>Susceptibility to auditory closed-loop stimulation of sleep slow oscillations changes with age</article-title>. <source>Sleep</source> <volume>43</volume>:<fpage>zsaa111</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsaa111</pub-id><pub-id pub-id-type="pmid">32562487</pub-id></citation></ref>
<ref id="B70">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schooler</surname> <given-names>J.</given-names></name></person-group> (<year>2011</year>). <article-title>Unpublished results hide the decline effect</article-title>. <source>Nature</source> <volume>470</volume>, <fpage>437</fpage>&#x02013;<lpage>437</lpage>. <pub-id pub-id-type="doi">10.1038/470437a</pub-id><pub-id pub-id-type="pmid">21350443</pub-id></citation></ref>
<ref id="B71">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Simon</surname> <given-names>K. C.</given-names></name> <name><surname>Whitehurst</surname> <given-names>L. N.</given-names></name> <name><surname>Zhang</surname> <given-names>J.</given-names></name> <name><surname>Mednick</surname> <given-names>S. C.</given-names></name></person-group> (<year>2021</year>). <article-title>Zolpidem maintains memories for negative emotions across a night of sleep</article-title>. <source>Affect. Sci</source>. <volume>3</volume>, <fpage>389</fpage>&#x02013;<lpage>399</lpage>. <pub-id pub-id-type="doi">10.1007/s42761-021-00079-1</pub-id><pub-id pub-id-type="pmid">35791418</pub-id></citation></ref>
<ref id="B72">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stanyer</surname> <given-names>E. C.</given-names></name> <name><surname>Baniqued</surname> <given-names>P. D. E.</given-names></name> <name><surname>Awais</surname> <given-names>M.</given-names></name> <name><surname>Kouara</surname> <given-names>L.</given-names></name> <name><surname>Davies</surname> <given-names>A. G.</given-names></name> <name><surname>Killan</surname> <given-names>E. C.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>The impact of acoustic stimulation during sleep on memory and sleep architecture: a meta-analysis</article-title>. <source>J. Sleep Res</source>. <volume>31</volume>:<fpage>e13385</fpage>. <pub-id pub-id-type="doi">10.1111/jsr.13385</pub-id><pub-id pub-id-type="pmid">34850995</pub-id></citation></ref>
<ref id="B73">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Staresina</surname> <given-names>B. P.</given-names></name> <name><surname>Bergmann</surname> <given-names>T. O.</given-names></name> <name><surname>Bonnefond</surname> <given-names>M.</given-names></name> <name><surname>van der Meij</surname> <given-names>R.</given-names></name> <name><surname>Jensen</surname> <given-names>O.</given-names></name> <name><surname>Deuker</surname> <given-names>L.</given-names></name> <etal/></person-group>. (<year>2015</year>). <article-title>Hierarchical nesting of slow oscillations, spindles and ripples in the human hippocampus during sleep</article-title>. <source>Nat. Neurosci</source>. <volume>18</volume>, <fpage>1679</fpage>&#x02013;<lpage>1686</lpage>. <pub-id pub-id-type="doi">10.1038/nn.4119</pub-id><pub-id pub-id-type="pmid">26389842</pub-id></citation></ref>
<ref id="B74">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Steriade</surname> <given-names>M.</given-names></name></person-group> (<year>2006</year>). <article-title>Grouping of brain rhythms in corticothalamic systems</article-title>. <source>Neuroscience</source> <volume>137</volume>, <fpage>1087</fpage>&#x02013;<lpage>1106</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroscience.2005.10.029</pub-id><pub-id pub-id-type="pmid">16343791</pub-id></citation></ref>
<ref id="B75">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stokes</surname> <given-names>P. A.</given-names></name> <name><surname>Rath</surname> <given-names>P.</given-names></name> <name><surname>Possidente</surname> <given-names>T.</given-names></name> <name><surname>He</surname> <given-names>M.</given-names></name> <name><surname>Purcell</surname> <given-names>S.</given-names></name> <name><surname>Manoach</surname> <given-names>D. S.</given-names></name> <etal/></person-group>. (<year>2022</year>). <article-title>Transient oscillation dynamics during sleep provide a robust basis for electroencephalographic phenotyping and biomarker identification</article-title>. <source>Sleep</source> <volume>46</volume>:<fpage>zsac223</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsac223</pub-id><pub-id pub-id-type="pmid">36453975</pub-id></citation></ref>
<ref id="B76">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tessier</surname> <given-names>S.</given-names></name> <name><surname>Lambert</surname> <given-names>A.</given-names></name> <name><surname>Chicoine</surname> <given-names>M.</given-names></name> <name><surname>Scherzer</surname> <given-names>P.</given-names></name> <name><surname>Souli&#x000E8;res</surname> <given-names>I.</given-names></name> <name><surname>Godbout</surname> <given-names>R.</given-names></name></person-group> (<year>2015</year>). <article-title>Intelligence measures and stage 2 sleep in typically-developing and autistic children</article-title>. <source>Int. J. Psychophysiol</source>. <volume>97</volume>, <fpage>58</fpage>&#x02013;<lpage>65</lpage>. <pub-id pub-id-type="doi">10.1016/j.ijpsycho.2015.05.003</pub-id><pub-id pub-id-type="pmid">25958790</pub-id></citation></ref>
<ref id="B77">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tononi</surname> <given-names>G.</given-names></name> <name><surname>Cirelli</surname> <given-names>C.</given-names></name></person-group> (<year>2006</year>). <article-title>Sleep function and synaptic homeostasis</article-title>. <source>Sleep Med. Rev</source>. <volume>10</volume>, <fpage>49</fpage>&#x02013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1016/j.smrv.2005.05.002</pub-id><pub-id pub-id-type="pmid">16376591</pub-id></citation></ref>
<ref id="B78">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ujma</surname> <given-names>P. P.</given-names></name></person-group> (<year>2018</year>). <article-title>Sleep spindles and general cognitive ability&#x02013;A meta-analysis</article-title>. <source>Sleep Spindles Cortical Up States</source> <volume>2</volume>, <fpage>1</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.1556/2053.1.2016.001</pub-id></citation>
</ref>
<ref id="B79">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Viechtbauer</surname> <given-names>W.</given-names></name></person-group> (<year>2010</year>). <article-title>Conducting meta-analyses in R with the metafor package</article-title>. <source>J. Stat. Softw</source>. <volume>36</volume>, <fpage>1</fpage>&#x02013;<lpage>48</lpage>. <pub-id pub-id-type="doi">10.18637/jss.v036.i03</pub-id></citation>
</ref>
<ref id="B80">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wamsley</surname> <given-names>E. J.</given-names></name> <name><surname>Tucker</surname> <given-names>M. A.</given-names></name> <name><surname>Shinn</surname> <given-names>A. K.</given-names></name> <name><surname>Ono</surname> <given-names>K. E.</given-names></name> <name><surname>McKinley</surname> <given-names>S. K.</given-names></name> <name><surname>Ely</surname> <given-names>A. V.</given-names></name> <etal/></person-group>. (<year>2012</year>). <article-title>Reduced sleep spindles and spindle coherence in schizophrenia: mechanisms of impaired memory consolidation?</article-title> <source>Biol. Psychiatry</source> <volume>71</volume>, <fpage>154</fpage>&#x02013;<lpage>161</lpage>. <pub-id pub-id-type="doi">10.1016/j.biopsych.2011.08.008</pub-id><pub-id pub-id-type="pmid">21967958</pub-id></citation></ref>
<ref id="B81">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Weigenand</surname> <given-names>A.</given-names></name> <name><surname>M&#x000F6;lle</surname> <given-names>M.</given-names></name> <name><surname>Werner</surname> <given-names>F.</given-names></name> <name><surname>Martinetz</surname> <given-names>T.</given-names></name> <name><surname>Marshall</surname> <given-names>L.</given-names></name></person-group> (<year>2016</year>). <article-title>Timing matters: open-loop stimulation does not improve overnight consolidation of word pairs in humans</article-title>. <source>Eur. J. Neurosci</source>. <volume>44</volume>, <fpage>2357</fpage>&#x02013;<lpage>2368</lpage>. <pub-id pub-id-type="doi">10.1111/ejn.13334</pub-id><pub-id pub-id-type="pmid">27422437</pub-id></citation></ref>
<ref id="B82">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Weng</surname> <given-names>Y.-Y.</given-names></name> <name><surname>Lei</surname> <given-names>X.</given-names></name> <name><surname>Yu</surname> <given-names>J.</given-names></name></person-group> (<year>2020</year>). <article-title>Sleep spindle abnormalities related to Alzheimer&#x00027;s disease: a systematic mini-review</article-title>. <source>Sleep Med</source>. <volume>75</volume>, <fpage>37</fpage>&#x02013;<lpage>44</lpage>. <pub-id pub-id-type="doi">10.1016/j.sleep.2020.07.044</pub-id><pub-id pub-id-type="pmid">32853916</pub-id></citation></ref>
<ref id="B83">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wilkinson</surname> <given-names>M. D.</given-names></name> <name><surname>Dumontier</surname> <given-names>M.</given-names></name> <name><surname>Aalbersberg</surname> <given-names>I. J.</given-names></name> <name><surname>Appleton</surname> <given-names>G.</given-names></name> <name><surname>Axton</surname> <given-names>M.</given-names></name> <name><surname>Baak</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>The FAIR guiding principles for scientific data management and stewardship</article-title>. <source>Sci. Data</source> <volume>3</volume>:<fpage>160018</fpage>.<pub-id pub-id-type="pmid">30890711</pub-id></citation></ref>
<ref id="B84">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wunderlin</surname> <given-names>M.</given-names></name> <name><surname>Z&#x000FC;st</surname> <given-names>M. A.</given-names></name> <name><surname>Hertenstein</surname> <given-names>E.</given-names></name> <name><surname>Feh&#x000E9;r</surname> <given-names>K. D.</given-names></name> <name><surname>Schneider</surname> <given-names>C. L.</given-names></name> <name><surname>Kl&#x000F6;ppel</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2021</year>). <article-title>Modulating overnight memory consolidation by acoustic stimulation during slow-wave sleep: a systematic review and meta-analysis</article-title>. <source>Sleep</source> <volume>44</volume>:<fpage>zsaa296</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsaa296</pub-id><pub-id pub-id-type="pmid">33406249</pub-id></citation></ref>
<ref id="B85">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zeller</surname> <given-names>C. J.</given-names></name> <name><surname>Z&#x000FC;st</surname> <given-names>M. A.</given-names></name> <name><surname>Wunderlin</surname> <given-names>M.</given-names></name> <name><surname>Nissen</surname> <given-names>C.</given-names></name> <name><surname>Kl&#x000F6;ppel</surname> <given-names>S.</given-names></name></person-group> (<year>2023</year>). <article-title>The promise of portable remote auditory stimulation tools to enhance slow-wave sleep and prevent cognitive decline</article-title>. <source>J. Sleep Res</source>. <volume>2023</volume>:<fpage>e13818</fpage>. <pub-id pub-id-type="doi">10.1111/jsr.13818</pub-id><pub-id pub-id-type="pmid">36631001</pub-id></citation></ref>
<ref id="B86">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>J.</given-names></name> <name><surname>Yetton</surname> <given-names>B.</given-names></name> <name><surname>Whitehurst</surname> <given-names>L. N.</given-names></name> <name><surname>Naji</surname> <given-names>M.</given-names></name> <name><surname>Mednick</surname> <given-names>S. C.</given-names></name></person-group> (<year>2020</year>). <article-title>The effect of zolpidem on memory consolidation over a night of sleep</article-title>. <source>Sleep</source> <volume>43</volume>:<fpage>zsaa084</fpage>. <pub-id pub-id-type="doi">10.1093/sleep/zsaa084</pub-id><pub-id pub-id-type="pmid">32330272</pub-id></citation></ref>
</ref-list> 
</back>
</article> 