Children (6–10 years) and their caregivers were recruited across the United States using flyers shared over social media, listservs, and through the Pitt + Me research registry. Children were required to be English speaking and without a significant behavior or sleep diagnosis that impact their daily life (per caregiver report). The caregiver was required to complete daily sleep diary questions during their involvement of the study and report regular involvement in their child’s bedtime routine (at least four nights a week). Participants completed informed consent and data were collected between September and December 2021 (COVID-19 pandemic, Delta variant prominence).

The aim of the original study was to examine differences in sleep health for children with SS compared to peers with NSS. Therefore, two groups of children were recruited. Participants were included in the SS group if their parent endorsed 5+/7 screening questions [based off the Sensory Profile-2 (Dunn, 2014) tactile/oral tactile sensitivity questions; provided as Supplementary Material in previous manuscript (Hartman et al., 2022)]. Additionally, a diagnosis of autism, attention-deficit hyperactivity disorder, or Down’s syndrome excluded participants in both groups, as these diagnoses have different neurological and medical components beyond sensory sensitivities that may impact sleep. A demographics survey was utilized to record key variables and describe participants.

Demographics of the whole sample are reported in our previous paper (Hartman et al., 2022). For this secondary analysis, a total of 35 children were able to complete the data collection protocol with sufficient data for this analysis (at least four evenings of settling down time data from actigraphy and sleep diary): 17 children with SS and 18 with NSS (Table 1). The average age in both groups was around 7.7 years. The SS group was majority male (65%) and white, non-Hispanic (59%). The NSS group was majority female (61%) and white, non-Hispanic (89%).

The wrist-worn ActiGraph GT9X (ActiGraph Corp, LLC, Pensacola, FL) was used to collect motor activity using an internal accelerometer, gyroscope, and magnetometer. Actigraphy data collection was set at a sampling rate of 30 Hz to collect data with 60 s epoch lengths, collected continuously across 2 weeks. Additionally, caregivers completed a morning and evening sleep diary for the two-weeks of data collection. Sleep diary questions of interest in this analysis included the timing of (i) the start of the settling down and (ii) perceived sleep onset. For this secondary data analysis, we included participants who had actigraphy and completed sleep diary entries for at least four evenings/nights to ensure that sufficient data was available for the analysis of the child’s settling down experience.

A total of 227 evenings of data were collected from the SS group with an average of 13.33 evenings per participant (SD = 2.44). Children in the NSS group provided a total of 203 evenings of data with an average of 11.27 evenings per participant (SD = 2.85).

Raw actigraphy data was processed through ActiLife (Actigraph Corp.), the accompanying proprietary software of the device. Vector magnitude, or activity counts, per minute were extracted for each participant. Sleep periods were first isolated using the Tudor-Locke (Tudor-Locke et al., 2015) algorithm that labeled each minute as “wake” or “sleep”. Then, the study team used a visual inspection and the caregiver reported sleep onset/offset timing within the sleep diary to adjust the identified sleep periods as needed. In the event that the caregiver reported timing different by 30+ minutes, the algorithm-identified time and visual inspection was prioritized. This prioritization was chosen a prioi based on the recommendations from the SBSM Guide to Actigraphy Monitoring (Ancoli-Israel et al., 2015) and the possibility that caregivers may not be with their child when they fall asleep and therefore may need to estimate sleep onset timing. After the sleep periods were isolated, every complete sleep period (no off-wrist time) was analyzed using the Sadeh algorithm (Sadeh et al., 1994). The Sadeh algorithm is a pediatric-specific algorithm that derives sleep variables from identified sleep periods. These variables include sleep duration, sleep efficiency, sleep onset and offset, among others. For this analysis, we only used the sleep onset variable to isolate the settling down period prior to the sleep period.

Next, we identified the start of settling down time within the actigraphy data. During the initial data collection, caregivers were interviewed and described their child’s regular bedtime routine. Caregivers were instructed to write down the start of their child’s settling down time each night in the sleep diary. This was operationalized in this way: the time when the bedtime routine had ended and the child is attempting to fall asleep. For this analysis, we extracted (i) the reported start of the settling down period from each sleep diary and (ii) sleep onset time indicated using the Sadeh algorithm with the cleaned actigraphy data (as outlined above). Using these time points, we cut and indexed the actigraphy data files containing three-dimensional accelerometer data and vector magnitude for each night per participant. Data files were labeled by group status (SS or NSS). Average settling down length of time was assessed using Hedges’ g effect size calculations, where g > 0.41 indicates minimal practical significance. (Ferguson, 2009). The data were then separated into trials using rolling windows to populate the dataset and to create features (see Classification results section below). Then, features were extracted from all of the trials. The features were picked to specifically demonstrate power and variation within the time windows and characterize some clinically meaningful aspects of the time period (measures of activity magnitude, variability of movement from moment to moment). These features were mean magnitude of activity, maximum magnitude, kurtosis, skewness, Shannon entropy (Shannon, 1948), standard deviation, and interquartile range (definitions and equations are found in the Supplementary Appendix).

The main aim of the analysis was to characterize differences in the actigraphy data during the settling down time for children with SS and NSS using supervised learning. We approached this binary classification problem using a 10-fold cross-validation with all of the features extracted from the data (see above paragraph). k-fold cross validation is used to test a model built to correctly identify a participant with SS from those with NSS across the entire dataset (called a binary classification problem). This approach allowed us to observe whether the approach, the dataset, and the method correctly separate the two groups in the dataset with sensitivity, specificity, and accuracy.

First, we aimed to examine how different algorithms performed the binary classification problem. We trialed random forests, support vector machines, linear and quadratic discriminant analyses, Naïve Bayes classification, and a multilayer perceptron model. Random forests performed the best as evidenced by the highest values of the average area under receiving operator characteristics curve (AUC). This finding indicates that sensitivity (identifying true positives) and specificity (identifying true negatives) were both high. Random forests are a collection of decision tree estimators used for regression and classification, where the decision trees are fit with subsamples of the entire dataset (Breiman, 2001). It is a type of supervised machine learning, where a model is trained with labels (SS or NSS). Training and test datasets are created from the entire dataset using a 4:1 split. The best parameter set for the task is found using a grid search algorithm within the training dataset using cross-validation. The parameters searched were the number of decision trees and maximum depth. The end-result was decided based on majority voting of decision trees.

Rolling windows were used to ‘epoch’ data entries into data windows and features were created from the windowed data. Rolling windows are widely used for feature engineering in machine learning algorithms as a robust approach to populate dataset and create more meaningful features. Data from each settling down time were examined in smaller sections (ranging from 7 to 13 min, see Table 2) and features like maximum magnitude were extracted. Then that “window” was moved down (“stride”) either 1 or 3 min and the new window of data was analyzed. Using rolling windows allowed for patterns to be detected within shorter time spans (small windows) rather than compressing the whole settling down period into a single trial (like averaging across the whole night).

First, the relation between window/stride-lengths of data windows and classification performance was examined to see if there is any specific correlation with lengths and performance metrics in a 10-fold cross validation analysis. Window length determines how long the data window will be in minutes, where stride determines the time difference between consecutive windows.

Next, the pair that resulted in the best performance was picked to do in-depth 10-fold stratified cross-validation analysis following the same method. Stratification was used to maintain the ratio between different labels—indicating whether the data came from a participant with SS or not—while also ensuring a balanced distribution of data across different subjects. Cross-validation was done to ensure the model was generalizable for our problem, and stratification was used to ensure uniform data imbalance across different folds. Within each cross-validation fold, a random-forest model with previously identified optimal parameters was trained with that fold’s training set and tested with the corresponding test set. Setting a parameter set instead of finding optimal parameters for each specific fold was chosen to have a more generalizable model. Based on predicted training labels, the decision threshold was changed from Prob >0.5, which is the default case for binary classification, to a threshold found by Youden’s J statistic, J = TPR-FPR where TPR is true positive rate and FPR is false positive rate based on training data’s ROC curve. This was done to make guesses based on the information from training data.

Finally, we conducted individual simple regression analyses to explore how clinically relevant features (namely, measures of magnitude, duration, and variance of activity) contributed to the variance in settling down length of time, a variable found to be significantly different in our previous work (Hartman et al., 2023). We calculated averages of each child’s activity magnitude, maximum activity magnitude, and settling down duration (in minutes) across all settling down periods. We also calculated median entropy and the standard deviation of the settling down duration as measures of variation. Using the average duration of settling down time as the dependent variable, we used each of the calculated variables above as predictors and controlled for child age and sex. As the sample size was small, we used this analysis to explore potential key measurement and intervention targets that may be substantially impacting settling down duration for children with SS.

Participants were similar in child age and sex, parent age, community type (urban, suburban, or rural) and family size (Table 1). The groups differed on race and ethnicity, with the SS group being slightly more diverse than the NSS group.

Using the procedure outlined in the previous section, settling down periods were measured for each group. On average, children with SS took 52.63 min (SD = 38.31 min) to settle down and fall asleep. This differed from the children with NSS, who took on average 26.19 min (SD = 14.20 min) to settle down and fall asleep (p = 0.075, Hedges’g = 0.95).

We conducted multiple simple regressions as exploratory analyses to examine how the identified features contribute to the variance noted in the average duration of settling down time between groups. Regression analyses indicated that mean maximum magnitude, median entropy, and mean magnitude independently accounted for 84%, 73%, and 57% of the variance in settling down length, respectively (Table 5). This suggests that these variables, constructed from features that successfully separate the two groups, are key contributors to the variance seen in settling down lengths between groups.

To our knowledge, this paper is the first to present application of machine learning techniques to examine the settling down period of activity for a pediatric sample. We used several techniques to strengthen the analysis. Time windows were used to compensate for inherent noise spikes in the actigraphy data and non-precise sleep diary recordings. 10-fold cross-validation was used to address concerns of bias related to random forest analysis. Despite these strengths, limitations remain. Most importantly, the small sample size and the semi-controlled nature of this study resulted in limited power to understand the significance and magnitude of the contribution each predictor had on the variance of settling down duration. Within this small sample, we did have baseline differences in racial and ethnic diversity, with the SS group being slightly more diverse than the NSS group. This could be reflective of a true difference between children with have SS and those without, however, our sample was too small to comment further. Both groups are not extremely diverse and are reflective of the typical sample of families that engage in research in the Pittsburgh area through a research registry.

Additionally, generalizing beyond the sample within this study is difficult due to lack of diversity and breadth of sampling. Future studies with larger, more generalizable samples are needed to better understand the differences during the settling down period for children with sensory sensitivities compared to peers without sensitivities. As a secondary data analysis, this paper is limited to the data originally collected. As such, we relied on caregiver-reported start of settling down to indicate where to cut the actigraphy data. Future research in this area may consider using an event marker on the actigraphy device or a more precise description of what settling down means to the caregiver-child dyad.

Also, the best parameter set was found using cross-validation based on the results acquired from an unobserved portion. The 10-fold cross-validation results given in Table 3 were acquired using the same portions used in the hyperparameter tuning. This may create a slightly optimistic bias in the analyses, but this was done due to the limited dataset size. Future research in this area should include more data collection from a more diverse population, and create a more generalizable approach by separating a portion of the dataset specifically for validating the machine learning approaches’ success.

We believe that the settling down period will continue to be an important period to examine for future sleep research. We aim to provide a novel way to characterize the settling down period for future studies to test within their own, hopefully larger, samples. A future study to test this current analysis with a larger sample of children with SS and NSS would further develop a foundation upon which this line of research can grow. Additionally, we suggest that future research confirm the increased and variable activity during the settling down period for children with SS and explore potential causes. Additionally, interventions targeting the settling down period can use this machine learning approach to measure change or characterize targets for specific pediatric populations who have unique barriers to good sleep health. Using machine learning techniques may allow for the settling down period to be analyzed in depth to see if it can be predicted solely by looking at actigraphy data and its extracted features.