Said et al. (1) evaluated the effectiveness of an educational school-based nutrition intervention called Sahtak bi Sahnak (your health on your plate) on dietary knowledge and adherence to dietary guidelines among secondary school adolescents in Lebanon. The authors concluded that the intervention was effective. However, upon closer inspection, we identified issues with the authors' statistical analysis and description of the cluster randomized controlled trial (cRCT) process. Our main concern is that the authors ignored the clustered nature of their randomization for part of their statistical analysis, including the third model (effect on unhealthy items adherence score) in Table 2 and the entire results in Table 3 of the original paper. The authors stated, “Where the random intercept was non-significant in the multilevel analysis, a multivariate regression model was performed to test the intervention effect, adjusting for the same significant covariates.” However, this approach lacks statistical justification because a non-significant random intercept does not imply that clustering and nesting effects can be disregarded in cRCTs.

When a study aims to evaluate the effect of an intervention in which the treated unit was a group rather than an individual, the design and analysis must address the issue of clustering and nesting effects (2, 3). In Said et al. (1) study, the intervention unit was the school (a total of 16 schools, 8 assigned to the treatment group and 8 schools assigned to the control group); therefore, the statistical analysis should not be conducted at the single level such as student without accounting for clustering and nesting effects. By “clustering effects,” we mean accounting for the potential and usually positive correlation among students within the same school, which can be adjusted for by using a random effect for clusters. The authors should have also accounted for the nesting effect by adjusting the degrees of freedom or by using different approximations. Ignoring these positive correlations within clusters or the limited degrees of freedom may inflate the type I error rate (2).

The authors accounted for such clustering effects in only two places (the first and second models in Table 2) by applying multilevel hierarchical estimates on the dietary knowledge score and healthy items adherence score. But they ignored the clustering for all other estimates (the third model in Table 2 and all the estimates in Table 3). Nevertheless, in these two models, the authors considered only two-level (adolescents and school), whereas their description suggests potentially more clustering levels, such as location and grades. A single-level analysis that does not account for such clustering and nesting effects for these multilevel observations produces results which, by being based on unsound analyses, are inherently unsubstantiated (2).

We also noticed that the authors did not clearly describe the process of randomization of the 16 schools while categorizing them into private vs. public, rural vs. urban, and grade 10 vs. grade 11. For the analysis to be validated, readers need to understand whether these locations and grades were clusters and whether they were appropriately randomized (4). Given the limited description in the paper, in its current form, we are unable to differentiate between whether the intervention design was random or was stratified for convenience. In a cRCT, every clustering unit should be randomized. Moreover, Said et al. (1) did not take clustering and nesting into consideration when they conducted sampling power calculations. Owing to the various levels (e.g., school types, locations, grades), the sample size and power calculations must consider the number of clusters per condition, their sizes, and their intracorrelation (5). It is also important that we highlight that Said et al. applied inappropriate chi-square and t-test in Table 1 to check the baseline differences between control and treated students. Usually, it is not advisable to test for baseline differences between study arms (some journals forbid it), as stated in the Consolidated Standards of Reporting Trials Guidelines, because it only tests the hypothesis of whether the randomization had been performed correctly. However, in this case, there is enough doubt about the randomization that such tests seem wise. Therefore, we eschewed the inappropriate use of the chi-square and t-test in Table 1 and instead used linear mixed models for continuous variables and generalized linear mixed models for multinomial outcomes adjusting the degrees of freedom for number of schools and found no significant differences in any variable of Table 1.

We thank Said et al. (1) for sharing their raw data and offering feedback to uphold the rigor of scientific research which have allowed us to reanalyze their work and present revised results in this letter (Tables 1, 2). While our reanalysis using their methods as explained in the paper generated some of their results that are close to original estimates in terms of both size of coefficients and statistical significance level, we have also failed to closely reproduce results in other cases. It is important to note that our analysis did not overturn Said et al. (1) conclusions or result in changes of formal statistical significance, and therefore no change in the paper's conclusion about the effects of the Sahtak bi Sahnak intervention Although the results were not qualitatively different, it is essential that we present the revised analysis using appropriate methods. Moreover, the confidence intervals were wider and the coefficient estimates differed between original and revised estimates, which has important implications particularly for research syntheses like meta-analyses. Our reanalysis assumed that there were only two levels—school and adolescents—in the intervention design. If there are more than two levels, which could not be confirmed because of the unclear description of the randomization process in the paper, we can neither support nor refute whether the Sahtak bi Sahnak intervention was effective in increasing dietary knowledge and adherence to dietary guidelines. Despite the qualitatively similar results, we present the corrected results because it is an ethical and professional scientific responsibility to correct any reported errors in published papers (6, 7). The statistical code used for reproducing the results and for analyzing the results with accounting for clustering and nesting (acknowledging that we could only account for the clustering and nesting that were explicitly described in the article) are available at the following link: https://doi.org/10.5281/zenodo.8030967.