The link between the exposure to work demands and the possible deterioration of employee’s health is an area of interest for occupational stress research (Pereira and Elfering, 2014). Work demands have been associated to a series of health complications such as cardiovascular diseases (Karasek et al., 1981; Rosario-Hernández et al., 2014), burnout (Brotheridge and Grandey, 2002), depression (Dormann and Zapf, 1999; Blackmore et al., 2007; Magnavita and Fileni, 2014; Rosario-Hernández et al., 2014), and psychosomatic symptoms (Pisanti et al., 2003; van der Doef et al., 2012; Rosario-Hernández et al., 2013).

On the other hand, impediments to recovering from work demands can impair employee’s health (Meijman and Mulder, 1998; Schwartz et al., 2003; Kivimäki et al., 2006; Zijlstra and Sonnentag, 2006; Fritz et al., 2010). Thus, the process of recovery appears to be influenced in the way in which people can disconnect from their work demands and those thoughts related to them (Cropley et al., 2006; Rook and Zijlstra, 2006; Sonnentag and Zijlstra, 2006; Sonnentag et al., 2008). In this way, recovery from work is necessary for workers to avoid chronic stress (Safstrom and Harting, 2013) and therefore, rumination is a mechanism suggested that can compromise a successful disconnection and recovery from work (Roger and Jamieson, 1988; Cropley et al., 2006). Cropley and Zijlstra (2011) indicate that work-related rumination can be considered as a set of repetitive thoughts directed to issues that revolve around work; it does not matter, really, if people ruminate or think about work issues when not at work and in fact, many people do it because find it rewarding and stimulating. However, Cropley and Zijlstra (2011) argue that rumination becomes a problem when affects health and well-being. Thus, Cropley and Zijlstra suggest that people not always worry or think negatively about work on their off time. In fact, thinking about work is not compatible to switch off, and therefore, makes it difficult to recover from work. On the other hand, thinking and reflexing about work issues can also have beneficial effects and can be associated to positive results.

Furthermore, Cropley and Zijlstra (2011) conceptualize work-related rumination as a construct with three factors, which they call affective rumination (AR), problem-solving pondering (PSP), and detachment (Det). AR is a cognitive state characterize by the appearance intrusive, penetrating, and recurrence thoughts about work. These thoughts are negative in affective terms (Pravettoni et al., 2007), which if are not controlled, can become cognitively and emotionally intrusive thoughts when off work. Meanwhile, Cropley and Zijlstra point out that most of studies related to rumination at work have focused on its negative aspect, which imply if people continue to think about their work when off, they continue to be with the “power button on” and this prevent them to recuperate during their off time. It is very clear that this type of rumination impact negatively recovery when not at work; however, thinking about work when not on it, not necessarily have negative implications, since it may have a positive side. For example, there are studies that suggest that thinking about work when off might have a positive impact on innovation and creativity (e.g., Baas et al., 2008). For instance, the results obtained by Baas et al. suggest that people tend to have a positive humor when the task at hand was found to be pleasantly and intrinsically helpful. Similarly, PSP, according to Cropley and Zijlstra (2011), is a mode of thinking characterized by lengthy mental examination or the appraisal of a past difficulty at work in order to discover a solution. Finally, detachment is the third factor of the work-related rumination, and it can be defined as a sense of being away from the work situation (e.g., Etzion et al., 1998). Cropley and Zijlstra (2011) indicate that there are people who manage to press the “off button” and can disconnect and forget about work.

Based on this conceptualization, Cropley et al. (2012) developed the Work-Related Rumination Scale (WRRS), which has been used in occupational health psychology research and has been translated into different languages to measure rumination at work in different studies. These translations have been done by Syrek et al. (2017) into German, Firoozabadi et al. (2018a) into Persian, Sulak Akyüz and Sulak (2019) into Turkish, and in Puerto Rico by Rosario-Hernández et al. (2013) into Spanish. The confirmatory factor analysis (CFA) results obtained of these translations of the WRRS are like those obtained on research by Cropley et al. (2012) and Querstret and Cropley (2012) because they also yielded a three-factor internal structure; AR, PSP, and detachment.

A total of 4,100 protocols from five different research conducted by the authors (Rosario-Hernández et al., 2013, 2015, 2018a,b, 2019, 2020) in Puerto Rico and each selected through a non-probabilistic sample, and distributed into this five large groups: sample 1 (n = 518, 12.6%), sample 2 (n = 1046, 25.5%), sample 3 (n = 1107, 27.0%), sample 4 (n = 626, 15.3%), and sample 5 (n = 803, 19.6%). The distributional differences in the five samples in sex (χ² [5] = 13.29, p = 0.02, Cramer’s V = 0.053), level of education (Kruskall–Wallis’ H [5] = 52.56, p < 0.01, χ² = 0.01) and age (Kruskall–Wallis’ H [5] = 74.97, p < 0.01, χ² = 0.01), although they were statistically significant, the effect size was trivial, that is, η² = 0.02. Regarding the job characteristics of the position, the type of employment (χ² [5] = 28.14, p < 0.01, Cramer’s V = 0.07), type of position (χ² [5] = 19.43, p < 0.01, Cramer’s V = 0.06), type of company (χ² [10] = 17.02, p < 0.01, Cramer’s V = 0.13) and years of work in the company (Kruskall–Wallis’ H [5] = 369.14, p < 0.01, η² = 0.08) were not substantially different between the five samples.

The characteristic of the whole sample such as gender, age, among other, ARe shown in Table 2. The sample was composed of 56.6% of females and the average education level was 16.73 ± 2.04, which is equivalent to a bachelor’s degree to one year of graduated studies.

The essential strategy of the present study was to analyze different sets of samples, obtained in different study contexts; this enhanced the inspection of the stability of the results by evaluating the measurement invariance, and in a general perspective, the replicability (de Rooij and Weeda, 2020). The estimated correlations between the latent variables based on the CFA approach were consistently different between the estimates based on the ESEM approach, to a degree it produced changes in the qualitative classification of the correlations. For example, practically all the latent correlations obtained in the CFA can be classified as high, according to the suggestions of Cohen (1992; 0.10, small; 0.30, medium; 0.50, large), or to empirically based classifications (≥0.32: 75th percentile, Bosco et al., 2015; ≥0.30: large, Gignac and Szodorai, 2016; ≥0.40, Lovakov and Agadullina, 2021). However, correlational estimates with CFA may appear to be not only high, but very high. With the ESEM, the classification of the correlational magnitude did not change, but the quantitative difference was closer to the points that separate a high magnitude from a moderate one, with the consequent impression that these correlations ARe high, but not very high. According to the mathematical theory behind ESEM, attenuation is produced by the estimation mechanism underlying the cross-factor loadings, in which the variance of the correlations moves toward the cross-loadings. These cross-loadings of the WRRS items ARe realistic representations of how the items ARe associated with their dimension and the rest of the dimensions, and due to the ESEM method these could be estimated. In contrast, the CFA imposes that these cross-loadings ARe zero, and therefore unrealistically represents the internal structure of the measurements in general, and of the WRRS in particular. Because ESEM is an approach that unites the exploratory and confirmatory approaches, the results within the exploratory framework generally carry information that leads to the analysis of factorial complexity (Fleming and Merino Soto, 2005). This result has two implications: first, that the dimensions of the WRRS maintain high correlations with each other, but not so high as to suggest a global dimension with significant interpretation; and that the correlations estimated in previous studies may be overestimated. Because the incorporation of ESEM to study the internal structure estimates the cross-loadings of the items with different factors than expected, one of the quality parameters of the internal structure was the factorial complexity, operationally defined as the degree to which the cross-loadings ARe different from zero. As a quality parameter, this complexity was moderately high in the first iteration of the analysis, with the full instrument as it is usually used. This highlights the consequent problem of the interpretability of the items because some of these items add invalid variance to their dimensions, because the items can represent more than one dimension. The practical implication is that, in research or professional applications, possibly a part of the contents of each dimension also incorporates other constructs of the WRRS model, to an extent that is questionable from measurement theory, that is, that a construct requires to be essentially one-dimensional to be interpreted. In the practice of construction of measures and validation, it is usual that the factorial simplicity of the items is presumed, that is, that the items purely represent their intended factors to be measured. With this conceptualization of measurement, the CFA applied to the WRRS is perfectly justified, because the cross-loadings do not exist because they ARe specified a priori with a value of zero.

In the three iterations of the ESEM analysis, the factorial complexity decreased due to the decisions made on the complex items, that is, they were removed on a statistical and substantive basis. One of the items removed was item 6 (detachment), whose responses need to be recoded to be joined to the other responses of its factor. Together with the strong magnitude of factorial complexity, its factorial loading in its expected dimension was very low; and both problems were reproducible in all five samples. It is known that the required recoding items usually produce method variance associated with their phrasing (DiStefano and Motl, 2009; Kam, 2018), and it is a problem commonly associated with the emergence of additional but spurious factors and low factor loadings. Therefore, the removal of this item, together with the rest of the removed items, produced an increase in the degree of fit of the WRRS model. A practical implication of this result for the user is that, as a first option to obtain more valid scores, remove this item from the calculation of the Detachment factor score; A second option is to evaluate the validity of this item, to corroborate its questionable operation, and for this the user can implement some dimensionality evaluation approach (e.g., CFA, ESEM, etc.).

Within the evaluation of the internal structure, the measurement invariance was satisfactory in the three levels evaluated (configuration, metric and thresholds, and intercepts), which helps to make comparisons according to sex, and age groups, in this study, early career (21–30 old age), in prime career (31–50 old age) and past peak career (≥51 old age), and sex. However, with respect to the age groups assessed for invariance, it is unclear whether the absence of intercept invariance (i.e., scalar) could have been produced by real differences or by the imbalance in the sample size of the samples compared in each of the five subgroups. An evaluation with different age grouping mode may be necessary to explore this with more certainty. Also, other models of equivalence assessment, including effect size, will be needed.

Our strategy to investigate measurement invariance was implemented to each of the independent subsamples (n = 5), and this provided an opportunity to observe the replicability of the measurement properties of the WRRS. In this last aspect, it is highlighted that the structural properties remain similar (unless, in sex and age groups), the measured parameters remain similar, given the natural variations of the administration conditions, and the variability of the individual disposition. Given that the data cleaning was antecedent to the main analyzes, in two manifestations of probable careless/insufficient effort responses (C/IE), it is possible to think about the link between the removal of the participants with IER and the measurement invariance achieved. We also observed that the difference between the five groups in the assessment of intercept invariance (i.e., scalar invariance) was larger than the cut-off points chosen and suggested by Chen (2007). This apparent lack of scalar invariance may be influenced by the chosen criteria of Chen (2007) and not be exactly appropriate for the assessment of scalar invariance. The reason is that these criteria were developed for the comparison of two groups (in our study, there were five groups), with the estimator for normally distributed continuous variables (i.e., maximum likelihood). To conclude that the invariance was not met at this level, a corroboration of the effect size of the non-invariance may be required (Nye et al., 2019).

Regarding reliability, the coefficients α and ω, the levels obtained can be considered moderately high in a general perspective and considering the interaction between the small number of items in each subscale, the sample size and the value obtained (Ponterotto and Ruckdeschel, 2007). These levels do not indicate using the WRRS for all uses, but predominantly for group applications and where decisions on individual subjects ARe not needed, because the coefficients ARe not high (i.e., 0.85 or more), the possibility of measurement error can still be considered high (Ponterotto and Ruckdeschel, 2007). The antecedent studies with the WRRS, where the interpretations ARe oriented toward group responses, do not conflict with this indication. On the other hand, given the similarity of the coefficients α and ω, it is assumed that some difference between the factorial loadings were trivial (Hayes and Coutts, 2020), and did not have a substantial effect on the distance between one coefficient and the other. This distance is usually associated with the degree of equality of the factorial loadings of the items, a requirement known as tau-equivalence to validate the α coefficient (Green and Yang, 2009; Hayes and Coutts, 2020). An implication of this similarity is that the estimation of internal consistency can be satisfactorily done with the coefficient α, and without requiring SEM modeling or SEM modeling approaches to estimate the coefficient ω. If the conditions of application in future uses, and the data cleaning will be effective, this implication can be induced to other contexts. Finally, given that the standard error of measurement was greater than half the standard deviation of the Detachment score, it is possible that it is necessary to incorporate revision strategies of this subscale, to improve the precision of the score (Wyrwich, 2004). These strategies may require adding an item, or refining the application of the instruments, or presenting the items in an orderly manner in each content subset.

In terms of the relationship between the three factors or zero order correlations of the WRRS, they tend to be high and positive between AR and PSP and on the other hand, these two also tend to correlate negatively and somewhat medium effect size to Detachment; except in one longitudinal study in which the relationships between AR and PSP were low and fluctuated between r = 0.07 and r = 0.19 (Vahle-Hinz et al., 2017). Probably one of the main concerns regarding the WRRS is whether the AR and PSP subscales can be distinguished and thus measure different constructs. Results from this brief systematic literature review is that they appear to measure two related but different constructs. For Cropley and Zijlstra (2011), emotional ARousal is one of the fundamental contrasts between AR and PSP states. Psychophysiological ARousal is strong in the AR state, which is Detrimental to recovery, whereas the PSP state is thought to exist without psychological or physiological ARousal, making it less harmful to recovery. According to Cropley and Zijlstra, AR has a negative valence, whereas problem-solving rumination has a positive valence, especially if the process of PSP results in a solution, which is supported by research that suggests that thinking about successfully completed tasks increases positive affect, self-efficacy, and well-being (Stajkovic and Luthans, 1998; Seo et al., 2004). As a result, it’s likely that ruminating with a problem-solving emphasis can help with recovery at least is not that Detrimental to health as AR. Moreover, Weigelt et al. (2019a) tested different models including one which has the three dimensions of the WRRS proposed by Cropley et al. (2012) and two other constructs that ARe also related to thinking about work such as positive and negative work reflection and results of their CFA supported that in fact, they were five different constructs.

As a final note, in the analysis it was Detected that the dispersion of the responses (induced from the different kurtosis values, and low ICC), which suggests not only little redundancy among the responses, but also that the items ARe sensitive to individual differences in responses, and therefore the items may be interesting units of content to explore.

Regarding the limitations of the study, first, the population representativeness is not guaranteed, because the non-random selection of the samples did not corroborate the population similarity. Second, the evaluation of the measurement invariance was done by a single procedure, since different methods can produce different percentage of Type I and Type II errors, it may be required to explore the equivalence with other methods (for example, differential operation approach of items). Third, the bifactor model was not implemented, and an assessment of multidimensionality in contrast to the dimensionality of a general factor may be required (Reise, 2012; Gignac, 2016; Rodriguez et al., 2016a,b). Finally, the reliability evaluation of the stability of the scores was not implemented; to complete the evaluation of this aspect, you should study the reproducibility of the score at different points of time, using a test–retest approach.

The final version of the instrument consisted of three moderately to highly related factors, items with increased factorial simplicity, satisfactory reproducibility of the responses to the items, high reliability of internal consistency in their scores, and strong invariance between the samples.

Data ARe available upon reasonable request to the corresponding author.

The studies involving human participants were reviewed and approved by Simón Carlo, Chair of the Review Institutional Review Board, Ponce Health Sciences University, Ponce, Puerto Rico. The patients/participants provided their written informed consent to participate in this study.

ER-H, LR-M, and CM-S: conceptualization, methodology, writing–original draft preparation, and writing–review and editing. CM-S and ER-H: formal Analysis. ER-H and LR-M: investigation. ER-H: supervision and funding acquisition. All author: contributed to the ARticle and approved the submitted version.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this ARticle ARe solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this ARticle, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.