This research is a scale adaptation study that involves several steps to ensure that both IREQ and MREQ are appropriate for use in the Ecuadorian context. To summarize, we first determined that conceptual equivalence is relevant, as mentioned earlier in this manuscript. Additionally, translation and content review were not necessary because we considered the work of Beranuy Fargues et al. (2009) and both questionnaires are in the Spanish language. Subsequently, to validate comprehension of the items, a pilot study was carried out using a modest sample size. Later, we conducted a larger study collecting a big sample (the sample was obtained by non-probabilistic convenience sampling) to perform a comprehensive psychometric evaluation, including exploratory factor analysis, confirmatory factor analysis, and measures of reliability and validity. Finally, we conducted the analysis taking into account the results and conclusions of this research.

This study was conducted following ethical guidelines for research in psychology. First, a research protocol was designed by Andrade and Ontaneda (2015), which helped psychology students of the “Universidad Técnica Particular de Loja”, apply the questionnaire to students in the 10th grade of general basic education, and in the first- and second-year secondary school, from public and private institutions, in 76 cities in Ecuador such as Quito, Cuenca, Loja, Guayaquil, etc. Subsequently, the participants were informed that their participation is voluntary and confidential. The directors of the educational centers, the parents and/or legal representatives and the teenagers signed the informed consent and assent form prior to the execution of the objectives of research. Lastly, data were collected in the classrooms of educational institutions during 2016,¹, using non-parametric incidental sampling techniques; thus, data collection depended on the availability of the respondents. The study was approved by the vice-director of research of the “Universidad Técnica Particular de Loja”.

We applied a socio-demographic test as an ad hoc survey designed for data collection related to the main aim of this study. It is composed by five short-answer questions which collect data about: age, gender, family type, school dropout and neighborhood.

Internet-related experiences questionnaire IREQ (Fargues et al., 2009), is a 10-item self-administered instrument that assesses internet abuse. The authors of this questionnaire found two factors: intrapersonal conflicts (items 4, 5, 6, 7, 9, 10) and interpersonal conflicts (items 1, 2, 3, 8). The response score varies between 1 (not at all) and 4 (very much); the higher the score, the higher the probability of having a problematic use of the internet.

Mobile-related experiences questionnaire MREQ (Fargues et al., 2009). This is a 10-item self-administered instrument that assesses problematic cell phone use. The questionnaire is organized into two factors (conflicts and communicational-emotional use). The response score varies between 1 (not at all) and 4 (very much). The higher the score, the higher the probability of having cell phone use.

This research was done following the ethical guidelines for research in psychology through a protocol developed by Andrade and Ontaneda (2015). In addition, this study followed the ethical principles included in the Declaration of Helsinki (de los Angeles Mazzanti Di Ruggier, 2011) in its updates, and in the current codes and guidelines such as: consent, personal data protection, confidentiality, non-discrimination, gratuity, and the possibility to drop out the study. On the other hand, the students were informed that their participation is voluntary and confidential. Furthermore, the parents and/or legal representatives signed the informed consent, and the students signed the informed assent, prior to the execution of the research objectives. Finally, the respondents were voluntary, confidential, and anonymous.

Screening data is necessary before the validation process, removing missing data, duplicate cases, and other anomalies. Afterwards, the Mahalanobis Distance was performed to identify multivariate outliers within dataset with a threshold value of p < 0.001 (Hu and Bentler, 1999; Andrade et al., 2022). Finally, Mardia's multivariate skewness and kurtosis were calculated to examine if the IREQ and MREQ items had multivariate normality (Mardia, 1970), see Tables 3, 4.² In addition, the descriptive statistics is shown in Table 2.

The Kaiser-Meyer-Olkin (KMO) is used to measure sampling adequacy which shows a measure of factorability (Kaiser and Rice, 1974); according to the guidelines the cutoff selected for determining the factorability of the dataset is KMO ≤ 0.60. Subsequently, to identify the structure of IREQ and MREQ, the exploratory factor analysis was performed to determine the number of factors.

In the scale adaptation study, exploratory factor analysis (EFA) is not mandatory, but we included it to examine the scale's structure. We evaluated the original two-factor model, but it yielded an unacceptable fit. Conversely, the one-factor model produced an acceptable fit.

In order to validate the IREQ and MREQ among Ecuadorian students, the Confirmatory Factor Analysis was performed using R package (Lavaan) (Rosseel, 2012) using maximum likelihood estimation to test structural model. Multisampling-modeling was conducted to investigate the IREQ and MREQ test measurement invariance regarding to participants gender (male and female). The goodness fit of the model was considered acceptable according to five indicators: (1) Chi-square p−value>0.5; (2) Comparative Fit Index > 0.90; (3) Tuker-Lewis fit Index (TLI) > 0.90 (Hu and Bentler, 1999); (4) Root Mean Square Error of Approximation (RMSEA) < 0.06 with its 90% confidence interval and (5) Standardized Root Mean Squared Residual (SRMSR) < 0.08 (Browne and Cudeck, 1992).

To examine the model invariance across the male and female samples, we conducted further tests on metric (factor loading) invariance scalar (measurement intercept) invariances in the model (Andrade et al., 2022). The overall invariance model was fitted with an equal factor structure, across two samples, and factor loadings and measurement intercepts to be freely estimated. Then, two restrictive invariance models were fitted to evaluate the invariance across samples. Lastly, comparisons were performed with the ratio test and the change in CFI and, to evaluate the invariance, the following criteria were considered: (i) the χ² difference between the most restrictive models (e.g., metric invariance, scalar invariance) are not significantly different from the less restrictive, baseline model (configural invariance) with α < 0.05 (optional); (ii) the change in CFI≥0.01 (mandatory), and (iii) the change in TLI≥0.02 (optional). This process was conducted for both questionnaires.

Concurrent validity was tested using correlational analysis, computing Crombach' alpha to ensure the internal consistency. In addition, we also compute the McDonald's omega due to criticism of Crombach' alpha (Malkewitz et al., 2023).

The validity of the IREQ was checked by exploratory and confirmatory factor analysis. The KMO test and Barlett's test produced satisfactory results. The residual matrix did not exceed 10% and the KMO coefficient of 0.89 shows that the matrix was of optimum suitability. To identify the underlying structure of the IREQ we utilized factor analysis using maximum likelihood and the proportion of explained variance is 0.286.

The results obtained by exploratory factor analysis show only one factor in contrast to the original solution proposed by Fargues et al. (2009), composed by two-factor model (intra-personal and inter-personal conflicts); thus, the item distribution and variance percentage differed from the original IREQ as showed in Table 3.

Confirmatory factor analysis corroborated the model obtained in the exploratory factor analysis. We tested a model with only one factor, this model showed an acceptable fit: χ²(35) = 180.946, p < 0.001; CFI = 0.990; TLI = 0.987; RMSEA = 0.045, 90% CI [0.038,0.051]; SRMR = 0.035 and the factor loadings were satisfactorily high (0.44–0.68). As presented in Figure 1, all items significantly loaded on the single factor.

In addition, the two-factor model was also tested. This model showed the following fit: χ²(34) = 298.915, p < 0.001; CFI = 0.991; TLI = 0.988; RMSEA = 0.043, 90% CI (0.039, 0.048); SRMR = 0.032 and the factor loadings were between (0.44–0.65). Also, to compare the two models we calculated the Akaike information criterion (AIC). The AIC value for bi-factor model was 103,535.6 while the AIC value for one-factor model was 51,850.79, thus, we selected the one factor model.

First, we test the validity of the MREQ. The KMO and Barlett's tests also produced satisfactory results. The residual matrix did not exceed 10% and the KMO coefficient of 0.89 shows that the matrix was of optimum suitability. In the same manner, to identify the underlying structure of the MREQ we utilized factor analysis using maximum likelihood and the proportion of explained variance is 0.31.

After performing the exploratory factor analysis the results show only one factor in contrast to the original solution developed by Fargues et al. (2009), composed by two-factor model (conflicts and communication-emotional use); thus, the item distribution and variance percentage differed from the original MREQ as showed in Table 4.

Confirmatory factor analysis was performed to test the model obtained in the exploratory factor analysis. The model with one factor model showed an acceptable fit: χ²(35) = 180.946, p < 0.001; CFI = 0.990; TLI = 0.987; RMSEA = 0.060, 90% CI (0.054, 0.066); SRMR = 0.035 and the factor loadings were satisfactorily high (0.44–0.68). As presented in Figure 2, all items significantly loaded on the single factor.

In addition, the two-factor model was also tested. This model showed the following fit: χ²(34) = 298.915, p < 0.001; CFI = 0.991; TLI = 0.988; RMSEA = 0.043, 90% CI (0.039, 0.048); SRMR = 0.032 and the factor loadings were between (0.58–0.65). Furthermore, to compare the two models we calculated the Akaike information criterion (AIC). The AIC value for one-factor model was 51,850.79 while the AIC value for bi-factor model was 90,369.5, thus, we selected the one factor model.

Tables 5, 6 shows the factor loadings of each item across whole group as well as across samples for IREQ and MREQ tests.

In order to have evidence that both tests have the same meaning across different groups, in Tables 7, 8 we performed the test of invariance for both IREQ and MREQ.

Analyzing IREQ results, the configural invariance shows a good model fit: χ²(108) = 568.859, ρ < 0.001; RMESA = 0.045; CFI = 0.985; TLI = 0.987; SRMR = 0.035. The metric invariance also had a good model fit: χ²(79) = 385.958, ρ < 0.001; RMESA = 0.043; CFI = 0.99; TLI = 0.988; SRMR = 0.036. The scaled Δχ²(29) = 182.901, ρ < 0.001, ΔCFI = 0.005; δRSMEA = 0.002. There was a little change in CFI (< 0.01) indicating that the metric invariance model was maintained across samples; meanwhile, the scalar invariance model has does not change in comparison with the configural model. Thus, our results have shown that both factor loadings and measurement intercepts were equal across male and female samples.

MREQ results show that the configural invariance indicates a good model fit: χ²(108) = 563.735, ρ < 0.001; RMSEA = 0.045; CFI = 0.986; TLI = 0.988; SRMR = 0.045. The metric invariance also had a good model fit: χ²(79) = 528.132, ρ < 0.001; RMSEA = 0.052; CFI = 0.986; TLI = 0.984; SRMR = 0.046. The scaled Δχ²(29) = 35.603, ρ < 0.001; ΔCFI = 0.00; ΔRSMEA = 0.007. There was no change in CFI indicating that the metric invariance model was maintained across samples; meanwhile, the scalar invariance model has does not change in comparison with the configural model. Thus, our results also have shown that both factor loadings and measurement intercepts were equal across male and female samples.

Models were evaluated to have significant change in goodness of fit compared between configural, metric, scalar and factor invariance when two of three criteria are met: the χ² difference was significant (ρ < 005) for both tests, the change in CFI between configural-metric and metric-scalar was equal to 0.005 while between scalar and factor invariance was equal to 0.007 for IREQ and no change was shown for MREQ between configural-metric and metric-scalar while the change between scalar and factor invariance was equal to 0.001; the change in TLI was equal to 0.001 between configural-metric and metric-scalar while the change between scalar and factor invariance was equal to 0.005 for IREQ. For MREQ the change in TLI between configural-metric and metric-scalar was equal to 0.004 while the change between scalar and factor invariance was equal to 0.001; finally, the RSMEA was equal to 0.002 (configural-metric and metric-scalar) and 0.009 (scalar and factor invariance) for IREQ and 0.007 (configural-metric and metric-scalar) and 0.002 (scalar and factor invariance) for MREQ, which is in concordance with Chen (2007) that it should be < 0.015.

According to Cronbach's alpha and McDonald's Omega shown in Tables 5, 6 the reliability for both IREQ and MREQ were high as overall as well as for male and female samples.

The correlations between and IREQ and socio-demographic variables [i.e., gender, family type, schooling status (dropout) and neighborhood] were explored to verify concurrent validity are shown in Table 9. There were no correlation between IREQ and the variables: Gender (ρ = −0.005, p < 0.001), family type (ρ = −0.0002), school dropout (ρ = 0.003, p < 0.001) and and neighborhood (ρ = 0.000)

In the same manner the correlations between MREQ and socio-demographic variables [i.e., gender, family type, schooling status (dropout) and neighborhood] were explored to verify concurrent validity. There were no correlation between MREQ and the variables: Gender (ρ = −0.05), family type (ρ = 0.000), school dropout (ρ = 0.001, p < 0.05) and neighborhood (ρ = 0.000).