COIL experiences are a digital educational alternative that connects two or more educational institutions in a student-centered online learning environment to address, understand, or discuss a global issue of interest to the parties. Some research has labeled it a pedagogy that creates environments for developing intercultural skills and competencies using technologies connecting two classrooms in different geographical areas (Appiah-Kubi and Annan, 2020; Zapatero et al., 2022).

On the other hand, Davis et al. (2023) define COIL as an innovative pedagogical approach used worldwide, providing teachers and students with the opportunity to engage in a global learning activity where they learn from and about each other (Marcillo-Gómez and Desilus, 2016). In addition to the predominant focus on interculturality and internationalization, it is also an innovative and cost-effective pedagogical tool that offers global learning opportunities from home (Vahed and Rodriguez, 2020; Liu and Shirley, 2021). Notably, COIL experiences have different conceptions, but all are closely related to internationalization, global learning, internet use, collaboration, and interculturality.

Up to this point, it can be observed that digital education is still under construction. Although it has been massified, ongoing inquiry into its results and implications continues. Various actors are involved in creation, production, and delivery. However, universities and higher education institutions are the foremost correspondents, as they have responded to the needs of different eras throughout history. Current education must integrate interculturality, address problems beyond the classroom, pay attention to digital nomads, innovate with tangible solutions that benefit communities and society as a whole through digital means, and meet specific demands (Tajpour and Hosseini, 2021; Hajimiri et al., 2023; Mukul and Büyüközkan, 2023). collaborative online international learning (COIL) courses address several of the abovementioned elements and serve as a strategy for active digital learning that vigorously fosters competencies in students who benefit from the dynamics of these experiences (Watla-iad and Kradtap Hartwell, 2022). This approach uses digital media to foster synchronous and asynchronous online collaboration among students and teachers in different geographical contexts. It creates equitable, innovative learning environments that align with 21st-century classrooms, enhancing cultural sensitivity and improving learning experiences (Borger, 2022). The strategy provided here is inherent to the digital age but requires further research and evidence to continue optimal and effective implementation.

COIL experiences are increasingly being implemented in universities for various reasons beyond internationalization or interculturality. They were not only used during the COVID-19 pandemic, although that situation gave them momentum at the time because it was a means to address internationalization processes without leaving home; they have been utilized for over two decades. They currently emerge as a coherent educational solution to the digital environments presented.

The nature of COIL experiences renders some competencies imminent and natural, such as interculturality, which typically involves two groups of students from educational institutions in different countries (West et al., 2022; Hackett et al., 2023). Other inherent competencies include strengthening a second or foreign language, in some cases English (Pouromid, 2019; Helena and Alena, 2021). However, other less direct competencies have been identified, such as job training (Nethsinghe et al., 2023). This competency may not be perceived in all studies or experiences, as it depends on the intention of the planned COIL. However, most COIL courses are designed in an initial phase of discussion and argumentation about a specific case, situation, or problem (Ingram et al., 2021; House et al., 2022). Some studies provide evidence of a strategy employing a real challenge or problem that promotes contextualized interdisciplinary work and requires a possible solution. In such cases, student involvement is greater, and the bond is very close; therefore, the competencies and skills developed differ. The content tends to be global and proposes complex situations (Suarez and Michalska Haduch, 2020; Salmon et al., 2022). As can be observed, studies report the benefits of developing various competencies that participants positively perceive. However, validated instruments that measure students' perception of COIL experiences in the three dimensions illustrated in Figure 1 are not found (Goto and Gutierrez-Gomez, 2024; Mestre-Segarra and Ruiz-Garrido, 2022; Wimpenny and Orsini-Jones, 2020).

For the design of the instrument, a literature review was conducted to determine which would be the appropriate dimensions to include in the scale, for this purpose, different proposals of measurement instruments developed to measure COIL experiences in higher education students were reviewed. As a result of this literature review and the three dimensions to measure COIL experiences were determined, this initial proposal of the scale had three dimensions, after the decision and the definition of the constructs, we proceeded to the design and adaptation of the items that would make up the scale. For cultural sensitivity, the internal review of the items by teachers and students who have participated in COIL experiences was taken into account, in order to determine that the items were clear to the context; after this internal review, the first version was prepared to proceed to content validation by expert judgment. The Questionnaire on the perception of the student's role in the Collaborative International Online Learning Experiences (COIL) can be consulted at: https://doi.org/10.6084/m9.figshare.26210861.v1.

For the selection of the members of the expert judgment panel, a search was carried out in Scopus for authors on the subject of COIL, the inclusion criteria being that the authors had at least 3 articles published on the subject as first author or corresponding author. After the selection of the candidates, an e-mail was sent to them inviting them to participate, for which 45 invitations were sent, however, only 18 experts participated.

The sample comprised 18 experts who validated the content of the previously designed questionnaire. In total, 11 women and seven men participated, with 12 holding a doctoral degree. The experts came from institutions in more than six countries, with the highest representation from the United States, Spain, Mexico, and Colombia. Their professional experience ranged from 6 to 32 years (see Table 1).

For the validation of the content, the Delphi method was used, which is a technique to achieve consensus among a group of people who are part of a panel of experts. This method organizes the communication process of a panel of experts focused on evaluating a problem in several rounds, in this case evaluating the design of a measurement instrument (García-Valdés and Suárez-Marín, 2013; López-Gómez, 2018). The intention of the Delphi methodology is to collate individual opinions until they reach a statistically generated consensus with collective intelligence (Nasa et al., 2021). In this case, for the validation of the content, evaluation criteria were used through which the experts should give their judgment on the items that made up the measurement scale, aspects such as clarity, coherence, relevance and sufficiency were considered. A matrix was developed which defined the elements to be evaluated and the values to be placed, the values ranged from 1 (“criteria not met”) to 4 (“high level”) (see Table 2).

The expert judgment process consisted of two stages: the first involved responding to questions related to professional variables, and the second involved evaluating the items comprising the instrument, applying the established evaluation criteria. Lastly, experts provided comments and suggestions for improvement for each item (Escobar-Pérez and Cuervo-Martínez, 2008). For the analyses, the measures of central tendency and normality of the data were taken into account. With respect to the normality of the data, acceptable values of skewness and kurtosis, which were within 2 and −2 standard deviations, were taken as reference (George and Mallery, 2001). Regarding the values obtained with Aiken's V coefficient, values above 0.75 were considered as criteria for permanence (Wilcox and Serang, 2017; Aw, 2019). Items with values below this threshold were considered to need further review.

Table 3 shows the results of the measures of central tendency, normality, and Aiken's V scores for the clarity criterion for each item of the International Online Collaborative Learning Perception Scale (COIL) assessed by the expert judges. The means indicate that experts considered the items to have a moderate level of clarity in their wording, except items 6 and 9, whose means suggested unclear wording. Regarding the distribution of the data, with the exception of items 3, 11, 14, 16 and 18, the rest presented normal values considered acceptable.

Regarding content validity through expert judgment, reference values were used to decide whether items should be retained based on items with an Aiken's V above 0.75. Items that did not meet this criterion should be reassessed for necessary adjustments in their wording. The obtained values ranged between 0.50 and 0.87 points. Eleven of the 19 items comprising the scale were considered clear in their wording and could be considered for measuring the construct. However, eight (items 1, 5, 6, 8, 9, 12, 16, 19) had values below the established reference value, indicating poor clarity of wording. Therefore, these items needed to be adjusted and reevaluated by expert judgment.

Table 4 reports the measures of central tendency, normality, and Aiken's V scores for the items' coherence criterion. The scores reflect that the experts considered the items coherent for what they intended to measure. Regarding normal distribution, the items exhibited skewness and kurtosis values within acceptable ranges, except for items 2, 3, 4, 8, 10 and 11, which exceeded acceptable normality values. Concerning content validity, Aiken's V values show that of the 19 items comprising the scale, 17 attained values considered acceptable (>0.75). However, two (items 9 and 17) need to be reviewed for coherence in the construct.

Table 5 reports the measures of central tendency, normality, and Aiken's V scores for the items' relevance criterion. The scores reflect that experts considered the items relevant for the intended measurement. Regarding the distribution of the data, with the exception of items 1, 2, 3, 4, 5, 6, 7, 8, 8, 10, 11, 13, 14, 16 and 18, the rest presented values of normality considered acceptable. Concerning content validity, Aiken's V values indicated that 18 of the 19 items comprising the scale were acceptable (>0.75). However, one item (item 9) must be reviewed to assess its relevance for inclusion in the Perception of collaborative online international learning (COIL) experiences scale.

Table 6 reports Aiken's V values for the clarity, coherence, relevance, and sufficiency criteria for the three dimensions comprising the scale and the overall scale. In Dimension 1, which corresponds to the interaction among students in digital environments, it was found that the clarity of item wording and sufficiency of items should be addressed by the authors who designed the scale to attain acceptable values and ensure that these criteria are satisfactorily met. In the second dimension, referring to collaborative work in multicultural teams to achieve objectives, it was found that the clarity criterion for item wording did not meet the minimum required value; however, the coherence, relevance, and sufficiency criteria achieved values above the suggested threshold. The third dimension of the instrument, concerning reflection on differences and similarities among peers during collaboration, met all criteria assessed by the experts, meaning that the items comprising this dimension were clear, coherent, relevant, and sufficient for the intended measurement. Finally, in the overall score of the scale, Aiken's V values indicated that the clarity criterion did not attain the acceptable score, prompting a review to identify which items need to be reformulated. However, in terms of coherence, relevance, and sufficiency, the overall Aiken's V scores had acceptable values.

This study contributes to research on collaborative international online learning experiences, specifically in the field of measurement. Therefore, this study can serve as a reference for evaluating students' COIL experiences. However, the results need to be approached with caution due to the inherent limitations of this study. For example, the evaluation of expert judgment was based on the opinion of a specific group, which could limit the generalizability of the results. A larger sample of experts could provide a more robust validation of the instrument. Further rounds among the experts are also needed to achieve consensus on the dimensions assessed. In addition, although this study presents a comprehensive assessment of content validity, it is necessary to evaluate other validity evidence such as construct validity, convergent and divergent validity, assessment of measurement invariance, and latent mean analysis that could provide information about the behavior of the data in the design. measurement model.

For future research, studies should be conducted that address the measurement of collaborative international online learning experiences, focusing on understanding the COIL construct through the proposed dimensions. In addition, it is essential to develop and validate specific instruments that focus on critical aspects of COIL contexts, such as intercultural communication and conflict resolution. Finally, longitudinal research is needed to provide a deeper understanding of the evolution of students' perceptions over time and the influence of COIL on students' academic performance and satisfaction.