There is still some theoretical discussion regarding the nature of ICC. Several key issues surrounding the ICC still need to be addressed, primarily due to the challenges posed by technological advancements and globalization (Chen, 2017). In general terms, ICC refers to the ability to move from one’s own culture to another and to communicate effectively and appropriately with people of different languages and cultures (Huang, 2021). At least two theoretical frameworks have sought to conceptualise ICC, including Byram’s (2020) Intercultural Communicative Competence Model, which integrates attitudes (such as openness and curiosity), knowledge (of social groups and cultural practices), skills of interpreting and relating, skills of discovery and interaction, and a strong ethical dimension, which includes responsibility and engagement in social justice; and Chen’s (2014) Triangular Model of ICC.

The Triangular Model of ICC offers a more manageable approach to operationalizing the constructs of the ICC. Although this model does not measure itercultural pedagogical competencies, it provides a sound framework for understanding individuals’ social disposition toward engaging effectively in intercultural interactions. The model highlights three core components: affective, cognitive, and behavioral. Each of these elements contributes to an individual’s overall competence. These components relate to three constructs: intercultural sensitivity, intercultural awareness, and intercultural effectiveness, respectively.

Intercultural Sensitivity refers to the emotional aspects of ICC, particularly an individual’s motivation to engage in intercultural interactions. It encompasses traits such as empathy, openness, and cultural sensitivity. Research has examined intercultural sensitivity in educational contexts, particularly in language-learning environments. For instance, Fritz et al. (2001) found a strong correlation between intercultural sensitivity and successful cross-cultural adaptation among international students. In the field of English as a Foreign Language (EFL), Karabinar and Guler (2013) used the Intercultural Sensitivity Scale (Starosta and Chen, 2010) to assess pre-service English teachers in Turkey. They found that higher levels of intercultural sensitivity were associated with more positive attitudes toward teaching in multicultural classrooms. More recently, Klenner et al. (2021) examined the factor structure of the Intercultural Sensitivity Scale (ISS) in a sample of Chilean university students and found that the original five-factor model proposed by Chen and Starosta (2000) remained stable in terms of dimensions and theoretical coverage. The study highlights the importance of culturally validating measurement tools and suggests that intercultural sensitivity may be conceptualized differently across cultural settings.

Intercultural Awareness involves an individual’s knowledge and empathetic capacity to understand their own and others’ cultures (Chen and Starosta, 1996), including cultural beliefs, values, norms, and worldviews. Understanding these aspects allows individuals to interpret culturally influenced messages accurately and communicate without significant misunderstandings. This allows intercultural awareness to develop and be enriched by the subject’s perception and understanding of their own culture and of other broader cultures (Cabrera and Barreiro, 2021). The Multicultural Awareness Scale (MAS) assesses individuals’ awareness of cultural diversity, sensitivity to cultural issues, and understanding of systemic inequalities. Studies have demonstrated the importance of intercultural awareness in educational and professional settings. The original validation study conducted among Malaysian undergraduate students demonstrated strong psychometric properties, with the MAS showing high internal consistency (Cronbach’s alpha = 0.89) and construct validity confirmed through exploratory factor analysis, revealing four dimensions: self-awareness, cultural awareness, interaction awareness, and similarity awareness (Rozaimie et al., 2011). Further research by Rozaimie and Ali (2014) used the MAS to investigate the relationship between multicultural awareness and intercultural relations in Malaysia’s multiracial society. Their findings supported the scale’s predictive validity, indicating that higher multicultural awareness was associated with more positive intercultural interactions and reduced psychological discomfort in diverse settings. In another study conducted by Rozaimie et al. (2017) in a sample of students from education and administration fields, a final reliability coefficient of 0.81 was obtained for the MAS. This value indicates that the 10 items of the MAS are reliable for assessing people with diverse ethnic origins’ awareness of cultural differences. Additionally, Yakar et al. (2018) applied the MAS to a healthcare context, evaluating intercultural communication competence among nurses. Their study confirmed the scale’s reliability in a professional setting (Cronbach’s alpha = 0.87) and emphasized the importance of multicultural awareness in enhancing patient care. Collectively, these studies highlight the MAS as a reliable and valid instrument for assessing multicultural awareness across diverse cultural and professional contexts.

Intercultural Effectiveness/Adroitness focuses on the ability to translate both affective and cognitive components into appropriate and effective behaviors during intercultural interactions. It encompasses verbal and non-verbal communication skills, as well as adaptability and flexibility in responding to the cultural expectations of various interaction contexts (Chen and Starosta, 1996). This involves showing sensitivity to cultural diversity by providing specific attention effectively (Yilmaz et al., 2020). The following section elaborates on this construct, which is the core of this study.

Intercultural Effectiveness/Adroitness is a particularly relevant aspect of ICC that EFL teachers need to develop. By being interculturally effective, EFL teachers enact the most appropriate behaviors to prompt ICC in their students. These behaviors can be seen in actions such as modeling interaction strategies for intercultural dialogue, considering not only the language but also the context in which it is used (Portalla and Chen, 2010), and implementing didactic strategies and classroom dynamics to prompt real-life communication with people from different cultural backgrounds (Byram, 2020), among others. In this regard, intercultural effectiveness for a teacher involves power dynamics within the educational process, leading to the reproduction and transformation of a culture (Bernstein, 1990). EFL teachers contribute to students’ broader development as global citizens by teaching intercultural effectiveness. This helps students not only master English but also become more culturally aware and effective communicators in an interconnected world (Byram et al., 2001).

Portalla and Chen (2010) developed the Intercultural Effectiveness Scale (IES) to measure this construct. Their study evaluated the IES among 204 undergraduate students enrolled in an introductory communication course in the northeastern United States. The final version of the IES comprises six factors and 20 items, with an overall reliability as measured by the Cronbach’s alpha coefficient of 0.85. The results demonstrated that students who scored well on the IES exhibited flexible cultural behaviors, an ability to distinguish appropriate cultural actions, and effective adaptation to various intercultural situations.

The IES has since been used in educational contexts, particularly in higher education. For example, Bates and Rehal (2017) utilized the IES to evaluate the outcomes of cross-cultural exploration and skill development in short-term undergraduate travel programs. Specifically, the IES was implemented in an international service-learning program in Romania, involving 13 students and two program leaders. Participants completed the IES assessment both before and after the program. The intervention considered three critical dimensions: continuous learning, interpersonal engagement, and resilience. Key findings revealed that the intervention helped students recognize their strengths and areas for improvement prior to traveling abroad. It also highlighted areas of growth, stability, or decline in their intercultural effectiveness. Furthermore, the IES provided program leaders with insights into the cross-cultural strengths and challenges of individual students and the group as a whole. Results demonstrated that cultural immersion and time abroad significantly contributed to student learning and development. Overall, the findings suggest that the IES is an effective tool for evaluating students’ intercultural skills and overall intercultural development.

The IES has been widely validated within the Turkish context. Arslan et al. (2015) validated the IES with 352 Turkish university students, focusing on linguistic adaptation, and confirmed the original factor structure of Portalla and Chen’s (2010) model among 204 Turkish nurses. Yilmaz et al. (2020) further validated the IES among 165 undergraduate nursing students at two universities in Turkey. They adapted the original 20-item scale to a 15-item, three-factor structure through linguistic and content validation. The adapted scale demonstrated sufficient internal consistency. Collectively, these studies underscore the reliability and validity of the IES in other language-speaking samples, thereby supporting its use as a robust tool for assessing intercultural effectiveness across diverse contexts.

Research has consistently demonstrated a strong correlation between the IES and scales measuring the other two constructs of ICC: Intercultural Sensitivity (IS) and Intercultural Awareness (IA). Portalla and Chen (2010) identified a significant correlation between the Intercultural Sensitivity Scale (ISS) and the IES (r = 0.74, p < 0.01), indicating that individuals with heightened intercultural sensitivity are more proficient at recognizing appropriate behaviors in intercultural interactions. Gungor et al. (2021) explored the connection between intercultural effectiveness, intercultural awareness, and xenophobia using the IES, the Multicultural Awareness Scale (MAS) (Rozaimie et al., 2011) and the Xenophobia Scale (van der Veer et al., 2011), respectively. The study involved 257 nursing students and 341 students from vocational health service programs. Findings showed that female students scored significantly higher in intercultural effectiveness, while male students performed better in intercultural awareness. Moreover, a weak negative correlation was identified between the total mean scores of the Xenophobia Scale and the Intercultural Effectiveness Scale (r = −0.182, p < 0.001). Additionally, a weak positive correlation was observed between the total mean score of the Intercultural Awareness Scale and the mean score of the Xenophobia Scale (r = 0.113, p < 0.05). The study concludes that incorporating intercultural effectiveness and sensitivity into healthcare programs’ curricula is essential for mitigating xenophobic biases. These results raise questions about the importance of this construct in other professions that involve working with people, such as the education field. In the Chilean context, Beltrán-Véliz et al. (2024) observed statistically significant positive correlations between the IES and ISS’s interaction confidence factor among 584 pre-service teachers, supporting theoretical expectations behavioral flexibility (r = 0. 336: p < 0.01), interaction relaxation (r = 0.661: p < 0.01), interactant respect (r = 0.387: p < 0.01), message skills (r = 0.204: p < 0.01), identity maintenance (r = 0.319: p < 0.01), and interaction management (r = 0.478: p < 0.01).

Considering the importance of intercultural effectiveness in the development of ICC and its pertinence to the Chilean TEFL context, it becomes relevant to have valid instruments to measure this construct, which would allow for collecting valuable data regarding EFL teachers’ levels of intercultural effectiveness. This information will aid in decision-making for pre-service and in-service teacher education.

The data collection method was a non-probabilistic/purposive sampling approach (Kelly, 2011). A total of 393 English teachers participated, ranging in age from 22 to 69 years (mean = 35.14, SD = 8.61). 73.3% of the participants were women, 26% were men, and 0.8% identified as other (self-identified as fluid and non-binary). Regarding this variable, according to data provided by the Chilean Ministry of Education, over 70% of teachers are women. Therefore, we consider the sample to be reasonably representative of the national context in terms of gender (CPEIP, 2020). Additionally, 16.8% of participants identified as Indigenous, primarily Mapuche. It should be noted that the intercultural construct does not refer to a specific culture, but rather to the interaction between different cultures based on dialogue, mutual respect, and recognition (Pereda, 2004). In this sense, the instrument (Intercultural Effectiveness Scale) is not specifically adapted for indigenous cultures. The teachers came from various educational institutions: public (46.6%), subsidized (37.4%), and private (16%), with an average of 9.25 years of professional experience (SD = 7.37).

Data collection was conducted through a single survey consisting of three parts. Part one comprised the sociodemographic variables age, sex, ethnicity, educational institution, and years of teaching experience. In Part 2, the Spanish adaptation of the IES (Beltrán-Véliz et al., 2024) (Appendix A) was employed. This scale measures intercultural effectiveness, understood as the behavioral construct of intercultural communication. The IES is answered using an ordinal response scale (1 = strongly disagree, 5 = strongly agree) and is composed of 20 items distributed across six factors: behavioral flexibility (4 items), interaction relaxation (5 items), interactant respect (3 items), message skills (3 items), identity maintenance (3 items), and interaction management (2 items). Although item 18 in the validation study by Beltrán-Véliz et al. (2024) showed an insufficient factor loading, it was retained from the original version (Portalla and Chen, 2010) to evaluate its performance in a different sample. In this psychometric study conducted in Chile, Beltrán-Véliz et al. (2024) reported a good fit for the six-factor correlated model (MLR-χ² (df = 155) = 488.297; p < 0.01; RMSEA = 0.058 [C.I. = 0.052–0.063]; CFI = 0.923; TLI = 0.906). In part 3, as a convergent validity measure, the MAS (Rozaimie et al., 2011) was applied. This instrument consists of nine items distributed in the following three factors: implicit culture awareness (4 items, i.e., ‘I accept and respect that male-female roles in families may vary significantly among different cultures.), tacit culture awareness (2 ítems, i.e. ‘It is important to identify immediately the ethnic groups of a person we meet or communicate with’), and cultural interaction awareness (3 ítems, i.e. ‘I discourage people from using racial and ethnic slurs or insult statement or behavior.’). The use of the MAS as a measure of convergent validity is theoretically justified, as both instruments share a conceptual foundation related to intercultural competence. While the IES focuses on behavioral skills, the MAS addresses the development of awareness of cultural diversity as the basis for such interactions. The MAS scale demonstrated acceptable internal consistency, as measured by Cronbach’s alpha, with a value of 0.717 (Rozaimie et al., 2011).

A language adaptation procedure was conducted, and the original scales were translated into Spanish, with the Chilean language context taken into account. This procedure followed the ITC Guidelines for Translating and Adapting Tests (International Test Commission, 2017). To ensure linguistic equivalence, a three-stage adaptation procedure was employed. First, forward translations were conducted by two independent bilingual experts; second, the back-translation phase was conducted by two native English-speaking academics; third, to refine the instrument and eliminate potential comprehension errors, monolingual speakers performed a final review of the items. The process culminated in a consensus between researchers and translators to finalize the scale. English teachers from across Chile were invited to participate in the study through a combination of purposive and snowball sampling methods. Initially, email invitations were distributed to potential participants using the principal investigator’s university’s alumni databases. These databases included contact information for graduates from education and English language teaching programs, many of whom are currently working as in-service teachers throughout the country. To expand the participant pool, the snowball sampling technique was employed (Parker et al., 2019). Teachers who received the invitation were encouraged to share it with colleagues who met the inclusion criteria, specifically those currently working as in-service English teachers in Chile. Additionally, the invitation and survey link were posted and circulated through dedicated social media groups and pages created specifically for the study, targeting English teachers in Chile. All participants provided informed consent via an online form approved by the Ethics Committee of the sponsoring university before completing the survey.

Firstly, descriptive statistics of the scale were analyzed to estimate measures of central tendency, dispersion, and shape. In addition, univariate normality assessments were conducted for the 20 items of the scale using SPSS v.25. To evaluate the theoretical structure of the scale, confirmatory factor analyses (CFAs) were conducted using Mplus v.8.1 (Muthén and Muthén, 2017). For its implementation, the polychoric correlations matrix and the unweighted least squares with mean and variance adjustment (ULSMV) estimation method were used. Regarding Mardia’s test, significant deviations from multivariate normality were observed, both in skewness (Skewness = 79.5, χ² = 5,170, p < 0.001) and kurtosis (Kurtosis = 561.3, z = 40.4, p < 0.001). Therefore, the data do not meet the assumption of multivariate normality (Mardia, 1970). To assess model quality, several goodness-of-fit indices were used to evaluate the CFA models: ULSMV-χ², comparative fit index (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA). For CFI and TLI, values of 0.90 or higher were considered acceptable (Schumacker and Lomax, 2016). For RMSEA, values less than or equal to 0.080 were considered a reasonable fit (Browne and Cudeck, 1992). To assess convergent validity, Pearson’s r correlation coefficient was used, following the criteria established by Cohen (2013) and Morales et al. (2003). To estimate reliability, McDonald’s ω was used as the primary reliability estimator, given its appropriateness for CFA-based models, while Cronbach’s α was reported as a complementary index (Green and Yang, 2015; Trizano-Hermosilla and Alvarado, 2016). Values of 0.7 or greater were considered satisfactory indicators of reliability, according to the criteria of Nunnally and Bernstein (1994).

Table 1 shows the descriptive results of the items. The highest mean is obtained by item 15, “I always show respect for my interlocutors during interaction with people from different cultures”. In contrast, the item with the lowest mean is item 6, “I have grammatical problems when interacting with people from different cultures”. Likewise, the univariate normality of the scale items was assessed. As shown in Table 1, none of the items complies with the univariate normality assumption, as they deviate significantly from what is expected under a normal distribution.

A confirmatory factor analysis was performed to evaluate the structure of the IES (Figure 1), which was based on the scale’s theoretical proposal of six correlated factors. The results showed appropriate goodness-of-fit indices (ULSMV-χ² (gl = 155) = 460.458; p < 0.01; RMSEA = 0.077 [C.I. = 0.069–0.085]; CFI = 0.929; TLI = 0.913; SRMR = 0.054). Considering these results, the IES confirms its factorial structure of six correlated factors in a population of Chilean EFL in-service teachers.

After confirming the factorial structure of the IES, its convergent validity was examined by correlating the IES factors with the three MAS factors (Table 2). Overall, the results indicate that several IES factors are positively associated with Implicit Culture Awareness (F7). This suggests that dimensions such as Behavioral Flexibility, Interaction Management, Interactant Respect, and Identity Maintenance are related to more implicit forms of cultural awareness. Significant associations were also found with Cultural Interaction Awareness (F8), particularly in factors related to interaction and adaptation, which supports the idea that intercultural sensitivity is expressed in both explicit and implicit ways. Finally, the consistent relationships between various IES factors and Tacit Culture Awareness (F9) demonstrate that the scale also connects with deeper, less conscious components of the intercultural experience. To sum up, this pattern of correlations confirms the convergent validity of the IES, indicating that its factors align coherently with the three MAS dimensions, which reflect different levels of cultural awareness in intercultural interactions.

Based on the validity evidence, reliability was primarily evaluated using McDonald’s ω coefficients. As shown in Table 3, the scale exhibits generally acceptable reliability values according to ω, with some variability across factors. The Interaction Relaxation factor had the highest value, while the Message Skills had the lowest. These results show that the IES instrument presents an adequate value for its application in the Chilean teaching population.

It should be noted that the Interaction Management factor comprises only two items. Therefore, the reliability estimates for this factor should be interpreted with caution, as internal consistency coefficients such as Cronbach’s α and McDonald’s ω are sensitive to the number of items. To enhance transparency, the inter-item correlation between the two items was examined and showed a moderate positive association, supporting an acceptable level of consistency for a two-item factor.

The results of this study reinforce the applicability of the IES for evaluating and fostering intercultural effectiveness in EFL teachers. First, the descriptive analysis highlights the variability of the items, with the highest scores relating to respect for intercultural interlocutors and the lowest reflecting difficulties with perceived grammatical accuracy. Second, confirmatory factor analysis confirms the six-factor structure of the IES, with acceptable fit indices aligning with theoretical expectations. Third, the correlational analysis demonstrates meaningful, statistically significant relationships between the IES factors and the implicit culture awareness factor of the MAS scale, further supporting convergent validity. Finally, reliability estimates, though variable, suggest a differential utility of the subscales, with some factors showing adequate consistency and others presenting limitations that should be considered when interpreting factor-level scores.