CS is the use of two languages within a single conversation, sentence, or linguistic unit (Poplack, 1980). For our purposes, we focus only on intra-sentential CS, which involves switching within a clause, as in (1).

Decades of research have supported the notion that CS is rule-governed. Poplack (1980), for example, showed that Spanish-English bilinguals must be highly proficient in both languages to produce acceptable intra-sentential switches. Crucially, Spanish-English bilinguals are capable of providing acceptability judgments on CS sentences, just as Spanish monolinguals do for Spanish sentences and English monolinguals for English sentences. For instance, Koronkiewicz (2014) found that Spanish-English bilinguals consistently rated sentences involving weak pronoun switches (e.g., ella ran the marathon ‘she’) significantly lower than those with strong pronoun switches (e.g., esas chicas ran the marathon ‘those girls’). This demonstrates that bilinguals who engage in CS possess clear intuitions about the acceptability of CS sentences. This is particularly relevant to our study as we employ the same grammaticality distinction as Koronkiewicz (2014), although we must recognize the grammatical diversity of CS, especially within reggaetón, that can mediate acceptability beyond strong/weak pronoun switches. Despite these now widely accepted findings, CS remains stigmatized against purist and standard ideologies, even within the bilingual community (Mata, 2023; Rangel et al., 2015; Anderson and Toribio, 2007).

Mata (2023), for instance, conducted a qualitative study of bilingual attitudes in San Diego County, which showed that while participants had positive attitudes toward bilingualism, they held negative attitudes toward CS. In a study that investigated the effects of negative attitudes on AJTs, Badiola et al. (2018) recruited 50 Spanish-English early bilinguals. All participants acquired both languages before the age of 7 and grew up in bilingual households. To gauge their attitudes toward CS, they were asked a series of language attitude questions. The authors found that participants with more negative attitudes toward CS also rated CS sentences lower. This brings to light a possible methodological consideration, where participants rate the sentences based on attitudes toward the linguistic practice, not the structure that is the experiment’s focus. Studies have indicated that negative attitudes and ideologies toward CS contribute to lower ratings on AJTs, reflecting broader societal disapproval toward mixing two linguistic systems.

The goal of an AJT, keeping in mind its generative origins, is to determine whether certain structures are considered grammatically acceptable and/or are part of a speaker’s I-language – the competence or internal/implicit grammatical rules of the speaker (Sprouse, 2013). AJTs have proven to be an invaluable tool in experimental methods, significantly contributing to the advancement of syntactic theory. Their effectiveness is well-supported by extensive evidence, including their high replicability (Sprouse and Almeida, 2017) and strong ecological validity (Hofweber et al., 2019). However, one limitation of AJTs is the decontextualization of the utterances. Language always exists in context and sociolinguistic literature on CS highlights important patterns to the when, why, and how of Spanish-English CS in bilingual communities (e.g., Zentella, 2008). The limitations of AJTs highlighted here do not pertain to their overall usefulness but rather to the tendency to overlook sociolinguistic contexts when employed. In this work, we follow usage-based approaches to language mixing (Backus, 2015). By which linguistic acceptability can be directly influenced by sociolinguistic contexts. In our approach, we take a usage based account of language to centralize the role of context, as in how frequency effects linguistic variation (Díaz-Campos and Pollock, 2023) and how language contact and language mixing are influenced by social forces (Parafita Couto et al., 2021; Rojas-Berscia, 2024). For example, Balam et al. (2020) demonstrate how the linguistic competence of a specific structure – bilingual compound verbs – is modulated by region, and Blokzijl et al. (2017) illustrate regional differences of mixed language determiner phrases across Miami and Nicaragua. An approach to language mixing that is based on context helps us to develop a multifaceted approach to bi/multilingualism, one that complements (and does not replace) the structural effects on language behavior. Following this line of thought, we must recognize that Spanish and English, as named languages (Erker, 2017), are socio-politically constructed to be separate languages (Urciuoli, 1995), despite each language being the result of centuries of language contact and language mixing. Popular understandings of language – that is, folk understandings of language from a non-linguistic perspective – also maintain that English is completely distinct from Spanish, and for this reason our analysis focuses on placing English against Spanish as we understand CS.

While the AJT methodology has its roots in generative syntax, a sociolinguistic perspective places the AJT well within the realm of perception and attitudes. Research on Latinx language perception, which specifically addresses the everyday language experiences of people in the U.S. of Latin American origin, has found that monolingual conceptualizations of Spanish and English are perceived more favorably than mixed language understandings (Rangel et al., 2015). Importantly, research on sociolinguistic perception has shown the importance of considering multiple inputs, such as a combination of audio and visual material (Callesano and Carter, 2019; Carter and Callesano, 2018; Hay and Drager, 2010; Niedzielski, 1999).

In the context of Miami, Carter and Callesano (2018) investigated the effect of national origin labels of perceptions of Spanish dialect variation. Although the authors did not find any effects of the label in their aggregate population sample, some significant label effects were found when analyzing data from only the Latinx population subsample. For example, after listening to examples of Cuban, Peninsular, and Colombian Spanish audio (and seeing ostensible family national origin background labels), Latinx participants were more likely to rate the voice as coming from a “poor” family when the background label was either “Cuba” or “Colombia,” as compared to when the background label read “Spain.” Additionally, the authors note that, regardless of the voice heard in the questionnaire, participants increased their ratings of “annual income” when they were told the speaker’s parents came from either Colombia or Spain as compared to Cuba (Carter and Callesano, 2018).

This is the methodological manipulation within perceptual dialectology that we follow in our study, but here we make note of the use of other types of labels/symbols that are perceived in addition to the speech stream, such as stuffed animals in experimental perceptions of vocalic variation (Hay and Drager, 2010) and racial phenotype in perceptions English language variation (Kutlu et al., 2022; Kurinec and Weaver, 2021). Collectively, these studies illustrate the importance of considering extra-linguistic input and context when addressing language perception. The additional context can take the form of images of faces with different skin color, such as White or Mestizo photographs (Chappell and Barnes, 2023) or photographs of “Hispanic” and “non-Hispanic” ethnic identities (Gutiérrez and Amengual, 2016). We base our AJT methodology in these sociolinguistic methods by incorporating an additional extra-linguistic stimulus, a textual label “reggaetón.”

Reggaetón music is one such context where Spanish-English CS is commonly found and associated with specific discursive and semiotic functions, particularly in relation to identity (Miranda, 2022; Martínez Vizcarrondo, 2011). The genre began to take shape in the late 1980s, coalescing as a fusion of elements from multiple genres: Jamaican dancehall, reggae en español (in Spanish) from Panama, and hip hop from the U.S. This style is characterized by its unmistakable dembow beat and often fast paced lyrics. Its development was heavily propelled by transnational migration across the Caribbean, Central America, and the U.S. (Marshall, 2009; Nwankwo, 2009; Twickel, 2009a). The song lyrics across the aforementioned genres often highlighted the lived experiences of urban Black communities, namely experiences of racism, marginalization, and criminalization; many youth identified with the lyrical content and conflicts with exclusionary national ideologies of race, across the aforementioned regions and beyond. This was particularly true for Black migrant communities in Panama, notably settled by numerous Caribbean migrant workers during the construction of the Panama Canal, who also brought their diverse musical backgrounds (Marshall, 2009). As a result of these shared transnational experiences, reggaetón shaped identity construction and resistance, especially for urban Black youth (Twickel, 2009b); with such strong international roots and an eventual presence on mainstream radio, the genre would then skyrocket in popularity by the early 2000s, leading to the eventual global recognition of artists like: El General, La Gata, Vico C, Tego Caledrón, Daddy Yankee, Ivy Queen, among many others.

Today, reggaetón’s international growth is underscored by the dominance of Puerto Rican artist, Bad Bunny, who held the title of the most streamed artist on Spotify worldwide from 2020 to 2022 (Raygoza, 2023; Spotify, 2020). This popularity demonstrates how music is ever present in the lives of social beings as both a regular practice and representation of identity, as music has an effect on social inclusion and cohesion (Welch et al., 2020). Applied in the context of reggaetón, there are linguistic elements utilized in musical performance which are representative of at least some speakers of Spanish and English, although not necessarily all who identify with and listen to reggaetón; culturally, reggaetón is significant because it has become emblematic of Caribbean culture and more widely associated with racial ideologies and cultural practices of diasporic Latinidad in the U.S. (Rivera-Rideau, 2015).

Despite reggaetón’s popularity, the genre carries a significant social weight stemming from its racialized history and is often criticized for its lyrical content, which often hypersexualizes women, glorifies partying and malianteo (“street activities” most often associated with racialized communities of lower socioeconomic status), and overall contrast with many conservative values (Rivera-Rideau, 2015). Despite these criticisms, the global popularity of reggaetón is undeniable and it can be argued that while not all Latinx bilinguals in the U.S. enjoy or listen to the genre, it is certainly recognized by all. In the context of CS and Latin music, previous research has focused on typifying CS strategies both syntactically and socio-pragmatically in identity construction (Balam and Shelton, 2023; Miranda, 2022; Loureiro-Rodríguez, 2017). Given the previously attested relationship between socio-pragmatics, identity, and CS, further investigation is necessary to determine the relationship between pragmatic motivations of CS and CS acceptability. As such, our current aim is to contextualize an AJT within the context of reggaetón. In the following sections we present our research questions, describe our materials and methodology, outline our analysis and results, present our discussion and conclusion of the results.

Considering that AJTs are a fruitful method to study CS acceptability, that CS does not exist in a vacuum and context is paramount, and the prevalence of CS in popular music genres such as reggaetón, we present the following research questions. As research utilizing the AJT methodology distinguishes between grammatical and ungrammatical sentences, each of the following research questions will be addressed by considering both the grammaticality of the sentences as well as the presence/absence of the reggaetón label.

A total of 137 U.S. Latinx Spanish-English bilingual participants, recruited using CloudResearch panels, took part in this study. Table 1 provides the demographics of the participants. Additional demographic information that showed no participant variation (e.g., self-reported race and ethnicity) are omitted. For the factor Family national origin, the level of missing indicates those participants who did not respond to that question. Moreover, participants were allowed to write-in their family national origin, and during the data cleaning stage, the first two authors collapsed the categories where possible in an effort to achieve groupings that were as balanced as possible, without eliminating participants or erasing their self-reported demographics. For Region, participants were asked to report their current zip code. The current analyses do not address the social variation of the participants; these data are part of a larger analysis and the demographic variables fall outside of the current goals and scope.

Following González-Vilbazo et al.’s (2013) methodological recommendations with CS research, all the test materials were written in CS so that the participants can enter into a CS mode. For example, the participants were welcomed to the study with the following sentence “Welcome to la encuesta! Por favor, responde a la siguiente pregunta before starting” (Welcome to the survey! Please, respond to the following question before starting). Prior to administering the survey to CloudResearch for assistance with data collection, we informally surveyed three Spanish-English CS consultants, two in Chicago and one in the Texas Valley, to provide feedback on the wording of our questions. In the survey, we use the term “spanglish” which we recognize as a contentious term in sociolinguistic literature (Otheguy and Stern, 2010). However, we note that our informal survey reviewers did not comment on its use and we recognize that our survey participants are already speakers of both Spanish and English.

The experiment was divided into five parts. The first part was a screener portion, in which the participants were asked “are you bilingual (English – Spanish)?” The participants had to answer either sí (yes) or no (no). The participants were not allowed to continue with the experiment if they had answered ‘no.’ The second screening question asked “What is the word con la que empieza esta pregunta?” (What is the word with which this question starts?). The participants had the following options: (i) word, (ii) empieza, (iii) what. The participants had to have chosen the third option, (iii) what, in order to continue with the experiment. Both questions were included to make sure that the participants were indeed Spanish-English bilinguals who were comfortable with the mixing of the two languages.

Following the screener was the consent portion, which was also delivered entirely in a mixture of both languages. This was followed by the CS portion of the experiment with a total of 40 sentences. The participants were given the following instructions: “En esta primera parte del survey, verás oraciones que cambian entre Spanish and English. Verás 4 questions per sentence and you will be presented with 40 sentences en spanglish. Some examples of Spanglish que estamos estudiando come from reggeatón music lyrics, mientras the others do not. You will see 4 preguntas con scales between 1 and 7. You may need to scroll left and right to see the question and the different options” (In this part of the survey, you will see sentences that change between Spanish and English. You will see 4 questions per sentence and you will be presented with 40 sentences in Spanglish. Some examples of Spanglish that we are studying come from reggaetón music lyrics, while the others do not. You will see 4 questions with scales between 1 and 7. You may need to scroll left and right to see the question and the different options).

In Figures 1, 2 we present screenshots of the Qualtrics survey with our four AJT questions. In only English, the first question reads as ‘Please rate on the following scale how good the sentences seems to you,’ the second as ‘Please indicate how likely it is that you would hear this sentence in your community,’ the third as ‘Please indicate how likely it is that you would say this sentence with other people your community,’ and the fourth as ‘Please indicate how strongly you agree that the sentence is correct.’ Figure 1 shows how the reggaetón source label was presented to participants and includes an acceptable example of CS – take a sip de cerveza cuando llegues a la party (take a sip of beer when you get to the party). Figure 2 shows the question presentation without the reggaetón label and includes an unacceptable CS – el otro día él was crying (the other day he was crying).

The CS items included in our study were determined by referencing popular reggaetón lyrics and modifications were made in two ways. First, semantic fields were adjusted to minimize vulgarity. This was done in order to best ensure believability that the non-labeled items did not come from reggaetón music lyrics. Second, half of the items were modified to be unacceptable; as all examples from reggaetón music are considered acceptable as they are productively used within reggaetón, we needed to manually modify the sentences to be unacceptable from a syntactic perspective (Appendix). We followed Koronkiewicz (2014) and changed strong (full nominal) subjects to weak (pronominal) subjects. Just as in Koronkiewicz (2014), the pronoun switch was the only structural manipulation that we implemented in the study. In Figure 2, the switch from Spanish to English occurs immediately after the weak subject (él ‘he’). In his research, Koronkiewicz (2014) found that CS with weak subject pronouns were regularly determined to be unacceptable, with an mean AJT score of 2.80 (2.41), which used a similar seven-point scale. All 40 sentences were randomized in our survey.

Following the CS AJT portion of the survey, participants completed a bilingual CS version of the Bilingual Code-Switching Profile [BCSP] (Olson, 2024), which provided information on each participant’s code-switching history, use, proficiency, and attitudes. Finally, the fifth section of the survey was a series of demographic questions, which included specific population samples. Crucially, participants provided their residential zip code, as we asked CloudResearch to provide a balanced sample across five states, namely New York (NY), Florida (FL), Illinois (IL), Texas (TX), and California (CA). Originally, our goal was to recruit bilingual, Latinx participants from five major Latinx cities – New York City, Miami, Chicago, El Paso, and Los Angeles – however, our combined efforts with CloudResearch proved unsuccessful at the city level, and we widened the search to the state level. Crucially, keeping with the scope of the current research questions, the following analyses are limited to grammaticality of the sentences, the presence of the reggaetón label, and how frequently participants listen to reggaetón.

In this study, participants were presented with four AJT questions (see Figures 1, 2). During the data exploratory phase, a correlational analysis revealed that the four scores were strongly correlated with each other, with correlation coefficients ranging from 0.79 to 0.84. Given these high correlations, it was determined that computing a mean score from the four individual scores would better represent the overall acceptability judgment for each sentence. Furthermore, the composite score demonstrated high correlations with each of the four individual scores, with coefficients of 0.93, 0.93, 0.94, and 0.94, respectively. This strong alignment between the individual scores and the composite score indicates that the composite score is a robust measure of the participants’ judgments.² Therefore, for subsequent analyses, we utilized this composite score to represent the acceptability judgments in our investigation of CS, frequency of interacting with reggaetón, and reggaetón labels.

To address our research questions, we employed multilevel mixed-effects regression models using the lme4 package in R. Mixed-effects modeling allows us to account for inherent clustering between observations or, in technical terms, nested data structures (Linck and Cunnings, 2015). In our study, participants responded to the same sets of questions, meaning responses were nested within each participant and each item. This modeling approach accounted for individual differences and item-specific effects, providing more precise estimates of how the variables of interest impact AJT ratings. In reporting the results of our mixed-effects model, we have calculated the p-values for the fixed effects using Satterthwaite’s approximations. Additionally, the confidence intervals for the fixed effects were computed using the Wald method, ensuring robust and reliable interval estimates for the model parameters.

We specified the random effects structure through a model selection process to identify the optimal random effects structure. Using the forward approach (e.g., Field et al., 2012; Hox et al., 2018) helped avoid early non-convergence issues. Initially, we fitted a random intercept-only null model with no fixed-effects variables. We computed an intraclass correlation (ICC) to indicate the proportion of variance due to the nested data structure (i.e., variance between Level 2 variables, such as participants and items). At each step of the model selection process, a fixed effect or its associated random component (the random slope for the fixed effect) was added to the model. The goal was to determine whether the addition would result in a better model fit. Model selection was complete when an addition no longer improved model fit or when the model became too complex to converge, resulting in a singular fit.

To compare model fit, we used a likelihood ratio test (χ² test of difference in deviance) with a significance level of 0.05. The likelihood ratio test was chosen because it allows for a direct comparison of nested models to determine if the added complexity significantly improves model fit. When the test was non-significant, indicating no substantial improvement in fit, the simpler model was preferred to avoid overfitting. Conversely, when the test was significant, the more complex model was chosen to capture the additional variance explained by the added complexity. This structured approach ensured that our final models were both parsimonious and adequately reflective of the underlying data structure, allowing us to robustly answer our research questions regarding the effects of sentence grammaticality and the reggaetón label on AJT ratings.

In the mixed-effect output, we also report two indices for model fit: marginal R² and conditional R². The marginal R² represents the proportion of variance explained by the fixed effects alone, providing an indication of the explanatory power of the predictors in which we are primarily interested. The conditional R² represents the proportion of variance explained by the entire model, including both fixed and random effects. This measure gives a sense of how well the model, as a whole, fits the data, accounting for individual differences and item-specific variability. Higher values of marginal R² indicate that the fixed effects are capturing a substantial portion of the variance in the dependent variable, while higher values of conditional R² indicate that the model, including the random effects, is capturing a large portion of the overall variance.

The descriptive statistics for AJT ratings, shown in Table 2, indicate that participants gave higher ratings for grammatical sentences (M = 4.57, SD = 1.83) compared to ungrammatical sentences (M = 4.17, SD = 1.96), pointing to a general preference for grammaticality in terms of acceptability. When considering the label status, the mean acceptability ratings were only slightly higher for sentences with the reggaetón label (M = 4.39, SD = 1.88) compared to those without it (M = 4.36, SD = 1.93). However, the confidence intervals for the label status conditions overlapped considerably, suggesting that the presence or the absence of reggaetón label in sentences had a less substantial effect on acceptability compared to grammaticality. In the current analysis, the effect of the reggaetón label was not statistically significant.

To further investigate the impact of grammaticality and label status on AJT ratings, we conducted mixed-effect modeling. The final model includes random intercepts for both participants and items, as well as random slopes for grammaticality within participants, allowing for individual variability in response patterns. The fixed effects results from the mixed-effects model are presented in Table 3. The intercept estimate was 4.15 (SE = 0.16, p < 0.001), indicating the average acceptability rating across all conditions. There was a significant effect of grammaticality (β = 0.42, SE = 0.13, p = 0.002), suggesting that grammatical sentences received higher ratings than ungrammatical ones. The effect of label status was not significant (β = 0.04, SE = 0.11, p = 0.67), indicating that the presence of a reggaetón label did not substantially influence acceptability ratings. Additionally, the interaction between grammaticality and label status was not significant (β = −0.03, SE = 0.15, p = 0.82), suggesting that the effect of grammaticality on acceptability ratings was consistent regardless of the label status.

Given the significant result related to grammaticality, we divided the dataset into two subsets: grammatical sentences and ungrammatical sentences, and conducted the remaining analyses separately for each subset. This approach aligns with the analytical methods employed in many AJT studies (refer to Badiola et al., 2018) and allows for a more straightforward interpretation of the results. Additionally, simplifying the mixed-effect model by avoiding interaction terms reduces the number of variables, thereby enhancing the reliability and robustness of the results. This reduction helps prevent the potential masking of the effects of individual variables by others, ensuring clearer and more interpretable findings.

To investigate whether AJT ratings are conditioned by participants’ enjoyment and frequency of listening to reggaetón, we extended our mixed-effects model to include these variables as fixed effects, while accounting for the effect of label status. Given that the dataset is divided into grammatical and ungrammatical sentences, separate models were developed for each subset. During exploratory data analysis, we found that the mean score for frequency was 4.85 (1.72) and the mean score for enjoyment was 5.11 (1.63), indicating relatively high levels of both measures. Furthermore, descriptive results showed that frequency and enjoyment are strongly correlated with each other (r = 0.85). To avoid the issue of multicollinearity, we decided to include only frequency of listening to reggaetón in the models. This decision was based on the consideration that frequency is a more objective measure of exposure, whereas enjoyment is more subjective and may vary more inconsistently across participants.

To address the research questions, we conducted mixed-effect models for both ungrammatical and grammatical sentences, incorporating the effects of frequency of listening to reggaetón and label status. The models included random intercepts for participants and items to account for individual variability and item-specific effects. The fixed effects in the models were frequency of listening to reggaetón, label status, and their interaction. The results from the mixed-effect models for both ungrammatical and grammatical sentences, summarized in Tables 4, 5 respectively, indicate that frequency of listening to reggaetón had a significant positive effect on acceptability ratings. For ungrammatical sentences, the estimated coefficient for frequency was β = 0.32 (SE = 0.08, p < 0.001), and for grammatical sentences, it was β = 0.38 (SE = 0.07, p < 0.001). This finding suggests that participants who frequently listen to reggaetón rated both types of sentences as more acceptable. In contrast, the effect of label status and the interaction between frequency and label status were not significant in either model, indicating that the presence of a reggaetón label did not significantly influence acceptability ratings, nor did it alter the effect of frequency on acceptability ratings. Our results have indicated a baseline effect of sentence grammaticality, by which participants distinguish grammatical and ungrammatical sentences as more and less acceptable, respectively. Finally, results indicate a significant effect of frequency of interaction with reggaetón, whereby the more participants report listening to reggaetón, the higher their AJT ratings are for both grammatical and ungrammatical sentences. In no case did the textual reggaetón label condition AJT ratings. In the following section we discuss these findings in light of previous literature and future directions.