Multilingualism is a phenomenon that is observed across the globe. It is estimated that more than half of the world’s population engages in daily communication in a second language (Eberhard et al., 2019). In what manner do different languages coexist within one mind of an individual with multilingual ability? The relationship between language, concepts, and the real world is a topic of significant interest in the field of linguistics. Does the mental representation of one’s mother tongue differ from that of a language acquired later in life? The nature of conceptual representation remains inadequately understood in first language, and even more so in second language (Vukovic and Shtyrov, 2014). One of the most significant debates in the field of cognitive science concerns the nature of the mental representation of concepts. The debate centers on the question of whether the conceptual representation is symbolic or perceptual (Kaup et al., 2024).

The classical view of the representation of concepts is that they are represented in the form of propositions (Fodor, 1975; Pylyshyn, 1985). In contemporary linguistics, language is conceptualized as a symbolic system in which the relationship between the signifier (e.g., a sound, a word or a phrase) and the signified (e.g., objects in the world) is arbitrary. The lower-level processes of perception and action play no role in the formation of cognition (Ojemann, 1991). Cognition is constructed from abstract, amodal representations, which are linked to their referents through formal rules (Newell and Simon, 1972; Collins and Loftus, 1975). From this perspective, linguistic input is converted into propositional symbols, integrated with the knowledge stored in long-term memory, in order to constitute comprehension. In this context, the term “concept” is used to refer to a category of abstract, amodal entities. However, if concepts are entirely represented symbolically, what are the fundamental differences between human language comprehension and artificial intelligence (AI)?

The view of symbolic representation has been challenged by the embodied view. This alternative framework suggests that concepts are fundamentally grounded in bodily action and perception, emphasizing the role of physical experience in shaping mental representations. Language is linked to the real world through the medium of concepts. Although linguistic symbols are arbitrary, concepts are derived from the real world. Embodied theories of cognition hold that thought is grounded in the same neural systems that govern sensation, perception and action (Barsalou, 1999; Glenberg and Kaschak, 2002). Thus, concepts are not typically processed in isolation but rather situated in background settings, events, and introspections. When an individual engages in an action or experiences an emotion in a given context, the brain generates a sensory-motor state that is congruent with the situation. As a situation is repeated, the concepts that process it become increasingly established in memory (Barsalou, 2008, 2023). Therefore, concepts are represented perceptually, rather than symbolically.

An increasing number of behavioral studies have indicated that concepts in first language are grounded in perceptual and experiential information (Stanfield and Zwaan, 2001; Pecher et al., 2009; Glenberg and Gallese, 2012; Marino et al., 2013; Fino et al., 2016). Furthermore, recent studies have indicated that the core language areas in the human neocortex, in conjunction with sensorimotor structures, form a highly interactive system that serves as the foundation for first language comprehension (Fernandino et al., 2013; Davis et al., 2015; Buccino et al., 2017; Vukovic et al., 2017; Gijssels et al., 2018). In the acquisition of their first language, individuals naturally tend to link words to corresponding objects and events within their environment. This recurrent coupling of linguistic symbols with tangible, real-world phenomena plays a pivotal role in the development of perceptually grounded concepts, with some notable exceptions, such as negated events (Vanek et al., 2024). Through this process, the human mind integrates abstract linguistic information with rich sensory experiences, thereby establishing a comprehensive cognitive framework for understanding and interacting with the world.

Four categories of embodiment have been proposed: unembodiment, secondary embodiment, weak embodiment, strong embodiment (Meteyard et al., 2012). In the unembodied view, sensorimotor information plays no role in conceptual representation (Machery, 2006; Meteyard et al., 2012). In the secondary embodiment view, concepts are amodal but linked to sensorimotor information (Patterson et al., 2007; Mahon and Caramazza, 2008). Both the unembodiment and secondary embodiment theories are compatible with the ideas of classical cognitive science, according to which cognition consists of computation over amodal and arbitrary symbols, and perceptual organs serve only as peripheral devices (Meteyard et al., 2012). The weak embodiment view holds that cognition requires some sensorimotor activation. Semantic representations are at least partly constituted by sensory-motor information (Vigliocco et al., 2004). The strong embodiment view holds that cognition cannot occur without sensorimotor activation (Tuena et al., 2023). Despite these advances, the nature of conceptual representation remains ambiguous. Specifically, whether the degree of embodiment varies across the first language and the second language of a multilingual individual is an open question that warrants further investigation. The present study investigates the differences in perceptual representations among multilingual individuals. By exploring these differences, we expect to contribute additional substantiation to the theories of weak and strong embodiment, thereby sharpening the comprehension of cognition’s embodied essence.

However, there are some other researchers who oppose the theories of embodiment. It is notable that not all studies provide evidence in support of embodied cognition. For instance, in certain studies, lesions to the motor cortex did not result in deficits in action word processing (Papeo et al., 2010; Maieron et al., 2013), indicating that activation of sensorimotor structures was not necessarily required in processing language. Moreover, the precise functional role of the sensorimotor activation remains a matter of debate. One hypothesis is that embodied mechanisms are an inseparable and functionally crucial part of language processing (Vukovic et al., 2017). An alternative hypothesis is that embodied mechanisms represent a by-product of language processing, and are therefore functionally redundant and irrelevant to efficient semantic comprehension (Mahon and Caramazza, 2008; Lotto et al., 2009).

There is still no conclusive answer to the question of the nature of conceptual representation. The conceptual representation of the multilinguals are of crucial importance for the advancement of our comprehension of language processes. Second language research is beneficial for the investigation of the developmental factors influencing the formation of embodied mechanisms. However, compared to first language (L1) research, studies on second language (L2) conceptual representations are relatively scarce, yielding somewhat mixed results. Some findings suggest that L2 concepts are perceptually represented as L1 concepts (De Grauwe et al., 2014; Dudschig et al., 2014; van Zuijlen et al., 2024). It is assumed that the perceptual representations in the first language could be equally transferred to the second language processing through the shared concepts. However, other findings show no perceptual representation in second language comprehension (Chen et al., 2020) or weaker embodiment in second language (Vukovic and Shtyrov, 2014; Foroni, 2015; Wang and Zhao, 2024), suggesting that there are quantitative or qualitative differences between L1 and L2 perceptual representations during language comprehension. Further research is required to gain a deeper understanding of the manner in which the diverse linguistic repertoires of multilinguals interact with conceptual representations and the real-world experiences of individuals.

What factors influence the display of perceptual representations in language comprehension? Under the embodiment view, memory works in the service of action and perception. Language proficiency is also an important potential factor affecting perceptual representations. Thus, this study investigates how memory and language proficiency influence the perceptual representation of not only the first language but also the second language and even the third language in multilingual individuals.

Memory plays a significant role in conceptual representation. The majority of contemporary researchers support the view that perception and cognition are inextricably linked. The perceptual system, for instance, is responsible for extracting information from the environment and transmitting it to higher cognitive functioning systems such as memory and thinking. Perceptual representation is an immediate and automatic form of representation in working memory. When relevant information in language comprehension enters long-term memory, it is represented as propositional symbols in the mind (Wang et al., 2005). However, Barsalou (1999) asserts that once a perceptual state is created, some of the perceptual information is extracted through selective attention and stored in long-term memory. Pecher et al. (2009) employed a delayed task to support the notion that native speakers exhibited perceptual representations in the long-term memory stage. Our study expects to examine whether multilinguals also possess perceptual representations in their long-term memory.

The present study used the sentence-picture verification paradigm (Pecher et al., 2009) to investigate perceptual representations in multilingual participants. This paradigm is a typical approach to investigating the activation of fine-grained perceptual features during language comprehension. There are two versions of this paradigm: an immediate version and a delayed version. In the immediate sentence-picture verification paradigm, participants listen to a sentence and then immediately see a picture, after which they decide whether the object in the picture is mentioned in the preceding sentence. The delayed sentence-picture verification paradigm comprises two phases: a sentence judgment phase and a picture recognition phase. In the sentence judgment phase, participants are presented with sentences and required to decide whether the sentences are meaningful or not. Subsequently, in the picture recognition phase, participants are instructed to look at pictures and decide whether the objects in the pictures were mentioned in the preceding sentences. The key feature of the sentence-picture verification paradigm is that the perceptual features of the objects mentioned in sentences can either match or mismatch those of the objects depicted in the pictures. To illustrate, the implied shape of an eagle in a sentence (e.g., “There was an eagle in the sky”) can match or mismatch the shape of an eagle in a picture (e.g., an eagle with wings stretched out or with wings drawn in). It is critically important to note that the target perceptual feature is task-irrelevant and only implied in the sentences.

The rationale of the paradigm is as follows: If concepts are represented perceptually, participants will engage in perceptual simulations, and demonstrate sensitivity to the implied features. In other words, they will be faster to verify pictures that match the perceptual features than those that do not. Typically, the reaction time is facilitated when the implied features match the visually presented targets (i.e., the match effect, as observed by Stanfield and Zwaan, 2001). We anticipated that perceptual representations would be stronger in the immediate task associated with working memory than in the delayed task linked to long-term memory.

Language proficiency can be another important factor affecting conceptual representations. Current studies mainly contrast the first language (L1) with the second language (L2). Yet, in addition to differences in proficiency, the divergent acquisition processes that define first and second languages stand as a fundamental distinction in their characterization. In an effort to examine the effects of proficiency, we involved a high-proficiency second language and a low-proficiency third language (L3) in our study. We hypothesized that perceptual representations would more robust and readily accessible in L1 due to their strong embodiment and lifelong exposure. High-proficiency L2 can embody perceptual representations similarly to L1, given extensive practice and usage. Low-proficiency L3 may have weaker perceptual representations due to less frequent use and less embodied experience.

The aim of the present study was to investigate how different languages were represented by multilinguals. In Experiment 1, we used the immediate sentence-picture verification paradigm to examine perceptual representations in the working memory stage. We found a significant difference in languages. Specifically, we found a match effect within the first language (Cantonese), but no match effect within the second language (Mandarin) or the third language (English). Prior to the study, we hypothesized that both L1 and high proficient L2 would be perceptually represented, with L1 exhibiting stronger embodiment than L2, and low proficient L3 exhibiting weak or even no embodiment. The results yielded partial support for these predictions. The first language was perceptually represented. However, despite the second language being widely spoken as an official language with high proficiency, and the third language being an occasionally used foreign language with lower proficiency, no match effect was observed in their comprehension. This suggests that language proficiency does not have a significant effect on the exhibition of perceptual representations.

In Experiment 2, we employed the delayed sentence-picture verification paradigm to investigate perceptual representations stored in long-term memory. Our results revealed no significant difference between languages at an overall level. Nonetheless, we observed a match effect within the first language (Mandarin), a finding that was absent in the second language (English). These findings suggest that a match effect was consistently present in L1 comprehension, irrespective of whether processing occurs at the working memory stage or the long-term memory stage. Furthermore, this effect held true whether the L1 was Cantonese or Mandarin. The results in our study supports the notion that conceptual representations in the L1 are inherently and persistently perceptual. Given the recurrence of these findings in L1 comprehension, which aligns with previous research (Tomasino et al., 2007; Pecher et al., 2009; Repetto et al., 2013; Nijhof and Willems, 2015; Zwaan, 2016), it is plausible to infer that concepts in the L1 are subconsciously reactivated through perceptual simulation during language comprehension.

Our findings lend support to the notion that first language comprehension is strongly embodied. According to the strong embodiment perspective, sensorimotor activation is a prerequisite for cognitive activities (Tuena et al., 2023). That is to say, when there is cognitive activity, there is sensorimotor activation. Our results from the sentence-picture verification paradigm, where the target perceptual feature was merely implied and task-irrelevant, revealed a match effect in first language processing, even under delayed task conditions. This indicates that perceptual experience is unconsciously engaged and plays an indispensable role in cognitive tasks.

It is more probable that the representation of human language is not exclusively symbolic, but is derived from the human experience of interacting with the world and others. The acquisition of first language through prolonged and varied engagement in authentic social scenarios, lays the groundwork for conceptual development. First language acquisition and cognitive development are intertwined and cannot be completely separated. Therefore, first language implies a stronger sense of emotion and morality than a second language. For instance, individuals may find it relatively effortless to express sentiments such as “I love you” in a foreign language, yet exercise considerable caution when conveying the same message in their native tongue due to the heightened emotional intensity inherent in the latter. This phenomenon is supported by a number of empirical studies (Foroni, 2015; Sheikh and Titone, 2015; Zhao et al., 2020), which indicate that the embodiment of emotion words in first language is stronger than that of emotion words in second language (but for counterevidence see, e.g., Vanek and Tovalovich, 2022).

Moreover, our findings revealed that the match effect in L1 was notably more pronounced in the immediate task compared to the delayed task. The manifestation of these perceptual representations appears to be modulated by memory dynamics. A notable limitation of our study pertains to the absence of task-independent memory assessments. We recognize that failing to incorporate a distinct measure of memory capacity into our experimental design might have introduced variability in participant performance. We recommend that future investigations in this domain should strive to include explicit tests of memory capacity as a control variable. Such enhancements to experimental designs will facilitate a more comprehensive analysis of the data and permit a finer-grained interpretation of the findings.

The situation becomes more complicated with regard to the perceptual representation in later-learned language comprehension. In this study, no match effect was found in L2 or L3, whether of high or low proficiency, whether in working memory stage or long-term memory stage. Our findings are consistent with those of some previous studies (Vukovic and Shtyrov, 2014; Foroni, 2015; Sheikh and Titone, 2015; Chen et al., 2020), providing further support for the view that conceptual representations in L2 are different from those in L1. For instance, Foroni’s (2015) study showed when participants read affirmative sentences, the magnitude of somatic activation was smaller in L2 than in L1; unlike in L1, there was no relaxation of the relevant muscles when participants read negative sentences in L2. Foroni (2015) concluded that embodiment was only partial in L2. Also, Vukovic and Shtyrov’s (2014) study found important quantitative differences between L1 and L2 sensorimotor brain activity in language comprehension. Therefore, it is reasonable to consider that concepts in first language are represented perceptually, whereas concepts in languages learned later are represented in a different way, probably mainly represented by propositional symbols.

Indeed, there have been other previous studies that have provided support for the notion of L2 embodiment. Previous studies have primarily focused on specific word types during the working memory phase, such as emotion words (Dudschig et al., 2014), cognate words (De Grauwe et al., 2014), action words (García-Gámez and Macizo, 2018), and spatial words (Ahlberg et al., 2017). It can be posited that these specific word types may be more likely to elicit embodied responses in immediate situations. These findings lend support to the hypothesis that second and third languages, acquired later in life, exhibit weak embodiment. The weak embodiment view posits that cognitive processes may benefit from sensorimotor activation, though such activation is not considered an essential component for the execution of cognitive tasks (Meteyard et al., 2012). It is possible that L2 comprehension may result in the generation of a certain degree of perceptual representation under certain conditions. Future research could concentrate on the circumstances under which L2 comprehension leads to the generation of perceptual representations.

Language proficiency can be one of the factors influencing the manifestation of perceptual representations. In our study, no match effect was observed in highly proficient L2 or L3 with low proficiency, even in the immediate task. This observation suggests that cognitive tasks can be effectively carried out without the reliance on embodied representations, providing empirical evidence for the weak embodiment in second and third language comprehension. The proficiency level of a language, whether it is the high-proficiency L2 or low-proficiency L3, does not appear to be a decisive factor in the manifestation of perceptual representation. Instead, the mode of language acquisition, whether it is the L1 or the languages learned later (L2 and L3), seems to exert a more significant influence. This suggests that the early, immersive, and experiential learning process characteristic of L1 acquisition may lead to deeper and more integrated perceptual representations than those learned later in life under more deliberate and analytical learning conditions. However, the participants in our studies were late multilinguals. L2 and L3 were learned explicitly in a school setting. Future research could explore whether early bilinguals have perceptual representations in two parallel languages, given that both languages are learned in real-life contexts and synchronized with cognitive development. It can be hypothesized that there is an overlap of embodied information between the two different languages.

Taken together, we speculate that multilinguals’ conceptual representations should not be viewed as an all-or-nothing mechanism; instead, they operate along a continuum where the degree of sensory-motor involvement can vary widely. Conceptual representations are both perceptual and symbolic.

On the one hand, the perceptual representation is arguably the most significant feature of human language. Human language is rich in emotion, cultural background and individual experience. Metaphors, rhetoric, and other devices are employed to convey complex emotions and subtle meanings. It is reasonable to posit that human language, particularly first language, must be perceptually representational, rather than merely propositional symbolic.

On the other hand, symbolic representation is also essential to human cognition and language. The symbolic nature of concepts is crucial to the productivity and flexibility of language and thought. More specifically, L1 concepts are likely to be mainly represented perceptually, whereas L2 concepts are usually characterized by symbolic representations. The acquisition of L1 has already established a cognitive framework for L2 comprehension. In order to reduce the cognitive load and facilitate the processing of linguistic information, L2 comprehension does not require the simulation of perceptual experiences in the mind. Instead, L2 comprehension can be linked to the semantic network of the L1. In general, L2 concepts are not directly derived from sensorimotor experiences. L2 embodiment is transferred from L1 through shared concepts. Sensorimotor activation in L1 and non-L1 languages should be different, with weaker connections to the sensorimotor cortex for the non-L1 (Dudschig et al., 2014). The manifestation of perceptual representations in later-learned L2 and L3 is subject to certain constraints.

In conclusion, our study suggests that concepts in first language are mainly represented perceptually because first language and cognition develop simultaneously from interactive human life. Concepts in second and third language are mainly represented symbolically, reflecting the cognitive economy of the brain. Moreover, language acquisition style is found to be a crucial factor impacting perceptual representation, while language proficiency level may not necessarily play a significant role. The manifestation of these perceptual representations is also subject to memory processes, with stronger perceptual representations in first language in working memory relative to long-term memory.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving humans were approved by School of Psychology, South China Normal University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

DC: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. JS: Investigation, Methodology, Writing – original draft. RW: Supervision, Writing – review & editing.