Research increasingly shows that speech and gesture form an integrated cognitive and interactional system, rather than parallel channels. Within embodied cognition, gestures ground linguistic meaning in sensorimotor experience (Johnson and Lakoff, 2002; Barsalou, 2008) with often indicating metaphorical mappings (Lakoff and Johnson, 1980). Thus gestures help activate, manipulate, and package information for speech, reflecting their role in the integrated cognitive system that underlies both thinking and speaking (Kita et al., 2017). Interactionist accounts emphasize that gesture–speech timing is socially organized through multiple semiotic resources, with gesture, talk, gaze, and other modalities functioning as coordinated parts of interaction (Goodwin, 2007; Kendon, 2004; Parisse et al., 2022). Both cognitive and interactional approaches converge in viewing fluency as an integrative outcome, realized through the smooth expression of intrapersonal (mind–body) and interpersonal (speaker–interlocutor) coordination. Fluency is thus multidimensional, encompassing speech, interaction, and gesture, a perspective that Kosmala (2024) dubs ‘inter-fluency.’

In this study, gestures include both hand movements and vocal tract actions such as the tongue and lips. Articulatory Phonology (AP; Browman and Goldstein, 1986, 1989) defines gestures as vocal-tract actions whose spatiotemporal coordination underlies phonological structure and phonetic implementation. Although AP presents the elaborate system of vocal tract gestures rather than the general body gestures, this perspective dissolves the boundary between speech and gesture, showing that linguistic segments and prosodic patterns emerge from the temporal coordination of bodily actions in an integrated communicative system (McNeill, 1992; Goldin-Meadow and Alibali, 2013). This kind of view is supported by evolutionary and experimental research, which suggests that manual gestures shape speech development and performance (Shattuck-Hufnagel and Ren, 2018; Pouw et al., 2021; Vainio, 2019; Gentilucci and Volta, 2008), with disfluencies often mirrored across modalities (Kosmala et al., 2023).

Phonology is a cognitive representation of physical actions. From this type of perspective, AP’s gestural score specifies the embodied primitives of vocal tract actions. These primitives are hierarchically built up to syllables, foot, and prosodic words (see Selkirk, 1980 et seq., for ‘Prosodic Hiearchy’). Prosodic words are the basis for phonological phrases and other larger categories, which function as a domain of various phonological rules. For example, rhythm rules were explained in Metrical Theory (Liberman and Prince, 1977; Hayes, 1995), which provides the cognitive scaffold that structures their temporal organization. The metrical grid encodes hierarchies of strong and weak beats that act as attractors for attention and timing, thereby licensing gestures at prominent positions such as stressed syllables, prosodic word boundaries, and phrasal edges. Recent models of oscillatory entrainment (Cummins and Port, 1998; Doelling et al., 2014) reinforce this interpretation by showing that prosodic rhythm reflects timing mechanisms, which align the execution of oral and manual gestures with rhythmic beats.

Although research on multimodality has grown steadily, systematic investigations linking gestures overall to phonological forms remain limited. While many gestures synchronize with pitch accents (Wagner et al., 2014), other articulators—the lips, tongue, cheeks, eyes, eyebrows, and head—appear to coordinate with different aspects of linguistic structure. Cross-linguistic studies illustrate this complexity: eyebrow raising, for instance, follows distinct temporal patterns in English and Japanese (de La Cruz-Pavía et al., 2020), and the tongue and lips help establish language-specific articulatory settings across utterances (Gick et al., 2004; Wilson et al., 2025). For EFL learners, the lack of explicit guidance on how such articulatory gestures should be timed and integrated risks reinforcing unnatural rhythm and persistent accentedness. What emerges, then, is a clear pedagogical imperative: gesture-informed teaching practices—drawing on both articulatory and manual cues—must be incorporated into pronunciation instruction, not as an optional supplement, but as an essential means of fostering naturalistic fluency and prosody.

L2-based work has employed gestures into pronunciation instruction to boost learners’ understanding of English suprasegmental traits. For prosody training, ‘beat’ gestures—cyclic up and down movements of a hand—when aligned with stressed syllables of English, has been found to help regulate speech rhythm (McCafferty, 2002), and to facilitate the students’ identification and production of syllables, word stress, and the rhythm of speech (Smotrova, 2017), since the beat gestures synchronize with prosodic peaks in English (Leonard and Cummins, 2011). Empirical studies report benefits for learners, such as reduced perceived accentedness (Gluhareva and Prieto, 2017), improved memory for pitch accents (Kushch et al., 2018), wider pitch range and durational contrast (Yamane et al., 2019), and enhanced pitch control and fluency (Cavicchio and Busà, 2023). Learner-produced beat gestures also show improvements of L2 English pronunciation, particularly among Catalan learners, where training with beat gestures yielded significantly lower accentedness than training without them (Llanes-Coromina et al., 2018; Prieto et al., 2025).

Compared to suprasegmental trainings, hand gesture benefits to segmental improvements seem to be more limited. Xi et al. (2024) found that learners using hand gestures mimicking lip aperture (wide for /æ/, narrow for /ʌ/) outperformed those mimicking tongue position or those using no gestures, suggesting that lip-focused cues are particularly effective. Hand gestures have been applied to vowel length contrasts as well (Hirata and Kelly, 2010; Hirata et al., 2014; Li et al., 2020, 2021), which we classify as suprasegmental (i.e., prosodic) feature. Within a framework of Autosegmental Phonology (e.g., Goldsmith, 1976; Hayes, 1995; Kubozono, 2017), vowel length is a property of its association to the prosodic (moraic) tier, where the length contrast is characterized in the number of morae nested by syllable unit (i.e., short vowel has one mora, while long vowel consists of two moras). This interpretation aligns with previous studies showing that manual gestures are particularly effective for suprasegmental features such as rhythm and fluency, whereas segmental accuracy is more directly supported by articulatory feedback. These findings suggest that lower-level segmental gestures, such as consonants and vowels, may benefit less from hand gestures than higher-level prosodic units. Instead, visual feedback on learners’ own oral articulatory gestures may provide a more effective pathway for improving segmental accuracy (Suemitsu et al., 2015; Antolík et al., 2019; Kocjančič et al., 2024; Yamane et al., 2025), a possibility that warrants further investigation in future research.

Although gestures have been examined at both segmental and suprasegmental levels, systematic comparisons of objective outcomes across these domains remain underexplored, highlighting the need for studies that directly evaluate their relative effectiveness. Furthermore, though some gesture-based pedagogies have been shown to benefit learners in other Asian EFL contexts (Ma and Jin, 2022; Wang et al., 2023), their specific impact on Japanese learners’ fluency development has yet to be systematically examined.

The present experiment is designed to address this gap by testing training effects at both levels of phonology, targeting Japanese learners of English. Our focus is not to capture effects at all levels claimed under the prosodic hierarchy, but to explore two domains—suprasegmental and segmental levels—as an initial step in understanding gesture–speech integration. Neurocognitive research further supports this perspective, as delta- (0.5–3 Hz) and theta-band (3–9 Hz) rhythms have been shown to align with prosodic and syllabic cycles (Giraud and Poeppel, 2012; Doelling et al., 2014), providing a biological bridge between abstract prosodic structure and gesture–speech integration. Gestures appear to pattern with this same rhythmic system. For example, beat gestures frequently precede word onsets by approximately 100 ms, effectively resetting listeners’ neural oscillations to sharpen temporal prediction and facilitate speech segmentation (Biau and Soto-Faraco, 2015; Biau et al., 2015). Together, these findings indicate that speech and gesture are not independent channels but coordinated expressions of a shared timing mechanism that underlies both perception and communication. Importantly, the present study integrates both suprasegmental and segmental targets within a single experimental design. By contrasting gesture types—hand gestures associated with suprasegmental development and mouth gestures with segmental refinement—it seeks to advance theoretical understanding of the rhythm–articulation interface while also offering pedagogical guidance for optimizing gesture-based L2 pronunciation training.

The purpose of this study is to compare the effects of two gesture-based training methods—manual gestures and articulatory gestures—on distinct linguistic features of Japanese EFL learners’ pronunciation. We also consider how the integration of these methods may provide complementary benefits for suprasegmental and segmental development.

Japanese learners, whose first language is based on a mora-timed rhythm (Port et al., 1987), tend to produce English with less durational variability in across all vowels in words, mirroring the more regular rhythm of Japanese. This produces English that sounds overly even and less natural to native listeners, often giving the impression of a slowed overall speech pace. The unnaturalness arises from the absence of vowel reduction in unstressed syllables and cliticization, processes through which the stress-timed rhythm of English facilitates phrasing and accelerates speech tempo. Thus, if gestures are carefully designed to guide learners toward temporal alignment with the prosodic peaks of English, they may come to chunk phrases, accelerate speech tempo, and thereby facilitate the development of ‘speed fluency’ (Lennon, 1990; de Jong, 2023), an area where Japanese speakers often face persistent difficulties (Tajima and Port, 2004; Kawase et al., 2024).

As for segmental skills, Japanese learners show consistently struggle with the vowel /æ/ (‘ash’; low front vowel), which is absent from their native five-vowel system /a, i, u, e, o/, and is often substituted with /a/ (‘lower-case a’; low central/back vowel) (Lambacher et al., 2005). This substitution arises because these two vowels share tongue height and show overlap in F1, although they differ in tongue backness. English /æ/ typically has F2 values around 1700–2050 Hz (Peterson and Barney, 1952; Hillenbrand et al., 1995), whereas Japanese /a/ F2 averages only 1,283 Hz for males and 1,530 Hz for females (Yazawa and Kondo, 2019). These values place Japanese /a/ much closer to English /ʌ/ (‘wedge’; mid back vowel) than to /æ/. Orthographic conventions in the Japanese kana loanword system, clearly reflecting this merger (e.g., lab / love → ラブ), seem to be a contributing factor to both perceptual and articulatory confusions. Given these challenges, vowel contrasts such as /æ/ versus /a/ emerge as ideal targets for gesture-based training interventions, on par with the importance of speed fluency training.

This study investigates how manual (hand) and articulatory (mouth) gestures can facilitate the acquisition of specific phonological features in L2 learners, advancing an embodied, multimodal approach to pronunciation instruction. The research addresses two primary questions:

We predict level-specific outcomes: learners trained with hand gestures will show greater improvement in suprasegmental fluency measured by speech rate, whereas learners trained with mouth gestures will demonstrate greater gains in segmental (vowel) accuracy measured by F2. Furthermore, we expect the timing of training to shape the trajectory of improvement: introducing hand training earlier will yield earlier fluency gains, while introducing mouth training earlier will yield earlier segmental gains.

Fifty Japanese university students from two classes of the English communication course participated in this study. Ten participants were excluded from the analysis because they failed to complete the entire experimental procedure. As a result, data from the remaining 40 participants (aged 18–19) were included in the analysis. All participants reported no history of hearing or speech impairments, were informed about the experimental guidelines, and provided written informed consent prior to participation. This study received ethical approval from the institutional review board of Hiroshima University.

Before the pretest session, participants were assigned to two distinct experimental groups according to their class affiliations. Two training methods were implemented over a four-week period. Because students were taught in intact classes determined by the institution, we assigned the two training methods in reverse order across classes to counterbalance potential order effects, ensuring that any improvements could not be attributed solely to the method presented first (Note: While assigning intact classes to different training orders helped mitigate potential order effects, we acknowledge that this quasi-experimental design does not provide the same level of control as full randomization, and we note this as a limitation of the study).

Specifically, the group that first received Hand Gesture Training (HGT) (Figure 1) followed by Mouth Gesture Training (MGT) (Figure 2) was designated as the Hand-Mouth group (HM group, n = 19), while the group that followed the reverse order was labeled as the Mouth-Hand group (MH group, n = 21) (see Table 1).

Regarding their English proficiency, all participants reported having learned English as a second language through school-based instruction and indicated no experience of long-term residence in an English-speaking country. As first-year students, none had received formal training in English pronunciation or taken courses in linguistics or phonetics. Although the HM and MH groups differed significantly in their TOEIC scores (p = 0.006), the overall proficiency of the participants was relatively low (M = 455.0 ± 105.9), approximately corresponding to CEFR levels A2–B1 and typical of Japanese first-year university students educated primarily through school-based instruction. Moreover, because the TOEIC (L&R) primarily assesses receptive skills (listening and reading) and do not fully represent overall English proficiency—particularly productive skills—we did not consider it appropriate to classify participants into high- and low-proficiency group on this basis. For practical reasons, they were instead assigned to two groups based on their institution-assigned class affiliations rather than their TOEIC scores.

In both training methods, a tongue twister titled “Betty Botter” was employed as the speech material. This tongue twister was selected because it includes the target vowels /æ/ in “batter” and /ʌ/ in “butter,” which consistently appear in the same phonetic environment, surrounded by two consonants /b/ and /t/. Furthermore, it consisted of 63 syllables, with each word containing no more than two syllables. Such a design ensured that the participants, who were EFL learners, would not be overwhelmed by the potential complexity. The complete text content of “Betty Botter” is as follows.

Betty Botter bought some butter;

To examine how different gesture modalities contribute to L2 pronunciation development, we implemented two training conditions that target distinct phonological levels. Hand Gesture Training (HGT) was designed to support suprasegmental development by aligning manual movements with rhythmic and stress patterns, thereby reinforcing learners’ awareness of timing and fluency. In contrast, Mouth Gesture Training (MGT) focused on segmental refinement by drawing learners’ attention to tongue and lip configurations that differentiate the difficult vowel contrasts /æ/ and /ʌ/. The HGT, aimed at improving the fluency of English oral reading, and the MGT, aimed at enhancing pronunciation accuracy of the target sounds, were assigned to participants in two groups (the HM group and the MH group) with a reversed training sequence to ensure counterbalancing. Both conditions used the Betty Botter passage as practice material, enabling a direct comparison of how suprasegmental versus segmental gesture-based instruction facilitates L2 pronunciation learning.

For HGT, we used ‘circular’ gestures. Circular gestures occur naturally in everyday speech to emphasize rhythm and prosody, and are particularly used in music performances such as choral music to enhance expression and the quality of the overall performance (Jansson et al., 2021; Kilpatrick, 2020). In the field of conducting, circular motions are a type of beat gesture, and form a foundational part of gestural vocabulary. These circular and rounded motions are commonly found in almost all types of beat patterns, such as a 4/4 beat pattern, and 2/4 beat pattern in conducting legato, or melodious, smooth and continuous melodic lines (Figure 3). Most notably within the Ilya Musin method, a highly respected school of conducting that emphasizes clarity and expressiveness through wrist-led movement (Musin, 1967; Ogrizovic-Ciric, 2009), the circular motions are also part of the beat gestures used in conducting single beats, compared to other common traditions of only beating up and down (Figure 4). In all circular gestures, consistent beat points were positioned at the onset of the hand’s upward motion, reflecting conductors’ metaphorical mapping of spatial rise onto musical crescendo (Meissl et al., 2022) or pitch rise (Morett et al., 2022). Drawing from this rationale and tradition, we integrated the circular motions from the Musin method into the training protocol, as they provide a controlled yet naturalistic means of coordinating physical gestures with rhythmic patterns in speech.

The details of HGT and MGT are given below.

In Week 1, learners received hand-gesture training to enhance awareness of stressed syllables, followed in Week 2 by training focused on phonological phrases (typically noun phrases and verb phrases). This progression from lower- to higher-level suprasegmental units is aligned with the principles of prosodic hierarchy.

WEEK 1 (Strokes at stressed syllable level):

WEEK 2 (Strokes at phonological phrase level):

In Week 1, learners engaged in listening and imitation tasks to build awareness of articulatory differences between two vowels. In Week 2, they progressed to lingual and lip shaping drills with real-time visual feedback, moving from awareness-raising to self-modulated practice to support changes in articulatory behavior.

WEEK 1 (Listening and imitation):

An instructor conducted listening quiz contrasting /æ/ and /ʌ/ using English Accent Coach (Thomson, 2012).

WEEK 2 (Face-mesh software training):

An instructor introduced web-based face mesh program (Shitara et al., 2023).

The pronunciation testing session was conducted in a soundproof booth. During the session, participants were seated at a table equipped with a condenser microphone (Audio Technica AT2020) for recording and a monitor for displaying prompt words. The pronunciation data captured by the microphone were transmitted to a laboratory computer via an audio interface (Focusrite Scalett Solo 2nd Gen) and recorded using Praat (Boersma and Weenink, 1992–2024) with a sampling frequency of 44,100 Hz.

In the paragraph-reading task, the tongue twister “Betty Botter,” which was used in the training session, was also employed in the pronunciation testing. During the testing, the tongue twister was displayed on the monitor, and participants were instructed to read it aloud one time. They were not instructed to use any hand gestures, allowing us to assess their performance independently of gesture use and thereby isolate the effects of the training. In the picture-naming task, three pictures for “batter” and three for “butter” were selected. These pictures were presented on the monitor once each in random order. Participants were required to name the item depicted in each picture, thereby determining whether it was “batter” or “butter.”

The entire experimental procedure consisted of two training phases and three test sessions over a four-week period. The schedule of the experiment is given in Figure 7. Before the first training phase, participants completed a pre-test, which included a paragraph-reading task and a picture-naming task, lasting approximately 20 min in total. Following the pre-test, the HM group underwent hand gesture training (HGT), while the MH group received mouth gesture training (MGT). Training sessions were conducted twice weekly in a classroom setting under instructor supervision, with each session lasting 20 min. After completing four training sessions (totalling 80 min), participants took a mid-test, which was identical to the pre-test. Participants then entered a second two-week training phase, in which the other type of training was implemented: the HM group received MGT, and the MH group underwent HGT. After completing another four training sessions, participants took a post-test, which was consistent with the pre-test and mid-test procedures.

In the reading task, 120 tokens (40 participants × 3 training phrases × 1 repetition) were collected. The total reading duration for each participant of the “Betty Botter” text was calculated, including all types of pauses and repetitions. About the types of pause, only silent pauses were observed. An examination for filled pauses yielded none, likely because the speakers had already become familiar with the texts. In the picture-naming task, a total of 720 tokens (40 participants × 2 vowel stimuli × 3 training phrases × 3 repetitions) were collected. In the words “batter” and “butter,” second formant (F2) of the vowels /æ/ and /ʌ/ was manually annotated and measured using Praat.

The speech rate was calculated by dividing the fixed total of 63 syllables in the “Betty Botter” text by each participant’s total reading duration (in seconds). The result is expressed as syllables per second (SPS).

When using Praat for F2 measurements, parameter settings were as follows: the number of formants was set to 5, and the window length was configured at 40 milliseconds. Furthermore, according to the frequency characteristics of participants’ voices, the formant ceiling value was fine-tuned within the range of 5,000–6,000 Hz to achieve optimal formant tracking.

The F2 values were measured at the midpoint of the intervals annotated for the vowels /æ/ and /ʌ/ in the words “batter” and “butter.” The onset of the intervals was defined as the first appearance of a periodic waveform following the consonant /b/, and the offset was marked as the last point of the periodic waveform prior to the consonant /t/. The raw F2 values were normalized using the Lobanov method, as implemented in NORM (Thomas and Kendall, 2007), based on each participant’s F2 values measured three times under all conditions, to reduce individual differences due to physiological structure. Subsequently, the normalized F2 values obtained from the three measurements under all conditions were averaged and utilized for statistical analysis.

For the speech rate measurements, a two-way mixed-design ANOVA was conducted to examine the effects of training phase (pre, mid, post) as one within-subjects factor, and training sequence (HM group vs. MH group) as one between-subjects factor. For the normalized F2 measurements, a three-way mixed-design ANOVA was conducted to examine the effects of vowel type (/æ/, /ʌ/) and training phase (pre, mid, post) as two within-subjects factors, and training sequence (HM group vs. MH group) as one between-subjects factor.

Both statistical analyses were conducted under the assumption of sphericity, as confirmed by Mauchly’s test (p > 0.05). Bonferroni correction was applied in post hoc pairwise comparisons to control the family-wise error rate. The significance level (α) was set to 0.05. All statistical analyses were performed using JASP (version 0.19.2).

The results point to the potential of gesture-based training as a targeted supplement to pronunciation instruction. Rather than treating pronunciation as a uniform skill, training can be designed to address suprasegmental and segmental development in complementary ways. Hand gestures may provide an accessible entry point for building fluency across proficiency levels, whereas mouth gestures may be more effective for learners who already possess the proficiency needed for fine-grained articulatory adjustments. The HM group’s improvement in vowel accuracy may have benefited from the fluency gains fostered by hand gestures, a pattern consistent with finding by Li et al. (2023), who showed that hand-based gesture training targeting suprasegmental features also led to improvements at the segmental level. Although the specific suprasegmental measures differed—intonation in Li et al. and speech rate in the present study—both sets of findings converge on the idea that suprasegmental-focused training can create favorable conditions for subsequent segmental improvement. More broadly, these findings echo evidence that suprasegmental-focused training often produces more noticeable gains in listener judgments than segmental drilling alone (Gordon and Darcy, 2019).

Sequencing hand and mouth training could further maximize their complementary benefits. At the same time, descriptive patterns observed in this study suggest that learner variability may influence training outcomes, underscoring the need for flexible instructional designs. Taken together, these findings demonstrate how gesture-based training can differentially support segmental and suprasegmental development, offering a basis for more nuanced and effective approaches to L2 pronunciation pedagogy.

While the present findings offer important insights into the developmental relationship between suprasegmental and segmental features in adult L2 speech, several limitations should be acknowledged. This study focused on two dependent variables—speech rate as a proxy for suprasegmental development and F2 values as a proxy for segmental articulation—which, although informative, cannot capture the full range of prosodic and articulatory changes involved in pronunciation learning. Future research should therefore include additional measures such as intonation, pitch range, stress placement, syllable duration, or consonant clarity to provide a more comprehensive picture of learning trajectories. Moreover, the training types (hand versus mouth gestures) were operationalized as broad modalities, yet the cognitive load, motor demands, and degree of linguistic integration likely varied across participants. Further studies could explore how differences in task complexity and attentional demands influence outcomes, helping to clarify the mechanisms that support learning. Finally, the relatively small sample size may have limited statistical power, possibly obscuring interaction effects or moderating influences. Addressing these issues will be essential for advancing our understanding of how gesture-based training supports L2 pronunciation development.