Each object has its own color, be it a color such as red, green, or blue, or be it colorless (i.e., white, black, or gray). A considerable amount of scientific research on color psychology has been conducted on all aspects of color. The theoretical basis of color and psychological functioning is based on the physiological model proposed by Goldstein (1942). Goldstein discussed that color naturally causes physiological reactions that manifest in emotional experience, cognitive focus, and motor behavior.

Color carries a psychologically relevant meaning, and the colors seen influence psychological functioning (Elliot and Maier, 2012). The first study on this issue (Schauss, 1979) suggested that pink reduced physical strength and aggression. Black (Frank and Gilovich, 1988) is associated with evil, death, and other negative concepts. Red is related to happiness and facilitates non-systematic cognitive processing (Soldat et al., 1997). Therefore, color can be regarded as a type of non-content visual stimulus.

According to perceptual theory, stimuli with distinguishing features, such as color, are conducive to visual search (Treisman and Gelade, 1980; Itti and Koch, 2000). Color coding impacts learners’ cognitive processing as material-oriented interventions in multimedia learning. The salience of color can attract learners’ attention and guide it toward the relevant visual information that designers want learners to process (Hillstrom and Chai, 2006; Jamet et al., 2008). From the perspective of perceptual theory, this is a bottom-up processing method that affects the learning process and learning effect through the visual design of multimedia.

In his Cambridge Handbook of Multimedia Learning, Mayer classifies color as visual signaling (Mayer, 2005a). The signaling principle, also known as the cueing principle, points out that people learn more deeply from multimedia materials when signals are added to guide their attention to the relevant elements of the material or highlight the organization of the material (Mayer, 2005a; de Koning et al., 2009).

When color coding is added to highlight the organization of learning materials, learners can obtain more information from multimedia messages. In the process of multimedia learning, learners must integrate the information conveyed through external representations into coherent mental representations. However, there is evidence that learners often need instructional support to identify correspondence and establish relationships between them (e.g., Seufert, 2003; Seufert and Brünken, 2006). Color coding is the non-lexical information added to learning materials that can attract learners’ attention and promote the selection, organization, and integration of instructional elements (de Koning et al., 2009). Many studies have shown that color coding in multimedia materials facilitates learning (Jeung et al., 1997; Kalyuga et al., 1999; de Koning et al., 2007; Ozcelik et al., 2009; Jamet, 2014; Richter et al., 2016).

Many researchers have explained from the perspective of cognitive load theory that color cues can reduce learners’ overall cognitive load and avoid overload, resulting in improved academic performance (Sweller, 1988; Paas et al., 2003).

Subjective measures are the most common methods used to assess cognitive processing (Mutlu-Bayraktar et al., 2019). Subjective self-reports on personal feelings can provide valuable information, but validity and corroboration problems arise (Bethel et al., 2007). Participants may not answer exactly how they feel, but in the way they believe others will answer. Furthermore, color is an implicit affective cue that seems to influence psychological functioning without people’s awareness (Friedman and Förster, 2010). Thus, physiological signals help to better understand the potential responses of participants during observations.

Eye tracking data can offer valuable information about learners’ cognitive processes. As a cognitive processing assessment method, it helps reveal information about the underlying cognitive processes during multimedia learning (Just and Carpenter, 1976; Antonenko et al., 2010; van Gog and Scheiter, 2010). Many studies have incorporated eye tracking measures to investigate perceptual processes as indicators of cognitive activity (e.g., Hyönä, 2010; Jarodzka et al., 2010; Park et al., 2015a,b). For instance, pupil diameter has been shown to be a sensitive measure of mental demands. Usually, the pupil dilates to meet increased task processing demands (van der Wel and van Steenbergen, 2018; Scharinger et al., 2020). Furthermore, fixation duration is related to the continuous psychological processes associated with present information (Just and Carpenter, 1976). Total fixation time is considered to be a sign of total cognitive processing engaged with fixation information (Just and Carpenter, 1980; Rayner, 1998). Therefore, fixation duration is a very useful measure of cognitive load (Holmqvist et al., 2017).

Electroencephalography (EEG) can non-invasively assess brain activity in an authentic environment. It measures the electrical activity generated by the brain through electrodes placed on the scalp. Changes in the power of neural oscillations measured with EEG are also a credible method for examining cognitive processes in multimedia learning (Antonenko and Niederhauser, 2010). EEG signals are voltage signals generated by neural activities, and these neural activities change with cognitive processes such as learning (Lin and Kao, 2018). Moreover, EEG power reflects the capacity or performance of cortical information processing. Many prior studies have used EEG to detect, estimate, or predict human brain activities (Gevins and Smith, 2003; Galán and Beal, 2012; Yoshida et al., 2014). These measurements vary with different levels of cognitive stimulation (Berka et al., 2004), which makes it a credible choice for measuring and continuously assessing cognitive load in a learning environment (Klimesch, 1999; Anderson and Bratman, 2008).

The alpha and theta changes in rhythms reflect what is happening in a subject’s information processing, even if the subject is not aware of such changes or cannot express them in words (Basar, 1999; Klimesch et al., 2005). A number of researchers have shown that alpha and theta activities are the most relevant measure of task difficulty or cognitive load amidst various task demands (Penfield and Jasper, 1954; Gale and Christie, 1987; Sterman et al., 1994; Gevins et al., 1997). Beta waves have also been demonstrated to be related to perception and cognition (Singer, 1993; Haenschel et al., 2000; Wrobel, 2000). Increased beta waves have been correlated with anxious thinking.

Electroencephalography signals are non-stationary, dynamic, and non-linear time series. Many researchers have used non-linear parametric measures to quantify EEG data (Gribkov and Gribkova, 2000). The degree of chaos in time series data can be measured using wavelet entropy (Rosso et al., 2001), sample entropy, and approximate entropy (Richman and Randall, 2000). Entropy is an accurate measure of complexity (Pincus, 1995; Chen et al., 2009), i.e., higher entropy refers to higher complexity (Parbat and Chakraborty, 2021).

Color acts as a non-conscious prime that affects psychological function in subtle ways (Bargh, 1990). Meanwhile, EEG has a high temporal resolution and has been proven to accurately reflect subtle changes that can be distinguished and quantified in a second-by-second time frame. Therefore, EEG can provide an index of the utilization of resources for the color coding effect on learners.

In the present study, we sought to determine the color coding effect on programming learning in multimedia learning. Based on the above analysis, we hypothesized that color coding is beneficial for learning. We recorded eye tracking and EEG data simultaneously to provide further insights on assessing cognitive processing. These recordings stemmed from both the central (i.e., brain) and autonomic (i.e., pupillary response) nervous systems. We focused on measuring the EEG theta, alpha, and beta frequency band power; pupil dilation; and fixation duration – all of which are specific indications of cognitive processing needs.

One-way analysis of variance (ANOVA) and Mann-Whitney U tests were used to make comparisons across the two groups. One-way ANOVA was used on the dependent variables that met the normality assumption to conduct an ANOVA. The Mann-Whitney U test was used as a non-parametric test of equivalence on dependent variables whose skewness and kurtosis values indicated that they did not meet the assumption of normality (Fritz et al., 2012; Legesse et al., 2020). The Mann-Whitney U test is most sensitive to changes in the medium.

Effect sizes were measured by ηp2 for the one-way ANOVA, with ηp2 = 0.01 considered a small effect, 0.06 a medium effect, and 0.14 a large effect (Cohen, 1988).

Effect size were estimated by Mann-Whitney U tests, the z value can be used to calculate an effect size, such as the r proposed by Cohen (1988). Cohen’s guidelines for r are that a large effect is 0.5, a medium effect is 0.3, and a small effect is 0.1 (Coolican, 2009).

The main purpose of the present study was to investigate the impact of color coding on programming learning in multimedia learning by measuring learning performance and cognitive processing using a combination of eye tracking and EEG data. The results indicated that the participants who studied the learning material with the color-coded design had higher learning performance than those who studied the learning material with the grayscale design. Our findings are in line with those of a previous study on the color coding effect (Ozcelik et al., 2009; Stark et al., 2018). Hence, we confirmed that color coding can promote programming learning in multimedia learning.

Color-coded materials minimize unnecessary search processes used to correlate verbal information, which may help improve performance. Our results show that the fixation duration on color-coded material was shorter than that on grayscale material. This indicates that participants are able to search for associated elements between visual and verbal information more quickly in the color-coded format than in the grayscale format. Utilizing the same color to relate elements in a slide could guide learners to focus on the salient and relevant information (Navalpakkam et al., 2005). This is because salient stimuli can automatically capture attention and affect the perceptual choice of subjects when processing information, so as to shorten the information search time and release more cognitive resources for information integration and material organization (Mayer, 2005b; Schnotz, 2014).

The current results are in line with the temporal contiguity effect in multimedia learning (Mayer, 2014). Our data provide evidence that color coding helps to form an association between verbal and program code representations because the items processed timely are related to each other (Raaijmakers and Shiffrin, 1981; Kahana, 1996). Color coding facilitates the identification of corresponding visual and verbal elements, and improves learning performance through the temporal proximity of relevant information (Ginns, 2006).

When learners are able to easily ascertain relevant information and integrate verbal and visual information, they can engage more deeply in the cognitive processing required for meaningful learning (Mayer and Chandler, 2001; Mayer and Moreno, 2003; Seufert, 2003). The present study observed higher theta and alpha powers in the color-coded learning conditions than in the grayscale learning conditions. The theta band is related to the working memory. The increases in theta oscillations are related to high working memory activity as well as successful memory encoding (Miller et al., 2018) and retrieval (Solomon et al., 2019; Pi et al., 2021). Therefore, deep processing of learning material in working memory can successfully encode memory (Khader et al., 2010; Lin et al., 2017), and working memory maintenance can forecast long-term memory (Hsieh and Ranganath, 2014). The learners who successfully memorized learning materials showed higher theta power than those who did not (Khader et al., 2010). Neural oscillations in the alpha band are related to an increase in internal attention-demanding cognitive processing (Klimesch, 2012; Fink and Benedek, 2014). Our study suggests that the color-coded group had a greater engagement of working memory and attention than the grayscale group, allowing information to be processed at a deeper level and thus resulting in better learning performance (Pi et al., 2021).

Fixation durations within the same activity can also reflect cognitive processing and cognitive load (Kruger and Doherty, 2016); a longer fixation duration suggests an increased cognitive load. Furthermore, the higher the difficulty of text comprehension and the greater the memory load during learning, the greater the pupil enlargement (Kahneman and Beatty, 1966, 1967). The results of the present study showed that the participants who viewed grayscale video lectures had longer fixation durations and greater pupil size than those who viewed color-coded video lectures. This indicated that the participants under the grayscale learning condition had a higher cognitive load than those under the color-coded condition. These results are consistent with the Cognitive Load Theory (Sweller, 1988, 1994) and the Cognitive Theory of Multimedia Learning (Mayer, 2005b). In the process of multimedia learning, learners need to psychologically integrate information transmitted through external representations into a coherent psychological representation. If learners utilize their limited cognitive resources to search the text when they hear new words in the narration, the visual scanning activity will produce extraneous cognitive processing (Paas and Sweller, 2014; Wang et al., 2020). Prior research has shown that learners usually need instructional support to identify correspondences and establish connections between them (e.g., Seufert, 2003; Seufert and Brünken, 2006). As a mode of signaling, color coding could guide learners’ attention toward essential material and thereby eliminate the need to process irrelevant material. Color coding highlights the organization of the essential material to guide learners’ cognitive processing. Specifically, it reduces extraneous mental processing demands to allow people to learn more deeply, which in turn leads to improved learning performance.

Using color coding could reduce the search for reference in the text and improve split-attention effects, resulting in lower perceived task difficulty (Ayres and Sweller, 2014; Xie et al., 2016), which is a reliable measure of extraneous cognitive load (Kalyuga et al., 2000). Subjective ratings of task difficulty increase as the extraneous cognitive load increases (Paas and van Merrienboer, 1993; Paas et al., 1994). The study results indicate that the participants in the grayscale group had more extraneous cognitive load than the participants in the color-coded group. Extraneous cognitive load is caused by the inappropriate presentation of the learning material (Sweller, 2004). When the extraneous cognitive load is increased, the germane cognitive load is reduced and learning is lessened. Learners use limited working memory resources to deal with the extraneous factors imposed by the teaching process rather than the intrinsic material. The more working memory resources are needed to deal with the extraneous cognitive load, the less resources are available to deal with the intrinsic cognitive load, which in turn reduces learning (Sweller, 2010). Meanwhile, color coding is an effective multimedia design method that minimizes extraneous overload (van Gog, 2014); it helps people study more deeply in multimedia learning because the added cues emphasize the organization of the essential material. The results indicated that the index β/(θ + α) could measure the extraneous cognitive load during the learning process.

The results also provide evidence for the emotional design hypothesis (e.g., Mayer and Estrella, 2014; Plass et al., 2014). Making essential elements visually appealing (i.e., colored) directs cognitive processing (i.e., selecting) in the learning process by guiding attention, maintaining cognitive processing, and helping learners better understand the essential material (i.e., organizing and integrating) – thereby leading to better learning (Pekrun, 2006).

This study provides evidence for the benefits of using color coding in programming learning during multimedia learning. We utilized multimodal physiological measures to access process information for a detailed analysis of the programming learning process using color-coded design and grayscale design in multimedia learning materials. We likewise combined EEG frequency band power data and eye tracking data to measure mental processing. The results demonstrate that color coding guides learners in finding corresponding visual and verbal information and paying attention to critical information for meaningful learning. The participants under the color-coded condition had a lower cognitive load, more positive learning experience, and better learning performance than those under the grayscale condition.

The results of the present study suggest that using color coding to highlight the format of program code in video lectures for programming learning can improve learning performance. Learning material might be better designed to help learners select relevant information and integrate verbal and visual representations so as to enhance learning (Mayer and Fiorelle, 2014). The conclusions of this study can also be extended to other courses.

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://github.com/zeron21/Multimodal_Data.git.

The studies involving human participants were reviewed and approved by the Ethics Committee of the School of Information and Electronic Engineering, Zhejiang University of Science and Technology. The participants provided their written informed consent to participate in this study.

All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.