Recognition of knowledge concepts in brain imaging data involves scanning the brain under different learning task states to obtain brain imaging data. Then, utilizing classification techniques to identify the corresponding brain images for different task states. This is crucial for understanding human learning (Bréchet et al., 2019) and represents a new direction emerging in recent years. According to different types of brain imaging data, KCR can be implemented via different techniques, including functional magnetic resonance imaging (fMRI), structural magnetic resonance imaging (sMRI), and electroencephalography (EEG). In recent years, with the development of fMRI and sMRI (Khvostikov et al., 2018) technologies, researchers have been able to obtain detailed information about brain function and structure, leading to significant advances in the field of cognitive neuroscience. Zeithamova et al. (2019) proposed a geometric deep learning framework for cross-modal brain anatomy and functional mapping, which is important for understanding the relationship between brain structure and function, as well as studying neurological disorders. Additionally, in the identification of EEG images, Li et al. (2016) encapsulated multi-channel neurophysiological signals into grid-like frames through wavelet transform and spectrogram transform. They further designed a hybrid deep learning model, combining Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), to extract task-related features, explore inter-channel correlations, and incorporate contextual information from these frames. However, there is few studies of recognizing the KC from fMRIs in a classroom (Zhang et al., 2022c). In general, KCR can be cast as a fMRI classification problem simply (Zhang et al., 2023).

Currently, there are many pathological analyses (Wang et al., 2021), neuro-disease diagnoses (Ronicko et al., 2020), and pattern recognition (Wang et al., 2019) methods based on fMRI data. They are mainly divided into traditional machine learning-based methods and deep learning-based methods, where deep learning methods can be further categorized into voxel-based classification methods based on convolutional neural networks and graph-based representation classification methods based on functional connectivity.

Traditional machine learning methods were initially applied to the correlation analysis of brain regions and cognitive functions in fMRI data. By examining the response of each voxel in the fMRI data of subjects under different stimuli (whether the voxel is “activated,” measured by the change in neural metrics at that point), mapping the voxels to cognitive stimuli was attempted. However, this approach ignored the correlation between voxels in different locations. Multi-voxel pattern analysis (MVPA) (Weaverdyck et al., 2020) applies multivariate analysis to multiple voxels in fMRI data to improve the representation of voxel relationships. Therein, linear discriminant analysis and support vector machines are also used in the comparisons. Kuncheva et al. (2010) proposed a support vector machine model based on random subspaces and compared its performance with other machine learning classifiers for fMRI classification. Ryali et al. (2010) achieved the dual objectives of discriminating brain regions and classifying fMRI data using logistic regression combined with L1 and L2 regularization and other machine learning techniques. However, the traditional machine learning-based fMRI classification is insufficient in representation learning.

Deep learning algorithms, benefiting from the power of neural networks, show the better performance in the fMRI classification. On the one hand, voxel-based deep learning methods have achieved research results in various fields. Feng et al. (2022) proposed a method that combines Deep Feature Selection (DFS) and Graph Convolutional Networks (GCN), to classify ASD and developing control groups, significantly improving the prediction performance. Researchers then focused on the temporal nature of fMRI data, integrating sequence learning ideas such as LSTM and Markov processes into fMRI data classification. These deep learning methods have shown improvement compared to traditional machine learning methods, but generally ignored the structure information, such as the interregional correlations of the brain (Li et al., 2023). On the other hand, functional connectivity (FC) is achieved based on the voxel-wise time series of fMRI images, reflecting the functional spatio-temporal relationships between brain regions (Lurie et al., 2020). Dynamic connectivity analyses (Zarghami and Friston, 2020) are investigated the neuronal basis of metastability. Generally, two main methods are often used to calculate FC: correlation analysis (Liégeois et al., 2020) and clustering decomposition (Cribben and Yu, 2017). In the former, brain regions with strong correlations are generally considered to be functionally connected, such as Pearson product-moment correlation and Spearman's rank correlation (Lei et al., 2023). The latter clusters brain regions to be functionally connected. FC has been utilized for diagnosing Autism Spectrum Disorder (ASD) (Shao et al., 2021), Alzheimer's disease (AD) (Zuo et al., 2024b, 2023a), and cognitive impairment (Zuo et al., 2024a, 2023b).

Besides, graph-based learning methods have been developed to identify the brain status, since the brain graph can be conducted by functional connectivity matrix (Bessadok et al., 2022; Zong et al., 2024). Kim and Ye (2020) develop an approach for graph analysis based on resting-state fMRI to diagnose the spectrum disorders. To consider the sequential features, Lei et al. (2023) developed a Spatio-Temporal Graph Convolutional Network (ST-GCN) for brain representation.

However, there is few studies that are to diagnose whether a KC from a computer course has been mastered by student (Zhang et al., 2022c). Inspired by this, this study aims to learn and encode both temporal and spatial information into the graph structure for subsequent spectral graph convolution methods to learn concepts from the graph structure. With the different representation, the study is to recognize the KC from brain fMRI, exploring a novel possibility of knowledge diagnoses for education (Zhang et al., 2020).

We evaluated the used classification methods for KCR by common metrics. For this multi-label classification, we calculated the metric by considering the samples belonging to the target class as positive samples. Let TP, TN, FP, and FN be true positive, true negative, false positive, and false negative for classification result, respectively. The following metrics evaluate model performance on the test set:

where i indicates the i-th class, P_o is the observed agreement, and P_e is the expected agreement by chance. The 10-fold cross-validation is adopted to achieve the classification accuracy. The process is as follows: we randomly divided the datasets into 10 folds where 9 folds are for training classifiers and the rest fold is for computing test accuracy. Finally, we reported the average values from the 10 folds. To evaluate the data imbalance issue, we also computed the Micro-average and Macro-average computed as in the previous work (Zhang et al., 2022d). In this study, we tried to ensure that the number of samples across classes was approximately balanced, leaving the issue of data imbalance for future consideration.

This subsection evaluates the time difference based methods. To check effectiveness of each component, we employed four different models for comparisons, shown in Table 4, together with their used parameters. Note that we adjusted the model in the training process to seek their best performance in our experiments.

From Table 4, these observations reveal that both CNN+MLP and LSTM+MLP reach better performance than MLP, while (CNN+LSTM) × MLP achieves the best performance. Besides, the MLP model attained good results after 3,000 training iterations, while other models achieved comparable performance after only 500 training iterations. This implies that more powerful feature representations not only enhance the classification accuracy of the models but also accelerate their convergence. However, the less steps likely incurs higher standard deviations.

In Table 5, we computed the performance evaluation results of each model utilizing time difference in terms of precision, accuracy, recall, F1 score, and Cohen's Kappa coefficient. The results indicate that considering either spatial or temporal features enhances classification performance. Moreover, the integration of both types of features appears to yield superior results compared to the use of either feature type alone. That implies that learning concept in the brain is not only a structural activity but also a temporal activity.