We collected MRIs from 55 typically developing (TD) fetuses [GA (mean ± SD, range): 32.9 ± 3.3 weeks, 27.4–38.7 weeks; sex (n, male/female/unknown): 23/12/20] for this study from prior prospective recruitment studies or clinical fetal MRIs that were performed to screen for fetal brain abnormalities but were clinically interpreted as normal by two board-certified radiologists. To construct a confirmed parcellation dataset, we included fetuses with successful cortical surface reconstruction and over 27 gestational weeks when gyral and sulcal folding starts visibly forming. This study was reviewed and approved by the Institutional Review Board at Boston Children’s Hospital. We acquired fetal brain MRIs on a Siemens 3 T Skyra scanner using a T2-weighted Half-Fourier Acquisition Single-Shot Turbo Spin-Echo (HASTE) sequence with the following parameters: in-plane resolution of 1 mm, field of view (FOV) of 256 mm × 256 mm, time repetition of 1.6 s, time echo of 120 ms, and slice thickness of 2–4 mm. After localization of fetal brains, multiplanar HASTE stacks were acquired at least three times in different orthogonal orientations to reconstruct reliable 3D motion-corrected volume of the fetal brain.

We used our fetal MRI processing pipeline (Im et al., 2017; Yun et al., 2020, 2021) to reconstruct the cortical surfaces, which consists of the brain extraction (Hong et al., 2021), the isotropic high-resolution volume reconstruction algorithm via a slice-to-volume registration (Kuklisova-Murgasova et al., 2012), and deep learning-based cortical plate (CP) segmentation (Hong et al., 2020). For the brain extraction, we used our in-house fetal brain extraction model based on a 2D U-net structure, which had been trained with 291 MRI stacks from 65 typical developing (TD) fetuses.¹ After brain extraction, we corrected intensity inhomogeneity via N4 bias field correction (Tustison et al., 2010) and created a motion-corrected 3D volume with 0.75 mm isotropic resolution using a slice-to-volume super-resolution technique (Kuklisova-Murgasova et al., 2012). Then, we applied our automatic CP segmentation algorithm that had been developed for the fetal brain (Hong et al., 2020; https://github.com/jwhong1125/fetal_CP_segmentation). The CP segmentation algorithm is based on 2D U-net models trained separately for the sagittal, coronal, and axial planes, which had been trained with 52 TD fetuses. It includes multi-view aggregation and test-time augmentation for precise CP segmentation onto the 3D volumes.

After CP segmentation on 3D volumes, we extracted 3D inner CP surfaces using matching-cube algorithms from the CIVET,² which generates tessellated triangular meshes for the boundary between the CP and its inner region. The algorithm tessellates a surface by collapsing an outer ellipsoid mesh enclosing the inner CP volumes and resampling it to the standard mech format with 81,920 triangles and 40,962 vertices (Liu et al., 2021). Lastly, we geometrically smoothed the resampled surface with the Taubin smoothing approach (Taubin, 1995) to obtain a natural shape of the surface without shrinking and voxelated patterns. In order to get brain surfaces on the standard mesh structure, we flipped the right hemisphere to the left, resampled onto the standard mesh, and re-flipped to the right. In this manner, both hemispheres share the same vertex indices and the neighborhood definition following the standard mesh structure, which enables the usage of both hemispheres together in the same model. After the resampling of individual cortical surfaces onto the standard mesh, both left and right surface models have the same vertex indices and neighborhoods. Therefore, both hemispheres were used for the training and evaluation of the parcellation models together.

We computed three folding feature maps, mean curvature, average convexity, and adaptive distance transform-based sulcal depth (Yun et al., 2013) as inputs for surface parcellation. The mean curvature measures the cortical folding in a fine view, the average convexity measures the cortical folding in a coarse view, and our adaptive distance transform-based sulcal depth measures the cortical folding by combining both the coarse and fine views. For the parcellation label, we manually parcellated individual cortical surfaces following the Freesurfer Desikan parcellation protocol that has been extensively used as a standard in neuroimaging studies (Desikan et al., 2006). The original Desikan parcellation map consists of 34 cortical regions in each hemisphere. However, the secondary and tertiary sulci are not fully developed in the fetal brain, so it is not feasible to parcellate subdivisions of the gyrus. We simplified the original map and defined 30 cortical regions in each hemisphere (Figure 1).

In this study, we implemented an attention-gated spherical U-net by modifying the attention module and applying it to the original spherical U-net as backbone architecture (Figure 2). The spherical U-net architecture has an encoder path and a decoder path each with five up/down-sampling steps. Each path consists of repeated layers of convolution, batch normalization, and leaky rectified linear units like the original U-net. However, there are several differences from the original U-net in handling spherical structures as input and output. The general 2D convolution layers are replaced with ring convolutions. The ring convolutions are designed to perform convolution operations on the mesh structures (Zhao et al., 2021). Likewise, the up-convolution and max pooling are replaced with surface-transposed convolutions and surface mean pooling. The final layer will be a vertex-wise filter to map the feature vector to the output surface shape.

Attention mechanisms can potentially enhance the network’s ability to focus on relevant features while ignoring irrelevant ones, thereby improving the accuracy and robustness of cortical surface parcellation (Oktay et al., 2018). The coefficient of attention, αi∈01, identify salient regions and prune feature response to preserve only the activations relevant to the specific task of the network which is the parcellation in this study. The output of the attention gate is the vertex-wise multiplication of input feature maps and attention coefficients, x^i,cl=xi,cl⋅αil (Figure 2). We used the multi-dimensional attention coefficients (Oktay et al., 2018) that enable each attention gate to learn to focus on multiple target regions of interest on each level of the icosahedron. The gating vector, gi∈ℝFg, is defined as multi-dimensional attention coefficients for each vertex i on the input icosahedron, where Fg corresponds to the number of feature maps in layer l. The gating vector contains contextual information to determine regions to be focused on lower-level feature maps. We used the additive attention (Bahdanau et al., 2014) to get the gating vector for spherical feature maps on each level of the icosahedron.

The attention gate is defined as a set of parameters Θatt containing: three linear transformations with one-ring convolution Wx∈ℝFl×Fint,Wg∈ℝFg×Fint,ψ∈ℝFl×1 and bias terms bψ∈ℝ,bg∈ℝFint, which are based on the original paper (Oktay et al., 2018) but the transform and dimensions are modified for spherical data shape on icosahedrons. Instead of the 1×1×1 convolutions, channel-wise one-ring convolutions were used to linearly map those concatenated inputs, i.e., xl and g, on icosahedrons toward an intermediate space with ℝFint dimension. After that, the rectified linear unit (ReLU) was applied to the concatenated features before linear transformation with another one-ring convolution and sigmoid activation function for vector concatenation-based attention (Jetley et al., 2018). The gating signal forms a grid on vertices conditioned to spatial information for each level of icosahedron from skip-connection, which enables the network to combine information from multiple scales of spherical feature maps to achieve better performance. The parameters within those attention gates can be optimized with the general back-propagation-based training of neural networks.

We trained and evaluated the model with fetal cortical surfaces extracted from 55 fetuses using 5-fold cross-validation while 10% of the training samples selected were used for validation during its training. In order to increase the number of training samples, we applied three-dimensional rotational augmentation on the icosahedron. We randomly rotated input folding maps and output parcellation maps on the sphere and re-tessellated rotated maps onto the icosahedron with 40,962 vertices using the barycentric interpolation (Berrut and Trefethen, 2004). After rotational augmentations, we normalized each input folding map’s value with the z-score transform (Devore, 1995). For the loss function, we used the Dice loss using the Adam optimizer (Kingma and Ba, 2014) with a learning rate of 1e−3. To get the best optimal weights of the model in each fold, we monitored the Dice coefficient for the validation set and applied the learning rate reducer with a factor of 0.1 with five-epoch patience for the stagnation of validation loss. The training continued for 100 epochs and the network weights that showed the highest dice coefficient for the validation set were stored as the optimal network for each fold. The entire automatic parcellation framework was developed using Tensorflow (Dillon et al., 2017) backend, and the training and evaluation process was conducted with Nvidia RTX A5000 GPUs.

We used the dice coefficient to evaluate automatic cortical surface parcellation performance, measuring the regional overlap between the predicted parcellation label and ground truth. The overall Dice coefficient is used to measure the general parcellation performance of the model according to the following formulas.

The l denotes the class label and i represents the vertex. The pil is the one-hot encoded prediction labels at vertex i for class l from automatic parcellation models and gil is the ground truth label at vertex i for class l based on manual parcellation.

We also computed boundary distance for each region to compare how predicted regional boundaries are aligned with boundaries defined on ground truth. We measured surface distance over vertices from the ground truth boundaries to the boundaries of the predicted regions and used their median to represent boundary distance. For region j and vertex i on predicted boundaries defined with k vertices, dij denotes the shortest distance toward the ground truth boundary of region j. Likewise, we also computed their mean to represent the overall boundary distance.

Additionally, we investigated the effect of parcellation performance on cortical surface measurement. For each region j, we computed the ground truth regional area, areaGTj, from the manual parcellation map and the predicted regional area, areapredj, from each automatic parcellation method for each cortical parcel. Then, we computed the absolute percent error for each region and measured their mean as the overall error for each automatic parcellation method.

We compared the automatic parcellation performance of the proposed attention-gated spherical U-net with surface registration-based parcellation (Fischl et al., 2004), SPHARM-net (Ha and Lyu, 2022), and original spherical U-net (Zhao et al., 2019). For surface registration-based parcellation, we aligned a 29 GA template surface with predefined regional labels, constructed with a different TD fetal cohort (Serag et al., 2012; Yun et al., 2019), to the individual cortical surface using a 2D sphere-to-sphere non-rigid warping (Robbins et al., 2004). Then, we resampled the label map for the individual surfaces.

All statistical analyses were conducted with IBM SPSS Statistics (Version 29), IBM Corp, and MATLAB, MathWorks Inc. For the statistical comparison of the parcellation performance, all the metrics and measurements are computed subject-wisely to investigate the improvement of parcellation performance in a paired manner. We first performed paired t-tests on the overall Dice coefficient, median boundary distance, and mean absolute percent error of area measurements from the proposed model against the registration-based parcellation, SPHARM-net, and original spherical U-net to statistically compare their automatic parcellation performances. For the regional evaluation, we performed paired t-tests on the dice coefficient, boundary distance, and absolute percentage error of area measurement for each region between methods. We used the false discovery rate (FDR) control method at a q-value (FDR adjusted p value) of 0.05 to adjust for multiple comparisons (Benjamini and Hochberg, 1995).

We examined the performance according to variations of GA and gyrification. We performed linear regression analysis, Dice=GA+c, to examine the relationship between GA and parcellation performance. Furthermore, we also divided the subjects into two subgroups based on GA, early third trimester (27–33 GA) and late third trimester (33–39 GA). Even fetal brains in both subgroups are under neurodevelopment for the maturation of the brain, cortical folding structures in the late third trimester are much more similar to infant or adult brains than those in the early third trimester. We performed paired t-tests on the overall dice coefficients for early and late third trimester subgroups to assess how neurodevelopmental stages affect brain parcellation performance.

Previous cortical surface parcellation methods are not sufficient for parcellation in fetal brains which have small size, limited gyrification, and large temporal variation since those methods rely on the presence of well-defined gyral and sulcal patterns. These differences require the parcellation model to capture information both adaptively and sensitively. The attention mechanism has emerged as a pivotal component in deep learning models, particularly in tasks that require discerning intricate patterns and relationships within data (Oktay et al., 2018; Schlemper et al., 2019). In the context of cortical surface parcellation, the attention mechanism plays a crucial role in instructing the model to focus on specific patterns on the cortical feature maps, which are relevant to infer the parcellation outputs (Jetley et al., 2018; Liu et al., 2022), thereby it contributes to improving the accuracy and precision of the automatic parcellation in fetal brains. The statistically significant improvements in overall dice coefficients (Table 1), as compared to previous methods, demonstrate the effect of attention mechanisms on fetal cortical parcellation. By focusing on relevant features and suppressing irrelevant ones, the attention mechanism allowed the model to capture fine-grained boundaries more accurately, which leads to the most similar boundaries of ground truth (Figure 7) and lowest boundary distances among the automatic parcellation approaches both globally and locally (Table 2; Figure 4). This is particularly important in cortical surface parcellation in fetal brains, where the distinction between different gyral regions can be subtle.

Furthermore, the attention mechanism aids in contextual understanding (Zhang et al., 2018). In the complex landscape of the brain, understanding the contextual information such as global and local folding patterns of the cortical surface is important to infer regional labels on the cortical surface. The attention mechanism allows the model to weigh the importance of different regions based on their context, leading to more accurate parcellation (Chen et al., 2018). The region-wise evaluation further reinforces the superiority of the attention-gated spherical U-net since each cortical region has a different rate of growth and gyrification. In the regions where cortical folding formed in the early developmental period, such as the precentral gyrus, postcentral gyrus, and insula, all of the parcellation methods showed stable performances. On the other hand, the proposed model showed greater performance in the cortical regions with late gyrification, such as the caudal middle frontal gyrus, cuneus, inferior parietal gyrus, par orbitalis, and transverse temporal gyrus (Figure 3). Not only did it outperform the surface registration-based method, but it also showed significant improvements against other deep learning models like SPHARM-net and the original spherical U-net. This suggests that the attention mechanism is not just a supplementary feature but a core component that substantially enhances the model’s performance.

Lastly, the improvement of the parcellation performance with the proposed model was not limited to the evaluation via dice coefficients. The proposed model showed the lowest error in the actual cortical surface measurements both globally and regionally (Tables 3, 4). This result emphasizes the effectiveness of the attention mechanism for automatic cortical surface parcellation, which eventually leads to precise regional analysis of the fetal brain.

Analyzing parcellation performance along GA provides valuable insights into the model’s adaptability and accuracy across different developmental stages. The fetal brains in the third trimester are under rapid growth of the cerebral cortex with gyrification, forming foldings on the surfaces. It means that the region belonging to the same parcellation label could have large temporal variations on its folding feature maps along with GA. For the precise parcellation of fetal brains across a wide range of GA, the model is required to find peculiar patterns composing each cortical region and focus on them, such as its sulcus/gyrus and their relative relationship, despite the large temporal variation in fetal brains.

The result shows the usefulness of the attention-gated spherical U-net for fetal cortical surface parcellation across a wide range of GA (Table 5; Figure 5). The surface registration-based method showed lower dice coefficients for all GAs and a bigger variation in subjects with higher GA, which led negative correlation between GA and dice coefficients. The SPHARM-net showed slightly lower performance around 31 weeks and decreased dice coefficients with GA. Our attention-gated spherical U-net was less influenced by the variations in brain development stages, suggesting that it could be a more reliable tool for fetal studies across different GA. Furthermore, the subgroup analysis dividing subjects into early (27–33 GA) and late third trimester (33–39 GA) subgroups further emphasizes the proposed model’s capability to adapt to the dynamic nature of fetal brain development during this period showing the highest overall Dice coefficients within both subgroups (Figure 6).

The proposed attention-gated spherical U-net showed improved performance in cortical parcellation of the fetal brain, there are some limitations to be addressed in future studies. First, the fetal MRIs used in this study were sourced from a single center. This may introduce biases related to specific imaging protocols, equipment, and patient demographics. Multi-center studies are essential to validate the generalizability of our findings across different settings and populations. Secondly, the number of subjects included in this study is relatively smaller than other cortical brain parcellation studies due to the rarity of fetal MRIs. For future research, a collaborative multi-center approach with a larger cohort is recommended to gather a diverse set of fetal MRI data, ensuring broader generalizability. Lastly, constructing a comprehensive dataset including possible variations by maternal health, socioeconomic status, racial and ethnic background, and other clinical factors will enhance the model’s adaptability and accuracy in reflecting real-world variability in fetal brain development.