A Disentangled Representation Based Brain Image Fusion via Group Lasso Penalty - Wang, Anqi; Luo, Xiaoqing; Zhang, Zhancheng; Wu, Xiao-Jun (Frontiers in Neuroscience, 2022)
Complementary and redundant relationships inherently exist between multi-modal medical images captured from the same brain. Fusion processes conducted on intermingled representations can cause the information distortion and the loss of discriminative modality information. To fully exploit the interdependancy between source images for better feature presentation and improve the fusion accuracy, we present the multi-modal brain medical image fusion method in a disentangled pipeline under the deep learning framework. A three-branch auto-encoder with two complementary branches and a redundant branch is designed to extract the exclusive modality features and common structure features from input images. Especially, to promote the disentanglement of complement and redundancy, a complementary group lasso penalty is proposed to constrain the extracted feature maps. Then, based on the disentanglement representations, different fusion strategies are designed for complementary features and redundant features respectively. The experiments demonstrate the superior performance of the proposed fusion method in terms of structure preservation, visual quality and running efficiency.