Brain Lesion Segmentation and Detection on Multi-parametric MRI Data: Marrying Deep Learning with Low-rank Factorization

Brain lesion segmentation plays a crucial role in clinical neuroimaging, aiding in diagnosis, treatment planning, disease monitoring, and research. Accurate identification of lesion location and extent empowers clinicians to make informed decisions about patient care and provides researchers insights into the mechanisms of these conditions. Magnetic resonance imaging (MRI) is the primary modality for diagnosing and assessing brain lesions. A combination of MRI sequences, known as multiparametric MRI (mpMRI), is often employed to provide a comprehensive assessment of the lesion tissue under study. This technique capitalizes on the unique tissue characteristics highlighted by each sequence, offering a richer representation than any single sequence alone could provide.

However, manual segmentation of brain lesions in mpMRI data is time-consuming, costly, and subjective, with potential variability between observers. Automated methods ensure consistent, reproducible results and eliminate both inter- and intra-observer variability, which is essential for longitudinal studies and multi-center trials. While deep learning models, particularly convolutional neural networks (CNNs), have revolutionized image segmentation, several challenges remain in their application to brain lesion segmentation in clinical practice. These include data scarcity, variability, class imbalance, high computational requirements, and the ``black box'' nature of deep learning models.

This thesis aims to develop efficient data-driven models for automated segmentation and detection of brain lesions, specifically brain tumors, Multiple Sclerosis (MS) lesions, and stroke lesions, using mpMRI. By fusing advanced deep learning with low-rank factorization techniques, we introduce a diagnostic tool that segments and detects brain lesions accurately and interpretably without added computational complexity.

In the first part of this thesis, we focus on the application of CNNs for the automated segmentation of new MS lesions in 3D FLAIR images. Our goal is to identify new lesions between two longitudinal MRI scans of an MS patient, which is a key indicator of disease progression. We introduce Pre-U-Net, a 3D encoder-decoder architecture consisting of pre-activation residual blocks. To mitigate limited training data and class imbalance, we employ data augmentation and deep supervision for optimal model training. Comparative analysis reveals that Pre-U-Net outperforms both U-Net and Res-U-Net on the MSSEG-2 dataset.

The centerpiece of this thesis presents a novel method that integrates low-rank factorization with deep learning for enhanced medical image segmentation. Recognizing the limitations of CNNs in exploiting global context and the quadratic complexity of attention in transformers, we introduce a family of models, called \emph{Factorizer}, which leverage the power of low-rank matrix factorization to construct a scalable and interpretable segmentation model. More specifically, we formulate nonnegative matrix factorization (NMF) as a differentiable layer and incorporate it into a U-shaped architecture. Moreover, we use the shifted window technique in combination with NMF to effectively aggregate local information. Our results indicate that Factorizers outshine CNNs and transformers in terms of accuracy, complexity, and interpretability, setting new benchmarks on the BraTS and ISLES'22 datasets. Notably, our experiments show that NMF components are highly meaningful, with each component highlighting specific regions, offering Factorizers a unique edge in interpretability over CNNs and Transformers. Moreover, our ablation studies reveal a distinctive feature of Factorizers that allows a significant speed-up in inference for a trained Factorizer model, without requiring additional steps or significant accuracy trade-offs.

In the final part, we propose a method utilizing low-rank tensor networks to enhance CNNs for brain tumor segmentation. Given that many effective 3D CNNs are prone to overfitting due to their complexity and limited training data, we introduce a 3D U-Net-like architecture integrated with residual blocks. By imposing low-rank constraints on convolutional layer weights, we aim to avoid overfitting. This approach allows the creation of networks with considerably fewer parameters. We assess our method performance in the BraTS 2020 challenge data.