Angiogenesis is a dynamic process and an important early step during breast cancer initiation and progression (1, 2). The morphologic quantitation and characterization of blood vessels in a biopsy specimen relies on indirect measures such as microvessel density (number of small and tortuous vessels, immunohistochemistry (factor VIII, CD31, CD34)) (2–6). High microvessel density predicts metastasis and poor survival in women with breast cancer, including women with early-stage breast cancer (Stage 1) (7–16). While morphologic characterization of vascularity has strong prognostic value, there is variability (1) between different pathologists—there is significant inter- and intra-reader variability and (2) within a tumor—frequently a single tissue slice does not adequately capture microvessel density of an entire tumor. The variability of morphological characterization of blood vessels becomes particularly limiting when evaluating (1) response to neo-adjuvant chemotherapy and (2) women who are at high-risk for developing a breast cancer (e.g., germline BRCA mutation). Consequently, imaging strategies are being developed and optimized for early detection of vascularization, and ultimately, neovascularization.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is routinely used to screen women at high-risk for breast cancer, evaluate the extent of local invasive breast cancer, and assist in planning of neo-adjuvant and adjuvant therapy (17). DCE-MRI utilizes timed pre- and post-contrast imaging; consequently, the semi-quantitative dynamics of contrast uptake and intensity (initial peak enhancements, delayed phase-washout) are routinely used for radiologic evaluation of DCE-MRI (18). DCE-MRI has proven to be highly sensitive and has revolutionized early detection of breast cancers, particularly in women who are at increased risk for breast cancer (19–31). Consequently, breast DCE-MRI has become standard of care and made it possible to obtain a high-resolution image of breast cancers and their associated vasculature. Despite the clinical success of breast DCE-MRI to image the breast with high sensitivity and resolution, the ability of DCE-MRI to image blood vessels with high precision remains a work in progress. Recently, through the new field of radiomics, contrast dynamics are being used to evaluate vascular density, vascular morphology, and detection of aggressive breast cancer biology. In addition to directly detecting and quantifying the vascularity, the indirect effects of altered vascularity such as perfusion and blood oxygen levels can be measured using both DCE-MRI alone and DCE-MRI in combination with other imaging modalities. These indirect approaches detect increased blood flow and hypoxia and have been shown to be diagnostic of breast cancer and predictive of treatment response (32–38). However, this review will focus on efforts to detect vessels in DCE-MRI of the breast.

DCE-MRI is standard-of-care for high-risk cancer screening and surgical staging and is not readily replaced. Furthermore, DCE-MRI time-sequence imaging data provides an opportunity to perform kinetic and pharmacokinetic modeling; this modeling holds promise to better characterize tumors (28, 30, 39, 40), monitor treatment effects (25, 31, 41), and plan radiation and surgical interventions (29, 42–44). Given the importance of DCE-MRI imaging, recent efforts have focused on detecting blood vessels in DCE-MRI images. However, the detection of blood vessels, particularly new blood vessels, in DCE-MRI images presents significant challenges.

First challenge is that DCE-MRI is not able to directly image biology. The presence of vessels is detected, both manually and computationally, based on high contrast enhancement and their linear morphology (shape). The fact that DCE-MRI provides dynamic 3D images of all tissue is simultaneously both the reason why it has so much potential benefit (wide-scale adoption and utility) and the reason why vessel detection is so difficult.

Second challenge is the resolution of MRI. Although MRI equipment with strong magnetic fields (>7.0 Tesla) can achieve a resolution <200 μm, the standard clinical MRI equipment used for breast cancer screening employs significantly weaker magnetic fields (1.5 or 3.0 Tesla) (45). This means that most MRI can only resolve features between 0.5 and 1.0 mm in size (46); small blood vessels range in size between 4 μm to < 1 mm (47). Consequently, DCE-MRI lacks the resolution to detect small or micro-vessels (48, 49).

A third issue is that breast tissue is inherently difficult to image. Unlike other tissue, where blood vessel detection has high accuracy, breast tissue presents many unique challenges (50–54). The human breast is highly heterogeneous both within the breast of individuals women and between women. Human breasts vary in size, the ratio of glandular tissue to adipose tissue, and the amount of collagen/extracellular matrix; furthermore, the human breast is subject to hormonal fluctuations during the menstrual cycle and undergoes complex changes during a woman's lifetime (puberty, pregnancy, lactation, involution, and menopause). The heterogeneity of the human breasts requires that imaging algorithms must be sufficiently flexible to have general applicability. Since breast tissue spans a range of densities and compositions, contrast enhancement changes within the breast and the intensity between background and tissue are not constant (48, 55). The breast also includes both adipose and fibroglandular tissues and both can appear on DCE-MRI images to have linear morphology (49, 55). Since blood vessels are detected based on contrast enhancement and morphology, accurately detecting blood vessels in the breast is very difficult. Additionally, the breast is highly deformable so movement during scans can change both the position and the shape of the breast resulting in imaging artifacts and making blood vessel detection even more difficult (49).

While it is challenging to perform vascular analysis of DCE-MRI (particularly in 3D), DCE-MRI is standard-of-care for breast cancer screening, surgical planning, and evaluation of response to neo-adjuvant chemotherapy. The advantages of DCE-MRI as a sensitive, non-invasive method to monitor for and detect lesions have made it a routine clinical procedure (19–21). Given the importance of DCE-MRI in clinical care, it is unlikely that it will soon or easily be replaced as the clinical standard-of-care by another imaging modality. Therefore, it is important to develop effective strategies to detect blood vessel density and morphology in DCE-MRI images. In this article, we aim to review computational approaches for detection and analysis of blood vessels in DCE-MRI images and present some of the strategies we have developed for co-registry of DCE-MRI images and detection of blood vessels and neovascularization.

Tumor Detection and Characteristics analysis is conducted both for research and to improve clinical decision making. Analysis uses morphologic features, enhancement kinetics, or a combination of both. To support clinical decision making, clinical aided systems work with human experts to improve both (1) early identification of tumors and (2) the sensitivity/specificity of breast imaging (57, 58). Many clinical aided systems have been developed and commercialized for mammography. In contrast, development of clinical aided systems for breast DCE-MRI is in its nascent stage and the systems that have been developed have not been widely tested outside the data set for which they were developed (58, 59). This lag in the development of clinical aided systems for breast MRI is primarily the result of lack of standardization of DCE-MRI across centres and institutions. Key differences in DCE-MRI protocols include the (1) specific gadolinium contrast agent used, (2) amount of contrast given, (3) magnet strength, and (4) image acquisition strategy. Despite these challenges, approaches have been developed and, within their data set, demonstrate excellent sensitivity and specificity.

The first type of Tumor Detection and Classification methods analyse the dynamics of the contrast agent. To identify tissue with malignant potential, the earliest methods of this type used kinetic curves, which show changes in contrast enhancement over time. The kinetic curve was constructed by plotting the signal intensity over time and classifying the curve based on the rate of contrast uptake and then washout. The shape of the washout curve was used to diagnose the tissue; benign or indeterminate tissue has slow uptake or plateauing intensity over time; malignant tissue has both rapid uptake and washout (60–66). The rapid uptake/washout of malignant tissue is attributed to the increased vascularity associated with tumors (66–68). These models, however, are often considered semi-quantitative and, due to inconsistencies between DCE-MRI protocols and machines, need to be tuned and adapted for each new study (59, 69). A more quantitative approach is to use pharmacokinetic models that apply mathematical equations to model the flow of contrast agent through the vasculature into the tissue (70); the most common model is a variant of the Tofts model (70–75). These flow models allow for measurement and comparison of physiologically relevant parameters; parameters measuring volume of contrast transferred (K^trans) and vascular volume (v_p) are able to distinguish between malignant and benign tumors (48, 69). The parameters K^trans and v_p, similar to kinetic curves measuring contrast uptake/washout, are assumed to correlate with the increased vascularity of malignant tissue. When compared to classifying kinetic curves, pharmacokinetic models have also been shown to out-perform kinetic curves (69, 76) and reduce the variability between DCE-MRI protocol and noisy signal (77). These models, however, also make assumptions about contrast flow and require multiple post-contrast time points; these assumptions limit their general applicability (74, 77, 78).

The second type of Tumor Detection and Classification methods involve machine learning. Machine learning approaches have been employed frequently and span the machine learning spectrum from clustering to deep learning neural networks. A number of reviews about machine learning in breast DCE-MRI focus on either the types of methods (79), the goal of the method (58, 80), or the conclusions of the study (57, 81). Starting with the machine learning approaches, we will provide a concise, non-exhaustive survey of the more popular machine learning methods being used and briefly describe each approach.

Fuzzy C-means clustering is one of the most widely used unsupervised approaches to analyse breast DCE-MRI images. The general strategy of Fuzzy-C means clustering algorithms is to classify input data into groups based on algorithms that minimizes in-group variability. Although many improvements have been made to the algorithm (82–86), Fuzzy C-means analysis of breast DCE-MRI images has been primarily limited to segmenting the boundaries of a lesion (84).

Machine learning algorithms that use a supervised training approach include logistic regression, linear discriminant analysis, random forests, and support vector machines. All these approaches require “training” on a fully classified set of data (i.e., DCE-MRI with all lesions labeled as either malignant or benign). After a training set is analyzed, each of these approaches can then be applied to a second labeled set (independent of the training) to test sensitivity and specificity. Although they share a training approach, these methods have important differences in how they perform classification. Logistic regression (for two classes) and linear discriminant analysis (for more than two classes) are both linear methods. Support vector machines and random forests (for both two and multi-class scenarios) are more robust non-linear methods. All these supervised machine learning methods have been used to classify malignant tissue (87–92).

The third type approaches used in Tumor Detection and Classification include artificial neural networks and deep learning (i.e., neural networks with multiple hidden layers). Neural network architecture contains an input layer, at least one hidden layer, and an output layer. Deep neural networks (i.e, deep learning methods) perform both feature extraction and classification. In the training phase, deep learning methods “learn” what features best classify the input data (81, 93). Because it is well suited for image recognition, the most common neural network architecture for DCE-MRI is convolutional neural networks (CNNs). The convolutional layers of a CNN extract imaging features while maintaining spatial relationships between features. The discovered features are aggregated by pooling layers before the output layers generate a classification. In addition to being applicable to Tumor Detection Classification (94, 95), the flexibility of CNNs address a wide range of breast DCE-MRI related research issues including fibroglandular tissue segmentation (96, 97); breast segmentation (95, 98); and detection of lymph node metastasis (99). The disadvantage to deep learning is that it requires a large, annotated training data set. However, transfer learning, where a network is pretrained on a large imaging database, has shown promise for reducing the required size of training data sets (80, 95–97). In other applications, deep learning has shown to out-perform standard computer vision and experts (94, 100); however, in MRI, deep learning approaches have not yet out-performed experts (58).

Radiomics is a field of research that views medical images as quantitative data; this data represents the phenotypic, genotypic, and molecular characteristics of the tissue. The goal of radiomic studies is to associate the quantitative features extracted from images to both qualitative (e.g., clinical variables and outcomes) and quantitative information (e.g., biomarkers, gene expression, or other relevant -omic measurements). Radiomic approaches have many research applications including diagnostic tools, clinical decision support, and hypothesis testing and generation (56, 101, 102). For comprehensive summary of radiomics in breast cancer, many researchers review radiomics studies based on their research goals (19, 56, 101–106). To achieve these research goals, radiomics has employed wide range of computational approaches. The computational approaches can be categorized based on the three different parts of the radiomic analysis: features extraction, feature selection, and model building (56). The goal of features selection and model building are, respectively, to classify features and then build predictive models so that unlabelled images can be classified. The main difference are the variables that are classified: in addition to detecting benign vs. malignant tissue, radiomic studies classify continuous variables to define cancer subtype, diagnosis, and prognosis. Since features selection and model building primarily use machine learning methods similar to those described in the machine learning section of this section, we will limit our review of computational approaches to those used in feature extraction.

Feature extraction is the process of identifying features from an image. The features can be any quantifiable data produced from either the full image or a segmented region of interest within the image. The main categories include morphology-, histogram-, texture-, and transform-based features (56). This process is not unique to radiomics and can include some already discussed features such as contrast kinetic features (i.e., washout dynamics and pharmacokinetic parameters). However, it is typically distinct from deep learning in that the extracted features are selected manually and not discovered by an algorithm (57).

The types of features extracted fall into two categories: semantic and agnostic (101). Sematic feature extraction attempts to capture prognostically meaningful characteristics such as the BI-RADS defined features of a lesion (e.g., density, spiculation, and vascularity). These features, which are visually identifiable by radiologists, primarily fall into the morphology- and histogram-based categories. Agnostic features are not directly observable and have to be computed or extracted from the image. The features most unique to radiomics are the texture- and transform-based features. The gray-level co-occurrence matrix (GLCM) is a texture-based feature used to represent tumor heterogeneity. GLCM is calculated by determining the frequency of neighboring pixels with specific intensities (106, 107). This texture-based analysis captures the spatial distribution of intensity. GCLMs and their second-order features have had success identifying lesions (87, 90, 108), characterizing molecular and histopathologic subtype (109–113), and predicting prognosis and therapy response (114–118). Transform-based features are features that are extracted after applying a function to the image to generate a new image. Examples of transformations include Fourier, Wavelet, and Laplacian of Gaussian transforms. Instead of the spatial dimension, these transformations convert the image to investigate texture in an alternative dimension such as time or frequency. Although not as widely used as GLCM, transform-based approaches in breast cancer have also identified, characterized, and predicted response (119–123).

The wide-spread adoption of radiomics has been hindered by both the (1) large amount of data needed for training and (2) limitations of DCE-MRI (e.g., contrast type and administration protocols, magnetic strength, etc.) (19, 57, 104). Combining radiomics with deep learning has shown promise for improved accuracy. Transfer learning, in particular, may be a type of deep learning that can reduce the size of data sets and time required for training.

It is difficult to automatically detect blood vessels in a non-specific imaging method such as DCE-MRI. DCE-MRI does not directly measure biology; therefore, the characteristics of the breast tissue increase the computational difficulty of measuring blood vessels. Initial efforts relied on experts who manually identified, measured, and scored blood vessels in DCE-MRI images. Since MRI provides no inherent method to distinguish vessels, detection is performed based on both their high enhancement in MRI due to rapid contrast uptake and their linear, network-like appearance (i.e., morphology) (49, 129, 132). Robust detection of wide range of vessels sizes requires image analysis at multiple scales (or resolutions). Some computational algorithms require reprocessing of an image at different scales, since a linear structure, if magnified sufficiently, no longer will appear linear. The number of and increment between scales depends on the (1) sensitivity of the algorithm and (2) MRI resolution. Finally, all contrast enhancement in DCE-MRI is reported as a numerical intensity; therefore, vessels can escape detection if: (1) nearby tissues exhibit rapid contrast uptake which occludes vessels; or (2) sufficiently small vessels do not contain enough contrast agent to achieve sufficient enhancement (49, 55, 133). Both these scenarios result in gaps appearing in the vascular network. These missing data require computational strategies to both detect and fill gaps when constructing a complete and accurate vascular network.

A robust computational algorithm must detect blood vessels with (1) similar accuracy between patients and (2) ideally, be independent of the MRI machine and protocol used. Differences in MRI machines can include different resolution (resulting from different magnetic field strengths), ranges of enhancement intensity, and software versions. Differences between breast DCE-MRI contrast administration protocols can result in contrast enhancement characteristics that cannot be directly compared. These include differences in the type of contrast agent, changes in contrast agent dose or administration rate, and different acquisition protocols or number of scans. Some of these factors can be minimized or corrected through analysis methods and image processing. For example, pharmacokinetic modeling avoids comparing intensities directly by instead comparing the model parameters after fitting. Intensity functions can be applied to the image before analysis to normalize the range of intensity within a DCE-MRI image (48). The intensity transfer function will enhance the low intensity tissue while suppressing the image background. Intensity thresholds can also be applied to remove background noise but generally require calibration within each image so that low intensity features are not lost (134).

Finally, a computational challenge that is specific to the breast is image co-registration. Although co-registration is not required for detecting vascularity in a single DCE-MRI image, co-registration is required to detect changes in vascularity (or other imaging feature) over time. Breast registration is difficult because the breast is highly deformable which results in a complex combination of both affine and non-rigid transformations (135–138). Additionally, many times the breast itself has changed over time either because of either natural biological cycles or malignant transformation. Together these make registration of serial breast MRI images exceptionally challenging.