SE was the first elastography imaging system introduced in the 1990s and can be divided into two methods (20): (a) static or quasi-static strain imaging: the operator manually compresses the tissue with an ultrasound transducer; and (b) acoustic radiation force impulse (ARFI) strain imaging: a short-duration high-intensity acoustic “pushing” pulse called ARFI can be used to displace the tissue (21).

According to Hooke’s law, E in strain imaging can be calculated by the equation E=σ/ϵ, where σ is the externally applied stress and ϵ is the strain, which equals the displacement (21). Due to the unknown applied stress to the tissue, the strain imaging system cannot provide the qualitative value of E. Therefore, in clinical practice, strain ratio (SR) is an often-used semiquantitative measurement. SR can be calculated as the ratio between the strain in the normal reference region and the strain in the region of interest. SR>1 indicates that the deformation of the target lesion is less than that of the normal reference tissue, indicating lower strain and greater hardness (1). There are some other common parameters: elasticity scores (ES) or grading systems, fat-to-lesion SR and elastography-to-B-mode size ratio (1).

SWE, which is more quantitative and reproducible than SE, can be classified into two methods: (a) transient elastography: The first commercial SWE system Fibroscan™ is based on transient elastography, which is widely used to estimate liver fibrosis (5); (b) point shear wave elastography and 2D-SWE: ARFI is used as the external excitation.

Moreover, in contrast to nonimaging USE methods (transient elastography and point shear wave elastography), 2D-SWE is an emerging technology that can measure shear wave velocity (SWV) or E in real time and generate quantitative elastograms (21). The semitransparent color elastogram is usually overlayed on the corresponding B-mode sonogram, with red usually representing hard tissue and blue representing soft tissue. The SWV and E are related to the colors in the color bar along the image (22). The system will report E by using the equation E=3ρc_s ^2, in which ρ represents the tissue density and c_s represents the SWV (1). SWV is higher in hard tissue and lower in soft tissue.

As a branch of AI, ML enables the creation of algorithms that are able to learn from data and make predictions, thus enabling computers to learn like humans (23). It is an interdisciplinary field involving computer science, statistics and a variety of other disciplines concerning automatic improvement (24).

According to the types of labels utilized in the training dataset, ML techniques can be broadly classified by supervised, unsupervised and reinforcement learning (25) ( Figure 2 ). Most ML algorithms related to radiology are supervised (11), which requires instances labeled with the desired classification outputs, named the “ground truth”. Examples of such algorithms include artificial neural networks (ANNs), support vector machine (SVM), random forest (RF), logistic regression, Naïve Bayes classifier, decision tree and K-nearest neighbor. In contrast, unsupervised learning is an algorithm in which instances are unlabeled and clusters of data need to be identified (26). Examples of such algorithms are K-means, fuzzy C-means clustering and Markov random fields. Since acquiring well-labeled databases is time-consuming, reinforcement learning, as a hybrid of supervised and unsupervised learning, uses less detailed information to train a model (27). It acquires data by learning from dynamic environment interaction (the computer will receive positive or negative reinforcement feedback) without being explicitly taught. The overview of ML strategies and their related algorithms and applications used in USE imaging has been presented in Table 1 .

In the field of radiology, ML algorithms often start with a set of available inputs (the image data, for example) and desired outputs (the classification of malignant and benign tumors, for example) (11). The input datasets are usually split into two sets: training and validation datasets. The training dataset serves to find the optimal weights and fit the model, while the validation dataset is used to optimize the parameters. After a model is developed, there is often an urgent need for an independent external test dataset to evaluate the performance and generalizability of the developed model (23).

The availability of large-scale labeled datasets, faster algorithms and more powerful parallel computing hardware, such as Graphics Processing Unit, have enabled the fast application of DL (28). DL methods enable to solve problems that have resisted the best attempts of the AI community for many years, since the use of large numbers of layers allow the improved universal approximation properties and the more features to be learned from the data with multiple levels of hierarchy and abstraction (29).

Convolutional Neural Networks (CNNs) have been considered as the state-of-art algorithms of DL methods in the field of radiology for computer vision tasks such as segmentation (30), detection (31, 32) and classification (33). There are four key ideas behind CNNs: local connections, shared weights, pooling and the use of many layers, resulting in the improved accuracy and efficiency of the whole system (29). The relationship between AI, ML, DL and CNNs is shown in Figure 3 . CNNs are formed by a stack of one input layer, one output layer and multiple hidden layers, which consist of convolutional layers, pooling layers and fully-connected layers (34) ( Figure 4 ). With convolution and pooling applied repeatedly, fully-connected layers are used to make classification or predictions (35). The combinations of layers are various and some deep neural network architectures have been successfully utilized in image analysis, such as GoogleNet (36), AlexNet (37), VGGNet (38) and ResNet (39).

Transfer learning (TL) strategies have recently been utilized in the medical world to avoid the overfitting problems caused by the lack of data. Within the TL method, knowledge can be shared and transferred between different tasks (40). The workflow involves two steps: pretrained on a large dataset (the ImageNet, for example) and fine-tuning on the target dataset (the limited ultrasound images, for example). In other words, by fine-tuning the DL architecture, the knowledge learned from one dataset can be transferred to another dataset obtained from another center.

Radiomics is defined as quantitative mapping, namely, to extract and analyze vast arrays of high-dimensional medical image features that are related to the prediction targets (41). Radiomics features, such as intensity, shape, texture or wavelet, reflect the underlying pathophysiology and provide the information on tumor phenotype and microenvironment (42). These features can be used solely or in combination with other relevant data sources (such as clinical reports, laboratory tests or genomic data) for robust evidence-based decision support to facilitate prediction, diagnosis, prognosis, or monitoring (41).

Radiomics features can be scored semantically by experienced radiologists, or can be mathematically computer-calculated to describe the size, shape and textures of the ROI, or can be created by DL algorithms (43). The calculated features are subsequently analyzed usually using traditional ML methods to build predictive models (44). Researches have already shown the capacity of radiomics analyses to increase diagnostic, prognostic, and predictive power (42).

Liver fibrosis is a principal factor in the development of CLD. Precise assessment of the amount and progression of liver fibrosis is of great value for the management and prognosis of CLD patients. Liver stiffness measured from USE has been regarded as the first-line noninvasive method; thus, AI-based USE models is proposed to reduce variability in stiffness measurements and to improve the diagnostic accuracy of liver fibrosis evaluation.

Earlier studies have extracted a series of hand-engineered features and then fed into various ML classifiers for predicting the different liver fibrosis levels. Textural features, liver stiffness and SWV have been proven to be the discriminant features, with ANN, SVM and RF being the most used classifiers (48–54).

More currently, AI with DL methods is expected to directly operate on the input images and reveal information that human experts cannot recognize. The better diagnostic performance of CNN than conventional ML classifiers for staging liver fibrosis has been demonstrated in the study of Brattain and coworkers (55). To obtain more standard SWE measurements and to detect significant fibrosis in patients with nonalcoholic fatty liver disease, they developed an automated framework called “SWE-Assist”. The model could automatically check the quality of SWE images, select a ROI, and classify the ROI with RF, SVM classifiers and CNNs. The CNN architecture yielded the largest classification improvement with an AUC of 0.89 in a large dataset of 3392 SWE images. A new method called DL radiomics of elastography (DLRE) has been proposed by Wang et al., which was expected to extract more radiomics features (56). In 398 patients with 1990 images from 12 hospitals, DLRE reached AUCs of 0.97 for F4, 0.98 for ≥ F3 and 0.85 for ≥ F2, which were significantly better than LSM and biomarkers. They also found that the model generated better performance as more images were acquired. As the DLRE model in Wang et al. showed limited accuracy in assessing ≥ F2, Lu et al. developed a new model named DLRE 2.0 based on the previous model by using the TL strategy for a more accurate evaluation of ≥ F2 liver fibrosis (57). They also developed three other DLRE-based models by gradually adding the tissue texture of the liver capsule and the liver parenchyma and serological results to DLRE 2.0. For evaluating ≥ F2, the performance of DLRE2.0 was significantly better than the previous DLRE model (AUCs: 0.84 vs 0.92, p < 0.05). However, no significant improvement was shown when adding more information. By using the TL strategy for classifying liver fibrosis, Xue et al. pretrained the InceptionV3 network on ImageNet to analyze multimodal BMUS and SWE images, generating higher diagnostic accuracy than non-TL with significantly higher AUCs (0.950, 0.932, and 0.930 for classifying F4, ≥ F3, and ≥ F2, respectively) (58). Furthermore, the model based on the multi-modal images yielded the highest diagnosis accuracy, outperformed the single-modality, LSM, and biomarkers.

How to define a reliable SWE image, especially a temporally stable SWE color box, is one of the limitations of USE. Gatos et al. tried to identify the temporally stable regions across SWE frames and to evaluate the impact of temporal stability for classifying various CLD combinations by means of a pretrained CNN scheme (59). The stability masked SWE images showed improved diagnostic performance compared to the unmasked ones. Therefore, in their study in 2020 (60), full and temporally stable masked SWE images were separately fed into GoogleNet, AlexNet, VGG16, ResNet50 and DenseNet201 with or without augmentation. All networks achieved maximum mean accuracies ranging from 87.2%–97.4% and AUCs ranging from 0.979–0.990.

Early detection and precise diagnosis of HCC are crucial for treatment selection and patient prognosis, making the differentiation of FLLs an important task. Since structural alterations can be reflected by changes in tissue stiffness, USE, especially SWE, has been widely used for differentiating benign from malignant FLLs (61, 62).

For different diagnostic purposes (all including FLL classification), radiomics approaches have been proposed with an emphasis on the extraction of high-dimensional features from different modality ultrasound images and the integration of the high-dimensional features with the low-dimensional data, such as expert-designed SWE parameters or serological clinical data (63, 64). Specifically, to classify FLLs, Wang et al. established two radiomics models: the ultrasomics score (based on radiomics features only) and the combined score (based on radiomics features and quantitvative SWE measurements) (63). With 1044 features extracted by ultrasomics, they found that the combined score had the best performance, yielding an AUC of 0.92. Chronic hepatitis B infection is the main risk factor for HCC in China. Early and accurate prediction of HCC occurrence in chronic hepatitis B patients is of great benefit for individual treatment and prognosis. Jin et al. established different HCC prediction models by combining high-throughput radiomics features using DL radiomics with serological and clinical information (64). Finally, the model including SWE, BMUS radiomics features, sex and age showed the best prediction performance (AUCs: 0.981, 0.942 and 0.900 in training, validation and testing cohorts for predicting 5-year prognosis of HCC).

The prediction of malignant subtype and clinical prognosis are also important for the decision-making and treatment of HCC patients. Yao et al. have demonstrated the value of the radiomics analysis system based on multi-modal ultrasound images (BMUS, SWE and SWV imaging) for benign and malignant classification, malignant subtyping, programmed cell death protein 1 prediction, Ki-67 prediction, and microvascular invasion prediction, with the AUCs ranging from 0.94-0.98 (65). The more detailed performance of AI based on USE for the evaluation of liver diseases is summarized in Table 2 .

Accurate delineation of breast masses on ultrasound images is an indispensable first step for image interpretation. Manual segmentation of breast masses is labor intensive and time-consuming (67). The various artifacts, uneven intensities and blurred boundaries in USE images have made automated segmentation of breast mass still a tough task (68). Sergiu and coworkers developed a probabilistic model for every pixel derived by a video sequence, followed by a Deterministic Annealing Expectation Maximization method used for automatic image segmentation of breast elastographic images (69). Only an error of 5% on the phantom test images was provided.

Early detection and treatment of breast cancer can significantly decrease mortality. Therefore, the differential diagnosis of benign and malignant breast masses is of great value in breast mass evaluation. Although the American College of Radiology Breast Imaging Reporting and Data System (ACR BI-RADS) can provide a standardized and systemic interpretation of breast ultrasound, the problems of inter- and intra-observer variability can be enormously addressed by AI solutions (70).

Early AI-based USE models for breast mass classification focused on the extraction of discriminant features and the utilization of various classifiers. The elasticity indices, such as SR and quantitative tissue elasticity, as well as texture features have served as the most commonly used handcrafted features (71–77). ANN and SVM classifiers are the most commonly used ML algorithms, all yielding satisfying performance. The fused ultrasound B-mode and elastographic features, have all been proven to generate better results than single-modal imaging (71, 72, 75, 77–83). It is also proved that the elasticity features along the rim surrounding the lesion is valuable for the classification of breast masses (84).

However, it is difficult to extract human-crafted features from USE images since they often contain irrelevant patterns (28), and the classification performance is greatly influenced by the selection of particular features. Recently, DL algorithms, especially CNNs, have made great strides in establishing AI-based USE models for breast mass classification (85–88). Zhang et al. developed a two-layer DL architecture based on SWE, comprising a pointwise gated Boltzmann machine and a restricted Boltzmann machine (85). The pointwise gated Boltzmann machine, restricted Boltzmann machine and SVM classifiers were used for automated feature learning, distinct representation learning and classification of breast tumors, respectively. Compared with the handcrafted statistical features, an accuracy of 93.4%, a sensitivity of 88.6%, a specificity of 97.1%, and an AUC of 0.947 were achieved. Fujioka et al. developed a DL model based on SWE images using six CNN architectures with different epochs for breast mass classification (86). The developed DL model reached a mean AUC of 0.870, showing equal or better diagnostic performance compared with the radiologists who analyzed the images using the 5-point visual color assessment and the mean elasticity value. Li et al. found that the CNN-based model using dual modal ultrasound images could provide a more pronounced improvement in diagnostic performance for inexperienced radiologists, with the AUC increasing from 0.794 to 0.830 (87).

Radiomics can provide automated quantification of high-throughput image features. It is expected to reveal disease characteristics that are invisible to the naked eye (42). Generally, the proposed ML-based radiomics models focused on the extraction of low- and high-order features and the utilization of various feature selection algorithms (89–91). The radiomics features involve shape, intensity, texture or wavelet features from different modality ultrasound images, such as SE (89), SWE (90), contrast-enhanced ultrasound (CEUS) (91) and B-mode ultrasound (BMUS) (90, 91). The frequently used feature selection algorithms include hierarchical clustering (89), the least absolute shrinkage and selection operator regression algorithm (90) and the genetic algorithm (91). The selected discriminant features are used alone or in combination with clinical data to train the ML classifiers for breast mass classification. More recently, the potential of using DL radiomics to facilitate the classification of breast masses has also been confirmed, which was expected to identify vast arrays of quantitative features (92, 93). Zhang et al. used a CNN to extract 768 radiomic features from segmented BMUS and SWE images to further build radiomics scores, which was then confirmed to have a better performance than the radiologist assessment using BI-RADS and quantitative SWE features for discriminating benign from malignant breast masses (92). However, this study has the limitation of complicated segmentation tasks. Zhou et al. utilized a CNN for both radiomics feature extraction and breast masses classification, and 4224 low-level and high-abstract features were extracted directly from 540 SWE images that does not need object segmentation (93). The model reached a classification accuracy of 95.8%, a sensitivity of 96.2% and a specificity of 95.7%.

Since TL is an effective strategy to augment accuracy and to reduce training time by transferring the knowledge learned from a source domain to a target domain, Fei et al. proposed a projective model-based multilayer kernel extreme learning machine to transfer parameters (94). The SE-based diagnosis with BMUS imaging used as the source domain generated the best performance, with an accuracy, sensitivity, and specificity of 87.12, 86.06, and 88.15, respectively. Liao et al. applied the VGG-19 network pretrained on the ImageNet dataset based on either single-modal or multi-modal images for breast tumor classification (95). The combination feature model based on BMUS and SE images yielded a correct recognition rate of 92.95% and an AUC of 0.98.

Accurate preoperative evaluation of ALN status in patients with breast tumors is important for surgical decisions and prognosis (96). To analyze ALN status, several AI models have been proposed based on the ultrasound images of ALNs (97) or of primary breast tumors (98, 99). For the differentiation of disease-free axilla and any axillary metastasis in early-stage breast cancer, Our team (98) and Zheng et al. (99) proposed to combine the radiomics features from BMUS and SWE with independent clinical risk factors, all generating satisfactory results. In addition, the DL radiomics model of Zheng et al. could also discriminate between a low and heavy metastatic axillary nodal burden, with an AUC of 0.905 (99).

Knowing how the tumor responds to treatment can be helpful for subsequent treatment selections. Furthermore, it is important to evaluate the pathological complete response (pCR) in breast cancer, since it is related to the rates of long-term survival. Fernandes et al. found that the SR could be predictive of the neoadjuvant chemotherapy response in locally advanced breast cancer (100). A significant difference in tumor stiffness was observed as early as 2 weeks into treatment. By using the preoperative data, the Naïve Bayes classifier achieved a classification of pCR and npCR with a sensitivity of 84%, a specificity of 85%, and AUC of 81%. The detailed results are summarized in Table 3 .

Segmentation plays an essential role in AI-based USE models, for that malignant thyroid nodules can be accurately diagnosed by using features of well-segmented nodules. However, USE images often have low image quality due to the existing high noise, making automated segmentation a tough task. To segment the thyroid nodules in a noisy environment, Huang et al. proposed a new segmentation method based on the adaptive fast generalized fuzzy clustering algorithm by utilizing the gray level and spatial position information of the original image (104). The proposed method obtained segmentation accuracies of 0.9981 in Gauss noise (0.03) and 0.9986 in Gauss noise (0.05), indicating that it had a strong ability to suppress noise and obtained more accurate results when clustering images with high noise.

Although the indices such as ES and SE have been introduced, SE is still a quantitative and subjective imaging method. Ding et al. computed a quantitative metric “hard area ratio” by transferring the original color thyroid elastograms from the red–green-blue color space to the hue-saturation-value color space (105). The SVM classifier obtained an accuracy of 93.6% when the hard area ratio and textural feature were used. Two studies used logistic regression analysis to investigate which sonographic features were associated with the malignancy of thyroid nodules and established formulas for predicting whether thyroid nodules were malignant or benign (106, 107).

Whether the ML-based diagnostic pattern can provide a more effective and accurate diagnosis than human experts for thyroid nodule classification still remains unknown. In a large study including 2064 thyroid nodules, Zhang et al. compared the diagnostic performance of nine ML classifiers trained on 11 BMUS features and 1 SE feature with experienced radiologists for thyroid nodule discrimination (108). The RF classifier generated the highest AUC of 0.938, performing better than radiologist diagnosis based on BMUS only (AUC= 0.924 vs. 0.834) and based on both BMUS and SE (AUC= 0.938 vs. 0.843). Recently, both ML-based visual and radiomics are popular methods used to diagnose thyroid nodules, Zhao et al. found that the ML-assisted US visual approach had the best diagnostic capability compared with radiomics approach and American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) (AUCs: 0.900 vs. 0.789 vs. 0.689 for the validation dataset, 0.917 vs. 0.770 vs. 0.681 for the test dataset) (109). When employing the ML-assisted US+SWE visual approach, the unnecessary fine-needle aspiration biopsy rate decreased from 30.0% to 4.5% for the validation dataset and from 37.7% to 4.7% for the test dataset.

Since DL is data-hungry, and the lack of standardized image data may lead to the overfitting problem, the TL strategy has also been applied to a few studies for thyroid nodule classification (110, 111). Qin et al. proposed to transferring feature parameters learned from VGG16, which was pretrained on ImageNet, to ultrasound images and used the hybrid features of BMUS and SE images to build an end-to-end CNN model (110). The proposed AI-based method yielded an accuracy of 0.9470, which was better than other single data-source methods. Pereira et al. compared the performance of conventional feature extraction-based ML approaches, fully trained CNNs, and TL-based pretrained CNNs for the detection of malignant thyroid nodules (111). The results showed that the pretrained network yielded the best classification performance with an accuracy of 0.83, which was better than that of fully trained CNNs. This may be caused by the relatively limited sample size used to train the fully trained network.

Although papillary thyroid cancer (PTC) is an indolent type of cancer, 20-90% of PTC patients are diagnosed with cervical lymph node metastasis (CLNM) (112), which is highly related to recurrence and a poor survival rate. Accurate CLNM estimations of PTC patients are clinically important. Liu et al. built a radiomics-based model, which extracted 684 radiomic features from both BMUS and SE images to estimate LN metastasis for PTC patients, yielding an AUC of 0.90, which was better than using features extracted from BMUS or SE separately (113). However, it only utilized the radiomics features, with no consideration on other clinical information. Our team developed a radiomics nomogram by incorporating SWE radiomics features as well as clinicopathological risk factors for predicting CLNM in PTC patients, which showed good diagnostic performance in the training set (AUC of 0.851) and the validation set (AUC of 0.832) (114). The detailed performance of AI based on USE for the evaluation of thyroid nodules is summarized in Table 4 .