The first step in the radiomics pipeline is the acquisition of a medical image, which becomes the basis for the analysis conducted throughout the rest of the process. The acquisition and reconstruction of medical images is subject to significant variability both within and between different institutions. In theory, any parameter that will affect the output distribution of voxel intensities will affect the calculated feature values, and thus the medical image acquisition and reconstruction parameters will greatly affect the outcome of the feature extraction process. This remains true regardless of the medical imaging modality used on the patient and can have a significant effect on the reproducibility of radiomics studies: features that demonstrate clinical relevance in one clinical setting may not be useful in a separate institution where a different imaging protocol is used. The slice thickness and pixel spacing of reconstructed medical images, for example, are known factors affecting radiomics feature output values (39, 40). A recent systematic review identified the reconstruction algorithm, number of iterations, and the level of gaussian smoothing as factors also affecting biomarker reproducibility for PET scans (41). When acquiring CT scans of patients, acquisition variables such as the tube current and voltage (42, 43), the pitch (44), and even the vendor of the scanner can affect the numerical output of calculated features (45, 46). MRI is particularly challenging as a modality since voxel intensity values are not standardised and can vary greatly depending on the acquisition parameters chosen (47), and studies have indeed demonstrated that some of these parameters, such as image noise (48), choice of reconstruction algorithm (49), dynamic range and matrix size (50), do affect the output feature values. Figure 1 displays a non-exhaustive list of the critical factors to consider that will affect biomarker outputs at each step in the radiomics workflow. The large dependence of radiomics features on acquisition and reconstruction protocols makes it imperative that these protocols are extensively documented when presenting the results of radiomics research to maximise study reproducibility (32).

A precise delineation of the ROI is a requirement for input into the feature extraction algorithm. The image voxels within this ROI define the anatomical/physiological area from which the features will be extracted in subsequent steps in the radiomics workflow; therefore, any variability in this segmentation will affect the numerical output for each feature. Segmentation of ROI’s can be done manually, semi-automatically or automatically. Manual segmentations performed by clinical experts are typically used, however, performing this task manually has well documented limitations such as inter-observer variability and a significant clinical time burden, which limits its feasibility for radiomics analyses in larger datasets (51, 52). Variability in manual segmentations has the effect of introducing bias into the evaluation of quantitative imaging biomarkers (53). Efforts should be taken to mitigate against this bias by performing multiple segmentations. Feature robustness analyses should be conducted both between multiple independent manual observer segmentations, and also between manual segmentations and semi- or fully automated algorithms, where possible (32).

Prior to radiomics feature analysis, there are a number of imaging post-processing steps that are typically conducted. Image discretization is one of these steps, involving the discretization of image voxel intensities from a continuous spectrum into a set of discrete intensity bins. Limiting the range of intensity values is necessary to make the calculation of subsequent radiomics features computationally tractable (22). It also has the benefit of noise reduction (54). There are two discretisation schemes available to radiomics researchers, namely, fixed bin number and fixed bin width, and the method chosen can affect the quantitative metrics subsequently extracted. A plethora of studies have demonstrated that the intensity discretisation scheme used can affect the reproducibility of radiomics features, and thus potentially affect the predictive model derived from them (40, 55–59). This fact, among many others discussed below, underscores the critical importance of transparent and comprehensive reporting of post-processing steps undertaken in radiomics studies.

Resampling image voxels to isotropic spacing is another necessary post-processing technique. Isotropic voxel spacing is necessary to ensure that the extracted texture features are rotationally invariant (54). Moreover, in-plane and through-plane spatial resolutions of medical images are commonly not unified across patient scans in radiomics datasets which can affect output feature values; therefore, resampling to a common spatial voxel resolution can be employed in an attempt to increase the reproducibility of feature values (30). There is as yet no consensus on whether up-sampling or down-sampling is preferred. Down-sampling images to a lower spatial resolution will necessarily result in information loss, whereas up-sampling images will result in the addition of false information. Different interpolation techniques exist to resample images to isotropic voxel spacing, such as nearest neighbour, trilinear, and cubic B-Spline; the method chosen can have a significant impact on the reproducibility of radiomic features. Recent studies have demonstrated that the chosen interpolation technique affects the number of reproducible features across all of the common modalities used in mPCa imaging, such as CT (60), PET (61), and MRI (62). Open and comprehensive reporting of the technique used is therefore necessary to ensure study reproducibility.

The crux of radiomics is the extraction of features from medical images. These features become the basis for which diagnostic and prognostic predictive models are generated which can be utilised to inform clinical decision making that is personalised to the specific biological attributes of the patient. In conventional radiomics practice, mathematically defined features that are hand-crafted using domain knowledge numbering in the hundreds, or sometimes thousands, are extracted from the ROI. However, with the recent surge in deep learning-based models and their applicability to medical images, it has become possible to mitigate the use of hand-crafted features and train complex neural network (NN) and convolutional neural network (CNN) models that are capable of learning the most salient features in unsupervised or supervised manners (63, 64). These two distinct types of features extracted from medical images will henceforth be referred to as hand-crafted features and machine-learnt features, respectively.

Hand-crafted features have the ability to capture either spatial or temporal heterogeneities within the defined ROI and can thus be categorised broadly as being either static or dynamic (65). Static radiomic features are time invariant and therefore characterise only spatial properties of the tumour. They are comprised of shape-based (morphological) and statistical features, which are further divided into first-, second-, and higher-order outputs. Morphological features describe the geometry of the lesion such as its compactness, sphericity, or surface to volume ratio (66, 67). First-order statistical features are derived from first-order histograms describing the distribution of voxel intensities within the specified tumour volume. Second-order statistical features, or what are often referred to as ‘texture features’, are among the most common descriptors used in radiomics predictive modelling. Texture features are able to characterise intensity spatial interrelationships between tumour voxels that first-order statistics fail to capture (65). Texture features can be extracted from a variety of defined matrices, such as the grey-level co-occurrence matrix (GLCM) (68), the grey-level run length matrix (GLRLM) (69), the grey-level size zone matrix (GLSZM) (70), the neighbourhood grey-tone difference matrix (NGTDM) (71), and the neighbourhood grey-level dependence matrix (NGLDM) (72), for use in predictive modelling. Higher-order statistical features involve the application of various mathematical transformations to the original image from which additional statistical features of first- and second-order can be extracted (21, 73), such as Laplacian of gaussian transformations, wavelet decompositions, and gabor filters for edge detection (74, 75). Dynamic features capture temporal information about the evolution of the disease over time that can provide more information than a simple snapshot of a lesion at a single time point. These features might unearth new biological characteristics of tumours that can be used in predictive modelling (76). There are, then, a very large array of possible features that can potentially be extracted from the ROI during a radiomics study. In practice, these features need not all be manually defined by the researcher, as the procedure of feature extraction can be performed by a number of commercial and open-source projects dedicated to the task.

Deep NN’s and CNN’s can be used to automatically learn high-level representations of input data such as medical images and generate machine learnt features. These techniques can be used to perform end-to-end predictive modelling tasks, encompassing automated hierarchical feature extraction and the utilisation of these features for the subsequent classification or regression task in a single step, or alternatively be used as standalone feature extractors (63). Deep CNN’s, for example, involve the repeated convolution of learnable filter grids across an input medical image whose values are tuned during the network training process to minimise a cost function such that the salient features relevant to the clinical task at hand are extracted. In this fashion, features are engineered automatically in a hierarchical way where simple characteristics are detected in the lower layers of the network, and increasingly abstract representations of the input image are learned as it progresses deeper through the network architecture (77, 78). There is a growing body of evidence demonstrating the usefulness of CNN-based features as a concomitant to traditional hand-crafted features in radiomics studies pertaining to various malignancies, such as soft tissue sarcoma (79), glioma (80), lung cancer (81), and also mPCa (82). Deep learning feature generators have greater versatility by not limiting themselves exclusively to manual human-defined features, however, this comes at the significant cost of reduced interpretability due to the black box nature of these algorithms. The benefits should therefore be weighed against the limitation of model interpretability, which is often desirable in the clinical context (83).

The dimensions of the extracted feature space may be large relative to the patient sample size, partly due to the nature of medical research where ethical considerations constrain access to patient data. Generating a predictive model with more explanatory variables than patient samples prevents generalisability of the model by over-fitting the sample on which the model was trained (83, 84). Reducing the dimensionality of the feature space improves the prediction capabilities of the final model, increases model interpretability, and shortens the training time. Selecting a subset of good features is therefore a crucial part of the radiomics workflow. Feature selection can be conducted according to the calculation of traditional statistical measures where features are eliminated through the application of thresholds, or, the dimensions of the feature space can be reduced by data-driven algorithms that project the data into lower dimensional spaces. Through the conduction of the feature selection process, quantitative imaging biomarkers should be chosen based on the possession of the following properties: repeatability, non-redundancy and reproducibility. Reproducibility has been stressed in the sections above, the main considerations being feature reproducibility with respect to segmentation and scanner variability.

Where possible, available test-retest data should be utilised to assess the repeatability of features across imaging scans taken under identical conditions (85). It is common to select arbitrary thresholds based on, for example, intraclass correlation coefficient (ICC) metrics calculated between the test-retest biomarker outputs, to exclude non-repeatable features (58, 86). This simple thresholding method is widely used, but its drawbacks should be noted. Firstly, comparisons of ICC values between different populations are invalid since the metric depends on the variance of the data, and thus also the underlying characteristics of the population under analysis (87, 88). Secondly, repeatability analysis alone is insufficient to determine the usefulness of a quantitative imaging biomarker. In particular, when it comes to response assessment, a feature with low repeatability may change drastically in response to therapy and could therefore be more informative as a predictive biomarker than a feature with high repeatability that changes only minimally during the same treatment. Therefore, it is not appropriate in all cases to remove non-repeatable features based on cut-off values of repeatability metrics. This point has been argued by Lin et al. (89), who posit a new metric for longitudinal assessment of predictive biomarkers termed the ‘response-to-repeatability’ ratio which weighs the biomarker sensitivity (measured as a change between baseline and follow-up scan) against its repeatability.

Redundant features that are highly correlated with each other are unlikely to provide any additional information useful for predictive modelling and can lead to model instability (90). Furthermore, their inclusion can increase the chances of overfitting and hamper model generalisability. Clusters of highly correlated features can be visualised and reduced to a single representative feature that is the most informative in the cluster (21, 87). Unsupervised data-driven algorithms that project the feature space into a lower dimension can also be used to select non-redundant feature subsets. Principal Component Analysis (PCA), which is a commonly employed method (28, 91, 92), is a linear dimensionality reduction technique that identifies successive axes that account for the largest variance in the data and projects the original feature space onto the hyperplane defined by those axes. The new, dimension reduced feature space is comprised of the principal components, each of which is defined by the projection of the original data onto one of the principal axes. The orthogonality of the principal components ensures that feature collinearity is minimised, which makes it an advantageous technique for the unsupervised removal of redundant features (28). Non-linear dimensionality reduction techniques such as local linear embedding (LLE) can also be used for the selection of salient features (84). It should be noted, however, that the projection of features into a lower dimensional space comes at the cost of reduced feature interpretability.

Having selected a subset of salient features, the final step in the radiomics workflow is the building of the predictive model. The development of classification models can be done using a variety of ML and deep learning techniques. It is impossible to know a priori which method will generate the best predictive model, and thus experimentation is advised along with comprehensive documentation of the techniques tested, hyperparameters used, and validation results. Scientific reproducibility of radiomics studies is a critical factor that can be facilitated by making implemented code available on platforms such as Github ¹ . A summary of some of the more prominent modelling techniques used, and example papers where they have been used in relation to mPCa, are provided in Table 1 .

Databases were searched using a logical search string [(“radiomics” OR “texture analysis” OR “radiological features” OR “radiomic features” OR “textural features” OR “texture features” OR “deep learning” OR “machine learning” OR “convolutional neural network” OR “CNN”) AND (metastatic OR metastases) AND (“prostate cancer” OR “prostate lesions”)] to identify potentially relevant papers published before the 23^rd of June 2021. Inclusion criteria were as follows: (1) human studies only, (2) analysis of medical imaging modalities only (radiomics analyses of histopathology slides were not included), (3) papers must either (i) extract a minimum of second order statistical features, or (ii) use deep learning techniques for feature extraction, (4) results must have diagnostic, prognostic, or predictive applicability to mPCa, (5) minimum of 10 sample size, and (6) full text of article must be available and accessible through our institution. A flow diagram illustrating the inclusion of identified studies is provided in the form of a PRISMA diagram (111) in Figure 2 .

The studies were partitioned into two sections following their inclusion into this review, namely, (i) Traditional radiomics, referring to papers that utilised traditional hand-crafted feature extraction techniques, and (ii) Deep radiomics, referring to papers that utilised deep learning networks for the extraction of deep features.

Identified papers utilising traditional hand-crafted feature extraction were subject to a RQS analysis. The RQS criteria is comprised of 16 defined items that correspond to critical points in the radiomics workflow. Each item in the RQS either awards or deducts points from a paper according to the study methodology. The RQS is designed to pinpoint methodological weaknesses in radiomics studies and encourage scientifically rigorous radiomics investigations (32). Table 2 provides a full description of these items and the points that can potentially be gained at each. A study may achieve a maximum of 36 points, or a minimum of -8 points. Each paper was assigned a percentage based on their score out of 36.

The use of the traditional radiomics paradigm of utilising hand-crafted features in mPCa is the predominant method of solving clinical challenges. Tables 3 , 4 summarise the salient characteristics for all identified traditional radiomics papers.

Utilising deep learning for automated end-to-end salient feature learning, extraction and modelling (which we may term ‘Deep Radiomics’) instead of traditional hand-crafted features is an area of study that has considerable potential. The rise of the field has been facilitated by an overall increase in the availability of computational resources and toolkits ^{3, 4, 5} that have made the designing and training of these networks easier than ever before. The potential of these models to become crucial pieces of the clinical workflow, supplementing physician decision making and contributing to individualised patient therapy, is enormous. In the deep radiomics publications identified in this review, the detection or classification of patient metastatic lesions is the overwhelming outcome measure.

With respect to imaging modalities, bone scintigraphy was the modality that was most commonly analysed (100, 125–131), and the salient characteristics of these papers are summarised in Table 5 . Papandrianos and colleagues (128) designed a CNN architecture for bone metastases diagnosis, where patient bone scintigrams were classified into three classes: no metastasis, degenerative (defined as the absence of metastasis but the presence of degenerative lesions), and metastasis present. Of the 778 patient bone scans examined 15% were reserved solely for testing and following a thorough exploration of the optimal CNN hyperparameters to be used, the final architecture achieved overall classification accuracy of 91.61% ± 2.46% (F1-score = 0.938). This result concords with a very similar study undertaken by the same authors, except in this instance only performing a dual-class classification problem (metastasis present vs. metastasis absent) by excluding any patients with degenerative lesions where their CNN model achieved a higher overall accuracy of 97.38% (129). Ntakolia et al. (127) performed the same three-class classification problem mentioned above also on 778 PCa patients who underwent bone scintigraphy, except this time deploying a lightweight version of the look-behind FCN (LB-FCN) (132, 133) and achieved a better overall accuracy of 97.41%. Their results demonstrated that state-of-the-art classification results can be achieved using a CNN with less learnable parameters and thus requiring less resources for training.

Deep learning techniques can also be utilised to classify individual identified lesions, rather than whole patient images, in bone scintigrams. Cheng et al. (100) used data from 371 breast cancer patients to pre-train a YOLO v4 network (134) to classify identified chest hotspots in bone scintigraphy images as either metastatic or benign. Employing a transfer learning approach, the model was then trained on a dataset of 194 PCa patients under a 10-fold cross-validation scheme which yielded a lesion-level classification sensitivity of 0.72 ± 0.04 and a precision of 0.9 ± 0.09. Their approach suggests the feasibility of utilising metastatic lesions in malignancies other than prostate cancer for pre-training a classification network, thus potentially offering a way for researchers to generate functioning networks even with limited datasets. A follow-up study (126) using instead the YOLO v3 network (135) managed to increase the sensitivity (0.82 ± 0.08) at the expense of precision (0.7 ± 0.11) for chest hotspot classification.

The remaining identified studies investigated deep learning on PET and/or CT images with various labelled radiotracers (99, 136–139). Table 6 shows the important characteristics of these studies. Lee et al. examined a cohort of 251 PCa patients with suspected BCR following definitive primary therapy (139). ¹⁸F-Fluciclovine PET images were labelled as either ‘normal’, if no recurrence was found, or ‘abnormal’, if there was either a recurrence in the prostatic bed or evidence of pelvic metastasis. Two 2D CNN’s (ResNet-50) were trained (a slice-based model which predicts individual patient PET slices, and a case-based model that makes a prediction for the entire PET image), and one case-based 3D CNN (ResNet-14) was trained to predict image abnormality. Prediction results on a dedicated test set at the patient level (AUC = 0.75, p = 0.013; Sens = 85.7%; Spec = 71.4%) were outperformed by the 2D slice-based CNN (AUC = 0.971, p < 0.001; Sens = 90.7%; Spec = 95.1%), however, both 2D models outperformed the 3D ResNet-14 (AUC = 0.699, p = 0.053; Sens = 71.4%; Spec = 71.4%). The authors hypothesise that the reason for the underperformance of the 3D CNN could be due to the fact that the 3D CNN had a higher number of learnable parameters and would therefore require a greater training dataset size to generate a sufficiently generalisable model.

One can take this analysis further by classifying individual lesions as benign or malignant. Masoudi et al. (138) demonstrated how deep learning can be used to classify the malignancy of individual bone lesions using only a patient staging CT. An expert radiologist identified, annotated, and segmented 2,880 bone lesions in 114 PCa patients, 41 of which were histopathologically confirmed to be metastatic. Fifteen patients were reserved solely for testing purposes. Using a constructed ensemble model comprised of the 2D ResNet-50 and a 3D ResNet-18 architecture, they achieved high accuracy in classifying bone lesions in the test set (Acc = 92.2%). The use of CT to classify the malignancy of patient lesions is not limited exclusively to bone lesions. Hartenstein et al. (137) trained a CNN for the binary classification of lymph nodes in contrast-enhanced CT images of 549 histologically confirmed PCa patients (2,616 labelled lymph nodes identified on ⁶⁸Ga-PSMA PET) and achieved a great accuracy of 89% (AUC = 0.95; Sens = 86%; Spec = 92%). External validation of these results was lacking, however, and the authors acknowledge that future studies should utilise histopathology confirmed lymph node invasion via extended pelvic lymph node dissection, rather than ⁶⁸Ga-PSMA PET, as the ground truth reference.

Deep learning can also be applied to perform fully automated detection of metastatic PCa lesions. Automated detection of metastases in patients can relieve some of the significant clinical time burden associated with manual observer analysis of medical images. Zhao et al. (99) conducted a proof-of-concept study for the automated detection and segmentation of metastatic lesions in ⁶⁸Ga-PSMA PET/CT images of mCRPC patients. The authors applied a 2.5D U-Net ensemble network that leveraged information from each anatomical plane separately to make predictions on the presence or absence of metastases in individual voxels, with manual delineations by expert physicians serving as the reference ground truth. Their results, which were restricted only to detecting metastatic disease in the pelvic area, demonstrated great accuracy in the detection of bone lesions (Sens = 0.99, Prec = 0.99) and lymph node lesions (Sens = 0.9, Prec = 0.94), but inferior accuracy in the detection of local prostate lesions (Sens = 0.61, Prec = 0.79). Although highly promising, extending this analysis to detect metastases throughout the body that aren’t restricted to the pelvic area remains a fertile pathway of future research. On top of the clear clinical benefit of having lesions detected automatically, there is potentially valuable prognostic information to be gained from the number of automatically detected lesions. Borrelli et al. (136) trained and tested two CNN’s, one that detected important patient organs, and another that used this output in conjunction with the patient ¹⁸F-Choline PET/CT image to automatically detect lymph node lesions in 399 biopsy-proven PCa patients. They determined that the number of automatically detected lymph node metastases was significantly associated with PCa-specific survival (HR = 1.19, 95% CI 1.05 – 1.33).

The RQS assessment of the identified papers reveals some significant limitations in the current landscape of mPCa radiomics research. The overall lack of prospective methodology and external validation of developed models, along with the other limitations presented above, are crucial contributing factors that are impeding the translation of radiomics models from being of purely academic interest to being properly realised clinical models capable of facilitating truly personalised patient interventions based on the specific phenotype of their malignancy. These downsides should be addressed as soon as possible so that the full potential of radiomics can be realised.

Having said this, it should also be noted that the RQS as a measure of methodological quality has its own limitations. For example, it takes insufficient account of the nature of the study and penalises papers perhaps too harshly in some respects, while not penalising sufficiently other aspects such as significant overfitting. For traditional radiomics papers whose purpose is not to develop a radiomics model, for example, Lin et al. (89) and their investigation into a potential new metric for response assessment, overfitting is not a consideration and thus feature reduction is not necessary. Similarly for studies using machine-learnt features for extraction and modelling, some aspects of the RQS might not be applicable. Employing feature cut-off analyses, for example, is unnecessary for a variety of ML models that are commonly used in radiomics research and, as has been pointed out elsewhere (36), might even compromise the interpretability of the final ML model. Furthermore, once you move to the realm of deep learning for feature extraction, where the black box nature of the algorithm makes the features that are extracted extremely difficult (if not impossible) to interpret, other parts of the RQS criteria become hard to meet. It is difficult to expect researchers to analyse the robustness of individual features to scanner variabilities when the features are deep features that are not precisely mathematically defined and very difficult to extract individually from a highly complex deep NN or CNN. Assessing individual feature robustness to segmentation variabilities with deep features is similarly impractical but could also be irrelevant. CNN’s, for example, can perform end-to-end feature extraction and predictive modelling on entire medical images without the need for segmentations in the first instance, which should be taken as a significant benefit since it removes what is known to be a large source of bias in radiomics research (32, 53). It would perhaps make more sense in these cases to assess robustness at the model level, rather than the individual feature level, but this is not what the RQS criteria specifies.

Considering these factors, we did not think it appropriate to perform RQS assessments on papers that employed deep learning from end-to-end, since it would make for an unfair comparison to those papers that utilised traditional hand-crafted features. However, even without conducting a RQS assessment of the identified deep radiomics studies, several clear methodological weaknesses exist among the reviewed studies. External validation was poor, with only a single study (131) conducting a validation of the model performance on an external test set. None of the papers were prospective in nature. The implicit bias present in retrospective studies, and the near complete lack of external validation of developed models is a large hurdle to the translation of these models into actual clinical practice. Whether using traditional mathematically defined features, or automatically learnt deep features, if radiomics is to achieve its full potential, then these downsides will need to be addressed in future research.

Model validation is a critical point that needs to be underscored. New predictive, prognostic or diagnostic models need to be validated in some fashion and performing this validation exclusively on the same data on which the model was trained will provide an inflated assessment of model performance. Internal validation should be performed as a necessary first step, where cross-validation techniques, bootstrapping, or a hold-out test set from the original study sample are used to assess model performance. Models that perform poorly on an internal validation sample are unlikely to generalise well to previously unseen populations (140). Internal validation alone, however, is insufficient. Radiomics studies should undertake, as a minimum, internal validation, but ideally external validation should also be conducted where possible (32, 33). If the purpose is to find robust, informative biomarkers and models that can be utilised in a cross-institutional fashion for diagnostic or prognostic purposes then it is imperative that external validation is also undertaken. External validation of a developed model or an identified biomarker, in which the predictive performance is assessed on a separate data sample, is necessary to understand their capacity to generalise to data other than that on which they were trained or acquired. The published literature reviewed in the present work demonstrate a poor performance when it comes to external validation, which is directly in line with a myriad of other studies that have highlighted the overall lack of external validation that is present in radiomics research pertaining to other malignancies (34, 36, 37). This is an issue that directly impacts the translational capacity of developed models into proper clinical practice, and future studies should seek to address this limitation.

The present review has revealed a relative paucity of studies that conducted a feature reproducibility analysis with respect to different segmentation techniques or scanner variabilities. Retrospective or multi-institutional datasets often do not have standardised imaging acquisition and reconstruction protocols. While efforts should be made to obtain a dataset of images with consistent or identical scanner parameters, this is not always feasible. Harmonisation techniques have been employed and validated that limit the influence of heterogeneous imaging parameters on the resulting radiomics signature, which ensures that any variability in the developed model is reflective of the underlying physiology/biology of the imaged lesion instead of being biased by inconsistent scanner protocols. ComBat is a commonly used harmonisation technique that employs an empirical Bayes framework to alter datasets to account for so-called ‘batch effects’, which refer to confounding experimental factors that affect the output of data other than the underlying biological variations that are of clinical interest (141). The method was originally introduced to reduce the influence of batch effects in genomic microarray research, but its applicability is not limited to this field and has since been utilised in radiomics research. Imaging acquisition protocols and reconstruction methods are batch effects in radiomics research that can be minimised by utilising the ComBat method, and evidence exists supporting its usefulness as a harmonisation technique for imaging modalities such as PET (142), CT (60, 143, 144), and MRI (145).

While the current evidence points to the usefulness of the ComBat harmonisation method, phantom studies confirming that the method improves feature reproducibility across scanner parameters in specific use cases should be performed where feasible. Indeed, it should be noted that there is no guarantee that the use of ComBat harmonisation, even if the method increases feature reproducibility across variable scanner parameters, will lead to a model with increased diagnostic, prognostic, or predictive power. This is demonstrated in a study by Peeken et al. (116) where the use of ComBat harmonisation resulted in an insignificant AUC value change of 0.01 in logistic regression models used to determine the malignancy status of segmented lymph nodes. Thus, while ComBat harmonisation can have a positive effect on the reproducibility of radiomics features, whether this leads to an improved model will depend on the particular modelling task and should be the subject of future research. Current mPCa radiomics research has experimented minimally with this technique, and future radiomics studies relating to mPCa should explore this technique further.

Studies reviewed in the present work utilised a myriad of different radiomics software for feature extraction. This is problematic for reproducibility, since it is known that even when extracting the same imaging biomarker, different software can yield different results (146). The standardisation of quantitative imaging biomarker definitions is therefore important to ensure maximum reproducibility. Standardisation initiatives such as the image biomarker standardisation initiative (IBSI) have attempted to address the problem of variable biomarker definitions, producing a set of standardised definitions that researchers can use for their quantitative imaging tasks (54, 147). Open source radiomics feature extraction software such as PyRadiomics and RaCaT (148, 149), implemented in the popular coding languages Python and C++, define biomarkers largely in accordance with the IBSI guidelines and carefully document any deviations, which can improve study reproducibility. Not specifying the software used or utilising in-house software that is not provided open source (and is therefore impossible to verify) should be minimised in favour of open-source projects such as those above in future works.

The reliance on traditional hand-crafted features to characterise ROIs is a limiting factor in radiomics research that can be supplemented by using deep learning methods. Automated feature generation with saliency to the predictive task at hand is a notable benefit that can streamline the radiomics workflow, reducing the need for manual or data-driven feature selection techniques and enable end-to-end predictive modelling with limited human intervention required. The deep radiomics papers reviewed in this work demonstrate the ability of these algorithms to achieve good results in classifying the malignancy of overall patient scans (127, 139), classifying individual bone lesions (126), or detecting the anatomical locations of metastatic lesions throughout the patient body (99). As already discussed, external validation is overwhelmingly lacking and needs to be addressed, however, the literature available to date suggest the significant potential for deep learning to contribute positively to the clinical management of mPCa. It should be noted, however, that the overwhelming majority of deep radiomics papers relating to mPCa developed models with diagnostic applicability. Only a single study attempted to perform any analysis that had prognostic value (136), where they associated the number of automatically detected lymph nodes with PCa-specific survival (HR = 1.19, 95% CI 1.05 – 1.33). The heterogeneity of mPCa and plethora of available treatment options such as hormone therapy, chemotherapy, PSMA-labelled isotope therapy and others make this disease a prime target for deep modelling capable of predicting optimal treatment regimens (150). This remains a relatively unexplored pathway, and future work should certainly delve into this rich area.

Deep learning does not need to be utilised for end-to-end feature extraction and modelling to aid substantively with specific aspects of the radiomics workflow. ROI segmentation is one area where deep learning has already made significant contributions. The recent explosion of fully convolutional networks (FCNs) has led to the development of numerous algorithms capable of performing fully automated segmentation of anatomical structures and patient lesions. FCNs such as the U-Net and its variants have revolutionised the field of medical image segmentation through their ability to output state-of-the-art delineations in seconds (151–153). Kostyszyn et al. (154), for example, have demonstrated the possibility of fully automated prostatic gross tumour volume segmentation on PSMA-PET images in patients with primary PCa. They found good concordance between the fully automated segmentation and the expert manual contour on an external validation cohort, achieving a median dice similarity coefficient (DSC) of 0.81 (range: 0.32-0.95). Other studies have demonstrated the possibility of fully automated and accurate segmentation of the prostate gland and its associated zones with deep learning across various imaging modalities such as CT (155–157) and MRI (158–160). Zhao et al. (99) developed a modified 2.5D U-Net architecture for the automated segmentation of metastatic prostate lesions in ⁶⁸Ga-PSMA PET/CT images in a proof-of-concept study. Their ensemble model, which leveraged information extracted from the axial, sagittal and coronal imaging planes simultaneously, was able to achieve mean DSCs of 0.645 and 0.544 for bone lesions and lymph node lesions when compared to expert manual segmentations, respectively, although the analysis was restricted to metastatic lesions contained in the pelvic area and fully-body validation has yet to be undertaken. By utilising fully automated segmentation algorithms reproducible segmentations can be produced and the issue of inter- and intra-observer variability is resolved, however, further research in this space is necessary because the lack of generalisability of these algorithms to independent datasets remains an impediment to their widespread clinical adoption (87).

Deep learning techniques can yield great benefits, but they are not without their downsides. In all cases the interpretability of the model is compromised relative to the traditional radiomics pathway. While hand-crafted features can often relate to very specific and clinically understandable aspects of tumour biology, deep features are highly abstract and difficult for humans to interpret. A generalised statement on whether this is an acceptable downside cannot be made, as it depends primarily on what the desired outcome of the clinical model is, and whether clinicians value having a greater intuitive understanding of the results of a particular model. Also, the complexity of deep learning models demands greater amounts of training data to produce acceptable results without overfitting. Techniques exist to mitigate against overfitting, such as dropout regularization (161), batch normalization (162) and artificially increasing the size of the dataset through data augmentation. Nevertheless it remains true that the vast majority of current models do not demonstrate generalisability to external datasets, which is evidenced by the studies reviewed in the present work. These downsides need to be considered when future radiomics studies in mPCa are undertaken.

There is no inherent limitation on the number of imaging modalities from which radiomics features can be extracted for use in predictive modelling. Hybrid imaging techniques capture expanded amounts of information that can be complementary in nature, where each modality characterises different information about the underlying biology of the tumour (163). ⁶⁸Ga-PSMA PET/CT imaging, for example, plays a crucial role in the detection and subsequent clinical management of metastatic PCa (11). ⁶⁸Ga-PSMA PET captures physiological information within the patient’s body about the distribution of the PSMA receptor, which is substantially overexpressed in the vast majority of PCa cases (17), while the CT provides high-resolution imaging reflecting the underlying density of the patient anatomy. Radiomics features extracted from each of these modalities will thus characterise the heterogeneity of the tumour in different and potentially complementary ways, which could improve model performance. There is evidence that utilising this approach in mPCa radiomics modelling can yield good results, both in the traditional radiomics methodology (93, 95, 98) and deep radiomics (99). Particularly in the field of deep radiomics, where the analysis of dual modalities can often be as simple as incorporating an additional channel in the network architecture, this method of analysis should be thoroughly explored.