Peripheral blood samples were obtained from asymptomatic men suspected of having PCa that attended the Urology Clinic at Leicester General Hospital (Leicester, UK) between 24 October 2012 and 15 August 2014. Samples were obtained from two cohorts of patients, termed the ‘TPTP’ and ‘TRUS’ cohorts (see below for more details). For both cohorts, patients were recruited and treated as described previously (10).

Phenotypic data were generated from a total of 130 males (42 diagnosed with benign disease and 88 diagnosed with cancer, as confirmed by TPTP or TRUS biopsy evidence) ( Tables 1 , 2 ). Of the 42 subjects diagnosed with benign disease; 2 were diagnosed with Atypical Small Acinar Proliferation (ASAP). 11 with Atypia, 13 with High Grade Prostatic Intraepithelial Neoplasia (PIN) and 16 with benign disease. Of the men diagnosed with PCa, 18 had low-risk, 44 had intermediate-risk, and 25 had high-risk cancer based on their D’Amico Risk Classification for Prostate Cancer (13). The D’Amico Risk for one patient was not available as no Gleason score values were provided. Further details regarding the TRUS (14) and TPTP (9, 10) biopsy techniques have been provided elsewhere.

Some of the data used in the present study have been previously published (11). These data were derived from 72 males having PSA levels < 20 ng/ml who had a TRUS and then a TPTP biopsy (11). The mean age for this cohort was 66 years old (age range of 50–84 years old).

Peripheral blood was collected from all individuals using standard clinical procedures, aliquots of which (30 ml) were transferred into sterile 50 ml polypropylene (Falcon) tubes containing 300 µl sterilised Lithium Heparin (1000 U/ml, Merck Millipore). Anti-coagulated samples were immediately transferred to the John van Geest Cancer Research Centre at Nottingham Trent University (Nottingham, UK) and processed immediately upon receipt (always within 3 hours of collection).

Absolute cell counts in whole blood samples were determined by the inclusion of BD Trucount™ beads (BD Biosciences; Mountain View, CA, USA), as per the manufacturer’s protocol. For the flow cytometric analysis, 100 μl of blood was mixed directly in the BD Trucount™ bead tube and T cell, B cell, and NK cell populations identified using the conjugated monoclonal antibodies (mAbs) detailed in Table 3 . Samples were incubated for 15 min at room temperature, protected from the light, after which erythrocytes were lysed by incubating samples for 15 min at room temperature in BD Pharm Lyse™ (BD Biosciences). Once staining was complete, cells were washed in phosphate buffered saline (PBS), resuspended in Coulter Isoton™ diluent. Data were acquired within 1 h using a 10-color/3-laser Beckman Coulter Gallios™ flow cytometer and analyzed using Kaluza™ v1.3 data acquisition and analysis software (Beckman Coulter). Controls used a “Fluorescence minus One”, “FMO” approach (15). A typical gating strategy for the analyses is presented in Figure 1 .

The bidirectional Long Short-Term Memory Deep Neural Network (biLSTM) is also known as a bidirectional Recurrent Neural Network (RNN). LSTM is an artificial recurrent neural network architecture used in the field of deep learning. Unlike the standard feedforward neural network, the LSTM has feedback connections which enable it to process entire sequences of data. A biLSTM is a type of LSTM with a bidirectional layer and learns bidirectional long-term dependencies in sequence data. The architecture of the proposed biLSTM for detecting the presence of PCa is shown in Table 4 . Although the biLSTM is widely applied to sequential data, it has been, and can also be successfully applied to non-sequential data.

A biLSTM model learns the input sequence both forward and backwards and concatenates both interpretations. The model duplicates the first recurrent layer in the network and creates two side-by-side layers, then provides the input sequence ‘as-is’ as input to the first layer and providing a reversed copy of the input sequence to the second (16). The training data are shuffled before each training epoch, and the validation data are shuffled before each network validation. Given that the mini-batch size does not evenly divide the number of training samples, the network discards the training data that do not fit into the final complete mini-batch of each epoch. Shuffling the data as mentioned above avoids discarding the same data at every epoch.

Two biLSTM models were implemented. The first biLSTM model takes as input immunophenotypic features and clinical data and is trained to detect the presence of PCa. The second model takes as input a set of biomarkers comprising immunophenotypic features and clinical data and is trained to predict the clinical risk the PCa when PCA has been identified as being present.

The models were built using combinations of phenotypic features and clinical data to determine the best combinations for training each model. Figure 2 shows how prediction models for detecting the presence of PCA and its clinical risk can be utilised to assist clinical diagnosis.

The dataset was initially split into datasets derived from men with benign prostate disease and patients with PCa, and each of these datasets was randomly split into ‘train’, ‘validation’ and ‘test’ datasets with a split ratio of 60:20:20, respectively. This random split process was repeated 30 times to create 30 different train, validation and testing sets. This allowed for exhaustive evaluations to be carried out using different sub-populations of the dataset for train, validation, and test purposes. The biLSTM Deep Neural Network models utilised the train sets for training, and the validation sets were utilised during the training process to improve the models’ learning. The test sets are unseen during training, and therefore the test results can be considered to represent mini clinical trials. The results at the end of the 30 runs were collected and analyzed. The methodology for evaluating the Deep Learning models is illustrated in Figure 3 .

A set of relevant metrics were adopted for evaluating the performance of the proposed biLSTM models, These were built using six different ‘feature’ sets: FC; PSA; FC+PSA; FC+Age; FC+Age+PSA; Age+PSA. ‘FC’ stands for flow cytometry features and comprises five phenotypic features CD8⁺CD45RA^-CD27^-CD28^- (CD8⁺ Effector Memory cells), CD4⁺CD45RA^-CD27^-CD28- (CD4⁺ Effector Memory cells), CD4⁺CD45RA⁺CD27^-CD28^- (CD4⁺ Terminally Differentiated Effector Memory Cells re-expressing CD45RA), CD3^-CD19⁺ (B cells), CD3⁺CD56⁺CD8⁺CD4⁺ (NKT cells), as identified previously as being able to discriminate between benign prostate disease and PCa (11).

Let model PCaPresence be a model for detecting the presence of PCa, and PCaRisk be a model for predicting whether a patient with PCa has D’Amico high-risk (H-risk) or low/intermediate risk (LI-risk) disease.

Sensitivity is also known as the True Positive Rate (TRP).

Specificity is also known as the True Negative Rate (TNR).

FPR stands for False Positive Rate.

The closer the values of Accuracy, Sensitivity (i.e. TPR, Sensitivity) and Specificity (i.e. TNR, Specificity) are to 100%, then the better the performance of a model.

The Receiver Operating Characteristic (ROC) evaluates the quality of a prediction model’s performance. The ROC curve has an optimal ROC point which comprises two values: the FPR and the TPR values. The optimal ROC point is computed by function (6) for finding the slope, S.

where, in a PCaPresence detection model let the positive class be the class containing patients with PCa, and the negative class be the class containing men having benign prostate disease. In the PCaRisk prediction model let the positive class be the H-risk group and the negative class be class containing the records of the patients belonging to the low and intermediate class (LI-risk). (N|P) is the cost of misclassifying a positive class as a negative class; and Cost (P|N) is the cost of misclassifying a negative class, as a positive class.

The optimal ROC point is identified by moving the straight line with slope S from the upper left corner of the ROC plot (FPR=0%, TPR=100%) down and to the right until it intersects the ROC curve. The Area Under the ROC Curve (AUC) is another important performance evaluation metric which reflects the capacity of a model to discriminate between the data obtained from individuals with benign prostate disease and patients with PCa. The larger the AUC, the better the overall capacity of the classification system to correctly distinguish between benign disease and PCa.

The dataset comprised 7 features, 5 of which were peripheral blood flow cytometric T and B cell phenotyping features identified in our previous study (11) and the remaining two of which were the clinical features PSA level and Age. The five phenotypic features were: CD8⁺CD45RA^-CD27^-CD28^- (CD8⁺ Effector Memory cells), CD4⁺CD45RA^-CD27^-CD28^- (CD4⁺ Effector Memory cells), CD4⁺CD45RA⁺CD27^-CD28^- (CD4⁺ Terminally Differentiated Effector Memory Cells re-expressing CD45RA), CD3^-CD19⁺ (B cells), CD3⁺CD56⁺CD8⁺CD4⁺ (NKT cells). The data for each immune phenotyping feature were standardized using z-score transformation. The standardized z-scores are scores (or data values) that have been given a common standard. This standard is a mean of zero and a standard deviation of 1. The PSA and Age values were not standardized ( Table 5 ).

The Kolmogorov-Smirnov and Shapiro-Wilk statistical tests demonstrated that the data are not normally distributed and non-parametric tests were therefore used for the analyses ( Table 6 ).

The Mann-Whitney U test revealed that there are no significant differences (p<0.05) between the groups for the flow cytometry features, but that the age and PSA levels in men with benign disease and those with PCa were different (p<0.05, Table 7 ). Given that there are 7 comparisons, the Bonferroni correction was applied and the α value was set to α= 0.007 to reduce Type I error. Using the adjusted α value revealed that there were no significant differences between the values of the features of the benign and PCa groups.

Table 7 also reports the effect size which is the magnitude of the difference between groups, and it is computed using r=Z√N, where Z is the output of the Mann-Whitney U Test, and N is the total number of samples. According to Cohen (17), the effect size is low if the value of r varies around 0.1, medium if r varies around 0.3, and large if r varies more than 0.5. This means that if the values of two groups do not differ by 0.2 standard deviations or more, then the difference is trivial, even if it is statistically significant. Hence, it can be concluded that there are no statistically significant differences between the features indicated in Table 7 in the benign prostate disease and PCa groups, and that any differences that do exist are small and trivial.

The nonparametric Spearman’s rank-order correlation shows there to be no strong positive or strong negative correlations amongst the outputs which will be utilised to build the machine learning classifier ( Figure 4 ).

TPTP is significantly better at diagnosing PCa than TRUS biopsies in biopsy naïve men with an elevated PSA <20 ng/ml and a benign feeling prostate (10). Nafie et al. have therefore proposed that TPTP should be regarded as the biopsy technique of choice in such cases (10).

The Kruskal-Wallis H test is an extension of the Mann-Whitney U test, is the nonparametric equivalent of the one-way analysis of variance and detects differences in distribution location. The major difference between the Mann-Whitney U and the Kruskal-Wallis H is simply that the latter can accommodate more than two groups. Both tests require independent (between-subjects) designs and use summed rank scores to determine the results. Therefore, for the analysis in this subsection the Kruskal-Wallis H test was suitable. The Kruskal-Wallis H (also known as the ‘one-way ANOVA on ranks’) rank-based nonparametric test, was used to determine whether there are any statistically significant differences between the immunophenotypic profiles of the patients when grouped based on biopsy methods and diagnosis. Therefore, a new variable was created, BiopsyDiagnosis, where the biopsy type (i.e. TPTP or TRUS) and diagnosis (i.e. benign prostate disease or PCa) were merged into a four separate labels: TPTPBenign, TRUSBenign, TPTPCancer, TRUSCancer, and the Kruskal-Wallis test was applied to determine significant difference between the TPTPBenign and TRUSBenign groups, and between the TPTPCancer and TRUSCancer patient groups. Table 8 shows the characteristics of each group of subjects and Table 9 the results of the Kruskal-Wallis H test which was applied to determine differences between the ranks of the abovementioned groups.

The α level for these tests was set to 0.005, however applying a Bonferroni correction which was applied to reduce the chance of a false positive (i.e. a Type I error) reduced the α value to 0.05 since there exist 10 possible comparisons. As shown in Table 9 , the absence of any significant differences (Asymp. Sig) between any of the immunophenotyping features of the TPTPBenign and TRUSBenign patients is a good indicator that data collected during TPTP and TRUS biopsy can be combined when training a machine learning model.

The performance of various biLSTM Deep Neural Network models (whose architecture is described above) for predicting the presence of PCa when using six different subsets of features was assessed. Table 10 shows the training, validation, and test results of the models. Table 10 shows that the FC+Age was the best model, achieving an accuracy of 86.92% on the validation set, and 86.79% on the test set. More specifically, the model was able to detect the presence of PCa in the validation set with Acc: 86.92% (± 0.10), Sensitivity: 83.70% (± 0.16); Specificity: 94.17% (± 0.11); AUC: 88.94% (± 0.07), ORP-FPR: 9.17% (± 0.20), ORP-TPR: 85.74% (± 0.14) ( Table 10 ). Results from the test set (set not used during training or validation) were Acc: 86.79% (± 0.10), Sensitivity: 82.78% (± 0.15); Specificity: 95.83% (± 0.11); AUC: 89.31% (± 0.07), ORP-FPR: 7.50% (± 0.20), ORP-TPR: 84.44% (± 0.14).

The validation results for predicting the presence of PCa using PSA revealed a 27.91% lower ORP-FPR when combining FC+PSA than when using PSA alone. For the test results, the ORP-FPR was 20.83% lower when combining FC+PSA than when using PSA alone. The standard deviation values of the FC+PSA were lower indicating a more stable model.

Combining Age with immunophenotypic features improved prediction accuracy and therefore age appears to be a good predictor for the presence of PCa when combined with the flow cytometry phenotypic features. As the correlation chart in Figure 4 shows there to be no strong positive or strong negative correlations between age and the rest of the features including diagnosis, we can rule out the fact that correlation is biasing the models’ predictions (i.e. since there are no strong positive or strong negative correlations between Age and the presence of PCa). However, a further statistical investigation was used to conclude whether age is biasing the performance of the prediction models. The two-sample Kolmogorov-Smirnov test, a nonparametric hypothesis test was applied for testing if the variable age has identical distributions in the two populations (i.e. the benign prostate disease and PCa groups). **Note that this is different to the results shown in Table 6 which checks whether the variables are normally distributed, and not whether the two groups follow the same distributions. Figure 5 shows the distribution of Age values across the benign prostate disease and PCa groups.**

The α value was set to 0.01 to minimize Type I error. The test returned p=0.033, Z=1.431 (and p>0.01) meaning that samples from the benign prostate disease and PCa groups are from the same continuous distribution at the 1% significance level. The next step was to determine whether there are any significant differences in the mean age ranks of these two groups that could be biasing the prediction. The α value was again set to 0.01 to prevent Type I errors and make it harder to predict significant differences. As the Mann-Whitney test revealed p=0.012, Z=-2.515 we can assume that there are no significant differences in the mean ranks of age at the 1% significance level.

However, to further ensure the correct conclusions are reached, the Moses Test of extreme reaction was carried out to recompute the differences between groups when the extreme outliers are not considered. The test is a distribution-free non-parametric test of the difference between two independent groups in the extremity scores (in both directions) that the groups contain. For the benign prostate disease and PCa groups, Moses tests whether extreme values are equally likely in both populations, or if they are more likely to occur in the population from which the sample with the larger range was drawn. The scores from the benign prostate disease and PCa groups are pooled and converted to ranks, and the test statistic is the span of scores (computed as the range plus 1) in one of the groups chosen arbitrarily. An exact probability is computed for the span and then recomputed after dropping a specified number of extreme scores from each end of its range. The exact one-tailed probability is calculated. After trimming the entire dataset, there were 5 patients ≤54 years old and 6 patients ≥84 years old. The information for the Moses extreme reaction test shows that the benign prostate disease and PCa groups have different age values with a Sig. = 0.006. However, when removing the extreme outliers, the Sig. value increases to Sig. = 0.082 hence the two groups have similar age values when the extreme outliers are removed.

Revisiting the results in Table 7 , and based on the observations described in this section, we can conclude that the algorithms are not biased towards age, and that age in combination with the immunophenotypic features forms a good predictor for the presence of PCa. It is important to mention that if age was biasing the output of the prediction model, then other machine learning models (FC+Age+PSA, Age+PSA) would have consistently delivered high prediction results, because machine learning models excel at detecting patterns in data and would have found the association (pattern) between the Age and the output variable (diagnosis) if this had existed. Based on these observations, it can be concluded that age is not biasing the output of the prediction model.

Men diagnosed with low-risk or small volume intermediate-risk PCa will very rarely require treatment compared to men who have been diagnosed with high-risk PCa. It is therefore important to detect men in the H-risk group accurately to prioritise treatment for those men, and to prevent unnecessary invasive procedures.

Consequently, we determined whether biLSTM models can differentiate between the clinical risk of PCa using the same features as those which have been used for building the models for predicting the presence of PCa ( Table 11 ). Given that there are 85 patients having low-risk (n=18), intermediate-risk (n=44) or high-risk (n=25) PCa, patients were grouped into L/I (low-intermediate) and H (high) risk groups. The biLSTM model that was designed for predicting risk was then utilised to predict risk (L/I or H). The test results in Table 11 show that the model which combined the flow cytometry features with PSA was able to predict clinical risk in the validation set with Acc: 94.51% (± 6.35), Sensitivity: 92% (± 21.09); Specificity: 95.56% (± 2.99); AUC: 93.78% (± 10.50), ORP-FPR: 4.44% (± 2.99), ORP-TPR: 92% (± 21.09). The results on the test set with Acc: 94.90% (± 6.29), Sensitivity: 92% (± 21.39); Specificity: 96.11% (± 0.00); AUC: 94.06% (± 10.69), ORP-FPR: 3.89% (± 0.00), ORP-TPR: 92% (± 21.39). These are a positive indicator, and it is expected that with a larger dataset the model will be able to learn better, and the standard deviation values will reduce. Comparing the results of the FC+PSA model with those which uses PSA values alone, the FCA+PSA model returned better validation and test results. It therefore appears that PSA is a good predictor of clinical risk when combined with FC values.

Experimental results described above using models for detecting the presence of PCa found that Age is a feature which, when combined with the immunophenotypic profiling, delivers a greater predictive accuracy than when it is used alone. Here, we follow a similar analysis for interrogating the impact of PSA. As indicated in the correlation chart ( Figure 4 ), there are no strong positive or strong negative correlations between PSA and the other features.

The two-sample Kolmogorov-Smirnov test was applied to check for identical distributions in the two populations (i.e. benign prostate disease and PCa). The test returned p=0.033, Z=1.431 (and p>0.01) meaning that samples from the benign prostate disease and the PCa groups are of the same continuous distribution at the 1% significance level. Next, we determined whether significant differences in the mean ranks of the benign prostate disease and PCa groups could be biasing the prediction. The alpha value was set to 0.01 to prevent Type I errors and make it harder to predict significant differences. The Mann-Whitney test returned p=0.007, Z=-2.717. Therefore, it can be assumed that there are significant differences in the mean ranks of age at the 1% significance level. These results show that the PSA could be influencing the risk of disease, which makes clinical sense given that the high-risk patients often (but not always) have higher PSA values than the low and intermediate-risk patients ( Table 11 ).