Models that are inherently explainable and interpretable. These are often simpler methods such as linear or logistic regression, the latter of which can be represented as a nomogram (14), a graphical representation of such models that visually convey the weight of different input variables. These models can be fully dissected, so there may be many ways of representing or explaining them.

Models that can not be interpreted easily, and are regarded as a “black box” out of which little or no knowledge can be derived. However, there are techniques that can provide explainability when working with black-box models, such as making use of Shapley values (15, 16) or making use of Deep Learning specific techniques (17) such as Layer-wise Relevance Propagation (18, 19). These are often post-hoc. In practice, these techniques will show a number of features and their importances expressed as a number. This could also be in the form of a heatmap. These feature importances may not always be as readily interpretable and may need training and education to comprehend adequately. Additionally, they do not necessarily explain why those features are important.

A classifier that may perform well in its evaluation metrics (sensitivity, specificity, ROC AUC, etc.) may still benefit from explainability methods. In particular, if models were to take into account many multimodal variables, the primary drivers of a given prediction may offer important insight for the user of the machine learning system.

Related to interpretable AI is explainable AI. Rather than being able to fully comprehend the inner workings of a model, an explainable AI model is able to be queried so that a reasonable explanation to the prediction is provided. Explainable AI can be viewed on different levels as well: Global, cohort, and local explainability. Global explainability provides information about the entire population or dataset. Due to the complex nature of the MS disease, valuable insights on a population level are scarce. Cohort explainability gives insight on subsets of the data, which can be more interesting when taking into account certain covariates. In this way, different groups of patients can be identified and correlations within these groups may offer more helpful insights than looking only at a global level. Lastly, local explainability provides insight on the model’s output for a single input example. Every patient has a different profile, and therefore local explainability may help acquire insight into the prediction of the model for that specific patient or observation.

In machine learning models, uncertainty plays a critical, yet understated role in understanding and interpreting predictions. Healthcare specifically can greatly benefit from uncertainty quantification, as it can add a layer of trust between the user and the model (20–22). Two major sources of uncertainty are aleatoric and epistemic uncertainty (23).

A well-calibrated machine learning model is one in which the model’s predicted probabilities closely match the probabilities observed in the actual data (29). Mathematically, this is represented as P(y|p^(y)=α)=α. This equation signifies that the probability of an event y occurring, given that the model predicts it with probability α, should ideally be α itself. As a practical example: a model that predicts the probability of 40% disease progression for a patient will ideally be correct 40% of the time of all patients who receive a similar prognosis. For methods such as neural networks, this is not often the case by default, and calibration needs to be improved. Additionally, calibration can also be applied to regressors that output a distribution, rather than a single value. In this case, the confidence interval (such as a 95% confidence interval, for example) can be calibrated to ensure that it matches the observations.

The need for calibration is evident in the lack of information an uncalibrated classifier or regressor provides. Often, as is the case with neural networks, a neural network classifier will collapse to output probabilities close to 100% or 0% consistently, rather than providing accurate probability estimates (29). As a result, a user of such a system needs to blindly trust the classifier rather than being able to take the confidence of the classifier into account.

Integrating ML tools into existing clinical workflows presents another layer of complexity. For these tools to be adopted, they must fit into the highly regulated environment of healthcare. This integration involves designing user interfaces and metrics that are intuitive for clinicians, ensuring that ML predictions are presented in a way that complements decision-making processes rather than complicating them (30). Furthermore, imperfect data pose a problem during the training and prediction stages of an ML model. Data collection can be a laborious task, and in some cases the data cannot be accurately represented due to individual differences in disease expression. This rings especially true in the case of MS.

As highlighted previously, UQ and explainability techniques have their merit, as they can highlight potential issues when making use of ML assisted decision systems. However, the end-user may not find much use in the way UQ results are represented in literature. Even explainabilty results have varying degrees of success concerning their usability (31). These techniques could benefit from user studies, as their usability hinges on the representation and, in turn, interpretation by the end-user. For example, rather than providing the clinician and/or patient with a numerical value signifying a “trustworthiness” score or certainty otherwise, larger trust could be gained by comparing the patient with other patients that have similar disease trajectories. This opacity can hinder trust and acceptance, especially in a high-stakes field like healthcare where understanding the “why” behind a diagnosis or prognosis is as crucial as the outcome itself (31).

Data diversity and availability are critical factors that significantly influence the development and performance of ML models in MS research. MS is a disease with a highly variable clinical course and a wide range of symptoms that differ from patient to patient. This heterogeneity necessitates a rich and diverse dataset that captures the broad spectrum of the disease. After all, deep learning techniques are prone to overfitting, and may have performance below acceptable levels as a result (21, 32). Initiatives such as MSBase (33, 34) attempt to address the issue of out-of-distribution performance by providing multi-center data. The amount of data by itself may give the end-user a reason to trust a model, given enough diversity. Data quality is another concern, with issues such as missing values, inconsistent data entry, and the need for standardization across different data sources complicating the development of reliable ML models. Introducing diversity by including measurements that stray away from purely medical imaging or clinical data may also provide a new avenue of research, potentially discovering novel biomarkers. Future work should focus on developing models that can adapt to individual patient variations and incorporating emerging data types such as Motor Evoked Potentials (35, 36) into ML models.