According to Gerke et al. (2020), AIHTs are different from traditional health technologies for three reasons: their capacity to continuously learn, their potential for ubiquity throughout the health care system, and the opaqueness of their recommendations. However, AIHTs suffer from ambiguities with respect to their definition and purpose as there is no agreed-upon definition that may help build an adapted and efficient policy and regulatory infrastructure (Pesapane et al., 2018). The drawback of AI exceptional features and of the high variability that exists among AI systems is that it poses definitional problems that affect AIHTs’ regulation and slow down their deployment in the healthcare sector (Love-Koh et al., 2018; Haverinen et al., 2019). Compared with traditional health technologies (such as drugs, vaccines and medical devices), AIHTs are not static products and have the capability to learn and improve over time (Parikh, Obermeyer, and Navathe 2019; Dzobo et al., 2020). AIHTs are therefore in stark contrast with most technologies in medicine, which are fairly well defined and usually implemented when they are fairly well understood.

Contributing to the distinctiveness of AIHTs in the health sector, self-learning and self-adaptation propensities clash with current regulatory frameworks and clinical practices (Alami et al., 2020; Fenech et al., 2020). It is easier to evaluate “locked” AIHTs, which are much more comparable to current health technologies (which cannot by themselves evolve). Currently, the majority of FDA-approved AIHTs have their capability to evolve locked (Dzobo et al., 2020; Miller, 2020). A locked algorithm will always yield the same result when it is fed by the same data, therefore it does not change overtime with uses. Locked algorithms are not per se safer. They could be more harmful than “unlocked” or “adaptive” algorithms if they end up yielding erroneous results (based on legacy training data that are outdated), misleading patient care or systematizing biases (Prabhu 2019). Thus, a locked AIHT may require new regulatory approvals if during real-world usage significant and unexpected patterns of results are observed (i.e., stable and expected process produces outcomes unexpected because of incorrect priors about the data fed into an AIHT) or if it is deemed necessary to update the algorithms to match advances in medical knowledge (Gerke et al., 2020; Miller, 2020). There also is the issue of when AIHT is used (or not used) on new populations that differ from the training data, which raises questions about how the training data upon which an AIHT was developed and whether certain populations may be unduly excluded from benefiting from its development and implementation. Therefore the concept of locked may be misleading and should not be conveyed as safer (Babic et al., 2019). Unlocked or adaptive algorithms that improve over time is the future according to some (Prabhu 2019) as they will outperform humans (Dzobo et al., 2020). But some issues are to be expected. Unlocked AIHT may change as they process new data and yield new outcomes without the knowledge or oversight of its users, which demands that their safety and security must be continually re-evaluated (Abràmoff et al., 2020). Also, unlike traditional healthcare technologies and locked algorithms, unlocked AIHTs are more vulnerable to cyber-attacks and misuse that can cause the algorithm to generate problematic and highly damaging outputs (Babic et al., 2019; Miller, 2020).

Another consideration that helps AIHT qualify for being an anomalous technology in the health sector is that the algorithms will need to be regularly updated (at high or even prohibitive prices) due to advances in medical knowledge and access to new datasets or at the risk of their usage becoming malpractice. To allow a rigorous analysis of the safety, efficiency, and equity of a given AIHT, it is necessary that the locked or unlocked state of the algorithm is always known to regulators and end-users (Char et al., 2020). Such transparency is necessary since due to the very distinct ethical and clinical implications that locked or unlocked AIHTs may generate. The update problem implies that a locked AIHT could quickly become outdated—potentially from the moment it is prevented from evolving (with or without supervision)—and that this could generate important risks as a result of the deployment and use of AIHTs in real-life contexts (Abràmoff et al., 2020). Although not all algorithms may need to evolve or be updated in the short term, at some point in time, updating or replacing an AIHT will involve additional post-acquisition costs. Post-acquisition updates and costs may seem counter-intuitive considering the distinctive characteristics attributed to AI, such as self-learning and continuous improvement. This may lead for certain organizations (in particular, in less affluent contexts or in periods of economic turmoil) not to deploy updates which will result in the uses of outdated algorithms and therefore sub-optimal benefits (if not harms) for some patients or services (Prabhu 2019). Since AIHT are considered as being more pervasive than physical technologies (such as drugs and other health products), some are arguying that managing the consequences of an outdated algorithm outweigh those of traditional health technologies (Babic et al., 2019; Prabhu 2019); even if it is very difficult to withdraw effectively a drug from the market, it is still possible to do so, while it may be much more challenging for AIHTs that are less visible, interpretable and tangible and more likely to be embedded in a hospital’s or health system’s IT systems.

Contrary to many health technologies, AI may have systemic effects that can be felt across an entire health care system, or even more so across health care systems in several jurisdictions (Dzobo et al., 2020). Gerhards et al. (2020) go as far as stating that AIHTs (especially those using machine learning) can yield significant changes to an entire healthcare system. These changes might not necessarily come from expected technological disruptions, but might come from the adaptation of the healthcare setting to certain methods and processes relying on AIHTs. This adaptation may initiate extensive and lasting transformations that are likely to affect all actors working in, using or financing the health system (Gerhards et al., 2020). The clinical use of some AIHTs may have the effect of transforming local health care administration practices by incorporating exogenous priors embedded within the technology. For instance, if a payer (public or private insurer) decides that a given AIHT recommendation become a precondition for reimbursement (i.e., making other care no longer reimbursable), this may have significant impacts on the way care is delivered, and will reduce patients’, clinicians’ and administrators’ autonomy in making shared and appropriate decisions when the human-recommended care is different than a new gold standard based on AI on data and priors (Vayena, Blasimme, and Cohen 2018). There is therefore a process of importing practices, potentially very different, which can strongly contrast with local habits and norms, requiring both adaptation and an impact assessment of these exogenous practices on the host environment.

AIHTs can have systemic real-world life-and-death consequences for patients (Miller, 2020), especially as AI will span across the life continuum from birth to death (Dzobo et al., 2020). AIHTs, unlike most drugs or medical devices, will also affect non-ill or non-frequent users of the health care system, be they due to AI increasing role in health surveillance, care optimization, prevention and public health, telemedicine (Love-Koh et al., 2018; Pesapane et al., 2018; Char and Burgart 2020). AI can help “democratizing health care” (i.e., in the sense of facilitating access) by extending care into patients’ homes (Reddy et al., 2020), places where more individualized and personalized care can be facilitated. While being increasingly present in patient care, AI will not address everything that has to do with the overall well-being of people. Some aspects less related to illness, such as spirituality and sociality, will most likely not be resolved and supported by AI systems (Dzobo et al., 2020). Therefore, a systematic response to using AI in health care may systemically neglect important aspects of care.

Mistakes due to AIHTs used in clinical care and within the health care system have the potential to widely harm the patient population. Some AI systems, especially in primary care settings, can have impacts on the entire population of a hospital or clinic (such as an AI-powered patient triage). This makes some people say that it is all the more necessary that all algorithms should be submitted to extensive scrutiny, with an increased attention on validation in clinical settings before they can be deployed in medical practice (Dzobo et al., 2020). In addition, a key challenge in implementing AI is that, without a comprehensive understanding of health needs (especially those not covered by AI), there is a risk of fragmenting healthcare delivery by silo use of AI systems. Such silo use may lead to weakening health systems capacity and efficiency in addressing patients needs.

AIHTs can have tropism effects on the healthcare system that may shape and normalize certain practices and expectations that are not necessarily accepted, widespread, cost-effective or standard in new contexts. An example of this would be an AIHT trained on medico-administrative data in a context where physicians have often modified their billing to enter the highest paying codes for clinical procedures, causing the algorithm to infer that these codes represent the usual, standard, or common practice to be recommended (Alami et al., 2020). Thus, the algorithmic inference would be biased because the procedure billed maximizes the clinician’s remuneration, but potentially was not the one performed; this can lead to a cascade of non-cost effective recommendations. Such tropism effects may increase the risk of inappropriate treatment and care, and may result in importing AIHT-fueled standards and practices that are exogenous and non-idiosyncratic to local organizations and that may perpetuate latent biases in training data that are not present in certain health systems or contexts of care (Abràmoff et al., 2020; Alami et al., 2020; Miller, 2020). Therefore, the large-scale systematization of certain behaviors or inclinations may end up resulting in significant costs and harms for organizations and health systems as well for patients and HCPs (Alami et al., 2020; Hu et al., 2019). For example, higher sensitivities to clinical thresholds could lead to overdiagnosis or overprescription, while lower sensitivities could result in undiagnosed and untreated segments of the population; it is in the potential scope of the impacts that exceptionalism lies and must be carefully assessed in HTA (Alami et al., 2020; Topol 2020).

According to Alami et al. (2020), instead of seeing AIHTs as a collection of distinct technologies, they should be regarded as a “health system transformation lever.” AI can serve as a strategic lever for improving health care and services access, quality and efficiency. Used in such way, AI could have significant society-wide impacts, including technological, clinical, organizational, professional, economic, legal, and ethical.

AI can become a key enabler of learning healthcare systems (LHS) to achieve their full potential (Babic et al., 2019), especially since AIHTs are themselves learning systems (Ho, 2020). AIHTs and LHS can complement each other as both strive when there are porous boundaries between research and development and with clinical and organizational practices. Using data from the health care system, AI can learn and recalibrate both its performance and behaviors, and over time inform and refine the practices of the health care system (Babic et al., 2019). AI can allow for ongoing assessment of accuracy and usage and continuous risk monitoring (Ho, 2020).

AI, as a lever, can also have a systemic impact of putting forward the response to needs for which there are ready-to-use technologies, causing to pay little attention to serious unmet needs (Alami et al., 2020). According to Grant et al. (2020), AI may represent the “next major technologic breakthrough” in health care delivery, offering endless possibilities for improving both patient care and yielding health care system-wide optimizations. However, this blurring of boundaries poses significant problems for adequate regulatory design and should not be taken lightly (Babic et al., 2019).

A large part of the explanation for AI exceptionalism comes in particular from the belief that an AI-generated outcome is inherently better than a human one (Char and Burgart 2020). This phenomenon, known as the automation bias, describes the fact that slowly but surely, AI is establishing itself as an authority over current practices and error-prone healthcare professionals. Part of this is reflected in the fact that it is now recognized as a problem that a person who disagrees with a result or recommendation generated by an AI must justify their opposition with much more data than those used by the AI to achieve that result (Char and Burgart 2020). The technological imperative—i.e., the mere fact that a sophisticated technological intervention exists creates pressure to use it because it is perceived to be superior to conventional practices, despite the risks—reinforces this belief and AI in medicine is currently having its technological imperative moment (Carter et al., 2020).

These high expectations toward AIHTs form the basis of the inevitability of AI in health. To the point where AI is seen as inevitably the future of medicine (Dzobo et al., 2020). Self-learning and the ability to perform arduous and repetitive tasks explains the growing interest for a greater place of AI in standard medical care. There are high hopes and, according to many commentators, good reasons to think that in a near future, virtually all physicians will be assisted by AI applications to expedite certain tasks and, in the median term, due to continuous learning, AIHTs might outperform humans in a wide range of areas (Dzobo et al., 2020; Abràmoff et al.2020; Gerke et al., 2020). It is not only for clinical or therapeutic reasons that AI seems to be inexorable; there is also competition within the AI ecosystem. The growing importance of AI and its inevitability also stems from the competition between political decision-makers from different jurisdictions to widely deploy AI in order not to lag behind others (Gerhards et al., 2020). Considering all the interests at stake, the massive investments and accelerated development of AI, the question is no longer whether AI will be part of routine clinical care, but when (Reddy et al., 2020).

Although the technological imperative is strong and that AI in health is very attractive and seems inevitable, caution is called for. In this regard, AI chasm is a powerful concept to rebalance and help manage expectations of overly rapid deployment and ubiquity of AI in health care (McCradden et al., 2020). AI chasm refers to the phenomenon that while AIHTs are very promising, very few will actually be successful once implemented in clinical settings. HTA agencies have an important role to play here to contain this phenomenon and reduce its frequency and spread (McCradden et al., 2020). One of the roles of evaluation and regulation is precisely to finely consider the implications of these technologies to overcome this phenomenon. This requires not only an analysis of technical efficiency and performance, but also an oversight of empirical and ethical validation to ensure the rights and interests of patients. This requires the development of regulatory tools that are well adapted to AIHTs so that there are clear procedures and processes to properly evaluate and screen AIHTs (Alami, Lehoux, Auclair, Guise, et al., 2020; Abràmoff et al., 2020). This is necessary to avoid ethical drifts and unacceptable (economic, health, social) costs that may be caused by technologies that are not adapted to the needs and specificities of a clinical or organizational context, or by milieus feeling pressure to deploy a technology and adapt its practices in a way that ultimately does not benefit patient care or sound health care management (Michie et al., 2017; Abràmoff et al.2020).

AI is currently in an era of promises rather than of fulfillment of what is expected from it (Michie et al., 2017). This new science has yet to move beyond average outcomes on individuals to actual personalized benefits based on their situation, characteristics, and desired outcomes. It is important to remain critical and vigilant with respect to the rush to adopt these new technologies, possibly more so than politicians are at present (Cowie 2017). Especially when thought leaders’ perspectives echo public wonderment and aspirations that AI transforms human life (Miller, 2020). With their development and implementation being largely driven by a highly speculative market and by proprietary interests, AIHTs are largely embedded into biocapital (Carter et al., 2020). That is to say, a vision of medical innovation that is based not on the actual creation of value, but on selling a certain vision of the future. It is through the sale of imaginary evoking unparalleled performance and disproportionate benefits to encourage all AI players to engage in the massive implementation of AI despite its uncertainties and shortcomings (Carter et al., 2020). That being said, in a study reported by Vayena et al. (2018), half of the surveyed American healthcare decision-makers expect that AIHTs will both improve medicine and fail meeting hyped expectations. Miller (2020) sums up the present phenomenon as follows: “No matter how sanguine the gurus touting game-changing AI technologies are, and no matter how much caregivers and patients hope that their benefits to medical practice and outcomes are not hype, all parties must remain vigilant.” AIHTs are in their phase of promises and hype, which is creating inconsequent expectations (Reardon 2019).

The consequences of these unreasonable expectations can be very significant. For instance, patients’ unsound expectations regarding the clinical outcomes of AIHTs can negatively affect their care experience (Alami et al., 2020). Certain areas of care, such as breast cancer, are particularly fertile ground to AI companies’ hype and promises (Carter et al., 2020), because it resonates with patient unfulfilled demands. The counterweight to these expectations is not yet in place as, despite all the hype, the science of AI is still young and possibly not yet mature, including gaps in clinical validation and perhaps imprecise health recommendations (Dzobo et al., 2020). HTA agencies and regulators have an important role to play, particularly in developing a regulatory infrastructure that is as exceptional as technology can be for health and as powerful as the “unfounded hype” can be, to use Mazurowski (2019) expression.

Reddy et al. (2020) identified three key AIHT-stemmed ethical challenges in care delivery: AI-fostered potential bias, patient privacy protection and trust of clinicians and the general public towards machine-led medicine. AI is also prone to generating new health inequalities; perhaps more potent than its ability to reduce existing ones (Fenech et al., 2020). An important caveat in terms of health care equity comes from the fact that those who compare very well with historic patient data will be the one benefiting the most from AIHTs. A cautious attitude is therefore called for with regards to patient and disease heterogeneity, taking into account that patterns detected by AI are largely deduced from smaller historical data sets (Dzobo et al., 2020). In addition to the current disparities, digital literacy and access to technologies are adding up, so that if nothing is done, large segments of the population may be excluded from enjoying the benefits of AIHTs, resulting in significant issues of justice (Fenech et al., 2020).

According to Fenech et al., 2020, AI is unlike any other health technology, due to its capability of being a general-purpose technology forcing to question the very essence of humans. This technology is particularly sensitive for the healthcare sector as it raises new existential questions that regulators and public decision-makers must now face. One of such key existential challenge for HTA, according to Haverinen et al. (2019), is that AI is becoming a new decision-maker. This adds an actor with a decision-making role on the fate of patients and the health care system in addition to the role of HCPs and increases the complexity of performing comprehensive HTA. For Ho (2020), a distinctive ethical concern that stems from AIHT is the technology’s unparalleled autonomy, which intensifies ethical and regulatory challenges, especially in terms of patient safety. While this obviously raises questions about liability (who is at fault for harm, and who is responsible for explaining it and being accountable to patients), it also requires thorough thinking about appropriate ways to ensure that care is humane and respects the dignity of persons (Pesapane et al., 2018; Vayena et al., 2018; Fenech et al., 2020).

Exceptionalism also stems from the fact that the field of medicine is structured around the transparency and explainability of clinical decisions, which poses serious problems for the acceptance, regulation and implementation of (too often inexplicable) AI in the health care system (Reddy et al., 2020). As Miller (2020) points out that technology insertion is never neutral, both the success of AI in health care and the integrity and reputation of health care professionals depend on the alignment between the ethos of medical ethics and the ability of AIHTs (notwithstanding its benefits and performance) to respond to the challenges that its very characteristics pose to the health care system (Reddy et al., 2020). It is therefore widely acknowledged that AI will have considerable benefits on health care (optimized process, increased quality, reduced cost, and expanded access) that will come at the price of raising ethical issues specific to the technology (Abràmoff et al., 2020). This moral cost and related ethical considerations partly explain that the field of AI ethics has recently “exploded” as academics, organizations and other stakeholders have been rushing to examine the ethical dimensions of AI development and implementation (Fiske et al., 2020). However, some are skeptical, such as Fenech et al. (2020) who was warning that data ethics is fashionable. While Bærøe et al. (2020) go as far as arguing that “exceptional technologies require exceptional ethics” and that “an intentional search for exceptionalism is required for an ethical framework tasked with assessing this new technology”.

AI uses larger than ever volumes of data generated by individuals, governments, and companies, and according to Fenech et al. (2020), the greater complexity of health data raise new questions about the governance of data use and storage, especially as AI technologies are only effective and relevant with up-to-date, labelled, and cleaned big data. In addition to data, the hardware infrastructure will need to be updated over time, requiring major financial investments to maintain the use of AI in the healthcare system (Dzobo et al., 2020). However, too few technical studies are helping to appreciate and help managing AIHTs’ complexity. In most studies, contextual, clinical and organizational considerations, implementation and uses of the technology are neglected, which complicates regulators’ assessment as they are mostly informed about the significance of AI applications’ technical performance (Alami et al., 2020). Caution should therefore be exercised, particularly since the complex ethical and regulatory issues involved deserve careful consideration before deploying these technologies in routine clinical care (Prabhu 2019).

AIHTs raise new regulatory challenges in part due to the greater variation in their performance between the test environment and the real-word context than those of drugs and medical devices. According to Gerke et al. (2020), AIHTs have potentially more risks and less certainty associated with their use in real-world contexts, which is central to regulatory concerns. However, most AIHTs have not been objectively validated in and for real-world usages (Alami et al., 2020). In that sense, one important caveat is that the adoption and impact of AIHTs are unlikely to be uniform or to improve performance in all health care contexts (Gerke et al., 2020). This is attributable concurrently to the technology itself (and its distinctive features that renders it disruptive), to the contexts of implementation (the systemic impact of the technology across the health care system, and clash with local practices) and to the human biases associated with the use of these technologies (inability to reason with AI-provided probabilities, small samples and noise induced extrapolation and false patterns identification, and undue risk aversion) (Gerke et al., 2020). For regulatory authorities, these represent great challenges for deciding whether marketing authorization is justified. But it is also puzzling for hospital, clinic and health care system purchasers to determine whether an AIHT will actually add value and increase performance of care and service delivery. There is a lot to be studied and understood on the broad systemic policy implications of AIHTs in real-world context of care and services (Alami, Lehoux, Auclair, de Guise, et al., 2020).

AIHTs’ exceptional characteristics have significant regulatory implications as regulation is emerging, but at a far slower pace than technological changes, which are virtually infinite (Char and Burgart 2020). Regulative complexity is furthered by the fact that existing standards (e.g., those of the Food and Drug Administration, European Medicines Agency, Health Canada) do not translate well for self-evolving technologies (Dzobo et al., 2020; Shaffer 2020; Topol 2020). This definitional deficit complicates the regulation of this technology and the implementation of appropriate policy infrastructure (Pesapane et al., 2018). Recent approvals of algorithms highlighted some limitations of existing regulatory standards and processes (Haverinen et al., 2019; Parikh et al., 2019). These considerations are threefold and concern the levels of requirements, the speed of AIHT developments and the equilibrium posture that regulators must adopt.

A challenge for existing regulatory regimes lies in the extensive information requirements on both the nature and effects of health technology, as well as clinical data on efficacy and patient safety, and population and societal impacts. However, regulators have yet to develop an infrastructure and processes that are appropriate and optimal for AI, and this requires more knowledge about how algorithms work (Dzobo et al., 2020). This complicates the problem because AI is a less transparent and explainable technology than drugs or medical devices can be (Abràmoff et al., 2020; Reddy et al., 2020). Privacy concerns are also important and there is yet no public agreement regarding data collection and sharing for commercial purposes, nor regarding for-profit data ownership (Michie et al., 2017). This calls for finding collective responses to these considerations, which must accompany the work of structuring HTA practices and infrastructures by regulators (Fenech et al., 2020).

Another dimension putting pressure on regulation is the speed of development. For regulating a fast-changing and unpredictably sector such as AI, time is of the essence to ensure that regulatory standards and practices are relevant (Pesapane et al., 2018). Currently, regulation has to constantly catch up with the private sector which leads to important gaps in terms of ethical examination of AIHTs (Shaffer 2020). Since, most developments are done by the private sector and HTA processes are not yet well designed and optimized, there is a problem of scrutiny (Abràmoff et al., 2020). So to keep up, regulation must be as fast as technological developments in AI, therefore it requires to conduct assessment and oversight at an unprecedented pace (Haverinen et al., 2019). However, this need to proceed quickly, in particular to match the private sector’s pace, must be put into perspective with the very acceptability of a significant presence of commercial interests in the big data and AIHTs sector.

Achieving the right balance is delicate for HTA agencies between accelerating the development of HTA policies and procedures and not falling prey to the sirens of AIHT’s hype (Cowie, 2017; Miller, 2020). Regulators want to see the health care system reap AI’s benefits quickly, but if their assessment is too hasty and the implementation of the first generations of AIHTs encounters difficulties or, worse, generates adverse effects, social and professional acceptability may be shattered and further delay the deployment of AI in healthcare. According to Reddy et al. (2020), it could need a single serious adverse incident caused by an AIHT to undermine the public’s and HCP communities’ confidence. Considering that AI’s acceptance is still fragile, and that AI is expected to have an expanded presence in all aspects of the health care system, HTA agencies will have to be extra careful in considering the ethical and regulatory implications of IA. If not well managed, these considerations will become major barriers that will play against the deployment of AI in healthcare (Pilotto et al., 2018; Vayena et al., 2018; Bærøe et al., 2020).