In recent decades, several “value frameworks” have been applied to evaluate the drug value in the health technology assessment (HTA) in the world (de Andrés-Nogales et al., 2020; de Andrés-Nogales et al., 2021; Hummel et al., 2012). These frameworks are employed to facilitate dynamic drug supervision, promote clinical rational drug use, adjust coverage and reimbursement decisions, and support drug marketing (Yu et al., 2023). In China, HTA assessment evidence based on comprehensive drug evaluations has also been used to update the national reimbursement drug list since 2017 (Chen et al., 2023). Despite differing evaluation goals, the introduction of these frameworks has expanded a broader concept of drug value. In the field of drug value evaluation, there is a growing trend towards incorporating a wide range of dimensions, including clinical, economic, societal and humanistic values. As proposed by Garrison LP Jr, the value of drugs cover life years gained, improvement in quality of life, cost savings within health system, productivity, cost savings outside health system, scientific spillovers, insurance value, real option value, value of hope and reduction in uncertainty (Garrison et al., 2017). Moreover, the criteria utilized to assess each dimension of value are continuously evolving. For instance, the criteria for evaluating drug efficacy have advanced from merely focusing on primary or endpoint outcomes to a comprehensive assessment of efficacy including therapeutic benefits type, multiple outcomes (primary outcome, secondary outcome, endpoint outcome, patients report outcome) and types of clinical research (Migliore et al., 2015; Vermersch et al., 2019; Institute for Quality and Efficiency in Healthcare, 2022). In situations requiring intricate decision-making, researchers tend to prioritize a holistic evaluation of drug value (Goetghebeur et al., 2012). The substantial and scattered evaluation criteria of drugs value posed inevitable challenges for the decision-making process. Collecting and categorizing these value criteria are increasingly essential to aid value-based decision-making.

Multi-Criteria Decision Analysis (MCDA), as a valuable approach for integrating diverse and complex trade-offs among different stakeholders, can provide comprehensive, transparent, and structured evaluations of drug value (Mt-Isa et al., 2014). Moreover, this approach has gained widespread use in the healthcare fields, particularly in the assessment of interventions for authorization, prioritization for coverage or reimbursement, selection for clinicians and patients, and the allocation of research funds (Marsh et al., 2014; European Medicines Agency, 2012). It is noteworthy that some prominent organizations, such as the European Medicines Agency, the Food and Drug Administration, and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), have acknowledged and endorsed MCDA as a valuable tool for assessing the benefits and safety of medicinal products (European Medicines Agency, 2012; Marsh et al., 2016; Angelis and Kanavos, 2021). Due to different evaluation purposes, various MCDA frameworks have been developed, including EVDEM (Evidence and Value: Impact on Decision Making) framework, Benefit-Risk framework, AVF (the Advance Value Framework), MAUT (Multi-attribute Utility Analysis), and others (Marsh et al., 2016). Each MCDA framework exhibits unique characteristic in selecting criteria, scoring and weighting techniques. For example, EVIDEM framework, introduced by Goetghebeur MM in 2012, employs a scoring scale ranging from 0–3 for 15 criteria (e.g., disease severity, cost-effectiveness, etc.) (Goetghebeur et al., 2012). Whereas, the benefit-risk MCDA framework utilizes a technique of “0–100 value scales” to determine the criteria score (Mendoza-Sanchez et al., 2018).

The objective of this study was to systematically collect and analyze drug value evaluation dimensions, criteria and techniques of weighting and scoring used in various MCDA frameworks based on the existing literature. This analysis aims to provide a foundation for the construction of a drug evaluation framework that could support regulatory approval, drug pricing, prescription decisions making, medical insurance reimbursement, and other related purposes.

Our analysis of 82 literature revealed that the evaluation of drug values covered 5 dimensions and 111 criteria (678 times). 94 (94/111, 84.68%) criteria were quantitative. Of these, 56 of the 94 criteria (59.57%) measured clinical value. Furthermore, we identified 11 scoring and 11 weighting techniques used in various MCDA frameworks. “Swing weighting” and “a direct rating scale” were the most frequently techniques used in MCDA literature. To the best of our knowledge, our study was the first attempt to categorize drug-oriented MCDA criteria based on their clinical, economic, societal, innovative and humanistic values.

The evaluation of drug value is a multifaceted process that requires thorough consideration of various dimensions, such as medical, ethical, economic, and social equity criteria (Garrison et al., 2017; Lakdawalla et al., 2018). Therefore, it is imperative to categorize the criteria for drug evaluation based on different value dimensions. However, previous literature reviews on MCDAs in healthcare have exhibited certain limitations, including oversimplified categorization of criteria. Lalla Aida Guindo classified the criteria based on the evaluation dimensions of EVIDEM framework (Guindo et al., 2012), while Tamas Zelei sorted out the decision criteria based on cost and outcome dimensions (Zelei et al., 2021). Additionally, a wider range of interventions was encompassed, such as pharmaceuticals, public health interventions (e.g., smoking cessation, obesity), screening, surgical strategies, devices, (Marsh et al., 2014). Further detailed research is required to explore the unique characteristics of different interventions in value evaluation. The National Institute for Health and Care Excellence (NICE) utilizes topic classification to develop health technology assessment, encompassing devices, diagnostics, interventional procedures, medicines, combination or integrated technologies, digital technologies and other technologies (National Institute for Health and Care Excellence, 2022). Our research focused on pharmaceuticals and categorized for drug-oriented MCDA criteria according to 5 dimensions (clinical, economic, societal, innovative and humanistic value dimensions). We found that clinical value criteria were commonly utilized, followed by economic value criteria, which aligns with previous studies (Park et al., 2012; Kwon et al., 2017; Tanios et al., 2013; Golan et al., 2011; Baji et al., 2016). Clinical value is the core factor in drug development and use, which can reflect the extent of a drug’s ability to meet clinical needs and provide clinical benefits for its target population (Liu et al., 2021).

In relation to clinical value, the regulatory and reimbursement processes for drugs prioritize the assessment of the effectiveness-safety balance, also referred to as efficacy-tolerability or benefit-risk balance (Dang et al., 2020). For example, the Food and Drug Administration (FDA) conducted a thorough benefit-risk evaluation of new drugs, taking into account the substantial evidence of safety and effectiveness provided by the sponsor (Food and Drug Administration, 2021). Similarly, the Institute for Quality and Efficiency in Healthcare (IQWiG) in Germany made a trade-off between benefit and harm aspect in the early benefit assessment of new drugs, and determined the extent of added benefit (minor, considerable, and major treatment effects) (Institute for Quality and Efficiency in Health Care, 2022). Our study also confirmed that “comparative safety/tolerability” was the most frequently used clinical value criterion, followed by “comparative effectiveness/efficacy.” Although there are subtle differences between efficacy and effectiveness, as well as safety and tolerability (Berger et al., 2012), we aggregated “effectiveness” and “efficacy” criteria into a single “comparative effectiveness/efficacy” criterion considering greater operability in practice, as well as “comparative safety/tolerability.”

With regard to the economic value, we found the most frequently utilized criteria were “cost of intervention,” “comparative other medical costs” and “comparative non-medical costs.” However, it is constantly discussing whether to incorporate cost (and related criteria such as cost-effectiveness, budget impact) as a criterion in MCDA (Golan et al., 2011; Marsh et al., 2018a; Hansen and Ombler, 2008). Hansen argued “cost is not a value criterion, but a measure of what has to be given up to achieve the value criteria” (Hansen and Devlin, 2019). Including cost related criteria in the calculation of MCDA values may violate the principle of “structurally independent” of the multidimensionality of value and damage opportunity cost in the allocation of limited resources (Hansen and Devlin, 2019; Wilson et al., 2022). Wilson compared the extent of consistency in ranking importance of the four criteria of “treatment effectiveness,” “cost of the intervention” “risk of serious harms,” and “risk of mild-to-moderate harms” based on MCDA and cost-effectiveness analysis models and found “cost of the intervention” should be excluded in MCDA for prioritizing intervention setting (Wilson et al., 2022). Golan and Angelis also excluded the cost of the treatments in the MCDA value metric. Golan used MCDA to aggregate health-related benefits of multiple dimensions and calculated the final value score by formula “cost/health-related benefits value” (Golan and Hansen, 2012). Angelis used costs per unit of MCDA value to calculate overall weighted preference value scores (i.e., the final value scores) (Angelis et al., 2020). However, cost related criteria were incorporated in EVIDEM frameworks (Goetghebeur et al., 2012). A set of criteria including “direct medical costs,” “direct non-medical costs” and “indirect costs” were used as the “modulators” of core model to measure economic consequences of intervention (Casellas Caro et al., 2022). With the soaring drug costs and limited financial resources, we think cost of the interventions should be considered in decision making, especially in prioritizing drugs for reimbursement.

Medical innovation is a complex concept lacking a definitive consensus, which mainly related to the realm of “therapeutic innovation” (Tanios et al., 2013). Therapeutic innovation is commonly associated with generating improved health outcomes that were previously unattainable and addressing the unmet medical need (Morgan et al., 2008). Nevertheless, we found the AVF frameworks offer a distinct viewpoint for quantitatively assessing the value of innovation, utilizing criteria such as the mechanism of action of the medicine, its spill-over effects, and its utility for patients (e.g., convenience) (Angelis and Phillips, 2021). The criterion of “mechanism of action” can be evaluated by Anatomical Therapeutic Chemical (ATC) Classification System of World Health Organization (WHO). The degree of innovation “spill-over effects,” as argued by Angelis, can be gauged by the number of new indications being explored for the medicine at each stage of clinical development (e.g., Phase I, Phase II, Phase III, Marketing Authorization phase). “Patient usefulness” could be perceived as ease and convenience, which were related to mode of administration, dosing schedule, medication restrictions, and product-specific designs.

Once the criteria are agreed upon in MCDA frameworks, the selection of the appropriate techniques of scoring and weighting are subsequently considered by the researchers. A published study in healthcare found AHP is the most frequently applied technique for weighting, followed by “Swing Weighting” (Camps et al., 2019). However, our research showed that “Swing Weighting” and “a direct rating scale” were the most frequently techniques used in the field of pharmaceuticals assessment. “Swing Weighting” could obtain more discriminative weights by obtaining the expected performance floating range of each criterion in advance. It should be noted that the weight of each criterion is typically determined by a panel of experts, independent of the specific drug (Hansen and Devlin, 2019; Camps et al., 2020; Guarga et al., 2019; Jiménez et al., 2018). Due to the weights and scores were closely related to the panel of experts participating in MCDA, the process of weight and score determination were subjective to some extent. So, sensitivity analysis was often used to assess the robustness of MCDA results by re-testing using alternative weighting and scoring techniques. Furthermore, as Marsh argued, it is important to incorporate well-trained experts and working groups to perform the MCDA (Marsh et al., 2018b).

Some limitations were worth mentioning. Firstly, the identified literature was only searched in seven publicly available databases, and supplemented with relevant review references and expert consultations. Grey articles were not acquired in this study, which may lead to a publication bias to a certain extent. Secondly, due to language limitations, only Chinese and English literature were identified. Excluding articles published in other languages may have had an impact on our results. Thirdly, this study did not recognize the critical dimensions and criteria targeted for different stages of life cycles in drug value evaluation. In future, we will continue to carry out the research on drug value assessment in different stages of life cycles.

This study comprehensively presented the current evaluation dimensions, criteria, and techniques for scoring and weighting in drug-oriented MCDA articles. By highlighting the frequently cited evaluation criteria and techniques for scoring and weighting, this analysis can serve as a resource to reasonably select these evaluation details in constructing the appropriate MCDA framework. The ultimate objective is to provide a solid foundation for the construction of a drug evaluation framework to advancing the structured decision in drug management. In future, as research on MCDA drug value evaluation deepens, more attention should be paid to the assessment criteria and techniques of weighting and scoring for different types of drugs. For instance, specific criteria should be considered when assessing traditional Chinese medicine in China.