The EMA serves as the central regulatory authority for medicines in the EU, overseeing the benefit/risk assessment for authorization and monitoring of medicines (A pharmaceutical strategy for Europe, 2024; European Medicines Agency, 2020). EMA also supports the development of medicines through various mechanisms, such as support for early access, scientific advice and protocol assistance and pediatric procedures. One of these mechanisms is the Qualification of Novel Methodologies (QoNM) procedure. This is provided by the EMA’s Committee for Medicinal Products (MPs) for Human Use (CHMP) based on recommendations from its Scientific Advice Working Party (SAWP). The review and assessment of requests is performed by a Qualification Team (QT), a group of experts appointed based on the specific expertise requirements. The QT is usually led by two rapporteurs (CHMP and/or SAWP member) and reports to SAWP and CHMP (Committee for medicinal products for human use, 2024). There can be two outcomes depending on the appropriateness of the presented evidence to support Qualification.

• CHMP qualification opinion (QO), public document

The QT drafts a List of Issues (LoI) which provides a preliminary scientific discussion and summarizes problems that have been identified in submitted qualification plans to be addressed by the applicant during a discussion meeting.

A LoI is the first readout of the assessment of a Qualification proposal that summarizes the scientific considerations and challenges that should be addressed by the applicant to achieve qualification or perform an optimized the qualification exercise going forward (in the context of a QA).

LoI include questions or concerns related to the validation strategy, reliability, accuracy and reproducibility of the methodology, as well as regulatory considerations that may need to be addressed. With them, applicants are provided with a focused overview of the limitations and considerations related to the specific qualification development, which combined with QA letters, can be a useful guide for the researchers to improve their qualification application and to have their methodology qualified.

To ensure transparency, draft QOs are shared before the final opinion is published (Table 1). Complementing a confidential QA letter, EMA may propose publishing a letter of support when a novel cannot yet be qualified based on the submitted data (Qualification of novel methodologies for drug development: guidance to applicants, 2024). These letters aim to encourage data-sharing and collaboration, enabling studies for the methodology’s eventual qualification and can also be leveraged for fundraising (Qualification of novel methodologies forb).

Successful QoNM supports and enables the development of novel medicines. The qualification process ensures that methodologies meet the necessary standards, e.g., validity, reliability, specificity and precision, required for drug development and regulatory decision making. QONM establishes the reliability and validity of these tools and helps integration and alignment of the qualified innovative approaches with the regulator so they can be considered by the CHMP during regulatory submissions.

The QoNM promotes accessibility within the scientific community. Through the publication of QOs, stakeholders gain insights into approved methodologies, their context of use (CoU) in the development of MPs, and the evidence supporting qualification.

The CoU refers to the specific intended use of a novel methodology in the development of MPs, including, e.g., the target population, the specific disease or condition, and the specific stage of development in which the methodology will be used. It is an important consideration in the QoNM, as a clear and concise description of the intended use is critical for regulatory assessment of the supportive evidence. This helps to ensure that the novel methodology is being used appropriately and effectively, and that it meets the necessary regulatory requirements.

CHMP, based on recommendations by SAWP, has used this procedure to assess regulatory acceptability for innovative methods, such as biomarkers, imaging methods, clinical outcome assessments, new animal models, statistical methods, innovative trial methodologies and big data approaches. However, the qualification of these methods is often compromised by weaknesses of applications and limitations tackled in this study. The subject of the LoI search are submissions that had an initial CoU for the proposed novel methodology, later revised by the CHMP, and that have been accepted for a QO, but have not necessarily been successful, which are identified as “agreeable CoU” in the report.

To identify the problematic aspects of qualification of applications, a review of regulatory documents, scientific literature, and guidance from the CHMP was conducted (Essential considerations for successful qualification, 2017). The views expressed in this article are the personal views of the authors and may not be understood or quoted as being made on behalf of or reflecting the position of the regulatory agency/agencies or organisations with which the authors are employed/affiliated. A major section of the items reviewed covers the LoI processed in the period from 1 January 2021 until the 20th of December 2022.

These documents provide a preliminary scientific discussion as well as raising issues to prepare a discussion meeting between QT and the applicant, of all 43 assessed procedures were extracted from EMA’s internal database.

Each LoI was examined, and the weaknesses of each qualification were split into major categories that appeared to be the most frequent among all applications. The review focused on the most common methodologies used in the development of MPs, such as biomarkers (Kraus, 2018; Gromova et al., 2020), imaging methods (Vermeulen et al., 2022; Izmailova et al., 2023) clinical outcome assessments (Clinical Outcome Assessments | U, 2021; Clinical outcomeassessment, 2021), statistical methods (Peterson and Altan, 2016; Kuhn et al., 2016), innovative trial methodologies (Innovative Science and Technology Approaches; Orloff et al., 2009; Beckman et al., 2022) and big data approaches (Qian et al., 2019; Zhu, 2020). Problematic aspects arising during the evaluation of novel qualifications include the validation strategy, target population and CoU as the most common ones. The summary of recommendations on how to address them are shown in Table 2.

A document containing a list of QoNM procedure applications processed in the interest interval was downloaded from EMA’s internal database. At the time of the analysis all the authors were affiliated with the Agency. Procedures that never started were excluded from the analysis. 43 procedures were examined, each being assessed individually on their intended and agreeable CoU, as well as a list of problematic aspects of qualification applications and a search for applications in clinical trials (CTs), related publications, or registration of patents to identify the research, development and impact activities surrounding the respective CoUs.

After identifying the applications on the EMA internal database, IRIS, within the desired timeframe, the document produced by both applicants, SAWP and CHMP were examined and a content analysis was performed. The topics discussed were identified and categorized and the intended CoU proposed by the applicant were compared to the agreeable CoU presented after modifications were required from CHMP.

Under the assumption that each qualification procedure is likely related to existing patents, publications and/or CTs, we performed searches in all three categories using the same specific set of keywords related to the methodology that has been qualified as part of a successful QO.

CT searches were executed using the ClinicalTrials.gov database, utilizing the expert search function.

Publication searches were conducted in the PubMed database, while patent searches were carried out in the Espacenet database, with filters employed to narrow the search to title, abstracts and claims as including other documents such as description resulted in too many irrelevant hits.

This data analysis led to three main findings regarding CoU, LoIs and the research, development and impact activities surrounding them, demonstrated through search results for publications, patents and CTs. The identified lists of issues were categorized, with validation strategy, doubts concerning the link between measurements and the predictive variables, being the most frequently identified issue category, appearing in nearly 50% of the procedures. Other commonly occurring problematic aspects of qualification applications include.

• Target population

Additionally, there are also other weaknesses, that are listed in Figure 1. While all these five weaknesses appear in more than 30% of the applications examined, after discussing with CHMP and SAWP experts in the QoNM procedure, it was decided to not focus of functional endpoints and CT application and conduct as, despite their incidence, they were considered a relatively niche issues, something that would have likely been clearer if a larger timeframe was analyzed.

The study identified four categories of QoNM submissions according to the CoU: New clinical outcome assessment, new digital tools and imaging methodologies, new biomarker qualification and new statistical methodologies. Digital Tools and imaging methodologies were grouped together due to their observed overlap and similarity in multiple applications where the QA/QO primarily focused on the digital component of the methods, technologies and methodologies. New clinical outcome assessments were found to be the most researched category, covering 13 out of 43 submissions each.

When it comes to comparing the proposed and agreeable CoU, analysis showed that in nearly 90% of these 43 procedures CHMP required a certain change in the proposed CoU, particularly regarding the lack of conciseness and specificity, as seen in the examples of Figures 2A–D.

Following the initial drug development, the initial findings are usually published. Further evidence generation (e.g., CTs) is conducted at later stages of development. Patents play a crucial role in securing innovation and later marketplace, as well as attracting funding opportunities.

To reach a point of applying for a QoNM, extensive research already had to be done considering protecting intellectual property, publishing research results and performing CTs. That is why it was decided to perform a search of each category to try to describe the research, development and impact activities surrounding the respective CoU.

Searching for publications, patents, and CTs related to agreeable CoU qualified for a QO in QoNM provides a comprehensive understanding of the scientific and regulatory landscape surrounding the methodology. This can help identify challenges or limitations of the qualification process and aligns with the EMA’s objective of enabling and leveraging research and innovation in regulatory science. Additionally, can facilitate the identification of gaps in knowledge, leading to a more thorough understanding of the methodology and its applications.

The search results, shown in Table 3, represent the number of search hits for patents, publications and CTs for each CoU. Search terms derived from CoU. Several cases, the ones with a QO, were investigated further to determine if they were applied in line with the agreeable CoU, as they are already in the public domain and do not raise confidentiality issues.

In 2022 Bakker et al. (Bakker et al., 2022) published a study examining the EMA’s biomarker qualification procedures from 2008 to 2020, investigating frequency, outcomes, challenges and biomarker characteristics. They emphasize the importance of robust validation strategies in discussions between applicants and regulators. Another study by Hendrikse et al. (Hendrikse et al., 2022) also examined biomarkers in the EU regulatory system in the same period, focusing on interactions between applicants and the EMA.

The research and the analysis results highlight the need for increased EMA support and showcases the importance of regulatory qualification for precision medicine and patient benefits. As seen from the number of submissions, new biomarker qualifications are still relevant. However, other categories of methodologies have also emerged, and the previously mentioned studies could serve as examples of a more thorough investigation that could be performed for each category in the future.

QoNM is a clearly defined, yet complex, process that requires planning and collaboration. The findings of this study, presented at the EMA multi-stakeholder workshop on QoNM (EMA multi, 2023) in April 2023, highlight the importance of the CoU, the challenges faced during qualification, namely, an inadequately described and justified CoU, validation strategy, target population, and the research, development and impact activities surrounding the respective CoU. By aligning with the EMA’s strategic plan, the qualification process can support the integration of science and technology, foster collaborative evidence generation, and enhance regulatory science within the EU.

Analysis of the identified CoU statements in QoNM submissions reveals a strong interest of applicants in clinical outcome assessment, statistical methodologies, and innovative digital and imaging tools together with new biomarkers and ATMP. Observing the difference between proposed and agreeable suggest that clear, specific and concise definition of the CoU is crucial in ensuring accurate and appropriate use of novel methodologies.

One example of a non-specific CoU proposed by the applicant (Stride velocity 95th centile as a secondary endpoint in Duchenne Muscular Dystrophy (DMD) measured by a valid and suitable wearable device, QO published (Opinions and letters of support on the qualification of novel methodologies for medicine development, 2024)) stated that the goal was to validate and enhance the measurement properties of SC95C and establish it as the primary endpoint in CTs across all phases, aimed at assessing the efficacy of new drugs for DMD patients by measuring maximal stride velocity. As the evidence did not match the targeted claim, a modification of the CoU was necessary, from primary to secondary endpoint (Figure 2B). Also, afterwards, the applicant requested to expand the CoU of SV95c to a secondary (efficacy) endpoint for other progressive neuromuscular diseases characterized by proximal muscle weakness. Nonetheless, the evidence presented was not supportive enough to validate SV95c as a secondary endpoint across different NMDs, as a result It was concluded that only the use of SV95C as an exploratory endpoint in NMD CTs could be supported. This highlights that clear and specific communication is crucial in ensuring developers and regulators reach a common understanding and that the intended use as supported by the presented evidence is accurately represented. Although not part of the analysis the time frame, should be mentioned that in July 2023 a new QO (European Medicines Agency, 2024) was published where SV95cCoU is extended to primary efficacy endpoint in DMD CTs, which indicates the value of a successful, stepwise qualification exercise for the applicant.

Another example, Prognostic Covariate Adjustment (PROCOVA™, QO published (Opinions and letters of support)), highlights the importance of focusing on the main points and not going beyond necessary. The applicant, in this case, overexplains why the proposed methodology is optimal, expanding the proposed CoU with no additional relevant information for this stage (Figure 2C).

The third example (IMI PREFER) highlights the need to specify the intended use in a concise way. The proposed CoU is more complex and lists several points in one long sentence. In comparison, the qualified CoU is more straightforward and split into bullet points to allow easier comprehension of the main themes (as seen from the QO (Opinions and letters of support)) (Figure 2D).

Since a specific set of keywords was chosen for the search, based on the agreed CoU for each submission, it helped narrow down the scope. Nonetheless, it needs to be considered the possibility that not all the hits in the presented Table 3 correspond to the methodology in question, on the contrary, the higher the number of hits, especially when in PubMed, of the higher is the possibility of finding false positives, despite the keyword set matching.

For most submissions, there was a greater number of hits for publications in comparison to the ones for CTs and patents. Although we are talking about new methodologies, certain information has already been published before and after the qualification date, resulting in more hits on PubMed. CTs showed few hits and even fewer related to the novel methodology in question. For example, if we look at data source and infrastructure for Enroll-HD registry-based studies for Huntington’s disease (Table 3) we see eight hits for CTs. However, exploring further, we discovered that only two were related to this novel methodology and its CoU, and one was in the recruiting stage at the time of this analysis.

Looking into the second example, utilize a scoring system 1-year post-transplant as a surrogate endpoint for the 5-year risk of death-censored allograft loss in kidney transplant subjects. (Table 3). Although no relevant CTs were found, a publication was identified describing an observational cohort study in line with the CoU. The fewer numbers can be explained by considering that the analysis only covered the previous 2 years and that these methodologies are new and innovative, therefore existing published data might be limited, and potential CTs might happen in the future. Patent search gave the least number of hits in total, which might also be explained by the timeframe. Another possible explanation could also be that a more refined search is necessary to get reliable information. It should be kept in mind that, while this analysis gives a rough overview of the journey of each novel methodology before the qualification stage, it is hampered by its scope and time limitations. As a result, the topic could be investigated further, and a deeper and more individually tailored analysis over a longer time would benefit the findings.

The analysis of 43 QoNM procedures between 2021–2022 highlighted several important findings and recommendations.

QoNM can be divided four different categories based on the agreeable CoU. Several weaknesses were identified, the most prevalent being to validation strategy, target population and CoU, all appearing in more than 35% of the applications.

The analysis highlighted concerns over validation strategy, appropriate selection of target population, relevant factors affecting treatment outcomes, justification of intended use and scope in QoNM procedures that applicants should carefully consider before the assessment. The analysis of the LoIs shall guide future applicants to optimize their submission package enabling an efficient Qualification procedure.

Aside from looking at these applications, the research, development and impact activities surrounding these CoU were also discussed, emphasizing the importance of monitoring publications, patents, and CTs to observe if the novel methodologies were employed in line with the agreeable CoU.

While the scope and time limitations of the analysis impede the development of a comprehensive understanding of the scientific and regulatory landscape surrounding QoNM, we believe that potential follow-up studies could be promising. Expanding the scope and timeframe of the analysis could enable the identification of additional challenges, limitations and supports the development of cases for regulatory approval and aid in identifying areas for further development, enhancing the overall understanding of the methodology and its potential applications. In summary, this analysis provides some high-level insights into the processes of Qualification of Novel Methodologies and offers recommendations to improve the content of QoNM applications and, consequently, higher likelihood of successful qualification. It is of value for researchers and regulatory authorities to consider these findings and recommendations to improve the qualification process and advance.