Coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), is a global health challenge. Since the first case was reported in December 2019, the virus has spread rapidly to become a global pandemic, with more than 760 million confirmed cases and more than 6.9 million deaths worldwide as of May 2023 (1–3). Several countermeasures have been implemented, including the development of COVID-19 vaccines (4, 5).

The SARS-CoV-2 spike (S) protein is a primary target of neutralizing antibodies (nAbs), which are essential for protective immunity against viral infection (6–9). The receptor-binding domain (RBD), located in the S1 subunit of the trimeric S protein, mediates viral attachment by binding to the host cell receptor angiotensin-converting enzyme 2 (ACE2). The interaction between RBD and ACE2 plays a critical role in viral entry, making the SARS-CoV-2 S1 subunit a primary target for vaccine development (10–12).

The mRNA vaccine BNT162b2 (Pfizer BioNTech; hereafter referred to as BNT) and the vector vaccine ChAdOx1 nCoV-19 (Oxford-AstraZeneca; hereafter referred to as ChAd), were among the first COVID-19 vaccines authorized by the European Medicines Agency (EMA) between December 2020 and April 2021 (13, 14). Both vaccines, which encode the full-length S protein of SARS-CoV-2, have demonstrated high vaccine efficacy (VE) in clinical trials (11, 15–17).

Concerns about the durability of immunity and the ability of emerging SARS-CoV-2 variants of concern (VOCs) to evade immune protection have led to ongoing efforts to improve vaccination strategies. In December 2021, the World Health Organization (WHO) and EMA recommended the use of heterologous ‘prime-booster vaccination’ using different types of COVID-19 vaccines for the first and second doses; also known as the “mix-and-match” approach (18–20).

This decision was based on interim results from several clinical trials suggesting that heterologous vaccination results in a stronger and longer-lasting immune response. In particular, the combination of vector and mRNA vaccines appeared to induce higher levels of neutralizing antibodies than homologous vaccination with the same type of vaccine (21–26). In July 2022, the ECDC and EMA updated their public health recommendation, suggesting a second booster dose at least 4 months after the first (27). Those who received two doses of vector vaccine could receive a third dose of mRNA vaccine. Others received a homologous triple vaccination with three doses of mRNA vaccine, resulting in a heterogeneous vaccinated population.

Serological and cell-based assays are two common approaches used to quantify immune response and immune protection following vaccination (28, 29). While cell-based assays measure cellular immune responses such as T-cell proliferation or cytokine production, serological assays allow for the rapid and cost-effective quantification of SARS-CoV-2-specific antibodies in human serum. Therefore, serological assays are more suitable for routine diagnostics and high-throughput analysis in clinical laboratories. Serological tests can provide valuable information on VE and the durability of antibody protection, helping to identify individuals with suboptimal immune responses who may benefit from alternative vaccination strategies (30).

Neutralizing antibodies against the SARS-CoV-2 S protein are particularly important for assessing VE and predicting immune protection in individuals (31). High nAb titers have been associated with a lower risk of SARS-CoV-2 infection and severe COVID-19 disease. Several studies have shown that individuals with higher levels of neutralizing antibodies are less likely to develop symptomatic COVID-19 following natural infection or vaccination (32–36).

Neutralizing antibody titers have therefore been proposed as a correlate of protection (CoP) from SARS-CoV-2 (37). In a systematic review, Perry et al. found a robust correlation between vaccine-induced antibody levels and VE, despite the profound heterogeneity in vaccination regimens, serological assays, VE endpoints, and populations. The authors conclude that humoral immunity is an integral part of protection against COVID-19 and propose anti-S or neutralizing antibody levels as the most likely immune marker for a SARS-CoV-2 CoP (38).

In 2020, the WHO established the First International Standard (IS) for anti- SARS-CoV-2 immunoglobulin (NIBSC Code: 20/136) to harmonize serological test results worldwide (39–41). Reference standards are intended to improve the accuracy, reliability, and reproducibility of serological tests and facilitate the intercomparison of measurements obtained with different assays and detection methods in different laboratory settings worldwide (42). Lack of standardization can lead to the inaccurate interpretation of serological results, hampering effective disease surveillance and vaccine development (43).

In this study, we compared three SARS-CoV-2 S1-specific routine immunoassays for their ability to monitor humoral immune response and immune protection in a heterogeneous vaccination cohort. Subjects received different homologous and heterologous three-dose vaccination regimens over a period of 8.5 months. The serological tests, which differ in assay method (ELISA, ECLIA, and PETIA), antigens (full S1 subunit vs. RBD only), and isotype specificity (IgG vs. total Ig) were compared using Spearman’s rank correlation and Passing–Bablok regression. To define an universal cut-off for immune protection, suitable for real-world settings, we correlated anti-S1/RBD antibody titers (in BAU/ml) with neutralization potential (percentage inhibition of RBD-ACE2 interaction) as assessed by a surrogate virus neutralization test (sVNT).

In summary, our results indicate that all S1-specific binding assays facilitate monitoring of the antibody response in vaccinated individuals. All assays resulted in similar antibody kinetics, and increasing antibody titers were associated with increasing inhibitory potential. However, we were unable to define a clear cut-off value in BAU/ml across all methods that would help distinguish subjects above and below 35 IH%, mainly due to the large proportional differences between the binding assays.

The heterogeneity of vaccination schemes and individual immune responses adds another layer of complexity that further complicates the definition of a common threshold in BAU/ml. As previously reported, homologous prime-boost vaccination (ChAd-ChAd) appears to result in lower antibody titers and neutralizing potential than heterologous vaccination (ChAd-BNT). For example, 5 months after the second dose (TP9), all subjects are >100 BAU/ml for ECLIA, whereas all ChAd-ChAd vaccinated subjects are <100 BAU/ml when measured by PETIA, although both are anti-RBD total Ig assays that should correlate comparably with sVNT ( Supplementary Figure S7 ). Interestingly, the substantial decline of S1-specific antibodies observed within 5 months after the first booster vaccination did not reflect a similar decline in inhibitory potential (sVNT IH%). It should be noted that neutralizing potential has been reported for anti-S1 antibodies raised against epitopes outside the RBD, whereas surrogate neutralization assays such as sVNT are limited to neutralizing anti-RBD antibodies. This may partially explain the observed differences in antibody waning between ECLIA/PETIA (anti-RBD) and ELISA (anti-S1) after the first booster.

The adaptive immune response to SARS-CoV-2 infection results in the activation and clonal expansion of virus-specific B cells. These differentiate into plasma cells and secrete soluble immunoglobins (Ig) into circulation that have different affinities for different viral proteins, mainly the viral spike (S) and nucleocapsid (N) protein in the case of SARS-CoV-2. Although pentameric IgM (low affinity/high avidity) provides the first line of defense, the subsequent seroconversion and production of high-affinity IgG are critical for long-term immune protection (49, 50). Neutralizing antibodies can inhibit the essential interaction between the RBD, located in the S1 subunit of the viral S protein, and the host cell receptor ACE2. Of note, IgA antibodies, which are responsible for mucosal immune defense, have been reported to exhibit ever higher neutralizing potential against SARS-CoV-2 than IgG antibodies (51). Immunization with vector or mRNA vaccines, on the other hand, results in S protein-specific antibodies raised against various epitopes in the S1 and S2 subunits.

The ability of different heterologous and homologous ChAd/BNT vaccination schemes to reduce SARS-CoV-2 infections or severe COVID-19 cases (VE) has been studied in large clinical trial populations (21–26, 52). A correlate of protection (CoP), on the other hand, is a measurable parameter that allows the prediction of immune protection in vaccinated individuals. Although spike-specific antibody titers have been proposed as a promising CoP for COVID-19, it is challenging to define which antibody titers are sufficient for immune protection (34, 37, 53). Several groups compared the antibody response in different ChAd/BNT vaccination cohorts using different routine binding assays (49, 54–56). These assays vary widely in antigen and isotype specificity as well as assay design and detection method. Spaeth et al. and Brehm et al. compared the performance of different N- and S-specific assays in SARS-CoV-2- positive subjects and patients with mild COVID-19 disease, respectively (57, 58). Here, we compared three anti-S1 binding assays in a heterogeneous vaccination cohort that use different parts of the same spike protein S1 subunit as antigen (RBD vs. full S1). It was not the aim of this study to compare the efficacy of different prime-boost vaccination regimens. However, it is worth noting that our results are consistent with previous reports, as homologous prime-boost vaccination with ChAd-ChAd seems to result in lower antibody titers than vaccination with ChAd-BNT or BNT-BNT (TP5). These differences are almost equalized after the third vaccination with BNT (TP11; Supplementary Figure S7 ) (59–63).

According to the WHO, an arbitrary unit of 1,000 BAU/ml can be used to assist the comparison of binding assays that detect “the same class of immunoglobulins with the same specificity” (45). Therefore, we asked ourselves the following question: How similar must anti-S1 binding assays be— in terms of isotype discrimination and assay principle— to meet this definition?

The ELISA assay (Euroimmun) detects IgG antibodies raised against the entire S1 subunit, whereas the ECLIA and PETIA assays both detect anti-RBD IgG, IgM, and IgA antibodies. Accordingly, PETIA and ECLIA show the smallest proportional difference, whereas ELISA and ECLIA show the lowest correlation and the largest proportional difference. Interestingly, PETIA and ECLIA interact similarly with the convalescent plasma-derived WHO IS (Roche U/ml = BAU/ml and Diasys AU/ml = BAU/ml) but yielded considerably different BAU/ml values in the heterogeneous vaccination cohort ( Supplementary Figure S1 , Supplementary Table S2 ) (39, 40). This might be explained by the difference in assay principles: in the Elecsys^® Anti-SARS-CoV-2 S assay (Roche), RBD-specific IgG, IgM, and IgA bind to a mix of biotinylated and ruthenylated RBD antigen. The resulting double-antigen sandwich (DAGS) complexes are immobilized on the solid phase via streptavidin-coated microparticles and quantified by electrochemiluminescence measurement. In the turbidimetric SARS-CoV-2 UTAB FS PETIA assay (Diasys), the RBD-antigen is coupled to polystyrene beads and binds to IgM, IgG, and IgA antibodies in the sample. The homogeneous PETIA assay can be more prone to non-specific reactions than heterogeneous technologies such as ELISA or (E)CLIA (64). Pentameric IgM tends to form larger antigen-antibody complexes than monomeric IgG or IgA and may result in a higher signal (65). In conclusion, the differential interaction of all three assays with the reference material, which is derived from SARS-CoV-2 infected individuals, versus vaccine serum strongly suggests that an alternative approach is required to harmonize different anti-S1 assays in a vaccination cohort.

The clinical benefits and intrinsic limitations of serological SARS-CoV-2-specific immunoassays are still vividly discussed (30, 66, 67). In particular, the repeated emergence of highly mutated VOCs, such as the Omicron variants, raised concerns that commercially available binding assays may become obsolete too quickly (66, 68, 69). More than thirty alterations have been identified within the Omicron spike protein, resulting in significantly reduced anti-RBD antibody binding and immune evasion (70–72). Wey et al. recently reported that the RBD-specific PETIA assay can quantify the antibody response to Alpha (B.1.1.7) and Kappa (B.1.617.1), while cross-reactivity to Omicron (B.1.1.529) is reduced by approximately 50% compared with wild-type virus and all other VOCs (64). In this study, we analyzed serum from subjects vaccinated in 2020/21, before the emergence of Omicron variants. We did not systematically compare the performance of all four binding assays in serum from patients infected with SARS-CoV-2 VOCs.

Other groups pointed out the inherent limitations of harmonization to the WHO IS, especially for SARS-CoV-2 binding assays that differ significantly in target antigen (N vs. S protein) and isotype specificity (IgG vs. IgM) (40, 59, 73–75). However, the early and widespread adoption of the WHO standard and the wide availability of conversion factors for commercial SARS-CoV-2 assays led to the following erroneous conclusion: conversion to BAU/ml allows the harmonization of two given SARS-CoV-2 binding assays. Of note, the WHO Expert Committee on Biological Standardization expressed concern that assigning the same unitage for binding assays based on different antigens would allow for an inappropriate use of the WHO IS (76). Our results confirm the distinct behavior of different anti-S1 binding assays: 1) assays that discriminate by isotype (IgG specific) but less by epitope (whole S1 subunit), and 2) assays that are more epitope specific (RBD only), but less isotype specific (total antibodies). Therefore, SARS-CoV-2 binding assays with the same antigen specificity and similar interaction with the WHO IS interact differently with vaccine serum. Furthermore, the correlation and proportional differences between ECLIA (heterogenous double-antigen sandwich assay) and PETIA (homogeneous turbidimetric assay) seem to change during the 8.5-month vaccination course, indicating a high susceptibility to the serum immunoglobulin composition (changing IgG/IgM/IgA ratio) ( Supplementary Figure S7 ). This discrepancy, which is most likely due to different assay principles, adds another item to the list of hurdles we must overcome if we are to achieve proper harmonization of binding assay results, especially in populations that have received different vaccine regimens of varying efficacy.

Although all COVID-19 vaccines are based on the full-length S protein, the presentation of antigen-derived peptides is strikingly different —not only between protein-based and nucleic acid-based vaccines but also between mRNA (BNT) and vector (ChAd) vaccines. This in turn leads to different CD8+ and CD4+ T cell activation, which shapes the subsequent antibody response (77–80). It is conceivable that the heterogeneity of current vaccine platforms negatively affects the comparability of binding assays that detect total antibodies directed against the same antigen.

A viable way to overcome this limitation in the future would be to harmonize against the material of defined antigen specificity and/or isotype composition. Of note, Freeman et al. characterized five antigen-specific fractions of a serum-based reference material, containing antibodies against the SARS-CoV-2 S protein (anti-S1/S2, -S1, -S2, and anti-RBD) and N protein, for the standardization of IgG and total Ig serological assays (81). Consistent with our observations, the anti-RBD IgG assay (sCOVG) values were approximately the same for anti-RBD and anti-S1 sera, whereas the anti-RBD total antibody assay (COV2T) values were doubled for anti-S1 serum. Interestingly, the authors conclude that it is unlikely that IgM and IgA antibodies contribute to this discrepancy, as the serotiters of both have been reported to decline substantially by 6 weeks after symptom onset when sample collection for the reference material began (80). Nevertheless, the substantial discrepancy between the two anti-RBD total antibody assays, which varies over the course of vaccination in our cohort ( Supplementary Figure S7 ), may still be due to differences in the two detection methods (PETIA vs. ECLIA).

The serum samples used for assay comparison were derived from a small non-representative cohort (n = 76) with a high female/male ratio and variable sample size per time point (n = 41–68; 32 subjects with ≥10 samples). Therefore, we do not report any significant difference between vaccination regimes, nor do we draw any conclusions about the superiority of heterologous prime-boost vaccination (ChAd-BNT vs. ChAd-ChAd). However, this assay comparison study has several limitations. Owing to the limited sample volume provided by the WHO, the 7-point WHO standard dilutions (3.9–250 BAU/ml) were assayed in singlets, which limits the accuracy of measurement. In addition, neutralizing antibodies were assessed by sVNT (inhibition of the RBD-ACE2 interaction), which does not reflect antiviral activity in vivo. However, surrogate assays are the only feasible way to estimate the neutralizing capacity of serum samples in clinical routine. The gold standard plaque reduction neutralization test (PRNT) is labor-intensive, time-consuming, and requires Biosafety Level 3 (BSL-3) facilities. Furthermore, sVNT is limited to neutralizing anti-RBD antibodies as detected by ECLIA and PETIA, whereas the ELISA assay detects antibodies against the entire S1 subunit (6–8).

Vaccine efficacy must be assessed using gold standard methods and studies must demonstrate a significant reduction in COVID-19 cases and/or severe disease progression in large study populations. However, in routine clinical practice, we must rely on cost-effective surrogate markers and surrogate assays to assess and evaluate the individual immune response in vaccinated individuals. Nevertheless, it is still under debate which marker(s) should be used for monitoring and what cut-off indicates adequate immune protection.

As discussed above, an in vitro CoP will never accurately predict vaccine efficacy and vaccine-induced immune protection in individuals, especially for highly evolving viruses, such as Coronaviridae. The recent COVID-19 pandemic has highlighted the need for rapid and flexible vaccine development and manufacturing, to ensure immune protection against emerging VOCs.

In summary, our results underscore the urgent need for rapidly evolving technology, not only for vaccines but also for serological binding assays, and for the continued development of both — bioanalytical methods and dedicated higher-order reference materials— to keep pace with rapidly mutating viruses. For future viral pandemics, if we are to use total antibody assays to monitor the vaccine-induced immune responses and predict immune protection in vaccinated individuals, we should strive to be more aware of method-specific differences and focus on the development of higher-order reference standards. Each reference material should be appropriate for the diagnostic task at hand, e.g., monitoring the antibody response post-infection versus post-vaccination.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving humans were approved by Ärztekammer Nordrhein, Tersteegenstr. 9, 40474 Düsseldorf, Germany. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

SS: Formal Analysis, Writing – original draft, Writing – review & editing, Investigation, Methodology, Visualization. CL: Writing – review & editing, Conceptualization, Data curation, Supervision. YS: Conceptualization, Data curation, Supervision, Writing – review & editing, Resources. BK: Writing – review & editing, Investigation, Methodology. JA: Investigation, Methodology, Writing – review & editing. HS: Investigation, Methodology, Writing – review & editing, Validation. TM: Investigation, Validation, Writing – review & editing, Data curation, Formal Analysis. CP: Validation, Writing – review & editing, Resources. MG: Resources, Writing – review & editing, Conceptualization, Data curation, Formal Analysis, Supervision, Writing – original draft.