A total of 120 residual plasma samples from clinical testing for EBV and HCMV at the Department of Laboratory Medicine, Peking Union Medical College Hospital were used, with concentrations ranging from 3 × 10 IU/mL to 5 × 10⁷ IU/mL (82 positive and 38 negative cases). Additionally, 121 residual oropharyngeal swab samples were used for RSV testing (90 positive and 31 negative cases). Reference materials included the 1st WHO International Standard for EBV (NIBSC code: 09/260, concentration 5 × 10⁶ IU/mL), the 1st WHO International Standard for HCMV for Nucleic Acid Amplification Techniques (WHO HCMV standard, NIBSC code: 09/162, concentration 5 × 10⁶ IU/mL), and the RSV Nucleic Acid Detection National Reference Material (code:370057-202001, concentration 3 × 10⁸ U/mL). All clinical specimens were tested in parallel using both the evaluated automated molecular detection system and the routine real-time quantitative PCR (RT-qPCR) platform currently employed in our hospital, which served as the reference standard. Diagnostic performance was assessed based on the results of the reference method.

To evaluate the long-term operational stability of the system, a 168-h continuous operation stress test was conducted in accordance with the Technical Guidelines for the Reliability Evaluation of Fully Automated Nucleic Acid Purification and Amplification Detection Systems issued by the National Medical Device Quality Supervision Authority. From April 19 to April 25, 2024, the instrument (PANA HM9000; Serial No.: TL52NL22120001) was continuously powered on for seven consecutive days in the manufacturer’s internal quality control laboratory. During each 24-h cycle, the system completed one full-capacity testing run, processing approximately 2000 samples per day, simulating maximum daily workload. Throughout the testing period, system status, error logs, and output quality were continuously monitored. Key performance indicators—including fault occurrences, interruption events, and error-free run rates—were recorded to assess consistency and robustness under sustained high-throughput conditions.

The following detection kits were used: EBV DNA detection kit (manufacturer: Suzhou Tianlong Technology, batch number: 20240409), HCMV DNA detection kit (manufacturer: Suzhou Tianlong Technology, batch number: P811240412), and RSV RNA qualitative detection kit (manufacturer: Suzhou Tianlong Technology, batch number: P1982326001). For nucleic acid extraction or purification, the quantitative nucleic acid extraction instrument was paired with the corresponding extraction or purification reagents (manufacturer: Xi’an Tianlong Technology, registration number: Shanxi Medical Device 20210023), and the qualitative nucleic acid extraction instrument was paired with its corresponding extraction or purification reagents (manufacturer: Xi’an Tianlong Technology, registration number: Shaanxi Medical Device 20210030). The PANA HM9000 Automated Molecular Detection Streamline (manufacturer: Xi’an Tianlong Technology; note: the subject of evaluation).

The qualitative performance of the detection system was evaluated by calculating the concordance rate between test results generated by the system and those obtained from a reference method using clinically characterized residual samples. This evaluation followed the CLSI EP12 guidelines, which provides a structured framework for assessing binary output examinations, such as positive/negative results, by comparing test outcomes against established reference categorizations (Clinical and Laboratory Standards Institute, 2023; Hiraishi et al., 2021). According to EP12, concordance rate serves as a fundamental indicator of agreement and reliability for qualitative assays, particularly when sensitivity, specificity, and predictive values are not the primary focus. In this study, all qualitative results for EBV, HCMV, and RSV were classified into binary outcomes and compared with results from the hospital’s validated RT-qPCR platform to determine positive, negative, and overall agreement.

Accuracy evaluation was conducted in accordance with the CLSI guideline EP09, which provides a structured approach for assessing bias through measurement procedure comparison using patient samples (Clinical and Laboratory Standards Institute [CLSI], 2018b). Following this guideline, Using the WHO EBV standard and WHO HCMV standard, EBV DNA and HCMV DNA negative plasma samples were diluted into five concentration gradients for testing (EBV: 5 × 10⁵ IU/mL, 5 × 10⁴ IU/mL, 5 × 10³ IU/mL, 5 × 10² IU/mL, 5 × 10 IU/mL; HCMV: 1 × 10⁶ IU/mL, 1 × 10⁵ IU/mL, 1 × 10⁴ IU/mL, 1 × 10³ IU/mL, 1 × 10² IU/mL). For each concentration, samples were extracted three times, and each extraction well was tested once. The accuracy was determined by comparing the mean detection values with the theoretical clinical values.

Linearity was assessed according to the CLSI guideline EP06, which provides a structured approach for evaluating whether a quantitative measurement procedure yields results that are directly proportional to the true analyte concentrations across a specified interval (Clinical and Laboratory Standards Institute [CLSI], 2020). Using the WHO EBV standard and WHO HCMV standard, EBV DNA, and HCMV DNA negative plasma samples were diluted into five concentration gradients for testing (EBV/HCMV: 1 × 10⁶ IU/mL, 1 × 10⁵ IU/mL, 1 × 10⁴ IU/mL, 1 × 10³ IU/mL, 1 × 10² IU/mL). Each concentration was extracted three times, and each extraction well was tested once. The logarithmic mean of the detected concentrations and the logarithmic values of the dilution ratios were linearly fitted, and the linear correlation coefficient (| r|) was calculated.

Intra-assay precision was evaluated following the CLSI guideline EP05, which outlines standardized protocols for assessing the repeatability and within-laboratory precision of quantitative measurement procedures (Clinical and Laboratory Standards Institute [CLSI], 2014). According to EP05, precision studies aim to characterize the variability under normal operating conditions across multiple runs and replicates. Using the WHO EBV standard, WHO HCMV standard, and the RSV Nucleic Acid Detection National Reference Material, EBV DNA, HCMV DNA, and RSV RNA negative samples were diluted to three concentration gradients—high, medium, and low (EBV DNA: 2 × 10⁴ IU/mL, 2 × 10³ IU/mL, 2 × 10² IU/mL; HCMV DNA: 1 × 10⁴ IU/mL, 1 × 10³ IU/mL, 1 × 10² IU/mL; RSV RNA: 1 × 10⁴ copies/mL, 2 × 10³ copies/mL, 6 × 10² copies/mL). Each concentration was tested 10 times, and the coefficient of variation (CV,%) was calculated (Plesser, 2018).

Evaluate Inter-Assay Precision according to CLSI EP05 guidelines (Clinical and Laboratory Standards Institute [CLSI], 2014). Using the WHO EBV standard, WHO HCMV standard, and the RSV Nucleic Acid Detection National Reference Material, EBV DNA, HCMV DNA, and RSV RNA negative plasma samples were diluted to three concentration gradients-high, medium, and low (EBV DNA: 2 × 10⁴ IU/mL, 2 × 10³ IU/mL, 2 × 10² IU/mL; HCMV DNA: 1 × 10⁴ IU/mL, 1 × 10³ IU/mL, 1 × 10² IU/mL; RSV RNA: 1 × 10⁴ copies/mL, 2 × 10³ copies/mL, 6 × 10² copies/mL). Each concentration was tested 5 times per day for 4 consecutive days, and the coefficient of variation (CV,%) was calculated (Goodman et al., 2016).

The limit of detection (LoD) was evaluated in accordance with CLSI guideline EP17, which defines LoD as the lowest analyte concentration that can be reliably distinguished from background noise and detected with a 95% probability. Following EP17 recommendations, using the WHO EBV standard, WHO HCMV standard, and the RSV Nucleic Acid Detection National Reference Material, EBV DNA, HCMV DNA, and RSV RNA negative samples were diluted to target concentration levels for detection (EBV DNA: 50 IU/mL, 35 IU/mL, 20 IU/mL, 10 IU/mL, 5 IU/mL; HCMV DNA: 60 IU/mL, 30 IU/mL, 20 IU/mL, 10 IU/mL; RSV RNA: 200 copies/mL, 150 copies/mL, 100 copies/mL). The detection rates at different concentrations were calculated, with the detection rate for the LoD required to be ≥ 95%.

Interference testing was conducted in accordance with CLSI guideline EP07, which provides a standardized approach to identifying and evaluating the effects of endogenous and exogenous substances that may alter the accuracy of measurement procedures. The goal is to determine whether potential interferents cause significant positive or negative bias in the test results (Clinical and Laboratory Standards Institute [CLSI], 2018a). For the EBV interference testing, weakly positive EBV DNA clinical samples (40 IU/mL) were divided into six groups. Five groups were spiked with the following interfering substances: 0.6 mg/mL bilirubin, 120 μG/mL acyclovir, 150 mg/mL hemoglobin, 60 μG/mL streptomycin, and 60 mg/mL triglycerides. The sixth group was mixed with an equivalent volume of saline as a control. Each sample was extracted three times, and each extraction well was tested once (Hays et al., 2022). For HCMV interference testing, weakly positive HCMV DNA clinical samples (60 IU/mL) were divided into five groups. Four groups were spiked with potential interfering substances: 0.2 mg/mL bilirubin, 10.4 μG/mL ganciclovir, 2.0 mg/mL hemoglobin, and 33 mg/mL triglycerides, respectively. The fifth group was supplemented with an equivalent volume of saline as a control. Each sample was extracted three times, and each well was tested once (Hays et al., 2022). For the RSV interference testing, weakly positive RSV RNA clinical samples (400 copies/mL) were divided into ten groups. Nine groups were spiked with the following interfering substances: 0.45 mg/mL azithromycin, 1% whole blood, 100 μG/mL oxymetazoline hydrochloride nasal spray, 100 μg/mL triamcinolone nasal spray, 16.5 g/L hematin, 18 g/L mucin, 2 mg/mL dexamethasone, 658.5 ng/mL arbidol hydrochloride, and 9 mg/mL sodium chloride. The tenth group was mixed with an equivalent volume of saline as a control. Each sample was extracted three times, and each extraction well was tested once (Hays et al., 2022). For all tests, the difference in Ct values (|DeltaCt|) between the experimental groups with interferents and the control group without interferents was required to be ≤ 1.

Evaluate Cross-Reactivity according to CLSI EP07 guidelines (Clinical and Laboratory Standards Institute [CLSI], 2018a). For the cross-reactivity test, EBV DNA negative clinical samples were divided into seven groups. Six groups were spiked with samples infected with various pathogens (≥ 10⁵ pfu/mL), including Hepatitis B virus (HBV), HCMV, adenovirus (AdV), Influenza A virus (IAV), Candida albicans, and Staphylococcus aureus (SA). The seventh group was mixed with an equivalent volume of saline as a control. For the cross-reactivity test, HCMV DNA negative clinical samples were divided into six groups. Five groups were spiked with samples infected with various pathogens (≥ 10⁵ pfu/mL), IAV, AdV, EBV, RSV, and Mycoplasma pneumoniae (MP). The sixth group was mixed with an equivalent volume of saline as a control. For the cross-reactivity test, RSV RNA negative clinical samples were divided into 12 groups. Eleven groups were spiked with samples infected with various pathogens (≥ 10⁵ pfu/mL), including IAV, Influenza B virus (IBV), AdV, MP, rhinovirus (RV), coronavirus (CoV), parainfluenza virus (PIV), Streptococcus pneumoniae (SPN), SA, EBV, and HCMV. The twelfth group was mixed with an equivalent volume of saline as a control. Each sample in all tests was extracted three times, and each extraction well was tested once. The detection result was considered acceptable if all results were negative.

Carryover contamination was evaluated following CLSI guideline EP47, which provides a structured framework for assessing reagent carryover effects in clinical measurement procedures. Reagent carryover is defined as the unintended transfer of analytes or reagents between test samples that may cause false-positive results or quantitative bias. EP47 recommends alternating high- and low-analyte concentration samples to simulate worst-case scenarios and evaluate the potential for cross-contamination (Clinical and Laboratory Standards Institute [CLSI], 2024). For EBV and HCMV, high-concentration positive clinical samples (≥ 1 × 10⁵ IU/mL) and negative samples were used. For RSV, high-concentration positive clinical samples (approximately 1 × 10⁵ IU/mL) were used. The samples were processed in alternating sequences of positive and negative samples for extraction. Each sample was extracted once, and each extraction well was tested once, with eight repetitions of the test. The detection result is considered acceptable if all negative samples test negative and all positive samples test positive.

Experimental data were analyzed using SPSS version 27.0 and R version 4.4.1 statistical software. Statistical analysis of the concordance rate was performed using SPSS version 27.0. Descriptive statistics were initially used to summarize the distribution of detection results. Cohen’s Kappa coefficient was calculated to assess the agreement between test kit results and diagnostic outcomes across different groups. Asymptotic standard error and approximate t-tests were used to evaluate the statistical significance of the Kappa values. To visually represent the performance metrics of the detection system, we used R packages such as “ggplot2” and “cowplot” to create effective and high-quality visualizations.

From April 19 to April 25, 2024, the instrument (Model: PANA HM9000; Serial No.: TL52NL22120001) was operated continuously for 7 days, with one full-capacity run (24 h) conducted each day. In each run, the system processed approximately 2,000 samples, simulating its maximum daily workload. All seven runs were completed successfully without any interruptions or error events. No system failures or responsible faults were recorded during the entire testing period, and all sample results met the predefined quality control standards.

The detection system was used to test 120 plasma samples and 121 oropharyngeal swab samples. The results showed that the positive concordance rate for EBV DNA, the positive concordance rate was 100% (31/31) and the negative concordance rate was 100% (9/9). For HCMV DNA was 100% (51/51), with a negative concordance rate of 100% (29/29). For RSV RNA, the positive concordance rate was 100% (91/91) and the negative concordance rate was 100% (30/30). The positive concordance rate, negative concordance rate, and overall concordance rate for all three nucleic acid detection kits were 100% (Kappa = 1, P < 0.001).

The experimental results showed that the absolute logarithmic deviations for the standard concentrations of both EBV DNA and HCMV DNA were within the ± 0.5 log unit range. A strong correlation was observed between the measured and theoretical concentrations for both EBV DNA and HCMV DNA, with the linear regression line closely aligning with the theoretical line (Figure 1).

The logarithmic values of the diluted standard samples were used, and the average slope method was applied to verify the linearity. The logarithmic means of the measured linear sample concentrations and the logarithmic values of the dilution ratios were linearly fitted, showing a high degree of linear correlation. The linear regression equation for EBV DNA was y = –0.988 x + 5.9639, with a correlation coefficient of R² = 0.9998, indicating a quantitative detection linear correlation coefficient |r| ≥ 0.98. The linear regression equation for HCMV DNA was y = –0.966 x + 5.7868, with a correlation coefficient of R² = 0.9999, also indicating a quantitative detection linear correlation coefficient |r| ≥ 0.98 (Figures 2A,B).

The results showed that the intra-assay precision for EBV DNA at high, medium, and low concentrations (2 × 10⁴ IU/mL, 2 × 10³ IU/mL, 2 × 10² IU/mL) was 1.76, 1.21, and 2.44%, respectively. For HCMV DNA, the intra-assay precision at high, medium, and low concentrations (1 × 10⁴ IU/mL, 1 × 10³ IU/mL, 1 × 10² IU/mL) was 0.94, 1.57, and 4.63%, respectively. For RSV RNA, the intra-assay precision at high, medium, and low concentrations (1 × 10⁴ copies/mL, 2 × 10³ copies/mL, 6 × 10² copies/mL) was 0.51, 0.61, and 0.72%, respectively. The coefficient of variation (CV,%) for the logarithmic values of the detected concentrations was ≤ 5% for all samples (Figure 2C).

The results showed that the inter-assay precision for EBV DNA at high, medium, and low concentrations (2 × 10⁴ IU/mL, 2 × 10³ IU/mL, 2 × 10² IU/mL) was 1.62, 1.68, and 3.69%, respectively. For HCMV DNA, the inter-assay precision at high, medium, and low concentrations (1 × 10⁴ IU/mL, 1 × 10³ IU/mL, 1 × 10² IU/mL) was 0.79, 1.10, and 2.89%, respectively. For RSV RNA, the inter-assay precision at high, medium, and low concentrations (1 × 10⁴ copies/mL, 2 × 10³ copies/mL, 6 × 10² copies/mL) was 0.64, 0.73, and 0.67%, respectively. The coefficient of variation (CV,%) for the logarithmic values of the detected concentrations was ≤ 5% for all samples (Figure 2D).

The results showed that for EBV DNA, the detection rates were 100% at concentrations of 50 IU/mL, 35 IU/mL, and 20 IU/mL; 95% at 10 IU/mL; and 75% at 5 IU/mL. The detection rate for the sample concentrations must be ≥ 95%, establishing the LoD for EBV DNA at 10 IU/mL. For RSV RNA, the detection rates were 100, 80, and 35% at concentrations of 200 copies/mL, 150 copies/mL, and 100 copies/mL, respectively. The detection rate for the sample concentrations must be ≥ 95%, establishing the LoD for RSV RNA at 200 copies/mL. The results for HCMV DNA showed a detection rate of 100% at concentrations of 60 IU/mL, 30 IU/mL, 20 IU/mL, and 10 IU/mL. The detection rate for the sample concentrations must be ≥ 95%, establishing the LoD for HCMV DNA at 10 IU/mL (Figure 3).

The detection results for EBV, HCMV, and RSV in the experimental groups with added interfering substances compared to the control groups without interferents showed that |ΔCt| was ≤ 1 for all tests (Figure 4). For EBV, a variety of substances were tested, including 0.6 mg/mL bilirubin, 120 μg/mL acyclovir, 150 mg/mL hemoglobin, 60 μG/mL streptomycin, and 60 mg/mL triglycerides. The |ΔCt| values of 0.11, 0.29, 0.01, 0.17, and 0.18, respectively, all well below the threshold (Figure 4A). Similarly, for HCMV detection, interferents included 0.2 mg/mL bilirubin, 10.4 μg/mL ganciclovir, 2.0 mg/mL hemoglobin, and 33 mg/mL triglycerides. The |ΔCt| values of 0.24, 0.15, 0.04, and 0.06, respectively, all within the acceptable range (Figure 4B). For RSV, the interfering substances included 0.45 mg/mL azithromycin, 1% whole blood, 100 μg/mL oxymetazoline hydrochloride nasal spray, 100 μG/mL triamcinolone nasal spray, 16.5 g/L hematin, 18 g/L mucin, 2 mg/mL dexamethasone, 658.5 ng/mL arbidol hydrochloride, and 9 mg/mL sodium chloride. The |ΔCt| values of 0.23, 0.42, 0.02, 0.1, 0.01, 0.29, 0.29, 0.28, and 0.43, respectively, all well within the acceptable range (Figure 4C).

The detection results were negative after adding cross-reactive substances to HCMV DNA, RSV RNA, and EBV DNA clinical samples that initially tested negative.

Testing was conducted by alternating between positive samples (at specific concentrations) and negative samples. The results showed that positive samples tested positive and negative samples tested negative, demonstrating that no carryover contamination occurred.