The workflow of this study is depicted in Figure 1 . A total of 199 BALF samples were collected from 190 patients admitted to the Department of Pulmonary and Critical Care Medicine, First Affiliated Hospital of Sun Yat-sen University (FAH-SYU). The test results of routine laboratory tests, combined with all information collected during all medical procedures, served as a clinical reference standard for pulmonologists to make a conclusive diagnosis. The remaining samples were stored at -80°C until May 2023, divided into two parts, namely R1 and R2, and sent individually to Kingmed Diagnostics Group Co., Ltd., Guangzhou and VisionMedical Co., Ltd., Guangzhou for tNGS and mNGS. Eligibility criteria: 1) all patients presented typical symptoms of RTI; 2) remaining samples were qualified and sufficient for sequencing; 3) with complete basic information and demographic records. Due to the limitations of mNGS, DNA and RNA samples were extracted and sent for library construction, respectively. Both companies produced pathogen reports independently without knowledge of each other’s and results of any routine laboratory tests until the completion of all data collection. Qualified tNGS and mNGS pathogen reports underwent downstream comparative analysis and evaluation of diagnosis value.

Different tests were conducted as part of standard routine practice. These tests were selectively conducted based on pathogen suspicion or for exclusion purposes. Results serve as evidence to assist in making a diagnosis. Regular laboratory tests were conducted as part of standard clinical practice. This included conventional microbiological tests (CMTs), such as microbial culture and bacterial and fungal stains, as well as immunological and molecular tests (e.g., qPCR) for pathogen identification and disease diagnosis. These tests were performed by the Department of Laboratory Medicine at FAH-SYU. In addition, acid-fast bacilli staining and culture, Xpert MTB/RIF, and T-cell-based tests for tuberculosis infection (T-SPOT.TB) were also performed for cases with clinical suspicion of Mycobacterium tuberculosis (MTB). Each of these methods was developed for the identification of specific types of pathogens and was utilized based on the individual cases. For practical considerations, these tests were selectively conducted based on clinical suspicion and were not used indiscriminately on all samples. Thus, data of specific methods in some samples may not be available, but results of at least two microbial tests were employed for all samples. Intermediate results of individual methods would not be considered positive unless consistency was observed from at least two methods. Results of all tests were combined and taken into consideration to conclude a diagnosis. The positive calling criteria for each method are summarized in Supplementary Table 1 .

A composite reference standard for etiology identification was established to serve as a criterion for physicians to make initial clinical diagnoses. In brief, two experienced physicians independently reviewed the medical records, including the chart interview process, routine laboratory test results, treatment responses and other information from all medical procedures, to determine whether the patients had infections or non-infections and to identify potential causal pathogens. Disagreements between physicians were resolved through in-depth discussions, during which senior physicians participated until a consensus was reached.

Samples were sent to Kingmed Diagnostics Group Co., Ltd., Guangzhou, China for tNGS sequencing. In brief, the development of the tNGS panel first started with a rigorous and exhaustive survey of different types of reference materials, including academic papers, books, and expert consensus, to determine the target spectrum. A corresponding reference database consisting of representative sequences of these microbes described above was curated from NCBI NT database. For tNGS, the panel was designed to cover a spectrum of 153 microorganisms, including 68 viruses, 65 bacteria, 14 fungi, 3 chlamydiae, and 3 mycoplasmas. A full list of all target species was detailed in Supplementary Table 2 . Primer design was guided by several critical parameters, including GC content, primer length, amplicon size, melting temperature, and potential secondary structures, followed by a specificity check. For more details of it, please refer to Yin’s study (Yin et al., 2024). The BALF samples with high viscosity were diluted 1:1 with 0.1 M dithiothreitol prior to nucleic acid extraction. Automated nucleic acid extraction was performed using MagPure Pathogen DNA/RNA Kit B (Magen Biotechnology, Guangzhou, Guangdong, China) on KingFisher™ Flex Purification System (Thermo Fisher Scientific, Waltham, MA, USA). Nuclease-free water (Invitrogen, Waltham, MA, USA) was used as a non-template control to detect contamination.

Multiplex PCR pre-amplification of target loci and library preparation were performed using a respiratory pathogen test kit, RP100 (KingCreate, Guangzhou, Guangdong, China). Generated libraries were quantified using Equalbit DNA HS Assay Kit (Vazyme Biotech, Nanjing, Jiangsu, China) with an Invitrogen™ Qubit™ 3.0/4.0 (Thermo Fisher Scientific, Waltham, MA, USA) Fluorometer to ensure that all samples had library densities ≥ 0.5 ng/μL, otherwise, they were subjected to library reconstruction. DNA fragment analysis was performed using the Qsep100 Capillary Electrophoresis System™ (BiOptic Inc., Jiangsu, China) and its compatible Standard Cartridge Kit. After library qualification, sequencing was performed using KM MiniSeqDx-CN Sequencing Kit on the sequencing platform KM MiniSeq Dx-CN (KingCreate, Guangzhou, Guangdong, China).

Generated sequencing data was analyzed using a customized bioinformatic workflow. Generated sequencing raw read data underwent quality control procedures. The fastp v0.20.1 (Chen et al., 2018) was employed for adapter trimming and low-quality filtering using default parameters followed by reference-based assembly using Bowtie2 v2.4.1 (Langmead and Salzberg, 2012) in ‘very-sensitive’ mode. For a specific pathogen species or group, the normalized matched read count must reach matched reads per 100,000 reads (RPhK) ≥ 10 to be considered positive. Interpreters conducted the interpretation of the generated pathogen reports, often cooperating with bioinformaticians, before releasing the report.

Criterion was set up for selecting representative reference sequences of microorganisms from the NCBI NT and NCBI Genome databases. The pathogen list was determined in reference to Johns Hopkins ABX Guide, Manual of Clinical Microbiology, clinical case reports, and peer-reviewed articles, which was detailed in Liu’s study (Liu et al., 2022). Using the QIAamp^® UCP Pathogen DNA Kit (Qiagen, Nordrhein-Westfalen, Germany) and following the manufacturer’s instructions, DNA extraction was performed on all samples. Human DNA was eliminated from the samples by using Benzonase (Qiagen, Nordrhein-Westfalen, Germany) and Tween 20 (Sigma, Indiana, U.S.). Total RNA was extracted from the specimens by using the QIAamp^® Viral RNA Kit (Qiagen, Nordrhein-Westfalen, Germany) and ribosomal RNA was removed by utilizing the Ribo-Zero rRNA Removal Kit (Illumina, California, USA). The generated cDNA was obtained through reverse transcription (Thermo Fisher Scientific, Waltham, MA, USA).

The Nextera XT DNA Library Prep Kit (Illumina, San Diego, CA, USA) was utilized to create libraries for both the DNA and cDNA samples. The Qubit dsDNA HS Assay Kit (Thermo Fisher Scientific, Waltham, MA, USA) was used to evaluate the quality of the libraries, followed by the High Sensitivity DNA Kit (Agilent, California, USA) on the Agilent 2100 Bioanalyzer (Agilent, California, USA). Each library was subjected to 75 cycles of single-end sequencing on an Illumina NextSeq 550Dx Sequencer (Illumina, San Diego, CA, USA) to generate approximately 20 million reads. Deionized water was included in the extraction process as a negative control to function as a non-template control alongside the specimens.

Trimmomatic (Bolger et al., 2014) was used for low-quality read filtering and adapter trimming. The Burrows-Wheeler Aligner software (Li and Durbin, 2009) was used to exclude reads of human genomes in reference to the human reference genome hg38. For microbial species identification, a curated reference database consisting of > 20000 genomes spanning bacteria, viruses, fungi, protozoa, to other multicellular eukaryotic organisms was employed, as depicted in a previous study (Liu et al., 2022). Microbial reads were aligned to the database using SNAP v1.0 beta.1810 (Zaharia et al., 2011). Positive detection of a given species or genus was reported if the reads per million (RPM) ratio (Miller et al., 2019), or RPM-r, was ≥ 5. The RPM-r was defined as the RPM of the sample divided by the RPM of the negative control. To minimize cross-species misalignments among closely related microorganisms, a penalty mechanism was introduced (Gu et al., 2021). Bioinformaticians conducted the interpretation of the pathogen identification results, generated the pathogen report and approved it before release.

The initial pathogen reports generated from both NGS approaches underwent a pairwise blind review by two experienced physicians. The physicians scrutinized each sample and pathogen report carefully to identify any ambiguous results. After completing this step, the chief researchers unblinded the tNGS and mNGS reports to analyze the results in preparation for the next step. In cases of controversial conclusions concerning any pathogen or patient, a review of the entire workflow was conducted with input from pulmonologists, bioinformaticians, and lab technicians to reach a final conclusion, which was based on the data generated by all methods and sampling.

To assess the diagnosis value of NGS methods in clinical decision-making and ensure an intuitive understanding, all reports were evaluated by pulmonologists and assigned labels to represent NGS’s contribution to pathogen identification in various scenarios. The five labels used are Agree, Support/Extend, Disagree, Unlikely, and Noncontributory. In general, these labels express agreement on the reports grouped by disease status and qualify whether NGS aided in identifying etiology. Agree and Support/Extend present positive attitudes on cases of infectious samples, while Disagree and Unlikely present disagreement, and Noncontributory presents an intermediate attitudes towards clinically insignificant microbes. The criteria for each label are described in detail in Table 1 .

Wilcoxon test and Kruskal Wallis test were used to test the pairwise variance for all quantitative measurements (Andrews, 1954). The normal distribution of data was determined by the Shapiro-Wilk test prior to t-test (Shapiro and Wilk, 1965). Linearity between two variables was checked using the Spearman rank correlation test (Fieller et al., 1957). These analyses were all conducted using the corresponding functions implemented in base R 4.1.1. The standard mean difference (SMD) was used to measure the degree of variance given two groups of variables with the assistance of the R package smd (https://rdrr.io/cran/smd/). A SMD value of “0.20 ≤ estimate ≤ 0.50” is considered to indicate a small effect size (Faraone, 2008).

Positive percent agreement (PPA, percent of the concordant positives), negative percent agreement (NPA, percent of the concordant negatives), sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and 95% confidence interval (CI) were all manually calculated following the formulas described in previous studies (Altman and Bland, 1994a, Altman and Bland, 1994b; Bland and Altman, 1996; van Stralen et al., 2009). Percent agreement, PPA and NPA were used for evaluating the agreement of measurement between two NGS approaches. To estimate the clinical agreement of these qualitative tests, the conclusion of clinical diagnosis was used as the reference to determine true positives, true negatives, false positives, false negatives. Based on these, specificity, sensitivity, and accuracy were further calculated.

Sample size was estimated as below:

In this formula, N represents the total sample size, C denotes confidence level, Z_α is the critical value of the standard normal distribution corresponding to the significance level α, β represents the power level, and Z_β is the critical value of the standard normal distribution corresponding to the power level. The effect size d, computed as minimum mean difference divided by standard deviation.

Assuming a confidence level of 0.99, the value of Z_α is 2.576. Assuming a power level of 0.95, the value of Z_β is 1.645. Based on preliminary experimental data (not published), the estimated effect size is 32%. The minimum required sample size is calculated as:

The estimated sample size is 173. Considering the possible insufficiency of remaining samples and possibility of unqualified sequencing test, additional 10% was added to this estimate. The final expected sample size is 190.

A total of 199 samples were collected from 190 patients with suspected pneumonia, among which 180 eligible samples passed the enrollment criteria and were included in the following analysis. All samples were randomly selected and collected between July 2021 and October 2022. Although different types of samples were collected for routine clinical practice, only BALF samples were selected for this study. Basic information of the study population is summarized in Table 2 . All the patients included were over 18 years old, with ages ranging from 19 to 89 ( Supplementary Figure 1 ). The majority of study subjects (over 75%) were aged between 40 and 80. 68% of the selected patients were male and 32% were female. All patients had typical pneumonia symptoms. The proportions of patients with cough, expectoration, fever, and hemoptysis account for 72.22%, 52.78%, 39.44%, and 10.00%, respectively ( Table 2 ). Some patients had chronic diseases, such as diabetes, cardiovascular disease, hypertension, and issues in other organs. Patients with cardiovascular disease and hypertension represent the highest proportions (29.44% and 31.67%). BALF samples were first taken from these patients for selected laboratory tests to serve as part of routine practice for helping pulmonologists with decision making. After hospitalization/administration of all inpatients, the remaining samples were sent for nucleic acid extraction to facilitate downstream NGS analysis

The general performance comparison between tNGS and mNGS is summarized in Figure 2 . A significant positive correlation in detection frequencies per species was observed between tNGS and mNGS (Spearman test, P=1.39e-14, Figure 2A ). However, tNGS reported more positives than mNGS, as evidenced by the deviated fitted line and a correlation coefficient less than 1 (ρ _spearman =0.85, Figure 2A ). It is interesting that although tNGS has a much narrower spectrum (153 species, detailed in Supplementary Table 2 ) than mNGS (~20,000), tNGS reported significantly more potential pathogens per sample than mNGS. Such a difference was more pronounced when considering only the targets covered by both the tNGS panel and mNGS (P=1.374e-13, Figure 2B ). Such variance could be further demonstrated at organism level. An overall difference could be observed between tNGS and mNGS regarding detection frequencies (P-value=1.212e-05, Paired Wilcoxon test). In detail, tNGS exhibited higher detection frequencies for 32 organisms, equal detection frequencies for 12 organisms, and lower detection frequencies for 4 organisms when compared to mNGS. Ten species were identified exclusively by either tNGS or mNGS, each appearing in only one sample ( Figure 2C ). Additionally, mNGS exclusively reported 28 species that are not covered in the tNGS panel ( Supplementary Table 3 ). However, 25 of them were considered clinically insignificant by composite reference standard. Although the overall percent agreements of all shared targets showed high concordance between both methods (82.78% ~ 100%) ( Supplementary Table 4 ), the PPAs of some species were low (e.g. Mycobacterium avium complex at 42.86%). These results highlight the differences in detection prevalence between mNGS and tNGS.

In reference to diagnoses, the overall accuracy, sensitivity, specificity, and negative percent value (NPV) of tNGS/mNGS all exceeded 95%, while the overall PPVs were relatively low at 42.09% (CI_95%: 27.6%~56.6%) and 53.47% (CI_95%: 38.7%~68.2%) ( Table 3 ). Further inspection revealed that this may be due to the predominance of opportunistic pathogens, as PPVs of opportunistic pathogens by tNGS and mNGS were 15.71% and 21.88%, respectively. An extreme case is Human gammaherpesvirus 4, as only 1 positive was considered clinically relevant out of all 46 positives (for all pathogens, see Supplementary Table 5 ). On the other hand, four selected pathogens - Cryptococcus neoformans, Aspergillus fumigata, Mycobacterium tuberculosis complex, and Chlamydia psittaci - considered empirically to be highly pathogenic, had high PPVs (66.7% to 100%) ( Table 3 ). Agreement of positive results between clinical diagnosis and other methods regarding these species is summarized in Figure 2D . tNGS and mNGS confirmed most of the positive results identified by clinical diagnosis and outperformed other methods. Taking Mycobacterium tuberculosis complex as an example, tNGS, mNGS and Xpert reported 14, 11, and 10 positives out of 16 clinically positive samples, respectively. This was also observed in Aspergillus fumigatus, Cryptococcus neoformans, and Chlamydia psittaci ( Figure 2D ).

To access the diagnosis value of NGS reports, we scrutinized all reports, adjudicated the reported organisms and categorized the reports based on agreement labeling (namely Agree, Support/Extend, Noncontributory, Unlikely, Disagree, see Table 1 for criteria), assessing whether NGS aided in etiology determination. Tags of Agree and Support/Extend indicate that NGS assisted in identifying potential pathogens, and the total number of reports marked by these two tags reached 124 (68.9%) and 118 (65.6%) for tNGS and mNGS, respectively. In contrast, the total counts of Disagree, Unlikely, and Noncontributory are relatively low, at 31.1% and 34.4%, where most reported pathogens were considered to be clinically insignificant ( Figure 3A ). A preference could be observed between tNGS and mNGS, as the etiology of some cases was identified only by either tNGS or mNGS. tNGS tended to report more cases of Unlikely while mNGS reported more cases of Disagree ( Figure 3B ). We assume that this may be due to sensitivity differences between NGS methods, which will be discussed later.

Figure 3A depicts the comprehensive efficacy of NGS in ascertaining etiological factors. Subsequent analyses were conducted on cases where NGS-based determinations of etiology were provided at a report level. We primarily focused on the cases that highlighted the enhanced capability of NGS to identify potential pathogens. At least one NGS method, either tNGS or mNGS, identified the primary pathogen Mycobacterium tuberculosis, supported by RLT, in 15 cases. However, no single RLT was capable of detecting Mycobacterium tuberculosis in all these instances ( Table 4 ). The RLT with the highest number of positives was the Xpert MTB/RIF assay, confirming the pathogen in 10 out of 15 cases, which was less effective than tNGS (14/15), and mNGS(11/15) ( Table 4 ). Furthermore, tNGS provided assistance in identifying potential Mycobacterium tuberculosis infections in cases where RLT results were negative. Additionally, in four cases, both tNGS and mNGS were instrumental in identifying non-tuberculous mycobacteria to the species level (S075, S085, S120, S149). Regarding the detection of Aspergillus fumigatus, it relies on serological tests, culture, and microscopic observation. Serological tests, such as ELISA, depend on an infection window to achieve optimal sensitivity, which may explain the positive results in only three out of the total seven cases. Moreover, the culture method - a time-intensive and comparatively less sensitive approach - yielded positive results in five cases. In contrast, both NGS methods achieved a 100% detection rate (7/7). In six cases, Chlamydia psittaci was detected exclusively by NGS methods, with no RLTs providing corroborative evidence. Diagnosis of these cases was definitively attributed to a composite reference standard, reinforced by the clinical context of primary pathogenicity and the patients’ history of frequent poultry exposure.