To evaluate the models’ performance and their applicability in identifying microbial organisms, we conducted experiments using a publicly available dataset of GC-IMS data from four different organisms (Weller and Christmann, 2023). These include three bacteria, Escherichia coli, Levilactobacillus brevis, and Pseudomonas fluorescens, and one fungus, Saccharomyces cerevisiae. Pure cultures of these organisms were prepared, along with six different mixtures combining each pair. The dataset consists of 10 different classes, with 31 unique cultures analyzed to generate 214 distinct GC-IMS spectra samples. In Table 1 , we present the number of Cultures and Samples for each class of the presented dataset.

As shown in Figure 1 , a representative GC-IMS spectrum can be visualized as a heatmap, where the x-axis indicates the drift time of the separated ions based on IMS and the y-axis represents the retention time derived by GC, corresponding to the separation characteristics of the sample. The color bar indicates the ion intensity, with specific ions (i.e., VOCs fingerprints) forming peaks at particular coordinates, generating rich informational spectra for analysis.

Given the high dimensionality of the input data (3,150 × 6,123), a standard procedure involves applying a pre-processing pipeline (illustrated in Figure 1 ) to reduce data size, denoise spectra, and extract the most relevant information. To this end, we utilize the open-source Python package gc-ims-tools (Christmann et al., 2022), in accordance to the dataset’s initial publication (Christmann et al., 2024). First, we apply a third-degree wavelet transformation to both the drift and retention time axes, significantly reducing the dimensionality of the spectra to one-tenth of the original data to each direction. Then, a baseline correction algorithm is applied to remove instrumental variations or background noise, using a white top-hat filter of size 15. Finally, to properly prepare the input data for the models, we apply various data points processing techniques. Drift Time Alignment adjusts the drift time axis relative to the Reactant Ion Peak (RIP), which is a standard technique for the task, due to instrumentation variations. Intensities Normalization between zero and one, is a common technique for efficiently training ML and DL models. While, Spectra Cutting is applied on the Region of Interest (ROI), which contains the most valuable information selecting the RIP-relative drift time range between 1.05 and 2.10 and the retention time between 70 and 780 s. The output of the pre-processing pipeline is shown in Figure 1 , where the spectral shape is reduced to (152 × 600), resulting in a significant 99.5% reduction in data size.

As outlined in Section 1, GC-IMS has emerged as a promising tool in various tasks for analyzing samples, particularly in food authentication. DL models have also shown great potential in training effective classifiers based on GC-IMS data. However, limited work has addressed the challenge of identifying bacterial cultures directly from data produced by GC-IMS prototypes. To address this, we employ eight different classification methods, widely used in chemometrics, including four ML models, PLS_DA (Christmann et al., 2024), PCA_LR (Vega-Márquez et al., 2020), PCA_SVM (Vega-Márquez et al., 2020), and XGBoost (Vega-Márquez et al., 2020), and four DL models, FCNN (Vega-Márquez et al., 2020), MLP (Vega-Márquez et al., 2020), CNN1D (Yan et al., 2024), and CNN2D (Yan et al., 2024).

Classical ML models are widely employed by researchers to address a variety of problems. Their main advantages include strong generalization on small datasets, fast training times, and relatively few parameters. PLS_DA is a commonly adopted method in chemometrics, providing both data compression and classification of input spectra. Similarly, PCA combined with either SVM or LR is frequently utilized, benefiting from PCA’s dimensionality reduction capabilities alongside the efficiency of SVM and LR. Additionally, XGBoost is included in our experiments due to its strong performance across a wide range of ML tasks. To ensure reproducibility of results, a random seed of 42 is set during all trainings. For dimensionality reduction, we retain a number of 50 components, while classification models (SVM, LR, XGBoost) are trained using their default hyper-parameters of their respective packages.

In contrast, DL models feature more parameters, longer training times, and more complex architectures. While they require larger datasets, they effectively capture correlations within input parameters, resulting in improved generalization and higher accuracy. We employ CNN2D to capture the spatial structure of the heatmaps, and CNN1D to assess the performance of convolutional layers on flattened heatmaps. Additionally, we include FCNN and MLP as standard baselines for DL-based methods. Figure 2 illustrates the four architectures: CNN2D ( Figure 2A ), CNN1D ( Figure 2B ), MLP ( Figure 2C ), and FCNN ( Figure 2C ).

The first model, CNN2D (2A), is adopted to leverage the spatial information within the spectra. The pre-processed spectra serve as input to the architecture, which consists of three hidden 2D convolutional layers with a 3×3 kernel size and an increasing number of filters, 32, 64, and 128, respectively. Each convolutional layer is followed by a Leaky ReLU activation and a 2×2 Max Pooling operation. The output of the final convolutional layer is flattened and passed through a fully connected hidden layer with Leaky ReLU activation and a Dropout rate of 0.5, resulting in 512 features. Lastly, a final fully connected layer is applied in the extracted features to generate the target classes.

Additionally, we implement a CNN1D (2B) model. After flattening the input spectra, and following the design of CNN2D (2A), we apply three 1D convolutional layers with a kernel size of 9, each followed by a Leaky ReLU activation and a 1D Max Pooling layer of size 4. The number of filters increases progressively (32, 64, 128). As previously, the output of the final convolutional layer is flattened and passed through a fully connected layer with Leaky ReLU activation and a Dropout rate of 0.5, resulting in 512 features. Finally, a fully connected output layer maps these features to the target classes.

Finally, we employ two models: MLP (2C) and FCNN (2C). In both, the input spectra are flattened and passed through fully connected hidden layers, each followed by a Leaky ReLU activation and a Dropout rate of 0.5, resulting in 512 features. Although their architecture is presented in the same way, the MLP model consists of a single hidden layer, whereas the FCNN model includes four hidden layers, resulting in more parameters and higher complexity. As in the previous models, a final fully connected layer processes these features to generate the target class predictions.

The original study that introduced this dataset evaluated models using a fixed train/test split. While this approach is common for large datasets, it has significant disadvantages in the context of small datasets like ours. First, with limited samples, a single split can lead to severe under-representation of certain classes in the test set, causing performance metrics to vary greatly depending on which samples are chosen. Second, the original paper does not provide sufficient details regarding how the split was constructed (e.g., sample or culture stratification, class balance, random seed), making it impossible to reproduce the original results and perform a fair, like-for-like comparison. To this end, and given the small number of samples available in the dataset for many classes (see Tables 1 , 2 ), creating a fixed test set would result in under-representation of several classes and make final performance metrics highly sensitive to the specific samples selected.

An alternative strategy that could prevent potential data leakage caused by culture-specific information is Leave-One-Culture-Out (LOCO) cross-validation, where all spectra from a given culture are held out for validation in each iteration. This would provide a rigorous assessment of generalization to unseen cultures. However, our dataset contains only 31 cultures, with as few as 2–4 cultures per class. Under these conditions, LOCO would result in extremely small training sets for some classes (2 cultures in a class, result in 50% split training and validation sets), leading to unstable performance estimates and impractical model training.

To overcome these limitations, we adopt a stratified five-fold cross-validation protocol at the sample level, as illustrated in Figure 3 . Cross-Validation is widely recommended for small datasets to improve robustness and reduce variance caused by test set selection, not only in ML and DL in general (Kohavi, 1995; Refaeilzadeh et al., 2009; Raschka, 2018), but also has been widely adopted in chemometrics and bioinformatics tasks for similar reasons (Beleites and Salzer, 2008; Esbensen and Geladi, 2010; Westad and Marini, 2015). In this setup, the dataset is randomly partitioned into five folds while preserving the overall class distribution in each fold. For each iteration, 80% of the samples are used for training and 20% for validation, ensuring that every sample appears exactly once in a validation set across five independent trainings. This approach maximizes the use of available data while maintaining comparability across experiments. To ensure fairness and like-to-like comparison between the proposed methods and the original study’s baselines, we include the PLS_DA model in our evaluation using the same hyperparameters. For reproducibility, all experiments are performed with a fixed random seed of 42.

For DL baseline models, we employ an Adam optimizer, a learning rate of 0.001, a batch size of 8 and cross-entropy loss function (L _CE ), as shown in Equation 1. Although hyperparameter fine-tuning could potentially improve the performance of each individual model, it was not applied in this study due to the limited dataset size and the primary objective of comparing baseline architectures rather than optimizing them, this would be out of scope for this study. Therefore, we report results using standard training hyperparameters.

During training, we evaluate through Accuracy (Equation 2) and F1-score (Equations 3–5) across the validation set, while Sensitivity (Equation 6) and Specificity (Equation 7) are also reported for the validation set for each class c, separately. Additionally, training and inference times along with the number of parameters are reported to provide a comprehensive comparison of each model for each experiment. In the following equations, C, N, TP, TN, FP, and FN represent number of classes, number of samples, true positives, true negatives, false positives, and false negatives, respectively. Moreover, y i , c represent the one-hot encoded ground truth label for sample i and class c, and y ^ i , c represent the predicted probability (from softmax) for sample i and class c.

For the needs of the given research, we conducted all the experiments on a server equipped with a NVIDIA GeForce RTX 4090 GPU. CUDA (12.0) along with Python (3.8.20) have been used, while various packages have been employed among gc-ims-tools (0.1.7) for the data pre-processing pipeline and PLS_DA training, scikit-learn (1.3.2) for the ML models of PCA_SVM and PCA_LR, xgboost (2.1.1) for the XGBoost model and torch (2.4.1+cu118) for the training of DL models.

Following, on Tables 3 , 4 , we present a list of the eight different experiments conducted on the dataset. More precisely, on Table 3 , we present two multi-class classification experiments, Pure and Pure & Mixed. As their names suggest, in the first experiment, we train models to classify between the four pure classes, whereas in the second, we train models to classify both pure and mixed cultures, resulting in ten different classes. The numbers indicate the assigned class labels, a standard approach in ML and DL training. Diving deeper into the classification of the pure cultures, we conduct two additional classification experiments, Bacteria & Fungi and Gram-positive & Gram-negative ( Table 3 ). The Bacteria & Fungi experiment is particularly important as it evaluates the models’ ability to distinguish bacterial from fungal behavior. Meanwhile, the Gram-positive & Gram-negative experiment focuses into bacterial characteristics, analyzing their GC-IMS fingerprints based on their cell wall type. These two experiments demonstrate a dataset imbalance, as reported in Table 2 , which is a common issue in the literature when training ML and DL models. The Gram-positive class contains 28 samples, while the Gram-negative class has more than twice as many, with 58 samples. Similarly, bacterial samples account for a subset of 86 spectra, whereas fungal samples are limited to just 31, approximately one third.

On the other hand, we explore the possibility of specifically identifying either the Presence (+) or the Pureness (*) of two distinct pathogenic bacteria, E. coli and P. fluorescens. As presented in Table 4 , to properly evaluate the models’ ability to detect the presence of a specific bacterium, we assign a value of 1 (one) to all the classes containing the given bacteria and 0 (zero) to all the remaining classes. Subsequently, by training from-scratch the models, we investigate their ability to identify the pureness of the bacteria among all the classes in which it is present. This experiment highlights the models’ sensitivity on the given bacteria. These two rounds of experiments are conducted for each bacteria, serving as a baseline for identifying pathogenic bacteria using ML and DL methods based on GC-IMS spectra. Similarly, dataset imbalance is also observed in these experiments ( Table 2 ). For the E. coli (+) experiment, the dataset includes 75 samples with presence and 139 with absence. In the E. coli (*) experiment, results are reported for 30 pure samples and 45 mixed samples. Likewise, for P. fluorescens (+), there are 73 samples with presence and 141 with absence, while the P. fluorescens (*) experiment includes 28 pure samples and 45 mixed samples.

As discussed earlier and in line with the initial dataset’s publication, we experiment with multi-class classification in two different scenarios: Pure and Pure & Mixed cultures. In the Pure experiment, we classify samples into four distinct categories, each representing a pure culture of a specific organism: E. coli, L. brevis, P. fluorescens, and S. cerevisiae. In contrast, the Pure & Mixed experiment evaluates the models’ ability to identify between pure cultures and all possible pairwise mixed cultures in the dataset, resulting in ten distinct classes.

Table 5 presents the classification results for the models described in Subsection 2.2, reporting the average and standard deviation of Accuracy and F1-Score across five cross-validation folds. The highest-performing model is highlighted in bold, while the second-best is underlined. In both experiments, the FCNN model demonstrates clear superiority, achieving 93.19% average accuracy and 93.04% average F1-score in the Pure experiment, and 92.53% average accuracy and 93.37% average F1-score in the Pure & Mixed experiment, all with a relatively small standard deviation. The CNN2D model consistently ranks second, demonstrating strong performance across all metrics and tasks, showcasing its ability to generalize the information based on their spatial information. The overall performance of DL models can be summarized as an out-performance compared to traditional ML baselines.

Following Table 6 , we analyze the Selectivity and Specificity of each class in both experiments using the best-performing model, FCNN. We observe that in both cases, the model achieves strong performance across all classes for both metrics. More specifically, introducing pairwise mixed cultures into the training set affects the performance on pure classes. For instance, the model’s ability to identify E. coli and L. brevis significantly decreases, whereas P. fluorescens and S. cerevisiae maintain or slightly improve their performance. On the other hand, the mixed cultures exhibit a sensitivity variation of up to 10%, while specificity remains consistently high, with differences of less than 1% between values across the different classes.

The classification between Bacteria & Fungi is a key experiment in our work, as it highlights the distinct correlations associated with bacterial compared to fungal cultures. In this experiment, we conducted a new training of the models based on their ability to classify between the two categories. The main challenge lies in effectively grouping all bacterial samples and identifying their correlations in comparison to fungi cultures.

As shown in Table 7 , both MLP and FCNN models achieve top performance, reaching an average accuracy and f1-score of 98.30% and 97.63%, respectively. These are followed by the other two DL baselines, CNN1D and CNN2D, while the ML baselines also demonstrate promising performance. Similarly, in Table 8 , we observe that for class zero (bacteria), the top-performing models achieve 100% sensitivity and 93.33% specificity, with an inverse pattern observed for class one (fungi).

Following the same setup, we conducted another innovative experiment, aiming to further analyze the behavior of a more precise bacterial categorization, namely Gram-positive & Gram-negative. To this end, we assigned class zero to Gram-negative bacteria, including E. coli and P. fluorescens, and class one to Gram-positive bacteria such as L. brevis.

In this task, as shown in Table 7 , MLP and FCNN again emerge as the top-performing models, achieving 97.71% accuracy and 97.35% f1-score. The ML baseline PLS_DA follows as the second-best performer in both metrics, while the remaining ML and DL baselines demonstrate significant lower performance. For the best-performing models, FCNN and MLP, sensitivity and specificity alternate between 100% and 93.33% across the two classes, as presented in Table 8 .

Finally, we evaluate the performance of the proposed models in the task of specifically identifying pathogenic bacteria, such as E. coli, as a highly pathogenic bacterium and P. fluorescens as a low pathogenic bacterium. To this end, our initial experiments involve ten different pure and mixed cultures to evaluate the models’ ability to detect the Presence (+) of a specific bacterium. We assign class zero to cultures that do not contain the specific bacterium and class one to those where it is present. Furthermore, among the cultures where the bacterium is present, we further classify the Pureness (*) of the culture in comparison to mixed ones, to dive deeper into the identification of pathogenic bacterial cultures.

As presented in Tables 9 , 10 , we observe that FCNN is the best-performing model for identifying the presence of E. coli, achieving 92.50% accuracy and an F1-score of 91.40%, closely followed by MLP with 92.04% accuracy and 90.86% F1-score. The remaining baselines exhibit lower performance. Sensitivity and specificity alternate between 96.45% and 84.76%, demonstrating that the model is highly specific in detecting the presence of pathogenic bacteria.

On the other hand, when distinguishing between pure and mixed cultures of the pathogenic bacterium E. coli, both FCNN and CNN1D achieved perfect performance, with 100% accuracy, F1-score, sensitivity, and specificity. The remaining baseline models also demonstrated strong performance, further supporting the reliability of the classification. These results highlight the models’ ability to accurately determine the pureness of specific pathogenic bacterial cultures. However, potential dataset-specific or culture-dependent noise should be considered when interpreting these perfect scores. Future work should involve evaluating these classes on larger and more diverse datasets to assess the consistency and generalization of the models.

In alignment with previous experiments, we investigate the classification performance of the models based on the Presence (+) and Pureness (*) of another pathogenic bacterium, P. fluorescens. Table 11 once again highlights the superiority of the FCNN model in both tasks. Specifically, FCNN achieves 93.44% accuracy and an f1-score of 92.70% in the P. fluorescens (+) experiment, followed by MLP and CNN2D, while CNN1D and PLS_DA also demonstrate promising results. Similarly, in the P. fluorescens (*) experiment, FCNN outperforms the other baselines, achieving 98.67% and 98.56% of accuracy and f1-score, respectively. CNN2D follows closely in performance, whereas the remaining baselines exhibit significantly lower results in comparison.

As shown in Table 10 , the sensitivity and specificity of the best-performing model, FCNN, are smoother compared to previous experiments. The values for the two classes in each experiment alternate between 91.62% and 94.31% for the presence of the pathogenic bacterium and between 100% and 96.67% for the pureness of the culture. These findings demonstrate that the models are highly specific in detecting the pureness of the pathogenic bacterium P. fluorescens.

To further evaluate the proposed DL baseline models, we further report two key evaluation aspects of DL research. First, we measured the training time for each of the four models: CNN1D, CNN2D, MLP, and FCNN. For each experiment and each iteration of the five-fold cross-validation evaluation method, we recorded the overall training time, compute the average and standard deviation in seconds, and presented the results in Table 12 .

Notably, FCNN, apart from being the best-performing model across all experiments, exhibited the fastest training time, averaging 24.54 seconds. This is nearly half the time required by the second-fastest model, CNN2D, which averaged 43.11 seconds. MLP followed with an average training time of 48.85 seconds but showed a high standard deviation of 78.16 seconds, and CNN1D had the longest training time, averaging 58.38 ± 71.34 seconds.

Additionally, Table 12 reports the overall number of trainable parameters relative to the number of target classes, as the last hidden layer’s parameters depend on the output classes, as illustrated in Figure 2 . MLP has the fewest trainable parameters at 46,694,912, closely followed by FCNN with 47,482,880 parameters, presenting only a 1.68% increase. CNN2D nearly doubles this amount with 81,423,360 parameters, while CNN1D has the highest number, requiring 93,285,376 trainable parameters.

These results highlight that FCNN not only achieves the highest accuracy across all experiments but also is trained fastest and is the second most compact model in terms of trainable parameters. MLP and CNN2D perform competitively depending on the experiment, while CNN1D consistently ranks as the least effective model among the four DL baselines, showcasing that CNN models are quite computationally insufficient in the presented tasks.