Ninety samples of fresh tomatoes (‘Provence’ variety) were purchased from a vegetable supermarket in Beijing, China. ‘Provence’ tomatoes have thin skins, juicy flesh, and a rich reddish color when ripe, and the used samples were free of any surface damage. To mitigate temperature variations that may lead to inaccuracies in measurements, the tomatoes were maintained at a temperature of 20°C and a relative humidity of 60% for a duration of 24 hours prior to the collection of spectral and SSC data. During the development of the model, the ratio of sample numbers between the calibration set and the prediction set was established at 3:1.

The spectral data of the tomato samples were collected by using a full transmission Vis-NIR online detection system shown in Figure 1 , and the main units include: a highly sensitive spectrometer (wavelength range: 560-1072 nm, wavelength interval: 0.25 nm, integration time: 5 ms), a speed-adjustable moving platform, a dark box, relative position sensors, an illumination unit consisting of a 150 W halogen lamp with focusing and attenuating device and a computer for the control system. The illumination unit and the spectrometer are placed 150 mm apart on each side of the conveyor belt in the dark box.

In order to assess the effect of the complex cavity inside the tomato on the accuracy of the on-line detection of SSC, each sample passed through the measuring device on a conveyor belt in two different orientations, Z1 orientation: the stem-calyx axis of the test samples were perpendicular to the conveyor belt, and the stems were facing upward; Z2 orientation: the stem-calyx axis of the test samples were parallel to the conveyor belt, and the stems were facing toward the spectrometer, and the orientation of the test samples were shown schematically in Figure 2A .

SSC refers to the percentage of solvable content in the tomato juice, which is mainly composed of nutrients such as soluble sugar (C₆H₁₂O₆) and organic acids (-COOH), and is an important indicator of tomato quality and fruit processing characteristics. After online spectrum acquisition, the traditional destructive method was used to measure the SSC of tomatoes immediately. The measuring instrument was digital Abbe refractometer (model PAL-1, Atago Co., Tokyo, Japan). The intact tomatoes were first chopped and pressed in a wall breaker to make tomato juice, then the tomato juice was filtered through gauze and the juice was dripped into a beaker. After sufficient shaking, about 0.3 ml of juice was dripped onto a digital refractometer (zero-corrected) and the SSC value was recorded manually.

At the time when the tomato samples just reached the detection position and left the detection position, the light signal from the spectrometer passed through only a small portion of the sample’s pericarp tissue. Due to the high optical intensity, these portions of the spectrum should be removed prior to modeling. The extent of removal in this study was about halfway between the two sides of the sample pericarp, which is close to the ideal range. The concluding spectral data obtained encompassed the majority of the pulp and cavity regions within each tomato sample, representing the primary internal quality areas of the entire tomato.

To further improve the spectral data quality, the pre-processing algorithms were generally used to reduce the noise and interference of the instrument and background before building the prediction model. Gaussian filter (GF) is a linear smoothing filter based on the Gaussian function (Zhang et al., 2024), The processing process is as follows: Firstly, the filter parameters are determined, then the GF is constructed, the weight of each data point in the filter is calculated and normalized according to the window size and standard deviation, and finally the data is convolved. Due to the influence of instrument accuracy and background noise, the spectral signals obtained by the spectrometer contain both useful information and random errors superimposed on them at the same time, and the use of smoothing algorithms reduces the noise and improves the signal-to-noise ratio (Chen et al., 2014). Standard normal variate (SNV) is mainly used to eliminate the effects of surface scattering as well as optical range changes (Dhanoa et al., 2023). The processing process is as follows: Firstly, for a given set of spectral data, the mean and standard deviation of all wavelength points in the spectrum are calculated; the data value for each wavelength point is then subtracted from the mean of the sample spectrum and divided by the standard deviation. Multiplicative scatter correction (MSC) is primarily employed to mitigate the scattering effects arising from heterogeneous particle distribution and variations in particle size (Li et al., 2018). The processing process is as follows: Firstly, a spectrum considered representative is selected as the reference spectrum. Then, for each spectrum to be processed, the linear relationship between it and the reference spectrum is calculated. Finally, the data of each wavelength point of the spectrum are corrected by using the obtained linear equation parameters. In this study, GF, SNV and SMC preprocessing algorithms were used to refine the tomato full transmission spectrum data.

Partial least squares regression (PLSR) is a multivariate factorial regression technique frequently employed in the field of spectral analysis. PLSR constructs predictive models by finding the optimal linear combinations of independent and dependent variables and extracting the latent variables (LVs) that maximize the correlation between input and output variables (Diniz et al., 2015). In addition, PLSR demonstrates computational efficiency, particularly in scenarios where the sample size is limited while the number of variables is extensive (Li et al., 2023). In this research, a PLSR model was developed to elucidate the quantitative relationship between the spectral matrix of tomatoes (X) and the matrix of SSC values (Y). The root mean square error of cross-validation (RMSECV) was employed to ascertain the optimal number of LVs.

Calibration correlation coefficient ( R c ), root mean square error of calibration (RMSEC), prediction correlation coefficient ( R p ), and root mean square error of prediction (RMSEP) were used to assess model performance. See Equation 1 for specific calculations. Typically, models with higher correlation coefficients ( R c and R p ) and lower root-mean-square errors (RMSEC, RMSECV, and RMSEP) can be considered to meet expectations. Matlab 2023b (Mathworks, Natick, MA) performed the development of all model programs.

where y i   , y i ^ and y i ¯ denote the measured, predicted and mean values of the i t h tomato sample in the calibration or prediction set, respectively, and n represents the total quantity of tomato samples included in either the calibration or prediction dataset.

The full spectrum contains 2048 wavelengths with a large number of uncorrelated and co-linear variables. Moreover, this adds complexity to the model, a large number of wavelengths may introduce interference, which can make the model run slower and less accurate (Luo et al., 2022). Therefore, it is important to find a variable selection algorithm to simplify the spectral data without reducing the accuracy of the model (Li et al., 2024).

Tomato is usually not a completely uniform structure. There may be differences in the internal organization of different parts of the tomato, such as the cell structure, water distribution, sugar content, etc. near the stem and in the middle of the tomato. By studying different spectral acquisition locations, the characteristics of different regions inside the tomato can be more comprehensively understood, so as to find the most suitable spectral acquisition strategy for specific tomato fruit varieties, and improve the universality and accuracy of the prediction model. Figure 2B showed the original spectral data collected by multi-point full transmission measurement way. It can be seen that the intensity of each spectral curve was relatively large. At the same time, due to the different acquisition locations and the variability of physical and chemical properties of tomatoes, there were obvious intensity differences between different spectral curves. Figure 2C showed the spectral curve after averaging the original data from multiple points. Generally speaking, Z1 orientation: the stem-calyx axis of the test samples were perpendicular to the conveyor belt, and Z2 orientation: the stem-calyx axis of the test samples were parallel to the conveyor belt, the trend and characteristics of the spectral curves obtained by the two acquisition methods were basically similar, but the intensity of the optical signal collected in the orientation of Z2 was higher than that of Z1, this phenomenon can be attributed to the impact of the internal cavity architecture of the tomato sample on the trajectory of light propagation. Due to the shorter optical path distance in the Z2 direction, which was less affected by the structure of the tomato cavity, the transmission spectral signals obtained were stronger.

For the raw transmission spectral data collected in different orientations, the corresponding PLSR models were established to evaluate their prediction performance. As can be seen from Table 1 , for Z1 orientation, R c and RMSEC of model were 0.951 and 0.258%, respectively, R p and RMSEP were 0.637 and 0.713%, respectively; and for Z2 orientation, R c and RMSEC of model were 0.917 and 0.338%, respectively,   R p and RMSEP were 0.757 and 0.583%, respectively. Compared with Z1, the model built in the Z2 orientation improved the accuracy of the prediction set while avoiding potential overfitting of the correction set. For the two detection orientations, the performance of the model was also improved with the increase of spectral intensity. Based on the above analysis, the raw spectral data obtained in the Z2 orientation were used for the subsequent modeling steps.

The spectrum obtained by the instrument encompasses not only the chemical characteristics of the sample but also extraneous information and noise, including electrical interference, background signals from the sample, and stray light. In this study, four preprocessing methods, including MSC, SNV, GF and GF and MSC, were used to process the original transmission spectrum of the tomato samples, and the spectral curves after preprocessing were shown in Figure 3 . It can be seen that the MSC and SNV algorithms reduced the influence of the heterogeneous particle distribution in tomato. Compared with the original spectra in Figure 2B , the spectral curves after GF treatment reduced the undesirable noise and presented smooth curves in the figure. The spectral curves after GF and MSC treatment also showed the same spectral trend as in the above analysis.

The results in Table 2 showed that the R c was improved after the pre-processing of MSC and SNV, and the RMSEC showed a significant decrease, which may be the result of suppressing the surface scattering of the samples. However, the improvement of the model performance in the prediction set was limited. After the GF treatment, although the performance of the model on the calibration set appeared to be degraded, the accuracy of the model on the prediction set was significantly improved. As a comparison, after GF and MSC combination preprocessing, model obtained the better performance ( R c   = 0.927, RMSEC = 0.316%, R p = 0.852, RMSEP = 0.456%). Obviously, the GF and MSC preprocessing process amplified the spectral properties and resulted in clearer and more consistent spectra, which in turn improved the stability of the data.

In addition, the presence of outliers within the dataset can significantly impact the characteristics of the normal samples, which leads to the inaccuracy of the established model. Before further modeling, the relevant algorithm should be used to remove the outliers from the original sample (Lepot et al., 2017; Song et al., 2022). A Monte Carlo outlier detection approach was employed to identify potential outliers within the samples. This method leverages the mean and standard deviation (STD) of the prediction error, facilitating the detection and exclusion of outliers from both spectral data and SSC (Zhang et al., 2020). As shown in Figure 4 , samples 24, 44, 46, 47, 48 were identified as potential outliers. To validate the accuracy of the algorithm, a PLS model was developed for SSC prediction of tomato samples based on the dataset before and after removal of each potential outlier. The results were shown in Table 2 . It can be seen that the prediction accuracies of the correction set and prediction set had been further improved ( R c   = 0.939, RMSEC = 0.291%, R p = 0.877, RMSEP = 0.417%) after outlier removal. This suggests that the elimination of sample outliers diminished the variability within the data set, thereby enhancing the outcomes of the modeling and quantitative analysis.