Variable Selection and Redundancy in Multivariate Regression Models - Westad, Frank; Marini, Federico (Frontiers in Analytical Science, 2022)
Variable selection is a topic of interest in many scientific communities. Within chemometrics, where the number of variables for multi-channel instruments like NIR spectroscopy and metabolomics in many situations is larger than the number of samples, the strategy has been to use latent variable regression methods to overcome the challenges with Multiple Linear Regression. Thereby, there is no need to remove variables as such, as the low-rank models handle collinearity and redundancy. In most papers on variable selection, the main objective is on comparison of prediction performance (RMSE or accuracy in classification) between various methods. Nevertheless, different methods with the same objective will, in most cases, given results that are not significantly different. In this paper, we present three other main objectives: i) to eliminate variables that are not relevant; ii) return a small subset of variables that has the same or better prediction performance as a model with all original variables; and iii) investigate the consistency of these small subsets.