Variant Approach for Identifying Spurious Relations That Deep Learning Models Learn - Tesch, Tobias; Kollet, Stefan; Garcke, Jochen (Frontiers in Water, 2021)
During the training phase, a deep learning (DL) model learns a function relating a set of input variables with a set of target variables. While the representation of this function in form of the DL model (e.g. the neural network) often lacks interpretability, several interpretation methods exist that provide descriptions of the function (e.g. measures of feature importance). On the one hand, these descriptions may build trust in the model or reveal its limitations. On the other hand, they may lead to new scientific understanding. In any case, they are only useful if the user is able to identify if parts of a description reflect spurious instead of causal relations (e.g. random associations in the training data instead of associations due to a physical process). However, this can be challenging even for experts. To address this challenge, we propose a variant approach and demonstrate its usefulness with two illustrative prediction tasks from hydrometeorology.