Interpretability Latent Space Method: Exploiting Shapley Representation to Explain Latent Space

Shapley values have become one of the most popular interpretation methods for feature attribution. These methods attribute the prediction of the input by the machine learning model to its basic feature. However, most of the previous work is based on Shapley's interpretation after the training is completed because of Shapley's calculation requirements (exponential time complexity), so that they can focus on post-hoc Shapley explanations. Therefore, we suggest trying to use Shapley value itself as a latent representation in the deep model to guide model training. First, we extract the latent space of the generation model as input of Shapley network. Then, we utilize Shapley network, which provides layer-wise transformation in the same forward pass and produce Shapley representation. Further, we apply Shapley representation to the calculation of the new model to explain the training process of the model. Finally, the experimental results show on the two datasets of MNIST and Fashion MNIST that our proposed method has certain interpretability.