Machine learning methods are more and more widely used in work, and the success of deep neural network pushes perceptual intelligence to a peak. However, people often need to understand why the model makes such a judgment. And in some extremely harsh scenarios, such as finance, subsidies, fraud and so on. The judgment basis and reason of the model becomes particularly important.
The interpretability of machine learning came into being and was gradually applied to industrial scenes. However, in some models with high black box degree, such as deep neural network, it is difficult to explain effectively. With the continuous development of machine learning interpretable technology, the problem is gradually overcome. However, traditional interpretable techniques rely more on the correlation between features and result variables, and may detect some opposite or even pathological explanatory relationships. At the same time, you can't answer "If an intervention changes, what is the decision or judgment of the model?" This counterfactual related problem.
Causal reasoning technology is a special method to study the influence of intervention on results. Usually, we regard correlation as an undirected relationship (features and results will affect each other, and if one party is adjusted, the other party will change accordingly), while causality is regarded as a directed relationship (features determine the results, and only changes in features will change the results, but there is no reverse relationship). Causality is usually stable in multiple scenarios, which is less disturbed than correlation. Therefore, the decision-making or judgment based on causality is more stable, which is also the relationship that we hope machine learning methods can learn. We hope that the method of machine learning is to judge that an object is a cat, because what it sees is the characteristics of hair, beard, sole pad and no tiger skin pattern. Not because there are wool balls, cat litter and shovel officers around cats, but because there are enough dogs. )
Counterfactual estimation and counterfactual reasoning are important parts of causal reasoning technology. (Personal understanding) Causal inference can be understood as wanting to answer, 1) What is the reason (why); 2) If ... The answer must be based on the results of various interventions. The lack of counterfactual results in observation is the fundamental problem of causal reasoning. Different from estimating causal effect (such methods mainly solve how to eliminate deviation and estimate counterfactual, and judge the difference of results under different interventions on this basis), the interpretable method of machine learning based on causality is mainly based on counterfactual explanation, "What kind of decision or judgment will the model make under other scenarios (interventions)?" To show the causal relationship learned by the machine learning model.
Judy Pearl has expounded the different levels of interpretability and emphasized that the generation of counterfactual interpretation is the highest level. Pearl's interpretability level can be roughly divided into the following three layers,
In the community, I saw many wonderful articles, which introduced the traditional machine learning interpretable technology in detail. This article (series) will elaborate the explainable method of machine learning based on causality in detail.
The explanation methods of machine learning based on causality can be roughly divided into three categories: model-based method, sample-based method and causality confirmation related method (other parts mentioned in the cited article are omitted because the direction is not important).
Model-based interpretation method is mainly to split the functions of each part of the model from the perspective of causal effect. For example, calculate the average causal effect caused by the changes of the nth layer and the mth neuron of the deep neural network. Compared with the traditional interpretable method, this method can answer "How much causal effect does a neuron of the neural network have on the final output?" In order to measure the importance of each component of the model, and give some quantitative results.
Model-based methods can be roughly divided into two categories: estimating causal effects and approximate construction. As mentioned above, the importance of a component can be determined by calculating the causal effect of each neuron (usually a component, not such a detail) on the output variable. Approximate construction method, by generating disturbance samples, constructs a bipartite graph, in which the fixed point is the element of input feature, the element of output result, and the edge is the influence relationship between elements. Through the division of bipartite graph, the sub-components which have close influence between input and output elements are obtained, and then the decision-making process and judgment basis of the model are explained.
Sample-based interpretation method aims to explain the decision-making and judgment process of the model by looking for sample examples. Counterfactual samples are modified on the basis of existing factual samples to infer and imagine possible results. Among them, the most typical method is counterfactual explanation. This method is to get the expected counterfactual results by making minimal changes to the characteristics of existing samples, and collect these slightly changed samples to explain the decision of the model. For example, a user is lost from the platform. We change his characteristics as little as possible, so that the model predicts that he will keep it. Observe the change of characteristics and explain on this basis. Here we still use the concept of disturbance, and the interpretable sample obtained by the minimum disturbance, that is, the counterfactual estimator. Counterfactual estimation does not really interfere with the sample, but simulates human imagination through the prediction of the model, so it is at the third level of the causal ladder.
The methods of generating counterfactual interpreters can be roughly divided into six categories, including heuristic method, weighting method, diversity-based method, mixed integer programming method, prototype-based method and GAN-based method. Heuristic method mainly uses distance measure to find the minimum disturbance, while weighting method assigns different weights to each feature dimension in distance measure (the weights are obtained by ANOVA F-value). The method based on mixed integer programming is mainly used to classify scenes with more features. Counterfactual samples generated by other methods may generate continuous values of classification features, resulting in unreasonable features. However, through mixed integer programming, reasonable counterfactual samples can be generated. Based on the prototype method, the direction of generation is determined by looking for a target sample with counterfactual results, which accelerates the convergence of the generation algorithm (the sample with counterfactual results cannot be directly selected because its disturbance may be great). There are many methods to generate counterfactual based on GAN. The simple one is to generate counterfactual samples with the expected counterfactual as the label based on conditionalGAN.
Model interpretability methods based on causal reasoning (counterfactual samples) have been increasingly applied to various scenarios. Next, let's look at two cases from natural language processing and recommendation system scenarios.
Identifying duplicate problems is an important task in the problem search scenario, and two sentences in the picture are identified as duplicate problems. The histogram on the right is the word importance calculated by SHAP value. It can be seen that the importance of the word "friend" is not ranked first. By interfering with Q2 to produce counterfactual samples, we find that "How can I help a person?"
In the case of product recommendation, the interpretability of counterfactual is also needed. For example, when we see a product recommended to users as a top 1 candidate, we may want to know which user behaviors have affected this result, and then change the recommendation result by changing some behaviors. The recommended system in the picture recommends the backpack with the top 1 position. Through the "minimum removable behavior set" identified by the algorithm (the minimum disturbance in counterfactual), it can be seen that "buying Adidas hiking shoes, commenting that Nikon camera is a good hiking companion, and giving high evaluation to Intenso charging treasure" makes the recommendation system decide to put Wolf Claw's backpack in the first place (because there are various indications that Alice likes hiking). If these behaviors are deleted, the ranking of iPad Air will become top 1.
In different scenarios, causal interpretable methods may still be limited by specific models, and there are some time complexity problems. Besides, there are still many problems to be solved. Some specific methods will be introduced later.
Here is a brief introduction to how to evaluate the interpretable method based on causal machine learning and the corresponding evaluation indicators. Different types of interpretation methods have their own unique evaluation criteria. Firstly, the evaluation criteria of interpretable methods based on counterfactual are introduced. Because there is no ground truth value of counterfactual, it is impossible to directly test the quality of counterfactual samples. Therefore, some indicators are defined by heuristics to ensure that the generated counterfactual meets expectations and has expected attributes.
There are other model-oriented interpretable methods and fairness evaluation indicators. The evaluation index of these methods is still a subject that needs to be overcome, so I won't introduce them here. Interested students can refer to the quotations.
J. pearl On the theoretical obstacles of machine learning from the seven sparks of causal revolution. CoRR，ABS/ 180 1.040 16，20 18。
[2] Pearl, J. Mackenzie D. (20 18). The book of cause: the new science of cause and effect. Basic books.
[3],,, Shendi Wang,, Jia, Lei Zhen. Explanatory counterfactual interpretation in artificial intelligence: a tutorial. ACM SIGKDD conference course on knowledge discovery and data mining (KDD 202 1).
[4] D. Alvarez-Melis and T. Jaakkola. Explain the causal framework of black box sequence prediction for sequence model. Proceedings of 2 0 17 conference on empirical methods of natural language processing, 412–421page, Copenhagen, Denmark, September 20 17. Association of computational linguistics.
Wu, T, Ribeiro, M.T., J., & Welding, D. S. (202 1). Polymerization juice: generating counterfactual for explaining, evaluating and improving the model. Proceedings of the 80th and 59th Annual Meeting of the Association of Computational Linguistics.
[5] A. Gazimatin, O. Balalao, R. Saha Roy, & Wycombe, G. (2020, 1 month). PRINCE: The provider with counterfactual explanation in recommendation system 88 can explain. Proceedings of the 3rd International Conference on Web Search and Data Mining (page 196-204).