To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. Understanding Black-box Predictions via Influence Functions In. In, Metsis, V., Androutsopoulos, I., and Paliouras, G. Spam filtering with naive Bayes - which naive Bayes? Goodman, B. and Flaxman, S. European union regulations on algorithmic decision-making and a "right to explanation". 10 0 obj values s_test and grad_z for each training image are computed on the fly training time, and reduce memory requirements. Frenay, B. and Verleysen, M. Classification in the presence of label noise: a survey. Datta, A., Sen, S., and Zick, Y. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. Fast exact multiplication by the hessian. The previous lecture treated stochasticity as a curse; this one treats it as a blessing. The final report is due April 7. We are given training points z 1;:::;z n, where z i= (x i;y i) 2 XY . It is known that in a high complexity class such as exponential time, one can convert worst-case hardness into average-case hardness. Fast convergence of natural gradient descent for overparameterized neural networks. J. Cohen, S. Kaur, Y. Li, J. Requirements chainer v3: It uses FunctionHook. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. Another difference from the study of optimization is that the goal isn't simply to fit a finite training set, but rather to generalize. RelEx: A Model-Agnostic Relational Model Explainer In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. Google Scholar Krizhevsky A, Sutskever I, Hinton GE, 2012. sample. On linear models and convolutional neural networks, Koh, Pang Wei. This could be because we explicitly build optimization into the architecture, as in MAML or Deep Equilibrium Models. functions. This Understanding Black-box Predictions via Influence Functions S. Arora, S. Du, W. Hu, Z. Li, and R. Wang. We motivate second-order optimization of neural nets from several perspectives: minimizing second-order Taylor approximations, preconditioning, invariance, and proximal optimization. How can we explain the predictions of a black-box model? arXiv preprint arXiv:1703.04730 (2017). The precision of the output can be adjusted by using more iterations and/or If the influence function is calculated for multiple NIPS, p.1097-1105. Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. Overview Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. Limitations of the empirical Fisher approximation for natural gradient descent. Model selection in kernel based regression using the influence function. This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. In. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. How can we explain the predictions of a black-box model? , . 10.5 Influential Instances | Interpretable Machine Learning - GitHub Pages J. Lucas, S. Sun, R. Zemel, and R. Grosse. where the theory breaks down, influence-instance. which can of course be changed. We'll also consider self-tuning networks, which try to solve bilevel optimization problems by training a network to locally approximate the best response function. PDF Understanding Black-box Predictions via Influence Functions - GitHub Pages The reference implementation can be found here: link. A tag already exists with the provided branch name. Adaptive Gradient Methods, Normalization, and Weight Decay [Slides]. I. Sutskever, J. Martens, G. Dahl, and G. Hinton. Visual interpretability for deep learning: a survey | SpringerLink ICML'17: Proceedings of the 34th International Conference on Machine Learning - Volume 70. International conference on machine learning, 1885-1894, 2017. Loss non-convex, quadratic loss . The ACM Digital Library is published by the Association for Computing Machinery. Li, B., Wang, Y., Singh, A., and Vorobeychik, Y. Stochastic gradient descent as approximate Bayesian inference. Liu, D. C. and Nocedal, J. # do someting with influences/harmful/helpful. approximations to influence functions can still provide valuable information. $-hm`nrurh%\L(0j/hM4/AO*V8z=./hQ-X=g(0 /f83aIF'Mu2?ju]n|# =7$_--($+{=?bvzBU[.Q. calculations, which could potentially be 10s of thousands. Therefore, if we bring in an idea from optimization, we need to think not just about whether it will minimize a cost function faster, but also whether it does it in a way that's conducive to generalization. CSC2541 Winter 2021 - Department of Computer Science, University of Toronto In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. I'll attempt to convey our best modern understanding, as incomplete as it may be. The next figure shows the same but for a different model, DenseNet-100/12. Pang Wei Koh, Percy Liang; Proceedings of the 34th International Conference on Machine Learning, . Negative momentum for improved game dynamics. The power of interpolation: Understanding the effectiveness of SGD in modern over-parameterized learning. Thus, we can see that different models learn more from different images. Disentangled graph convolutional networks. In this lecture, we consider the behavior of neural nets in the infinite width limit. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. This packages offers two modes of computation to calculate the influence