TY - GEN
T1 - Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis
AU - Bernhard, Haslhofer
AU - Kriglstein, Simone
AU - Rauber, Andreas
A2 - Wachsenegger, Anahid
A2 - Arai, Kohei
N1 - iVmB?
gedruckter Proceedings-Beitrag wurde einer Peer Review unterzogen?
File?
PY - 2023/9
Y1 - 2023/9
N2 - Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. However, it is still largely unclear how well users comprehend the provided explanations and whether these increase the users’ ability to predict the model behavior. We approach this question by conducting a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. Moreover, we investigate the effect of counterfactual explanations and misclassifications on users’ ability to understand and predict the model behavior. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model’s decision boundary. Furthermore, we find that counterfactual explanations and misclassifications can significantly increase the users’ understanding of how a machine learning model is making decisions. Based on our findings, we also derive design recommendations for future post-hoc explainability methods with increased comprehensibility and predictability.
AB - Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. However, it is still largely unclear how well users comprehend the provided explanations and whether these increase the users’ ability to predict the model behavior. We approach this question by conducting a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. Moreover, we investigate the effect of counterfactual explanations and misclassifications on users’ ability to understand and predict the model behavior. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model’s decision boundary. Furthermore, we find that counterfactual explanations and misclassifications can significantly increase the users’ understanding of how a machine learning model is making decisions. Based on our findings, we also derive design recommendations for future post-hoc explainability methods with increased comprehensibility and predictability.
KW - eXplainable Artificial Intelligence
KW - Machine Learning Interpretability
KW - Human Computer Interaction
KW - eXplainable Artificial Intelligence
KW - Machine Learning
U2 - 10.1007/978-3-031-37717-4_46
DO - 10.1007/978-3-031-37717-4_46
M3 - Conference Proceedings with Oral Presentation
SN - 978-3-031-37716-7
T3 - Lecture Notes in Networks and Systems (LNNS)
SP - 712
EP - 733
BT - Intelligent Computing. SAI 2023
T2 - Computing Conference 2023
Y2 - 22 June 2023 through 23 June 2023
ER -