Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis

Anahid Naghibzadeh-Jalali (Author and Speaker), Haslhofer Bernhard, Simone Kriglstein, Andreas Rauber

Research output: Chapter in Book or Conference ProceedingsConference Proceedings with Oral Presentationpeer-review

Abstract

Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. However, it is still largely unclear how well users comprehend the provided explanations and whether these increase the users’ ability to predict the model behavior. We approach this question by conducting a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. Moreover, we investigate the effect of counterfactual explanations and misclassifications on users’ ability to understand and predict the model behavior. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model’s decision boundary. Furthermore, we find that counterfactual explanations and misclassifications can significantly increase the users’ understanding of how a machine learning model is making decisions. Based on our findings, we also derive design recommendations for future post-hoc explainability methods with increased comprehensibility and predictability.
Original languageEnglish
Title of host publicationIntelligent Computing. SAI 2023
Subtitle of host publicationProceedings of the 2023 Computing Conference, Volume 1
EditorsKohei Arai
Pages712-733
ISBN (Electronic)978-3-031-37717-4
DOIs
Publication statusPublished - Sept 2023
EventComputing Conference 2023 - Clayton Hotel Chiswick, London, United Kingdom
Duration: 22 Jun 202323 Jun 2023

Publication series

NameLecture Notes in Networks and Systems (LNNS)
Volume711

Conference

ConferenceComputing Conference 2023
Country/TerritoryUnited Kingdom
CityLondon
Period22/06/2323/06/23

Research Field

  • Experience Business Transformation
  • Former Research Field - Data Science

Keywords

  • eXplainable Artificial Intelligence
  • Machine Learning Interpretability
  • Human Computer Interaction

Fingerprint

Dive into the research topics of 'Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis'. Together they form a unique fingerprint.

Cite this