Encoding semantic attributes - Towards explainable AI in industry

The transformation of industrial environments is progressing at a fast pace as more and more autonomous systems are installed and operated. Save and explainable AI algorithms are thus essential, especially for collaborative interactive systems that operate in human spaces. We propose the “Semantic Encoder”, a 2D-vision based CNN model trained on a purely synthetic dataset, to address the explainability aspect by extracting semantic descriptions of real objects based on their visual appearances. We can use the extracted semantic information to simply describe depicted samples or to differentiate between normal and anomalous samples, with the possibility to explain what caused the anomaly detection. The semantic description can be further used to sort samples by classifying them or to find a sample with specific semantic properties. We evaluate the Semantic Encoder with respect to its informative power by comparing the computed semantic features with features extracted by a VGG-16 model and classical image processing methods. The results are quantified based on the Generalized Discriminative Value (GDV). We also investigate how accurately anomalous samples are detected by computing ROC and PR curves. We use the semantic parameters to understand what causes good and inaccurate anomaly detection decisions. In addition, we evaluate the quality of the classification based sorting by examining confusion matrices and classification accuracy.