Head and Shoulders: Automatic Error Detection in Human-Robot Interaction

Pauline Trung, Manuel Giuliani (Speaker), Michael Miksch, Gerald Stollnberger, Susanne Stadler, Nicole Mirnig, Manfred Tscheligi

Research output: Chapter in Book or Conference ProceedingsConference Proceedings with Oral Presentationpeer-review

Abstract

We describe a novel method for automatic detection of errors in human-robot interactions. Our approach is to detect errors based on the classification of head and shoulder movements of humans who are interacting with erroneous robots. We conducted a user study in which participants interacted with a robot that we programmed to make two types of errors: social norm violations and technical failures. During the interaction, we recorded the behavior of the participants with a Kinect v1 RGB-D camera. Overall, we recorded a data corpus of 237,998 frames at 25 frames per second; 83.48% frames showed no error situation; 16.52% showed an error situation. Furthermore, we computed six different feature sets to represent the movements of the participants and temporal aspects of their movements. Using this data we trained a rule learner, a Naive Bayes classifier, and a k-nearest neighbor classifier and evaluated the classifiers with 10-fold cross validation and leave-one-out cross validation. The results of this evaluation suggest the following: (1) The detection of an error situation works well, when the robot has seen the human before; (2) Rule learner and k-nearest neighbor classifiers work well for automated error detection when the robot is interacting with a known human; (3) For unknown humans, the Naive Bayes classifier performed the best; (4) The classification of social norm violations does perform the worst; (5) There was no big performance difference between using the original data and normalized feature sets that represent the relative position of the participants.
Original languageEnglish
Title of host publicationICMI 2017 Proceedings of the 19th ACM International Conference on Multimodal Interaction
Pages181-188
Number of pages8
DOIs
Publication statusPublished - 2017
EventICMI 2017 - 19th ACM International Conference on Multimodal Interaction -
Duration: 13 Nov 201717 Nov 2017

Conference

ConferenceICMI 2017 - 19th ACM International Conference on Multimodal Interaction
Period13/11/1717/11/17

Research Field

  • Not defined

Keywords

  • Human-robot interaction
  • error detection
  • error situation

Fingerprint

Dive into the research topics of 'Head and Shoulders: Automatic Error Detection in Human-Robot Interaction'. Together they form a unique fingerprint.

Cite this