Head and Shoulders: Automatic Error Detection in Human-Robot Interaction

Pauline Trung, Manuel Giuliani (Vortragende:r), Michael Miksch, Gerald Stollnberger, Susanne Stadler, Nicole Mirnig, Manfred Tscheligi

Publikation: Beitrag in Buch oder TagungsbandVortrag mit Beitrag in TagungsbandBegutachtung


We describe a novel method for automatic detection of errors in human-robot interactions. Our approach is to detect errors based on the classification of head and shoulder movements of humans who are interacting with erroneous robots. We conducted a user study in which participants interacted with a robot that we programmed to make two types of errors: social norm violations and technical failures. During the interaction, we recorded the behavior of the participants with a Kinect v1 RGB-D camera. Overall, we recorded a data corpus of 237,998 frames at 25 frames per second; 83.48% frames showed no error situation; 16.52% showed an error situation. Furthermore, we computed six different feature sets to represent the movements of the participants and temporal aspects of their movements. Using this data we trained a rule learner, a Naive Bayes classifier, and a k-nearest neighbor classifier and evaluated the classifiers with 10-fold cross validation and leave-one-out cross validation. The results of this evaluation suggest the following: (1) The detection of an error situation works well, when the robot has seen the human before; (2) Rule learner and k-nearest neighbor classifiers work well for automated error detection when the robot is interacting with a known human; (3) For unknown humans, the Naive Bayes classifier performed the best; (4) The classification of social norm violations does perform the worst; (5) There was no big performance difference between using the original data and normalized feature sets that represent the relative position of the participants.
TitelICMI 2017 Proceedings of the 19th ACM International Conference on Multimodal Interaction
PublikationsstatusVeröffentlicht - 2017
VeranstaltungICMI 2017 - 19th ACM International Conference on Multimodal Interaction -
Dauer: 13 Nov. 201717 Nov. 2017


KonferenzICMI 2017 - 19th ACM International Conference on Multimodal Interaction

Research Field

  • Nicht definiert


  • Human-robot interaction
  • error detection
  • error situation


Untersuchen Sie die Forschungsthemen von „Head and Shoulders: Automatic Error Detection in Human-Robot Interaction“. Zusammen bilden sie einen einzigartigen Fingerprint.

Diese Publikation zitieren