Sensing the Machine: Evaluating Multi-modal Interaction for Intelligent Dynamic Guidance

Recent research has demonstrated the potential for representing intelligent guidance using multi-modal cues, yet few guidelines or processes exist to guide the design of such a system. In this paper, we seek to address this gap by investigating the design of multi-modality assistant systems for setting the optimal parameters in industrial plants. We then present the results of a user study conducted with 22 participants to evaluate the effectiveness and experience of different combinations of visual (Highlights and Ambient) and haptic (Clicks and Vibration) modalities for providing intelligent dynamic guidance. The results demonstrate that providing the intelligent guidance with the multi modality of Highlights+Ambient resulted in shorter duration and higher practicality than Ambient lights alone. Moreover, multi modality of Highlights+Ambient+Vibration was rated with lower usability than Highlights+Ambient as well as higher mental demand than merely Highlights.