DeepCrypt - Deep Learning for Video QoE Monitoring and Fingerprinting of User Actions

We introduce DeepCrypt, a deep-learning based approach to analyze YouTube video streaming Quality of Experience (QoE) from the Internet Service Provider (ISP) perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived on-line from the encrypted stream of bytes, DeepCrypt can infer six different video QoE indicators capturing the user-perceived performance of the service, including the initial playback delay, the number and frequency of rebuffering events, the video playback quality, the video encoding bitrate, and the number of quality changes. DeepCrypt offers deep visibility into the behavior of the end-user, enabling the fingerprinting and detection of different user actions on the video player, such as video pauses and playback scrubbing (forward, backward, out-of-buffer), offering a complete visibility on the video streaming process from in-network traffic measurements. Extensive evaluations over a large and heterogeneous dataset composed of mobile and fixed-line measurements, using the YouTube HTML5 player, the native YouTube mobile app, as well as a generic HTML5 video player built on top of open source libraries, and considering measurements collected at different ISPs, confirm the out-performance of DeepCrypt over previously used shallow-learning models, as well as the deep-model generalization to different video players and network setups. To the best of our knowledge, this is the first paper tackling such a combined QoE and user-actions approach for video streaming applications, using deep learning models over heterogeneous measurements.