In today’s world where the number of online videos are growing gradually nothing has become more important for video streaming services than measuring the quality of experience. The authors of this paper introduced a Chrome browser plug-in (YouSlow) that can detect various causes of YouTube video abandonment, such as start-up latency, rebufferings and bitrate changes. They describe abandonment as an act of closing a video during playback, either because of losing interest or being exasperated by above-mentioned playback events. Adaptive bitrateMost famous video service providers use adaptive bitrate (ABR) streaming in which a video player tends to change video bitrates according to network conditions. To improve ABR streaming, they decided to monitor live playback events directly from video players instead of focusing on quality of service (QoS) metrics which do not echo user experience accurately. The importance of this improvement is that the fewer playback events happen, the more viewers tend to watch videos until the end.
First, they tried to measure the playback events. They started with start-up latency which is the elapsed time between the play button’s click and main video’s start time. Pre-roll advertisements and playout buffers can cause these start-up latency. It’s feasible by YouSlow to find out whether the reason behind the start-up latency is one of those annoying pre-roll ads or not enough buffered data. The result of studying this playback event shows that 99% of the cases a pre-roll ad is the reason behind the start-up latency.Bitrate changes
Next playback event that has been tested is bitrate switches. Bitrate switches are the number of times an ABR player decides to increase or decrease the bitrate. Based on their observation, fewer than three bitrate switches happen during more than 98% of video sessions. There are eight different bitrates for YouTube videos which are highres, hd1440, hd1080, hd720, large, medium, small and tiny. Two video bitrates which stand out as the most watched video bitrates are large and medium. Furthermore, as they are able to get the approximate location of the playback event using an IP geo-location database, distributions of played bitrates among countries were not hard to capture. Finally, they found out that there’s a delicate correlation between bitrate changes and rebufferings.
What is rebuffering? You may have experienced those moments where you were watching a video and suddenly the video was paused and then resumes again. Rebuffering can happen when the player requests a higher bitrate than what is actually available in the network. This is how bitrate change and rebuffering correlate. Authors’ observation shows that in 99% of cases fewer than four rebufferings happen during the entire video session. Video QoE analysisIn this section, the authors focused on the metrics that allowed them to analyze the impact of playback events (e. g. , start-up latency, rebufferings and bitrate changes) on video QoE and viewing interruption. We should bear in mind that YouTube videos are mostly short video clips such as music videos and sports highlights. While visiting different websites, you may come across YouTube videos that are related to the topic of what you have searched. Viewers can easily abandon these videos if they lose interest during playback. The authors planned to separate videos that were abandoned due to poor conditions such as frequent bitrate changes and rebufferings from videos that were closed due to lack of interest. When a video is paused due to rebufferings and the viewer close the video during that interval, it is considered as an abandonment. The story is more complicated with bitrate changes. Videos that are closed due to bitrate switches or lack of interest are hardly distinguishable. By assuming that an act of abandonment is when a viewer closes the video within five seconds of bitrate change, we can separate bitrate changes from lack of interest. However, they only consider this separation if the viewer has watched at least half of the full content of the video. By this definition, their dataset can only include longer videos. In this case, videos with a duration of 30 seconds or shorter are left out. This can be considered as one the weaknesses of their analysis because a lot of videos are excluded. To improve that, they can change their definition of abandonment a little so that 30 seconds or shorter videos can fit in as well. With all that being said, there are other reasons for abandonment that are out of human hands. For example, an electricity shutdown or a system crash. However, these occasions rarely happen and are too hard to detect.
In their analysis report, they have mentioned that there are more abandonments when rebuffering intervals are short. What can cause these short rebufferings? The answer is requiring more than your network capacity. This means if an ABR player wants a higher bitrate and the network it belongs to can’t handle it, short rebufferings will happen. A simple metric from which they can predict the abandonment rate is rebuffering ratio. Rebuffering ratio is the fraction of time when a viewer experiences rebufferings while watching a video. For a 90 second video, if we face 10 seconds of rebuffering the rebuffering ratio would be 10/(10+90)=0. 1. Based on the authors’ result, as the rebuffering ratio increased, more viewers abandoned the videos. However, based on their saying, additional factors such as rebuffering duration and total playback length were avoided due to high complexity. These factors can become future studies in order to strengthen their research.
They count the number of video sessions abandoned by the viewers during either the start-up latency or the rebuffering. By dividing this number by the total number of video sessions, they have achieved the abandonment rate. Based on the authors’ confession, comparing the abandonment rates between start-up latency (e. g. , initial buffering and pre-roll ads) would have strengthened their results. However, this was not feasible due to lack of samples for the buffering case. They calculate bitrate change ratio and showed that the more the bitrate changes happen, the more viewers tend to leave the video session.
Having in mind that rebufferings and bitrate changes are dependent on each other, they claim that by monitoring rebuffering ratio and bitrate changes over playback would improve user engagement while a video is being played. The significance of their contribution is to maintain the abandonment rate lower than a specified percent.
In summary, YouSlow is a video QoE analysis tool that can detect various playback events. Their formula for calculating rebuffering ratio and bitrate change ratio make sense to be proper QoE metrics but we have to keep in mind that they avoided so many complex metrics which can affect their analysis. I’m not sure if these avoidances were made due to lack of samples or impossibility of using current technology. However, for the future works, they can focus on adding metrics such as rebuffering duration and playback length.