Big Blog

Find company updates, big product releases, and some of our most popular posts

Share

Share

Variability of Video Calling Experience

With millions of people using Skype around the world, we can glean global insights on the quality and openness of the Internet. From anonymous feedback that people on Skype provide us through their rating of video calls, we’ve put together an analysis of the varying quality and experience of Skype video calling by location. Our goal is to allow people on Skype to compare their experience with Skype video calling to that of others in the same location to determine if using a different network provider might improve their call quality.

Key findings

  • In poor network conditions users give a low user rating to the calls made from the network.
  • There are clear differences between and within countries in the (call quality) performance of different networks and network providers.
  • Knowing the performance of the network you use compared to others in the same location can provide users with an opportunity to evaluate using other networks/providers, if your call quality is being impaired.

Detailed results

We start by comparing the ratings given to Skype video calls by people located in various countries. We conclude with a look into the variability of quality within countries.
network1.png

In the figure above, we have divided countries into four equal groups based on average feedback ratings provided by people making Skype video calls during the month of September 2011. The highest quarter represents countries with the best video calling experience and lowest quarter represents the lowest quality video calling experience. Please note that the countries shaded gray are those from which we have not yet received sufficient feedback to draw clear conclusions on call quality.

While the figure above demonstrates the differences in Skype video call quality globally, we are also interested in developing a better understanding of the differences in call quality within countries. By comparing our users feedback from different networks within each country, we can better understand what opportunities exist for Skype users to improve their calling experience, as well as better understand the source of the variation of calling quality.

Each Internet Service Provider (ISP) uses at least one officially registered Autonomous System Number (ASN) to identify them within the Internet core routing fabric. In the figure below, we have plotted the average user-feedback rating for each ASN within a country. Following the plot horizontally for a given country will give you an impression of the variation in quality of the Skype video calling experience. In this study, we have used a relative quality scale, where the highest rating is given to the ASN with highest user feedback rating. All other ASNs are rated as a percentage from the highest rating.

In the data that we are providing, we have not distinguished between mobile and fixed-network connection providers. Instead, we have marked networks that have more than 50% of video calls made using a point-to-point protocol (PPP) connection. ISPs use PPP to give dial up Internet access to their customers, for example over 3G mobile networks, whose network characteristics may differ significantly from regular, fixed-line types of connections. We also distinguish large networks for which we received a lot of user feedback.
network2.png

At Skype we are constantly working to deliver the best possible call quality and highest satisfaction for all Skype users. We want everyone using Skype to have a high-quality video calling experience; independent of the network they are on. As can be seen from the figure above, there is significant variation in user experience by location/country. We have explored some of the reasons for low user ratings by modelling them with objective network quality parameters such as allocated bandwidth, round-trip time, jitter and packet loss. Through this modelling, we’ve found that there is correlation between different network quality technical parameters and average feedback ratings of the network. We conclude that in many cases, users in the countries featured in the second figure who are experiencing poor video call quality may be able to significantly increase their call quality by evaluating / selecting alternative network providers.

Data used

The data used in our analysis is based on anonymized feedback on Skype two-way video calls made in September 2011. We used data only from calls made between the released (non Beta) 5.x versions of Windows clients (earlier than versions 5.5 and 5.6). This data was selected to avoid bias due to differences in platform and version usage. Additionally, for calculating average user feedback rating for networks, we used only calls with a direct (UDP transport stream) connection in order to avoid bias resulting from differences due to transport streams and relayed calls.

User feedback was collected based on random sampling. After each call, there is a random mechanism which decides whether to trigger a pop-up window on any of the participant’s clients, where the user can rate the call quality in the form of a subjective opinion score that ranges in between 1 = very bad, 2 = poor, 3 = fair, 4 = good and 5 = excellent. Subjective opinion scores from each consistent set of users were subsequently averaged for networks and countries. Since we ask user feedback from only a small fraction of calls, we excluded smaller networks from the study, and used only networks where we had more than 1000 user feedbacks collected during September. To be able to present differences within countries, we used only countries with at least 3 different networks.

Notes
All data analysis was done using the R Project software for Statistical Computing [http://www.r-project.org/].
The data includes GeoLite data created by MaxMind, available from http://www.maxmind.com/.
The shapefile used by R libraries was obtained from http://thematicmapping.org/downloads/world_borders.php, originally downloaded from the Mapping Hacks website: http://www.mappinghacks.com/data/. The world borders dataset is made available under a Creative Commons Attribution-Share Alike License. [http://creativecommons.org/]

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s