Why ratings are a trap…

For many DTV platforms, OTT providers and streaming platforms, showcasing the very best content you have to users is the obvious place to go when thinking about your optimal customer experience. Your popular shows have the best talent, the biggest budgets and you and your team love them to bits! These shows really work for acquisition marketing and drive loads of new customers in. Putting these top shows all over your UX is also the best way to get the highest ratings and that is the key metrics everybody judges me on. Surely that is success?

However, here at The Filter have found that focusing just on your most popular content can lead to some major issues for customers:

  1. A platform can appear very static as the same content is surfaced to users’ week on week. This can often lead users to having a perception that a platform has a limited catalogue.
  2. Platforms can also present content to users that is just not relevant to them. If you have never watched a horror show or just don’t like sports, no matter how popular other people find it, you will just not engage with it.

The key metrics you should be focused on as a platform is not ratings (total number of views for your top shows) but on much more customer centric metrics like content assets watched per month, bounce rate (how many times a user comes to your platform and does not end up watching something) and, very importantly, time taken to start watching a show.

In reality, a limited catalogue if often not to blame for this perception as most platforms we work with have a deep and rich catalogue. But if users cannot easily see and find the content that they want to watch then and now, then do not expect them to hang around to find out. In the current battle between OTT providers, the winners over time will be the ones with the best customer experience as well as content.

To understand if popularity bias is a problem on your platform, look at the frequency decay for your shows i.e., what percentage of content gets the most plays. We often see a classic viewing curve emerge where a small number of titles account for the majority of plays. The example below, demonstrates how easy it is to force users to watch just a small percentage of titles.

We find that the optimal mix is 20% of your titles should account for 80% of plays.

The other way to see if popularity bias is a problem is by taking a time-based series of screen shots of your home page for a typical customer. Do this every day for a few weeks. Then put them on a wall and look at the differences over time. How does it vary? We often find that it does not vary as much as you would expect and moreover, the UX is dominated by the same top shows. If customers come back to the same shows every time they visit, they will start to look for other options.

So why does this happen? Often this can happen through a combination of factors, most notably content prioritisation by editorial teams and model configuration for algorithms you might be using.

Editorial teams, quite intuitively, will want to promote the very latest content, push the new releases or the new original content that is a key commercial driver to that platform. Top executives talk about shows that get ‘x’ million views. With this mindset, flagship content will remain front and centre on a platform, whether relevant to a user or not. Even worse, these flagship shows are often continually presented to an individual after they have finished watching the content.

You then add model configuration into the mix for popularity models such as Trending or Most Popular. They are designed to take user input data across the platform and then present back the popular content to users. These rails are very effective in gathering clicks and views as users gravitate to these ‘social proof’ recommendations. If you then add to this mix collaborative filtering models which inherently look at user’s behaviour and other content viewed, then it is easy to see why this type of popularity ‘squeeze’ can manifest into a narrow content view for all users. In effect popularity becomes self-fulfilling and over time will get worse.

What can be done to avoid this trap? Key to avoiding this problem is be clear on what metrics are important to you and will have most effect on the big numbers, like say customer lifetime value. For SVOD platforms this is generally churn and for AVOD this is active users which, in our experience, can both be correlated to total views per month. Your most popular content has a vital role to play in this but not on its own. Here are some of The Filter’s thoughts on how to avoid overly focusing on the most popular content:

  1. Get Editorial teams to work alongside algorithms – One way of pushing the latest content on your platform but keeping it fresh it to use simple mechanisms to rotate, edit and manage ‘hero’ content on the platform. Our ‘Hero’ tool lets editorial teams pick a sub-set of users to show in this prime real estate but then lets a model learn what is best to show each user based on their behaviour and preferences and excludes content if they are watching already.
  2. Create dynamic algorithms that change frequently – by adapting recommendation algorithms to train and change often, you can create a sense of the platform being dynamic and changing regularly. This prevents over saturation of content, presents different content to users regularly and gives the feeling that new content is being added or brought to a user’s attention which improves the feeling of catalogue depth.
  3. Increase personalisation – by increasing the level of personalisation to a platform makes every rail or page more relevant to a user. This can be achieved not just in recommendation rails, but in how genre or category rails are served, in what order and let the UI be driven by user behaviours and what they like to watch. Leveraging our AI capabilities to determine the optimal rail position will stop users from seeing the same content and reduce the popularity bubble that can occur over time.
  4. Cater for Early Visits – have a different customer experience for the first time a user arrives and for when they are a new user with limited viewing history. In a customer’s early life on a platform, they probably are coming in for the most popular content that featured in your acquisition marketing. But as you gradually get to know them and their tastes and as the excitement of the popular shows wanes, you should offer a much more diverse experience centred on their likes.
  5. Steer by the right KPIs – change the KPI balance to be one where engagement matters more than just pure views or plays. Getting users to watch multiple titles over a period of time is far more valuable than achieving lots of users watching one thing. The net affect will be a reduction in churn and improved profitability.


About The Filter

The Filter offers personalisation of key customer touch points based on proven data science. We put the right content in front of a customer at the right time. More views means more loyalty, lower churn and more revenue.

The Filter prides itself on working closely with clients both during set-up and afterwards to make sure that the client’s customers are always getting the optimal recommendations. We are an agency not a SaaS provider. We have a genuinely deep TV understanding which means our many data science solutions work exceptionally well for your viewers. We work with you to tailor our AI to your customer experience, your audiences and your content. We also integrate effortlessly into your data ecosystem allowing you to benefit from the power of personalisation easily and quickly.

If you would like to know more, please email damien.read@thefilter.com