Not Finding Popularity Bias in Recommendations from Music Streaming Services @ ISMIR 2022

One of the emerging topics in machine learning research is “fairness” and “bias” in recommendation algorithms. For example, when a job site only recommends certain applicants for certain high-value jobs, it can impact who applies for and ultimately gets hired into these desirable jobs. The recommender system is then rewarded for making a “successful” recommendation and thus learns to keep recommending similar applicants in the future. This kind of feedback loop can lead to a biased decision-making process in which the “rich get richer”.

In the context of music, it is well-known that consumption follows a long-tail distribution (see my post about the Equality of Outcomes) in which a small number of artists receive the lion’s share of attention. It should be noted that this has been the case long before music streaming existed and may result from both social pressures (see one of my all-time favorite papers on Artificial Cultural Markets but Salganik, Dodds, and Watts) and familiarity bias (a.k.a, the Mere Exposure Effect).

However, popularity bias in music recommender systems could be another contributing factor to this “unfair” marketplace. Popularity bias is the concept that the artists recommended to users are more popular that the artists that the users typically listen to. The conspiracy theory here is that major record labels, which partly own and license music to streaming music services, would like songs by their “mainstream” artists to be exposed to listeners, and as a result, crowd out music by independent artists. Aside from this conspiracy theory, academic research has shown that standard recommendation algorithms often do result in algorithmic popularity bias both in the context of music recommendation (Abdollahpouri 2019) and music recommendation (Kowald 2020).

Last year, my students Sean McQuillan, Vera Crabtree, John Hunter, Sunny Zhang and I decided to look at this concept of recommendation-based popularity bias in two contexts: algorithmic bias and commercial bias. While we did find evidence of algorithmic popularity bias for three state-of-the-art recommendation algorithms (see our technical report), we found no evidence of commercial popularity bias in three streaming music platforms (Spotify, YouTube, and Amazon Music) when conducting a small-scale simulated user experiment.

This is a surprising result because it is counterevidence to the conspiracy theory that record labels are working in cahoots with streaming services to use personalized recommendations as a means to dominate the marketplace of music consumption. That said, personalized recommendation (e.g., “Daily Mix” playlists on Spotify) is just one of many ways that major record labels and streaming services can influence listening preference and consumption patterns. (See the excellent work by Luis Aguiar on “Platform Power and Platform Bias in the Music Industry” for more detail.)

We will be presenting our work “Exploring Popularity Bias in Music Steaming Services” a Late-Breaking Demo paper ” at this week’s International Society for Music Information Retrieval (ISMIR) conference. Please check out the paper if you want to know more about how we conducted the simulated user experiment and, in particular, the limitations of this work.