https://medium.com/voice-tech-podcast/a-simple-way-to-explain-the-recommendation-engine-in- ai-d1a609f59d97 The digital public sphere is currently dominated by recommendation engines that act as curation and moderation tools for social media, video, and news. These are designed to tackle the challenge of too much information to consume by distilling it to only what the user might be interested in. While these systems perform reasonably well at this task, their mathematical formulations unfortunately lead to subtly dysfunctional outcomes. The reward functions utilized to train these systems are optimized for engagement, and therefore prioritize extreme, inflamed, and divisive content as a means of increasing overall consumption time. The root cause for this phenomenon is the optimization function which sits at the heart of most recommender systems. Collaborative filtering, deep ranking networks, and reinforcement learning-based approaches all share the property that they are optimized to maximize some proxy metric of user engagement. Empirical research in the space has repeatedly shown that this is an inappropriate proxy to optimize over, since human psychology is such that engagement is increased through negative, inflamed, and controversial content. A natural consequence of this phenomenon is that recommender systems begin to exhibit a propensity to surface divisive, extreme, or outlier content in order to maximize overall consumption time. One particularly insidious result of engagement-driven algorithms is content drift. This occurs when the recommendation algorithm nudges a user to gradually consume more and more niche content in a particular domain. In politically charged domains, this can lead to users being fed extremely radicalizing content that would not have been directly consumed or even discovered by the user. This is particularly concerning since engagement is rewarded even when the algorithm is correct in its assessment that the user will never change their mind. While challenging to measure, the link between recommendation engines and increased polarization is a topic of active research by computational social scientists. In the early days, it was unclear whether such an effect even existed, as users may well have actively sought out similar content on their own. More recent work using audit tools, large language model-based sockpuppet accounts, and carefully designed field experiments has begun to quantify this effect in detail. The audit technique has the advantage of being able to precisely measure the effect of a recommender system by having full knowledge of the counterfactual situation (had the user not interacted with the recommender system). These works consistently find that on divisive topics (especially conspiracy theories and politics) there is an increased probability of getting radicalized by a recommendation engine. Neutral topics, on the other hand, tend to get normalized, as the system nudges towards a particular viewpoint. It is important for regulators, platform moderators, and computer scientists to develop a shared understanding of the different ways in which recommendation engines can drive users towards more extreme views, and to what degree. This is a critical foundation for future algorithmic auditing and is a key component of the EU's proposed Digital Services Act. Counteracting the engagement-driven extremism of current systems is an active area of research by computer scientists working on recommendation systems. One promising approach is to utilize a multi-objective function during training that, besides maximizing consumption, also penalizes the discovery of extreme, divisive, or radicalizing content. This approach requires careful engineering to define appropriate metrics for such values as informational diversity or bridging divides, but has the advantage of being relatively straightforward to implement.
Sources
https://knightcolumbia.org/research/algorithmic-amplification-and-society https://www.techpolicy.press/algorithms-shift-polarization-why-does-policy-still-miss-the-real-pro blem/ https://arxiv.org/html/2603.20723v1 https://academic.oup.com/pnasnexus/article/doi/10.1093/pnasnexus/pgaf062/8052060 https://pubmed.ncbi.nlm.nih.gov/41308156/