Rithwik Chitla, Ishaan Mali, Benescia Kashyap, Raphael Fohine

DOI: https://doi.org/10.5281/zenodo.22726664

Large Language Models (LLMs) have demonstrated strong potential as recommendation engines, leveraging their broad world knowledge and natural language understanding to generate personalized item suggestions. However, LLMs inherit and amplify popularity bias present in their pretraining corpora, consistently recommending well-known items while neglecting the vast majority of the catalog. This paper presents a systematic study of popularity bias across four inference and training regimes applied to Qwen2.5 models of two scales (1.5B and 3B parameters) on the MovieLens-100K benchmark. We show that zero-shot and few-shot prompting produce heavily skewed recommendations, and that naive Supervised Fine-Tuning (SFT) inherits the dataset’s popularity distribution (71.4\% head items in raw training pairs). We introduce a popularity-capped sampling strategy that rebalances SFT training data to 73.9\% tail items, substantially improving fairness. Building on this, we apply Direct Preference Optimization (DPO) using preference pairs constructed from user ratings, where long-tail liked items serve as chosen responses and popular disliked items serve as rejected responses. Our best configuration (Qwen2.5-3B, SFT + DPO) achieves a 59.2\% reduction in Average Recommendation Popularity (ARP) and a Tail Coverage (TailCov) of 95.45\%, compared to 37.21\% for the zero-shot baseline. These results demonstrate that alignment-stage training is a powerful and practical tool for fairness in LLM-based recommendation.

artificial intelligence, bias mitigation, direct preference optimization, large language models, recommendation systems

We have presented a systematic study of popularity bias in LLM-based recommendation, covering four training regimes applied to Qwen2.5 models at 1.5B and 3B scales on MovieLens-100K. We showed that raw SFT inherits and perpetuates dataset-level popularity skew, and that a simple capping strategy combined with DPO preference alignment can reduce ARP by 59.2% and achieve 95.45\% tail coverage for the 3B model. Our findings establish a practical, compute-efficient recipe for fairness-aware LLM recommendation that runs on a single consumer-grade GPU. Future work will extend to larger datasets, multi-item recommendation, and composite fairness objectives incorporating user-side demographic equity.

[1]Y. Hou, J. Zhang, Z. Lin, H. Lu, R. Xie, J. McAuley, and W. X. Zhao, "Large Language Models are Zero-Shot Rankers for Recommender Systems," Proceedings of the 46th European Conference on Information Retrieval (ECIR), pp. 364–381, 2024.
[2]K. Bao, J. Zhang, Y. Zhang, W. Wang, F. Feng, and X. He, "TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation," Proceedings of the 17th ACM Conference on Recommender Systems (RecSys), pp. 1006–1014, 2023.
[3]J. Zhang, R. Xie, Y. Hou, W. X. Zhao, L. Lin, and J. Wen, "Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach," Proceedings of the 17th ACM Conference on Recommender Systems (RecSys), 2023.
[4]H. Abdollahpouri, M. Mansoury, R. Burke, and B. Mobasher, "Managing Popularity Bias in Recommender Systems with Personalized Re-ranking," Proceedings of the AAAI Workshop on Reducing Online Misinformation through Credible Information Retrieval, 2019.
[5]Y. Deldjoo, M. Schedl, P. Cremonesi, and G. Pasi, "Explaining Popularity Bias in Recommendation Systems," Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1834–1838, 2021.
[6]Y. Zhang, F. Feng, X. He, T. Wei, C. Song, G. Ling, and Y. Zhang, "Causal Intervention for Leveraging Popularity Bias in Recommendation," Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 11–20, 2021.
[7]T. Wei, F. Feng, J. Chen, Z. Wu, J. Yi, and X. He, "Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System," Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1791–1800, 2021.
[8]R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, "Direct Preference Optimization: Your Language Model is Secretly a Reward Model," Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023.
[9]Y. Saito, S. Yaginuma, Y. Nishino, H. Sakata, and K. Nakata, "Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback," Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM), pp. 501–509, 2020.
[10]L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., "Training Language Models to Follow Instructions with Human Feedback," Advances in Neural Information Processing Systems (NeurIPS), vol. 35, pp. 27730–27744, 2022.
[11]K. Tian, E. Mitchell, H. Yao, C. D. Manning, and C. Finn, "Fine-tuning Language Models for Factuality," International Conference on Learning Representations (ICLR), 2024.
[12]T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, "QLoRA: Efficient Finetuning of Quantized LLMs," Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023.
[13]F. M. Harper and J. A. Konstan, "The MovieLens Datasets: History and Context," ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 5, no. 4, pp. 1–19, 2015.
[14]Qwen Team, "Qwen2.5 Technical Report," arXiv preprint arXiv:2412.15115, 2025.
[15]L. von Werra, Y. Belkada, L. Tunstall, E. Beeching, T. Thrush, N. Lambert, and S. Huang, "TRL: Transformer Reinforcement Learning," GitHub repository, 2022. [Online]

Rithwik Chitla, Ishaan Mali, Benescia Kashyap, Raphael Fohine (2026). From Bias to Fairness: Mitigating Popularity Bias in LLM-Based Recommendation Systems via Supervised Fine-Tuning and Direct Preference Optimization. International Journal of Computer Science Engineering Techniques, 10(5), 158–167. ISSN: 2455-135X. DOI: https://doi.org/10.5281/zenodo.22726664