For citation:
Romanov A. I., Batraeva I. A. Attention based collaborative filtering. Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 2022, vol. 22, iss. 1, pp. 103-111. DOI: 10.18500/1816-9791-2022-22-1-103-111, EDN: CJRZZE
Attention based collaborative filtering
Attention mechanism invention was an important milestone in the development of the Natural Language Processing domain. It found many applications in different fields, like churn prediction, computer vision, speech recognition, and so on. Many state-of-the-art models are based on attention mechanisms, especially in NLP. As this technique is very powerful, we decided to investigate its application in solving a collaborative filtering problem. In this paper, we propose a standard framework for developing a recommender system engine based on transformer architecture. We could not reproduce current state-of-the-art results on MovieLens datasets, but in our implementation attention based model achieves competitive scores on MovieLens 1M and MovieLens 10M datasets.
- Gomes-Uribe C. A., Hant N. The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Transactions on Management Information Systems, 2016, vol. 6, no. 4, pp. 1–19. https://doi.org/10.1145/2843948
- Liu S., Bismas P. K. A Hybrid Recommender System for Recommending Smartphones to Prospective Customers. CoRR, 2021, vol. 23, pp. 2–3.
- Stromqvist Z. Matrix factorization in recommender systems. How sensitive are matrix factorization models to sparsity? Department of Statistics Uppsala University, 2018. 26 p.
- Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, Tat-Seng Chua. Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017, vol. 10, pp. 335–344. https://doi.org/10.1145/3077136.3080797
- Wang K., Peng H., Jin Y. Sha Ch., Wang X. Local Weighted Matrix Factorization for Top-n Recommendation with Implicit Feedback. Data Science and Engineering, 2016, vol. 1, pp. 252–264. https://doi.org/10.1007/s41019-017-0032-6
- Rendle S., Freudenthaler C., Gantner Z., Schmidt-Thieme L. Bayesian Personalized Ranking from Implicit Feedback. Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, 2009, vol. 10, pp. 452–461.
- Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, Tat-Seng Chua. Neural Collaborative Filtering. Proceedings of the 26th International Conference on World Wide Web, 2017, vol. 10, pp. 173–182. https://doi.org/10.1145/3038912.3052569
- Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser L., Polosukhin I. Attention is all you need. Conference on Neural Information Processing Systems, 2017, vol. 15, pp. 4–5.
- Mingsheng F., Hong Q., Dagmawi M., Li L. Attention based collaborative filtering. Neurocomputing, 2018, vol. 311, pp. 88–98. https://doi.org/10.1016/j.neucom.2018.05.049
- Velickovic P., Casanova A., Lio P., Cucurull G., Romero A., Bengio, Y. Graph attention networks. 6th International Conference on Learning Representations, ICLR 2018 – Conference Track Proceedings, 2018, vol. 12, pp. 2–3. https://doi.org/10.17863/CAM.48429
- Harper F. M., Konstan J. A. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 2015, vol. 5, iss. 4, pp. 1–19. https://doi.org/10.1145/2827872
- Rendle S., Zhang L., Koren Y. On the Difficulty of Evaluating Baselines. ArXiv preprint arXiv:1905.01395, 2019, vol. 19, pp. 1–3.
- 1566 reads