Recommender Systems Evaluation: A 3D Benchmark

AutorAlan Said, Domonkos Tikk, Klara Stumpf, Yue Shi, Martha Larson and Paolo Cremonesi
QuelleACM RecSys 2012 International Workshop on Recommendation Utility Evaluation: Beyond RMSE (RUE '12) 
LinksBibTeX   |   Uni-Bibliothek 

Recommender systems add value to vast content resources by matching users with items of interest. In recent years, immense progress has been made in recommendation techniques. The evaluation of these has however not been matched and is threatening to impede the further development of recommender systems. In this paper we propose an approach that addresses this impasse by formulating a novel evaluation concept adopting aspects from recommender systems research and industry. Our model can express the quality of a recommender algorithm from three perspectives, the end consumer (user), the service provider and the vendor (business and technique for both). We review current benchmarking activities and point out their shortcomings, which are addressed by our model. We also explain how our 3D benchmarking framework would apply to a specific use case.