Emotion-based Image Recommendation System

Abstract

Recommendation systems play a crucial role in today’s digital society by helping users make decisions when so much information is available. Among other types of content, images are ingrained in people’s daily lives and can significantly influence their emotional states. Because of this intense emotional impact, images have become important tools in therapeutic contexts for the treatment of dementia and post-traumatic stress. By considering emotional responses, an image recommendation system can personalize and make recommendations according to emotions, increasing relevance and effectiveness. This work focuses on developing an image recommendation system where emotions felt by the user are a feature for the recommendation. This study was done with a new dataset of the EmoRecSys Project that is still under construction. To do that, we developed an emotion-based recommendation system based on the user’s declared explicit emotions from the dataset. To evaluate the performance of the proposed recommendation system, two more recommendation systems that are more widely studied in the literature were developed: pixel and metadata, as well as a random recommendation system that works as a baseline. The systems’ performance was evaluated using precision, recall, f1-score, and normalized discounted cumulative gain. The results at top@10 for precision were 0.119, 0.113, and 0.111 for the emotion-, pixel-, and metadata-based, respectively. Regarding recall, at top@10, the recalls are 0.542, 0.505, and 0.492 for the emotion-, pixel-, and metadata-based, respectively. In the normalized discounted cumulative gain, emotions, pixels, and annotations have top@10 of 0.302, 0.299, and 0.285. The results show that the recommendation of images based on emotions achieve better results than using state of the art features, such as pixel and metadata similarities