Evaluation of online emoji description resources for sentiment analysis purposes

Abstract

Emoji sentiment analysis is a relevant research topic nowadays, for which emoji sentiment lexica are key assets. Manual annotation affects directly their quality (where high quality usually corresponds to high self-agreement and inter-agreement). In this work we present an unsupervised methodology to evaluate emoji sentiment lexica generated from online resources, based on a correlation analysis between a gold standard and the scores resulting from the sentiment analysis of the emoji descriptions in those resources. We consider in our study four such online resources of emoji descriptions: Emojipedia, Emojis.wiki, CLDR emoji character annotations and iEmoji. These resources provide knowledge about real (intended) emoji meanings from different author approaches and perspectives. We also present the automatic creation of a joint lexicon where the sentiment of a given emoji is obtained by averaging its scores from the unsupervised analysis of all the resources involved. The results for the joint lexicon are highly promising, suggesting that valuable subjective information can be inferred from authors’ descriptions in online resources.