Evaluation of abstraction capabilities and detection of discomfort with a newscaster chatbot for entertaining elderly users
DATE:
2021-08-17
UNIVERSAL IDENTIFIER: http://hdl.handle.net/11093/2572
EDITED VERSION: https://www.mdpi.com/1424-8220/21/16/5515
UNESCO SUBJECT: 3325 Tecnología de las Telecomunicaciones ; 5802.01 Educación de Adultos ; 5701.04 Lingüística Informatizada
DOCUMENT TYPE: article
ABSTRACT
We recently proposed a novel intelligent newscaster chatbot for digital inclusion. Its controlled dialogue stages (consisting of sequences of questions that are generated with hybrid Natural Language Generation techniques based on the content) support entertaining personalisation, where user interest is estimated by analysing the sentiment of his/her answers. A differential feature of our approach is its automatic and transparent monitoring of the abstraction skills of the target users. In this work we improve the chatbot by introducing enhanced monitoring metrics based on the distance of the user responses to an accurate characterisation of the news content. We then evaluate abstraction capabilities depending on user sentiment about the news and propose a Machine Learning model to detect users that experience discomfort with precision, recall, F1 and accuracy levels over 80%.