Learning to converse with empathy in open-domain dialogue systems

Abstract

Conversational agents are chat-oriented systems that interact and communicate with humans to serve various purposes. They can focus on specific tasks and help users with certain goals such as booking a restaurant, or simply converse with humans (which is more commonly known as chatbots). The goal of such chatbots (the focus of this thesis) is to mimic human-to-human conversations and have a prolonged and engaging dialogue with human users. Such agents have been modeled with several different methods from hand-crafted rules (e.g. ELIZA, PARRY, ALICE) to, more recently, deep neural networks. On top of being one of the major challenges in artificial intelligence, chatbots can also serve many other practical purposes ranging from psychological counseling to customer service. One of the major milestones to attain in order to build human-like chatbots is to model and incorporate empathy because human conversations often involve the sharing of emotions and feelings. More specifically, in the context of conversational agents, an empathetic dialogue system should be able to not only understand how the user currently feels from the dialogue history but also properly address that emotion by appropriately responding towards it. In addition, properly addressing the user’s emotions have been shown to be beneficial in multiple aspects such as enhancing user satisfaction, decreasing dialogue breakdown, and relieving user’s stress. In light of such multi-faceted benefits, we focus on teaching empathy to conversational agents in this thesis. Some of the initial works on empathetic dialogue systems include rule-based chatbots which created dialogue managers based on user emotions and affective listeners that responded in both content and affect level. However, the limitations of such rule-based systems are clear in terms of scalability to dataset size and generalizability to different domains and situations. More recently, neural conversation models have been successful in generating fluent and relevant responses, but their responses were widely known to be dull and generic due to the maximum likelihood objective that does not factor in any kind of emotional exchange, or empathy. On such note, several recent works focused on mainly two directions in attempting to model empathy in dialogues. The first line of work has been successful in controlling and conditioning the generated responses to certain sentiments, emotions, and emojis. Meanwhile, others have worked on more data-driven approaches by training a model to jointly predict the current emotional state while generating a response. While both approaches have been successful to a certain extent, they have neglected some crucial issues of empathetic response generation. The first approach - controlled text generation - assumes that the emotion to condition the response is given as an input, but we often do not know which emotion is appropriate to be empathetic. The latter takes the assumption that understanding the users current emotion will let the model implicitly learn how to respond empathetically. However, recognizing the current emotional state does not guarantee that the model has learned to respond appropriately toward that emotion. To cope with such issues, we propose an end-to-end approach that mainly addresses the problem of responding appropriately. Instead of considering only the current user emotion as in the previous literature, we look at the future emotion of the user towards the generated system utterance. More precisely, we propose to model this as a reinforcement learning problem, in which the reward signal to maximize is given as the (predicted) sentiment of the next user turn, namely sentiment look-ahead. We use reinforcement learning as it is the most natural formulation of sentiment look-ahead. Intuitively, an empathetic person would first consider the consequences before speaking, which, in turn, would naturally improve his or her conversational partners feelings. Finally, we verify the effectiveness of our approach through preliminary experiments with human evaluations on the idea of sentiment look-ahead. Based on such, we analyze the errors and weaknesses, and further propose three different improved implementations of sentiment look-ahead along with much more thorough evaluation metrics that correlate well with human judges. Based on the experiments and automatic/human evaluations, our best performing reward model significantly outperforms other models in terms of Empathy, Relevance, and Fluency, verifying the effectiveness of our novel viewpoint about empathy.

Date of Award	2020
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology