The popularization of social networking services and the improvement in computer technology offer the potential of open-domain conversational dialogue systems that can be trained in an end-to-end manner without any handcoding. Specifically, a sequence to sequence (seq2seq) model [Sutskever et al., 2014; Vinyals and Le, 2015] is the most common approach in the field of dialogue generation, and it can generate fluent responses. However, dialogue systems which only rely on utterances and responses as training data sometimes generate generic and uninformative responses [Vougiouklis et al., 2016; Ye et al., 2018; Ke et al., 2018].
In order to solve these problems, various works were proposed to generate more human-like responses by exploiting a set of textual facts, such as review posts [Ghazvininejad et al., 2018; Moghe et al., 2018], character description as persona [Zhang et al., 2018], and fragments of articles from web-sites
Figure 1: Typical example of fact-based dialogue session. The words with underline correspond to the facts. The new topics that are drifted from the candidates of topics are shown in bold.
[Galley et al., 2019; Tam et al., 2019]. These works mainly focus on integrating facts based on a topic of input utterances into dialogue systems. However, this scheme tends to converse passively while keeping the given topic and has a dif-ficulty to bring diverse and proactive communication, as we can often observe in our daily conversation.
To address this issue, this paper makes a step towards creating another type of fact-based dialogue system: endowing it with the ability of a convergent and divergent thinking over both context and facts. This is inspired by human thought method [Guilford, 1967]. In general, as shown in Figure 1, humans usually converses by exploiting available information (i.e., convergent thinking) as well as exploring many possible directions (i.e., divergent thinking) within the interactions. For example, #1 and #2 utterances conduct the convergent thinking to talk about the topic of the character spider-man by utilizing the facts. Another remarkable phenomenon is that the conversation usually drifts from one topic to other similar topics. Utterances in item #3 and #4 perform the divergent thinking to introduce the new topics of Tokyo and Kyoto by expanding from the topic candidates of NYC and Tokyo that are present in facts and context, respectively.
This has the potential of enabling the system to chat naturally with humans on open-domain topics and proactively lead the conversation smoothly, which results in more informative and diverse conversation. Motivated by this, we propose a fact-based neural conversational model equipped with a convergent and divergent decoding, which can converse about the current topic or drift from one topic to another. Specifically, our model can enable the decoder to attend and copy tokens from source inputs including context and facts (i.e., convergent decoding) or inputs-related topics that are produced by a topic drifter (i.e., divergent decoding), considering the decision made by a decoding switcher. We test the effectiveness of our proposed model on DSTC7 dataset [Gal- ley et al., 2019] that is one of the most large-scale and challenging benchmarks based on real-world conversation. Both automatic and human evaluation results show that our method significantly outperforms state-of-the-art methods in producing more appropriate, informative, and diverse responses.
The contributions of this paper can be summarized below:
• To the best of our knowledge, we first incorporate a concept of convergent and divergent thinking over both context and facts into fact-based dialogue system.
• We propose a fact-based dialogue system with a convergent and divergent decoding, which can converse about the current topic or drift from one topic to another.
• Experiments show that our model outperforms state-of-the-art models by a large margin in terms of both automatic and human evaluations.
2.1 Task Definition
The task of fact-based dialogue generation can be stated as follows: given a dialogue context and a collection of K supporting facts
1, ..., K), the goal is to generate a response as a sequence of tokens
. We formalize this setting by following DSTC7 sentence generation task [Galley et al., 2019]. The length of the context is limited to the past six utterance at most, and these are concatenated by the special symbol <SEP> representing a break in the utterance. Moreover, the supporting facts are shown in text fragments extracted from various web-sites based on the topics of conversation.
2.2 Model Overview
The architecture overview of our proposed model is presented in Figure 2, and it consists of four major components: context & facts encoders (2.3), topic drifter (
2.4), decoding switcher (
2.5), and decoder (
2.6). The context & facts encoders encode
and
into hidden vectors
and
, respectively. The topic drifter outputs inputs-related topics (drift words)
and
that can be expanded from candidates of current topics in
and
, respectively. In the next utterance, the decoding switcher predicts the probabilities of executing the convergent decoding or divergent decoding. Based on the decision made by the decoding switcher, the decoder attends and copies tokens from source inputs;
Figure 2: Architecture overview of our model.
context and facts (convergent decoding) or drift words that are produced by the topic drifter (divergent decoding).
2.3 Context and Facts Encoders
We implement the encoder with forward and backward Gated Recurrent Units (GRU) [Chung et al., 2014]. The encoder converts a dialogue context and a collection of facts into two hidden state sequences and
, respectively.
where the bracket denotes the vector concatenation. Here,
and
where d is the hidden size are vectors concatenated with respect to all the forward hidden states
and backward states
as well. Moreover,
is the pre-trained GloVe word embedding [Pennington et al., 2014]. To reduce the number of parameters, both encoders share weights and word embeddings.
2.4 Topic Drifter
Topic words are one of the crucial factors for generating an informative response [Ke et al., 2018]. However, repeating a response based on a certain topic hurts diversity and smoothness of the conversation. To overcome this problem, the topic drifter outputs inputs-related words called drift words to expand from candidates of current topics.
The topic drifter performs two processes to output the drift words from context and facts. Firstly, candidates of current topic words and
are extracted from context and facts, respectively. Specifi-cally, we apply the existing topic words extracter [Tang et al., 2019] which combines TF-IDF and Part-Of-Speech features for scoring word salience. Secondly, the cosine similarity between a word from vocabulary constructed using the training set and each extracted topic word is applied, and the most related
words are outputted. After these processes, we finally obtain contextual drift words
and factual drift words
, where L =
and
.
2.5 Decoding Switcher
Our proposed model has two types of decoding methods for generating a response. The first one is a convergent decoding that can copy tokens from the given inputs including context and facts, and the second one is a divergent decoding that can output a response that contains the drift words obtained by the topic drifter. The decoding switcher selects which the decoding type should be used in the next system utterance. Specifi-cally, the switcher is modeled as a binary classifier that takes a dialogue context and a set of supporting facts, and it predicts the probability , where
means the convergent decoding and
means the divergent decoding. The probability of this binary classifier is calculated as:
where and
are learnable parameters and
, simply the average pooling over all last hidden states of the facts encoder.
Furthermore, we introduce a switch loss to better supervise this switcher explicitly. Intuitively, when any facts are not matched the conversation, drifting the topic is necessary (i.e., should be higher) because the incorporation of irrelevant facts may be noisy signals of the model. To this end, we define a correct label
for each sample. This label is set to 1 if the output word exists in the topic words of the facts and 0 otherwise. The switch loss is defined as follows:
2.6 Decoder
Based on the decision made by the decoding switcher, the decoder generates a response word-by-word sequentially with controlling the output distribution.
The t-th hidden state of the decoder is computed autoregressively with the forward GRU where the initial hidden state
is
and the initial word
is the special symbol <SOS> denoting the start of sentence.
The probability distribution over the fixed vocabulary is obtained by passing the decoder hidden state
,
where and
are parameters to be learned.
Multi-Source Attention and Copy Mechanism After estimating the t-th decoder hidden state , we obtain the context vectors
,
,
and the copy distributions
, by leveraging the attention mechanism [Bahdanau et al., 2015] and the copy mechanism [See et al., 2017] for the dialogue context, facts, contextual drift words, and factual drift words.
For the dialogue context, the attention weight and the context vector
are calculated based on the hidden states
obtained from the context encoder and the decoder hidden state
(Eq. 7). Then, the copy distribution
is calculated by integrating the attention weights (Eq. 8).
In order not to attend words in irrelevant facts, for calculating the attention of facts, the hierarchical attention mechanism [Yavuz et al., 2019] which combines both the word-level and sentence-level attention is applied. First, and
are calculated for each word (Eq. 9). Subsequently,
and
are calculated for each fact (Eq. 10). To compute the copy distribution of facts, Eq. 11 is used.
For the contextual and factual drift words, we compute the attention weights and the context vectors
(Eq. 12, 13) based on the embeddings of drift words and the t-th decoder hidden state. In addition, the attention weights are summed up to obtain the distribution over the contextual and factual drift words
and
(Eq. 14, 15).
Convergent and Divergent Decoding
The final output distribution is determined according to the decision made by the decoding switcher. Let
be a output probability of
from the convergent decoding, and
be a output probability of
from the divergent decoding. Then, these probabilities can be calculated based on the following formula:
where is learnable parameter. The final distribution of
is defined as the following mixture of the distributions:
As can be seen Eq.20, the predicted by the decoding switcher only affects the copy distributions in Eq. 16 and Eq. 17 (i.e., not affects the vocabulary distribution). In order to control the output distribution more efficiently, if the output word
exists in the copy sources, using the copy distributions preferentially is ideal. To this end, we devise a copy loss. Let
be a mixing factor of the t-th copy distribution. Then, the copy loss is calculated as follows:
where is a correct label defined by checking whether the t-th output word
exists in the copy sources.
2.7 Multi-Task Learning
Our model is trained using minimization of the weighted-sum of three losses as:
where and
are hyper-parameters to balance loss, and
is the negative log-likelihood loss in the seq2seq learning [Sutskever et al., 2014].
3.1 Dataset
Table 1: Statistics of DSTC7 dataset.
We train and evaluate our model on the DSTC7 dataset [Galley et al., 2019] because it is very large scale and good characterization of real-world conversations. Table 1 shows the statistical information of the DSTC7 dataset used in our experiment. The dialogue data is crawled from Reddit1, and the facts data is crawled from 226 types of web-sites by using a word matching algorithm. In addition, we follow [Tam et al., 2019] and use IDF-based knowledge extractor, and a maximum of four sentences are collected as a set of facts. All responses and utterances are truncated to have a maximum length of 32, and maximum fact length are set to 50.
3.2 Implementation Details
The implementation of all the models was conducted with PyTorch2. The encoder and decoder have a one-layer GRU structure with 128 hidden states, but these did not share parameters. The word embedding size was set to 300 and the embedding state is initialized using GloVe. During training, the batch size was set to 64 and Adam [Kingma and Ba, 2014] with an initial learning rate of 0.0005 was employed. All models were trained at most 10 epochs, and the model with the minimum loss was selected. The vocabulary was obtained from the dialogue and knowledge data, and the size was set to 30,000. The balancing parameters for loss functions and
were set to 1.0. The hyper-parameter
for the topic drifter was set to 5. The label smoothing technique [Szegedy et al., 2016] with 0.9 for the binary cross entropy was applied. The accuracy of the decoding switcher achieved 68.7% on the dev dataset. For inference, topk sampling [Fan et al., 2018] with k = 10 provided the best results for all models3. This result is also reported in the previous work [Qin et al., 2019].
3.3 Baseline Models
Several models implemented by ourselves are selected to be evaluated as follows:
• S2S is a normal seq2seq that does not have access to the external knowledge [Vinyals and Le, 2015].
• kgS2S is a knowledge-grounded seq2seq model from [Ghazvininejad et al., 2018], where the knowledge is stored in memory units and fed into the decoder.
• PGN is a pointer generator network [See et al., 2017], where the output distribution is integrated with both the vocabulary and context distributions.
• kgPGN is an extended PGN [Tam et al., 2019] that can also copy from the knowledge. This model won the DSTC7 competition.
• DeepCopy is an extended kgPGN model by introducing a hierarchical pointer network. [Yavuz et al., 2019].
In addition to these models, we will also report the automatic evaluation results of the DSTC7 submission models that are taken from the overview paper [Galley et al., 2019].
Furthermore, we also conducted several ablation tests to verify the effectiveness of loss functions and
. Specifically, we set
to 0 during training, respectively. Moreover, we tried to generate different responses by setting the probability
in Eq. 4 to different values in both the training and inference stage. Specifically, we set
to 0 (convergent decoding) and 1 (divergent decoding). Then, in order to suppress the impact of
during training, we set
to 0.
3.4 Evaluation Metrics
Both automatic and human evaluation metrics are used to analyze the model’s performance. For the automatic evaluation, three types of metrics are used: 1) BLEU is used to measure the 4-gram level relevance between the generated responses and the ground-truth. 2) Dist-1,2 are used to evaluate the diversity of N-gram (N = 1, 2) in the generated responses. BLEU and Dist-1,2 are evaluated with the public source code of DSTC74. 3) PMI aims to evaluate how smoothly the generated responses are connected to the context and facts. Similar to [Tanaka et al., 2019b], we calculate this metric based on the point-wise mutual information (PMI) [Church and Hanks, 1990] as follows:
where denotes the content words.
For the human evaluation, the generated responses are manually assessed based on three metrics: 1) Appropriateness (whether the responses are appropriate/relevant to the given context), 2) Informativeness (whether the responses provide meaningful information relevant to the given context), and 3) Fluency (whether the responses are fluent and could plausibly have been produced by humans). We randomly sampled 100 context-response pairs, shuffled the order of systems, and asked three students who have studied in the field of dialogue systems to rate the pairs in terms of model quality on 0 to 2 scales (2 for the best). In Section 4.2, the macro-average score will be reported.
4.1 Automatic Evaluation
Table 2: Automatic evaluation results. The best results are highlighted in bold, and higher value denotes the better performance. BLEU and Dist scores are shown in percentage. The results with * are taken from the DSTC7 paper. denotes the kgPGN model.
Table 2 shows the automatic evaluation results. It can be seen that our model achieves better performance on all the metrics than all the baselines and the submissions to the DSTC7. This indicates that our proposed model can produce diversified responses that are smoothly connected to the source inputs. Moreover, ablation tests show that both the switch loss and the copy loss
significantly boost the diversity scores, and the
also helps to improve the scores of BLEU and PMI. This shows that learning to select which the decoding type should be used explicitly and encouraging our model to use the copy distributions preferentially are vital to improve the performance of our model. Moreover, the combination of two loss functions is necessary if the best performance is to be obtained. The results of the models by setting the probability in Eq. 4 to 0 (convergent decoding) and 1 (divergent decoding) show that our models with
achieve the highest BLEU and Dist-1,2, respectively. This indicates the
can be effectively used to control the behavior of the generated responses.
Figure 3: Effect for adjusting the . Dash lines denote the results of proposed model that utilizes
predicted by the decoding switcher.
Furthermore, the effect of the probability on the generated responses was further evaluated. Specifically, we calculated the score of BLEU, Dist-2, and PMI corresponding to different
values (see Figure 3). According to the results, we can observe that BLEU decreases and Dist-2 increases when the
increases. In other words, there is a trade-off relationship between relevance and diversity. This observation is in line with the previous work [Nakamura et al., 2018]. Most importantly, any models do not achieve higher the score of PMI than our model using the dynamic value. This indicates that our decoding switcher is more effective to produce a response that can be connected to both context and facts smoothly.
4.2 Human Evaluation
Table 3: Human evaluation results.
As shown in Table 3, on the human evaluation, our model shows the highest performance on the all the metrics among all the models. Specifically, Appropriateness and Informativeness are significantly improved while retaining Fluency.
Table 4: Generated examples. The context is manually shortened to fit in the table, without significantly affecting meaning. The new topics that are drifted from the candidates of topics are shown in bold. Minaj, Kanye, and Pitbull are famous for an American rapper. The decoding switcher predicts the in the left and right examples, respectively.
This result indicates that our model can generate more meaningful responses that is related/appropriate to the conversation. Further observation shows that our model using the decoding switcher significantly outperforms our models with in all the metrics. This indicates that selecting the decoding type forcefully hurts response fluency and usefulness for the user. This verify the effects of our model using the decoding switcher. This is also in line with the score of PMI in the automatic evaluation in Table 2 and Figure 3.
Table 4 shows the two examples of the responses produced by all the models. From the results, our proposed model with the decoding switcher can generate more informative and contentful responses that are appropriate to the context than the other systems, while the baselines tend to talk about the current or previous topics passively and sometimes result in the redundant and unnatural conversation. These cases also show that the can be effectively used to flexibly control whether to converse about the exiting topics or drift the topic.
We can observe that there are also some cases where our models with do not perform well. These our models sometimes generate the non-fluent response (
in the left example) and the non-appropriate response to the context (
in the right example). By contrast, our model using the decoding switcher can generate more conversationally appropriate and fluent responses.
Sequence to sequence (seq2seq) approach has gained great attention in the field of dialogue generation. However, the problem of tending to generate generic and non-informative responses such as “I don’t know” and “Thanks” still remains [Vougiouklis et al., 2016; Ye et al., 2018; Ke et al., 2018].
The incorporation of facts (unstructured knowledge) has been shown to be an effective way to solve these problems. [Ghazvininejad et al., 2018] extended encoder-decoder model where the decoder is provided with an encoding of the context along with the facts encoding. To capture the multi-turn dialogue context and facts effectively, [Tanaka et al., 2019a] proposed a fact-based hierarchical seq2seq model. [Lian et al., 2019] proposed a knowledge selection mechanism where both prior and posterior distributions over knowledge were used. To explicitly output informative words, [Tam et al., 2019] and [Yavuz et al., 2019] proposed a model that enable the decoder to copy and attend tokens from any of the facts available as external knowledge in addition to the dialogue context. These works mainly focus on integrating facts based on a topic of input utterances into dialogue systems, which implicitly assume the topic to be kept on a dialogue and usually converse passively. However, this does not meet the common practice observed in our daily communication, and these systems have a difficulty to generate diverse responses that provide meaningful information proactively. To address this issue, we first incorporate the concept of the convergent and divergent thinking over both context and facts that can be observed in human thought method [Guilford, 1967] into fact-based dialogue system.
One of the most related works to ours may be [Liu et al., 2018], who proposed a knowledge diffusion model (NKD) that can not only match the knowledge base (structured knowledge) for the input utterance but diffuse them to similar entities with another type of knowledge base, but we have following differences. Firstly, the NKD is designed to depend on the movie specific-domain knowledge base, making them not applicable for open-domain dialogue generation. Moreover, such knowledge requires a lot of works to build up. Meanwhile, we applied a very large scale and open-domain facts (unstructured knowledge that can be relatively easily constructed) to show the effectiveness of our model in real-world conversation. Secondly, the NKD cannot explicitly control whether to exhibit similar entities or not, while our model can flexibly control the behavior of responses by the convergent and divergent decoding.
This paper proposed a fact-based dialogue model with a convergent and divergent decoding that introduces a concept of a convergent and divergent thinking over both context and facts. In contrast to existing approaches, our model can flexibly control either conversing about the current topic or drifting the topic, which brings more diverse and proactive conversation. Evaluation results show that our method sig-nificantly outperforms state-of-the-art baselines in producing more appropriate, informative, and diverse responses. This paper shed some lights in exploring the potential of incorporation of the convergent and divergent thinking into dialogue systems. As for future work, we plan to apply a reinforcement learning to further improve the performance of the model.
[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR 2015, 2015.
[Chung et al., 2014] Junyoung Chung, C¸ aglar G¨ulc¸ehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.
[Church and Hanks, 1990] Kenneth Ward Church and Patrick Hanks. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22–29, 1990.
[Fan et al., 2018] Angela Fan, Mike Lewis, and Yann N. Dauphin. Hierarchical neural story generation. In ACL 2018, pages 889–898, 2018.
[Galley et al., 2019] Michel Galley, Chris Brockett, Xiang Gao, Jianfeng Gao, and Bill Dolan. Grounded response generation task at dstc7. In AAAI 2019 DSTC7, 2019.
[Ghazvininejad et al., 2018] Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. A knowledge-grounded neural conversation model. In AAAI 2018, pages 5110– 5117, 2018.
[Guilford, 1967] Joy Paul Guilford. The nature of human intelligence. 1967.
[Ke et al., 2018] Pei Ke, Jian Guan, Minlie Huang, and Xiaoyan Zhu. Generating informative responses with controlled sentence function. In ACL 2018, pages 1499–1508, 2018.
[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.
[Lian et al., 2019] Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, and Hua Wu. Learning to select knowledge for response generation in dialog systems. In IJCAI 2019, pages 5081–5087, 2019.
[Liu et al., 2018] Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. Knowledge diffusion for neural dialogue generation. In ACL 2018, pages 1489–1498, 2018.
[Moghe et al., 2018] Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M. Khapra. Towards exploiting background knowledge for building conversation systems. In EMNLP 2018, pages 2322–2332, 2018.
[Nakamura et al., 2018] Ryo Nakamura, Katsuhito Sudoh, Koichiro Yoshino, and Satoshi Nakamura. Another
diversity-promoting objective function for neural dialogue generation. CoRR, abs/1811.08100, 2018.
[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP 2014, pages 1532–1543, 2014.
[Qin et al., 2019] Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao. Conversing by reading: Contentful neural conversation with on-demand machine reading. In ACL 2019, pages 5427–5436, 2019.
[See et al., 2017] Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer-generator networks. In ACL 2017, pages 1073– 1083, 2017.
[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS 2014, pages 3104–3112, 2014.
[Szegedy et al., 2016] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR 2016, pages 2818–2826, 2016.
[Tam et al., 2019] Yik-Cheung Tam, Jiachen Ding, Cheng Niu, and Jie Zhou. Cluster-based beam search for pointer-generator chatbot grounded by knowledge. In AAAI 2019 DSTC7, 2019.
[Tanaka et al., 2019a] Ryota Tanaka, Akihide Ozeki, Shugo Kato, and Akinobu Lee. An ensemble dialogue system for facts-based sentence generation. CoRR, abs/1902.01529, 2019.
[Tanaka et al., 2019b] Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh, and Satoshi Nakamura. Conversational response re-ranking based on event causality and role factored tensor event embedding. CoRR, abs/1906.09795, 2019.
[Tang et al., 2019] Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric P. Xing, and Zhiting Hu. Target-guided open-domain conversation. In ACL 2019, pages 5624–5634, 2019.
[Vinyals and Le, 2015] Oriol Vinyals and Quoc Le. A neual conversational model. In ICML, Deep Learning Workshop, 2015.
[Vougiouklis et al., 2016] Pavlos Vougiouklis, Jonathon S. Hare, and Elena Simperl. A neural network approach for knowledge-driven response generation. In COLING 2016, pages 3370–3380, 2016.
[Yavuz et al., 2019] Semih Yavuz, Abhinav Rastogi, GuanLin Chao, and Dilek Hakkani-T¨ur. Deepcopy: Grounded response generation with hierarchical pointer networks. CoRR, abs/1908.10731, 2019.
[Ye et al., 2018] Zhihao Ye, Ruichu Cai, Zhaohui Liao, Zhifeng Hao, and Jinfen Li. Generating natural answers on knowledge bases and text by sequence-to-sequence learning. In ICANN 2018, pages 447–455, 2018.
[Zhang et al., 2018] Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In ACL 2018, pages 2204–2213, 2018.