# Topic-switch adapted Japanese Dialogue System based on PLATO-2

Donghuo Zeng∗1 Jianming Wu∗1 Yanan Wang∗1 Kazunori Matsumoto  Gen Hattori
Kazushi Ikeda

###### Abstract

Large-scale open-domain dialogue systems such as PLATO-2 have achieved state-
of-the-art scores in both English and Chinese. However, little work explores
whether such dialogue systems also work well in the Japanese language. In this
work, we create a large-scale Japanese dialogue dataset, Dialogue-Graph, which
contains 1.656 million dialogue data in a tree structure from News, TV
subtitles, and Wikipedia corpus. Then, we train PLATO-2 using Dialogue-Graph
to build a large-scale Japanese dialogue system, PLATO-JDS. In addition, to
improve the PLATO-JDS in the topic switch issue, we introduce a topic-switch
algorithm composed of a topic discriminator to switch to a new topic when user
input differs from the previous topic. We evaluate the user experience by
using our model with respect to four metrics, namely, coherence,
informativeness, engagingness, and humanness. As a result, our proposed PLATO-
JDS achieves an average score of 1.500 for the human evaluation with human-bot
chat strategy, which is close to the maximum score of 2.000 and suggests the
high-quality dialogue generation capability of PLATO-2 in Japanese.
Furthermore, our proposed topic-switch algorithm achieves an average score of
1.767 and outperforms PLATO-JDS by 0.267, indicating its effectiveness in
improving the user experience of our system.

Figure 1: Overview of the topic-switch based PLATO-JDS. Given a user input
with previous dialogue, the topic-switch consists of a topic discriminator
that predicts if the user input is the same topic as previous dialogue, and
switch to a new topic when it differs form previous dialogue’s. The PLATO-JDS
is a pretrained Japanese dialogue generation model for generating a response
by giving specific inputs from the topic-switch.

##  1\. Introduction

Transformer-based methods are becoming fundamental techniques to developing
human-like chatbots [Devlin et al., 2019, Brown et al., 2020, Golovanov et
al., 2019, Wolf et al., 2019, Zhou et al., 2021, Shalaby et al., 2020,
Adiwardana et al., 2020, Radford et al., 2019]. Large-scale open-domain
dialogue systems such as PLATO-2 [Bao et al., 2021a] is designed to scale the
model up to billion parameters leading to high-quality open-domain chatbots,
have achieved state-of-the-art scores in both English and Chinese. However,
such large-scale dialogue systems are rarely explored in Japanese.

In this work, we create a large-scale Japanese dialogue dataset, Dialogue-
Graph by collecting 1.656 million dialogue data in a tree structure from News,
TV subtitles and Wikipedia corpus. Then, we train PLATO-2 using Dialogue-Graph
to build a large-scale Japanese dialogue system, PLATO-JDS. Moreover, we study
some cases generated by PLATO-JDS and the result suggests that PLATO-JDS is
difficult to suitably switch to a new topic during a dialogue. To solve this
issue to further improve the user experience of our system, we introduce a
topic-switch algorithm composed of a topic discriminator to switch previous
topic to a new topic when user input differs from previous topic. The topic
discriminator is a BERT-based binary classifier [Devlin et al., 2019] that
used to predict whether the user input belongs to previous topic or not. The
topic discriminator is a BERT-based binary classifier [Devlin et al., 2019]
used to predict whether the user input belongs to previous topic or not. As
shown in Figure 1, if the result is ”Yes”, the concatenation of the user input
and previous dialogue is used as the input to PLATO-JDS, otherwise, only the
user input is used.

We evaluate PLATO-JDS in terms of four metrics, namely, coherence,
informativeness, engagingness and humanness [Bao et al., 2021a], and the
results demonstrate that our proposed PLATO-JDS achieves scores of 1.600,
1.467, 1.467, and 1.467 on four metrics, respectively for human evaluation
with the human-bot chat strategy, and the average score is 1.500 and close to
the maximum score of 2.000, suggesting that PLATO-2 can be adopted to achieve
high-quality dialogue generation in Japanese. Furthermore, our proposed topic-
switch algorithm achieves the average score of 1.767 and outperforms PLATO-JDS
by 0.267, indicating its effectiveness in improving the user experience of our
system.

Our contributions include: 1) we manually collected 1.656 million dialogue
data to build a tree-structured large Japanese dataset Dialogue-Graph; 2) we
trained a Japanese PLATO-2, PLATO-JDS by using Dialogue-Graph. Both human and
automatic evaluations demonstrate that PLATO-JDS is effective to achieve high-
quality dialogue generation; 3) Moreover, we introduced a topic-switch
algorithm to further improve the user experience of PLATO-JDS.

##  2\. Related Work

Dialogue systems in the Japanese language domain have been developed by
employing basic deep learning techniques, such as the AI love counseling
system [Nakatsuji, 2019]. There are also some chatbot applications, such as
Clova [Bae et al., 2022] and Rinna [Wu et al., 2016] that employ state-of-the-
art NLP methods to improve the performance of dialogue generation. The
Japanese language model [Tianyu and Sawada, 2021] has trained two types of
models on a public corpus, including the GPT-2 [Radford et al., 2019] and
RoBERTa [Liu et al., 2019] models. Considering a large-scale dataset can be
used to train high-performance dialogue systems and should be taken into
account. Current works [Sugiyama et al., 2021, Rikters et al., 2019] try to
collect a large-scale dataset to improve Japanese dialogue system. Even so,
few large-scale Japanese conversation systems have been trained by using a
large Japanese language dataset, such as those in the millions in terms of
size.

Recently, large-scale transformer-based dialogue generation models have
significantly improved the performance of open-domain chatbots [Zhang et al.,
2020, Bao et al., 2020, Bao et al., 2021a, Bao et al., 2021b, Roller et al.,
2021, Lewis et al., 2020]. DialoGPT [Zhang et al., 2020] is a transformer-
based dialogue generation model trained by using 147M dialogue-like exchanges
from Reddit comments and shows good performance in generating context-based
responses by giving a single user input. PLATO [Bao et al., 2020] focuses on
performing one-to-many dialogue generation [Kim et al., 2020] (e.g., given a
single user input, the model can reply with more than one response) that is
able to improve DialoGPT. Furthermore, PLATO-2 [Bao et al., 2021a] scales the
PLATO model up to billions of parameters to achieve a new state-of-the-art
open-domain dialogue systems by introducing curriculum learning [Bengio et
al., 2009], which contains pretraining on a one-to-one generation subtask and
finetuning on a one-to-many generation subtask. However, PLATO-2 is evaluated
on English and Chinese data, and there is no work exploring its effectiveness
on Japanese data to facilitate the development of Japanese dialogue systems.
To construct a high-quality dialogue system, control generation methods
[Dathathri et al., 2019, Keskar et al., 2019, Madotto et al., 2020, Smith et
al., 2020, Du and Ji, 2021] have been widely applied in dialogue system. In
this study, PLATO-JDS is trained by curriculum learning, which controls
coherent and fluent responses through two steps, as shown in Figure 5.

To improve the user experience of dialogue systems, it is necessary to make
advanced models have very good topic adaptability. The work [Xu et al., 2021]
extracts topic segments from dialogue history in an unsupervised way to select
the most appropriate response for the topic-switch as needed. The work [Xia et
al., 2022] also adopts the topic segments method but also learns inter-segment
relationships to improve the topic-switch. The work [Sugiyama et al., 2021]
collect a mixed dataset with three characteristics: personality, empathy, and
knowledge, to fine-tune BlenderBot to conduct a chit-chat system, which can
switch topics frequently. Although these models track global topic flow
throughout multi-turn dialogue, they have difficulty eliminating the
interference of historical dialogues when users suddenly switch topic that is
not related to the previous dialogue. Instead of inputting all the historical
dialogues and user’s input as previous methods, we propose a topic-switch
algorithm with a topic discriminator to determine what the inputs of the
trained PLATO-JDS are.

##  3\. Dataset

In this section, we describe the details of Japanese dataset Dialogue-Graph in
§3.1 and the data preprocessing in §3.2.

Figure 2: Dialogue-Graph: Start from topic 0:food with an initial utterance
${n}^{0}_{0,0}$:”What is your favorite food?”, the Dialogue-Graph grows to a
one-to-many dialogue data containing eight dialogue turns in max. Here, one-
to-many indicates that one user input can accommodate many responses.

###  3.1 Data collection

In this work, we use the crowd-driven system [Ikeda and Hoashi, 2018] to
collect our dialogue data. This system can greatly improve the efficiency of
dialogue data collection because it uses an asynchronous approach to dialogue
creation, where workers can create an utterance at any time without waiting
for the previous workers to finish creating the utterances. Specifically, when
a worker starts using the system, the system will assign a dialogue that
requires the workers’ input, and the worker creates an utterance based on the
given topic as well as the previous utterances.

We recruit 60 participants to collect the dialogue dataset. We assumed that a
dialogue consists of alternating utterances between role A and role B, all
participants were divided into two groups of 30 individuals playing role A and
role B. One role can create utterances for multiple dialogues, and a dialogue
is created by multiple workers. In the end, we select only high-quality
conversations by evaluation method the same as [Ikeda and Hoashi, 2018] and
remove those dialogues that are not common sense, politically sensitive, and
ethical concerns. To increase the efficiency of data collection and allow
participants to create dialogues faster, we set the system to be accessible
from 12:00 am to 3:00 pm. The dialogue data collection took over three months,
costing 4 yen per utterance input and 32 yen per dialogue (about $0.30).

Inspired by the studies [Galitsky, 2018, Cheng et al., 2020], we also create
tree-structure dialogues, with 9 levels from the first head node to the leaf
node, i.e. the length of the conversation is 8 turns. We use 50 topics and
each topic has an initial utterance, the same as the work [Ikeda and Hoashi,
2018], for example, the topic is family and his initial utterance is ”Do you
live with someone?”. A worker created 8 utterances based on the starting
utterance, which became a dialogue. Then, we evaluated the collected dialogues
using the same evaluation method as in [Ikeda and Hoashi, 2018]. This method
leverages the quality of the dialogues from three perspectives: efficiency,
quality, and workers’ interest. After the data collection was completed, we
hired the same 60 participants to do the evaluation.

In Figure 2, we define Dialogue-Graph
$\mathcal{G}^{Dia}=(\mathcal{N},\mathcal{E})$,
$\mathcal{N}=\\{{n}^{t}_{i,j}|0\leq t\leq 49,0\leq i\leq 8,j\geq 0\\}$, where
$\mathcal{N}$ is the set of dialogue node ${n}^{t}_{i,j}$,
$\mathcal{E}=\\{{e}^{t}_{i,j\rightarrow m,n}|0\leq t\leq 49,0\leq i,m\leq
8,j,n\geq 0\\}$ is the set of dialogue turns, such that
${e}^{0}_{0,0\rightarrow 1,0}$ represents a single utterance turn from
${n}^{0}_{0,0}$ to ${n}^{0}_{1,0}$. Here, $t,i/m$ and $j/n$ indicate the
number of topic types, utterance turns and candidates of the current dialogue
node, respectively, the maximum number of $t$ and $i/m$ are $50$ and $8$, the
minimum number of $j/n$ is $3$.

Figure 3: Training data samples for training a topic discriminator and PLATO-
JDS.

###  3.2 Preprocessing

To prepare training data for training the topic discriminator and PLATO-JDS ,
we follow three policies to preprocess data from Dialogue-Graph. We split each
preprocessed dataset to training, validation and testing set with a common
rate of $90\%:5\%:5\%$.

  * •

pos. vs neg. As shown in Figure 3 (a), we extract arbitrary dialogue turns
from the same topic and annotate them as positive samples. On the other hand,
we annotate the counterpart as negative samples.

  * •

one-to-one mapping. We prepare training data for the stage 1 of PLATO-JDS. As
shown in Figure 3 (b), we employ the breadth-first search method (BFS) to
extract 1.68M data samples in total. BFS is designed to traverse dialogue
nodes in the graph to obtain utterance pairs. Starting from the root node, we
extract all utterance turns as training data (e.g.,
${n}^{0}_{0,0}\rightarrow{n}^{0}_{1,0}$,
${n}^{0}_{0,0}\rightarrow{n}^{0}_{1,3}$). The definition of an utterance turn
is provided in §3..

  * •

one-to-many mapping. We prepare training data for the stage 2 of PLATO-JDS. As
shown in Figure 3 (c), we use the depth-first search method (DFS) method to
generate 1.26M data samples. DFS is designed to traverse dialogue nodes in the
graph to obtain all eight-turn dialogue as training data (e.g.,
${n}^{0}_{0,0}\rightarrow{n}^{0}_{1,0}\rightarrow{n}^{0}_{2,1}\rightarrow{n}^{0}_{3,1}\rightarrow{n}^{0}_{4,1}\rightarrow{n}^{0}_{5,0}\rightarrow{n}^{0}_{6,0}\rightarrow{n}^{0}_{7,0}\rightarrow{n}^{0}_{8,2}$).

##  4\. Approach

To achieve PLATO-JDS as shown in Figure 1, we design a topic-switch algorithm
to utilize a topic discriminator to decide what the model inputs is by giving
a user input and previous dialogue, then we feed it to a trained PLATO-JDS to
generate a response. In this section, we first describe the detail of the
topic discriminator in §4.1 and PLATO-JDS in §4.2 in detail, then we describe
how the topic-switch algorithm works in §4.3.

Figure 4: Overview of topic discriminator. It consists of data preprocessing
described in § 3.2, tokenization method employed to obtain word sequences, the
backbone of BERT for training a binary classifier.

###  4.1 Topic discriminator

The topic discriminator is a binary classifier that is designed to determine
whether current user input belongs to a topic common to the previous dialogue.
As shown in Figure 4, we build a BERT-based topic discriminator and train it
by using the preprocessed Dialogue-Graph data described in §3.2. We annotate
data that belongs to a common topic as positive samples and the counterpart as
negative samples. We concatenate one turn containing an utterance pair as a
single sentence to input into the tokenization method followed by the backbone
of BERT. The objective of this training task is to discriminate whether two
utterances in one pair belongs to a common topic by optimizing the binary
cross entropy (BCE) loss as follows.

${L_{BCE}=-\frac{1}{N}\sum_{i=1}^{N}(Y_{i}\cdot\log\hat{Y_{i}}+(1-Y_{i})\cdot\log(1-\hat{Y_{i}}))}$
(1)

where $N$ is the number of samples, $Y_{i}$ denotes the ground-truth label (0
or 1), while $\hat{Y_{i}}$ is the predicted probability of the topic
discriminator.

Here, we employ the SentencePiece [Kudo and Richardson, 2018] tokenization
method to segment a raw input sentence directly into word sequences.
SentencePiece is a language-agnostic tokenization method based on a subword
idea to avoid unknown tokens while reducing the number of tokens. The Byte
Pair Encoding (BPE) [Gage, 1994, Sennrich et al., 2016] algorithm is a subword
division algorithm employed by SentencePiece to shortlist high-frequency words
as tokens and divide low-frequency words into two or more subwords as tokens.
We use SentencePiece to split the Japanese dataset Dialogue-Graph into 48k
tokens. Finally, we employ the same input representations as the original
PLATO-2 model including token embedding, role embedding, turn embedding and
position embedding, respectively. Here, token embedding is a pretrained
embedding for different subwords. Role embedding is designed to distinguish
the role of speakers during one dialogue. Turn embedding is assigned according
to relative order when there are multi-turn dialogues in the conversion.
Position embedding is obtained based on the token position in each dialogue.

###  4.2 PLATO-JDS

We follow the original PLATO-2 model [Bao et al., 2021a] to train it on
Japanese data to perform dialogue generation. PLATO-2 is a large-scale
transformer based dialogue generation framework that is trained via curriculum
learning [Bengio et al., 2009]. Curriculum learning is a two-stage training
strategy and is shown in Figure 3. In the first stage, a coarse-grained
generation model is pretrained to learn response generation under different
dialogue contexts by using the preprocessed one-to-one mapping data shown in
Figure 3 (b). In the second stage, a fine-grained generative model and an
evaluation model are trained by using the preprocessed one-to-many mapping
data shown in Figure 3 (c). We adopt the same input representations as the
topic discriminator model.

Stage 1, General response generation. One-to-one mapping is a conventional
approach and it is an efficient way to learn high-level properties of response
generation. Given a $i^{th}$ response $R(i)$ and its previous dialogue context
$C(i)$, the model is trained by minimizing the negative log-likelihood (NLL)
loss as follows,

$L_{NLL}^{General}=-\leavevmode\nobreak\ \textbf{E}\sum_{i=1}^{L}\log
p(R(i)|C(i))$ (2)

where $L$ is the length of the generated response $R(i)$.

Stage 2.1, Diverse Response Generation. To further train the model to capture
the relationship of one-to-many mapping, a discrete latent variable $z$ is
introduced. The model estimates the latent distribution of training samples
$p(z|C(i),R(i))$ and then generates sampled latent variable $p(R(i)|C(i),z)$.
The $NLL$ loss of diverse response generation and the bag-of-words (BOW) loss
are defined as follows.

$\begin{split}\begin{aligned} L_{NLL}^{Diverse}&=-\textbf{E}_{z\sim
p(z|C(i),R(i))}\sum_{i=1}^{L}\log p(R(i)|C(i),z)\\\\[5.0pt]
L_{BOW}^{Diverse}&=-\textbf{E}_{z\sim p(z|C(i),R(i))}\sum_{i=1}^{L}\log
p(R(i)|C(i),z)\\\\[5.0pt]
L^{Diverse}&=L_{NLL}^{Diverse}+L_{BOW}^{Diverse}\end{aligned}\end{split}$ (3)

where $L$ is the length of generated response $R(i)$. The final generation
model is optimized by the above integrated loss $L^{Diverse}$.

Stage 2.2, Coherent Response Selection. Once diverse responses generated from
one-to-many generation, the highest quality response is selected as the final
output. The selection model is trained by estimating the coherence between a
given utterance and its response by leveraging the capacity of distributed
representation of the masked language model (MLM). Two objective loss of
response coherence estimation (RCE) and MLM are as follows.

$\displaystyle L_{RCE}^{Selection}$ $\displaystyle=-\log
p(l_{r}=1|C(i),R(i))-\log p(l_{r}=0|C(i),\hat{R(i)})$ (4) $\displaystyle\break
L_{MLM}^{Selection}$ $\displaystyle=-\textbf{E}\sum_{i\in L}\log
p({x_{t}|{{}^{\textstyle x}\big{/}_{\textstyle L}}})$ $\displaystyle\break
L^{Selection}$
$\displaystyle=L_{RCE}^{Selection}+L_{MLM}^{Selection}\vspace{-2mm}$

where $l_{r}$ denote the generated response is consistent with dialogue text
($R(i)$) or not ($\hat{R(i)}$), $x$ denotes input tokens of context and
response. $\\{x_{i}\\}_{i\in L}$ denotes mask tokens, and ${{}^{\textstyle
x}\big{/}_{\textstyle L}}$ represents rest unmasked tokens.

Figure 5: Overview of the PLATO-JDS model and its data processing. The data
preprocessing described in § 3.2, tokenization method as same as the topic
discriminator (§ 4.1), and the backbone dialogue generation model of PLATO-2.

###  4.3 Topic-switch algorithm

A pretrained PLATO-JDS is used to generate a response by inputting the
concatenation of the previous dialogue and current user input. We design a
topic-switch algorithm to decide whether to include the previous dialogue used
for response generation. We define the topic-switch algorithm in Algorithm 1.
We utilize a topic discriminator to predict whether the topic of current user
input belongs to the previous dialogue topic by comparing with an experimental
threshold. If we get a score that is less than the threshold, the current user
input is probably unrelated to the previous dialogue and the input for the
PLATO-JDS module will be automatically switched to current user input.
Conversely, the input for the PLATO-JDS module will include previous dialogue.
Here, we set the threshold at 0.61 based on the trade-off between the
precision and recall, where the recall is 99.75% and the precision is 95.72%.

Algorithm 1 Topic-switch algorithm

Input: $\varepsilon$: a threshold for deciding if switch to a new topic, $i$:
the number of turns, $C(i)$: the previous dialogue of turn $i$, $\Phi(i)$: the
user utterance of turn $i$, $\Gamma(a,b)$: the topic discriminator to predict
whether $a$ and $b$ belong to a same topic.
Output: the context of next utterance $C(i)$

1:$\beta=\Gamma(C(i),\Phi(i))$

2:if $\beta\leq\varepsilon$ then

3: $C(i)\leftarrow\Phi(i)$ $\triangleright$ Switch to a new topic

4:else if $\beta>\varepsilon$ then

5: $C(i)\leftarrow C(i)+\Phi(i)$

6:end if

7:$i\leftarrow i+1$

##  5\. Experiment

In this section, we discuss the evaluation metrics in §5.1. Then, we explain
why we choose PLATO-2 as the backbone of dialogue generation module by
discussing the result in the English data reported in PLATO-2’s work in §5.2.
To evaluate our proposed topic-switch based PLATO-JDS, we compare our model
with PLATO-JDS and report the comparison results in §5.4. Furthermore, we
discuss the result of a case study in §5.5, and the training detail in §5.3.

###  5.1 Evaluation Metric

We evaluate our model from human and automatic perspectives. To achieve a fair
comparison, we utilize the same evaluation metrics used in the PLATO-2’s study
[Bao et al., 2021a]. In terms of human evaluation, we asked 15 people (11 male
and 4 female) to perform a three-level evaluation (0:bad,1:neutral,2:good)
with respect to the utterance-level metrics of ”coherence” and
”informativeness”, and the dialogue-level metrics of ”engagingness” and
”humanness” to evaluate the user experience. The definitions of these metrics
are as follows:

  * •

Coherence: leveraging the response whether it is relevant to the current topic
and consistent with the context.

  * •

Informativeness: measuring the response whether it is informative and is
appropriate.

  * •

Engagingness: judging the dialogue whether the evaluator likes to talk with
the speaker in a long dialogue.

  * •

Humanness: assessing the dialogue whether the speaker is a human being or the
response is natural.

In terms of automatic evaluation, to evaluate the model’s capacity on lexical
diversity, we employ the corpus-level metrics [Li et al., 2016a] distinct-1
and distinct-2 (distinct-1/2 in Table 1, 2), which are defined as the number
of distinct $unigrams$ and $bigrams$ as scaled by the total number of
generated tokens in response generation.

###  5.2 Baseline

We set PLATO-2 as our baseline since it is the state-of-the-art dialogue
generation model on the English and Chinese domains. PLATO-2 has achieved the
best performance in both human and automatic evaluations compared to the
Blender model [Roller et al., 2021] that mitigates undesirable toxic or bias
traits of large corpora by introducing blended skills. However it requires
extensive manual annotations. Considering the cost-performance ratio, we only
implement PLATO-2 as PLATO-JDS and compare it with our proposed topic-switch
based PLATO-JDS , both trained on Dialogue-Graph in this work. We perform both
evaluations through bot-bot and human-bot chat strategies and provide the
detail in §5.4. The description of PLATO-JDS is provided in §4.2.

###  5.3 Training Details

The PLATO-JDS training include two stages. In stage 1, we employed negative
log-likelihood (NLL) loss to capture the general characteristics of response
generation via one-to-one mapping learning. In stage 2.1, we took the sum of
NLL loss and bag-of-words (BOW) loss to learn the fine-grained generation.
Here, BOW loss is used to reflect the training process of discrete latent
variables. In stage 2.2, the objective loss was the sum of losses of response
coherence estimation and Masked Language Model (MLM) loss [Bao et al., 2021a].
More details of the above loss functions can be found in § 4.2. Moreover, we
set the maximum sequence length of context and response to 512, and the size
of position embedding to 256. We trained PLATO-JDS for 25 days by using NVIDIA
RTX A5000 GPU (24G x 2).

In addition, we show the loss curve of model training and validation process
in the stages 1, 2.1, and 2.2 in Figure 7 in Appendices. It shows that
overfitting does not occur during model training, and demonstrates that the
trained model is reliable.

###  5.4 Result

####  5.4.1 Bot-bot chat strategy

The bot-bot chat strategy is designed to evaluate dialogue generation by
performing dialogue simulation between two chatbots to reduce time consumption
and expense of full human evaluations, which require humans to spend time
talking to the chatbot and scoring generated response. This strategy is
commonly applied to achieve a fair evaluation of dialogue generation [Li et
al., 2016b, Bao et al., 2021a]. We randomly selected 200 questions from our
dataset as the start to produce ten-turns dialogues per question for models.
As the result shown in Table 1, our topic-switch based PLATO-JDS outperformed
PLATO-JDS on all human evaluation metrics, which suggests that the responses
generated by our topic-switch based PLATO-JDS are coherent in the current
dialogue context, and contains much more information than PLATO-JDS. In
addition, from the scores of ”Engagingness” and ”Humanness” metrics, we
believe that the topic-switch based PLATO-JDS is capable of improving the user
experience on PLATO-JDS, which also proves the effectiveness of generating
human-like responses. Furthermore, the result of distinct-1/2 demonstrated
that our topic-switch based PLATO-JDS contains a larger number of distinct
$unigrams$ and $bigrams$ than PLATO-JDS, which indicates its capacity in terms
of lexical diversity. In addition, from the average length of utterance and
the average number of topics, our topic-switch based PLATO-JDS achieved higher
point than PLATO-JDS, which further verifies the validity of the topic-switch
on switching to a new topic depending on the user input.

Metric Model w/o topic-switch w/ topic-switch Human evaluation Coherence 1.667
1.800 Informativeness 1.533 1.867 Engagingness 1.533 1.733 Humanness 1.533
1.800 Average score 1.567 1.800 Automatic evaluation Distinct-1/2 0.343/0.712
0.388/0.730 Length (avg.) 14.016 14.667 Topics (avg.) 4.6 6.8

Table 1: The bot-bot chat strategy: comparison results of PLATO-JDS (w/o
topic-switch) and topic-switch based PLATO-JDS (w/ topic-switch).

####  5.4.2 Human-bot chat strategy

In addition to the bot-bot chat strategy, we also collected human-bot dialogue
records to evaluate our model in a real-world setting. We employed 15
participants to chat with the bot, and each participant was requested to talk
to the bot for 50 turns. We generated a total of 750 dialogue data. The
results are summarized in Table 2. We obtained a very close comparison result
as well as the result described in §5.4.1. Although lower scores were obtained
for our topic-switch based PLATO-JDS compared with those in the bot-bot chat
strategy, we achieved higher scores on all metrics compare to PLATO-JDS.

To further confirm the effectiveness of the topic-switch, we also compared the
number of topics on average in the generated dialogue. As shown in Tables 1
and 2, topic-switch based PLATO-JDS manages two more topics and longer
dialogue than the PLATO-JDS, which demonstrates the effectiveness of the
topic-switch on making PLATO-JDS to generate a response based on a new topic.
As a result, adapting the topic-switch algorithm that can improve PLATO-JDS in
both human and automatic evaluations and suggest the capability in improving
the user experience.

Metric Model w/o topic-switch w/ topic-switch Human evaluation Coherence 1.600
1.733 Informativeness 1.467 1.800 Engagingness 1.467 1.800 Humanness 1.467
1.733 Average score 1.500 1.767 Automatic evaluation Distinct-1/2 0.339/0.709
0.376/0.715 Length 13.970 14.333 Topics (avg.) 4.2 6.5

Table 2: The human-bot chat strategy: comparison results of PLATO-JDS (w/o
topic-switch) and topic-switch based PLATO-JDS (w/ topic-switch).

###  5.5 Case study

We pick up two human-bot dialogue samples of PLATO-JDS with or without topic-
switch by using a same start utterance in the same topic ”beauty and health”,
which is to study how topic-switch affect the response generation. As shown in
Figure 6, when user input changes to a new topic that differs from context in
previous dialogue, we noted that our model with the topic-switch is able to
generate an appropriate response but PLATO-JDS can not. We believe that our
proposed topic-switch forced PLATO-JDS module to switch to a new topic to
generate its relevant response, thereby demonstrating the effectiveness of the
topic-switch. Furthermore, the case study also demonstrated that the coherence
among all the evaluation results described in §5.4.

Figure 6: Two dialogue samples generated with the PLATO-JDS with or without
topic-switch models. Starting from the same utterance ”I have a pain in my
stomach” that belongs to the topic of ”Health”, when the user input change to
the new topic of ”Alcohol” in the red dotted box to chat; our topic-switch
based PLATO-JDS can switch to the ”Alcohol” topic to generate an appropriate
response but PLATO-JDS can not.

##  6\. Conclusion

We built a PLATO-2 based Japanese dialogue system PLATO-JDS, and proposed a
topic-switch algorithm to make PLATO-JDS switch to a new topic if user input
differs from the previous topic. We built the largest Japanese dialogue
generation dataset Dialogue-Graph by collecting 1.656M dialogue data from
News, TV subtitles and Wikipedia corpus. our proposed PLATO-JDS achieved the
average score of 1.500 for the human evaluation with human-bot chat strategy,
and our proposed topic-switch algorithm further improved PLATO-JDS by 0.267
and achieved 1.767 human evaluation. The results suggest that PLATO-2 can be
adopted to achieve high-quality dialogue generation in Japanese and the topic-
switch is effective for improving the user experience of our system.

## References

  * [Adiwardana et al., 2020] Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020).  Towards a human-like open-domain chatbot.  arXiv preprint arXiv:2001.09977.
  * [Bae et al., 2022] Bae, S., Kwak, D., Kim, S., Ham, D., Kang, S., Lee, S.-W., and Park, W. (2022).  Building a role specified open-domain dialogue system leveraging large-scale language models.  In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2128–2150, Seattle, United States. Association for Computational Linguistics.
  * [Bao et al., 2020] Bao, S., He, H., Wang, F., Wu, H., and Wang, H. (2020).  PLATO: Pre-trained dialogue generation model with discrete latent variable.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 85–96, Online. Association for Computational Linguistics.
  * [Bao et al., 2021a] Bao, S., He, H., Wang, F., Wu, H., Wang, H., Wu, W., Guo, Z., Liu, Z., and Xu, X. (2021a).  PLATO-2: Towards building an open-domain chatbot via curriculum learning.  In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2513–2525, Online. Association for Computational Linguistics.
  * [Bao et al., 2021b] Bao, S., He, H., Wang, F., Wu, H., Wang, H., Wu, W., Wu, Z., Guo, Z., Lu, H., Huang, X., et al. (2021b).  Plato-xl: Exploring the large-scale pre-training of dialogue generation.  arXiv preprint arXiv:2109.09519.
  * [Bengio et al., 2009] Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009).  Curriculum learning.  In Proceedings of the 26th Annual International Conference on Machine Learning (ICML) (Volume 382: Long Papers), page 41–48, New York, NY, USA. Association for Computing Machinery.
  * [Brown et al., 2020] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020).  Language Models are Few-Shot Learners.  In Advances in Neural Information Processing Systems (Volume 33: Long Papers), pages 1877–1901, Online. Curran Associates, Inc.
  * [Cheng et al., 2020] Cheng, J., Agrawal, D., Alonso, H. M., Bhargava, S., Driesen, J., Flego, F., Ghosh, S., Kaplan, D., Kartsaklis, D., Li, L., et al. (2020).  Conversational semantic parsing for dialog state tracking.  arXiv preprint arXiv:2010.12770.
  * [Dathathri et al., 2019] Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., and Liu, R. (2019).  Plug and play language models: A simple approach to controlled text generation.  arXiv preprint arXiv:1912.02164.
  * [Devlin et al., 2019] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019).  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * [Du and Ji, 2021] Du, W. and Ji, Y. (2021).  Sidecontrol: Controlled open-domain dialogue generation via additive side networks.  arXiv preprint arXiv:2109.01958.
  * [Gage, 1994] Gage, P. (1994).  A new algorithm for data compression.  C Users Journal.
  * [Galitsky, 2018] Galitsky, B. (2018).  Building dialogue structure from discourse tree of a question.  In Workshops at the thirty-second AAAI conference on artificial intelligence.
  * [Golovanov et al., 2019] Golovanov, S., Kurbanov, R., Nikolenko, S. I., Truskovskyi, K., Tselousov, A., and Wolf, T. (2019).  Large-scale transfer learning for natural language generation.  In Korhonen, A., Traum, D. R., and Màrquez, L., editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL (Volume 1: Long Papers), pages 6053–6058, Florence, Italy. Association for Computational Linguistics.
  * [Ikeda and Hoashi, 2018] Ikeda, K. and Hoashi, K. (2018).  Utilizing crowdsourced asynchronous chat for efficient collection of dialogue dataset.  In Proceedings of the Sixth AAAI Conference on Human Computation and Crowdsourcing, HCOMP (Volume 6: Long Papers), pages 60–69, Zúrich, Switzerland. AAAI Press.
  * [Keskar et al., 2019] Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., and Socher, R. (2019).  Ctrl: A conditional transformer language model for controllable generation.  arXiv preprint arXiv:1909.05858.
  * [Kim et al., 2020] Kim, B., Ahn, J., and Kim, G. (2020).  Sequential latent knowledge selection for knowledge-grounded dialogue.  arXiv preprint arXiv:2002.07510.
  * [Kudo and Richardson, 2018] Kudo, T. and Richardson, J. (2018).  SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Association for Computational Linguistics.
  * [Lewis et al., 2020] Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2020).  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
  * [Li et al., 2016a] Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B. (2016a).  A diversity-promoting objective function for neural conversation models.  In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110–119, San Diego, California. Association for Computational Linguistics.
  * [Li et al., 2016b] Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., and Gao, J. (2016b).  Deep reinforcement learning for dialogue generation.  In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Austin, Texas. Association for Computational Linguistics.
  * [Liu et al., 2019] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019).  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692.
  * [Madotto et al., 2020] Madotto, A., Ishii, E., Lin, Z., Dathathri, S., and Fung, P. (2020).  Plug-and-play conversational models.  arXiv preprint arXiv:2010.04344.
  * [Nakatsuji, 2019] Nakatsuji, M. (2019).  Can ai generate love advice?: Toward neural answer generation for non-factoid questions.  arXiv preprint arXiv:1912.10163.
  * [Radford et al., 2019] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. (2019).  Language models are unsupervised multitask learners.  OpenAI blog (Volume 1 (8): Long Papers), page 9.
  * [Rikters et al., 2019] Rikters, M., Ri, R., Li, T., and Nakazawa, T. (2019).  Designing the business conversation corpus.  In Proceedings of the 6th Workshop on Asian Translation, pages 54–61, Hong Kong, China. Association for Computational Linguistics.
  * [Roller et al., 2021] Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., and Weston, J. (2021).  Recipes for building an open-domain chatbot.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300–325, Online. Association for Computational Linguistics.
  * [Sennrich et al., 2016] Sennrich, R., Haddow, B., and Birch, A. (2016).  Neural machine translation of rare words with subword units.  In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.
  * [Shalaby et al., 2020] Shalaby, W., Arantes, A. S., GonzalezDiaz, T., and Gupta, C. (2020).  Building chatbots from large scale domain-specific knowledge bases: challenges and opportunities.  In IEEE International Conference on Prognostics and Health Management, ICPHM 2020, pages 1–8, Detroit, MI, USA. IEEE.
  * [Smith et al., 2020] Smith, E. M., Gonzalez-Rico, D., Dinan, E., and Boureau, Y.-L. (2020).  Controlling style in generated dialogue.  arXiv preprint arXiv:2009.10855.
  * [Sugiyama et al., 2021] Sugiyama, H., Mizukami, M., Arimoto, T., Narimatsu, H., Chiba, Y., Nakajima, H., and Meguro, T. (2021).  Empirical analysis of training strategies of transformer-based japanese chit-chat systems.  arXiv preprint arXiv:2109.05217.
  * [Tianyu and Sawada, 2021] Tianyu, Z. and Sawada, K. (2021).  日本語自然言語処理における事前学習モデルの公開.  人工知能学会研究会資料 言語・音声理解と対話処理研究会, 93:169–170.
  * [Wolf et al., 2019] Wolf, T., Sanh, V., Chaumond, J., and Delangue, C. (2019).  Transfertransfo: A transfer learning approach for neural network based conversational agents.  arXiv preprint arXiv:1901.08149.
  * [Wu et al., 2016] Wu, X., Ito, K., Iida, K., Tsuboi, K., and Klyen, M. (2016).  りんな: 女子高生人工知能.  言語処理学会第 22 回年次大会発表論文集, pages 306–309.
  * [Xia et al., 2022] Xia, J., Liu, C., Chen, J., Li, Y., Yang, F., Cai, X., Wan, G., and Wang, H. (2022).  Dialogue topic segmentation via parallel extraction network with neighbor smoothing.  In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval., pages 2126–2131, Madrid, Spain. ACM.
  * [Xu et al., 2021] Xu, Y., Zhao, H., and Zhang, Z. (2021).  Topic-aware multi-turn dialogue modeling.  In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, pages 14176–14184, Online. AAAI Press.
  * [Zhang et al., 2020] Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., and Dolan, B. (2020).  DIALOGPT : Large-scale generative pre-training for conversational response generation.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270–278, Online. Association for Computational Linguistics.
  * [Zhou et al., 2021] Zhou, H., Ke, P., Zhang, Z., Gu, Y., Zheng, Y., Zheng, C., Wang, Y., Wu, C. H., Sun, H., Yang, X., et al. (2021).  Eva: An open-domain chinese dialogue system with large-scale generative pre-training.  arXiv preprint arXiv:2108.01547.

## Appendix

Table 3 list all the 50 topics in our data collection. Table 4 list the start
utterance of each topic. Fig. 8 draw the number of dialogue with the change of
topics. Table 5 displays the statistic of our Dialogue-Graph dataset. Table 7
shows the comparison result of bot-bot chat of PLATO-2 on English data. Table
6 displays the details of PLATO-JDS training setting. Fig. 7 draws the loss of
PLATO-JDS during the training.

話題 Topic 話題 Topic 美容と健康(食生活) Beauty&health (eat habits) 映画 movie 自己紹介 self-
introduction 家族 family アウトドア outdoors 恋愛 love 休日の過ごし方 way to spend holidays 趣味
hobby 友人関係 friendship スポーツ do sports スポーツ watch sports 勉強 study 習い事 lesson
学生生活 student life アルバイト part-time job 料理 cooking スイーツ sweets お酒 alcohol テレビ番組
TV program 通勤 commute インテリア indoor setting 家事 housework 車、ドライブ car, drive 仕事
work 好きな動物 Favorite animal 季節 season 家電 Home appliances 携帯電話 cellphone 旅行(国内)
Travel (domestic) 出身地 birthplace 美容と健康(運動) Beauty&health (exercise) 語学
language マッサージ message カフェ Cafe 読書・マンガ Reading / manga 音楽 music 欲しいもの what I
want 結婚 marriage 旅行(海外) travel (overseas) 食べ物 food どこでもドア anywhere door ランチ
lunch 好きな芸能人 favorite entertainer 好きな街 favorite town ハマっているもの addicted things
将来の夢 future dream ファッション fashion 宝くじ lottery 睡眠(生活習慣？） Sleep (lifestyle?) SNS
SNS

Table 3: 50 topics list in English and Japanese

Topic start utterance family Are you living alone? marriage Are you married??
friendship Do you meet friends in school? hobby What do you do for fun?? do
sports Do you do some sports? study Do you have any studying now? cooking Do
you like cooking? sweets Do you like sweets? ……

Table 4: Example of topics and the corresponding start utterances.

# Number of # Utterance with max number of tokens 95 Utterance with min number
of tokens 1 Utterance with avg. number of tokens 12.28 Dialogue with max
number of tokens 258 Dialogue with min number of tokens 23 Dialogue with avg.
number of tokens 109.3 Number of Turns 8 Total dialogues 1.656M Total
utterances 9,837,742 Max number of all dialogues for a topic 22,397 Min number
of all dialogues for a topic 21,903 Avg. number of all dialogues across topics
22,092.04 Variance of all dialogues across topics 14106.32

Table 5: Statistics of our Dialogue-Graph dataset.

Model(Stage) | Hyper-parameters | PPL | Loss | NLL/RCE loss | Bow loss | MLM loss
---|---|---|---|---|---|---
Batch size | Learning rate | Steps
Stage 1 | 2,700 | 4e-4 | 16M | 1.92 | 0.52 | 0.52 | - | -
Stage 2.1 | 5,400 | 4e-5 | 1.10M | 1.20 | 4.40 | 0.23 | 4.17 | -
Stage 2.2 | 2,900 | 4e-5 | 2.54M | - | 0.58 | 0.39 | - | 0.19
|  |  |  |  |  |  |  |

Table 6: Details of PLATO-JDS training setting. The $NLL$ loss is computed in
stage 1, the $NLL$ and $Bow$ losses are calculated in stage 2.1, and the $RCE$
and $MLM$ losses are obtained in stage 2.2

(a) One to one mapping model training in the stage 1 of the PLATO-JDS .

(b) One to many mapping model training in the stage 2.1 of PLATO-JDS

(c) Response evaluation model training in the stage 2.2 of PLATO-JDS

Figure 7: Three loss graphs of PLATO-JDS model training in the stage-1,
stage-2.1, and stage-2.2.

Metric Model PLATO DialoGPT Blender PLATO-2 Human Evaluation Coherence 0.568
0.720 1.856 1.920 Informativeness 0.564 0.712 1.816 1.892 Engagingness 0.340
0.340 1.820 1.840 Humanness 0.280 0.100 1.540 1.740 Automatic Evaluation
Distinct-1/2 0.042/0.255 0.150/0.508 0.117/0.385 0.169/0.613

Table 7: Comparison results of bot-bot chat strategy of PLATO-2 on English
data. A higher value indicates higher quality. All the results come from
original PLATO-2 paper (Bao et al., 2021a) Figure 8: The number of dialogues
with the change of topics.