#  : Complex Dialogue Utterance Splitting and Reformulation
for Multiple Intent Detection

Haoran Meng1111Equal contribution. Xin Zheng2 4111Equal contribution. Tianyu
Liu3111Equal contribution. 222Corresponding authors. Zizhen Wang3 He Feng3
Binghuai Lin3 Xuemin Zhao3 Yunbo Cao3 Zhifang Sui1222Corresponding authors.
1 MOE Key Laboratory of Computational Linguistics, Peking University, China
2 Institute of Software, Chinese Academy of Sciences, China
3 Tencent Cloud Xiaowei 4University of Chinese Academy of Sciences, China
haoran@stu.pku.edu.cn;zhengxin2020@iscas.ac.cn;{rogertyliu,zizhenwang,
mobisysfeng,binghuailin,xueminzhao,yunbocao}<EMAIL_ADDRESS>
###### Abstract

While interacting with chatbots, users may elicit multiple intents in a single
dialogue utterance. Instead of training a dedicated multi-intent detection
model, we propose , a dialogue utterance splitting and reformulation task that
first splits multi-intent user query into several single-intent sub-queries
and then recovers all the coreferred and omitted information in the sub-
queries. DialogUSR can serve as a plug-in and domain-agnostic module that
empowers the multi-intent detection for the deployed chatbots with minimal
efforts. We collect a high-quality naturally occurring dataset that covers 23
domains with a multi-step crowd-souring procedure. To benchmark the proposed
dataset, we propose multiple action-based generative models that involve end-
to-end and two-stage training, and conduct in-depth analyses on the pros and
cons of the proposed baselines.

##  1 Introduction

Thanks to the technological advances of natural language processing (NLP) in
the last decade, modern personal virtual assistants like Apple Siri, Amazon
Alexa have managed to interact with end users in a more natural and human-like
way. Taking chatbots as human listeners, users may elicit multiple intents
within a single query. For example, in Figure 1, a single user query triggers
the inquiries on both high-speed train ticket price and the weather of
destination. To handle multi-intent user queries, a straightforward solution
is to train a dedicated natural language understanding (NLU) system for multi-
intent detection. Rychalska et al. (2018) first adopted hierarchical
structures to identify multiple user intents. Gangadharaiah and Narayanaswamy
(2019) explored the joint multi-intent and slot-filling task with a recurrent
neural network. Qin et al. (2020) further proposed an adaptive graph attention
network to model the joint intent-slot interaction. To integrate the multi-
intent detection model into a product dialogue system, the developers would
make extra efforts in continuous deployment, i.e. technical support for both
single-intent and multi-intent detection models, and system modifications,
i.e. changes in the APIs and implementations of NLU and other related modules.

Figure 1: The task illustration for . It serves as a plug-in module that
empowers multi-intent detection capability for deployed single-intent NLU
systems.

To provide an alternative way towards understanding multi-intent user queries,
we propose complex dialogue utterance splitting and reformulation (DialogUSR)
task with corresponding benchmark dataset that firstly splits the multi-intent
query into several single-intent sub-queries and then recover the coreferred
and omitted information in the sub-queries, as illustrated in Fig 1. With the
proposed task and dataset, the practitioners can train a multi-intent query
rewriting model that serves as a plug-in module for the existing chatbot
system with minimal efforts. The trained transformation models are also
domain-agnostic in the sense that the learned query splitting and rewriting
skills in DialogUSR are generic for multi-intent complex user queries from
diverse domains.

We employ a multi-step crowdsourcing procedure to annotate the dataset for
DialogUSR which covers 23 domains with 11.6k instances. The naturally
occurring coreferences and omissions account for 62.5% of the total human-
written sub-queries, which conforms to the genuine user preferences.
Specifically we first collect initial queries from 2 Chinese task-oriented NLU
datasets that cover real-world user-agent interactions, then ask the
annotators to write the subsequent queries as they were sending multiple
intents to the chatbots, finally we aggregate the human written sub-queries
and provide completed sub-queries if coreferences and omissions are involved.
We also employ multiple screening and post-checking protocols in the entire
data creation process, in order to ensure the high quality of the proposed
dataset.

For baseline models, we carefully analyze the transformation from the input
multi-intent queries to the corresponding single-intent sub-queries and
summarize multiple rewriting actions, including deletion, splitting,
completion and causal completion which are the local edits in the generation.
Based on the summarized actions, we proposed three types of generative
baselines: end-to-end, two-stage and causal two-stage models which are
empowered by strong pretrained models, and conduct a series of empirical
studies including the exploration on the best action combination, the model
performance on different training data scale and existing multi-intent NLU
datasets.

We summarize our contributions as follows111Code and data are provided in
https://github.com/MrZhengXin/multi_intent_2022.:

1) The biggest challenges of multi-intent detection (MID) in the deployment is
the heavy code refactoring on a running dialogue system which already does a
good job in single-intent detection. It motivates us to design DialogUSR,
which serves as a plug-in module and eases the difficulties of incremental
development.

2) Prior work on MID has higher cost of data annotation and struggles in the
open-domain or domain transfer scenarios. Only NLU experts can adequately
annotate the intent/slot info for a MID user query, and the outputs of MID NLU
models are naturally limited by the pre-defined intent/slot ontology. In
contrast, DialogUSR datasets can be easily annotated by non-experts, and the
derived models are domain-agnostic in the sense that the learned query
splitting, coreference/omission recovery skills are generic for distinct
domains

3) Presumably MID is more difficult than single intent detection (SID) given
the same intent/slot ontology. From the perspective of task (re)formulation,
DialogUSR is the first to convert a MID task to multiple SID tasks (the
philosophy of ’divide and conquer’) with a relatively low error propagation
rate, providing an alternative and effective way to handle the MID task.

Figure 2: The overview for the data collection procedure of DialogUSR. Firstly
we sample initial queries from task-oriented NLU datasets (Sec. 2.1), then we
hire crowdsource workers to write follow-up queries (Sec. 2.2). To aggregate
the annotated queries, we propose text filler templates (marked in red, Sec.
2.3) and post-processing procedure. Finally we ask annotators to recover the
missing information in the incomplete utterances (marked in blue, Sec. 2.4).

##  2 Dataset Creation

We collect a high quality dataset via a 4-step crowdsourcing procedure as
illustrated in Fig 2.

###  2.1 Initial Query Collection

In order to determine the topic of the multi-intent user query, we sample an
initial query from two Chinese user query understanding datasets for task-
oriented conversational agents, namely SMP-
ECDT222http://ir.hit.edu.cn/SMP2017-ECDT(Zhang et al., 2017) and
RiSAWOZ333https://github.com/terryqj0107/RiSAWOZ (Quan et al., 2020). Then we
ask human annotators to simplify the initial queries that have excessive
length (longer than 15 characters), or are too verbose or repetitive in terms
of semantics444The sentence simplification phase makes the annotated multi-
intent queries sound more natural, as users are not likely to elicit a lengthy
query. Given the fact that we would add 2 or 3 following sub-queries to the
initial queries, they should be simplified to keep a proper query length (Fig
2).. RiSAWOZ is a a large-scale multi-domain Chinese Wizard-of-Oz NLU dataset
with rich semantic annotations, which covers 12 domains in _tourist
attraction_ , _railway_ , _hotel_ , _restaurant_ , etc. SMP-ECDT is released
as the benchmark for the “domain and intent identification for user query”
task in the evaluation track of Chinese Social Media Processing conference
(SMP) 2017 and 2019. It covers divergent practical user queries from 30
domains which are collected from the production chatbots of iFLYTEK. We use
the two source datasets as our query resources as they comprise a variety of
common and naturally occurring user queries in daily life for task-oriented
chatbot and cover diverse domains and topics.

###  2.2 Follow-up Query Creation

After specifying an initial query, we ask human annotators to put themselves
in the same position of a real end user and imagine they are eliciting
multiple intents in a single complex user query while interacting with
conversational agents. The annotators are instructed to write up to 3
subsequent queries on what they need or what they would like to know about
according to the designated initial query. Although most subsequent queries
stick to the topic of the initial query, we allow the human annotators to
switch to a different topic which is unrelated to the initial query555In fact,
we neither encourage nor discourage topic switching in the annotation
instruction.. For example in Figure 1, the second sub-query asks about the
weather in Nanjing, where the initial query is an inquiry on the railway
information. We observe that 37.3% annotated multi-intent queries involve
topic switching by manually checking 300 subsampled instances in the training
set, which conforms to the user behaviour in the real-world multi-intent
queries.

Figure 3: The domain statistics of DialogUSR, which covers diverse domains in
the conversational agents.

###  2.3 Query Aggregation

In the pilot study, we tried to ask human annotators to manually aggregate the
sub-queries but found that the derived queries are somewhat lack of variations
in the conjunctions between the sub-queries, as the annotators tend to always
pick up the most common Chinese conjunctions like ’and’, ’or’, ’then’. We even
observed sloppy annotators trying to hack the annotation job by not using any
conjunctions at all for each query (most queries are fluent even without
conjunctions). In a nutshell, we find it challenging to screen the annotators
and ensure the diversity and naturalness of the derived query in the human-
only annotation. We then resort to human-in-the-loop annotation, sampling from
a rich conjunction set to connect sub-queries and post-checking the sentence
fluency of aggregated queries by GPT-2. After each round of annotation (we
have 6 rounds of annotations), we randomly pick up 100 samples and check their
quality, finding that over 95% of samples are of high quality. Actually most
sentences in the Fig 9 (appendix) are fluent and natural (especially in
Chinese) without cherry-picking.

More concretely we propose a set of pre-defined templates that correspond to
different text infilling strategies between consecutive queries. Specifically,
with a 50% chance we concatenate two consecutive queries without using any
text filler. For the other 50% chance, we sample a piece of text from a set of
pre-defined text fillers with different sampling weights, such as “UTF8gbsn首先”
(first of all), “UTF8gbsn以及” (and), “UTF8gbsn我还想知道” (I also would like to
know), “UTF8gbsn接下来” (then), “UTF8gbsn最后” (finally), and then use the sampled
text filler as a conjunction while concatenating consecutive queries. Although
being locally coherent, the derived multi-intent query may still exhibit some
global incoherence and syntactic issues, especially for longer text. We thus
post-process the derived query with a ranking procedure as an additional
screening step. For each annotated query set, we generate 10 candidate multi-
intent queries with different sampled templates and rank them according to
language model perplexity using a GPT-2 (117M) model. We only keep the the
candidate with lowest perplexity to ensure the fluency and syntactic
correctness. To avoid trivial hacks in the complex query splitting, we remove
all the punctuations in the aggregated query, which conforms to the default
settings of most production chatbots, i.e. no punctuations in the spoken
language understanding phase after going through the automatic speech
recognition module.

Figure 4: The ratio of the incomplete utterances in the gold outputs of
DialogUSR. The blue bar signifies incomplete utterances which requires
rewriting while the orange bar represents complete utterances.

###  2.4 Query Completion

After assembling the multi-intent user queries, we observe that incomplete
utterances, such as co-references and omissions, are frequently occurring
which account for 62.5% of total human-written subsequent queries. Note that,
in the annotation instruction, we do not explicitly ask the crowdsource worker
to use coreferences or omissions while writing the subsequent queries in the
_follow-up query creation_ phase. The naturally occurring incomplete
utterances reflect genuine user preferences while sending out multiple
intents. To gather sufficient information while splitting multi-intent queries
into independent single-intent queries, we ask another group of
annotators666The _query completion_ phase starts when _follow-up query
creation_ phase has finished. We hire another group of annotators that did not
participate in the follow-up query writing task to screen the quality of
rewritten queries while doing _query completion_. to write the completed
utterances by recovering omitted and co-referred information for the
incomplete queries.

###  2.5 Data Annotation Settings

To perform human annotation, we hired crowdsource workers from an internal
data annotating group. The workers were limited to those who have abundant
hand-on experiences in annotating conversational data with good records
(recognized as experts in the internal assessment, rejection rate $\leq$ 1%).
Additionally, all the workers were screened via a 10-case qualification test
that covers various annotation tasks in Sec 2.1 to Sec 2.4 (correctly
annotating 8 out of 10 cases). They were paid 0.6$ per datapoint, which is
more than prevailing local minimum wage. We split the entire annotation
procedure into multiple rounds and hire another group of human judges to post-
check the quality of annotated dataset and filter unqualified instances after
each round. In this way, we create a high-quality crowdsourcing dataset.

##  3 Dataset Analysis

#### Dataset Statistics

In total, after accumulating annotations for several rounds, we obtain 11,669
instances. We conduct 6 rounds of annotation, increasing the annotation scale
with each round (ranging from $\sim$100 instances/round to $\sim$4000
instances/round). On average, an aggregated multi-intent complex query from
the proposed DialogUSR dataset comprises 36.7 Chinese characters by assembling
3.6 single-intent queries (including initial and follow-up queries). After
recovering missing information in the query completion phase (Sec 2.4), the
average lengths of completed initial query, first follow-up query, second
follow-up query and third follow-up query are 11.9, 12.3, 12.4, 10.8
respectively. We split the dataset into train, validation and test sets with
sizes of 10,169 , 500, 1,000 respectively.

#### Domain Statistics

The domain statistics of DialogUSR is depicted in Fig 3. Thanks to the diverse
domains of our source datasets, DialogUSR covers 23 domains that chatbot users
frequently query on in their daily life. Additionally, as mentioned in Sec
2.2, the annotators proactively switch topics or domains in the data creation
procedure. We find that, on average, a complex query in DialogUSR involves 1.4
domains, showing the potential usage of recognizing user intents across
different domains. The models training on the DialogUSR dataset can deal with
divergent situations in the practical usage while accommodating the utility of
personal virtual assistant.

#### Incomplete Utterance Analysis

Existing multi-intent detection datasets, such as MixATIS and MixSNIPS Qin et
al. (2020), were created using simple heuristic rules, e.g. adding a
particular conjunction “and” while concatenating two single-intent queries.
The simple heuristic datasets largely undermine the multi-intent detection in
the real-world conversational agents, where users naturally interact with
chatbots with coreferences and omissions. As highlighted in Sec 2.4, nearly
two thirds of human-written subsequent queries are incomplete. We further show
the incomplete ratio of follow-up queries for different domains in Fig 4. In
the incomplete utterances, according to our statistics, only 2.4% of them
belong to the coreferred phenomenon, showing that users prefer not using
pronouns to refer to previously mentioned entities.

Figure 5: The overview for the actions taken to transform a multi-intent complex user query (Q1) to the executable single-intent queries (Q4). We use red, blue and green to highlight the text fillers, omitted information and query delimiters, respectively. Model | BLEU | METEOR | ROUGE | SACC | Exact Match (EM)
---|---|---|---|---|---
Comp. | Rew. | Avg.
End-to-end (mT5-base) | 49.37 | 57.54 | 60.14 | 84.40 | 88.26 | 56.89 | 56.17
End-to-end (mT5-large) | 57.07 | 59.83 | 66.47 | 93.50 | 93.53 | 63.04 | 63.90
End-to-end (mT5-xl) | 64.37 | 62.52 | 72.71 | 97.90 | 97.52 | 70.07 | 71.45
End-to-end (mBART-large) | 62.32 | 62.24 | 71.09 | 98.60 | 97.03 | 68.11 | 69.48
Two-stage (once,mT5-base) | 49.38 | 58.03 | 60.21 | 83.80 | 89.64 | 56.95 | 56.17
Two-stage (casual,mT5-base) | 55.29 | 60.26 | 65.11 | 85.30 | 89.80 | 61.72 | 60.79
Two-stage (once,mT5-large) | 57.86 | 60.39 | 67.03 | 94.10 | 92.54 | 63.79 | 64.53
Two-stage (casual,mT5-large) | 62.09 | 62.04 | 70.68 | 94.20 | 93.53 | 67.72 | 68.24
Two-stage (once,mT5-xl) | 64.37 | 62.82 | 72.50 | 98.70 | 97.04 | 70.33 | 71.78
Two-stage (casual,mT5-xl) | 63.73 | 62.86 | 72.46 | 98.80 | 95.07 | 70.28 | 71.62
Table 1: The benchmark for the baseline models (Fig 5). “Comp.” and “Rew.”
correspond to the complete and rewritten (incomplete due to coreferences or
omissions) queries. We report the median scores over 5 runs.

##  4 Baseline Models

###  4.1 Task Overview

As depicted in Figure 5, the input (Q1) and the output (Q4) of DialogUSR have
a large text overlap. The transformation from Q1 to Q4 can be viewed as
several local edits that retain the main body of the input query. We thus
define several implicit actions that guide the transformation: 1) The Split
action (Q1$\rightarrow$Q2) divides the complex multi-intent query into
specific single-intent query with a special token. In our implementation we
use the semicolon (;) and set up a heuristic rule that puts the semicolons
before the text fillers if the latter appear. 2) The Delete action
(Q2$\rightarrow$Q3) removes the text fillers and keep the salient queries for
the subsequent actions. 3) The Complete action (Q3$\rightarrow$Q4) recovers
the coreferred and omitted information in the recognized single-intent queries
so that they can be effectively parsed by the existing (single-query) NLU
module. 4) The Causal Complete strategy consists of the Split action
(Q1$\rightarrow$Q2) and several Complete actions that echo with the token-by-
token auto-regressive text generation. The difference is that Causal Complete
strategy in DialogUSR recovers the missing information in the incomplete user
utterances with a query-by-query fashion (Q5$\rightarrow$Q6$\rightarrow$Q7).

###  4.2 End-to-end Generative Models

The most straightforward way is to train a sequence-to-sequence model to learn
the transformation from the multi-intent query (Q1) to the decomposed single-
intent ones (Q4) in the end-to-end fashion. The models are trained to
implicitly split the raw query (without punctuation) (Q1$\rightarrow$Q2),
delete the conjunctions (Q2$\rightarrow$Q3) and recover the missing
information (Q3$\rightarrow$Q4) in one single turn of generation. Specifically
given the multi-intention complex query, the model is trained to output the
sequence of multiple completed independent queries “$Q_{1};Q_{2};...;Q_{n};$
</s>”, where “;”, $n$, “</s>” represent the query separation token, the number
of queries and the end-of-sentence token, respectively.

###  4.3 Two-stage Generative Models

In stead of performing all three actions in one single turn, we try to guide
the transformation by a step-by-step generation (Moryossef et al., 2019; Liu
et al., 2019, 2021). Notably, the Split, Delete and Complete actions in Fig 5
can be arbitrarily permuted throughout the generation process, e.g. firstly
removing text filler then split the complete the complex query
(Delete$\rightarrow$Split$\rightarrow$Complete). However we observe the
performance drop if we explicitly employ a 3-step generation due to the error
propagation.

#### Two-stage model (once)

we resort to a two-stage procedure that firstly splits the complex query
(Q1$\rightarrow$Q2) and then recovers the incomplete utterances
(Q2$\rightarrow$Q4). As the Split action is relative easy, i.e. achieving
nearly 100% accuracy on the query separation, the error accumulations are
largely mitigated.

#### Two-stage model (casual)

Due to the fact that the former sub-queries would not be affected by the
subsequent queries, we propose a “causal”-style query-by-query generation
(Q5$\rightarrow$Q6$\rightarrow$Q7) in which the current sub-query to be
reformulated only conditions on the prior sub-query instead of seeing the
bidirectional context. Specifically, the Causal complete action takes place
after the Split action. In the $t$-th episode of Causal complete action, we
feed the model with incomplete queries “$q_{1};...;q_{t}$”, and then train the
model to generate the completed query $Q_{t}$. In this way, we greatly reduce
the search space without the sacrifice on model performance. From an
engineering standpoint, the proposed Causal complete action is a natural fit
for the “streaming” conversational agent, i.e. simultaneous query splitting
and information recovery followed by single-intent NLU while the users are
eliciting multiple intents.

##  5 Experiment Settings

#### Model Setting

We experiment with a variety of pretrained models via Hugging Face
Transformers Wolf et al. (2020), including mT5 (Xue et al., 2021) with three
different parameter scales, namely T5-base (580M), T5-large (1.2B), T5-xl
(3.7B) , and mBART-large (Liu et al., 2020b) with 340M parameters as the
backbones for the end-to-end and two-stage models. They are all multi-lingual
pre-trained models that support both Chinese and English DialogUSR. We use the
Adam optimizer (Kingma and Ba, 2015) with the learning rate of $0.00003$ and
train the models for maximum 9 epochs on 4-8 A100 Gpus.

#### Evaluation Metrics

Viewing DialohUSR as a sequence generation task, i.e. concatenating the
segmented single-intent queries with semicolons like Q4 in Fig 5, we use
BLEU-4 (Papineni et al., 2002), METEOR (Lavie and Denkowski, 2009), ROUGE-L
(Lin, 2004), which are three commonly used automatic evaluation metrics to
measure the ngram similarity with the reference in the token level. We also
propose two new sentence-level metrics, namely Split Accuracy (SACC) and Exact
Match (EM) to evaluate the model performance for DialogUSR. Specifically SACC
measures the ratio of correct query splitting. We consider a multi-query to be
correctly separated if the models split it into exactly the same number of
queries as the reference:

$\text{SACC}=\frac{1}{n}\sum_{1\leq i\leq
n}{\mathbb{I}_{\text{len}(Q_{pred}^{(i)})=\text{len}(Q_{ref}^{(i)})}},$

where $n$ is the number of instances, $\mathbb{I}$ is the indicator function,
$Q_{pred}^{(i)}$ and $Q_{ref}^{(i)}$ are the $i$-th predicted and reference
query list. As for EM, we consider it correct if the predicted query is
exactly the same as the reference one:

$\text{EM}=\frac{\sum_{i}\sum_{j}\mathbb{I}_{Q_{pred\\_j}^{(i)}=Q_{ref\\_j}^{i}}}{\sum_{1\leq
i\leq n}\text{len}(Q_{ref}^{(i)})},$

where $Q_{pred\\_j}^{(i)}$ and $Q_{ref\\_j}^{(i)}$ represent the $j$-th
predicted and reference query of the $i$-th instance. We calculate Exact Match
in three different situations: EM-Complete, where we only consider the queries
that does not need further modification (Q5 in Fig 5); EM-Rewritten, where
delete or complete actions are needed (Q6, Q7 in Fig 5); and finally EM-
Average, in which we consider all the queries.

Combination | BLEU | SACC | EM
---|---|---|---
DE $\rightarrow$ (SP+CP) | 46.08 | 77.00 | 51.25
(DE+CP) $\rightarrow$ SP | 34.23 | 70.10 | 47.57
(SP+DE) $\rightarrow$ CP | 47.60 | 82.00 | 54.15
(SP+CP) $\rightarrow$ DE | 45.67 | 80.20 | 52.42
CP $\rightarrow$ (SP+DE) | 45.66 | 78.80 | 52.28
SP $\rightarrow$ DE $\rightarrow$ CP | 47.66 | 83.40 | 54.48
SP $\rightarrow$ (DE+CP) | 49.37 | 84.40 | 56.17
Table 2: The exploration on the most effective action combination for the two-stage (once) model using the mT5-base models. SP, DE, CP are the abbreviations of the Split, Delete and Complete actions in Fig 5. Model | MixSNIPS | MixATIS
---|---|---
BLEU | EM | BLEU | EM
T5-base | 99.46 | 95.13 | 96.94 | 74.88
T5-large | 99.60 | 97.64 | 98.52 | 88.77
T5-xl | 99.62 | 98.14 | 99.87 | 98.55
Table 3: End-to-end model performance on the MixSNIPS and MixATIS datasets.
Figure 6:  The model performance (mT5-base) of different training data scale
(left) and sub-queries (right).  Figure 7:  The demonstration of generated
outputs for different baseline models. The query marked in red is wrong due to
the missing destination of the train, while the query marked in blue is a
paraphrase of the the reference.

##  6 Analysis and Discussions

#### Baseline performance

Table 1 shows the performance of the baseline models on DialogUSR. For both
end-to-end and two-stage generative baselines, enlarging the model parameters
of mT5 models leads to a considerable performance gain, which indicates that
powerful pretrained models with larger capacity are important in learning
query transformation in DialogUSR. In terms of the comparisons between end-to-
end and two variants of two-stage models, we observe that for mT5-base and
mT5-large, the causal-style two-stage model is the clear winner among the
three models, which shows that the query-by-query transformation
(Q5$\rightarrow$Q6$\rightarrow$Q7 in Fig 5) is the most effective way to
recover the missing information while reformulating the queries. For mT5-xl,
the performance gap between two-stage and end-to-end baselines is largely
reduced, indicating powerful trained models may close the gap between
different baselines.

We also report the model performance on the existing multi-intent detection
datasets, namely MixSNIPS and MixATIS. As mentioned in Sec 3, both of them are
created by inserting specific conjunctions between two complete single-intent
queries from the SNIPS (Coucke et al., 2018) or ATIS (Hemphill et al., 1990)
datasets, without any coreference or omission phenomenon. In other words, both
of them can be effectively solved with an end-to-end model using the Delete
and Split actions. The large performance gap of the same model on the MixATIS/
MixSNIPS and the proposed DialogUSR verifies that the multi-intent query
splitting and reformulation task is far from solved.

#### Findings in the different action combinations

As elaborated in Sec 4.3 and Fig 5, the Split, Delete and Complete actions can
be permuted during the generation. We thus try to find the most effective
action combination for the two-stage (once) model as shown in Table 2. We find
that 1) The 3-stage models777We try different actions permutations on the
3-stage models and put the most effective combination in Table 2.
(SP$\rightarrow$DE$\rightarrow$ CP) are not necessary in the multi-stage
generation compared with its two-stage variants (SP $\rightarrow$ (DE+CP))
because of the risk of error propagation (performance drop) and larger
computational overhead. 2) The Split action should be placed in the first
stage, as placing it in the second stage exhibit large performance drop, e.g.
SP $\rightarrow$ (DE+CP) and (DE+CP) $\rightarrow$ SP. Presumably this is
because the query splitting transformation may not be robust to the
potentially ill-formed rewritten queries due to the lack of exposure to the
noisy training data. 3) The Delete and Complete actions should be merged and
placed in the second stage of generation. These two actions together can be
viewed as a rewriting operation that deletes the conjunctions and recovers the
missing information.

#### Detailed analysis on model outputs

As the DialogUSR is actually a domain-agnostic query rewriting task, we
investigate the performances of the baseline models with different training
data scale in Fig 6 (left). With less training data, we observe a clear boost
while employing the two-stage models. Fig 6 (right) shows the model
performance while generating the sub-queries in different positions, e.g. Q5,
Q6, Q7 in Fig 5) correspond to the first, second and third queries while
splitting the multi-intent complex query. We observe a large performance drop
while comparing the first query and the subsequent queries, because in real-
world scenarios most users would not include coreferences or omissions in the
query, which make it much easier to split and complete the first sub-query.

We also provide a case study for the generated outputs from different baseline
models in Fig 7. Both the models trained with the two-stage strategy produce
correct and executable single queries, while the end-to-end model misses the
destination information in the third query, which would end up with the false
parsing results in the downstream NLU modules of conversational agents.

##  7 Related Work

#### Incomplete Utterance Restoration

To convert multi-turn incomplete dialogue into multiple single-turn complete
utterance, two major paradigms are available currently. One straight-forward
way is to consider it as a sequence-to-sequence problem, using models
including RNN (Pan et al., 2019; Elgohary et al., 2019), Trans-PG+BERT (Hao et
al., 2021) and T5 with importance token selection (Inoue et al., 2022). And
since the source and target utterances are highly overlapped, another approach
is to edit rather than generate from scratch, specifying the operation by
sequence tagging. Pan et al. (2019) proposed Pick-and-Combine model, while Liu
et al. (2020a) introduced Rewritten U-shaped Network which imitates semantic
segmentation by predicting the word-level edit matrix, and with similarity
Huang et al. (2021) used a semi auto-regressive generator. Later, Hao et al.
(2021) proposed RUST to address the robustness issue and Jin et al. (2022)
proposed hierarchical context tagging to achieve higher phrase coverage.

#### Multi-intent detection

Spoken language understanding (SLU) which consists of intent detection and
slot filling is the core in spoken dialogue systems(Tur and De Mori, 2011).
Intent detection mainly aims to classify a given utterance with its intents
from user inputs. Considering this strong correlation between the two tasks,
some joint models are proposed based on the multi-task learning framework.
(Zhang and Wang, 2016; Goo et al., 2018; Qin et al., 2019; Yao et al., 2014;
Li et al., 2018). Li et al. (2018) proposed the gate mechanism to explore
incorporating the intent information for slot filling. Convolutional-LSTM and
capsule network have been proposed to solve the problem (Xia et al., 2018).
Gangadharaiah and Narayanaswamy (2019) shows that 52% utterances are multi-
intent in the Amazon internal dataset which indicate that in real world
scenario, however, users often input utterance containing multi-intent.
Therefore, Rychalska et al. (2018) first adopted hierarchical structures to
identify multiple user intents. Qin et al. (2020) associate multi-intent
detection with slots filling via graph attention network.

Larson and Leach (2022) offers a thorough overview on the existing multi-
intent detection datasets. Except from MixATIS and MixSNIPS datasets, TOP
(Gupta et al., 2018) contains multi-intent queries annotated in a hierarchical
manner which dramatically improves the expressive power while DialogUSR
contains queries and rewriting queries which can bridge the single-intent
dection and multi-intent detection and also decoupling the query intent
detection section and multi-intent query separation section. NLU++ (Casanueva
et al., 2022) has been collected, filtered and carefully annotated by dialogue
NLU experts while DialogUSR queries are created by human annotators and
aggregated by rules and evaluated by model which lead to a lower cost of data
annotation than NLU++.

##  8 Conclusion

We propose DialogUSR, a dialog utterance splitting and reformulation task and
corresponding dataset, for multi-intent detection in the conversational
agents. The model trained on DialogUSR can serve as a domain-agnostic and
plug-in module for the existing product chatbots with minial efforts. The
proposed dataset contains 11.6k high quality instances that cover 23 domains
with a multi-step annotation process. We propose multiple action-based
generative baselines to benchmark the dataset and analyze their pros and cons
through a series of investigations.

## Limitations

The proposed DialogUSR focuses on a single task for the research community and
lacks of implementation details in the product conversational agents. The
approaches on how the proposed DialogUSR interacts with other modules, e.g.
dialog manager, ranking module for candidate NLU parsing results, remains an
interesting and important research area. We position our work in the line of
researches which enhances advanced conversational AI (i.e. multi-turn or
multi-intent) by _query rewriting_ , and leave multi-intent slot-filling
entity annotation to the further work.

## Acknowledgement

We thank all the anonymous reviewers for their insightful feedback. This paper
is supported by the National Key Research and Development Program of China
2020AAA0106700 and the National Natural Science Foundation of China (NSFC)
project U19A2065.

## References

  * Casanueva et al. (2022) Inigo Casanueva, Ivan Vulić, Georgios Spithourakis, and Paweł Budzianowski. 2022.  NLU++: A multi-label, slot-rich, generalisable dataset for natural language understanding in task-oriented dialogue.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 1998–2013, Seattle, United States. Association for Computational Linguistics.
  * Coucke et al. (2018) Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018.  Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces.  _arXiv preprint arXiv:1805.10190_.
  * Elgohary et al. (2019) Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019.  Can you unpack that? learning to rewrite questions-in-context.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5918–5924, Hong Kong, China. Association for Computational Linguistics.
  * Gangadharaiah and Narayanaswamy (2019) Rashmi Gangadharaiah and Balakrishnan Narayanaswamy. 2019.  Joint multiple intent detection and slot labeling for goal-oriented dialog.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 564–569, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Goo et al. (2018) Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. 2018.  Slot-gated modeling for joint slot filling and intent prediction.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 753–757, New Orleans, Louisiana. Association for Computational Linguistics.
  * Gupta et al. (2018) Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018.  Semantic parsing for task oriented dialog using hierarchical representations.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2787–2792, Brussels, Belgium. Association for Computational Linguistics.
  * Hao et al. (2021) Jie Hao, Linfeng Song, Liwei Wang, Kun Xu, Zhaopeng Tu, and Dong Yu. 2021.  RAST: Domain-robust dialogue rewriting as sequence tagging.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 4913–4924, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Hemphill et al. (1990) Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990.  The ATIS spoken language systems pilot corpus.  In _Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990_.
  * Huang et al. (2021) Mengzuo Huang, Feng Li, Wuhe Zou, and Weidong Zhang. 2021.  Sarg: A novel semi autoregressive generator for multi-turn incomplete utterance restoration.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(14):13055–13063.
  * Inoue et al. (2022) Shumpei Inoue, Tsungwei Liu, Nguyen Hong Son, and Minh-Tien Nguyen. 2022.  Enhance incomplete utterance restoration by joint learning token extraction and text generation.  _CoRR_ , abs/2204.03958.
  * Jin et al. (2022) Lisa Jin, Linfeng Song, Lifeng Jin, Dong Yu, and Daniel Gildea. 2022.  Hierarchical context tagging for utterance rewriting.  In _Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022_ , pages 10849–10857. AAAI Press.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Larson and Leach (2022) Stefan Larson and Kevin Leach. 2022.  A survey of intent classification and slot-filling datasets for task-oriented dialog.  _arXiv preprint arXiv:2207.13211_.
  * Lavie and Denkowski (2009) Alon Lavie and Michael J. Denkowski. 2009.  The meteor metric for automatic evaluation of machine translation.  _Machine Translation_ , 23(2–3):105–115.
  * Li et al. (2018) Changliang Li, Liang Li, and Ji Qi. 2018.  A self-attentive model with gate mechanism for spoken language understanding.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3824–3833, Brussels, Belgium. Association for Computational Linguistics.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Liu et al. (2020a) Qian Liu, Bei Chen, Jian-Guang Lou, Bin Zhou, and Dongmei Zhang. 2020a.  Incomplete utterance rewriting as semantic segmentation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2846–2857, Online. Association for Computational Linguistics.
  * Liu et al. (2019) Tianyu Liu, Fuli Luo, Pengcheng Yang, Wei Wu, Baobao Chang, and Zhifang Sui. 2019\.  Towards comprehensive description generation from factual attribute-value tables.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5985–5996, Florence, Italy. Association for Computational Linguistics.
  * Liu et al. (2021) Tianyu Liu, Xin Zheng, Baobao Chang, and Zhifang Sui. 2021.  Towards faithfulness in open domain table-to-text generation from an entity-centric view.  In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_ , pages 13415–13423. AAAI Press.
  * Liu et al. (2020b) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020b.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742.
  * Moryossef et al. (2019) Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019.  Step-by-step: Separating planning from realization in neural data-to-text generation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2267–2277, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Pan et al. (2019) Zhufeng Pan, Kun Bai, Yan Wang, Lianqiang Zhou, and Xiaojiang Liu. 2019.  Improving open-domain dialogue systems via multi-turn incomplete utterance restoration.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1824–1833, Hong Kong, China. Association for Computational Linguistics.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  * Qin et al. (2019) Libo Qin, Wanxiang Che, Yangming Li, Haoyang Wen, and Ting Liu. 2019.  A stack-propagation framework with token-level intent detection for spoken language understanding.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2078–2087, Hong Kong, China. Association for Computational Linguistics.
  * Qin et al. (2020) Libo Qin, Xiao Xu, Wanxiang Che, and Ting Liu. 2020.  AGIF: An adaptive graph-interactive framework for joint multiple intent detection and slot filling.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1807–1816, Online. Association for Computational Linguistics.
  * Quan et al. (2020) Jun Quan, Shian Zhang, Qian Cao, Zizhong Li, and Deyi Xiong. 2020.  RiSAWOZ: A large-scale multi-domain Wizard-of-Oz dataset with rich semantic annotations for task-oriented dialogue modeling.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 930–940, Online. Association for Computational Linguistics.
  * Rychalska et al. (2018) Barbara Rychalska, Helena T. Glabska, and Anna Wróblewska. 2018.  Multi-intent hierarchical natural language understanding for chatbots.  _2018 Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS)_ , pages 256–259.
  * Tur and De Mori (2011) Gokhan Tur and Renato De Mori. 2011.  _Spoken language understanding: Systems for extracting semantic information from speech_.  John Wiley & Sons.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Xia et al. (2018) Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, and Philip Yu. 2018.  Zero-shot user intent detection via capsule neural networks.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3090–3099, Brussels, Belgium. Association for Computational Linguistics.
  * Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021.  mT5: A massively multilingual pre-trained text-to-text transformer.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 483–498, Online. Association for Computational Linguistics.
  * Yao et al. (2014) Kaisheng Yao, Baolin Peng, Yu Zhang, Dong Yu, Geoffrey Zweig, and Yangyang Shi. 2014\.  Spoken language understanding using long short-term memory neural networks.  In _2014 IEEE Spoken Language Technology Workshop (SLT)_ , pages 189–194.
  * Zhang et al. (2017) Wei-Nan Zhang, Zhigang Chen, Wanxiang Che, Guoping Hu, and Ting Liu. 2017.  The first evaluation of chinese human-computer dialogue technology.  _arXiv preprint arXiv:1709.10217_.
  * Zhang and Wang (2016) Xiaodong Zhang and Houfeng Wang. 2016.  A joint model of intent determination and slot filling for spoken language understanding.  In _Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016_ , pages 2993–2999. IJCAI/AAAI Press.

##  Appendix A Implementation Detail

We run the experiments with Huggingface Transformers library on 4 Nvidia A100
GPU, with the training batch size of 96. For experiments on full training set,
we set warm up step as 50. For beam search of seq2seq model, the beam size is
4. We train and test our models with 8 A100 GPUs.

Model-Size | Inference (ms) | Train (s)
---|---|---
End-to-end, base | 71.94 | 289
End-to-end, large | 114.95 | 450
End-to-end, xl | 118.59 | 712
Two-stage(once), base | 156.85 | 570
Two-stage(once), large | 201.11 | 875
Two-stage(once), xl | 211.26 | 1392
Two-stage(casual), base | 173.90 | 1118
Two-stage(casual), large | 253.06 | 1911
Two-stage(casual), xl | 277.07 | 2859
Table 4: Model Inference Speed and Training Time

| Conj0_Prob | Conj1_Prob | Conj2_Prob | Conj3_Prob
---|---|---|---|---
None | 50% | 50% | 50% | 50%
UTF8gbsn先 | 25% | 0% | 0% | 0%
UTF8gbsn首先 | 25% | 0% | 0% | 0%
UTF8gbsn然后 | 0% | 10% | 10% | 10%
UTF8gbsn还有 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn我还想知道 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn另外我想知道 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn再一个就是 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn以及 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn和 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn还要 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn并且 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn再然后 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn另外 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn其次 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn同时 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn除了这个还有 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn接着 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn紧接着 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn接下来 | 0% | 2.50% | 2.50% | 2.35%
UTF8gbsn最后 | 0% | 0.00% | 0.00% | 2.35%

Table 5: Conjunction probability distribution in four queries cases.

| Conj0_Prob | Conj1_Prob | Conj2_Prob
---|---|---|---
None | 50% | 50% | 50%
UTF8gbsn先 | 25% | 0% | 0%
UTF8gbsn首先 | 25% | 0% | 0%
UTF8gbsn然后 | 0% | 10% | 10%
UTF8gbsn还有 | 0% | 2.50% | 2.35%
UTF8gbsn我还想知道 | 0% | 2.50% | 2.35%
UTF8gbsn另外我想知道 | 0% | 2.50% | 2.35%
UTF8gbsn再一个就是 | 0% | 2.50% | 2.35%
UTF8gbsn以及 | 0% | 2.50% | 2.35%
UTF8gbsn和 | 0% | 2.50% | 2.35%
UTF8gbsn还要 | 0% | 2.50% | 2.35%
UTF8gbsn并且 | 0% | 2.50% | 2.35%
UTF8gbsn再然后 | 0% | 2.50% | 2.35%
UTF8gbsn另外 | 0% | 2.50% | 2.35%
UTF8gbsn其次 | 0% | 2.50% | 2.35%
UTF8gbsn同时 | 0% | 2.50% | 2.35%
UTF8gbsn除了这个还有 | 0% | 2.50% | 2.35%
UTF8gbsn接着 | 0% | 2.50% | 2.35%
UTF8gbsn紧接着 | 0% | 2.50% | 2.35%
UTF8gbsn接下来 | 0% | 2.50% | 2.35%
UTF8gbsn最后 | 0% | 0.00% | 2.35%

Table 6: Conjunction probability distribution in three queries cases. Input |  UTF8gbsn昭通到南昌的汽车最早几点发车走高速吗另外我想知道自驾要多久到
---|---
|  The bus from Zhaotong to Nanchang; what time does it leave at the earliest;
does it take the highway; also I would like to know how long it will take to
get there by self-driving
End-to-end |  UTF8gbsn昭通到南昌的汽车 [SP] 昭通到南昌的汽车最早几点发车 [SP] 昭通到南昌的汽车走高速吗 [SP] 昭通到南昌自驾要多久到
|  The bus from Zhaotong to Nanchang [SP] What time does the bus from Zhaotong
to Nanchang leave at the earliest [SP] Does the bus from Zhaotong to Nanchang
take the highway [SP] How long it will take to get there from Zhaotong to
Nanchang by self-driving
Two-stage (Once) |  UTF8gbsn昭通到南昌的汽车 [SP] 昭通到南昌的汽车最早几点发车 [SP] 昭通到南昌的汽车走高速吗 [SP] 自驾昭通到南昌要多久到
|  The bus from Zhaotong to Nanchang [SP] What time does the bus from Zhaotong
to Nanchang leave at the earliest [SP] Does the bus from Zhaotong to Nanchang
take the highway [SP] How long it will take to get there by self-driving from
Zhaotong to Nanchang
Two-stage (causal) |  UTF8gbsn昭通到南昌的汽车 [SP] 昭通到南昌的汽车最早几点发车 [SP] 昭通到南昌的汽车走高速吗 [SP] 自驾昭通到南昌要多久到
|  The bus from Zhaotong to Nanchang [SP] What time does the bus from Zhaotong
to Nanchang leave at the earliest [SP] Does the bus from Zhaotong to Nanchang
take the highway [SP] How long it will take to get there by self-driving from
Zhaotong to Nanchang
Input |  UTF8gbsn推荐一部动作电影我要看三十集左右的电视剧然后找一部90年代的动作电影
|  Recommend an action movie; I want to watch a TV series with about 30
episodes; and then look for an action movie in the 90s
End-to-end |  UTF8gbsn推荐一部动作电影 [SP] 我要看三十集左右的动作电视剧 [SP] 找一部90年代的动作电影
|  Recommend an action movie [SP] I want to watch an action TV series with
around thirty episodes [SP] Looking for an action movie from the 90s
Two-stage (Once) |  UTF8gbsn推荐一部动作电影 [SP] 我要看三十集左右的电视剧 [SP] 找一部90年代的动作电影
|  Recommend an action movie [SP] I want to watch a TV series with around
thirty episodes [SP] Looking for an action movie from the 90s
Two-stage (causal) |  UTF8gbsn推荐一部动作电影 [SP] 我要看三十集左右的动作电视剧 [SP] 找一部90年代三十集左右的动作电影
|  Recommend an action movie [SP] I want to watch an action TV series with
around thirty episodes [SP] Looking for an action movie from the 90s with
around thirty episodes
Input |  UTF8gbsn首先查下下周三杭州是什么天气另外我想知道下周三有没有去杭州的航班我还想知道适合穿什么衣服
|  First, I want to check what weather will be in Hangzhou next Wednesday; in
addition, I want to know if there is a flight to Hangzhou next Wednesday; I
also want to know what clothes to wear.
End-to-end |  UTF8gbsn查下下周三杭州是什么天气 [SP] 下周三有没有去杭州的航班 [SP] 下周三杭州的天气适合穿什么衣服
|  Check what weather will be in Hangzhou next Wednesday [SP] Is there a
flight to Hangzhou next Wednesday [SP] What clothes should I wear for the
weather in Hangzhou next Wednesday
Two-stage (Once) |  UTF8gbsn 查下下周三杭州是什么天气 [SP] 下周三有没有去杭州的航班 [SP] 下周三杭州适合穿什么衣服
|  Check what weather will be in Hangzhou next Wednesday [SP] Is there a
flight to Hangzhou next Wednesday [SP] What clothes should I wear in Hangzhou
next Wednesday
Two-stage (causal) |  UTF8gbsn查下下周三杭州是什么天气 [SP] 下周三有没有去杭州的航班 [SP] 下周三杭州适合穿什么衣服
|  Check what weather will be in Hangzhou next Wednesday [SP] Is there a
flight to Hangzhou next Wednesday [SP] What clothes should I wear in Hangzhou
next Wednesday
Input |  UTF8gbsn推荐一款越野的三厢车有没有四驱的我想要高配车
|  Recommend an off-road sedan; is there any four-wheel drive; I want a high-
end car
End-to-end |  UTF8gbsn推荐一款越野的三厢车 [SP] 有没有四驱的越野的三厢车 [SP] 我想要高配的越野的三厢车
|  Recommend an off-road sedan [SP] Is there a four-wheel drive off-road sedan
[SP] I want a high-profile off-road sedan
Two-stage (Once) |  UTF8gbsn推荐一款越野的三厢车 [SP] 有没有四驱的越野三厢车 [SP] 我想要高配的四驱越野三厢车;
|  Recommend an off-road sedan [SP] Is there a four-wheel drive off-road sedan
[SP] I want a high-end four-wheel drive off-road sedan
Two-stage (causal) |  UTF8gbsn推荐一款越野的三厢车 [SP] 有没有四驱的越野的三厢车 [SP] 我想要高配的四驱的越野三厢车 [SP]
|  Recommend an off-road sedan [SP] Is there a four-wheel drive off-road sedan
[SP] I want a high-end four-wheel drive off-road sedan
Table 7: Examples of model rewriting queries

##  Appendix B Query Aggregation Detail

In Table 5, we provide conjunction probability distribution when we have four
queries need to be aggregated. Conjunction0 is placed at the head of
consecutive query1. Conjunction1, Conjunction2 and Conjunction3 is placed at
the tail of consecutive query1, query2 and query3 respectively. As described
in Table 5, Conjunction0 have 50% chance to be empty and 25% probability to be
UTF8gbsn“先”(first) and another 25% chance to be UTF8gbsn“首先”(first of all).
Similarly, Conjunction1 is placed at the middle of query1 and query2 with the
probability described in the table and so on. Table 6 shows the probability
distribution of conjunctions when three consecutive queries that need to be
aggregated. We generate 10 candidate multi-intent queries by joining
consecutive queries with conjunctions described in Table 5 and Table 6. After
query aggregation, we calculate the perplexity of ten candidate multi-intent
queries and select the most fluent sentence as multi-intent query in
DialogUSR.

##  Appendix C DialogUSR Cases In All Domains

In Figure 8, for every twenty three domains, we respectively provide one case
to show our dataset. All twenty three domains in DialogUSR are listed in
Figure 8 including Attraction, TV, Railway, Weather, Restaurant, Flight,
Movie, Hotel, Car, Hospital, Courses, Cook, News, App, Navigation, Music,
Translation, Mail, Dial, Disease, Time, Sports. Query indicates the multi-
intent query in DialogUSR and Query1 to Query4 represent the single-intent
queries in DialogUSR.

As mentioned in Follow-up Query Creation section, we observe that 37.3% multi-
intent queries involve topic switching and this phenomenon can be found in
case of Translation, Time, Phone Courses etc. In Translation case, query1 to
query3 is about translation while query4 What is the route to
Tiananmen(UTF8gbsn去天安门的路线是什么) is about Navigation. As shown in Figure 4, a
large amount of sub-queries in multi-intent query is missing information
therefore they need to be rewritten by human annotators. This situation can be
easily found in many cases, for example, TV case, Railway case, Weather case,
Restaurant case etc. For example, in Railway case, the sub-query I would also
like to know what time is the latest train? (UTF8gbsn我还想知道最晚的车次是几点？) lack the
key information and human annotator rewirte the sub-query as What is the
latest train number to Zhengzhou (UTF8gbsn到郑州最晚的车次是几点). We also provide a
English version of twenty three cases in every domain in Fig 9.

##  Appendix D Broader Impact and Ethnic Consideration

Data in DialogUSR does not involve user privacy. The data source we collect
from SMP-ECDT and RiSAWOZ is open source for research and is licensed under
the MIT License which is a short and simple permissive license with conditions
only requiring preservation of copyright and license notices.

Our generative baseline models have very low risk in terms of producing
discriminatory, insulting words or divulging privacy due to the fact all the
training data are strictly screened and do not include private user
information or insulting content. All involved annotators voluntarily
participated with decent payment.

Figure 8: DialogUSR dataset instances in all domains. Punctuations are added
in the last column for better readability.  Figure 9: English version of
DialogUSR dataset instances in all domains.


#  Two non-perturbative $\alpha^{\prime}$ corrected or loop corrected string
cosmological solutions

Li Song<EMAIL_ADDRESS>College of Physics, Sichuan University,
Chengdu, 610065, China  Deyou Chen<EMAIL_ADDRESS>School of Science,
Xihua University, Chengdu 610039, China

###### Abstract

In this paper, we present two non-perturbative string cosmological solutions
without curvature singularities for the bosonic gravi-dilaton system. These
solutions are general in the sense that they can straightforwardly match the
perturbative solution to arbitrarily high orders in the perturbative region.
The first solution includes non-perturbative $\alpha^{\prime}$ corrections
based on Hohm-Zwiebach action. We then use the simple phenomenological map
between $\alpha^{\prime}$ corrected theory and loop corrected theory in string
cosmology to construct a non-perturbative loop corrected non-singular
solution. Both solutions are non-singular everywhere. Therefore, the pre- and
post-big-bangs are smoothly connected by these solutions.

##  I Introduction

Resolving the big-bang singularity is a widely concerned problem. In
Einstein’s general theory of gravity, spacetime is generated from the big-
bang. On the other hand, in string cosmology, the story is quite different due
to the famous scale-factor duality Veneziano:1991ek ; Sen:1991zi ;
Tseytlin:1991wr ; Tseytlin:1991xk . The scale-factor duality had been firstly
obtained from the equations of motion (EOM) of the string’s low energy gravi-
dilaton effective action. It shows that the EOM with the FLRW-like ansatz is
invariant under the transformations between the scale factor and its
inversion, namely $a\left(t\right)\leftrightarrow 1/a\left(t\right)$. Keep in
mind that the string dilaton plays a central role in this duality. The main
differences between T-duality and scale-factor duality are that the scale-
factor duality does not require the background compactified, and it belongs to
a continuous group of $O\left(d,d\right)$. Remarkably, the scale-factor
duality introduces a pre-big-bang cosmology Gasperini:1996fu ;
Veneziano:2000pz ; Gasperini:2002bn ; Gasperini:2007vw ; Gasperini:1992em . It
implies that the story of our universe was not only born from the initial big-
bang singularity, but there exists a long evolution in a region of pre-big-
bang. Pre- and post- big-bang scenarios are disconnected by the big-bang
singularity. On the other hand, when the universe evolves to the big-bang
singularity, the growth of the string coupling $g_{s}=\exp\left(2\phi\right)$
and the Hubble parameter $H\left(t\right)$ invalidate the perturbative theory.
The theory of quantum gravity around this region requires the low energy
effective action to include two kinds of corrections: (1) the string curvature
scale, which includes higher-derivative $\alpha^{\prime}$ corrections, (2) the
strong coupling regime, which requires the quantum loop corrections.

In literature, some phenomenological higher loop models Gasperini:1992em ;
Gasperini:2003pb ; Gasperini:2004ss  were proposed to resolve the big-bang
singularity. As for the higher-derivative $\alpha^{\prime}$ corrections, at
the cost of losing the scale-factor duality or the $O(d,d)$ symmetry, some
simplified models were proposedGasperini:1996fu ; Easson:2003ia  to smoothly
connect the Pre- and post- big-bang scenarios. One may wonder if we can obtain
a nature global picture of the non-singular universe when the action includes
the $\alpha^{\prime}$ corrections to all orders.

To answer this question, let us focus on the recent developments of non-
perturbative string cosmology to all orders in $\alpha^{\prime}$. In refs.
Meissner:1991zj ; Meissner:1996sa , in addition to manifesting the scale-
factor duality in the EOM, Meissner and Veneziano found that when the massless
closed string fields only depending on time, the low energy effective action
of closed string with the zeroth and the first orders in $\alpha^{\prime}$
could be simplified and rewritten in an $O\left(d,d\right)$ invariant form.
After that, Sen proved that this result can be extended to all orders in
$\alpha^{\prime}$ corrections of full string field theory Sen:1991zi ;
Sen:1991cn . In other words, the closed string spacetime action can be
rewritten into the $O\left(d,d\right)$ covariant form to all orders in
$\alpha^{\prime}$ without imposing any extra constraint or symmetry. Based on
these works, Hohm and Zwiebach Hohm:2015doa ; Hohm:2019ccp ; Hohm:2019jgu
demonstrated that the $O\left(d,d\right)$ covariant spacetime action can be
dramatically simplified into a simple form. This result in turn provides
possible non-perturbative dS or AdS vacua to the bosonic string theory
Hohm:2019ccp ; Wang:2019mwi ; Krishnan:2019mkv ; Nunez:2020hxx ;
Bieniek:2023ubx . Moreover, the Hohm-Zwiebach action makes it possible to
analyze the stringy effects on the cosmological singularity and black hole
singularity seriously. In ref. Wang:2019kez , a set of non-singular non-
perturbative string cosmological solutions is constructed for the first time.
This set of solutions is subsequently extended to more general solutions which
can match the perturbative solution to arbitrary order in $\alpha$’ expansion
Wang:2019dcj . More recent developments on the smoothing cosmological and
black hole singularities refer to refs. Ying:2021xse ; Ying:2022xaj ;
Klinkhamer:2019ocj ; Bernardo:2019bkz ; Bernardo:2020zlc ; Quintin:2021eup ;
Codina:2021cxh ; Ying:2022cix ; Codina:2023fhy ; Gasperini:2023tus .

The purpose of this paper is to give non-singular non-perturbative solutions
with $\alpha^{\prime}$ corrections or loop corrections. We first construct an
$\alpha^{\prime}$ corrected solution based on Hohm-Zwiebach action. The
solution we present is non-perturbative in the sense that it covers the whole
region from the pre-big-bang to the post-big-bang and non-singular everywhere.
In the perturbative region $t\to\infty$, this solution matches the
perturbative solution to arbitrary order in $\alpha^{\prime}$. So, though we
currently only know the first two orders in $\alpha^{\prime}$ corrections,
once higher orders are calculated in the future, our solution can
straightforwardly match them by fixing the parameters. Our solution is
parallel to these given in Wang:2019dcj . In ref. Wang:2019dcj , a
phenomenological map between the $\alpha^{\prime}$ corrected theory and loop
corrected theory are found. With the help of this map, we then construct a
non-perturbative loop corrected solution, which is non-singular everywhere and
includes higher loop contributions. So, both the $\alpha^{\prime}$ corrected
or the loop corrected solution we are going to construct resolve the big-bang
singularity and connects the pre-big-bang and post-big-bang scenarios
smoothly.

The reminder of this paper is outlined as follows. We give a non-singular non-
perturbative solution to any order in $\alpha^{\prime}$ expansion in section
2. In section 3, we present a loop corrected non-singular non-perturbative
solution. Section 4 is the conclusion.

##  II The general $\alpha^{\prime}$ corrected solution

Since we are going to find both $\alpha^{\prime}$ corrected solutions and loop
corrected solutions, we write the full perturbative structure of the closed
string effective action,

$\displaystyle I$ $\displaystyle=$
$\displaystyle\int\>d^{d+1}x\sqrt{-g}\bigg{\\{}e^{-2\phi}\Big{[}(R+4(\partial\phi)^{2}-\frac{1}{12}{\cal
H}^{2})+\frac{\text{$\alpha^{\prime}$}}{4}(R_{\mu\nu\sigma\rho}R^{\mu\nu\sigma\rho}+\cdots)+{\cal
O}(\alpha^{\prime 2})\big{]}$ (1) $\displaystyle+$
$\displaystyle\Big{[}(c_{R}^{1}R+c_{\phi}^{1}(\partial\phi)^{2}+c_{{\cal
H}}^{1}{\cal
H}^{2})+\alpha^{\prime}(c_{\alpha^{\prime}R}^{1}R_{\mu\nu\sigma\rho}R^{\mu\nu\sigma\rho}+\cdots)+{\cal
O}(\alpha^{\prime 2})\Big{]}$ $\displaystyle+$ $\displaystyle
e^{2\phi}\Big{[}(c_{R}^{2}R+c_{\phi}^{2}(\partial\phi)^{2}+c_{{\cal
H}}^{2}{\cal
H}^{2})+\alpha^{\prime}(c_{\alpha^{\prime}R}^{2}R_{\mu\nu\sigma\rho}R^{\mu\nu\sigma\rho}+\cdots)+{\cal
O}(\alpha^{\prime 2})\Big{]}$ $\displaystyle+$
$\displaystyle\cdots\bigg{\\}}.$

This action contains three massless fields, the metric $g_{\mu\nu}$, dilaton
$\phi$ and the antisymmetric field $b_{\mu\nu}$ whose field strength is ${\cal
H}_{\mu\nu\rho}=3\partial_{[\mu}b_{\nu\rho]}$. We will set $b_{\mu\nu}=0$ in
this paper. All $c_{[\cdots]}^{i}$ are not known until now. We first consider
the $\alpha^{\prime}$ corrected solutions. To this end, we focus on the loop
tree level, i.e. the first line of the above action. In the FLRW background,

$ds^{2}=-dt^{2}+a^{2}\left(t\right)\delta_{ij}dx^{i}dx^{j}.$ (2)

the loop tree level action with all orders in $\alpha^{\prime}$ corrections is
given by Hohm and Zwiebach in Refs. Hohm:2019ccp ; Hohm:2019jgu :

$\displaystyle I_{\alpha^{\prime}}$ $\displaystyle=$ $\displaystyle\int
d^{D}x\sqrt{-g}e^{-2\phi}\left(R+4\left(\partial\phi\right)^{2}+\frac{1}{4}\alpha^{\prime}\left(R^{\mu\nu\rho\sigma}R_{\mu\nu\rho\sigma}+\ldots\right)+\alpha^{\prime
2}(\ldots)+\ldots\right),$ (3) $\displaystyle=$ $\displaystyle\int
dte^{-\Phi}\left(-\dot{\Phi}^{2}+\sum_{k=1}^{\infty}\left(\alpha^{\prime}\right)^{k-1}c_{k}\mathrm{tr}\left(\dot{\mathcal{S}}^{2k}\right)\right),$
(4)

The notation $\mathcal{S}$ is defined as

$\mathcal{S}=\left(\begin{array}[]{cc}0&a^{2}\left(t\right)\\\
a^{-2}\left(t\right)&0\end{array}\right).$ (5)

In the action (4), we can only determine the coefficients $c_{1}=-\frac{1}{8}$
and $c_{2}=\frac{1}{64}$ for the bosonic string theory through the one-loop
and two-loops beta functions of non-linear sigma model, and $c_{k\geq 3}$ are
undetermined constants. Define

$\displaystyle H\left(t\right)$ $\displaystyle=$
$\displaystyle\frac{\dot{a}\left(t\right)}{a\left(t\right)},$ $\displaystyle
f\left(H\right)$ $\displaystyle=$ $\displaystyle
d\sum_{k=1}^{\infty}\left(-\alpha^{\prime}\right)^{k-1}2^{2\left(k+1\right)}kc_{k}H^{2k-1}$
$\displaystyle=$
$\displaystyle-2dH-2d\alpha^{\prime}H^{3}+\mathcal{O}\left(\alpha^{\prime
3}\right),$ $\displaystyle g\left(H\right)$ $\displaystyle=$ $\displaystyle
d\sum_{k=1}^{\infty}\left(-\alpha^{\prime}\right)^{k-1}2^{2k+1}\left(2k-1\right)c_{k}H^{2k}$
(6) $\displaystyle=$ $\displaystyle-
dH^{2}-\frac{3}{2}d\alpha^{\prime}H^{4}+\mathcal{O}\left(\alpha^{\prime}\right),$

where $H\left(t\right)$ is the Hubble parameter. Note that

$g^{\prime}(H)=Hf^{\prime}(H),\quad{\rm and}\quad
g(H)=Hf(H)-\int_{0}^{H}f(x)dx,$

where $f^{\prime}(H)\equiv\frac{d}{dH}f(H)$. The action (4) is simplified to
the Hohm-Zwiebach action

$\displaystyle I_{HZ}$ $\displaystyle=$ $\displaystyle\int
dte^{-\Phi}\left(-\dot{\Phi}^{2}+g(H)-Hf(H)\right),$ (7)

After variation, the EOM of Hohm-Zwiebach action (7) is

$\displaystyle\ddot{\Phi}+\frac{1}{2}Hf\left(H\right)$ $\displaystyle=$
$\displaystyle 0,$
$\displaystyle\frac{d}{dt}\left(e^{-\Phi}f\left(H\right)\right)$
$\displaystyle=$ $\displaystyle 0,$
$\displaystyle\dot{\Phi}^{2}+g\left(H\right)$ $\displaystyle=$ $\displaystyle
0.$ (8)

In the perturbative region $t\to\infty$, the perturbative solution can be
obtained iteratively by using (6),

$\displaystyle H\left(t\right)$ $\displaystyle=$
$\displaystyle\frac{\sqrt{2}}{\sqrt{\alpha^{\prime}}}\left[\frac{t_{0}}{t}-160c_{2}\frac{t_{0}^{3}}{t^{3}}+\frac{256\left(770c_{2}^{2}+19c_{3}\right)}{3}\frac{t_{0}^{5}}{t^{5}}\right.$
$\displaystyle\left.-\frac{2048\left(88232c_{2}^{3}+4644c_{3}c_{2}+41c_{4}\right)}{5}\frac{t_{0}^{7}}{t^{7}}+\mathcal{O}\left(\frac{t_{0}^{9}}{t^{9}}\right)\right],$
$\displaystyle\Phi\left(t\right)$ $\displaystyle=$
$\displaystyle-\frac{1}{2}\log\left(\beta^{2}\frac{t^{2}}{t_{0}^{2}}\right)-32c_{2}\frac{t_{0}^{2}}{t^{2}}+\frac{256\left(44c_{2}^{2}+c_{3}\right)}{3}\frac{t_{0}^{4}}{t^{4}}$
(9)
$\displaystyle-\frac{2048\left(6976c_{2}^{3}+352c_{3}c_{2}+3c_{4}\right)}{15}\frac{t_{0}^{6}}{t^{6}}+\mathcal{O}\left(\frac{t_{0}^{8}}{t^{8}}\right),$

with

$\displaystyle f\left(H\left(t\right)\right)$ $\displaystyle=$
$\displaystyle-2dH-128c_{2}d\alpha^{\prime}H^{3}+768c_{3}d\alpha^{\prime
2}H^{5}-4096c_{4}d\alpha^{\prime 3}H^{7}+\mathcal{O}\left(\alpha^{\prime
4}H^{9}\right),$ $\displaystyle=$
$\displaystyle\frac{\sqrt{d}}{t_{0}}\left[-\frac{2t_{0}}{t}+64c_{2}\frac{t_{0}^{3}}{t^{3}}-\frac{512\left(50c_{2}^{2}+c_{3}\right)}{3}\frac{t_{0}^{5}}{t^{5}}\right.$
$\displaystyle\left.+\frac{4096\left(2632c_{2}^{3}+124c_{3}c_{2}+c_{4}\right)}{5}\frac{t_{0}^{7}}{t^{7}}+\mathcal{O}\left(\frac{t_{0}^{9}}{t^{9}}\right)\right],$
$\displaystyle g\left(H\left(t\right)\right)$ $\displaystyle=$ $\displaystyle-
dH^{2}-96c_{2}d\alpha^{\prime}H^{4}+640c_{3}d\alpha^{\prime
2}H^{6}-3584c_{4}d\alpha^{\prime 3}H^{8}+\mathcal{O}\left(\alpha^{\prime
4}H^{10}\right),$ $\displaystyle=$
$\displaystyle\frac{1}{t_{0}^{2}}\left[-\frac{t_{0}^{2}}{t^{2}}+128c_{2}\frac{t_{0}^{4}}{t^{4}}-\frac{2048\left(50c_{2}^{2}+c_{3}\right)}{3}\frac{t_{0}^{6}}{t^{6}}\right.$
$\displaystyle\left.+\frac{8192\left(24448c_{2}^{3}+1136c_{3}c_{2}+9c_{4}\right)}{15}\frac{t_{0}^{8}}{t^{8}}+\mathcal{O}\left(\frac{t_{0}^{10}}{t^{10}}\right)\right],$

where $\beta$ is an integration constant,
$t_{0}\equiv\frac{\sqrt{\alpha^{\prime}}}{\sqrt{2d}}$ and universal
$c_{1}=-\frac{1}{8}$ is used. One is ready to see that this solution is
singular at the non-perturbative region $t=0$. From the scale factor duality,
$H(t)\to-H(t)$, $\Phi(t)\to\Phi(t)$, $f(t)\to-f(t)$ and $g(t)\to g(t)$ is also
a solution.

A non-perturbative non-singular solution of the EOM (8) should meet two
conditions: (1) It must match the perturbative solution (9) in the
perturbative region $\frac{\sqrt{\alpha^{\prime}}}{t}\to 0$; (2) It must be
non-singular for any $\frac{\sqrt{\alpha^{\prime}}}{t}$. Since until now
$c_{k\geq 3}$ are still unknown, a good solution only needs to match the first
two orders in $\alpha^{\prime}$. Such a solution has been constructed in ref.
Wang:2019kez . However, an important question is that, can we construct a
general solution which can easily match all given $c_{k\leq n}$ for some
$n>2$? In other words, a general solution is expressed in terms of $c_{k}$. In
ref. Wang:2019dcj , two such solutions were given. In this work, we give
another simpler one. Every term in our solution is non-singular. Let us first
define a dimensionless parameter

$\tau\equiv\frac{t}{t_{0}}=\sqrt{\frac{2d}{\alpha^{\prime}}}t.$ (10)

After some long calculation, we find a non-perturbative solution of the EOM
(8),

$\displaystyle H(t)$ $\displaystyle=$
$\displaystyle\frac{\sqrt{\frac{2}{\alpha^{\prime}\beta^{2}\lambda_{0}}}\left(e^{-\sum_{k=1}^{\infty}\frac{\lambda_{k}}{\tau^{2k}+1}}\left(\left(\tau^{2}+1\right)^{2}\sum_{k=1}^{\infty}\left(\frac{8k^{2}\lambda_{k}\tau^{4k-2}}{\left(\tau^{2k}+1\right)^{3}}-\frac{2k(2k-1)\lambda_{k}\tau^{2k-2}}{\left(\tau^{2k}+1\right)^{2}}\right)+\tau^{2}-1\right)\right)}{\left(\tau^{2}+1\right)^{3/2}}$
(11) $\displaystyle\Phi(t)$ $\displaystyle=$
$\displaystyle\frac{1}{2}\log\frac{\lambda_{0}}{1+\tau^{2}}+\sum_{n=1}^{\infty}\frac{\lambda_{n}}{1+\tau^{2n}},$
(12) $\displaystyle f(H(t))$ $\displaystyle=$
$\displaystyle-2d\sqrt{\frac{2\beta^{2}\lambda_{0}}{\left(\tau^{2}+1\right)\alpha^{\prime}}}e^{\sum_{k=1}^{\infty}\frac{\lambda_{k}}{\tau^{2k}+1}}=-2dH(t)-2d\alpha^{\prime}H(t)^{3}+{\cal
O}(\alpha^{\prime 2})$ (13) $\displaystyle g(H(t))$ $\displaystyle=$
$\displaystyle-\frac{2d}{\alpha^{\prime}}\left(\sum_{k=1}^{\infty}\frac{2k\lambda_{k}\tau^{2k-1}}{\left(\tau^{2k}+1\right)^{2}}+\frac{\tau}{\tau^{2}+1}\right)^{2}=-dH(t)^{2}-\frac{3}{2}d\alpha^{\prime}H(t)^{4}+{\cal
O}(\alpha^{\prime 2}).$ (14)

Applying $c_{1}=-1/8$ and $c_{2}=1/64$, expanding this solution in the
perturbative region $t/\sqrt{\alpha^{\prime}}\to\infty$, to match the
perturbative solution (9), we identify

$\displaystyle\lambda_{0}$ $\displaystyle=$ $\displaystyle
1/\beta^{2},\quad\lambda_{1}=0,\quad\lambda_{2}=\frac{11+1024c_{3}}{12},$
$\displaystyle\lambda_{3}$ $\displaystyle=$
$\displaystyle-\frac{4}{15}(13+2816c_{3}+1536c_{4})...$ (15)

It is obvious that every single term in the sums such as
$\sum_{n=1}^{\infty}\frac{\lambda_{n}}{1+\tau^{2n}}$ of the solution is non-
singular everywhere. But we want to point out that there could be a tiny
possibility that the summation is not non-singular for fine-tuned parameters
$\lambda_{n},n>0$ Gasperini:2023tus1 . If choosing $\lambda_{n\geq 1}=0$, the
solution reduces to

$\displaystyle H(t)$ $\displaystyle=$
$\displaystyle-\frac{\sqrt{2}}{\sqrt{\alpha^{\prime}}}\frac{\left(1-\tau^{2}\right)}{\left(1+\tau^{2}\right)^{3/2}},$
$\displaystyle\Phi(t)$ $\displaystyle=$
$\displaystyle-\frac{1}{2}\log\beta^{2}-\frac{1}{2}\log\left(1+\tau^{2}\right),$
$\displaystyle f(t)$ $\displaystyle=$
$\displaystyle-\frac{2\sqrt{2}d}{\sqrt{\alpha^{\prime}}}\frac{1}{\sqrt{1+\tau^{2}}},$
$\displaystyle g(t)$ $\displaystyle=$
$\displaystyle-\frac{2d}{\alpha^{\prime}}\frac{\tau^{2}}{\left(1+\tau^{2}\right)^{2}}.$
(16)

which is exactly the solution given in ref. Wang:2019kez .

##  III The general loop corrected solution

After constructing the non-perturbative non-singular $\alpha^{\prime}$
corrected solution, we want to find the corresponding loop corrected non-
perturbative non-singular solution. We do not know the form and coefficients
of the higher loop terms in the full perturbative action (1). For the FLRW
background (2), an effective loop corrected action is sketched as

$\displaystyle I_{\rm Loop}$ $\displaystyle=$ $\displaystyle\int
d^{d+1}x\sqrt{-g}e^{-2\phi}\left[R+4\left(\partial_{\mu}\phi\right)^{2}-V\left(e^{-\Phi\left(x\right)}\right)\right],$
(17) $\displaystyle=$ $\displaystyle\int
dte^{-\Phi}\big{[}-\dot{\Phi}+dH^{2}-V(e^{-\Phi})\big{]},$

where the $O(d,d)$ non-local dilaton is Gasperini:1992em ; Gasperini:2003pb ,

$e^{-\Phi\left(t\right)}=V_{d}\int
dt^{\prime}\left|\frac{d\left(2\phi\right)}{dt^{\prime}}\right|\sqrt{-g\left(t^{\prime}\right)}e^{-2\phi\left(t^{\prime}\right)}\delta\left(2\phi\left(t\right)-2\phi\left(t^{\prime}\right)\right)=V_{d}\sqrt{-g\left(t\right)}e^{-2\phi\left(t\right)}.$
(18)

The EOM is

$\displaystyle 2\ddot{\Phi}_{L}-2dH_{L}^{2}-\frac{\partial
V}{\partial\Phi_{L}}$ $\displaystyle=$ $\displaystyle 0,$
$\displaystyle\dot{\Phi}_{L}^{2}-dH_{L}^{2}-V$ $\displaystyle=$ $\displaystyle
0,$ $\displaystyle\dot{H}_{L}-H_{L}\dot{\Phi}_{L}$ $\displaystyle=$
$\displaystyle 0,$ (19)

where the subscribe $L$ indicates that the quantities belong to the loop
corrected theory. For an positive integer $n$ and arbitrary parameters
$m_{n}$, $\sigma_{n}$, a class of solutions of the above EOM was constructed
in refs Gasperini:2003pb ; Gasperini:2004ss ,

$\displaystyle\Phi^{(n)}_{L}(t)$ $\displaystyle=$
$\displaystyle\frac{1}{2n}\log\left(\frac{\sigma_{n}^{2n}}{1+\left(m_{n}t\right)^{2n}}\right),$
(20) $\displaystyle H^{(n)}_{L}(t)$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{d}}\frac{m_{n}}{\sigma_{n}}e^{\Phi^{(n)}_{L}\left(t\right)}=\frac{m_{n}}{\sqrt{d}}\left[\frac{1}{1+\left(m_{n}t\right)^{2n}}\right]^{1/2n}.$

with the potential

$V^{(n)}_{L}=\left(\frac{m_{n}}{\sigma_{n}}\right)^{2}e^{2\Phi^{(n)}_{L}\left(t\right)}\left[\left(1-\sigma_{n}^{-2n}e^{2n\Phi^{(n)}_{L}\left(t\right)}\right)^{\frac{2n-1}{n}}-1\right].$
(21)

As argued in refs Gasperini:2003pb ; Gasperini:2004ss , since the factor
$e^{\Phi}$ roughly represents the coupling constant, the integer $n$
effectively represents the loop number. The potential indicates the non-
perturbative contributions by the $n$th loop.

In ref. Wang:2019dcj , a map is found between the $\alpha^{\prime}$ corrected
theory and loop corrected theory,

$\displaystyle H_{L}$ $\displaystyle\leftrightarrow$ $\displaystyle
f\left(H_{\alpha^{\prime}}\right),$ $\displaystyle-V_{L}$
$\displaystyle\leftrightarrow$ $\displaystyle
g\left(H_{\alpha^{\prime}}\right)+df\left(H_{\alpha^{\prime}}\right)^{2}$
$\displaystyle\Phi_{L}$ $\displaystyle\leftrightarrow$
$\displaystyle\Phi_{\alpha^{\prime}}+\Phi_{0}.$ (22)

where $\Phi_{0}$ is a constant. Through this map, we thus can find a loop
corrected non-perturbative non-singular solution from our $\alpha^{\prime}$
corrected solution (14). Since $e^{n\Phi_{L}^{(n)}}$ indicates the
contribution from the $n$-th loop, we use eqn. (III) to express the solution
in terms of $e^{\Phi_{L}^{(n)}}$. Then, after some tedious calculation, we
obtain the loop corrected solution,

$\displaystyle\Phi_{L}(t)$ $\displaystyle=$
$\displaystyle\Phi_{L}^{(1)}+\sum_{n=1}^{\infty}e^{2n\Phi_{L}^{(n)}}$
$\displaystyle H_{L}(t)$ $\displaystyle=$
$\displaystyle\frac{m_{1}}{\sigma_{1}}\exp\left[\Phi_{L}^{(1)}+\sum_{n=1}^{\infty}e^{2n\Phi_{L}^{(n)}}\right]$
$\displaystyle V_{L}(\Phi_{L}^{(n)}(t))$ $\displaystyle=$
$\displaystyle\left(\dot{\Phi}_{L}^{(1)}+\sum_{n=1}^{\infty}2n\dot{\Phi}_{L}^{(n)}e^{2n\Phi_{L}^{(n)}}\right)^{2}-\frac{dm_{1}^{2}}{\sigma_{1}^{2}}\exp\left[2\Phi_{L}^{(1)}+2\sum_{n=1}^{\infty}e^{2n\Phi_{L}^{(n)}}\right],$
(23)

where we set $m_{n}=\sqrt{2d/\alpha^{\prime}}$ and $\sigma_{i}$’s are free
parameters to be determined by coefficients calculated from the effective low
energy action. Obviously, this solution is non-singular around the non-
perturbative region $t\sim 0$.

##  IV Conclusion

In this paper, based on Hohm-Zwiebach action, we first constructed a class of
general $\alpha^{\prime}$ corrected non-perturbative non-singular string
cosmology solutions. Currently, only the first two orders in $\alpha^{\prime}$
correction are available. Our solution matches these results in the
perturbative region as required. Once higher orders in $\alpha^{\prime}$
correction are available in the future, we can straightforwardly fix the
corresponding parameters in our solution to match the perturbative solutions.
There exists a phenomenological map between $\alpha^{\prime}$ corrected theory
and loop corrected theory given in ref. Wang:2019dcj . We used this map to
constructed the corresponding loop corrected solution, which is also non-
singular and non-perturbative. Since both solutions are non-singular
everywhere, the pre- and post-big-bangs are thus smoothly connected by them.

###### Acknowledgements.

Li Song is supported by Sichuan Science and Technology Program,
NO:2022YFG0317. Deyou Chen is supported by Tianfu talent plan and FXHU

## References

  * (1) G. Veneziano, “Scale factor duality for classical and quantum strings,” Phys. Lett. B 265, 287 (1991). doi:10.1016/0370-2693(91)90055-U
  * (2) A. Sen, “O(d) x O(d) symmetry of the space of cosmological solutions in string theory, scale factor duality and two-dimensional black holes,” Phys. Lett. B 271, 295 (1991). doi:10.1016/0370-2693(91)90090-D
  * (3) A. A. Tseytlin, “Duality and dilaton,” Mod. Phys. Lett. A 6, 1721 (1991). doi:10.1142/S021773239100186X
  * (4) A. A. Tseytlin and C. Vafa, “Elements of string cosmology,” Nucl. Phys. B 372, 443 (1992) doi:10.1016/0550-3213(92)90327-8 [hep-th/9109048].
  * (5) M. Gasperini, M. Maggiore and G. Veneziano, “Towards a nonsingular pre - big bang cosmology,” Nucl. Phys. B 494, 315 (1997) doi:10.1016/S0550-3213(97)00149-1 [hep-th/9611039].
  * (6) G. Veneziano, “String cosmology: The Pre - big bang scenario,” doi:10.1007/3-540-45334-2-12 [hep-th/0002094].
  * (7) M. Gasperini and G. Veneziano, “The Pre - big bang scenario in string cosmology,” Phys. Rept. 373, 1 (2003) doi:10.1016/S0370-1573(02)00389-7 [hep-th/0207130].
  * (8) M. Gasperini and G. Veneziano, “String Theory and Pre-big bang Cosmology,” Nuovo Cim. C 38, no. 5, 160 (2016) doi:10.1393/ncc/i2015-15160-8 [hep-th/0703055].
  * (9) M. Gasperini and G. Veneziano, “Pre - big bang in string cosmology,” Astropart. Phys. 1, 317 (1993) doi:10.1016/0927-6505(93)90017-8 [hep-th/9211021].
  * (10) M. Gasperini, M. Giovannini and G. Veneziano, “Perturbations in a nonsingular bouncing universe,” Phys. Lett. B 569, 113 (2003) doi:10.1016/j.physletb.2003.07.028 [hep-th/0306113].
  * (11) M. Gasperini, M. Giovannini and G. Veneziano, “Cosmological perturbations across a curvature bounce,” Nucl. Phys. B 694 (2004) 206 doi:10.1016/j.nuclphysb.2004.06.020 [hep-th/0401112].
  * (12) D. A. Easson, “Towards a stringy resolution of the cosmological singularity,” Phys. Rev. D 68, 043514 (2003) doi:10.1103/PhysRevD.68.043514 [hep-th/0304168].
  * (13) K. A. Meissner and G. Veneziano, “Symmetries of cosmological superstring vacua,” Phys. Lett. B 267, 33 (1991). doi:10.1016/0370-2693(91)90520-Z
  * (14) K. A. Meissner, “Symmetries of higher order string gravity actions,” Phys. Lett. B 392, 298 (1997) doi:10.1016/S0370-2693(96)01556-0 [hep-th/9610131].
  * (15) A. Sen, “Twisted black p-brane solutions in string theory,” Phys. Lett. B 274, 34 (1992) doi:10.1016/0370-2693(92)90300-S [hep-th/9108011].
  * (16) O. Hohm and B. Zwiebach, “T-duality Constraints on Higher Derivatives Revisited,” JHEP 1604, 101 (2016) doi:10.1007/JHEP04(2016)101 [arXiv:1510.00005 [hep-th]].
  * (17) O. Hohm and B. Zwiebach, “Duality Invariant Cosmology to all Orders in $\alpha^{\prime}$,” arXiv:1905.06963 [hep-th].
  * (18) O. Hohm and B. Zwiebach, “Non-perturbative de Sitter vacua via $\alpha^{\prime}$ corrections,” arXiv:1905.06583 [hep-th].
  * (19) C. Krishnan, “de Sitter, $\alpha^{\prime}$-Corrections and Duality Invariant Cosmology,” arXiv:1906.09257 [hep-th].
  * (20) P. Wang, H. Wu and H. Yang, “Are non-perturbative AdS vacua possible in bosonic string theory?,” accepted by Phys. Rev. D, arXiv:1906.09650 [hep-th].
  * (21) C. A. Núñez and F. E. Rost, “New non-perturbative de Sitter vacua in $\alpha^{\prime}$-complete cosmology,” JHEP 03, 007 (2021) doi:10.1007/JHEP03(2021)007 [arXiv:2011.10091 [hep-th]].
  * (22) P. Bieniek, J. Chojnacki, J. H. Kwapisz and K. A. Meissner, “Stability of the de-Sitter spacetime. The anisotropic case,” [arXiv:2301.06616 [hep-th]].
  * (23) P. Wang, H. Wu, H. Yang and S. Ying, “Non-singular string cosmology via $\alpha^{\prime}$ corrections,” arXiv:1909.00830 [hep-th].
  * (24) P. Wang, H. Wu, H. Yang and S. Ying, “Construct $\alpha^{\prime}$ corrected or loop corrected solutions without curvature singularities,” JHEP 01, 164 (2020) doi:10.1007/JHEP01(2020)164 [arXiv:1910.05808 [hep-th]].
  * (25) S. Ying, “Resolving naked singularities in $\alpha^{\prime}$-corrected string theory,” Eur. Phys. J. C 82, no.6, 523 (2022) doi:10.1140/epjc/s10052-022-10427-5 [arXiv:2112.03087 [hep-th]].
  * (26) S. Ying, “Two-dimensional regular string black hole via complete $\alpha^{\prime}$ corrections,” [arXiv:2212.03808 [hep-th]].
  * (27) F. R. Klinkhamer, “More on the regularized big bang singularity,” Phys. Rev. D 101, no.6, 064029 (2020) doi:10.1103/PhysRevD.101.064029 [arXiv:1907.06547 [gr-qc]].
  * (28) H. Bernardo, R. Brandenberger and G. Franzmann, “O$(d,d)$ covariant string cosmology to all orders in $\alpha^{\prime}$,” JHEP 02, 178 (2020) doi:10.1007/JHEP02(2020)178 [arXiv:1911.00088 [hep-th]].
  * (29) H. Bernardo and G. Franzmann, “$\alpha^{\prime}$-Cosmology: solutions and stability analysis,” JHEP 05, 073 (2020) doi:10.1007/JHEP05(2020)073 [arXiv:2002.09856 [hep-th]].
  * (30) J. Quintin, H. Bernardo and G. Franzmann, “Cosmology at the top of the $\alpha$’ tower,” JHEP 07, 149 (2021) doi:10.1007/JHEP07(2021)149 [arXiv:2105.01083 [hep-th]].
  * (31) T. Codina, O. Hohm and D. Marques, “General string cosmologies at order $\alpha$’3,” Phys. Rev. D 104, no.10, 106007 (2021) doi:10.1103/PhysRevD.104.106007 [arXiv:2107.00053 [hep-th]].
  * (32) S. Ying, “Three dimensional regular black string via loop corrections,” JHEP 03, 044 (2023) doi:10.1007/JHEP03(2023)044 [arXiv:2212.14785 [hep-th]].
  * (33) T. Codina, O. Hohm and B. Zwiebach, “2D Black Holes, Bianchi I Cosmologies, and $\alpha^{\prime}$,” [arXiv:2304.06763 [hep-th]].
  * (34) M. Gasperini and G. Veneziano, “Non-singular pre-big bang scenarios from all-order $\alpha^{\prime}$ corrections,” [arXiv:2305.00222 [hep-th]].
  * (35) We are indebted to the anonymous referee to help us clarify this confusion.