# Federated Traffic Synthesizing and Classification Using Generative
Adversarial Networks

1School of Electronic Inform. & Commun., Huazhong Univ. of Science &
Technology, China  2School of Artificial Intelligence, Xidian University,
Xi’an, China  3Pazhou Lab, Guangzhou, China  4Department of Electrical
Engineering, University of South Florida, FL  Chenxin Xu1, Rong Xia1, Yong
Xiao13,Yingyu Li1, Guangming Shi23, and Kwang-cheng Chen4

###### Abstract

With the fast growing demand on new services and applications as well as the
increasing awareness of data protection, traditional centralized traffic
classification approaches are facing unprecedented challenges. This paper
introduces a novel framework, Federated Generative Adversarial Networks and
Automatic Classification (FGAN-AC), which integrates decentralized data
synthesizing with traffic classification. FGAN-AC is able to synthesize and
classify multiple types of service data traffic from decentralized local
datasets without requiring a large volume of manually labeled dataset or
causing any data leakage. Two types of data synthesizing approaches have been
proposed and compared: computation-efficient FGAN (FGAN-I) and communication-
efficient FGAN (FGAN-II). The former only implements a single CNN model for
processing each local dataset and the later only requires coordination of
intermediate model training parameters. An automatic data classification and
model updating framework has been proposed to automatically identify unknown
traffic from the synthesized data samples and create new pseudo-labels for
model training. Numerical results show that our proposed framework has the
ability to synthesize highly mixed service data traffic and can significantly
improve the traffic classification performance compared to existing solutions.

##  I Introduction

Traffic classification and identification have been considered as fundamental
for a wide range of applications such as network anomaly detection and
identification, as well as resource slicing for different prioritized services
and network access control. Traditional traffic classification [1, 2, 3]
approaches are mostly centralized data processing approaches based on either
supervised learning which often relies on a large volume of high-quality
labeled datasets to train a model or clustering which divides data samples
based on some observable features.

With fast growing demands on innovative services and applications, the next
generation telecommunication technology is expected to support a plethora of
new services and applications[4, 5]. Traditional traffic classification
solutions are facing unprecedented challenges. More specifically, with new
services and applications being introduced more frequently than ever, it
becomes more difficult to collect a sufficient number of high-quality labeled
datasets for each emerging service in its early stage of development.
Clustering-based approaches suffer from limited accuracy and often incapable
of identifying new unknown services, especially when the total volume of
traffic data is relatively low. Furthermore, with the increasing awareness of
data protection and user privacy, centralized data processing solutions
relying on datasets uploaded to a single computer server will soon be
impossible to support future services with stringent privacy and data
protection requirements.

Federated learning (FL) is an emerging machine learning framework that enables
collaborative model training based on decentralized datasets [6, 7]. It has
been considered as one of key enabling technologies for data processing and
model construction based on decentralized datasets distributed across a large
networking system. In spite of merits, FL suffers from several known
limitations. In particular, traditional FL (e.g. FedAvg [7]) is still a
supervised learning approach and requires a large number of labeled datasets
for training the model. Moreover, FL is known to suffer from slow convergence
rate especially when the datasets are mostly unlabeled and heterogeneous (e.g.
non-IID).

In this paper, we focus on traffic synthesizing and classification across
decentralized datasets distributed throughout a large geographic area. A set
of local fog servers, each can access a local dataset (e.g. dataset collected
in a local sub-region), have been deployed for training a global traffic
classification model without requiring any transferring or data leakage of
local datasets. We propose Federated Generative Adversarial Networks and
Automatic Classification (FGAN-AC), a novel decentralized traffic synthesizing
and classification framework. FGAN-AC seamlessly integrate a federated
generative learning model called FGAN into an automatic classification
framework. In FGAN, two types of generative learning algorithms have been
introduced: computation-efficient FGAN (FGAN-I) and communication-efficient
FGAN (FGAN-II). The former algorithm FGAN-I deploys a single model
(discriminator) at each fog server to compare the synthetic data generated by
the global generator with its local dataset. The later FGAN-II algorithm
deploys a complete Generative Adversarial Networks (GANs) [8] consisting of
two models, a generator and a discriminator, at each fog server. The fog
servers will then coordinate their model training via intermediate parameters
exchanging which usually require much less communication overhead compared to
exchanging real or fake (synthetic) data. We show that both generative
learning algorithms create synthetic samples that capture the mixture of
distribution of all the decentralized datasets. Based on these synthetic data
samples, an automatic classification and model updating framework has been
proposed to automatically identify unknown traffic and assign pseudo-labels
for model training. Extensive simulations have been conducted to evaluate the
performance of the proposed framework. Our results show that FGAN-AC is able
to synthesize highly mixed service data traffic and can significantly improve
the traffic classification performance.

##  II Related Work

Federated Learning: Federated learning is an emerging distributed machine
learning solution enabling collaborative model training based on decentralized
datasets [9, 10, 11, 12]. It has been proved to be an very effective solution
for reducing communication overhead [9] and increasing data privacy [10].
Recent results suggest that combining FL and generative learning is possible
to create synthetic data samples that capture the mixture of distributions
associated with decentralized datasets [13, 14, 15]. For example, in [14], the
authors proposed a framework that could apply multiple discriminators and
generators to synthesize dataset with complex features.
Deep Learning Based Traffic Classification: Deep neural network is considered
as a promising solution for traffic classification [1, 2, 3, 16, 17]. In
particular, the authors in [17] introduced a novel solution with attention
mechanism that extracted features of the payload segments and output the
classification results through Softmax classification layer. The authors in
[18] proposed an classification model that can identify unknown classes of
data and then automatically updating its model. Different from the above
works, we integrate federated generative learning into a automatic
classification framework to create synthetic samples to assist the automatic
traffic classification. To the best of our knowledge, this is the first work
that exploits data synthesizing for improving automatic traffic classification
performance.

##  III System Model And Architecture Overview

###  III-A System Model

We consider a fog computing networking system that can access and process a
set of decentralized datasets. In particular, the system consists of the
following elements:

Local datasets consist of service data generated by users in different service
coverage areas. We assume there are no overlapping among different datasets
and each dataset consists of a limited number of types of service traffic.

Local fog servers are deployed at different service areas to process its local
datasets. We assume each fog server is associated with a local dataset. Due to
the privacy consideration, each dataset can only be accessed by the associated
fog server.

Global coordinator is the computing server connected to local fog servers
across all service areas. The coordinator cannot access any local dataset, but
can communicate with all the fog servers and process their uploaded
intermediate results.

###  III-B Architecture Overview

The main objective is to construct a joint training model based on the
decentralized datasets to synthesize and classify different service traffic
data. The joint training process will be coordinated by the global
coordinator. We propose a novel framework called FGAN-AC that can capture the
distributions of multiple decentralized datasets and perform global service
synthesizing and classification without resulting any data leakage. In the
FGAN-AC approach, we propose a three-step framework for automatic data
synthesizing and classification described as follows:

Figure 1: Architecture overview

Decentralized Service Synthesizing: In this step, a large volume of synthetic
service traffic will be created by the global coordinator to capture the mixed
distributions of traffic data from all local datasets. We borrow the idea from
federated learning to efficiently aggregate the models deployed and trained
over different fog servers. Two types of FGAN approaches are proposed to
coordinate among fog servers as shown in Fig. 2: computation-efficient FGAN,
labeled as FGAN-I, and communication-efficient FGAN, labeled as FGAN-II.

(a) FGAN-I

(b) FGAN-II

Figure 2: FGAN Architecture.

FGAN-I: In this approach, only a single model is deployed at each fog server
and the coordinator. More specifically, the coordinator will be implemented
with a generator to create synthetic data to be transmitted to each fog server
where a local discriminator is deployed to distinguish between the real
traffic data and the synthetic data received from the generator. The generator
in the coordinator will be compete with all the discriminators in fog servers
to generate a mixture of data samples related to the services in local
datasets.

FGAN-II: In this approach, a GANs model consisting of both a generator and a
discriminator will be deployed at each fog server as well as the coordinator.
In this case, all fog servers will coordinate their intermediate model
training parameters with each other via the coordinator. The coordinator will
first aggregate all the uploaded parameters and then broadcast the updated
ones to each fog server.

Global Service Classification: In this step, a global service classification
model will be trained based on the synthesized dataset. In particular, we
first adopt a deep clustering approach, named as Deep Embedded Clustering
(DEC) [19], to create pseudo-labels for high-quality synthesized service data
samples. Different from traditional clustering algorithms which heavily rely
on the original dataset, DEC further exploits the latent features of the input
data to help separate different data traffic. Secondly, we feed the above
pseudo-labeled dataset to a classifier located at the coordinator for
supervised model training. Finally, the global coordinator will broadcast the
well-trained classification model to all the connected fog servers for local
data classification and unknown traffic filtering. Trained with global mixed
service samples, the broadcasted classification model can not only distinguish
local dataset for each individual fog server, but also identify the potential
service data that only available in other datasets.

Autonomous Model Updating: In this step, a self-updating mechanism is adopted
to autonomously identify newly arrived unknown service traffics and assign new
pseudo-labels for them. And then those newly synthesized service samples will
be merged with the original dataset to re-train the classifier. After the
training process is completed, the global coordinator will broadcast the
updated classifier again for future traffic identification.

##  IV FGAN-AC Architecture

In this section, we present more detailed descriptions of the three main steps
of FGAN-AC.

###  IV-A Decentralized Service Synthesizing

As mentioned earlier, one main objective of this paper is to synthesize
sufficient service traffic samples that can capture the distribution of the
real service datasets located in multiple fog servers. In particular, we
utilize a popular deep generative neural network, GANs, to produce high-
quality samples. GANs consist of two neural networks referred to as generator
$G$ and discriminator $D$ that competitively play a two player min-max game.
The generator tries to fool the discriminator with synthetic samples mapped
from Gaussian noises, and the discriminator aims to differentiate the real
data samples from the fake ones. The payoff value of GANs can be formulated
as:

$\displaystyle\min_{G}\max_{D}V(G,D)=\mathbb{E}_{\boldsymbol{x}\sim
P_{data}}[\log D(\boldsymbol{x})]$ (1)
$\displaystyle+\mathbb{E}_{\boldsymbol{z}\sim
P_{z}}[\log\left(1-D(G(\boldsymbol{z}))\right)],$

where $\boldsymbol{x}$ denotes data samples drawn from real dataset,
$P_{data}$ is the data distribution of real dataset, and $\boldsymbol{z}$ is
drawn from Gaussian space $P_{z}$. In this step, we train a decentralized GANs
model over multiple datasets to efficiently capture the distribution of mixed
service traffic collected by different fog servers. In particular, we
introduce two types of FGAN approaches: computation-efficient FGAN-I and
communication-efficient FGAN-II.

####  IV-A1 FGAN-I

As shown in Fig. 2(a), in FGAN-I, a global generator $G$ located at the
coordinator is competing with $N$ decentralized discriminator
$\\{D_{1},D_{2},\dots,D_{N}\\}$, each associated with an exclusive dataset
$d_{n}$. We can write the pay-off value of FGAN-I as follows:

$\displaystyle\min_{G}V(G)$
$\displaystyle=\frac{1}{N}\sum_{n=1}^{N}\mathbb{E}_{\boldsymbol{z}\sim
P_{z}}[\log\left(1-D_{n}(G(\boldsymbol{z}))\right)],$ (2a)
$\displaystyle\max_{D_{n}}V(D_{n})$
$\displaystyle=\mathbb{E}_{\boldsymbol{x}\sim P_{d_{n}}}[\log
D_{n}(\boldsymbol{x})]$ $\displaystyle+\mathbb{E}_{\boldsymbol{z}\sim
P_{z}}[\log\left(1-D_{n}(G(\boldsymbol{z}))\right)],$ (2b)

where $P_{d_{n}}$ is the data distribution of real dataset $d_{n}$. In each
joint training round, $G$ first generates two batches of fake samples with
batch size $b$, denoted as $\boldsymbol{x}^{(d)}$ and $\boldsymbol{x}^{(g)}$,
and sends them to all the connected fog servers. $\boldsymbol{x}^{(d)}$ is
used for training the discriminator, and $\boldsymbol{x}^{(g)}$ is used for
the discriminator to calculate loss value of the generator. Secondly, each
discriminator $D_{n}$ deployed on a fog server draws a batch of samples
$\boldsymbol{x}^{(r)}$ from its local dataset $d_{n}$ and performs a $E$-step
updating by ascending the gradient of loss value $L_{disc}^{n}$. Thirdly, each
updated discriminator $D_{n}$ will utilize $\boldsymbol{x}^{(g)}$ to calculate
a loss value $L_{gen}^{n}$ for the generator. After receiving all the
feedbacks from $N$ fog servers, the global generator $G$ will also update by
descending the gradient of an aggregated loss value $L_{gen}$. We summarize
the training process of FGAN-I in Algorithm 1. The above mentioned loss
functions for model updating are formulated as follows:

$\displaystyle\mathop{L}\nolimits_{disc}^{n}$
$\displaystyle=\frac{1}{b}\sum\limits_{x\in\mathop{\boldsymbol{x}}\nolimits^{(r)}}{\log(\mathop{D}\nolimits_{n}(x))+\frac{1}{b}}\sum\limits_{x\in\mathop{\boldsymbol{x}}\nolimits^{(d)}}{\log(1-\mathop{D}\nolimits_{n}(x))},$
(3a) $\displaystyle L_{gen}^{n}$
$\displaystyle=\frac{1}{b}\sum\limits_{x\in\mathop{\boldsymbol{x}}\nolimits^{(g)}}{\log(1-D_{n}(x))},$
(3b) $\displaystyle L_{gen}$
$\displaystyle=\frac{1}{N}\sum\limits_{n=1}^{N}{L_{gen}^{n}}.$ (3c)

Algorithm 1 FGAN-I

Input: maximum global training round $I$, local training epoch $E$;
Output: global generator $G$ ;
for global training round $i\leq I$ do

  * Global generator broadcasts two batches of synthesized samples ${\boldsymbol{x}}^{(g)}$ and ${\boldsymbol{x}}^{(d)}$;

  * for each local discriminator parallel do

  * $L_{gen}^{n}\leftarrow{LocalUpdate}{({\boldsymbol{x}}^{(g)},{\boldsymbol{x}}^{(d)})}$

  * end for

  * Aggregate local loss values by Equation (3c)

  * Update generator by descending the gradient of the aggregated loss;

end for

Function $LocalUpdate$ ${({\boldsymbol{x}}^{(g)},{\boldsymbol{x}}^{(d)})}$
for local training round $e\leq E$ do

  * 1) Sample a batch of real data ${\boldsymbol{x}}^{(r)}$;

  * 2) Update discriminator by ascending the gradient of Equation (3a);

  * 3) Calculate the loss value of global generator by Equation (3b);

  * 4) Upload the loss value to global generator;

end for
return loss value of global generator

In FGAN-I, only a single component of GANs is deployed at each fog server and
coordinator, which significantly reduces the computation complexity. However,
a large amount of synthesized data samples need to be transmitted between the
global coordinator and each fog server in the training process. When the batch
size is large or the data sample is complex which is common in practice, the
communication overhead of FGAN-I will be extremely heavy.

####  IV-A2 FGAN-II

In order to avoid redundant data sample transmission, in this approach, both
the generator and the discriminator are deployed at each fog server as well as
the global coordinator as shown in Fig. 2(b). Different from FGAN-I, we borrow
the idea from federated learning [7] to efficiently aggregate information
obtained from multiple decentralized datasets.

Algorithm 2 FGAN-II

Input: local GANs with parameters $w_{n}$ and $\theta_{n}$, global GANs with
parameters $w$ and $\theta$, global training round $I$, number of local
training epoch $E$;
Output: $w$ and $\theta$;
for global training round $i\leq I$ do

  * for each local GANs parallel do

  * $(w_{n},\theta_{n})\leftarrow{LocalUpdate}{(w,\theta)}$;

  * end for

  * Aggregate received parameters by Equation (4);

end for
return $w$,$\theta$

Function LocalUpdate ($w,\theta$)
$w_{n}\leftarrow w$, $\theta_{n}\leftarrow\theta$;
for i=1 $\to$ E do

  * Update $\theta_{n}$ by ascending the gradient of Equation (5a);

  * Update $w_{n}$ by descending the gradient of Equation (5b);

end for
return $w_{n},\theta_{n}$

Firstly, in each global training round, the global GANs model located at the
coordinator broadcasts its latest model parameters, denoted as $w$ for
generator and $\theta$ for discriminator, to each fog server. Secondly, local
GANs models located at different fog servers update their parameters according
to the received $w$ and $\theta$, and then performs an $E$-step local model
updating by ascending the gradient of loss function $L_{D}^{n}$ and descending
the gradient of loss function $L_{G}^{n}$. After the local training process is
completed, local parameters denoted as $w_{n}$ and $\theta_{n}$ will be report
to the global coordinator. At last, the global coordinator aggregates all the
received local parameters by:

$\displaystyle
w=\frac{1}{N}\sum_{n=1}^{N}{\mathop{w}\nolimits_{n}},\quad\theta=\frac{1}{N}\sum_{n=1}^{N}{\mathop{\theta}\nolimits_{n}}.$
(4)

Following the notations in FGAN-I, $\boldsymbol{x}^{(r)}$ is a batch of real
service samples, $\boldsymbol{x}^{(g)}$ and $\boldsymbol{x}^{(d)}$ denotes a
batch of synthesized samples used for training generator and discriminator,
respectively. The loss functions of the discriminator and the generator at
each fog server $n$ can be formulated as:

$\displaystyle\mathop{L}\nolimits_{D}^{n}=\frac{1}{b}\sum_{x\in\mathop{\boldsymbol{x}}\nolimits^{(r)}}{\log(\mathop{D}\nolimits_{n}(x))}+\
\frac{1}{b}\sum_{x\in\mathop{\boldsymbol{x}}\nolimits^{(d)}}{\log(1-\mathop{D}\nolimits_{n}(x))},$
(5a)
$\displaystyle\mathop{L}\nolimits_{G}^{n}=\frac{1}{b}\sum_{x\in\mathop{\boldsymbol{x}}\nolimits^{(g)}}{\log(1-\mathop{D}\nolimits_{n}(x))}.$
(5b)

We present a more detailed description of FGAN-II in Algorithm 2.

In FGAN-II, instead of transmitting the synthesized data samples, we only
perform coordination on the intermediate training parameters that are
independent from the data dimension and training batch size. Unfortunately,
both fog servers and the coordinator need to allocate more computation
resources for the joint training process.

###  IV-B Global Service Classification

In this step, we establish a global service classifier based on the synthetic
samples produced by the generator at the global coordinator. Trained with
high-quality synthesized global mixed service traffic data across all the
decentralized local datasets, this classifier is able to identify different
services at each fog server. Two procedures are needed to generate high
accuracy global service classification: service sample labeling and global
classifier training. We present a step-by-step description in Algorithm 3.

####  IV-B1 Service Sample Labeling

In this procedure, we adopt a deep clustering method, referred to as DEC [19],
to create a pseudo-label for each sample in the synthesized dataset, denoted
as
$T=\\{\hat{\mathop{x}}_{1},\hat{\mathop{x}}_{2},\dots,\hat{\mathop{x}}_{n_{T}}\\}$.
In particular, we optimize a pre-trained encoder and cluster centroid by
minimizing the KL divergence between a target distribution $p_{ij}$ and a soft
assignment distribution $q_{ij}$ as follows:

$\displaystyle
L=KL(P||Q)=\sum\nolimits_{i}{\sum\nolimits_{j}{\mathop{p}\nolimits_{ij}\log\frac{{\mathop{p}\nolimits_{ij}}}{{\mathop{q}\nolimits_{ij}}}}}.$
(6)

Let $Z_{i}$ be the latent feature of $\hat{\mathop{x}}_{i}$ extracted by the
above mentioned encoder, and $\mathop{\mu}\nolimits_{j}$ be the centroid of
cluster $j$. Then, $q_{ij}$ can be formulated as:

$\displaystyle{q_{ij}}=\frac{{{\rm{1+(}}||{\rm{}}{Z_{i}}-{\mu_{j}}|{|^{2}}{)^{{\rm{-1}}}}}}{{\sum\limits_{j=1}^{k}{{{(1+(||{\rm{}}{Z_{i}}-{\mu_{j}}|{|^{2}}))}^{-1}}}}},$
(7)

where $k$ is number of clusters. In order to improve the cluster purity, the
target distribution $p_{ij}$ is formulated as first squaring the soft
assignment distribution and then normalizing it

$\displaystyle\mathop{p}\nolimits_{ij}=\frac{{{{\mathop{\mathop{q}\nolimits_{ij}}\nolimits^{2}}\mathord{\left/{\vphantom{{\mathop{\mathop{q}\nolimits_{ij}}\nolimits^{2}}{\mathop{f}\nolimits_{j}}}}\right.\kern-1.2pt}{\mathop{f}\nolimits_{j}}}}}{{{{\sum\nolimits_{j}{\mathop{\mathop{q}\nolimits_{ij}}\nolimits^{2}}}\mathord{\left/{\vphantom{{\sum\nolimits_{j}{\mathop{\mathop{q}\nolimits_{ij}}\nolimits^{2}}}{\mathop{f}\nolimits_{j}}}}\right.\kern-1.2pt}{\mathop{f}\nolimits_{j}}}}},\vspace{-0.5cm}$
(8)

where $\mathop{f}\nolimits_{j}=\sum\nolimits_{i}{\mathop{q}\nolimits_{ij}}$ is
the soft clustering probability. In this way, data points
$\\{Z_{1},Z_{2},\cdot,Z_{n_{T}}\\}$ can be assigned with a higher confidence.
After the training process is completed, all the samples
$\hat{\mathop{x}}_{i}$ in the synthesized dataset $T$ will be labeled via the
optimal solution of maximizing $q_{ij}$.

To find the optimal number of clusters, Bayesian information criterion (BIC)
is also adopted in our algorithm. For each possible cluster number $k$, we
calculate a BIC value for clustering performance evaluation by:

$\displaystyle BIC_{k}=n_{T}\times\ln(\frac{R}{n_{T}})+k\times\ln(n_{T}),$ (9)
$\displaystyle
R=\sum\limits_{j=1}^{k}{\sum\limits_{\mathop{Z}\nolimits_{i}\in\mathop{\mu}\nolimits_{j}}{\sqrt{\mathop{(\mathop{Z}\nolimits_{i}-\mathop{\mu}\nolimits_{j})}\nolimits^{2}}}},$
(10)

where $R$ is the sum of all the Euclidean distances between latent features
$Z_{i}$ and its corresponding cluster centroid $\mu_{j}$, $n_{T}$ is the
number of samples in the synthesized dataset $T$ and optimal number of
cluster, denoted as $k^{*}$, will be found by first calculating BIC values
with all possible $k\in\\{1,2,...,K_{max}\\}$, then choosing the $k$ whose BIC
value decreasing the most.

Algorithm 3 Autonomous Service Classification

Input: synthesized dataset
$T=\\{\hat{\mathop{x}}_{1},\hat{\mathop{x}}_{2},\dots,\hat{\mathop{x}}_{n_{T}}\\}$;
maximum number of clustering $K_{max}$; stopping threshold $\delta$; maximum
iterations $I_{max}$;
Output: global classifier $C$;
for $k=\\{1,2,...,K_{max}\\}$ do

  * for iteration round $i\leq I_{max}$ do

  * 1) initialize encoder and cluster centroid $\mu_{j}$;

  * 2) mapping $\hat{\mathop{x}}_{i}$ to $Z_{i}$ with the encoder;

  * 3) compute the distance between $Z_{i}$ and $\mu_{j}$ by equation (7);

  * 4) label each $Z_{i}$ with the closest cluster index;

  * 5) save labeled dataset as:

  * $T_{new}=\\{(\hat{x}_{1},y_{1}),(\hat{x}_{2},y_{2}),\dots,(\hat{x}_{n_{T}},y_{n_{T}})\\}$;

  * 6) optimize encoder and $\mu$ according the clustering loss in equation (6);

  * 7) re-calculate $y$ with the updated clustering model;

  * 8) if sum$(y_{old}\neq y)/n_{T}\leq\delta$ then

  * Stop training;

  * end for

  * Calculate ${BIC}_{k}$ value by equation (9);

  * $\Delta BIC$=$BIC_{k}$-$BIC_{k-1}$;

end for
Find the optimal clustering model with maximum $\Delta BIC$ value;
Calculate assignment probability $q_{ij}$;
Label $\hat{x}_{i}$ with $\mathop{\arg\max}_{j}q_{ij}$;
Train service classifier $C$ with $T_{new}$;
Broadcast $C$ to all the fog servers;

####  IV-B2 Global Classifier Training

After autonomous service sample labeling, the original synthesized dataset $T$
will update to
$T_{new}=\\{(\hat{x}_{1},y_{1}),(\hat{x}_{2},y_{2}),\dots,(\hat{x}_{n_{T}},y_{n_{T}})\\}$,
where $y_{i}$ is the pseudo-label of $\hat{x}_{i}$. Then, this labeled dataset
will be utilized for training a global service classifier $C$. When the model
converges, this well-trained classifier will be broadcasted to all the
connected fog servers for local service identification.

###  IV-C Autonomous Model Updating

In this step, we introduce a self-updating scheme to automatically update both
the synthesizing and the classification model when new types of service
traffic arrive. The proposed self-updating scheme consists of two procedures
described as follows:

####  IV-C1 Unknown Service Filtering

For a given service traffic sample, the well-trained service classifier $C$
can output a vector indicating the probabilities of it belongs to the known
service classes. We can monitor whether a new type of service emerges by
periodically feeding a bulk of real service traffic data to the classifier
$C$. Let $\bf o$ be the multi-dimensional output of classifier $C$ when
feeding with a real service sample $s$, and we use $o^{*}=max(\bf o)$ to
denote the confidence score of $s$. When $s$ belong to a newly appeared
service type for all the connected fog servers, $o^{*}$ will be relatively
small, indicating the well-trained classifier $C$ cannot assign this sample to
any known service type with a high probability. Thus, we introduce a threshold
value $\alpha$ to determine whether a traffic sample belongs to a known
service type or not. That is, samples with $o^{*}<\alpha$ will be treated as
unknown service types.

####  IV-C2 Data Synthesizing and Classifier Updating

Once the new type of service is identified in the above procedure, FGAN will
be trained again with the updated datasets, which contain both the unknown
service samples and the original dataset. In this way, a new mixture of
traffic samples will be synthesized which capturing the updated services data
distribution. Similarly, the newly synthesized dataset will first be labeled
with pseudo-labels, and then used for classifier training. Finally, the re-
trained classifier which is able to distinguish more diversified services will
be broadcasted to each fog server for service classification again.

##  V  Performance Evaluation

###  V-A Dataset Description and Environmental Setup

To evaluate the performance of our proposed framework on both service data
synthesizing and classification, we conduct extensive simulations based on a
real-world dataset “ISCX VPN-nonVPN dataset” (ISCXVPN2016) which consisting of
rich diversities of traffic samples including Emails, Facebook, SCP, etc. We
consider data samples associated with 10 servers among which 8 services are
considered as the known traffic classes and 2 services are assumed to be
unknown services as shown in Table I. In our simulations, data samples of both
known and unknown services are uniformly randomly allocated to fog servers.

All the simulations are conducted on a workstation with Intel(R) Core(TM)
i5-8500<EMAIL_ADDRESS>16.0 GB RAM@2133 MHz, 2 TB HD and four NVIDIA Corporation
GP102 [TITAN X] GPUs and implemented with Pytorch library.

TABLE I: Detailed Information of the Dataset for Evaluation | Application
---|---
Existing classes | $Email$ | $Facebook$
$Netflix$ | $SFTP$
$Skype$ | $Vimeo$
$SCP$ | $Twitter$
Unknown classes | $Youtube^{*}$ | $VOIPbuster^{*}$

###  V-B Evaluation Results

To evaluate and compare the data synthesizing performance of FGAN-I and FGAN-
II, we measure their communication overheads using the numbers of transmitted
data bits and computational loads using the running time per global training
round in Table II . We also use Maximum Mean Discrepancy (MMD) to evaluate the
quality of samples synthesized by two FGAN methods which can be calculated by:

$\displaystyle MMD^{2}\left(P_{r},P_{g}\right)=$ (11)
$\displaystyle\mathop{\mathbb{E}}\limits_{x_{r},x_{r}^{\prime}\sim
P_{r},x_{g},x_{g}^{\prime}\sim P_{g}}$
$\displaystyle\left[g\left(x_{r},x_{r}^{\prime}\right)-2g\left(x_{r},x_{g}\right)+g\left(x_{g},x_{g}^{\prime}\right)\right],$

where $x_{r}$ and $x_{r}^{\prime}$ are real samples drawn from real data
distribution $P_{r}$, $x_{g}$ and $x_{g}^{\prime}$ are synthesized samples
drawn from fake data distribution $P_{g}$. $g(\cdot)$ represents a kernel
function that maps the sample space to the Hilbert space. It can be observed
that the communication overhead of FGAN-I is proportional to the batch size
$b$, while the communication overhead of FGAN-II is mainly determined by the
size of the model parameters. Compared to FGAN-II, FGAN-I consumes less
running time per training round. With the number of training rounds increases,
both FGAN-I and FGAN-II can obtain lower MMD scores which means that the
quality of synthesized sample is improved.

TABLE II: Evaluation of FGAN-I and FGAN-II $I$ | $b$ | $N$ | MMD | Com.overhead (MB) | Time per round (s)
---|---|---|---|---|---
FGAN-I | FGAN-II | FGAN-I | FGAN-II | FGAN-I | FGAN-II
5 | 64 | 30 | 2.25 | 1.94 | 6.3 | 4.08 | 651.32 | 1571.77
10 | 64 | 30 | 0.67 | 1.45 | 6.3 | 4.08 | 651.32 | 1571.77
20 | 64 | 30 | 0.26 | 0.56 | 6.3 | 4.08 | 651.32 | 1571.77
5 | 128 | 30 | 0.96 | 0.96 | 12.5 | 4.08 | 347.41 | 1051.89
10 | 128 | 30 | 0.83 | 0.83 | 12.5 | 4.08 | 347.41 | 1051.89
20 | 128 | 30 | 0.35 | 0.38 | 12.5 | 4.08 | 347.41 | 1051.89
5 | 128 | 50 | 1.07 | 1.33 | 12.5 | 4.08 | 405.74 | 1288.84
10 | 128 | 50 | 0.79 | 0.62 | 12.5 | 4.08 | 405.74 | 1288.84
20 | 128 | 50 | 0.34 | 0.43 | 12.5 | 4.08 | 405.74 | 1288.84
TABLE III: Performance Evaluation Results Of The Classifiers | Recall | Precision | F1 score
---|---|---|---
MLP | original | 0.3746 | 0.3787 | 0.3725
updated | 0.3219 | 0.3273 | 0.3298
1D-CNN | original | 0.4128 | 0.4181 | 0.4135
updated | 0.3767 | 0.3748 | 0.3654
2D-CNN | original | 0.4584 | 0.4564 | 0.4627
updated | 0.4289 | 0.4263 | 0.4257

To evaluate the performance of classification , we consider three evaluation
metrics: Recall, model precision and F1 score, we consider two scenaries for
each solution: (1) original: only 8 known services have been evaluated and
classified, and (2) updated: total 10 services including 8 known and 2 unknown
services have been evaluated. We use TP to denote the true positive decision
that assigns two similar samples to the same cluster, and TN to denote the
true negative decision that assigns two different samples to different
clusters. We name the possible erroneous outputs of a classifier as false
positive (FP) decision that assigns two different samples to the same cluster
and false negative (FN) decision that assigns two similar packets to different
clusters. We can then calculate those evaluation metrics as follows:

$\displaystyle R=\frac{{TP}}{{TP+FN}},\quad P=\frac{{TP}}{{TP+FP}},\quad
F1=\frac{{2P\times R}}{{P+R}}$ (12)

We considere three network architectures for classifier: Multilayer Perceptron
(MLP), 1D-CNN and 2D-CNN. Note that we resize the input data to 1 $\times$
1600 for both MLP and 1D-CNN, and 40 $\times$ 40 for 2D-CNN. It can be
observed that the 2D-CNN-based classifier offers the best performance. The
MLP-based classifier has the lowest classification accuracy among three
solutions which is around 37%. We can also observe that compared to the
original classifier, the performance of the updated classifier also degrades,
which may be attributed to two aspects: some unknown service samples are not
detected, and some errors occured in the clustering process for the unknown
service types, to slightly worsen the subsequent classification.

Figure 3: Classification results for unknown classes(2D-CNN)

We present the classification result of the unknown services in Figure 3 which
we present a confusion matrix to show the detailed classification results of
two unknown service traffics. The confusion matrix calculates the
classification accuracy by comparing the actual class of each data sample with
the estimated one. We can observe that the accuracy for Youtube has reached
roughly 42.71%, while the accuracy for VOIPbuster is around 39.79%.

##  VI Conclusion

In this paper, we have proposed a novel framework that not only synthesizes
the mixture of distribution of decentralized dataset but also classifies
multiple unknown classes of data traffic. Furthermore, we have compared two
types of FGAN approaches and propose an automatic traffic classification
framework to autonomous classify unknown services and assign new pseudo-
labels. Extensive simulations have been conducted and the numerical results
demonstrate that our proposed framework is able to synthesize high-quality
mixed service data traffic as well as significantly improve the service
classification performance.

## Acknowledgment

This work was supported in part by the National Natural Science Foundation of
China under Grants 62071193 and 61632019, the Key R & D Program of Hubei
Province of China under Grant 2020BAA002, and China Postdoctoral Science
Foundation under Grant 2020M672357.

## References

  * [1] P. Wang _et al._ , “Datanet: Deep learning based encrypted network traffic classification in sdn home gateway,” _IEEE Access_ , vol. 6, pp. 55 380–55 391, Sep. 2018.
  * [2] M. Lotfollahi _et al._ , “Deep packet: A novel approach for encrypted traffic classification using deep learning,” May. 2019.
  * [3] I. Masi _et al._ , “Learning pose-aware models for pose-invariant face recognition in the wild,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 41, no. 2, pp. 379–393, Feb. 2019.
  * [4] Y. Xiao, G. Shi, Y. Li, W. Saad, and H. V. Poor, “Toward self-learning edge intelligence in 6g,” _IEEE Communications Magazine_ , vol. 58, no. 12, pp. 34–40, Dec. 2020.
  * [5] Y. Xiao and M. Krunz, “Distributed optimization for energy-efficient fog computing in the tactile Internet,” _IEEE J. Sel. Area Commun._ , vol. 36, no. 11, pp. 2390–2400, Nov. 2018.
  * [6] Augenstein _et al._ , “Generative models for effective ml on private, decentralized datasets,” _arXiv preprint arXiv:1911.06679_ , 2019.
  * [7] B. McMahan _et al._ , “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in _The Proceedings of AISTATS_ , vol. 54, Fort Lauderdale, FL, Apr. 2017, pp. 1273–1282.
  * [8] Goodfellow _et al._ , “Generative adversarial networks,” _Advances in Neural Information Processing Systems_ , vol. 3, pp. 2672–2680, Mar. 2014.
  * [9] J. Konecný _et al._ , “Federated learning: Strategies for improving communication efficiency,” _NIPS_ , vol. abs/1610.05492, Mon. 2016.
  * [10] Bonawitz _et al._ , “Practical secure aggregation for privacy-preserving machine learning,” Oct. 2017, pp. 1175–1191.
  * [11] W. Y. B. Lim _et al._ , “Federated learning in mobile edge networks: A comprehensive survey,” _IEEE Communications Surveys Tutorials_ , vol. 22, no. 3, pp. 2031–2063, Sep. 2020.
  * [12] Wen _et al._ , “A unified federated learning framework for wireless communications: towards privacy, efficiency, and security,” Toronto, ON, Canada, Jul. 2020, pp. 653–658.
  * [13] C. Hardy _et al._ , “Md-gan: Multi-discriminator generative adversarial networks for distributed datasets,” in _2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)_ , United States,Epub, Nov. 2019, pp. 866–877.
  * [14] Rasouli _et al._ , “Fedgan: Federated generative adversarial networks for distributed data,” _arXiv preprint arXiv:2006.07228_ , Jun. 2020.
  * [15] D. J. Im _et al._ , “Generative adversarial parallelization,” _arXiv e-prints_ , p. arXiv:1612.04021, Dec. 2016.
  * [16] S. Deena _et al._ , “Recurrent neural network language model adaptation for multi-genre broadcast speech recognition and alignment,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 27, no. 3, pp. 572–582, Sep. 2019.
  * [17] Li _et al._ , “Byte segment neural network for network traffic classification,” in _2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS)_ , 2018, pp. 1–10.
  * [18] J. Zhang _et al._ , “Autonomous unknown-application filtering and labeling for dl-based traffic classifier update,” in _IEEE INFOCOM 2020_ , Toronto, Jul. 2020, pp. 397–405.
  * [19] J. Xie _et al._ , “Unsupervised deep embedding for clustering analysis,” in _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, June_ , vol. 48, pp. 478–487.


# Optimal Transport for Change Detection on LiDAR Point Clouds

###### Abstract

Unsupervised change detection between airborne LiDAR data points, taken at
separate times over the same location, can be difficult due to unmatching
spatial support and noise from the acquisition system. Most current approaches
to detect changes in point clouds rely heavily on the computation of Digital
Elevation Models (DEM) images and supervised methods. Obtaining a DEM leads to
LiDAR informational loss due to pixelisation, and supervision requires large
amounts of labelled data often unavailable in real-world scenarios. We propose
an unsupervised approach based on the computation of the transport of 3D LiDAR
points over two temporal supports. The method is based on unbalanced optimal
transport and can be generalised to any change detection problem with LiDAR
data. We apply our approach to publicly available datasets for monitoring
urban sprawling in various noise and resolution configurations that mimic
several sensors used in practice. Our method allows for unsupervised multi-
class classification and outperforms the previous state-of-the-art
unsupervised approaches by a significant margin.

Index Terms—  Change detection, LIDAR, Point clouds, Optimal transport,
Building change detection.

##  1 Introduction

Detecting changes occurring in multi-temporal remote sensing data plays a
crucial role in monitoring several aspects of real life, such as disasters,
deforestation, and urban planning [1, 2]. In the latter context, identifying
both newly built and demolished buildings is essential to help landscape and
city managers to promote sustainable development [3, 4]. While the use of
airborne LiDAR point clouds has become widespread in urban change detection
[5, 6], the most common approaches require the transformation of a point cloud
into a regular grid of interpolated height measurements, i.e. Digital
Elevation Model (DEM) [7, 8, 9]. However, the DEM’s interpolation step causes
an information loss related to the height of the objects, affecting the
detection capability of building changes, where the high resolution of LiDAR
point clouds in the third dimension would be the most beneficial.

Notwithstanding recent attempts to detect changes directly on 3D point clouds
using either a semantic segmentation processing step [10, 11] or developing a
novel Siamese network based on a Kernel Point Convolution (KPC) [12], all of
which supervised, hence require a fully labelled training dataset. However,
the creation of a training dataset for change detection on a bi-temporal pair
of 3D point clouds requires manually labelling millions of points to take into
account both the difference in the spatial resolution (i.e., the number of
points per square meter) and the lack of registration between the two point
clouds of each pair.

Fig. 1: Method pipeline with chunk splitting and OT change detection applied
to chunk $i$.

These limitations call for developing unsupervised methods for identifying
change directly on 3D point clouds. To the best of our knowledge, based on the
survey papers [5, 13], there exists only one unsupervised approach able to
identify both positive and negative changes (new and demolished buildings,
respectively) directly on 3D point clouds, namely the Multiscale Model to
model Cloud Comparison (M3C2) [14], which is a distance-based change detection
method. Specifically, each point cloud of a bi-temporal pair is stored in an
octree structure, a recursive subdivision of the 3D space, to ensure a fast
identification of the nearest neighbours of each point. The identification of
changes is performed by thresholding the Hausdorff distance between each point
of the first cloud and each neighbour’s points of the second cloud (i.e., all
the points contained in the corresponding homologous cell of the octree)
laying on a local normal surface. Despite the M3C2 being capable of quickly
processing bi-temporal pairs with millions of points, it is sensitive to the
direction and amplitude of changes [13], which in turn affect the change
detection performance.

Motivated by the previous arguments, we introduce a principled change
detection pipeline based on Optimal Transport (OT), capable of distinguishing
between newly built and demolished buildings. OT theory provides us with a
methodology to project the first point cloud on the support of the second one
through the so-called displacement interpolation [15] by considering both
clouds as two uniform probability distributions. The changes are then computed
point-wise between the projected and the second cloud. OT theory does not
require that the two clouds have the same point density and is insensitive to
the direction and amplitude of changes. Courty et al. [16] proposed to use OT
to identify changes occurred on a coastal cliff in a pair of LiDAR point
clouds. However, their algorithm requires the conservation of mass between the
two point clouds, which models them as two probability distributions and
detects only change, irrespective of it being a positive or negative change.
We argue that this shortcoming affects the change detection performance. To
address this limitation, we propose to use unbalanced optimal transport [17]
to cope with the creation and destruction of mass related to building changes
occurring on a bi-temporal pair of LiDAR point clouds. We demonstrate the
efficacy of our approach on the only publicly available airborne LiDAR dataset
for change detection [5] by showing superior performance over the M3C2 and the
OT-based method introduced in [16].

##  2 Methodology

We aim to detect the changes occurring on a bi-temporal pair of airborne LiDAR
point clouds, namely $pc_{0}$ and $pc_{1}$ collected at time $t_{0}$ and
$t_{1}>t_{0}$ over the same geographical area. In this setting, detecting
change is solved by projecting the support of
$\mathcal{X}_{0}\in\mathbb{R}^{3}$ onto the support of
$\mathcal{X}_{1}\in\mathbb{R}^{3}$, i.e., matching the probability
distributions $\mu_{0}$ and $\mu_{1}$ modelling $pc_{0}$ and $pc_{1}$
respectively. After this projection step, changes are computed as the
pointwise distance between the points of the projected space, namely
$\hat{\mathcal{X}_{1}}$, and the space $\mathcal{X}_{1}$.

An effective way to match two probability distributions is addressed by
Optimal Transport (OT) theory [18, 19, 20] that enables to transport
$\mathcal{X}_{0}$ onto $\mathcal{X}_{1}$ following a least effort principle,
i.e. a transport called OT plan, that is optimal given a cost metric $C$ on
$(\mathcal{X}_{0}\times\mathcal{X}_{1}$). The problem of computing the OT plan
$\hat{P}$ is defined as $\hat{P}=\arg\min_{P\in\Pi}H(P)$ s.t.
$H(P)=<P,C>+\lambda R(P)$ and $\Pi=\\{P\in\mathbb{R}^{n_{0}\times
n_{1}}|\,P1_{n_{1}}=\mu_{0},P^{T}1_{n_{0}}=\mu_{1}\\}$, where $1_{l}$ is the
$1$-dimensional vector of ones, $<>$ is the Frobenius dot product, $R(P)$ is
the entropic regularisation term and $\lambda$ the associated scalar. To
detect a change in point cloud pairs, Courty et al. [16] proposed to compute
the projection of $\mathcal{X}_{0}$ onto $\mathcal{X}_{1}$ using the
barycenter projection [18] defined as
$\hat{\mathcal{X}_{1}}=diag(\hat{P}^{T}1_{n_{0}})^{-1}\mathcal{X}_{0}$, where
the OT plan $\hat{P}$ minimises $H$.

We advocate that the use of unbalanced OT [17] will increase the performance
because it is more effective to cope with the creation and destruction of mass
related to building changes than the previous version [16]. The unbalanced
version adds another regularisation term to $H$:
$R_{u}(P)=KL(P1_{n_{1}},[1/n_{0}]_{n_{0}})$ that penalises mass conservation,
which is not valid when the matching between the two distributions
$\mu_{1},\mu_{2}$ (modelling $pc_{0},pc_{1}$) is not surjective [21, 22] as in
the case of urban changes.

Dataset | M3C2 | OT | UOT
---|---|---|---
Low res - low noise | 0.2987 | 0.2702 | 0.4065
High res - low noise | 0.5373 | 0.3421 | 0.5520
Low res - high noise | 0.3872 | 0.2694 | 0.3926
Photogrammetry | 0.3501 | 0.2849 | 0.3989
MultiSensor | 0.3778 | 0.3036 | 0.4817
Table 1: Intersection over union benchmark with state-of-the-art on the five
datasets for building change detection.

##  3 Experimental Results

Fig. 2: Comparison between OT [16] and our method on two point clouds
($pc_{0}$, $pc_{1}$) with a different point density. Unbalanced OT detected
changes which are not properly identified by OT, such as the demolished
building (bottom right of the GT).

In this section, we evaluate our change detection pipeline on the only
publicly available dataset for building change detection [5], which contains
bi-temporal pair of airborne LiDAR point clouds. This dataset is composed of
five sub-datasets characterised by different spatial resolution and
acquisition noise levels: from high resolution with low noise to low
resolution with noisy point clouds. Fig. 1 shows our pipeline, composed of the
pre-processing step, which divides each point cloud into chunks, and the
change detection module that first computes the barycenter projection using
unbalanced OT and, subsequently, the pointwise distance. Dividing the point
cloud into chunks is necessary because they usually contain millions of points
which are too large for OT computation due to the quadratic complexity [20].
We compared the unbalanced OT with both the previous OT version [16] and the
M3C2 method [14] on the five sub-datasets. Following the experimental setting
presented in [5], we used empirical thresholds for these three methods to
identify points in $pc_{1}$ that changed with respect to $pc_{0}$ by comparing
if the pointwise distance exceeds the threshold. The threshold is computed by
trying several values and systematically selecting the best one for each sub-
dataset as this is reported in similar studies for a fair comparison [5].
Table 1 shows that our pipeline, using unbalanced OT, outperforms the other
two state-of-the-art methods and is robust to the resolution level and input
noise. In particular, we demonstrated experimentally (quantitatively in Table
1, and qualitatively in Figure 2) our related hypothesis, which is that
unbalanced optimal transport handles the creation and destruction of mass
related to building changes more efficiently, while the (balanced) OT
algorithm [16] is less effective in all the five sub-datasets.

Computationally speaking, a cloud containing millions of points is too large
for optimal transport, which has quadratic time complexity. To compute the
optimal transport plan for each chunk on a $16$ GB GPU, we fix the upper limit
for a cloud point to $30\,000$ points. For example, this requirement will slit
the largest dataset, i.e., high resolution and low noise, into $162$ chunks.
As the data can be noisy and non-uniformly distributed over the spatial
dimensions, we cannot ensure that each chunk has the same number of points.
The chunks computed from the largest dataset range from a minimum size of
$8\,072$ points to a maximum of $29\,346$ with a median value of $24\,276$
points. The memory footprint on a GPU ranges from $3.3$ to $9.7$ GB with a
median of $8.8$ GB, and the computation time ranges from $4.8$ to $10.7$
seconds with a median of $7.94$ seconds. The entire pipeline is fully
reproducible on HPC111https://github.com/MarcoFiorucci/OT-4-change-detection.

##  4 Conclusions

We introduced an unsupervised change detection pipeline based on unbalanced
OT, which provides us with a general method to identify changes directly on
LiDAR point clouds. We successfully validated our framework showing that it
surpasses two state-of-the-art methods for identifying both newly built and
demolished buildings. We believe that OT can effectively address different
problems of interest in the remote sensing community, such as flood and fire
detection, climate changes and cultural heritage. This work paves the way for
a principled method for change detection on different remote sensing non-
registered time series by combining OT theory with image processing and
machine learning.

### Acknowledgement

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 101027956.

## References

  * [1] A. Asokan and J. Anitha,  “Change detection techniques for remote sensing applications: a survey.,”  Earth Science Informatics, vol. 12, 2019.
  * [2] Wenzhong Shi, Min Zhang, Rui Zhang, Shanxiong Chen, and Zhao Zhan,  “Change detection based on artificial intelligence: State-of-the-art and challenges,”  Remote Sensing, vol. 12, no. 10, 2020.
  * [3] Ryan Reynolds, Lu Liang, XueCao Li, and John Dennis,  “Monitoring annual urban changes in a rapidly growing portion of northwest arkansas with a 20-year landsat record,”  Remote Sensing, vol. 9, no. 1, 2017.
  * [4] Mustafa Erdogan and Altan Yilmaz,  “Detection of building damage caused by van earthquake using image and digital surface model (dsm) difference,”  International Journal of Remote Sensing, vol. 40, no. 10, pp. 3772–3786, 2019.
  * [5] Iris de Gélis, Sébastien Lefèvre, and Thomas Corpetti,  “Change detection in urban point clouds: An experimental comparison with simulated 3d datasets,”  Remote Sensing, vol. 13, no. 13, 2021.
  * [6] Uwe Stilla and Yusheng Xu,  “Change detection of urban objects using 3d point clouds: A review,”  ISPRS Journal of Photogrammetry and Remote Sensing, vol. 197, pp. 228–255, 2023.
  * [7] Tuong Thuy Vu, M. Matsuoka, and F. Yamazaki,  “Lidar-based change detection of buildings in dense urban areas,”  in IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium, 2004, vol. 5, pp. 3413–3416 vol.5.
  * [8] K. Zhou, R. Lindenbergh, B. Gorte, and S. Zlatanova,  “Lidar-guided dense matching for detecting changes and updating of buildings in airborne lidar data,”  ISPRS Journal of Photogrammetry and Remote Sensing, vol. 162, pp. 200–213, 2020.
  * [9] Xuzhe Lyu, Ming Hao, and Wenzhong Shi,  “Building change detection using a shape context similarity model for lidar data,”  ISPRS International Journal of Geo-Information, vol. 9, no. 11, 2020\.
  * [10] Sudan Xu, George Vosselman, and Sander Oude Elberink,  “Detection and classification of changes in buildings from airborne laser scanning data,”  Remote sensing, vol. 7, no. 12, pp. 17051–17076, 2015.
  * [11] Chenguang Dai, Zhenchao Zhang, and Dong Lin,  “An object-based bidirectional method for integrated building extraction and change detection between multimodal point clouds,”  Remote sensing, vol. 12, no. 10, 2020.
  * [12] Iris de Gélis, Sébastien Lefèvre, and Thomas Corpetti,  “Siamese kpconv: 3d multiple change detection from raw point clouds using deep learning,”  ISPRS Journal of Photogrammetry and Remote Sensing, vol. 197, pp. 274–291, 2023.
  * [13] Abderrazzaq Kharroubi, Florent Poux, Zouhair Ballouch, Rafika Hajji, and Roland Billen,  “Three dimensional change detection using point clouds: A review,”  Geomatics, vol. 2, no. 4, pp. 457–485, 2022.
  * [14] Dimitri Lague, Nicolas Brodu, and Jérôme Leroux,  “Accurate 3d comparison of complex topography with terrestrial laser scanner: Application to the rangitikei canyon (nz),”  ISPRS journal of photogrammetry and remote sensing, vol. 82, pp. 10–26, 2013.
  * [15] Justin Solomon, Fernando de Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, and Leonidas Guibas,  “Convolutional wasserstein distances: Efficient optimal transportation on geometric domains,”  ACM Trans. Graph., vol. 34, no. 4, jul 2015.
  * [16] Nicolas Courty, Rémi Flamary, Devis Tuia, and Thomas Corpetti,  “Optimal transport for data fusion in remote sensing,”  in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2016.
  * [17] Khiem Pham, Khang Le, Nhat Ho, Tung Pham, and Hung Bui,  “On unbalanced optimal transport: An analysis of Sinkhorn algorithm,”  in Proceedings of the 37th International Conference on Machine Learning, Hal Daumé III and Aarti Singh, Eds. 13–18 Jul 2020, vol. 119 of Proceedings of Machine Learning Research, pp. 7673–7682, PMLR.
  * [18] C. Villani,  Optimal Transport: Old and New,  Grundlehren der mathematischen Wissenschaften. Springer Berlin Heidelberg, 2008.
  * [19] F. Santambrogio,  Optimal Transport for Applied Mathematicians: Calculus of Variations, PDEs, and Modeling,  Progress in Nonlinear Differential Equations and Their Applications. Springer IP, 2015.
  * [20] Gabriel Peyré and Marco Cuturi,  “Computational optimal transport: With applications to data science,”  Foundations and Trends® in Machine Learning, vol. 11, no. 5-6, pp. 355–607, 2019.
  * [21] Thibault Séjourné, Jean Feydy, François-Xavier Vialard, Alain Trouvé, and Gabriel Peyré,  “Sinkhorn divergences for unbalanced optimal transport,” 2019.
  * [22] Laetitia Chapel, Rémi Flamary, Haoran Wu, C’edric F’evotte, and Gilles Gasso,  “Unbalanced optimal transport through non-negative penalized linear regression,”  in Neural Information Processing Systems, 2021.