# Mixing Signals: Data Augmentation Approach for Deep Learning Based
Modulation Recognition

Xinjie Xu*, Zhuangzhi Chen*, Dongwei Xu, Huaji Zhou, Shanqing Yu, Shilian
Zheng, _Member, IEEE_ ,
Qi Xuan, _Senior Member, IEEE_ , and Xiaoniu Yang  *: The first two authors
contributed equally.

###### Abstract

With the rapid development of deep learning, automatic modulation recognition
(AMR), as an important task in cognitive radio, has gradually transformed from
traditional feature extraction and classification to automatic classification
by deep learning technology. However, deep learning models are data-driven
methods, which often require a large amount of data as the training support.
Data augmentation, as the strategy of expanding dataset, can improve the
generalization of the deep learning models and thus improve the accuracy of
the models to a certain extent. In this paper, for AMR of radio signals, we
propose a data augmentation strategy based on mixing signals and consider four
specific methods (Random Mixing, Maximum-Similarity-Mixing,
$\theta-$Similarity Mixing and n-times Random Mixing) to achieve data
augmentation. Experiments show that our proposed method can improve the
classification accuracy of deep learning based AMR models in the full public
dataset RML2016.10a. In particular, for the case of a single signal-to-noise
ratio signal set, the classification accuracy can be significantly improved,
which verifies the effectiveness of the methods.

###### Index Terms:

Deep learning, Automatic modulation recognition, Data augmentation, Mixing
Signal

##  I Introduction

Automatic modulation recognition (AMR) are widely used in wireless
communication system [1]. In modern communication environment, radio signals
are showing trends such as heavy load, wide spectrum range, and high
recognition accuracy requirements, which makes traditional feature extraction
based AMR difficult [2]. Especially, in the modern communication environment,
radio signals are showing trends such as heavy load, wide spectrum range, and
high recognition accuracy requirements, which makes that feature extraction
and recognition based on prior knowledge face many difficulties. In recent
years, deep learning has also reached new high, exerting its advantages in
various fields [3]. Deep learning model can automatically extract features by
processing and integrating big data, which coincides with the problem of AMR.
Therefore, more and more scholars are beginning to use deep learning
algorithms to distinguish the modulation types of radio signals [4, 5, 6, 7,
8], so signal recognition tend to be automated and avoided manual feature
extraction based on experience, improving the ability to recognize complex
radio signals finally.

Deep learning based methods often relies on a large amount of data for model
training, however, the collection and labeling of real data usually consume a
lot of costs, thus data augmentation, as a software-based method, becomes a
convenient and effective strategy for creating large-scale dataset [9].
Traditional data augmentation generates new data through simple
transformations such as rotation, cropping, slicing and warping [10, 11, 12,
13] of the real data, this kind of methods are simple and convenient, but have
little effect in improving the generalization of the model because they only
changes little on the existing data. The deep learning based data augmentation
usually use adversarial generative network (GAN) [14, 15, 16] or transfer
learning [17, 18, 19] to synthesize new data, however, these generative models
themselves also require a lot of data to train, so when the amount of real
data is small, this kind of methods are also not very effective. Between these
two kinds of methods, for image recognition, there is one kind of methods that
based on the multi-sample mixing, such as Mixup [20], these methods create new
samples by mixing multiple existing samples, increasing the diversity of
samples compare with simple sample transformation and are simpler and more
convenient compared with the deep learning methods.

In this paper, we focus on the data augmentation of radio signals for the task
of AMR. At present, there are few researches on data augmentation of radio
signals, mainly through rotation, flip [21] and GAN [22, 23] to expand the
dataset, but the signals generated by these methods will face the problem
mentioned above, especially when the amount of real data is small. Therefore,
inspired by the multi-sample mixing strategy used in image data augmentation,
for AMR of radio signals, we propose data augmentation framework based on
signals mixing according to the characteristics of signals, which can improve
the performance of deep learning based AMR models, especially when the amount
of real data is small. The main contributions of this paper can be summarized
as follows:

  1. 1.

We propose Mixing Signals, a simple but effective data augmentation strategy
for signals, which generate a large number of samples of known samples by
pairwise mixing the samples of the same modulation type.

  2. 2.

Under Mixing Signals strategy, four specific methods are proposed to achieve
the pairwise signals mixing. We found that these methods can effectively
improve the classification accuracy of the deep learning model, especially
when the amount of real dataset is less than full dataset (at the scenarios of
single SNR).

  3. 3.

The effectiveness of these methods are explored on different scenarios and
models, and the modulation recognition accuracy of the deep learning model is
increased by more than 10% with the samples of single SNR. What’s more,
comparing with the traditional methods (flip, rotation, gaussian noise) to
comprehensively, we evaluate the practicability of various signal-oriented
data augmentation methods and analyze the difference in timing and
distribution of samples before and after data augmentation.

The rest of paper is structured as follows. Sec. II introduces the automatic
modulation recognition and data augmentation techniques in recent years. In
Sec. III, we present the details of our methods and principle. Sec. IV shows
the simulation setting and results with analysis. Finally, we conclude the
paper and introduce future work in Sec. V.

##  II related works

###  II-A Automatic Modulation Recognition

To overcome the problems existing in the long-distance data transmission, the
radio signals must be modulated to load the signal spectrum into the high-
frequency channel for transmission. Then the receiver analyzes the modulation
type of the high-frequency signal through the AMR algorithm, so as to
demodulate the signal to realize long-distance transmission. Traditional AMR
are typically categorized into likelihood-based methods [24, 25] and feature-
based methods [26, 27]. The former is based on hypothesis theory, which
analyzes the signal cumulant and compares it with a given threshold for
classification. Although it has a complete theoretical foundation, in
practical applications, it is limited to a large amount of calculation and the
challenge of handling diverse real samples. The latter can identify the
received signal in the absence of prior information by extracting the
instantaneous amplitude, phase and frequency, Fourier [28] and wavelet
transform [29] of the signal, and then choose a suitable classifier for
identification. But this method requires high measurement accuracy of feature
parameters and is sensitive to noise, so it has low efficiency in engineering.

With the rise of deep learning, deep neural networks (DNNs) have been widely
used in many fields such as computer vision [30], machine translation [31] and
speech recognition [32]. Many researchers are also combining modulation
recognition with deep learning to offer new ideas for solving the difficulty
of modulation recognition [4]. O’shea et al., has applied deep learning to
signal classification earlier, and proposed an end-to-end Convolutional Neural
Networks (CNN) in the literature [5]. They put the modulated signal into the
model and get its modulation type by residual block. Huynh-The et al. [6]
proposed the MCNet. The network architecture is designed several specific
convolution blocks to learn the correlation of spatiotemporal signals through
different asymmetric convolution kernels. Xu et al. [7], proposed a novel
three-stream deep learning framework to extract the features from individual
and combined in-phase/quadrature(I/Q) symbols of the modulated signals. In
addition to using the original signal directly, recent studies have included
methods for converting the signal to another modality. For example, the SigNet
proposed by Chen et al. [33], which adopts a signal-to-matrix operator to
convert the original signal into a square matrix, and then combines it with
CNN to achieve the task of modulation recognition. Xuan et al. [34] introduced
signals into the field of graph machine learning, and achieved the goals by
Adaptive Visibility Graph (AVG) algorithm, which aims to adaptively map
signals into graphs. Different from popular CNNs, the bidirectional long
short-term memory network (DCN-BiLSTM) [8] can extract the contextual
information of signals well and address the long-term dependence problems.

###  II-B Data augmentation

Data augmentation is a method that can reduce over-fitting and improve the
generalization ability of the model. It is widely used in the training process
of neural network models [35]. This method only needs to process the training
data to expand the dataset to alleviate many problems such as small amount of
data and unbalanced samples, thereby improving the robustness of the model.

For image classification tasks, general data augmentation includes rotation,
scaling, cropping, and translation [21]. At present, data augmentation based
on multiple samples have gradually attracted attention. Mixup proposed by
Zhang et al. [20] and CutMix proposed by Yun et al. [36] are all based on
combining multiple samples with prior knowledge to construct neighbors of real
samples for effectively expanding new training samples. In addition to these
methods by mixing samples, data augmentation based on machine learning have
also received attention, such as GAN [14], style transfer [17] to generate new
samples, and AutoAugment [37] algorithm to learn the best augmentation
strategy by reinforcement learning(RL).

For time series data augmentation, time domain transformation is the most
direct data augmentation method for time series data, mainly including Window
slicing, Window warping, Magnitude warping, Jittering, etc. Window slicing
[13] randomly crops the time series and interpolates the time series back to
the original length. Window warping [38] is a unique time series augmentation
method, which selects a random time range, then compresses (down samples) or
extends (up samples), while other time ranges remain unchanged. Similar to
window warping, Magnitude Warping [13] is to multiply the magnitudes at the
beginning of any position by a certain magnitude. Noise injection is a method
by injecting small amount of noise into time series without changing the
corresponding labels. This includes injecting Gaussian noise, spike, step-like
trend, and slope-like trend, etc [39].

Different from image and basic time series, thanks to radio signal
classification requires more prior knowledge, the current research of signal
augmentation is limited, mainly including rotation, flipping, adding Gaussian
noise [21] and the using ACGAN, mentioned by TANG et al. [22], to generate new
constellation diagram for the improvement of model’s robustness. Haradal et
al. [15] proposed a method for generating time series data based on GAN, and
enhanced the classification of biological signals. However, when the data set
is short, GAN cannot learn the data distributions of samples, which leads to a
lack of realism in the generated samples. There are also many problems
existing such as unstable training, disappearance of gradients and mode
collapse, so GAN is not suitable for insufficient radio signal dataset.

##  III Method

In this section, we introduce the details of Mixing Signals (MS). Firstly, we
introduce the AMR task and the overall framework of using MS for AMR data
augmentation, then, three specific methods are introduced to achieve MS for
new signal generation, and finally, we point out the principle of MS to
analyze the reason for its effect.

###  III-A Automatic Modulation Recognition

Usually a communication system includes a transmitter, a channel and a
receiver. The input symbol $s(t)$ is converted into a transmission signal by
the modulation function $F$, and then the signal is transmitted to the
receiver through the signal $h(t)$. The obtained signal can be shown as

$\displaystyle x(t)=F(s(t))h(t)+v(t)$ (1)

where $v(t)$ is Additive White Gaussian Noise (AWGN). AMR seeks to obtain the
modulation function $F$, which serves as the basis for demodulation, which can
be seen as a multi-class classification problem of radio signals.

Radio signals are modulated by I/Q modulators and thus can be represented by
two channels of I/Q, where I is in-phase, Q is quadrature. IQ signal is
defined as:

$\displaystyle x(t)=I(t)+jQ(t)=e^{j\varphi(t)}$ (2)

from which a $L$-length signal can be sampled as:

$\left\\{\begin{array}[]{lr}I=[i_{1},i_{2},...,i_{L}]\\\
Q=[q_{1},q_{2},...,q_{L}]\end{array}\right.and\ x=\left[\begin{matrix}I\\\
Q\end{matrix}\right]$ (3)

Therefore, in AMR task, a signal sample is represented by $(x,y)$ with $x\in
R^{2\times L}$ and its modulation category label $y$. Then, we feed the sample
$x$ into the deep learning model $f$ to get the predicted label $\hat{y}$,
based on which we can infer the modulation function $F$.

Figure 1: The framework of data augmentation by mixing signals.

###  III-B Data Augmentation by Mixing Signals

The overall framework of data augmentation by mixing signals is shown in Fig.
1. Specifically, we use MS to largely increase the size of training set, so as
to better fit the data distribution and further improve the generalization
ability of the DNN model in the test process. In particular, we propose the
following three MS methods.

Figure 2: Random Mixing augmentation

####  III-B1 Random Mixing (RM)

This method mixes two training samples of the same class to generate two new
samples by randomly slicing, as shown in Fig. 2. We define the mixing
operation as

$\begin{matrix}{\hat{x}}_{A}=M_{A}\bigodot x_{A}+\left(1-M_{B}\right)\bigodot
x_{B}\\\ {\hat{x}}_{B}=\left(1-M_{A}\right)\bigodot x_{A}+M_{B}\bigodot
x_{B}\\\ {\hat{y}}_{A}={\hat{y}}_{B}=y_{A}=y_{B}\\\ \end{matrix}$ (4)

where $({\hat{x}}_{A},{\hat{y}}_{A})$ and $({\hat{x}}_{B},{\hat{y}}_{B})$ are
two newly generated samples, while $(x_{A},y_{A})$ and $(x_{B},y_{B})$ are two
original ones. And $\bigodot$ is element-wise multiplication to multiply the
elements at the corresponding positions.
$M_{A},M_{B}\in\left\\{0,1\right\\}^{L}$ are the binary mask for $x_{A}$ and
$x_{B}$, where the elements $M_{A}(p_{A}+i)=M_{B}(p_{B}+i)=1$ for
$i=0,1,2,\ldots,l$ and the others are set to 0. Note that $l$ could be any
integer in $(0,L)$, but in practice, it is better to be close to $L/2$ in
order to make the original signals mixing as much as possible, so as to
generate distinguishing descendants. Once $l$ is settled down, the parameters
$p_{A}$ and $p_{B}$ are then chosen in $[0,L-l]$. Typically, modulation
changes the amplitude and frequency of the carrier signal, exchanging segments
will affect these two properties little. Therefore, RM can preserve the
original modulation type of the signal by mixing the samples of the same
category and allows the model to learn features more effectively.

####  III-B2 Maximum Similarity Mixing (MSM)

Here, we further consider the similarity between the exchanging signals. We
argue that the newly generated signals could be more consistent with the
original ones, if the more similar parts are exchanged, and thus may benefit
the subsequent classification task. Denoting $T_{A}$ and $T_{B}$ as the
exchanging parts of $x_{A}$ and $x_{B}$, respectively, which have the same
length $l$, we seek to maximize their similarity, or minimize their distance,
as following:

$\mathop{\arg\min}_{p_{A},p_{B}}{D\left(T_{A},T_{B}\right)}$ (5)

We then obtain the following MSM method, by considering definition of
distances between vectors. Here, we use DTW as metrics. Note that DTW changes
the alignment between time series data points from ”one-to-one” to ”one-to-
many” of Euclidean distances to accommodate phase shifts and amplitude changes
between time series.

DTW-MSM: This algorithm[40] uses dynamic programming to calculate the degree
of alignment of two time series data.

$D\left(T_{A},T_{B}\right)=DTW\left(T_{A}^{l},T_{B}^{l}\right)$ (6)

Then, according to the recursive formula(9),we can calculate the solution from
$DTW\left(T_{A}^{1},T_{B}^{1}\right)$ to
$DTW\left(T_{A}^{l},T_{B}^{l}\right)$, one by one.

$D\left(T_{A},T_{B}\right)=DTW\left(T_{A}^{l},T_{B}^{l}\right)+\left\\{\begin{array}[]{lr}DTW\left(T_{A}^{i-1},T_{B}^{j}\right)\\\
DTW\left(T_{A}^{i},T_{B}^{j-1}\right)\\\
DTW\left(T_{A}^{i-1},T_{B}^{j-1}\right)\\\ \end{array}\right.$ (7)

And the distance corresponding position of two signals is defined as

$dis\left(T_{A}^{i},T_{B}^{j}\right)=\sqrt{\left|T_{A}^{i}-T_{B}^{j}\right|^{2}}$
(8)

Through the two-steps grid searching, we can get a more accurate global
optimal solution, so that any two signals can be mixed to expand dataset and
achieve the purpose of improving the comprehensive performance for the
classification model.

####  III-B3 $\theta-$Similarity Mixing ($\theta-$SM)

To compensate for the pros and cons of random mixing and Maximum Similarity
Mixing, we introduce a hyperparameter $\theta$ to evaluate the similarity of
swapped region, called $\theta-$Similarity Mixing. This method aims to find a
balance between disordered signals and optimal signals, so that the augmented
dataset contains both the regularization of random signals and the
authenticity of the original data.

For any two samples to be mixed, we randomly select two signal segments
$T_{A}$ and $T_{B}$, and then calculate their similarity $D(T_{A},T_{B})$ by
DTW. If its similarity D is less than a given parameter $\theta$, the signal
segment is mixed. Obviously, with the gradual increase of $\theta$, the
exchanged segments are no longer constrained by the similarity, making it tend
to RM. Meanwhile, with the gradual decrease of $\theta$, the exchanged
segments tend to be consistent, and the method will evolve into MSM.

####  III-B4 n-times Random Mixing (nRM)

MSM and $\theta-$Similarity Mixing describe the similarity from the amplitude,
frequency and other features of the signal itself, which means that two
samples mix once. Therefore, we further explore the feature diversity of data
and extend the RM method. On the basis of random mixing, we use multiple
mixing to make the samples contain more modulation information, so it’s called
n-times Random Mixing (nRM). Method As shown in Fig.3, we perform multiple
iterations over the initial signal by MS.

Just like RM, we further break up the original signal. We find that the
performance of the model is mainly restricted by the amplitude and frequency
of the carrier signal, and its timing characteristics have little effect on
modulation recognition. Therefore, this method is still valid for AMR tasks.

Figure 3: n-times Random Mixing augmentation Figure 4: Constellation diagrams
of 11 modulated signals with different methods.

###  III-C The principle of MS

In the field of computer vision, there are currently Mixup[20], Cutout[41],
CutMix[36] and other methods. Among them, although Mixup makes full use of the
pixels in the images, it will introduce unnatural artifacts. And Cutout makes
the model to focus on the parts with poor discrimination in the object, but
does not make full use of all the pixels, which reduces the training
efficiency. CutMix ensures that two objects can be identified from the partial
attempts of a single image by sharing activation, so as to improve training
efficiency.

Since there are no obvious multiple objects in the signal sample, and usually
its information of frequency and phase is contained in the waveform. So some
of its features also exist in the whole waveform, rather than in the parts. If
we still adopt the mode of sharing the active object, the data may be
introduced noise information. More seriously, this will lead to a fundamental
change in the type of modulation, especially in FM signals. Therefore, we are
inspired by the two aspects and adopt the same object fusion for
electromagnetic signals to prevent serious distortion of the waveform.

In order to observe the differences between the various methods, we made a
visual analysis of the 18db signal in the RML2016 dataset. The constellation
diagram is shown in the Fig.4, from top to bottom, the order is original, RM-
based, MSM-based, $\theta-$SM-based and nRM-based signals. From the Figure, We
find that although mixing signals changes the sequence of the signal, it
doesn’t affect the modulation type of the signal to a certain extent. Compared
with rotating and flipping, mixing signals maintains the original channel
characteristics based on the combination of real samples, because of which all
mixing signals maintain high similarity with the original signals. So,
generated samples is effective to enhance the generalization ability of DNN.

##  IV Simulations

In this section, we evaluate MS for its performance to improve the
classification models on radio signal datasets.

###  IV-A Dataset, scenarios and methods

####  IV-A1 Dataset

We evaluate the proposed data augmentation strategy with the public dataset
RML2016.10a [42]. The dataset is simulated by GNU Radio, contains 11
modulation types and 20 Signal to Noise Ratio (SNR) levels ranging from -20dB
to 18dB. Each sample has 128 sampling points and IQ channels. We divide the
dataset into training set and test set according to the ratio of 4:1, the
training set and test set contain 176,000 and 44,000 signal samples.

####  IV-A2 Scenarios

Specifically, we set up two scenarios to verify our methods:

  * $\bullet$

Scenarios I: Simulation with partial training samples based on a single
signal-to-noise ratio. Since most models cannot be effectively trained on low
signal-to-noise ratio signals and data with low signal-to-noise ratios are of
little value in practical applications, in this simulation, we ignore the low
SNR (less than -6 dB) samples, and choose three different levels of SNR, -6dB,
0dB, 10dB and 18dB respectively, as the research target. In order to set up
partial training samples scenario, we select the -6dB, 0dB, 10dB and 18dB
samples in the training set respectively as three new training sets, and all
test samples of -6dB, 0dB, 10dB and 18dB are set to the corresponding test set
respectively. Each training set size is 8800 (800 samples for per modulation
type) and each test set is 2200(200 samples for per modulation type).

  * $\bullet$

Scenarios II: Simulation with full training set, which means the training set
size is 176000 and test set size is 44000.

For the former, we verify the effect of our methods in the case that only a
part of real data can be obtained, such case is more in line with the
application scenario of our proposed methods, and for the latter, we verify
the effect of our methods in the case of relatively sufficient real data.

####  IV-A3 The details of methods

  * $\bullet$

Filp: Flip[21] includes horizontal flip by switching the I value to its
opposite and vertical flip by switching the Q value to its opposite.

  * $\bullet$

Rotation: Rotation[21] includes rotating by a given angle for Constellation
Diagram, such as 90°and 180°.

  * $\bullet$

Gaussian noise: Adding gaussian noise[21] with different standard deviations.

  * $\bullet$

MS: Mixing Signals performs pairwise mixing of samples of the same label by
different strategies(RM, MSM, $\theta-$SM and nRM).

Figure 5: Analysis of the effect of the mixing length $l$ for MS. X-axis means
the proportion of the mixing region to the total signal sample.

(a) 0dB

(b) 10dB

(c) 18dB

Figure 6: The influence of augmented data size for the methods. The
experiments use ResNet as classifier to train the data of RML2016.10a. The
x-axis(N) represents enhancement factor $N$.

###  IV-B Influence of some hyperparameters on mixing signals results

In order to maximize the benefits of the MS strategy, we firstly conduct a
simple analysis of the hyperparameters to determine more appropriate values,
which are used as the basis for the experiments. We mainly analyze the
hyperparameter length of fragment $l$, enhancement factor $N$, similarity
threshold $\theta$ for $\theta-$Similarity Mixing and the times of mixing $n$
for nRM. Especially for parameter $N$, it’s obvious that deep learning model
get preciser and robuster performance with the increasing of the enhancement
factor. However, these hyperparameters necessarily have reasonable values to
balance the performance and training efficiency, which is the goal of the next
experiments.

For MS, hyperparameters $l$ and $N$ affect all MS strategies, so we experiment
with controlled variables at first. To evaluate the effect of $l$ on this
method, we fix the enhancement factor $N$ and simulate in scenario I. In order
to fully reflect the MS methods, we increase the amount of enhanced samples to
4 times of the initial dataset, which means the enhancement factor $N=5$.
Then, We traverse the ratio of the length of the mixing region for the total
signal length $L$ with a step length of 5%, and use the partial
dataset(RML2016.10a) to conduct multiple experiments. Ultimately, we compare
the average classification accuracy with different experiments to select a
appropriate value. As shown in Fig.5, with the gradual increasing of $l$, the
recognition capability of the model presents a trend of firstly rising and
then falling. Especially, when the proportion is 0.5, the new signal sample at
this time preserves more features of the two original signals before and after
Mixing Signals, such as the key information of frequency and amplitude, so the
generalization ability of the model reaches the extreme value. Therefore, in
order to obtain a more superior classification model, we set the proportion of
$l/L$ to 0.5 in the next MS experiments.

Secondly, we continue to analyze the enhancement factor $N$ on the basis of
$l/L=0.5$ on the first scenarios. Among them, rotation gradually expands the
dataset at fixed angles, and MS randomly selects two samples of the same
category for mixing. In this section, We evaluate the influence of $N$ on
different methods in the dataset, and the average classification accuracy is
shown in Fig.6. Since the CNN cannot learn operations with the rotation of
large angles, rotation takes advantage when the enhancement factor is low.
However, with the increase of the enhancement factor, the nonlinear activation
in the model has a regular effect on the rotation of smaller angles, so that
the rotation with denser angles can no longer compensate for the defects of
the model, resulting rotation has little effect on the model. And MS can still
make the model obtain more detailed recognition patterns by expanding the
amount of data, especially while the enhancement factor $N=13$, the
enhancement effect is more outstanding and we take this way as the basis of
the following experiment.

For the hyperparameter $n$ in nRM, we adopt the same way, take scenario I as
an example, set $N=13$ as baseline. When the length of the mixing region is
small, the method may have the situation that the mixing region fail to
overlap, which is inconsistent with the situation that the mixing region
overlaps. Therefore, we consider the influence of the length $l$ of the mixing
region and the number of mixing $n$ on the method, the result is shown in the
Fig.7. In the Figure, the x-axis is the mixing length ratio $l/L$, and the
y-axis is the number of mixing $n$. It can be found from the heatmap that with
the increase or decrease of $l$, the unfused region gradually shrinks, and the
method tends to RM, so the performance reach its best when l/L=0.5, which is
consistent with the experimental results of the influence of the parameter
$l/L$. However, the number of mixing $n$ has a weak influence on this method,
so we take the parameters $l/L=0.5$ and $n=4$ when the model performance
reaches the best (91.82%).

Figure 7: The influence of $l/L$ and $n$ for nRM on 18dB dataset. TABLE I: Comparison result of proposed method and others on partial dataset Model | SNR |  w/o data augmentation | Flip | Rotation |  Gaussian noise | Mixing Signals(MS)
---|---|---|---|---|---|---
RM | DTW-MSM |  $\theta$-SM | nRM
CNN | -6dB | 51.14% | 47.91% | - | 50.41% | 54.59% | 54.18% | 53.27% | 59.64%
0dB | 66.68% | 70.14% | 72.77% | 69.09% | 71.18% | 69.82% | 71.77% | 74.55%
10dB | 72.23% | 75.19% | 80.14% | 73.32% | 76.09% | 74.05% | 77.36% | 78.00%
18dB | 71.45% | 74.86% | 81.45% | 73.18% | 76.14% | 76.05% | 77.46% | 82.23%
ResNet | -6dB | 61.86% | 60.41% | 65.46% | 60.41% | 63.96% | 63.73% | 61.32% | 62.68%
0dB | 77.27% | 80.86% | 88.59% | 74.27% | 88.68% | 85.18% | 82.91% | 86.50%
10dB | 80.64% | 84.91% | 84.86% | 73.09% | 89.55% | 92.86% | 83.36% | 88.36%
18dB | 78.64% | 85.05% | 85.18% | 67.05% | 88.59% | 87.18% | 88.23% | 91.82%
MCLDNN | -6dB | 62.68% | 65.14% | - | 63.00% | 64.09% | 64.91% | 64.09% | 64.32%
0dB | 81.43% | 77.86% | 81.77% | 80.18% | 84.50% | 83.73% | 83.14% | 84.55%
10dB | 83.86% | 84.41% | 91.36% | 85.68% | 91.68% | 88.91% | 90.05% | 90.36%
| 18dB | 83.09% | 84.95% | 90.91% | 86.18% | 90.09% | 91.82% | 90.09% | 90.86%
TABLE II: Comparison result of proposed method and others on low SNRs Model | SNR |  w/o data augmentation | Flip | Rotation |  Gaussian noise | Mixing Signals(MS)
---|---|---|---|---|---|---
RM | DTW-MSM |  $\theta$-SM | nRM
CNN | -10dB | 30.05% | 32.73% | 32.00% | 30.68% | 33.05% | 32.86% | 31.23% | 35.32%
-18dB | 10.27% | 11.05% | 10.64% | 10.59% | 10.96% | 10.73% | 10.36% | 11.55%
ResNet | -10dB | 38.86% | 38.73% | 37.73% | 38.77% | 39.64% | 39.09% | 39.27% | 39.73%
-18dB | 13.14% | 13.05% | 12.23% | 12.32% | 13.46% | 12.64% | 12.64% | 13.00%
MCLDNN | -10dB | 40.32% | 39.73% | 38.00% | 38.77% | 40.14% | 39.14% | 40.46% | 38.55%
-18dB | 12.09% | 11.86% | 11.50% | 11.46% | 12.36% | 11.96% | 11.14% | 11.86%

###  IV-C The performance of MS on the first scenarios

####  IV-C1 Comparison with other data augmentation methods

In this section, we compare flip, rotation and MS on the first scenarios with
different models, such as CNN, ResNet and MCLDNN. Note that flip only includes
horizontal flip and vertical flip, thus, its enhancement factor $N=3$. TableI
shows that the comparison results of the augmented models and the original
models with proposed method and others. It can be found that the MS method
achieves similar classification accuracy comparing with flip and rotation, and
can achieve an accuracy improvement of 1.82%$\sim$13.19% on partial samples,
which verifies the MS can improve the classification performance of the
Convolutional Neural Network. By pairwise combining signal samples, more
features are merged into one sample, so that the timing features can be better
captured by the model. Especially in Fig.6, We found that with the increasing
of enhancement factor, rotation fail to get better results.

Comparing CNN and ResNet, for models with simpler structures, the effect of
mixing signals is not as good as rotation, such as CNN, which is easier to
capture features with obvious changes, but cannot easily capture changes in
signal time-domain exchange. Furthermore, compared with complex models, such
as ResNet and MCLDNN, these models are good at mining the information in the
data itself without requiring more complex samples, which makes the MS better
than Flip and Rotation. From the perspective of model depth, we compare CNN,
MCLDNN and resnet, and we can find that this method(MS) is more suitable for
deep networks. Under the action of convolution operation, the receptive field
obtained by the high-dimensional features of the shallow network is smaller,
which makes the model pay more attention to local features, while the deep
network pays more attention to the overall features. Then MS can be understood
as a regular term that introduces local features, so that the deep network
(ResNet) takes into account both the whole and the local, while the shallow
network pays more attention to the local. So it is obvious that ResNet can
obtain more knowledge, which also supports the experimental results of more
accuracy improvement.

Furthermore, in order to reflect the influence of the data augmentation method
on low SNRs, we retrain the full-sample pre-trained model to enhance the
generalization ability of low SNR signal classification. The results are shown
in TableII. We still test CNN, ResNet, and MCLDNN. In the case of low signal-
to-noise ratio, data augmentation has little benefit, and the impact of noise
cannot be offset by expanding the dataset.

####  IV-C2 Comparison with coupled methods

In the previous section, we explore the performance of individual augmentation
methods on the model. In this section, we explore whether there is a coupling
phenomenon between different methods, and the interaction between traditional
methods and MS to further improve the performance of MS. To this end, we take
the 18db signal and ResNet as an example to explore their interaction, and set
the parameters $l/L$=0.5, $\theta$=0.2, $N$=13, $n$=4. At first, we use
different methods to expand the dataset by 6 times to keep the data size after
the pairwise combination, then N=13, the experimental results are shown in the
TableIII. Comparing the TableI and the TableIII, our proposed method can make
up for the disadvantages of traditional methods to a certain extent, and
obtain better performance. However, the performance of the model is also
affected because a part of the mixing samples are replaced by rotated or
noised samples. Especially for the noised samples, the performance of the
model is significantly degraded compared to the single MS method. For the
coupling between MSs, the table shows that there is no strong coupling
phenomenon, on the contrary, different combinations of methods will inhibit
the model from acquiring features in a certain enhanced mode.

TABLE III: Comparison with coupled methods Coupled Methods | Accuracy
---|---
Rotation+RM | 87.36%
Rotation+DTW-MSM | 91.18%
Rotation+$\theta$-SM | 90.68%
Rotation+nRM | 91.32%
Gaussian Noise+RM | 69.36%
Gaussian Noise+DTW-MSM | 83.68%
Gaussian Noise+$\theta$-SM | 83.86%
Gaussian Noise+nRM | 76.82%
RM+DTW-MSM | 85.36%
RM+$\theta$-SM | 86.55%
RM+nRM | 87.68%
DTW-MSM+$\theta$-SM | 86.46%
DTW-MSM+nRM | 89.18%
$\theta$-SM+nRM | 85.73%

###  IV-D The performance of MS on the second scenarios

Since RM and nRM still have a good effect in experiments with partial dataset,
in this section, we evaluate the data augmentation between flip, rotation, RM
and nRM on different models on the second scenarios. In whole experiments,
training set is doubled, so the training set and test set contain 352,000 and
44,000 signal samples. We take $l/L=0.5$ for MS and $n=4$ for nRM.

Figure 8: Comparison result of different methods Figure 9: Comparison result
of Random-Mixing on different model

As shown in Fig.8, our proposed method can increase the signal classification
accuracy by about 5% when SNR is between 4dB and 18dB. However, when SNR is
lower than -4db, various augmentation methods cannot work, and even inhibit
the classification ability of the model. Although the MS has more side effects
on the classification of low SNR signals. As the noise gradually decreases, RM
can make full use of the effective features in the samples, especially at 8db,
which improve the classification accuracy of the model about 6.14% at most. In
Fig.9, we observed that this classification suppression phenomenon is common
in the DNN model, and when the SNR is -6db, the model’s distinction between
the signal itself and the noise is in the critical. However, more noisy data
will only cause the model to fit the noise and fail to capture the modulation
characteristics. Only in relatively clean data, MS can play a regular role for
the model, filter out more noise, and improve the classification of the model.

According to the above results, we no longer use signals less than -6dB into
the data process of MS, and the result (Resnet+Random-Mixing2) is shown in
TableIV. Hence, removing part of the noise interference can obtain a better
classification accuracy.

In general, MS is more sensitive to noise than rotation. As shown in Figures 6
and 8, while there is a lot of noise, the amplified data will decrease the
model’s recognition ability, but it also shows excellent performance in data
with high SNR, which improves the overall modulation recognition capability.

TABLE IV: The result of removing signals less than -6dB Method | Accuracy
---|---
Resnet | 58.74%
Resnet+RM | 59.63%
Resnet+RM2 | 60.83%

##  V Conclusion

This paper proposes a data augmentation method for signals. Through mixing
multiple samples, a large number of generated samples are expanded to train
the deep learning model, so that the generalization ability of the modulation
recognition model could be improved. Especially in partial dataset, this
method improves the accuracy of the model by more than 10% at most, verifying
its good performance in signal augmentation. In addition, we consider some MS
strategies, and conducet experiments on partial datasets and full datasets. It
was found that the method we proposed is more suitable for samples with less
noise and hindered the classification of samples with low SNR. In future work,
we will consider more MS strategies or mixing tests, and exchange the more
obvious feature regions in the signal for training and testing in a certain
way, so as to further improve the accuracy and robustness of the deep learning
model.

## References

  * [1] E. Azzouz and A. K. Nandi, “Automatic modulation recognition of communication signals,” 2013.
  * [2] M. Pedzisz and A. Mansour, “Automatic modulation recognition of mpsk signals using constellation rotation and its 4th order cumulant,” _Digital Signal Processing_ , vol. 15, no. 3, pp. 295–304, 2005.
  * [3] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” pp. 770–778, 2016.
  * [4] F. Meng, P. Chen, L. Wu, and X. Wang, “Automatic modulation classification: A deep learning enabled approach,” _IEEE Transactions on Vehicular Technology_ , vol. 67, no. 11, pp. 10 760–10 772, 2018.
  * [5] T. J. O’Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning based radio signal classification,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 168–179, 2018.
  * [6] T. Huynh-The, C.-H. Hua, Q.-V. Pham, and D.-S. Kim, “Mcnet: An efficient cnn architecture for robust automatic modulation classification,” _IEEE Communications Letters_ , vol. 24, no. 4, pp. 811–815, 2020.
  * [7] J. Xu, C. Luo, G. Parr, and Y. Luo, “A spatiotemporal multi-channel learning framework for automatic modulation recognition,” _IEEE Wireless Communications Letters_ , vol. 9, no. 10, pp. 1629–1632, 2020.
  * [8] K. Liu, W. Gao, and Q. Huang, “Automatic modulation recognition based on a dcn-bilstm network,” _Sensors_ , vol. 21, no. 5, p. 1577, 2021.
  * [9] C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” _Journal of big data_ , vol. 6, no. 1, pp. 1–48, 2019.
  * [10] A. Buslaev, V. I. Iglovikov, E. Khvedchenya, A. Parinov, M. Druzhinin, and A. A. Kalinin, “Albumentations: fast and flexible image augmentations,” _Information_ , vol. 11, no. 2, p. 125, 2020.
  * [11] M. M. Krell and S. K. Kim, “Rotational data augmentation for electroencephalographic data,” in _2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)_ , pp. 471–474. IEEE, 2017.
  * [12] T. Fields, G. Hsieh, and J. Chenou, “Mitigating drift in time series data with noise augmentation,” in _2019 International Conference on Computational Science and Computational Intelligence (CSCI)_ , pp. 227–230. IEEE, 2019.
  * [13] T. T. Um, F. M. Pfister, D. Pichler, S. Endo, M. Lang, S. Hirche, U. Fietzek, and D. Kulić, “Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks,” in _Proceedings of the 19th ACM international conference on multimodal interaction_ , pp. 216–220, 2017.
  * [14] A. Waheed, M. Goyal, D. Gupta, A. Khanna, F. Al-Turjman, and P. R. Pinheiro, “Covidgan: data augmentation using auxiliary classifier gan for improved covid-19 detection,” _Ieee Access_ , vol. 8, pp. 91 916–91 923, 2020.
  * [15] Q. Zhang and Y. Liu, “Improving brain computer interface performance by data augmentation with conditional deep convolutional generative adversarial networks,” _arXiv preprint arXiv:1806.07108_ , 2018.
  * [16] Q. Xuan, Z. Chen, Y. Liu, H. Huang, G. Bao, and D. Zhang, “Multiview generative adversarial network and its application in pearl classification,” _IEEE Transactions on Industrial Electronics_ , vol. 66, no. 10, pp. 8244–8252, 2018.
  * [17] P. A. Cicalese, A. Mobiny, P. Yuan, J. Becker, C. Mohan, and H. V. Nguyen, “Stypath: Style-transfer data augmentation for robust histology image classification,” in _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pp. 351–361. Springer, 2020.
  * [18] X. Zhang, J. Ma, Y. Li, P. Wang, and Y. Liu, “Few-shot learning of parkinson’s disease speech data with optimal convolution sparse kernel transfer learning,” _Biomedical Signal Processing and Control_ , vol. 69, p. 102850, 2021.
  * [19] A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese, “Taskonomy: Disentangling task transfer learning,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 3712–3722, 2018.
  * [20] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” _arXiv preprint arXiv:1710.09412_ , 2017.
  * [21] L. Huang, W. Pan, Y. Zhang, L. Qian, N. Gao, and Y. Wu, “Data augmentation for deep learning-based radio modulation classification,” _IEEE Access_ , vol. 8, pp. 1498–1506, 2019.
  * [22] B. Tang, Y. Tu, Z. Zhang, and Y. Lin, “Digital signal modulation classification with data augmentation using generative adversarial nets in cognitive radio networks,” _IEEE Access_ , vol. 6, pp. 15 713–15 722, 2018\.
  * [23] Z. Huaji, B. Jing, W. Yiran, J. Licheng, S. ZHENG, S. Weiguo, X. Jie, and Y. Xiaoniu, “Few-shot electromagnetic signal classification: A data union augmentation method,” _Chinese Journal of Aeronautics_ , 2021.
  * [24] F. Wang, O. A. Dobre, C. Chan, and J. Zhang, “Fold-based kolmogorov–smirnov modulation classifier,” _IEEE Signal Processing Letters_ , vol. 23, no. 7, pp. 1003–1007, 2016.
  * [25] B. Dulek, “Online hybrid likelihood based modulation classification using multiple sensors,” _IEEE Transactions on Wireless Communications_ , vol. 16, no. 8, pp. 4984–5000, 2017.
  * [26] Y. Liang, X. Xiang, Y. Sun, X. Da, C. Li, and L. Yin, “Novel modulation recognition for wfrft-based system using 4th-order cumulants,” _IEEE Access_ , vol. 7, pp. 86 018–86 025, 2019.
  * [27] Y. Zeng, M. Zhang, F. Han, Y. Gong, and J. Zhang, “Spectrum analysis and convolutional neural network for automatic modulation recognition,” _IEEE Wireless Communications Letters_ , vol. 8, no. 3, pp. 929–932, 2019\.
  * [28] A. Hazza, M. Shoaib, S. A. Alshebeili, and A. Fahad, “An overview of feature-based methods for digital modulation classification,” in _2013 1st international conference on communications, signal processing, and their applications (ICCSPA)_ , pp. 1–6. IEEE, 2013.
  * [29] D. Zhang, “Wavelet transform,” in _Fundamentals of Image Data Mining_ , pp. 35–44. Springer, 2019.
  * [30] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” _arXiv preprint arXiv:2111.06377_ , 2021.
  * [31] M. Pham, J. M. Crego, and F. Yvon, “Revisiting multi-domain machine translation,” _Transactions of the Association for Computational Linguistics_ , vol. 9, pp. 17–35, 2021.
  * [32] M. Pham, J. M. Crego, and F. Yvon, “Revisiting multi-domain machine translation,” _Transactions of the Association for Computational Linguistics_ , vol. 9, pp. 17–35, 2021.
  * [33] Z. Chen, H. Cui, J. Xiang, K. Qiu, L. Huang, S. Zheng, S. Chen, Q. Xuan, and X. Yang, “Signet: A novel deep learning framework for radio signal classification.” IEEE, 2021.
  * [34] Q. Xuan, J. Zhou, K. Qiu, Z. Chen, D. Xu, S. Zheng, and X. Yang, “Adaptive visibility graph neural network and its application in modulation classification.” IEEE, 2022.
  * [35] C. Summers and M. J. Dinneen, “Improved mixed-example data augmentation,” in _2019 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , pp. 1262–1270. IEEE, 2019.
  * [36] S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo, “Cutmix: Regularization strategy to train strong classifiers with localizable features,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , pp. 6023–6032, 2019.
  * [37] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, “Autoaugment: Learning augmentation strategies from data,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 113–123, 2019.
  * [38] A. Le Guennec, S. Malinowski, and R. Tavenard, “Data augmentation for time series classification using convolutional neural networks,” in _ECML/PKDD workshop on advanced analytics and learning on temporal data_ , 2016.
  * [39] T. Wen and R. Keyes, “Time series anomaly detection using convolutional neural networks and transfer learning,” _arXiv preprint arXiv:1905.13628_ , 2019\.
  * [40] T. Rakthanmanon, B. Campana, A. Mueen, G. Batista, B. Westover, Q. Zhu, J. Zakaria, and E. Keogh, “Searching and mining trillions of time series subsequences under dynamic time warping,” in _Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pp. 262–270, 2012.
  * [41] T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” _arXiv preprint arXiv:1708.04552_ , 2017.
  * [42] DEEPSIG, “Rf datasets for machine learning,” https://www.deepsig.ai/datasets, 2018.