# Stop Oversampling for Class Imbalance Learning: A Review

Ahmad B. Hassanat  Ahmad S. Tarawneh  Ghada A. Altarawneh  Faculty of
Information Technology, Mutah University, Karak, Jordan  Dept. of Algorithms
and Their Applications, Eötvös Loránd University, Budapest, Hungary  Dept.
Accounting, Mutah University, Karak, Jordan

# Stop Oversampling for Class Imbalance Learning: A Critical Review

Ahmad B. Hassanat  Ahmad S. Tarawneh  Ghada A. Altarawneh  Faculty of
Information Technology, Mutah University, Karak, Jordan  Dept. of Algorithms
and Their Applications, Eötvös Loránd University, Budapest, Hungary  Dept.
Accounting, Mutah University, Karak, Jordan

###### Abstract

For the last two decades, oversampling has been employed to overcome the
challenge of learning from imbalanced datasets. Many approaches to solving
this challenge have been offered in the literature. Oversampling, on the other
hand, is a concern. That is, models trained on fictitious data may fail
spectacularly when put to real-world problems. The fundamental difficulty with
oversampling approaches is that, given a real-life population, the synthesized
samples may not truly belong to the minority class. As a result, training a
classifier on these samples while pretending they represent minority may
result in incorrect predictions when the model is used in the real world. We
analyzed a large number of oversampling methods in this paper and devised a
new oversampling evaluation system based on hiding a number of majority
examples and comparing them to those generated by the oversampling process.
Based on our evaluation system, we ranked all these methods based on their
incorrectly generated examples for comparison. Our experiments using more than
70 oversampling methods and three imbalanced real-world datasets reveal that
all oversampling methods studied generate minority samples that are most
likely to be majority. Given data and methods in hand, we argue that
oversampling in its current forms and methodologies is unreliable for learning
from class imbalanced data and should be avoided in real-world applications.

###### keywords:

Oversampling , SMOTE , Imbalanced datasets

##  1 Introduction

When training a dataset with examples from one class greatly outnumbering
those from the other, a phenomenon known as class imbalance emerges. The
majority class is usually referred to as such, whereas the minority class is
referred to as such. There may be more than one majority class and more than
one minority class in a single dataset. The main cause of class imbalance is
that classifiers trained on unequal training sets have a prediction bias,
which is linked to poor performance in the minority class(es), Depending on
the dataset utilized, the bias could range from a little imbalance to a severe
imbalance [1, 2, 3, 4, 5]. This problem has grown and has become a significant
difficulty since the minority class is frequently of critical importance, as
it represents favorable examples that are rare in nature or expensive to
obtain [6]. This is true when considering contexts such as Big Data analytics
[7, 8, 9, 10, 11, 12, 13], Biometrics [14, 15, 16, 17, 18, 19, 20, 21, 22],
gene profiling [23], credit card fraud detection [24, 25], face image
retrieval [24], content-based image retrieval [26, 27], disease detection [28,
29, 30, 31, 32], internet of things [33, 34, 35, 36, 37, 38, 39, 40, 41, 42,
43], Natural Language Processing [44, 45], network security [46, 47, 48, 49,
50, 51, 52], image recognition [53, 54, 55, 56, 57, 58], Anomaly Detection
[59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69], etc. In formal terms, a
supervised machine learning dataset $D$ with $n$ instances belonging to $m$
classes $C1,C2,C3,...,Cm$ is said to be a class imbalanced dataset if and only
if for any $|Ci|>>|Cj|$, where $i$ and $j$ are indexes $1,2,3,...,m$. There
are several approaches to solving class imbalance problem before starting
classification, such as:

  * 1.

More samples from the minority class(es) should be acquired from the discourse
domain.

  * 2.

Changing the loss function to give the failing minority class a higher cost
[70].

  * 3.

Oversampling the minority class.

  * 4.

Undersampling the majority class.

  * 5.

Any combination of previous approaches.

Each of the aforementioned approaches has its own set of benefits and
drawbacks [71, 72]. Oversampling, on the other hand, is the most often used
approach among them, as seen by the multitude of oversampling methods
published in the last two decades. However, this does not necessarily imply
that the oversampling approach is beneficial. Oversampling approaches boost
the quantity of minority-class instances by creating new ones out of thin air
based only on their similarity to one or more of the minority’s examples. This
is troublesome since such methods may raise the likelihood of the learning
process being overfitted [73, 74, 75, 76, 75][71] [72] [73] [74] [75]. On
paper, the overfitted synthetic datasets produce good machine learning
results, however this is not always the case in practice. Another more
critical problem of oversampling is that the fabricated examples could exist
in the real world belonging to a different class, regardless of how similar it
is to the minority’s examples, as we always have examples from class A that
are the closest to examples from a different class B. Therefore, we argue
that, even if such synthesizing generates favorable outcomes on paper,
negative results can be easily obtained in practice. The major goal of this
study, in addition to reviewing a large number of oversampling methods, is to
prove our counterclaim on the use of oversampling as a solution to the problem
of class imbalance, which is as follows:

Oversampling in its current forms and methodologies is a misleading approach
that should be avoided since it feeds the learning process with falsified
instances that are pushed to be members of the minority class when they are
most likely members of the majority.

To the best of our knowledge, the only methodology for proving an oversampling
method’s goodness is its classification accuracy metrics after the
classification of the oversampled datasets, with no tests for the validity of
the synthesized instances and if they are appropriate for training a model for
real-world use. Therefore, we find oversampling practitioners are pleased with
their machine learning outcomes in the lab, but they should consider how much
harm could be done in practice outside of the lab, particularly in medical and
other vital applications. The harm is exacerbated when we realize that several
of these methods have become integral parts of APIs and machine learning
packages, such as Python imbalanced-learn API [77] and Smote-Variants API
[78]. We prove our counterclaim in this paper by using a number of typical
oversampling methods on several benchmark datasets, concealing some of the
majority examples, and then comparing the created examples to the hidden
majority examples to determine if they approximately match. Finding such
counter examples proves our counterclaim.

The following is the structure of this paper: The literature review of class
imbalance problem is presented in the second section. The mythology of proving
our counterclaim is illustrated in Section Three. And the experimental results
are listed and discussed in section four.

##  2 Literature review of oversampling methods

In the literature, there are various ways to machine learning from class
imbalance data. One of the most prevalent ways, particularly SMOTE-like
approaches, is oversampling. On January 26, 2022, a Google Scholar search for
the term ”SMOTE” yielded 77,300 results, while a search for ”oversampling”
yielded 297,000 results. This is merely a foreshadowing of the developing
trend of oversampling. Figure 1 depicts the nearly exponential increase in the
number of articles that dealt with, employed, or addressed oversampling and/or
SMOTE.

Figure 1: The number of publications that have the terms ”oversampling” and/or
”SMOTE”

The relevance of the well-defined class imbalance problem and the simplicity
of oversampling solutions are the reasons for this abnormal surge in
oversampling research. Anyone with a rudimentary understanding of machine
learning can come up with a novel way to produce fresh similar examples given
some minority examples. There could be an infinite number of such solutions.

Several studies, such as [79, 80, 81], have reviewed various oversampling
approaches; nevertheless, they are not thorough and have not paid adequate
attention to validating the oversampling approach to the problem of class
imbalance.

One of the earliest and most extensively utilized approaches for class
imbalance is the Synthetic Minority Oversampling Technique (SMOTE) [82]. It
interpolates synthetic examples between nearest neighbors from the training
set’s collection of minority class cases. As a result, by merging the
properties of seed instances with randomly picked k-nearest neighbors, a
synthetic sample is generated. The earliest version of the SMOTE algorithm
relied solely on synthetic oversampling. They also used a combination of
synthetic oversampling and undersampling, which might be useful [83]. SMOTE
was tested on nine benchmark datasets and proven to improve classification
performance.

SVMSMOTE [84], which is based on SMOTE, focuses on constructing SVM
modifications to successfully handle the problem of class imbalance.
Oversampling, cost-sensitive learning, and undersampling are some of the
heuristics used in SVM modeling. This method produced promising results when
compared to other oversampling methods.

Borderline-SMOTE [85] is an SMOTE-based minority oversampling method that only
oversamples the minority examples around the borderline. In comparison to
SMOTE and other random oversampling methods investigated, their findings show
that this solution improves classification results for the minority class.

Oversampling by a synthetic inverse minority is used in Reverse-SMOTE
(R-SMOTE) [86], a technique based on SMOTE and the inverse near-neighbor idea.
R-SMOTE beats other over-sampling methods in terms of precision,
F-measurement, and accuracy, according to this study that compared traditional
sampling procedures to alternative methods, including SMOTE. In the
comparison, three benchmark datasets were employed.

Constrained Oversampling (CO) [87] is a technique for reducing noise in
oversampling. This method is used to extract the overlapping regions in a
dataset. Ant Colony Optimization is then used to define the boundaries of
minority regions. Most significantly, in order to create a balanced dataset,
fresh samples are synthesized via oversampling under constraints. This method
varies from others in that it includes noise-reduction constraints in the
oversampling process. CO outperforms a range of oversampling benchmarks,
according to their results.

In addition, the Majority Weighted Minority Oversampling Technique (MWMOTE)
[88] was offered as a solution to the problem of class-imbalance learning.
MWMOTE finds and weights difficult-to-learn informative minority class samples
based on their distance from nearby majority class samples. It then creates
synthetic samples from the weighted informative minority class samples using a
clustering algorithm. The primary premise of MWMOTE is that all generated
samples must belong to one of the minority class clusters. In terms of
numerous assessment measures, the provided results suggest that MWMOTE is
superior than or similar to some other existing approaches.

Adaptive synthetic (ADASYN) [89] was given with the goal of eliminating bias
and moving the classification decision boundary in the direction of the hard
examples. The primary idea behind ADASYN is to use a weighted distribution for
different minority class examples based on their learning difficulty, with
more synthetic data created for more difficult minority class examples than
for easier minority class examples. The efficacy of this method is proved by
the results of experiments conducted on a variety of datasets using five
different evaluation measures.

Synthetic Minority Over-Sampling Technique Based on Furthest Neighbor
Algorithm (SOMTEFUNA) [6] is another exciting and recent method for machine
learning from imbalanced datasets. To produce fresh synthetic minority
examples, this method employs the farthest neighbor examples. SOMTEFUNA has a
number of advantages over some other approaches, one of which being the lack
of tuning parameters, which makes it easier to be used in real-world
scenarios. Using Naive Bayes and Support Vector Machine classifiers, the
method compared the benefits of resampling to common methods such as SMOTE and
ADASYN. The reported findings show that SOMTEFUNA is a viable alternative to
the other oversampling methods, according to its reported results.

Sampling WIth the Majority (SWIM) [90] is a synthetic oversampling method that
is robust in cases of significant class imbalance. SWIM’s fundamental feature
is that it uses the density of the well-sampled majority class to direct the
creation process. SWIM’s model was built using both the radial basis function
and the Mahalanobis distance. SWIM was put to the test on 25 benchmark
datasets, and the findings show that it beats some of the most common
oversampling methods.

Other ways of oversampling include, but are not limited to, the work of [91,
92, 93, 94, 78, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107,
108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119]

The validation process is what all oversampling methods have in common, which
is basically the evaluation of the classifier’s performance employed to
classify the oversampled datasets using one or more accuracy measures such as
Accuracy, Precision, Recall, F-measure, G-mean, Specificity, Kappa, Matthews
correlation coefficient (MCC), Area under the ROC Curve (AUC), True positive
rate, False negative (FN), False positive (FP), True positive (TP), True
negative (TN), and ROC curve. Table 1 lists 72 oversampling methods, including
their known names, references, the number of datasets utilized, the number of
classes in these datasets, the classifiers employed, and the performance
metrics used to validate the classification results after oversampling.

Table 1: Summary of the methods used in this study. In this table, C4.5 is Decision Tree C4.5, LR is LR, LDA is Linear Discriminate Analysis, NB is Naive Bayes, RF is Random Forest and ANN is Artificial Neural Network.  ID | Method Name | Reference | No. Datasets | No. Classes | Classifiers | Performance measures
---|---|---|---|---|---|---
M1 | SMOTE | [82] | 6 | Binary |  | C4.5
---
Ripper
NB
| ROC curve
---
AUC
M2 | SMOTE TomekLinks | [120] | 13 | Binary | C4.5 | AUC
M3 | SMOTE ENN | [120] | 13 | Binary | C4.5 | AUC
M4 | Borderline SMOTE1 | [85] | 4 | Binary | C4.5 |  | TP rate
---
F-values
M5 | Borderline SMOTE2 | [85] | 4 | Binary | C4.5 |  | TP rate
---
F-values
M6 | ADASYN | [89] | 5 | Binary | C4.5 |  | Accuracy, Precision
---
Recall, F-measure
G-mean
M7 | AHC | [121] | 1 | Binary |  | C4.5
---
KNN, SVM
NB
AdaBoost
| Recall, Specificity
---
Accuracy
ROC, G-mean
weighted accuracy
M8 | distance SMOTE | [122] | 10 | Binary | Linear Regression |  | Precision
---
Recall
F-measure
M9 | polynom fit SMOTE | [123] | 1 | Binary | SVM | TP rate, TN rate
M10 | Stefanowski | [124] | 9 | Binary |  | C4.5
---
MODLEM
| Accuracy
---
Specificity
Recall
M11 | ADOMS | [125] | 12 | Binary | ANN | G-mean
M12 | Safe Level SMOTE | [126] | 2 | Binary | NB, SVM |  | Precision, Recall
---
F-values, AUC
M13 | MSMOTE | [127] | 3 | Binary |  | C4.5
---
AdaBoost
| Precision, Recall
---
F-values
M14 | DE oversampling | [128] | 10 | Binary | SVM | F-measure, AUC
M15 | SMOBD | [129] | 9 | Binary | SVM | G-mean, AUC
M16 | SUNDO | [130] | 4 | Binary | CART, SVM |  | Accuracy, TP
---
TN, FP, FN
M17 | MSYN | [131] | 10 | Binary | C4.5 | AUC, F-measure
M18 | SVM balance | [132] | 1 | Binary |  | ANN
---
RF
LR
| Accuracy, Recall
---
Specificity, AUC
M19 | TRIM SMOTE | [133] | 11 | Binary | C4.5 | AUC, F-measure
M20 | SMOTE RSB | [134] | 44 | Binary | C4.5 | AUC
M21 | ProWSyn | [135] | 10 | Binary |  | ANN
---
C4.5
| F-measure
---
G-mean, AUC
M22 | SL graph SMOTE | [136] | 8 | Binary |  | KNN, RIPPER
---
C4.5
NB
F-measure, AUC
M23 | LVQ SMOTE | [137] | 8 | Binary |  | Logistic Tree
---
ANN
NB
RF
SVM, OLVQ3
| Recall
---
Specificity
G-mean
M24 | SOI CJ | [138] | 20 | Binary | C4.5 |  | Precision, Recall
---
F-measure, AUC
M25 | ROSE | [139] | 20 | Binary |  | C4.5
---
LR
AUC
M26 | SMOTE OUT | [140] | 18 | Binary | SVM | F-measure
M27 | SMOTE Cosine | [140] | 18 | Binary | SVM | F-measure
M28 | Selected SMOTE | [140] | 18 | Binary | SVM | F-measure
M29 | LN SMOTE | [141] | 15 | Binary |  | C4.5
---
NB
| F-measure
---
G-mean
Recall
M30 | MWMOTE | [88] | 20 | Binary |  | C4.5
---
AdaBoost, KNN
ANN
G-mean, AUC
M31 | PDFOS | [142] | 6 | Binary | RBF |  | F-measure
---
G-mean, AUC
M32 | RWO sampling | [143] |  |  |  |
M33 | NEATER | [144] | 22 | Binary |  | C4.5
---
RF, SVM
G-mean, AUC
M34 | DEAGO | [145] | 2 | Binary | ANN | AUC
M35 | Gazzah | [146] | 2 | 100 | SVM |  | True Negative Rate
---
True Positive Rate
M36 | MCT | [147] | 14 | Binary |  | C4.5
---
KNN, SVM
NB
| FN, FP
---
M37 | SMOTE IPF | [148] | 9 | Binary | C4.5 | AUC
M38 | KernelADASYN | [149] | 7 | Binary | C4.5 |  | Precision
---
Recall
F-measure
G-Mean
M39 | MOT2LD | [150] | 15 | Binary | CART |  | G-mean
---
F-measure
M40 | V SYNTH | [151] | 6 | Binary |  | LR
---
LDA
| Kappa
---
Recall
Specificity
Accuracy
FPR, FNR
M41 | OUPS | [152] | 45 | Binary |  | SVM
---
KNN
LDA
LR
| Recall
---
Specificity
G-mean
M42 | SMOTE D | [153] | 66 | Binary |  | C4.5
---
KNN, SVM
F-measure
M43 | SMOTE PSO | [154] | 18 | Multi | SVM | AUC and G-mean
M44 | CURE SMOTE | [155] | 12 | Multi |  | CART
---
RF
| F-measure
---
G-mean
AUC
M45 | CE SMOTE | [156] | 10 | Binary | C4.5 |  | F-measure
---
G-mean
M46 | Edge Det SMOTE | [157] | 9 | Binary | SVM |  | F-measure
---
G-mean
M47 | CBSO | [158] | 8 | Binary |  | ANN
---
C4.5
| accuracy
---
F-measure
G-mean
M48 | ASMOBD | [129] | 9 | Binary | SVM | G-mean and AUC
M49 | Assembled SMOTE | [159] | 4 | Binary | SVM |  | Precision
---
Recall
F-score
Accuracy
M50 | SDSMOTE | [160] | 4 | Binary |  | C4.5
---
AdaBoost
Bagging
| AUC
---
F-measure
M51 | DSMOTE | [161] | 11 | Binary |  | Naïve Bayes
---
KNN, SVM
| Accuracy
---
Recall
Precision
F-measure
G-mean
M52 | G SMOTE | [162] | 4 | Binary | 1NN and SVM | AUC
M53 | NT SMOTE | [163] | 3 | Binary | SVM and C4.5 | Accuracy
M54 | Lee | [164] | 8 | Binary | SVM |  | Precision
---
Recall
G-mean
M55 | SMOTE PSOBAT | [165] | 30 | Binary |  | ANN
---
C4.5
| Kappa
---
Accuracy
M56 | MDO | [166, 167] | 20 | Multi |  | C4.5, KNN
---
RIPPER
| AUC
---
G-mean
Precision
Recall
F-measure
M57 | Random SMOTE | [168] | 10 | Multi | KNN | G-mean
M58 | VIS RST | [169] | 6 | Binary |  | AdaBoost
---
C4.5
| Accuracy
---
True positive rate
true negatives rate
F-measure, and AUC
M59 | GASMOTE | [170] | 10 | Binary | C4.5 | F-measure, G-mean
M60 | SMOTE FRST 2T | [171] | 1 | Binary | C4.5 | AUC, FP, FN
M61 | AND SMOTE | [172] | 12 | Binary | CART | AUC
M62 | NRAS | [173] | 41 | Binary |  | ANN
---
SVM, LDA
LR
| Recall
---
Specificity
G-mean
M63 | AMSCO | [174] | 30 | Binary | ANN |  | Precision
---
Recall
F-measure
G-mean
M64 | SSO | [175] | 6 | Binary | ANN |  | F-measure
---
G-mean
M65 | NDO sampling | [176] | 1 | Binary |  | LR
---
Decision Tree C5
ANN, SVM
F-measure, Recall
M66 | DSRBF | [177] | 13 | Multi | ANN |  | Accuracy
---
Recall
M67 | Gaussian SMOTE | [104] | 1 | Binary | SVM |  | Accuracy
---
Recall
Specificity
M68 | Supervised SMOTE | [178] | 2 | Binary | SVM |  | Accuracy
---
Recall
Specificity
MCC , AUC
M69 | SN SMOTE | [179] | 39 | Binary |  | KNN, ANN
---
C4.5
| Accuracy
---
Recall
Specificity, AUC
M70 | CCR | [180] | 32 | Binary |  | CART, KNN
---
NB, SVM
| F-measure
---
G-mean, AUC
M71 | ANS | [181] | 14 | Binary |  | ANN, KNN
---
NB, SVM
C4.5
F-measure, AUC
M72 | cluster SMOTE | [182] | 2 | Binary | RIPPER | AUC

As can be seen from the previous discussion and Table 1, all the
aforementioned oversampling methods use the classification accuracy measures
of the synthesized data to verify their goodness, assuming that the
synthesised examples belong to the minority class. On paper, however, the
accuracy measures appear to be good if the data is over-fitted, which is
common when using Oversampling methods [72, 73, 74, 75, 76, 71].

Another critical problem with the oversampling approach is the assumption that
the synthetic examples belong to the minority class; do they truly belong to
the minority class?

None of the previous literature has answered this critical question. This
study aims to provide a validation system for oversampling methods, in order
to determine to what degree these methods synthesize unrealistic examples;
assuming they are belonging to the minority when they are not.

##  3 Method and Data

The proposed validation system for oversampling methods works by hiding a
subset of the majority’s examples, which is referred to as the hidden subset.
Although the hidden majority examples are part of the population, we excluded
them from the training dataset since we assumed they were not obtained from
the real-world domain of discourse. Because all oversampling approaches do not
access the entire real-world population, this assumption is correct.

It is important to make sure that the class imbalance problem still exists
after concealing the hidden subset.

After that, we apply the oversampling method that needs to be validated on the
remaining dataset in order to generate new examples, which is referred to as
the synthetic subset. The hidden subset is then returned to the training set.

The generated examples in the synthetic subset are claimed to belong to the
minority class by all oversampling methods. We compare the similarity between
these examples (the synthetic subset) and all examples in the original
training set before oversampling to see if these synthesized examples belong
to the minority or the majority.

Figure 2 illustrates the proposed validation system.

Figure 2: Flow diagram of the proposed validation system

In order to determine the degree of similarity, we need a similarity measure
such as Euclidean distance (ED), Manhattan distance (MD), Hassanat distance
(HD) [183], etc. In this paper, we opt for HD as being invariant to noise,
outliers and data scale, since the nature of this metric prevents each feature
from having a distance greater than one, regardless of the scale of the
features in the targeted dataset. Furthermore, HD had been shown to outperform
a wide range of machine learning similarity measures, including the most
common ones like ED and MD [184, 185, 186, 187, 188].

HD can be expressed mathematically as in equation1.

$D(p_{i},q_{i})=\left\\{\begin{array}[]{ll}1-\frac{1+min(p_{i},q_{i})}{1+max(p_{i},q_{i})}&\mbox{,
}min(p_{i},q_{i})\geq 0\\\
1-\frac{1+min(p_{i},q_{i})+|min(p_{i},q_{i})|}{1+max(p_{i},q_{i})+|min(p_{i},q_{i})|}&\mbox{,
}min(p_{i},q_{i})<0\end{array}\right.$ (1)

and for the total distance between two examples is

$HD(\mathbf{p},\mathbf{q})={\sum_{i=1}^{N}D(p_{i},q_{i})}$ (2)

Where $p$ and $q$ are feature vectors and $N$ is the number of features in
each vector.

It is worth mentioning that we are proposing a validation system, not an
evaluation system, the similarity measure using HD is meant to find the number
of examples taken from the synthetic subset that are similar to the minority
as the core of our validation system. Those generated examples, which are more
similar to the majority indicate the error of the oversampling method
validated. This error is calculated according to equation 3.

$Error=\frac{CM}{SS}$ (3)

where $CM$ is the number of synthetic examples that are close to majority
examples and $SS$ is the total number of examples in the synthetic subset.

##  4 Datasets

We employ three real-life datasets to put our validation system to the test,
namely Yeast4, Yeast5, and Yeast6, which are routinely used by many
oversampling methods. On [189], all of the datasets are freely available.
Table 2 contains information about these datasets.

Table 2: Description of the datasets used in this study ID | Name | No. Attributes | No. Classes | No. Minor | No. Major
---|---|---|---|---|---
1 | Yeast4 | 10 | 2 | 51 | 1433
2 | Yeast5 | 10 | 2 | 44 | 1440
3 | Yeast6 | 10 | 2 | 35 | 1449

Table 2 shows that the datasets have different minority and majority
distributions, despite the fact that the number of attributes and classes are
the same. It is not necessary to address the problem with multi-class datasets
to prove our counter claim, as most oversampling approaches only use binary
class datasets, as shown in Table 1.

##  5 Experiments and Results

In our experiments, we used all of the oversampling methods listed in Table 1
on each of the three datasets listed in Table 2, after eliminating some
majority examples at random. We employed varied numbers of hidden examples,
namely 10%, 25%, and 50% of the majority examples of each dataset examined, to
see the effect of the number of hidden examples on the validation process.
Furthermore, each experiment is repeated five times, with the average of the
results for each hidden ratio for each oversampling method on each dataset
being reported. Table 2 shows the number of erroneous synthetic examples (NE),
which are ones that are generated as minority examples but appear to be more
comparable to majority examples, as the proposed validation system suggests.
It also shows the number of synthetic examples (SE) generated by each
oversampling method, in addition to the error rate (ER) which is calculated
using Equation 3. All the result reported in Table 2 were obtained by hiding
only 10% of the majority examples.

Table 3: Oversampling Methods’ Validation Results on three datasets using 10% hidden percent. | Yeast4 | Yeast5 | Yeast6 |  |
---|---|---|---|---|---
Method | NE | SE | ER | NE | SE | ER | NE | SE | ER | Avg. ER | Avg. Rank
M1 | 463.2 | 1239 | 0.37 | 108 | 1252 | 0.09 | 294 | 1270 | 0.23 | 0.23 | 24
M2 | 470 | 1235 | 0.38 | 117 | 1252 | 0.09 | 279 | 1270 | 0.22 | 0.23 | 25
M3 | 414.2 | 1130.2 | 0.37 | 91 | 1222 | 0.07 | 266 | 1187 | 0.22 | 0.22 | 20
M4 | 270.6 | 1239 | 0.22 | 277 | 1252 | 0.22 | 391 | 1270 | 0.31 | 0.25 | 38
M5 | 471 | 1239 | 0.38 | 597 | 1252 | 0.48 | 757 | 1270 | 0.60 | 0.48 | 58
M6 | 568.8 | 1239 | 0.46 | 219 | 1252 | 0.17 | 534 | 1270 | 0.42 | 0.35 | 50
M7 | 50 | 50 | 1.00 | 42 | 43 | 0.98 | 34 | 34 | 1.00 | 0.99 | 71
M8 | 482.6 | 1239 | 0.39 | 145 | 1252 | 0.12 | 258 | 1270 | 0.20 | 0.24 | 28
M9 | 765 | 1224 | 0.63 | 204 | 1232 | 0.17 | 244 | 1260 | 0.19 | 0.33 | 47
M10 | 53 | 59 | 0.90 | 9 | 11 | 0.82 | 17 | 17 | 1.00 | 0.91 | 68
M11 | 523.4 | 1239 | 0.42 | 165 | 1252 | 0.13 | 372 | 1270 | 0.29 | 0.28 | 43
M12 | 109.8 | 125.2 | 0.88 | - | - | - | 55 | 55 | 1.00 | 0.94 | 70
M13 | 363 | 1239 | 0.29 | 261 | 1252 | 0.21 | 308 | 1270 | 0.24 | 0.25 | 37
M14 | 425.8 | 771.4 | 0.55 | - | - | - | 250 | 648 | 0.39 | 0.47 | 56
M15 | 479 | 1239 | 0.39 | 116 | 1252 | 0.09 | 357 | 1270 | 0.28 | 0.25 | 39
M16 | 285 | 286 | 1.00 | 289 | 292 | 0.99 | 298 | 300 | 0.99 | 0.99 | 72
M17 | 309.8 | 1239 | 0.25 | 45 | 1252 | 0.04 | 79 | 1270 | 0.06 | 0.12 | 6
M18 | 478.8 | 1239 | 0.39 | 103 | 1252 | 0.08 | 269 | 1270 | 0.21 | 0.23 | 22
M19 | 329 | 1239 | 0.27 | 26 | 1252 | 0.02 | 124 | 1270 | 0.10 | 0.13 | 7
M20 | 938.8 | 2478 | 0.38 | 0 | 43 | 0.00 | 575 | 2540 | 0.23 | 0.20 | 13
M21 | 729.4 | 1239 | 0.59 | 221 | 1252 | 0.18 | 385 | 1270 | 0.30 | 0.36 | 51
M22 | 282.6 | 1239 | 0.23 | - | - | - | 406 | 1270 | 0.32 | 0.27 | 42
M23 | 775.8 | 1239 | 0.63 | 330 | 1252 | 0.26 | 621 | 1270 | 0.49 | 0.46 | 54
M24 | 52 | 1239 | 0.04 | 120 | 1252 | 0.10 | 41 | 1270 | 0.03 | 0.06 | 2
M25 | 867 | 1239 | 0.70 | 456 | 1252 | 0.36 | 707 | 1270 | 0.56 | 0.54 | 61
M26 | 442 | 1239 | 0.36 | 76 | 1252 | 0.06 | 246 | 1270 | 0.19 | 0.20 | 14
M27 | 475.4 | 1239 | 0.38 | 86 | 1252 | 0.07 | 265 | 1270 | 0.21 | 0.22 | 19
M28 | 470.2 | 1239 | 0.38 | 121 | 1252 | 0.10 | 303 | 1270 | 0.24 | 0.24 | 30
M29 | 413.8 | 1239 | 0.33 | 167 | 1252 | 0.13 | 272 | 1270 | 0.21 | 0.23 | 23
M30 | 644.8 | 1239 | 0.52 | 223 | 1252 | 0.18 | 369 | 1270 | 0.29 | 0.33 | 48
M31 | 872.2 | 1239 | 0.70 | 395 | 1252 | 0.32 | 599 | 1270 | 0.47 | 0.50 | 59
M32 | 927.8 | 1239 | 0.75 | 698 | 1252 | 0.56 | 987 | 1270 | 0.78 | 0.69 | 63
M33 | 1041.6 | 2478 | 0.42 | 315 | 2504 | 0.13 | 813 | 2540 | 0.32 | 0.29 | 45
M34 | 1236 | 1239 | 1.00 | 523 | 1252 | 0.42 | 36 | 1270 | 0.03 | 0.48 | 57
M35 | 504 | 791 | 0.64 | 148 | 807 | 0.18 | 159 | 841 | 0.19 | 0.34 | 49
M36 | 602 | 1239 | 0.49 | 191 | 1252 | 0.15 | 414 | 1270 | 0.33 | 0.32 | 46
M37 | 518 | 1239 | 0.42 | 110 | 1252 | 0.09 | 292 | 1270 | 0.23 | 0.25 | 34
M38 | 1200 | 1239 | 0.97 | 855 | 1252 | 0.68 | 1264 | 1270 | 1.00 | 0.88 | 67
M39 | 307 | 1226 | 0.25 | 129 | 1248 | 0.10 | 93 | 1262 | 0.07 | 0.14 | 8
M40 | 1199 | 1239 | 0.97 | 639 | 1252 | 0.51 | 1262 | 1270 | 0.99 | 0.82 | 66
M41 | 938 | 1264 | 0.74 | 547 | 1268 | 0.43 | 711 | 1272 | 0.56 | 0.58 | 62
M42 | 1239 | 1239 | 1.00 | 1010 | 1251 | 0.81 | 1271 | 1271 | 1.00 | 0.94 | 69
M43 | 78 | 153 | 0.51 | 32 | 132 | 0.24 | 47 | 105 | 0.45 | 0.40 | 53
M44 | 288 | 1239 | 0.23 | 188 | 1252 | 0.15 | 89 | 1270 | 0.07 | 0.15 | 9
M45 | 710 | 1239 | 0.57 | 72 | 1252 | 0.06 | 139 | 1270 | 0.11 | 0.25 | 35
M46 | 440 | 1239 | 0.36 | 85 | 1252 | 0.07 | 278 | 1270 | 0.22 | 0.21 | 17
M47 | 753 | 1239 | 0.61 | 363 | 1252 | 0.29 | 633 | 1270 | 0.50 | 0.47 | 55
M48 | 302 | 1239 | 0.24 | 35 | 1252 | 0.03 | 32 | 1270 | 0.03 | 0.10 | 4
M49 | 476 | 1239 | 0.38 | 123 | 1252 | 0.10 | 308 | 1270 | 0.24 | 0.24 | 32
M50 | 451 | 1239 | 0.36 | 85 | 1252 | 0.07 | 256 | 1270 | 0.20 | 0.21 | 16
M51 | 14 | 1239 | 0.01 | 2 | 1252 | 0.00 | 0 | 1270 | 0.00 | 0.00 | 1
M52 | 534 | 1239 | 0.43 | 85 | 1252 | 0.07 | 296 | 1270 | 0.23 | 0.24 | 33
M53 | 413 | 1239 | 0.33 | 18 | 1252 | 0.01 | 285 | 1270 | 0.22 | 0.19 | 11
M54 | 452 | 1239 | 0.36 | 103 | 1252 | 0.08 | 320 | 1270 | 0.25 | 0.23 | 26
M55 | 225 | 592 | 0.38 | 76 | 869 | 0.09 | 169 | 724 | 0.23 | 0.23 | 27
M56 | 592 | 1239 | 0.48 | 453 | 1252 | 0.36 | 441 | 1270 | 0.35 | 0.40 | 52
M57 | 470 | 1239 | 0.38 | 116 | 1252 | 0.09 | 304 | 1270 | 0.24 | 0.24 | 29
M58 | 232 | 411 | 0.56 | 69 | 224 | 0.31 | 95 | 148 | 0.64 | 0.50 | 60
M59 | 64 | 165 | 0.39 | 12 | 133 | 0.09 | 18 | 116 | 0.16 | 0.21 | 15
M60 | 552 | 1482 | 0.37 | 115 | 1500 | 0.08 | 279 | 1270 | 0.22 | 0.22 | 21
M61 | 212 | 1239 | 0.17 | 73 | 1252 | 0.06 | 128 | 1270 | 0.10 | 0.11 | 5
M62 | 140 | 1239 | 0.11 | 84 | 1252 | 0.07 | 28 | 1270 | 0.02 | 0.07 | 3
M63 | 275 | 805 | 0.34 | 97 | 1101 | 0.09 | 182 | 789 | 0.23 | 0.22 | 18
M64 | 579 | 1235 | 0.47 | 236 | 1250 | 0.19 | 257 | 1270 | 0.20 | 0.29 | 44
M65 | 558 | 1239 | 0.45 | 143 | 1252 | 0.11 | 307 | 1270 | 0.24 | 0.27 | 41
M66 | 521 | 1239 | 0.42 | 101 | 1252 | 0.08 | 275 | 1270 | 0.22 | 0.24 | 31
M67 | 918 | 1239 | 0.74 | 727 | 1252 | 0.58 | 1025 | 1270 | 0.81 | 0.71 | 64
M68 | 365 | 1239 | 0.29 | 140 | 1252 | 0.11 | 100 | 1270 | 0.08 | 0.16 | 10
M69 | 496 | 1239 | 0.40 | 104 | 1252 | 0.08 | 326 | 1270 | 0.26 | 0.25 | 36
M70 | 1048 | 1240 | 0.85 | 898 | 1253 | 0.72 | 1076 | 1270 | 0.85 | 0.80 | 65
M71 | 381 | 1239 | 0.31 | - | - | - | 105 | 1270 | 0.08 | 0.20 | 12
M72 | 411 | 1239 | 0.33 | 46 | 1252 | 0.04 | 498 | 1270 | 0.39 | 0.25 | 40

The averages of five trials on each dataset for each approach are provided in
Table 2. In addition, for each approach, the average error is calculated for
the error rates on all three datasets. The last column in the table shows the
average rank of each method based on the three datasets; the lower the rank,
the better the oversampling performance; for example, rank 1 shall be awarded
to the method with the smallest error.

A thorough examination of Table 2 demonstrates that all oversampling methods
result in errors in the synthesized examples. That is, they generate examples
that are meant to be minority, yet are similar to the majority or fall within
the majority class’s decision boundary. Despite the fact that all methods
generate such examples, the quantity of fake examples generated differs from
one method to another. On the Yeast4 dataset, for example, the oversampling
method (M51) generates 14 incorrect examples, whereas other methods, such as
M70, generate more than 1K incorrect examples. That is why M51 is ranked
first, whereas M70 is ranked much higher. Similar findings were achieved when
25% and 50% of the majority examples were used as hidden examples, thus there
is no need to include them in tables; however, we show them in Figure 3.

(a) 10%

(b) 25%

(c) 50%

Figure 3: Box plot of the average error rates of all oversampling methods on
three datasets with varied hidden percentages.

The average error rate of all oversampling methods increases somewhat as the
hidden percent increases, as seen in Figure 3. This is logical since when
oversampling methods synthesize their minority examples, they become unaware
of some majority examples; in fact, we expected a significant error rise as
the size of the hidden subset grew larger. In terms of the effect of the
dataset on the average oversampling error, we can observe in the same figure
that some datasets, such as (Yeast5), are easier to be oversampled than
others, such as (Yeast6) and (Yeast4). However, the difference is not
substantial, and more importantly, as the Box plots show, the standard
deviation of the error rates produced by all oversampling methods on each
dataset is extremely high.

In order to compare oversampling methods, we ranked them according to their
average error across three datasets. The average ranks of the methods are
plotted against the average errors they produced on three datasets (Yeast4,
Yeast5 and Yeast6) as shown in Figure 4.

Despite the fact that all of the methods discussed generate false examples,
Figure 4 indicates that certain methods do better than others in avoiding the
formation of false examples. As a result, if oversampling is unavoidable, the
method’s reliability should be verified using a validation tools such as ours.
Methods like M51, for example, have error rates close to 0%, whereas others
like M7 and M16 have error rates near to 100%.

Figure 4: Methods ranking based on their average error rate on thee datasets

As a result, all Oversampling methods validated produce misleading examples,
regardless of the hidden percentage or dataset used. Figure 5 visualizes the
Oversampling results on three datasets using M1, displaying various sorts of
examples, including hidden, minority, majority, and synthesized examples while
hiding 10% of the majority examples. For the sake of illustration, we limited
the data to only two features.

(a) Yeast4

(b) Yeast5

(c) Yeast6

Figure 5: Visualization of validating the SMOTE method on three datasets. When
the figure is scaled up, more information is obtained.

As seen in Figure 5, many synthetic examples are created on the basis of or
near hidden examples, producing almost identical feature values. Even in
higher dimensional feature space, such a situation has the potential to occur.
The common mistake that all oversampling methods make is to feed such data to
a classifier, assuming that all of the examples are realistic and labeled
based on reality. The classifier has no other knowledge and learns based on
the false assumption, which produces excellent results in labs but unexpected
behavior in real-world applications.

The results shown thus far do not necessarily imply that incorrect example
synthesis occurs just when the majority examples are hidden from the
oversampling method. Even though the majority examples are completely visible
to the methods, some methods generate false examples. The published findings
of all of the oversampling methods demonstrate this, as none of them claimed
to be an accurate method with no errors.

We validated the best performers on a fourth machine learning dataset,
Vehicle3, because some oversampling approaches passed our validation test by
presenting a relatively small number of unrealistic examples, and to further
support our counterclaim against the validity of the oversampling approach in
general. The validation results of the best performers are shown in Table 4.

Table 4: ER on Vehicle3 dataset using 25% hidden data and some of the methods with the least ER Method | NE | SE | ER
---|---|---|---
M51 | 168 | 264 | 0.64
M24 | 88 | 264 | 0.33
M62 | 122 | 264 | 0.46
M61 | 61 | 264 | 0.23
M19 | 91 | 264 | 0.34
M39 | 102 | 236 | 0.43
M44 | 194 | 264 | 0.73
M68 | 149 | 264 | 0.56

As can be seen in Table 4, when we changed the dataset, the errors of the
”best” oversampling methods increased significantly, demonstrating once again
that these oversampling methods fill in the features space gap without
considering whether the generated examples are truly belong to the minority,
and falsely consider them as such. This makes the training of these examples
deceptive, and it could lead to the classifier being overfitted on incorrect
data if robust generalization techniques are not used. As a result, when
applied to real-world tasks, it is possible that the entire machine learning
system fails spectacularly, particularly in critical applications such as
security, autonomous driving, aviation safety and medical applications, where
even one unrealistic synthesized example could do catastrophic harm.

##  6 Conclusion

Oversampling methods have been used and developed for decades to handle the
problem of class imbalance learning, and there is a near exponential growing
trend for such type of research. The main question of this research is
oversampling approach in its current form and methods provide applicable and
viable solution for learning from class imbalance data? We claim that the
current oversampling approach is deceptive and could lead to severe failures
in real-world applications. In order to answer the main question and to prove
our counterclaim, we reviewed a large number of oversampling methods and
analyzed their performance in terms of providing unrealistic examples, for
this purpose we propose a new validation system for oversampling methods,
which we utilized to validate over 70 different oversampling methods. Our
validation results on four real-world common datasets reveal that all of the
oversampling methods investigated generate false examples, assuming that they
are minorities when they are not, causing classifiers to perform well in labs
but more likely fail in practice.

The Oversampling methods investigated in this paper are ranked according to
how many incorrect examples they generate. When used to solve real-life
problems, the ranking shows that some methods are less harmful than others.
When the datasets were changed, however, they were found to be useless.
Therefore, we recommend avoiding such methods when dealing with sensitive
applications such as security, autonomous driving, aviation safety, and
medical applications that use machine learning from class imbalanced data.
Instead, we seriously encourage using ensemble approaches to problems of class
imbalance, such as Easy Ensemble. [190], Random Data Partitioning [71], etc.
Because these methods do not create data out of thin air and do not, as the
Undersampling approach suggests, deny the learning process from critical data.

More research should be done in the future to confirm the validity or
invalidity of oversampling approach, investigating more methods and
incorporating more data. Furthermore, we recommend that additional research be
conducted on real-world applications, including measurements of incorrect
predictions made with and without the use of oversampling methods, as well as
comparisons with ensemble methods.

## References

  * [1] G. Haixiang, L. Yijing, J. Shang, G. Mingyun, H. Yuanyue, G. Bing, Learning from class-imbalanced data: Review of methods and applications, Expert Systems with Applications 73 (2017) 220–239.
  * [2] M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, F. Herrera, A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42 (4) (2011) 463–484.
  * [3] G. M. Weiss, Foundations of imbalanced learning, Imbalanced Learning: Foundations, Algorithms, and Applications (2013) 13–41.
  * [4] J. Wu, Z. Zhao, C. Sun, R. Yan, X. Chen, Learning from class-imbalanced data with a model-agnostic framework for machine intelligent diagnosis, Reliability Engineering & System Safety 216 (2021) 107934.
  * [5] M. Peng, Q. Zhang, X. Xing, T. Gui, X. Huang, Y.-G. Jiang, K. Ding, Z. Chen, Trainable undersampling for class-imbalance learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 4707–4714.
  * [6] A. S. Tarawneh, A. B. Hassanat, K. Almohammadi, D. Chetverikov, C. Bellinger, Smotefuna: Synthetic minority over-sampling technique based on furthest neighbour algorithm, IEEE Access 8 (2020) 59069–59082.
  * [7] R. F. Mansour, S. Abdel-Khalek, I. Hilali-Jaghdam, J. Nebhen, W. Cho, G. P. Joshi, An intelligent outlier detection with machine learning empowered big data analytics for mobile edge computing, Cluster Computing (2021) 1–13.
  * [8] N. O. Aljehane, R. F. Mansour, Optimal allocation of renewable energy source and charging station for phevs, Sustainable Energy Technologies and Assessments 49 (2022) 101669.
  * [9] R. F. Mansour, J. Escorcia-Gutierrez, M. Gamarra, V. G. Díaz, D. Gupta, S. Kumar, Artificial intelligence with big data analytics-based brain intracranial hemorrhage e-diagnosis using ct images, Neural Computing and Applications (2021) 1–13.
  * [10] A. B. Hassanat, Two-point-based binary search trees for accelerating big data classification using knn, PloS one 13 (11) (2018) e0207772.
  * [11] A. Hassanat, Norm-based binary search trees for speeding up knn big data classification, Computers 7 (4) (2018) 54.
  * [12] A. Hassanat, Furthest-pair-based decision trees: Experimental results on big data classification, Information 9 (11) (2018) 284.
  * [13] A. B. Hassanat, Furthest-pair-based binary search tree for speeding big data classification using k-nearest neighbors, Big Data 6 (3) (2018) 225–235.
  * [14] A. B. Hassanat, S. Jassim, Visual words for lip-reading, in: Mobile Multimedia/Image Processing, Security, and Applications 2010, Vol. 7708, International Society for Optics and Photonics, 2010, p. 77080B.
  * [15] A. B. Hassanat, Visual speech recognition, Speech and Language Technologies 1 (2011) 279–303.
  * [16] A. B. Hassanat, E. Btoush, M. A. Abbadi, B. M. Al-Mahadeen, M. Al-Awadi, K. I. Mseidein, A. M. Almseden, A. S. Tarawneh, M. B. Alhasanat, V. S. Prasath, et al., Victory sign biometrie for terrorists identification: Preliminary results, in: 2017 8th International Conference on Information and Communication Systems (ICICS), IEEE, 2017, pp. 182–187.
  * [17] A. B. Hassanat, On identifying terrorists using their victory signs, Data Science Journal 17.
  * [18] A. S. Tarawneh, D. Chetverikov, C. Verma, A. B. Hassanat, Stability and reduction of statistical features for image classification and retrieval: Preliminary results, in: 2018 9th International Conference on Information and Communication Systems (ICICS), IEEE, 2018, pp. 117–121.
  * [19] M. Z. Al-Shamaileh, A. B. Hassanat, A. S. Tarawneh, M. S. Rahman, C. Celik, M. Jawthari, New online/offline text-dependent arabic handwriting dataset for writer authentication and identification, in: 2019 10th International Conference on Information and Communication Systems (ICICS), IEEE, 2019, pp. 116–121.
  * [20] A. Hassanat, M. Al-Awadi, E. Btoush, A. Al-Btoush, E. Alhasanat, G. Altarawneh, New mobile phone and webcam hand images databases for personal authentication and identification, Procedia Manufacturing 3 (2015) 4060–4067.
  * [21] A. I. Al-Btoush, M. A. Abbadi, A. B. Hassanat, A. S. Tarawneh, A. Hasanat, V. S. Prasath, New features for eye-tracking systems: Preliminary results, in: 2019 10th International Conference on Information and Communication Systems (ICICS), IEEE, 2019, pp. 179–184.
  * [22] A. B. Hassanat, V. S. Prasath, B. M. Al-Mahadeen, S. M. M. Alhasanat, Classification and gender recognition from veiled-faces, International Journal of Biometrics 9 (4) (2017) 347–364.
  * [23] H. Xu, C. Zhang, G. S. Hong, J. Zhou, J. Hong, K. S. Woon, Gated recurrent units based neural network for tool condition monitoring, in: 2018 International Joint Conference on Neural Networks (IJCNN), IEEE, 2018, pp. 1–7.
  * [24] U. Fiore, A. De Santis, F. Perla, P. Zanetti, F. Palmieri, Using generative adversarial networks for improving classification effectiveness in credit card fraud detection, Information Sciences 479 (2019) 448–455.
  * [25] N. Ghatasheh, H. Faris, R. Abukhurma, P. A. Castillo, N. Al-Madi, A. M. Mora, A.-Z. Ala’M, A. Hassanat, Cost-sensitive ensemble methods for bankruptcy prediction in a highly imbalanced data distribution: a real case from the spanish market, Progress in Artificial Intelligence 9 (4) (2020) 361–375.
  * [26] A. S. Tarawneh, A. B. Hassanat, C. Celik, D. Chetverikov, M. S. Rahman, C. Verma, Deep face image retrieval: A comparative study with dictionary learning, in: 2019 10th International Conference on Information and Communication Systems (ICICS), IEEE, 2019, pp. 185–192.
  * [27] A. S. Tarawneh, C. Celik, A. B. Hassanat, D. Chetverikov, Detailed investigation of deep features with sparse representation and dimensionality reduction in cbir: A comparative study, Intelligent Data Analysis 24 (1) (2020) 47–68.
  * [28] M. Hammad, M. H. Alkinani, B. Gupta, A. A. Abd El-Latif, Myocardial infarction detection based on deep neural network on imbalanced data, Multimedia Systems (2021) 1–13.
  * [29] M. Fatima, M. Pasha, et al., Survey of machine learning algorithms for disease diagnostic, Journal of Intelligent Learning Systems and Applications 9 (01) (2017) 1.
  * [30] A. Alqatawneh, R. Alhalaseh, A. Hassanat, M. Abbadi, Statistical-hypothesis-aided tests for epilepsy classification, Computers 8 (4) (2019) 84.
  * [31] M. Aseeri, A. B. Hassanat, S. Mnasri, A. S. Tarawneh, K. Alhazmi, G. Altarawneh, M. Alrashidi, H. Alharbi, K. Almohammadi, D. Chetverikov, et al., Modelling-based simulator for forecasting the spread of covid-19: A case study of saudi arabia, Int. J. Comput. Sci. Netw. Secur 20 (2020) 114–125.
  * [32] A. B. Hassanat, S. Mnasri, M. Aseeri, K. Alhazmi, O. Cheikhrouhou, G. Altarawneh, M. Alrashidi, A. S. Tarawneh, K. Almohammadi, H. Almoamari, A simulation model for forecasting covid-19 pandemic spread: Analytical results based on the current saudi covid-19 data, Sustainability 13 (9) (2021) 4888.
  * [33] S. Mnasri, A. Van Den Bossche, N. Nasri, T. Val, The 3d redeployment of nodes in wireless sensor networks with real testbed prototyping, in: International Conference on Ad-Hoc Networks and Wireless, Springer, 2017, pp. 18–24.
  * [34] S. Mnasri, N. Nasri, V. Thierry, The 3d indoor deployment in dl-iot with experimental validation using a particle swarm algorithm based on the dialects of songs, in: 2018 14th International Wireless Communications & Mobile Computing Conference (IWCMC), IEEE, 2018, pp. 928–933.
  * [35] S. Mnasri, A. Van den Bossche, N. Nasri, T. Val, The 3d deployment multi-objective problem in mobile wsn: optimizing coverage and localization, International Research Journal of Innovative Engineering-IRJIE 1 (5).
  * [36] S. Mnasri, N. Nasri, M. Alrashidi, A. Van den Bossche, T. Val, Iot networks 3d deployment using hybrid many-objective optimization algorithms, Journal of Heuristics 26 (2020) 663–709.
  * [37] W. Abdallah, S. Mnasri, T. Val, Genetic-voronoi algorithm for coverage of iot data collection networks, in: 30th International Conference on Computer Theory and Applications (ICCTA 2020), 2020, pp. 16–22.
  * [38] W. Abdallah, S. Mnasri, N. Nasri, et al., Emergent iot wireless technologies beyond the year 2020: A comprehensive comparative analysis, in: 2020 International Conference on Computing and Information Technology (ICCIT-1441), IEEE, 2020, pp. 1–5.
  * [39] S. Mnasri, N. Nasri, A. Van den Bossche, T. Val, A new multi-agent particle swarm algorithm based on birds accents for the 3d indoor deployment problem, ISA transactions 91 (2019) 262–280.
  * [40] S. Mnasri, F. Abbes, K. Zidi, K. Ghedira, A multi-objective hybrid bcrc-nsgaii algorithm to solve the vrptw, in: 13th International Conference on Hybrid Intelligent Systems (HIS 2013), IEEE, 2013, pp. 60–65.
  * [41] S. Tlili, S. Mnasri, T. Val, A multi-objective gray wolf algorithm for routing in iot collection networks with real experiments, in: 2021 National Computing Colleges Conference (NCCC), IEEE, 2021, pp. 1–5.
  * [42] S. Mnasri, N. Nasri, A. Van Den Bossche, T. Val, A hybrid ant-genetic algorithm to solve a real deployment problem: a case study with experimental validation, in: International Conference on Ad-Hoc Networks and Wireless, Springer, 2017, pp. 367–381.
  * [43] S. Mnasri, N. Nasri, A. Van Den Bossche, T. Val, A comparative analysis with validation of nsga-iii and moea/d in resolving the 3d indoor redeployment problem in dl-iot, in: 2017 International Conference on Internet of Things, Embedded Systems and Communications (IINTEC), IEEE, 2017, pp. 15–20.
  * [44] M. Alghamdi, W. Teahan, Experimental evaluation of arabic ocr systems, PSU Research Review.
  * [45] A. B. Hassanat, G. A. Altarawneh, Rule-and dictionary-based solution for variations in written arabic names in social networks, big data, accounting systems and large databases, Research Journal of Applied Sciences, Engineering and Technology 8 (14) (2014) 1630–1638.
  * [46] E. Hamadaqa, A. Abadleh, A. Mars, W. Adi, Highly secured implantable medical devices, in: 2018 International Conference on Innovations in Information Technology (IIT), IEEE, 2018, pp. 7–12.
  * [47] S. Mulhem, A. Abadleh, W. Adi, Accelerometer-based joint user-device clone-resistant identity, in: 2018 Second World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), IEEE, 2018, pp. 230–237.
  * [48] A. Mars, A. Abadleh, W. Adi, Operator and manufacturer independent d2d private link for future 5g networks, in: IEEE INFOCOM 2019-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), IEEE, 2019, pp. 1–6.
  * [49] A. Alabadleh, S. Aljaafreh, A. Aljaafreh, K. Alawasa, A rss-based localization method using hmm-based error correction, Journal of Location Based Services 12 (3-4) (2018) 273–285.
  * [50] A. Aljaafreh, K. Alawasa, S. Alja’afreh, A. Abadleh, Fuzzy inference system for speed bumps detection using smart phone accelerometer sensor, Journal of Telecommunication, Electronic and Computer Engineering (JTEC) 9 (2-7) (2017) 133–136.
  * [51] A. Abadleh, E. Al-Hawari, E. Alkafaween, H. Al-Sawalqah, Step detection algorithm for accurate distance estimation using dynamic step length, in: 2017 18th IEEE International Conference on Mobile Data Management (MDM), IEEE, 2017, pp. 324–327.
  * [52] A. Abadleh, S. Han, S. J. Hyun, B. Lee, M. Kim, Construction of indoor floor plan and localization, Wireless Networks 22 (1) (2016) 175–191.
  * [53] A. B. Hassanat, V. S. Prasath, K. I. Mseidein, M. Al-awadi, A. M. Hammouri, A hybridwavelet-shearlet approach to robust digital imagewatermarking, Informatica 41 (1).
  * [54] A. B. Hassanat, S. Jassim, Color-based lip localization method, in: Mobile Multimedia/Image Processing, Security, and Applications 2010, Vol. 7708, International Society for Optics and Photonics, 2010, p. 77080Y.
  * [55] A. B. Hassanat, M. Alkasassbeh, M. Al-awadi, A. Esra’a, Color-based object segmentation method using artificial neural network, Simulation Modelling Practice and Theory 64 (2016) 3–17.
  * [56] P. Narloch, A. Hassanat, A. S. Tarawneh, H. Anysz, J. Kotowski, K. Almohammadi, Predicting compressive strength of cement-stabilized rammed earth based on sem images using computer vision and deep learning, Applied Sciences 9 (23) (2019) 5131.
  * [57] A. B. Hassanat, V. S. Prasath, M. Al-kasassbeh, A. S. Tarawneh, A. J. Al-shamailh, Magnetic energy-based feature extraction for low-quality fingerprint images, Signal, Image and Video Processing 12 (8) (2018) 1471–1478.
  * [58] A. B. Hassanat, M. Alkasassbeh, M. Al-awadi, A. Esra’a, Colour-based lips segmentation method using artificial neural networks, in: 2015 6th international conference on information and communication systems (ICICS), IEEE, 2015, pp. 188–193.
  * [59] M. Al-kasassbeh, T. Khairallah, Winning tactics with dns tunnelling, Network Security 2019 (12) (2019) 12–19.
  * [60] G. Al-Naymat, M. Al-Kasassbeh, E. Al-Harwari, Using machine learning methods for detecting network anomalies within snmp-mib dataset, International Journal of Wireless and Mobile Computing 15 (1) (2018) 67–76.
  * [61] A. A. Zuraiq, M. Alkasassbeh, Phishing detection approaches, in: 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), IEEE, 2019, pp. 1–6.
  * [62] M. Almseidin, A. A. Zuraiq, M. Al-Kasassbeh, N. Alnidami, Phishing detection based on machine learning and feature selection methods, International Association of Online Engineering.
  * [63] A. Abuzuraiq, M. Alkasassbeh, M. Almseidin, Intelligent methods for accurately detecting phishing websites, in: 2020 11th International Conference on Information and Communication Systems (ICICS), IEEE, 2020, pp. 085–090.
  * [64] M. Almseidin, I. Piller, M. Al-Kasassbeh, S. Kovacs, Fuzzy automaton as a detection mechanism for the multi-step attack, International Journal on Advanced Science, Engineering and Information Technology 9 (2) (2019) 575–586.
  * [65] M. Al-Kasassbeh, S. Mohammed, M. Alauthman, A. Almomani, Feature selection using a machine learning to classify a malware, in: Handbook of computer networks and cyber security, Springer, 2020, pp. 889–904.
  * [66] M. Almseidin, M. Al-Kasassbeh, S. Kovacs, Detecting slow port scan using fuzzy rule interpolation, in: 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), IEEE, 2019, pp. 1–6.
  * [67] Z. Alothman, M. Alkasassbeh, S. Al-Haj Baddar, An efficient approach to detect iot botnet attacks using machine learning, Journal of High Speed Networks 26 (3) (2020) 241–254.
  * [68] A. Rawashdeh, M. Alkasassbeh, M. Al-Hawawreh, An anomaly-based approach for ddos attack detection in cloud environment, International Journal of Computer Applications in Technology 57 (4) (2018) 312–324.
  * [69] M. Alkasassbeh, A novel hybrid method for network anomaly detection based on traffic prediction and change point detection, arXiv preprint arXiv:1801.05309.
  * [70] S. Wang, W. Jiang, K.-L. Tsui, Adjusted support vector machines based on a new loss function, Annals of Operations Research 174 (1) (2010) 83–101.
  * [71] A. B. Hassanat, A. S. Tarawneh, S. S. Abed, G. A. Altarawneh, M. Alrashidi, M. Alghamdi, Rdpvr: Random data partitioning with voting rule for machine learning from class-imbalanced datasets, Electronics 11 (2) (2022) 228.
  * [72] A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, F. Herrera, Learning from imbalanced data sets, Vol. 10, Springer, 2018.
  * [73] P. Branco, L. Torgo, R. P. Ribeiro, A survey of predictive modeling on imbalanced domains, ACM Computing Surveys (CSUR) 49 (2) (2016) 1–50.
  * [74] K. K. Hauner, R. E. Zinbarg, W. Revelle, A latent variable model approach to estimating systematic bias in the oversampling method, Behavior Research Methods 46 (3) (2014) 786–797.
  * [75] M. Al-Nashashibi, W. Hadi, N. El-Khalili, G. Issa, A. AlBanna, A new two-step ensemble learning model for improving stress prediction of automobile drivers, The International Arab Journal of Information Technology 18 (6) (2021) 819–829.
  * [76] P. Fergus, M. Selvaraj, C. Chalmers, Machine learning ensemble modelling to classify caesarean section and vaginal delivery types using cardiotocography traces, Computers in biology and medicine 93 (2018) 7–16.
  * [77] G. Lemaître, F. Nogueira, C. K. Aridas, Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning, The Journal of Machine Learning Research 18 (1) (2017) 559–563.
  * [78] G. Kovács, Smote-variants: A python implementation of 85 minority oversampling techniques, Neurocomputing 366 (2019) 352–354.
  * [79] G. Haixiang, L. Yijing, J. Shang, G. Mingyun, H. Yuanyue, G. Bing, Learning from class-imbalanced data: Review of methods and applications, Expert Systems with Applications 73 (2017) 220–239.
  * [80] Y. Sun, A. K. Wong, M. S. Kamel, Classification of imbalanced data: A review, International journal of pattern recognition and artificial intelligence 23 (04) (2009) 687–719.
  * [81] J. Tanha, Y. Abdi, N. Samadi, N. Razzaghi, M. Asadpour, Boosting methods for multi-class imbalanced data classification: an experimental review, Journal of Big Data 7 (1) (2020) 1–47.
  * [82] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: synthetic minority over-sampling technique, Journal of artificial intelligence research 16 (2002) 321–357.
  * [83] C. Drummond, R. C. Holte, et al., C4. 5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling, in: Workshop on learning from imbalanced datasets II, Vol. 11, Citeseer, 2003, pp. 1–8.
  * [84] Y. Tang, Y.-Q. Zhang, N. V. Chawla, S. Krasser, Svms modeling for highly imbalanced classification, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39 (1) (2008) 281–288.
  * [85] H. Han, W.-Y. Wang, B.-H. Mao, Borderline-smote: a new over-sampling method in imbalanced data sets learning, in: International conference on intelligent computing, Springer, 2005, pp. 878–887.
  * [86] R. Das, S. K. Biswas, D. Devi, B. Sarma, An oversampling technique by integrating reverse nearest neighbor in smote: Reverse-smote, in: 2020 International Conference on Smart Electronics and Communication (ICOSEC), IEEE, 2020, pp. 1239–1244.
  * [87] C. Liu, S. Jin, D. Wang, Z. Luo, J. Yu, B. Zhou, C. Yang, Constrained oversampling: An oversampling approach to reduce noise generation in imbalanced datasets with class overlapping, IEEE Access.
  * [88] S. Barua, M. M. Islam, X. Yao, K. Murase, Mwmote–majority weighted minority oversampling technique for imbalanced data set learning, IEEE Transactions on knowledge and data engineering 26 (2) (2012) 405–425.
  * [89] H. He, Y. Bai, E. A. Garcia, S. Li, Adasyn: Adaptive synthetic sampling approach for imbalanced learning, in: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), IEEE, 2008, pp. 1322–1328.
  * [90] C. Bellinger, S. Sharma, N. Japkowicz, O. R. Zaïane, Framework for extreme imbalance classification: Swim—sampling with the majority class, Knowledge and Information Systems (2019) 1–26.
  * [91] C. Tian, L. Zhou, S. Zhang, Y. Zhao, A new majority weighted minority oversampling technique for classification of imbalanced datasets, in: 2020 International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), IEEE, 2020, pp. 154–157.
  * [92] P. Domingos, Metacost: A general method for making classifiers cost-sensitive, in: Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, 1999, pp. 155–164.
  * [93] Y. E. Kurniawati, A. E. Permanasari, S. Fauziati, Adaptive synthetic-nominal (adasyn-n) and adaptive synthetic-knn (adasyn-knn) for multiclass imbalance learning on laboratory test data, in: 2018 4th International Conference on Science and Technology (ICST), IEEE, 2018, pp. 1–6.
  * [94] W. Zhang, R. Ramezani, A. Naeim, Wotboost: Weighted oversampling technique in boosting for imbalanced learning, in: 2019 IEEE International Conference on Big Data (Big Data), IEEE, 2019, pp. 2523–2531.
  * [95] B. S. Raghuwanshi, S. Shukla, Smote based class-specific extreme learning machine for imbalanced learning, Knowledge-Based Systems 187 (2020) 104814.
  * [96] G. Douzas, F. Bacao, Self-organizing map oversampling (somo) for imbalanced data set learning, Expert systems with Applications 82 (2017) 40–52.
  * [97] G. A. Pradipta, R. Wardoyo, A. Musdholifah, I. N. H. Sanjaya, Radius-smote: A new oversampling technique of minority samples based on radius distance for learning from imbalanced data, IEEE Access 9 (2021) 74763–74777.
  * [98] B. Krawczyk, M. Koziarski, M. Woźniak, Radial-based oversampling for multiclass imbalanced data classification, IEEE transactions on neural networks and learning systems 31 (8) (2019) 2818–2831.
  * [99] J. Hong, H. Kang, T. Hong, Oversampling-based prediction of environmental complaints related to construction projects with imbalanced empirical-data learning, Renewable and Sustainable Energy Reviews 134 (2020) 110402.
  * [100] M. H. IBRAHIM, Odbot: Outlier detection-based oversampling technique for imbalanced datasets learning, Neural Computing and Applications (2021) 1–26.
  * [101] L. Wang, H. Wang, G. Fu, Multiple kernel learning with minority oversampling for classifying imbalanced data, IEEE Access 9 (2020) 565–580.
  * [102] S. Bej, N. Davtyan, M. Wolfien, M. Nassar, O. Wolkenhauer, Loras: An oversampling approach for imbalanced datasets, Machine Learning 110 (2) (2021) 279–301.
  * [103] T. Zhu, Y. Lin, Y. Liu, Improving interpolation-based oversampling for imbalanced data learning, Knowledge-Based Systems 187 (2020) 104826.
  * [104] G. Douzas, F. Bacao, F. Last, Improving imbalanced learning through a heuristic oversampling method based on k-means and smote, Information Sciences 465 (2018) 1–20.
  * [105] H. Faris, R. Abukhurma, W. Almanaseer, M. Saadeh, A. M. Mora, P. A. Castillo, I. Aljarah, Improving financial bankruptcy prediction in a highly imbalanced class distribution using oversampling and ensemble learning: a case from the spanish market, Progress in Artificial Intelligence 9 (1) (2020) 31–53.
  * [106] Z. Jiang, J. Yang, Y. Liu, Imbalanced learning with oversampling based on classification contribution degree, Advanced Theory and Simulations 4 (5) (2021) 2100031.
  * [107] G. Douzas, F. Bacao, J. Fonseca, M. Khudinyan, Imbalanced learning in land cover classification: Improving minority classes’ prediction accuracy using the geometric smote algorithm, Remote Sensing 11 (24) (2019) 3040.
  * [108] Y. Zhang, X. Li, L. Gao, L. Wang, L. Wen, Imbalanced data fault diagnosis of rotating machinery using synthetic oversampling and feature learning, Journal of manufacturing systems 48 (2018) 34–50.
  * [109] Z. Wang, H. Wang, Global data distribution weighted synthetic oversampling technique for imbalanced learning, IEEE Access 9 (2021) 44770–44783.
  * [110] G. Liu, Y. Yang, B. Li, Fuzzy rule-based oversampling technique for imbalanced and incomplete data learning, Knowledge-Based Systems 158 (2018) 154–174.
  * [111] X. Wu, Y. Yang, L. Ren, Entropy difference and kernel-based oversampling technique for imbalanced data learning, Intelligent Data Analysis 24 (6) (2020) 1239–1255.
  * [112] J. Engelmann, S. Lessmann, Conditional wasserstein gan-based oversampling of tabular data for imbalanced learning, arXiv preprint arXiv:2008.09202.
  * [113] Q. Li, G. Li, W. Niu, Y. Cao, L. Chang, J. Tan, L. Guo, Boosting imbalanced data learning with wiener process oversampling, Frontiers of Computer Science 11 (5) (2017) 836–851.
  * [114] C.-R. Wang, X.-H. Shao, An improving majority weighted minority oversampling technique for imbalanced classification problem, IEEE Access 9 (2020) 5069–5082.
  * [115] R. Malhotra, S. Kamal, An empirical study to investigate oversampling methods for improving software defect prediction using imbalanced data, Neurocomputing 343 (2019) 120–140.
  * [116] G. Kovács, An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets, Applied Soft Computing 83 (2019) 105662.
  * [117] M. R. K. Dhurjad, M. Banait, A survey on oversampling techniques for imbalanced learning, Int. J. Appl. Innov. Eng. Manage. 3 (1) (2014) 279–284.
  * [118] J. Li, Q. Zhu, Q. Wu, Z. Fan, A novel oversampling technique for class-imbalanced learning based on smote and natural neighbors, Information Sciences 565 (2021) 438–455.
  * [119] Z. Jiang, T. Pan, C. Zhang, J. Yang, A new oversampling method based on the classification contribution degree, Symmetry 13 (2) (2021) 194.
  * [120] G. E. Batista, R. C. Prati, M. C. Monard, A study of the behavior of several methods for balancing machine learning training data, ACM SIGKDD explorations newsletter 6 (1) (2004) 20–29.
  * [121] J. Wang, M. Xu, H. Wang, J. Zhang, Classification of imbalanced data by using the smote algorithm and locally linear embedding, in: 2006 8th international Conference on Signal Processing, Vol. 3, IEEE, 2006, pp. 1–4.
  * [122] J. De La Calleja, O. Fuentes, A distance-based over-sampling method for learning from imbalanced data sets., in: FLAIRS Conference, 2007, pp. 634–635.
  * [123] S. Gazzah, N. E. B. Amara, New oversampling approaches based on polynomial fitting for imbalanced data sets, in: 2008 the eighth iapr international workshop on document analysis systems, IEEE, 2008, pp. 677–684.
  * [124] J. Stefanowski, S. Wilk, Selective pre-processing of imbalanced data for improving classification performance, in: International Conference on Data Warehousing and Knowledge Discovery, Springer, 2008, pp. 283–292.
  * [125] S. Tang, S.-P. Chen, The generation mechanism of synthetic minority class examples, in: 2008 International Conference on Information Technology and Applications in Biomedicine, IEEE, 2008, pp. 444–447.
  * [126] C. Bunkhumpornpat, K. Sinapiromsaran, C. Lursinsap, Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem, in: Pacific-Asia conference on knowledge discovery and data mining, Springer, 2009, pp. 475–482.
  * [127] S. Hu, Y. Liang, L. Ma, Y. He, Msmote: Improving classification performance when training data is imbalanced, in: 2009 second international workshop on computer science and engineering, Vol. 2, IEEE, 2009, pp. 13–17.
  * [128] L. Chen, Z. Cai, L. Chen, Q. Gu, A novel differential evolution-clustering hybrid resampling algorithm on imbalanced datasets, in: 2010 Third International Conference on Knowledge Discovery and Data Mining, IEEE, 2010, pp. 81–85.
  * [129] S. Wang, Z. Li, W. Chao, Q. Cao, Applying adaptive over-sampling technique based on data density and cost-sensitive svm to imbalanced learning, in: The 2012 International Joint Conference on Neural Networks (IJCNN), IEEE, 2012, pp. 1–8.
  * [130] S. Cateni, V. Colla, M. Vannucci, Novel resampling method for the classification of imbalanced datasets for industrial and other real-world problems, in: 2011 11th International Conference on Intelligent Systems Design and Applications, IEEE, 2011, pp. 402–407.
  * [131] X. Fan, K. Tang, T. Weise, Margin-based over-sampling method for learning from imbalanced datasets, in: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer, 2011, pp. 309–320.
  * [132] M. A. H. Farquad, I. Bose, Preprocessing unbalanced data using support vector machine, Decision Support Systems 53 (1) (2012) 226–233.
  * [133] K. Puntumapon, K. Waiyamai, A pruning-based approach for searching precise and generalized region for synthetic minority over-sampling, in: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer, 2012, pp. 371–382.
  * [134] E. Ramentol, Y. Caballero, R. Bello, F. Herrera, Smote-rs b*: a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using smote and rough sets theory, Knowledge and information systems 33 (2) (2012) 245–265.
  * [135] S. Barua, M. M. Islam, K. Murase, Prowsyn: Proximity weighted synthetic oversampling technique for imbalanced data set learning, in: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer, 2013, pp. 317–328.
  * [136] C. Bunkhumpornpat, S. Subpaiboonkit, Safe level graph for synthetic minority over-sampling techniques, in: 2013 13th International Symposium on Communications and Information Technologies (ISCIT), IEEE, 2013, pp. 570–575.
  * [137] M. Nakamura, Y. Kajiwara, A. Otsuka, H. Kimura, Lvq-smote–learning vector quantization based synthetic minority over–sampling technique for biomedical data, BioData mining 6 (1) (2013) 1–10.
  * [138] A. I. Sanchez, E. F. Morales, J. A. Gonzalez, Synthetic oversampling of instances using clustering, International Journal on Artificial Intelligence Tools 22 (02) (2013) 1350008.
  * [139] G. Menardi, N. Torelli, Training and assessing classification rules with imbalanced data, Data mining and knowledge discovery 28 (1) (2014) 92–122.
  * [140] F. Koto, Smote-out, smote-cosine, and selected-smote: An enhancement strategy to handle imbalance in data level, in: 2014 International Conference on Advanced Computer Science and Information System, IEEE, 2014, pp. 280–284.
  * [141] T. Maciejewski, J. Stefanowski, Local neighbourhood extension of smote for mining imbalanced data, in: 2011 IEEE symposium on computational intelligence and data mining (CIDM), IEEE, 2011, pp. 104–111.
  * [142] M. Gao, X. Hong, S. Chen, C. J. Harris, E. Khalaf, Pdfos: Pdf estimation based over-sampling for imbalanced two-class problems, Neurocomputing 138 (2014) 248–259.
  * [143] H. Zhang, M. Li, Rwo-sampling: A random walk over-sampling approach to imbalanced data classification, Information Fusion 20 (2014) 99–116.
  * [144] B. A. Almogahed, I. A. Kakadiaris, Neater: filtering of over-sampled data using non-cooperative game theory, Soft Computing 19 (11) (2015) 3301–3322.
  * [145] C. Bellinger, N. Japkowicz, C. Drummond, Synthetic oversampling for advanced radioactive threat detection, in: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), IEEE, 2015, pp. 948–953.
  * [146] S. Gazzah, A. Hechkel, N. E. B. Amara, A hybrid sampling method for imbalanced data, in: 2015 IEEE 12th International Multi-Conference on Systems, Signals & Devices (SSD15), IEEE, 2015, pp. 1–6.
  * [147] L. Jiang, C. Qiu, C. Li, A novel minority cloning technique for cost-sensitive learning, International Journal of Pattern Recognition and Artificial Intelligence 29 (04) (2015) 1551004.
  * [148] J. A. Sáez, J. Luengo, J. Stefanowski, F. Herrera, Smote–ipf: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering, Information Sciences 291 (2015) 184–203.
  * [149] B. Tang, H. He, Kerneladasyn: Kernel based adaptive synthetic data generation for imbalanced learning, in: 2015 IEEE Congress on Evolutionary Computation (CEC), IEEE, 2015, pp. 664–671.
  * [150] Z. Xie, L. Jiang, T. Ye, X. Li, A synthetic minority oversampling method based on local densities in low-dimensional space for imbalanced learning, in: International Conference on Database Systems for Advanced Applications, Springer, 2015, pp. 3–18.
  * [151] W. A. Young, S. L. Nykl, G. R. Weckman, D. M. Chelberg, Using voronoi diagrams to improve classification performances when modeling imbalanced datasets, Neural Computing and Applications 26 (5) (2015) 1041–1054.
  * [152] W. A. Rivera, P. Xanthopoulos, A priori synthetic over-sampling methods for increasing classification sensitivity in imbalanced data sets, Expert Systems with Applications 66 (2016) 124–135.
  * [153] F. R. Torres, J. A. Carrasco-Ochoa, J. F. Martínez-Trinidad, Smote-d a deterministic version of smote, in: Mexican Conference on Pattern Recognition, Springer, 2016, pp. 177–188.
  * [154] J. Cervantes, F. Garcia-Lamont, L. Rodriguez, A. López, J. R. Castilla, A. Trueba, Pso-based method for svm classification on skewed data sets, Neurocomputing 228 (2017) 187–197.
  * [155] L. Ma, S. Fan, Cure-smote algorithm and hybrid algorithm for feature selection and parameter optimization based on random forests, BMC bioinformatics 18 (1) (2017) 1–18.
  * [156] S. Chen, G. Guo, L. Chen, A new over-sampling method based on cluster ensembles, in: 2010 IEEE 24th International Conference on Advanced Information Networking and Applications Workshops, IEEE, 2010, pp. 599–604.
  * [157] Y.-I. Kang, S. Won, Weight decision algorithm for oversampling technique on class-imbalanced learning, in: ICCAS 2010, IEEE, 2010, pp. 182–186.
  * [158] S. Barua, M. M. Islam, K. Murase, A novel synthetic minority oversampling technique for imbalanced data set learning, in: International Conference on Neural Information Processing, Springer, 2011, pp. 735–744.
  * [159] B. Zhou, C. Yang, H. Guo, J. Hu, A quasi-linear svm combined with assembled smote for imbalanced data classification, in: The 2013 International Joint Conference on Neural Networks (IJCNN), IEEE, 2013, pp. 1–7.
  * [160] K. Li, W. Zhang, Q. Lu, X. Fang, An improved smote imbalanced data classification method based on support degree, in: 2014 International Conference on Identification, Information and Knowledge in the Internet of Things, IEEE, 2014, pp. 34–38.
  * [161] S. Mahmoudi, P. Moradi, F. Akhlaghian, R. Moradi, Diversity and separable metrics in over-sampling technique for imbalanced data classification, in: 2014 4th International Conference on Computer and Knowledge Engineering (ICCKE), IEEE, 2014, pp. 152–158.
  * [162] T. Sandhan, J. Y. Choi, Handling imbalanced datasets by partially guided hybrid sampling for pattern recognition, in: 2014 22nd International Conference on Pattern Recognition, IEEE, 2014, pp. 1449–1453.
  * [163] Y. H. Xu, H. Li, L. P. Le, X. Y. Tian, Neighborhood triangular synthetic minority over-sampling technique for imbalanced prediction on small samples of chinese tourism and hospitality firms, in: 2014 Seventh International Joint Conference on Computational Sciences and Optimization, IEEE, 2014, pp. 534–538.
  * [164] J. Lee, N.-r. Kim, J.-H. Lee, An over-sampling technique with rejection for imbalanced class learning, in: Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication, 2015, pp. 1–6.
  * [165] J. Li, S. Fong, Y. Zhuang, Optimizing smote by metaheuristics with neural network and decision tree, in: 2015 3rd International Symposium on Computational and Business Intelligence (ISCBI), IEEE, 2015, pp. 26–32.
  * [166] L. Abdi, S. Hashemi, To combat multi-class imbalanced problems by means of over-sampling techniques, IEEE transactions on Knowledge and Data Engineering 28 (1) (2015) 238–251.
  * [167] L. Abdi, S. Hashemi, To combat multi-class imbalanced problems by means of over-sampling and boosting techniques, Soft Computing 19 (12) (2015) 3369–3385.
  * [168] Y. Dong, X. Wang, A new over-sampling approach: random-smote for learning from imbalanced data sets, in: International Conference on Knowledge Science, Engineering and Management, Springer, 2011, pp. 343–352.
  * [169] K. Borowska, J. Stepaniuk, Imbalanced data classification: A novel re-sampling approach combining versatile improved smote and rough sets, in: IFIP International Conference on Computer Information Systems and Industrial Management, Springer, 2016, pp. 31–42.
  * [170] K. Jiang, J. Lu, K. Xia, A novel algorithm for imbalance data classification based on genetic algorithm improved smote, Arabian journal for science and engineering 41 (8) (2016) 3255–3266.
  * [171] E. Ramentol, I. Gondres, S. Lajes, R. Bello, Y. Caballero, C. Cornelis, F. Herrera, Fuzzy-rough imbalanced learning for the diagnosis of high voltage circuit breaker maintenance: The smote-frst-2t algorithm, Engineering Applications of Artificial Intelligence 48 (2016) 134–139.
  * [172] J. Yun, J. Ha, J.-S. Lee, Automatic determination of neighborhood size in smote, in: Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication, 2016, pp. 1–8.
  * [173] W. A. Rivera, Noise reduction a priori synthetic over-sampling for class imbalanced data sets, Information Sciences 408 (2017) 146–161.
  * [174] J. Li, S. Fong, R. K. Wong, V. W. Chu, Adaptive multi-objective swarm fusion for imbalanced data classification, Information Fusion 39 (2018) 1–24.
  * [175] T. Rong, H. Gong, W. W. Ng, Stochastic sensitivity oversampling technique for imbalanced data, in: International conference on machine learning and cybernetics, Springer, 2014, pp. 161–171.
  * [176] L. Zhang, W. Wang, A re-sampling method for class imbalance learning with credit data, in: 2011 International Conference of Information Technology, Computer Engineering and Management Sciences, Vol. 1, IEEE, 2011, pp. 393–397.
  * [177] F. Fernández-Navarro, C. Hervás-Martínez, P. A. Gutiérrez, A dynamic over-sampling procedure based on sensitivity for multi-class problems, Pattern Recognition 44 (8) (2011) 1821–1833.
  * [178] J. Hu, X. He, D.-J. Yu, X.-B. Yang, J.-Y. Yang, H.-B. Shen, A new supervised over-sampling algorithm with application to protein-nucleotide binding residue prediction, PloS one 9 (9) (2014) e107676.
  * [179] V. García, J. S. Sánchez, R. Martín-Félez, R. A. Mollineda, Surrounding neighborhood-based smote for learning from imbalanced data sets, Progress in Artificial Intelligence 1 (4) (2012) 347–362.
  * [180] M. Koziarski, M. Woźniak, Ccr: A combined cleaning and resampling algorithm for imbalanced data classification, International Journal of Applied Mathematics and Computer Science 27 (4).
  * [181] W. Siriseriwan, K. Sinapiromsaran, Adaptive neighbor synthetic minority oversampling technique under 1nn outcast handling, Songklanakarin J. Sci. Technol 39 (5) (2017) 565–576.
  * [182] D. A. Cieslak, N. V. Chawla, A. Striegel, Combating imbalance in network intrusion datasets., in: GrC, Citeseer, 2006, pp. 732–737.
  * [183] A. B. Hassanat, Dimensionality invariant similarity measure, Journal of American Science 10 (8) (2014) 221–226.
  * [184] H. A. Abu Alfeilat, A. B. Hassanat, O. Lasassmeh, A. S. Tarawneh, M. B. Alhasanat, H. S. Eyal Salman, V. S. Prasath, Effects of distance measure choice on k-nearest neighbor classifier performance: a review, Big data 7 (4) (2019) 221–248.
  * [185] R. Ehsani, F. Drabløs, Robust distance measures for knn classification of cancer data, Cancer informatics 19 (2020) 1176935120965542.
  * [186] C. R. Kancharla, J. Vankeirsbilck, D. Vanoost, J. Boydens, H. Hallez, Latent dimensions of auto-encoder as robust features for inter-conditional bearing fault diagnosis, Applied Sciences 12 (3) (2022) 965.
  * [187] R. Veerachamy, R. Ramar, Agricultural irrigation recommendation and alert (aira) system using optimization and machine learning in hadoop for sustainable agriculture, Environmental Science and Pollution Research (2021) 1–20.
  * [188] M. Farooq, S. Sarfraz, C. Chesneau, M. U. Hassan, M. A. Raza, R. A. K. Sherwani, F. Jamal, Computing expectiles using k-nearest neighbours approach, Symmetry 13 (4) (2021) 645.
  * [189] D. Dua, C. Graff, Uci machine learning repository (2017).
URL http://archive.ics.uci.edu/ml

  * [190] X.-Y. Liu, J. Wu, Z.-H. Zhou, Exploratory undersampling for class-imbalance learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39 (2) (2008) 539–550.