# SLA2P: Self-supervised Anomaly Detection with Adversarial
Perturbation††thanks: The code will be made publicly available after the
acceptance of the paper for publication.

Yizhou Wang†, Can Qin†, Rongzhe Wei‡, Yi Xu†, Yue Bai† and Yun Fu†
† Northeastern University, ‡ Purdue University
<EMAIL_ADDRESS>{qin.ca, xu.yi<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Anomaly detection is a fundamental yet challenging problem in machine learning
due to the lack of label information. In this work, we propose a novel and
powerful framework, dubbed as SLA2P, for unsupervised anomaly detection. After
extracting representative embeddings from raw data, we apply random
projections to the features and regard features transformed by different
projections as belonging to distinct pseudo classes. We then train a
classifier network on these transformed features to perform self-supervised
learning. Next we add adversarial perturbation to the transformed features to
decrease their softmax scores of the predicted labels and design anomaly
scores based on the predictive uncertainties of the classifier on these
perturbed features. Our motivation is that because of the relatively small
number and the decentralized modes of anomalies, 1) the pseudo label
classifier’s training concentrates more on learning the semantic information
of normal data rather than anomalous data; 2) the transformed features of the
normal data are more robust to the perturbations than those of the anomalies.
Consequently, the perturbed transformed features of anomalies fail to be
classified well and accordingly have lower anomaly scores than those of the
normal samples. Extensive experiments on image, text and inherently tabular
benchmark datasets back up our findings and indicate that SLA2P achieves
state-of-the-art results on unsupervised anomaly detection tasks consistently.

##  1 Introduction

Anomalies, also known as outliers, are defined as “data instances that
significantly deviate from the majority of data instances” (Pang et al.,
2020a). Correspondingly, anomaly detection (AD) refers to the process of
finding these anomalous data points out in a data-driven fashion, which has
long been a fundamental problem in machine learning and has various real-world
applications, including medical health (Min et al., 2017; Khan and Yairi,
2018), fraud detection (Adewumi and Akinyelu, 2017; Shen et al., 2007; Fu et
al., 2016), cybersecurity (Tan et al., 2011; Kwon et al., 2019) and video
surveillance (Chen et al., 2015; Sultani et al., 2018) etc. In view of whether
and to what degree labels are available, anomaly detection tasks can be
generally classified into three categories: 1) Supervised anomaly detection
involves training models on a labeled dataset consisting of both inliers and
outliers and then applying them to test data. 2) Semi-supervised anomaly
detection (SSAD), or one-class classification, deals with the setting that the
training dataset is only composed of normal data and the trained model is
supposed to detect anomalous data in the testing phase. 3) Unsupervised
anomaly detection (UAD), which is the most common and challenging case, obeys
the condition that solely unlabeled data with both inliers and outliers are
provided and the anomaly detection technique is required to be capable of
detecting the outliers (Chandola et al., 2009). These three categories
consider anomalies as points that are intrinsically from distinct classes from
normal points’. Besides, there exists another type of AD called out-of-
distribution (OOD) detection (Liang et al., 2018; Hendrycks et al., 2019; Hsu
et al., 2020; Liu et al., 2020), which aims to distinguish samples that have
disjoint distribution from training samples (usually from a different
dataset).

We mainly focus on the unsupervised setting, i.e., UAD, which is most widely
applicable because it is costly to obtain labels for AD in many real-world
scenarios (Pang et al., 2020a; Chalapathy and Chawla, 2019; Chandola et al.,
2007). UAD has wide-range applications in practice, including large-scale
dataset construction, website management and news management etc. We must
clarify that in some literatures, the so-called “unsupervised anomaly
detection” actually refers to SSAD (e.g., (Somepalli et al., 2020; Kwon et
al., 2020; Venkataramanan et al., 2019)) or OOD (e.g., (Schirrmeister et al.,
2020)) by our definition, which is beyond the scope of this paper.

In this work, we introduce a novel SeLf-supervised framework for unsupervised
Anomaly Detection using Adversarial Perturbation, which we name as SLA2P. With
the extracted embeddings from unlabeled raw data, we project them into
different subspaces by multiplicating random matrices. Despite having no
explicit idea of the subspaces, we deem the transformation to be digging
various unknown aspects of latent information of the data. Then we train a
deep neural network (DNN) classifier on the transformed features in order to
distinguish which subspace they are projected into. The training process of
the classifier is equipped with early stopping technique to prevent
overfitting and the reason behind this is that outliers fail to be trained
well owing to their smaller population size and more various modes. The
trained classifier is supposed to mostly learn useful information concerning
characteristics of the normal data, hence its predictive uncertainties can be
utilized to design anomaly scores, with anomalies having more uniform
predictions while normal instances having sharper predictions. Such prediction
distribution disparities can be further amplified by adding adversarial
perturbations to these transformed features. The aim of the perturbations is
to decrease the softmax scores of the predicted labels of the transformed
features by the classifier. The practice boosts UAD performance empirically,
and we are not aware of any other analogous approach in the anomaly detection
field. To summarize, our contributions are four-fold:

  * •

We propose a novel framework SLA2P for UAD, which can be applied to image,
text and inherently tabular datasets. Our approach is the first attempt to use
feature-level transformation to create pretext classification task as
surrogate supervision for UAD task .

  * •

We investigate UAD task by involving adversarial perturbation technique in a
self-supervised fashion to achieve considerable performance improvement.

  * •

Our method achieves state-of-the-art (SOTA) results on $7$ challenging
datasets, surpassing the current best methods by a considerable margin.
Besides, it is higly robust and can maintain excellent detection performance
under particularly high or small anomaly ratios.

  * •

We further theoretically analyze SLA2P by illustrating the “similarity-
preserving” property of random projections, which bridges image-level and
feature-level transformation based self-supervised methods.

##  2 Related Works

#### Unsupervised anomaly detection.

Traditional methods for UAD are mostly based on classic unsupervised learning
tools, including density estimation methods (Breunig et al., 2000; Yang et
al., 2009; Kim and Scott, 2012), clustering methods (Ester et al., 1996; He et
al., 2003), dimension reduction methods (Shyu et al., 2003; Paffenroth et al.,
2018) and one-class support vector machine methods (Schölkopf et al., 2000;
Amer et al., 2013). Thanks to the marvelous representation ability of DNNs,
many reconstruction-based methods using DNNs are developed for UAD in recent
years. They mainly employ deep generative models (Zenati et al., 2018; Perera
et al., 2019; Lai et al., 2020b) or autoencoders (AE) (Chen et al., 2017;
Pidhorskyi et al., 2018; Abati et al., 2019) to reconstruct data and determine
abnormality of data via its reconstruction error. For instance, DAGMM (Zong et
al., 2018) feed the latent representations of the AE into a gaussian mixture
model and jointly optimize them. RSRAE (Lai et al., 2020a) incorporates a
robust PCA layer (Lerman and Maunu, 2018) into deep AEs, which is designed to
project normal data into their subspace while leaving anomalies out. Sound as
these carefully designed architectures seem, it is inevitable that such
reconstruction-based methods focus more on low-level or element-wise error
rather than high-level semantic information, as pointed out by Wang et al.
(2019).

#### Anomaly detection with pretrained networks.

Transferring discriminative embeddings of pretrained nets to AD has been
widely studied and achieved great success (Sabokrou et al., 2018; Rippel et
al., 2020; Ruff et al., 2020). (Andrews et al., 2016) shows that transfer-
representation-learning approaches offer viable representations for AD tasks
without prior knowledge of the data. Methods in (Kumagai et al., 2019;
Vercruyssen et al., 2020) improve AD performance on target domains via
transferring information of related domains. Burlina et al. (2019) employs
pretrained VGGNets (Simonyan and Zisserman, 2015) to perform novelty
detection. Recently DNNs pretrained on ImageNet have been used to extract
features for unsupervised anomaly detection and segmentation on images
(Bergmann et al., 2019, 2020; Bergman et al., 2020; Venkataramanan et al.,
2019) and videos (Nazare et al., 2018; Pang et al., 2020b).

#### Self-supervised learning.

Self-supervised Learning (SSL) has been an increasingly prevailing
unsupervised learning method. SSL methods learn representations via designing
a pretext task between inputs and self-defined signals (Bojanowski and Joulin,
2017; Gidaris et al., 2018) or contrastive learning (Oord et al., 2018; Chen
et al., 2020). Employing SSL techniques to assist semi-supervised AD has shown
promising results recently (Golan and El-Yaniv, 2018; Bergman and Hoshen,
2020; Tack et al., 2020; Sehwag et al., 2021). In UAD task, E3Outlier (Wang et
al., 2019) trains a discriminitive DNN via SSL and use the network outputs of
the DNN to design anomaly scores. Nevertheless, such practice can only be
applied to image data and require manually pre-defined transformations. By
contrast, our method can be applied to both image and tabular data, and in our
framework random transformations are able to exhibit promising performance.

#### Input perturbation.

Input adversarial perturbation technique is first proposed by (Goodfellow et
al., 2014) to generate adversarial data samples to fool the classifier. In OOD
detection, several works utilize the opposite perturbation to the input raw
data using the gradient of the maximum softmax score of the predicted label
attained from pretrained network (Liang et al., 2018; Hsu et al., 2020). Lee
et al. (2018) add perturbation to increase the proposed Mahalanobis distance-
based confidence score for OOD detection. In anomaly/outlier detection area,
there are few efforts in utilizing input perturbations to enhance AD
performance as we can not directly utilize the softmax outputs of some
pretrained network. To the best of our knowledge, we make the first attempt to
wisely combine perturbation technique with self-supervised learning to address
AD problems.

Figure 1: Overview of our proposed framework SLA2P on image data. Given
unlabeled image data with both inliers and outliers, we first extract
representative features from raw data and normalize them into a unit sphere
vector space (features of anomalies are shaded). We then apply random
projections to the embeddings by multiplicating matrices sampled randomly from
standard normal distribution. Transformations by different matrices give rise
to pseudo labels, on which we train a DNN classifier. Next we adversarially
perturb the transformed samples using the gradients of the softmax scores of
the predicted labels of the classifier. Finally, the anomaly scores are
generated leveraging the predictive uncertainty estimates of the network on
the perturbed transformed features. Best viewed in color.

##  3 Problem Statement

We consider data space $\mathcal{X}\subset\mathbb{R}^{d}$ and we are given an
unlabeled dataset $X\subset\mathcal{X}$ . $X$ contains both normal data
(inliers) $X_{\text{in}}$ of size $n$ and anomalous data (outliers)
$X_{\text{out}}$ of size $np$, where $0<p<1$ is the anomaly rate defined as
the ratio of anomalies to normal data. The essential objective of UAD is to
design a universal indicator function
$I(\mathbf{x}):\mathcal{X}\rightarrow\\{0,1\\}$ such that $I(\mathbf{x})=1$
for $\mathbf{x}\in X_{\text{in}}$ and $I(\mathbf{x})=0$ for $\mathbf{x}\in
X_{\text{out}}$. However, directly pursuing such function is hard and
inefficient, as there is a trade-off between type-I error (rate of normal
samples which are classified as anomalies) and type-II error (rate of
anomalous samples that are classified as normal) (Golan and El-Yaniv, 2018).
The standard practice to deal with this problem is to instead design a scoring
function $S(\mathbf{x}):\mathcal{X}\rightarrow\mathbb{R}$ s.t. higher scores
indicate more normality while lower scores indicate more anomality.

##  4 Method

Our proposed framework SLA2P is illustrated in Figure 1, which can be
partitioned into three sub-processes.

###  4.1 Feature Extraction with L2 normalization

Given a raw datum $\mathbf{x}$, we employ a feature extractor $\mathbf{f}$ to
vectorize it into its discriminative embedding
$\mathbf{f}(\mathbf{x})\in\mathbb{R}^{d}$. The choice of $f$ depends on the
data type. We use a pretrained DNN for images and standard text pre-processing
methods for text data. For inherently tabular data, the raw data is qualified
to serve as its embedding, i.e., choosing $\mathbf{f}$ to be identity mapping
is enough. The detailed setting for different datasets is elaborated in Sec.
6. After feature extraction, we apply L2 normalization to the embeddings:

$\mathbf{v}(\mathbf{x})=\frac{\mathbf{f}(\mathbf{x})}{\left\|\mathbf{f}(\mathbf{x})\right\|_{2}}.$

L2 normalization projects the embeddings into a sphere in $\mathbb{R}^{d}$,
which only reduces the feature dimension by one but eliminates the differences
of magnitudes (norms) among the unlabeled data (Gu et al., 2020). Our idea is
that anomalies differ from normal data in intrinsic and latent distribution,
not magnitudes. Therefore the normalization step is supposed to help the
subsequent modules of SLA2P to concentrate on semantic level discrepancies and
attenuate low-level variances between normal and anomalous data. Our
theoretical argument also corroborates its effectiveness in that the proof of
our similarity perserving Theorem 1 requires the L2-norm of the processed
vectors to be $1$ (see Sec. 5 for more details).

###  4.2 Feature-level Self-supervised Learning

#### Motivation from geometric transformations of images.

Geometric transformations as self-supervision have obtained tremendous success
on image unsupervised learning tasks (Golan and El-Yaniv, 2018; Gidaris et
al., 2018; Wang et al., 2019) and the core idea is that DNNs are considered to
be able to excavate useful information of image data if they are able to
recoginize their transformed patterns (Gidaris et al., 2018). Nevertheless,
the majority of data in machine learning tasks is in tabular form that does
not possess the same structure as 2D images, and there are not in-hand
“geometric transformations” for data in vector forms. In fact, geometric
transformations are usually performed by applying affine transformations to
homogeneous coordinates of images. Given a coordinate pair $(x,y)^{\top}$, it
is firstly represented as a three-dimensional vector by adding an additional
dimension with element 1. Then the new coordinates
$(x^{\prime},y^{\prime})^{\top}$ are yielded by multiplicating a
transformation matrix:

$\left(\begin{matrix}x^{\prime}\\\ y^{\prime}\\\
1\end{matrix}\right)=\left(\begin{matrix}a&b&c\\\ d&e&f\\\
0&0&1\end{matrix}\right)\left(\begin{matrix}x\\\ y\\\ 1\end{matrix}\right).$

When $a=\cos\theta,b=\sin\theta,d=-\sin\theta,e=\cos\theta,c=f=0$, the
manipulation performs rotation by angle $\theta$; when
$a=e=1,b=d=0,c=x_{0},f=y_{0}$, it performs translating image along x axis by
$x_{0}$ and along y axis by $y_{0}$.

Spurred by this, we propose to extend such kind of transformation to vector
form data via replacing the coordinate vector with the vectors themselves. In
this way, we can perform transformations in the feature-level to conduct self-
supervised learning. The designing of transformation matrix still remains a
problem since a rotation matrix of vectors does not have the same meanings as
a rotation matrix for image 2D coordinates. Fortunately, our experiments show
that utilizing random matrices with each element i.i.d. sampled from standard
normal distribution is enough to exhibit outstanding results. Next, we give a
detailed description of our method of using random matrices for self-
supervision.

#### Random matrix multiplications as surrogate supervision.

Let the random projection matrix set
$\mathcal{A}=\\{\mathbf{A}^{(1)},\mathbf{A}^{(2)},\cdots,\mathbf{A}^{(M)}\\}$.
For any $1\leq m\leq M$, $\mathbf{A}^{(m)}=(a_{ij})\in\mathbb{R}^{k\times d}$
($k\ll d$) are independent and identically distributed (i.i.d.) sampled and
the elements of the matrix are also i.i.d. sampled from standard normal
distribution

$a_{ij}\sim\mathcal{N}(0,1).$

We design the random transformation to be multiplying the aforementioned
random matrices to data embeddings:

$\mathbf{v}^{(m)}=\mathbf{A}^{(m)}\mathbf{v},$

thereby we have generated a self-labeled data set

$D_{\mathcal{A}}\triangleq\big{\\{}\big{(}\mathbf{v}^{(m)}(\mathbf{x}),m\big{)}~{}\big{|}~{}\mathbf{x}\in\mathcal{X},\mathbf{A}^{(m)}\in\mathcal{A}\big{\\}}.$

Here we treat $m$ as the pseudo label of $\mathbf{A}^{(m)}\mathbf{v}$, and
hence we have a self-defined classification dataset with $mn(1+p)$ transformed
features. The random projections not only project feature embeddings into
different spaces, but also perform dimension reduction. We observe that a
random matrix multiplication project one subspace or convex set in
$\mathbb{R}^{d}$ to another subspace or convex set in $\mathbb{R}^{k}$, as
claimed in Proposition 1. Considering that extracted features of normal data
tend to be in a certain subspace or convex set, Proposition 1 guarantees that
the transformed features of normal data are still in one subspace or convex
set.

######  Proposition 1.

For any real matrix $\mathbf{A}\in\mathbb{R}^{k\times d}$, $\mathcal{V}$ is a
linear subspace in $\mathbb{R}^{d}$ and $\mathcal{U}$ is a convex set in
$\mathbb{R}^{d}$, then $\mathbf{A}\mathcal{V}$ is a linear subspace in
$\mathbb{R}^{k}$ and $\mathbf{A}\mathcal{U}$ is a convex set in
$\mathbb{R}^{k}$.

#### Training pseudo label classifier.

It is straightforward that we can train a multi-class classifier
$C_{\bm{\theta}}$ with parameters $\bm{\theta}$ on the self-labeled dataset,
i.e., the classifier is supposed to classify the transformed features
$\\{\mathbf{v}^{(m)}(\mathbf{x}_{i})\\}$ into pseudo-class $m$ for every
$1\leq m\leq M$. The loss function is standard cross-entropy loss. We denote
the softmax output vector of $C_{\bm{\theta}}$ as
$\mathbf{y}(\cdot|\bm{\theta})$ with $m$th element
$y^{(m)}(\cdot|\bm{\theta})$ and then our self-defined pretext task can be
formulated as

$\min_{\bm{\theta}}\frac{1}{n(1+p)}\sum_{i=1}^{n(1+p)}\mathcal{L}(\mathbf{x}_{i}|\bm{\theta}),$

where

$\displaystyle\mathcal{L}(\mathbf{x}_{i}|\bm{\theta})$
$\displaystyle=-\frac{1}{M}\sum_{m=1}^{M}\log\Big{(}y^{(m)}\big{(}\mathbf{v}^{(m)}(\mathbf{x}_{i})|\bm{\theta}\big{)}\Big{)}$
$\displaystyle=-\frac{1}{M}\sum_{m=1}^{M}\log\Big{(}y^{(m)}\big{(}\mathbf{A}^{(m)}\mathbf{v}(\mathbf{x}_{i})|\bm{\theta}\big{)}\Big{)}.$

During the training stage, we employ an early stopping technique to refrain
from overfitting. We set a hyperparameter $\mu$ as classification accuracy
threshold such that the training stops as soon as the classification accuracy
of the classifier in the current batch reaches $\mu\in(0,1)$. This strategy
plays a vital role in our framework, since if the classifier $C_{\bm{\theta}}$
can distinguish all the transformed data the anomalous data fail to be
detected easily by the classification result.

###  4.3 Network Uncertainty Based Scoring with Adversarial Perturbation

We define our anomaly score function $S(\mathbf{x})$ based on the predictive
uncertainties of the classifier $C_{\theta}$. Our goal is to make normal data
have higher scores than anomalies. Instead of using the training results of
the original transformed samples like (Golan and El-Yaniv, 2018; Wang et al.,
2019) did, we add adversarial perturbations to the transformed features before
feeding them into $C_{\theta}$.

#### Adversarial perturbation to the transformed features.

We adversarially perturb the transformed features employing the gradient of
the negative log softmax score of the predicted class of the trained
classifier $C_{\theta}$ w.r.t. the input sample. Mathematically, for any
$1\leq m\leq M$ and $\mathbf{x}$, we let

$\widetilde{\mathbf{v}}^{m}(\mathbf{x})=\mathbf{v}^{(m)}(\mathbf{x})+\eta(-\nabla_{\mathbf{v}^{(m)}(\mathbf{x})}\log\widehat{y}^{(m)}(\mathbf{x}|\bm{\theta})),$
(1)

where
$\widehat{y}^{(m)}(\mathbf{x}|\bm{\theta})=\max_{i}y^{(i)}(\mathbf{v}^{(m)}(\mathbf{x})|\bm{\theta})$,
and $\eta$ is the perturbation magnitude. Considering $C_{\theta}$ has been
trained well on most transformed data samples, such practice aims to lower the
softmax score of the pseudo class with the highest prediction probability,
i.e., to make the projected features more difficult to classify. Note that our
adversarial perturbation form is different from (Goodfellow et al., 2014) as
we do not involve the pseudo labels when computing the gradient. Empirically,
we discover that after such operation the anomaly scores of both inliers and
outliers will decrease, but the inliers will be more robust to the
perturbations than the outliers.

#### Negative Brier Score.

Inspired by reconstruction-based AD methods (Chen et al., 2017; Zong et al.,
2018; Lai et al., 2020a) which employ L2 distance error of reconstruction and
input as score, we utilize Euclidean distance for scoring by considering the
one-hot encoding of the correct label as ground truth in the classification
problem:

$S(\mathbf{x})=-\frac{1}{M}\sum_{m=1}^{M}\left\|\mathbf{y}\big{(}\widetilde{\mathbf{v}}^{(m)}(\mathbf{x})|\bm{\theta}\big{)}-\mathbf{e}_{m}\right\|_{2}^{2},$

where $\mathbf{e}_{m}$ is the $m$-th canonical basis vector representing the
label vector of $m$-th pseudo class. $-S(\mathbf{x})$ refers to the mean
squared error of the prediction outputs of $C_{\theta}$ and ground truth one-
hot pseudo labels, which actually corresponds to a classic proper score rule
known as Brier Score (Brier et al., 1950). The ideal case is that the normal
data to have lower error and accordingly higher anomaly score.

##  5 Theoretical Foundation of SLA2P

Random projections do not manifestly preserve semantic information of data,
which contrasts with geometric transformations ( e.g., a rotated or translated
plane is still a plane, not a bird). Due to randomness, one may have no
explicit idea of what subspace the features are projected into and whether the
random transformations have an influence on the discrepancies among original
features. Intriguingly, we find that as long as the dimension $k$ is
sufficiently large, the L2 distance and inner product of transformed
embeddings, which are two effective measures of similarities between high-
dimensional vectors, are roughly in proportion to their original values by a
constant factor $k$.

######  Theorem 1 (Similarity-preserving Property).

Given any fixed $m$ and index pair of the unlabeled dataset $(i,j)$, for any
positive numbers $0<\epsilon<1$ and $0<\delta<1$,
(1) when $k>\frac{4\log\frac{2}{\delta}}{\epsilon^{2}-\epsilon^{3}}$, we have
with probability at least $1-\delta$ over the random sampling of matrix
$\mathbf{A}^{(m)}$,

$(1-\epsilon)\left\|\mathbf{v}_{i}-\mathbf{v}_{j}\right\|_{2}^{2}\leq\frac{\left\|\mathbf{v}^{(m)}_{i}-\mathbf{v}^{(m)}_{j}\right\|_{2}^{2}}{k}\leq(1+\epsilon)\left\|\mathbf{v}_{i}-\mathbf{v}_{j}\right\|_{2}^{2},$

(2) when $k>\frac{4\log\frac{4}{\delta}}{\epsilon^{2}-\epsilon^{3}}$, we have
with probability at least $1-\delta$ over the random sampling of matrix
$\mathbf{A}^{(m)}$,

$\mathbf{v}_{i}\cdot\mathbf{v}_{j}-\epsilon\leq\frac{\mathbf{v}^{(m)}_{i}\cdot\mathbf{v}^{(m)}_{j}}{k}\leq\mathbf{v}_{i}\cdot\mathbf{v}_{j}+\epsilon.$

The proof of Theorem 1 is given in the supplementary. As stated in Theorem 1,
random projections are likely to preserve the similarities among features,
i.e., originally distant features ( e.g., one normal and one anomalous
instance) will still be comparatively distant and features originally closely
distributed ( e.g., normal data) tend to be near each other after projection.
This lays the theoretical foundation of our SLA2P framework in that, although
the transformations are random, they still preserve the structure and inner
relationship of the data points, which is coherent with handcrafted geometric
transformations for 2D image data.

##  6 Experiments

###  6.1 Datasets

To manifest the generality and flexibility of SLA2P, we empirically evaluate
our framework on three image datasets and four tabular datasets. The image
datasets are CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al.,
2009) and Caltech 101 (Fei-Fei et al., 2004). The tabular datasets are
composed of text datasets 20 Newsgroups (Lang, 1995),
Reuters-21578111http://www.daviddlewis.com/resources/testcollections/reuters21578/
and inherently tabular datasets Arrhythmia and KDDCUP99 (Dua and Graff, 2017).
For the image and text classification datasets, each class of data serves as
inliers in turn, and the average result over all the classes is reported as
the overall performance. The detailed descriptions are as follows.
CIFAR-10 is composed of 60,000 32$\times$32 images of 10 classes, with 6,000
images per class. In each experiment, the inliers are 6,000 images from one
class and the outliers are $6000\times p$ images randomly selected from the
rest classes.
CIFAR-100 consists of 60,000 32$\times$32 images labeled according to 100
distinct categories. The 100 categories are grouped into 20 superclasses, with
3,000 images per superclass. In each experiment, the inliers are 3,000 images
from one superclass, and the outliers are $3000\times p$ images randomly
chosen from the rest of superclasses.
Caltech 101 contains 9,146 images of 101 different classes. The size of each
image is $300\times 200$. Following the experiment setting of (Lai et al.,
2020a), we select 11 classes that contain at least 100 images and randomly
sample 100 out of them for each class. In each experiment, the inliers are 100
images from one certain class and the outliers are $100\times p$ images
selected from the rest 10 classes at random.
20 Newsgroups is a text classification dataset with roughly 20,000 newsgroup
documents, partitioned nearly evenly across 20 newsgroups, i.e., classes. We
randomly sample 360 documents per class. In each experiment, the inliers are
360 documents from a certain class, and the outliers are $360\times p$
documents randomly chosen from the rest classes.
Reuters-21578 is also a text classification dataset containing 90 text
categories with multi-labels. We choose the five largest classes with single
labels and sample 360 documents for each class at random. In each experiment,
the inliers are the documents from a fixed class, and $360\times p$ outliers
are sampled randomly from the remaining four classes.
Arrhythmia is a small-scale medical dataset containing attributes on the
diagnosis of cardiac arrhythmia in patients. The dataset contains $16$
classes. We construct the anomalous dataset using the smallest classes
$3,4,5,7,8,9,14,15$ and the normal set using the rest. There are $452$ data
instances in total, among which $66$ are anomalies.
KDDCUP99 is a large-scale intrusion detection dataset. Following (Zong et al.,
2018) we use the entire UCI $10\%$ dataset, where the non-attack classes are
treated as anomalies. There are $97278$ abnormal instances and $396743$ normal
ones.

###  6.2 Experimental Setup

We compare SLA2P with six SOTA anomaly detection methods: IF (Liu et al.,
2008), OCSVM (Schölkopf et al., 2000; Amer et al., 2013), DAGMM (Zong et al.,
2018), E3Outlier (Wang et al., 2019), RSRAE (Lai et al., 2020a) and PANDA
(Reiss et al., 2021). We adopt the commonly used Area under the Receiver
Operating Characteristic curve (AUROC) and Area under the Precision-Recall
curve (AUPR) as evaluation metric. For AUPR, we consider outliers as
“positive” when computing. All the experiments, including baseline methods,
are run $5$ times independently using the same random seeds $0,1,2,3,4$ for
reproducibility and fair comparison and the averaged scores are reported in
the main paper. The source code and more experimental details (including each
single experiment result) are provided in the supplementary.

#### Setup of SLA2P.

For image data, we use ResNets (He et al., 2016) pretrained on ImageNet
without the last fully connected layer to extract embeddings. To match the
image input shapes, we first resize raw images to $224\times 224$ using
Bilinear Interpolation and then feed it into the extractor network. For text
data, we apply TFIDF transformer and Hashing-vector (Rajaraman and Ullman,
2011) sequentially to pre-process raw data into vectors, which is the same
process as in (Lai et al., 2020a). For inherently tabular data, we directly
use raw data vectors as the input. On Reuters dataset, we set $M=512$ and
$k=128$ due to high dimensionality. On KDDCup99 dataset, we set $M=64$ and
$k=128$ because of high computational burden. On the rest of the datasets, we
set $M=256$ and $k=256$. For the early stopping threshold, we set $\mu=0.75$
for 20 Newsgroups, $\mu=0.3$ for Reuters and $\mu=0.6$ for all the rest
datasets. We choose the perturbation magnitude as $\eta=10$ for 20 Newsgroups,
$\eta=1e2$ for Reuters, $\eta=1e4$ for CIFAR-100 and $\eta=1e3$ for all the
rest datasets. In fact, setting $\mu=0.6$ and $\eta=1e3$ is universally
adequate for different UAD tasks and there is an implicit relationship between
these two hyperparameters (see ablation study). Assuming the input dimension
of classifier network is $q$, for all the experiments, the classifier network
is a $3$-layer fully connected network with structure FC($q$, $2q$)-FC($2q$,
$4q$)-FC($4q$, $M$). Batch normalization (Ioffe and Szegedy, 2015) is applied
to each layer and LeakyReLU (Xu et al., 2015) is employed as activation
functions. We use Adam (Kingma and Ba, 2015) to optimize the classifier
network parameters with learning rate $1e$-$3$ and weight decay $5e$-$4$.

#### Setup of baseline methods.

Our experiments indicate that all the benchmark methods, except E3outlier
(which directly manipulates images), works better using extracted
representative embeddings as input than using raw data as input. Hence for
fair comparison, we compare our method to the baseline methods with extracted
features as input in the sequel. For the implementation of the benchmarks, we
adapt the code from package scikit-learn (Pedregosa et al., 2011) for IF and
OCSVM. We implemented DAGMM and E3Outlier using the
code222https://github.com/demonzyj56/E3Outlier from (Wang et al., 2019) with
minimal modifications such that they are applicable to our datasets and adapt
to our experimental protocal. RSRAE results are obtained running the official
public code 333https://github.com/dmzou/RSRAE. We adapt PANDA for UAD setting
by setting the training set and the testing set to be the same with both
inliers and outliers using the official public code
444https://github.com/talreiss/PANDA. All the data preprocessing processes
follow the original papers.

###  6.3 Results

Dataset | p | IF | OCSVM | DAGMM |  E3Outlier | RSRAE | PANDA | SLA (ours) |  SLA2P (ours)
---|---|---|---|---|---|---|---|---|---
| 0.1 | 83.24 / 43.55 | 85.19 / 47.57 | 59.86 / 15.73 | 85.89 / 45.09 | 83.58 / 42.48 | 86.69 / 59.43 |  90.81 / 59.46 |  91.56 / 62.91
CIFAR-10 | 0.3 | 79.36 / 56.95 | 79.52 / 56.69 | 67.04 / 37.81 | 80.58 / 58.53 | 75.05 / 50.90 | 83.63 / 64.20 |  87.68 / 70.08 |  87.69 / 70.03
| 0.5 | 75.94 / 62.25 | 74.83 / 60.04 | 62.73 / 44.84 | 77.44 / 64.26 | 71.27 / 56.92 | 82.75 / 68.20 |  84.64 / 73.25 |  84.71 / 73.66
| 0.1 | 77.35 / 33.35 | 79.98 / 35.58 | 54.47 / 12.09 | 80.36 / 33.61 |  89.21 / 55.94 | 84.70 / 58.33 | 87.51 / 52.13 |  91.45 / 71.70
CIFAR-100 | 0.3 | 73.90 / 49.65 | 74.86 / 49.62 | 56.78 / 29.59 | 78.60 / 53.15 |  85.33 / 66.40 | 82.32 / 64.21 | 85.03 / 66.39 |  85.48 / 69.29
| 0.5 | 70.81 / 56.51 | 71.21 / 55.96 | 55.13 / 38.16 | 76.22 / 61.00 | 81.68 / 69.46 | 77.83 / 68.88 |  82.48 / 71.21 |  81.98 / 70.76
| 0.1 | 89.88 / 59.72 | 92.98 / 67.16 | 72.61 / 34.02 | 90.72 / 62.29 | 94.28 / 74.44 | 96.32 / 80.39 |  96.36 / 80.98 |  96.42 / 81.12
Caltech 101 | 0.3 | 87.12 / 69.86 | 86.80 / 69.40 | 73.53 / 49.20 | 87.37 / 70.97 | 87.52 / 69.94 | 94.67 / 85.72 |  95.09 / 86.17 |  95.35 / 87.70
| 0.5 | 84.29 / 72.97 | 81.67 / 69.82 | 69.14 / 53.70 | 84.82 / 73.93 | 82.59 / 71.78 |  93.97 / 88.91 | 93.57 / 87.73 |  96.26 / 94.31

Table 1: AUROC/AUPR (%) results on image datasets for UAD with ResNet-50 as
the feature extractor. The best in bold and the second best underlined. The
mean scores over $5$ independent runs are reported.

We report average AUROC and AUPR of all the methods on all the datasets. To
exhibit the generality of our approach, we present its performance under
varying anomaly ratios $p=10\%,30\%,50\%$. We name our approach without
adversarial perturbation as SLA and include its performance as well. Results
on image datasets using pretrained ResNet-50 network are given in Tab. 1. The
results using pretrained ResNet-101 are provided in the supplementary for
completeness, where slightly better performances are achieved owing to the
usage of deeper pretrained DNN extractor. We also summarize the results on
text datasets and inherently tabular datasets in Tables 2 and 3 respectively.

#### Peformance on image datasets.

As indicated in Tab. 1, the performance of our two proposed methods,
especially SLA2P, consistently outperform existing methods on all the image
datasets, including CIFAR-10 and CIFAR-100, which are abidingly considered the
most challenging benchmark datasets for UAD (Wang et al., 2019). In addition,
SLA2P is robust to the anomaly ratio and can still attain exceptional
performance when $p$ is large, with AUROC $84.71\%$ on CIFAR-10, $82.48\%$ on
CIFAR-100 and $96.26\%$ on Caltech 101 when the number of anomalies reaches
half the number of normal samples. This is commendable as when the number of
anomalies increase, no matter what UAD method is used, it is inevitable that
the adverse effect of the training of outliers to the training of inliers will
increase accordingly (Lai et al., 2020a, b).

Dataset | p | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | 0.1 | 56.45 / 15.09 | 82.04 / 43.82 | 54.84 / 12.76 | 89.03 / 49.79 |  90.77 / 50.57 |  93.20 / 71.77
0.3 | 58.35 / 34.56 | 75.07 / 52.28 | 54.07 / 27.38 |  88.86 / 68.61 | 86.29 / 63.58 |  90.16 / 74.21
0.5 | 57.00 / 43.77 | 71.33 / 58.49 | 51.94 / 35.93 |  86.73 / 73.55 | 84.48 / 71.18 |  86.55 / 78.13
Reuters-21578 | 0.1 | 59.30 / 18.74 | 90.95 / 61.69 | 65.07 / 22.33 | 92.44 / 64.44 |  93.51 / 64.58 |  95.77 / 81.40
0.3 | 58.86 / 35.99 | 83.24 / 63.25 | 59.33 / 33.61 |  88.70 / 73.07 | 88.07 / 71.95 |  95.52 / 89.68
0.5 | 55.85 / 42.10 | 74.88 / 62.36 | 62.01 / 46.54 |  83.39 / 72.93 | 82.87 / 72.64 |  86.09 / 79.92

Table 2: AUROC/AUPR (%) results on text datasets for UAD. The mean scores over
$5$ independent runs are reported.

| Arrhythmia |  | KDDCUP99
---|---|---|---
| AUROC (%) | AUPR (%) |  | AUROC (%) | AUPR (%)
IF | 79.78±0.93 | 46.29±1.30 |  | 92.72±1.17 | 69.06±3.91
OCSVM | 79.47±0.00 | 47.75±0.00 |  | 51.98±0.00 | 43.23±0.00
DAGMM | 67.16±4.14 | 30.68±1.98 |  | 77.55±5.90 | 47.28±5.80
RSRAE | 80.31±2.89 | 42.03±2.84 |  | 60.09±2.30 | 27.36±1.22
SLA (ours) | 79.85±0.69 | 46.78±1.22 |  | 97.00±0.46 | 88.89±0.92
SLA2P (ours) | 98.93±0.08 | 90.59±0.66 |  | 97.59±0.34 | 90.59±0.86

Table 3: UAD performance on inherently tabular datasets.

#### Peformance on tabular datasets.

Besides image datasets, our framework is also able to exhibit superior
performance on tabular datasets as demonstrated in Tables 2 and 3. SLA2P
achieves SOTA results on 20 Newsgroups, significantly outperforming current
SOTA RSRAE by $3\%$-$4\%$ in AUROC and around $10\%$ in AUPR for different
anomaly ratios. On Reuters, our method works much better than current SOTA,
with up to $6.8\%$ gain in AUROC and $16.6\%$ gain in AUPR when $p=0.3$ over
RSRAE. On inherently tabular datasets, SLA2P exhibits dominating performance
with over $40\%$ AUPR gain on Arrhythmia, over $20\%$ AUPR gain on KDDCUP99,
and the AUROCs and AUPRs of these two classical datasets all exceed $90\%$.
The extraordinarily excellent performance of SLA2P on the large-scale dataset
KDDCUP99 reveals its practicality .

(a) AUROC on CIFAR-10.

(b) AUROC on CIFAR-100.

(c) AUROC on Caltech.

(d) AUROC on 20 Newsgroups.

(e) AUROC on Reuters.

(f) AUPR on CIFAR-10.

(g) AUPR on CIFAR-100.

(h) AUPR on Caltech.

(i) AUPR on 20 Newsgroups.

(j) AUPR on Reuters.

Figure 2: AUROC and AUPR scores with standard deviation bars when the anomaly
ratio is tiny. Best viewed in color.

#### Peformance under tiny anomaly ratios.

The anomaly ratio can be extremely small in some UAD tasks and circumstances.
Therefore we further compare our methods to the baseline methods when the
anomaly rates are tiny: $p=1\%,2\%,3\%,4\%,5\%$. Clearly seen from Fig. 2, our
SLA2P still achieves favorable performance compared to other baseline methods.
For all the aforementioned image and text datasets except 20 Newsgroups, SLA2P
dominates in AUROC and AUPR with minimal numerical deviations. Furthermore, as
demonstrated by the steadiness of the SLA2P curve in Fig. 2, SLA2P is
remarkably stable with the alternation of $p$, especially in AUROC, despite
the fact that we require sampling random matrices in our pipeline.

###  6.4 Ablation Study and Sensitivity Analysis

(a) CIFAR-10, inlier ’cat’
(w/o perturbation).

(b) CIFAR-100, inlier ’non-insect invertebrates’ (w/o perturbation).

(c) 20, ’comp.sys.mac.hardware’
(w/o perturbation).

(d) Reuters, inlier ’topics’
(w/o perturbation).

(e) Arrhythmia
(w/o perturbation).

(f) CIFAR-10, inlier ’cat’
(w/ perturbation).

(g) CIFAR-100, inlier ’non-insect invertebrates’ (w/ perturbation).

(h) 20, ’comp.sys.mac.hardware’
(w/ perturbation).

(i) Reuters, inlier ’topics’
(w/ perturbation).

(j) Arrhythmia
(w/ perturbation).

Figure 3: Distribution visualization of the anomaly scores before and after
adversarial perturbation of our framework. Best viewed in color.

(a) CIFAR-10.

(b) 20 Newsgroups.

(c) Arrhythmia.

Figure 4: AUPR heatmap of our SLA2P. Best viewed in color.

#### Effect of adversarial perturbation.

To further demonstrate the function of the adversarial perturbations on the
transformed features, we plot the anomaly score distributions before and after
the perturbations. We choose experiments with inlier class ’cat’ of CIFAR-10,
’non-insect invertebrates’ of CIFR-100, ’comp.sys.mac.hardware’ of 20
Newsgroups, ’topics’ of Reuters. We also comprise Arrhythmia experiment. As
shown in Fig. 3, despite our self-supervised SLA framework, there still exist
outliers whose anomaly scores interlaces with those of inliers. In contrast,
after the perturbation, the scores of both normal and anomalous data decreases
but those of the inliers are more robust to the perturbations, making the
anomalies more separable.

#### Relationship between $\mu$ and $\eta$.

We provide fine-grained AUPR results when $p=0.1$ on CIFAR-10, 20 Newsgroups
and Arrhythmia tuning $\mu\in\\{0.4,0.5,0.6,0.7,0.8\\}$ and $\eta$ in log
scale. As illustrated in Fig. 4, SLA2P is highly robust w.r.t. the early
stopping threshold $\mu$ and the perturbation magnitude $\eta$. There is a
wide range of hyperparameter permutations that make the AUPR score of SLA2P
over $60\%$ on CIFAR-10, over $55\%$ on 20 Newsgroups and over $50\%$ on
Arrhythmia (all better than current SOTA baselines). Also, within the range of
the hyperparameter values exhibiting competitive AD performance, higher $\mu$
typically requires larger $\eta$ to guarantee satisfactory detection
performance. This makes sense by intuition as the more transformed samples are
classified well, the larger the stepsize of the perturbations should be to
separate the anomalies out.

#### Robustness to transformed dimension $k$.

We investigate the robustness of AD performance of SLA2P to the transformation
matrix dimension $k$. We conduct ablation study on Caltech 101 dataset with
$k$ varing in the interval $[25,350]$. The experiment is repeated for $5$
times independently and the AUROC and AUPR curve w.r.t. dimension $k$ are
depicted in Fig. 5. From the figure we can draw the conclusion that UAD
performance of SLA2P is generally robust w.r.t. $k$ with moderate growth when
$k$ increases and the growth tends to be flatten out when $k$ becomes
comparatively larger. This phenomenon may be elucidated by Theorem 1 as $k$
needs to be large enough to preserve the similarities between transformed
features with high probability.

(a) AUROC w.r.t. $k$.

(b) AUPR w.r.t. $k$.

Figure 5: UAD performance on Caltech 101 with varying $k$ of our proposed
SLA2P method. Best viewed in color.

##  7 Conclusion

This paper proposes a novel framework, SLA2P, for unsupervised anomaly
detection. We employ random projections as a feature-level self-supervised
approach and incorporate adversarial perturbation into it to design
discriminative anomaly scores. We further justify SLA2P’s effectiveness both
theoretically and empirically. We hope this work may also shed light to
pretext task based perturbation techniques for other unsupervised learning
problems.

## References

  * Abati et al. (2019) Abati, D., Porrello, A., Calderara, S. and Cucchiara, R. (2019).  Latent space autoregression for novelty detection.  In CVPR.
  * Adewumi and Akinyelu (2017) Adewumi, A. O. and Akinyelu, A. A. (2017).  A survey of machine-learning and nature-inspired based credit card fraud detection techniques.  International Journal of System Assurance Engineering and Management.
  * Amer et al. (2013) Amer, M., Goldstein, M. and Abdennadher, S. (2013).  Enhancing one-class support vector machines for unsupervised anomaly detection.  In Proceedings of the ACM SIGKDD workshop on outlier detection and description.
  * Andrews et al. (2016) Andrews, J., Tanay, T., Morton, E. J. and Griffin, L. D. (2016).  Transfer representation-learning for anomaly detection.  JMLR.
  * Arriaga and Vempala (1999) Arriaga, R. I. and Vempala, S. (1999).  An algorithmic theory of learning: Robust concepts and random projection.  In 40th Annual Symposium on Foundations of Computer Science (Cat. No. 99CB37039).
  * Bergman et al. (2020) Bergman, L., Cohen, N. and Hoshen, Y. (2020).  Deep nearest neighbor anomaly detection.  arXiv preprint arXiv:2002.10445.
  * Bergman and Hoshen (2020) Bergman, L. and Hoshen, Y. (2020).  Classification-based anomaly detection for general data.  In ICLR.
  * Bergmann et al. (2019) Bergmann, P., Fauser, M., Sattlegger, D. and Steger, C. (2019).  Mvtec AD - A comprehensive real-world dataset for unsupervised anomaly detection.  In CVPR.
  * Bergmann et al. (2020) Bergmann, P., Fauser, M., Sattlegger, D. and Steger, C. (2020).  Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings.  In CVPR.
  * Bojanowski and Joulin (2017) Bojanowski, P. and Joulin, A. (2017).  Unsupervised learning by predicting noise.  In ICML.
  * Breunig et al. (2000) Breunig, M. M., Kriegel, H.-P., Ng, R. T. and Sander, J. (2000).  Lof: identifying density-based local outliers.  In Proceedings of the 2000 ACM SIGMOD international conference on Management of data.
  * Brier et al. (1950) Brier, G. W. et al. (1950).  Verification of forecasts expressed in terms of probability.  Monthly weather review.
  * Burlina et al. (2019) Burlina, P., Joshi, N. and Wang, I. (2019).  Where’s wally now? deep generative and discriminative embeddings for novelty detection.  In CVPR.
  * Chalapathy and Chawla (2019) Chalapathy, R. and Chawla, S. (2019).  Deep learning for anomaly detection: A survey.  arXiv preprint arXiv:1901.03407.
  * Chandola et al. (2007) Chandola, V., Banerjee, A. and Kumar, V. (2007).  Outlier detection: A survey.  ACM Computing Surveys.
  * Chandola et al. (2009) Chandola, V., Banerjee, A. and Kumar, V. (2009).  Anomaly detection: A survey.  ACM computing surveys (CSUR).
  * Chen et al. (2017) Chen, J., Sathe, S., Aggarwal, C. C. and Turaga, D. S. (2017).  Outlier detection with autoencoder ensembles.  In SDM.
  * Chen et al. (2020) Chen, T., Kornblith, S., Norouzi, M. and Hinton, G. E. (2020).  A simple framework for contrastive learning of visual representations.  In ICML.
  * Chen et al. (2015) Chen, Z., Tian, Y., Zeng, W. and Huang, T. (2015).  Detecting abnormal behaviors in surveillance videos based on fuzzy clustering and multiple auto-encoders.  In ICME.
  * Dua and Graff (2017) Dua, D. and Graff, C. (2017).  UCI machine learning repository.
http://archive.ics.uci.edu/ml

  * Ester et al. (1996) Ester, M., Kriegel, H.-P., Sander, J., Xu, X. et al. (1996).  A density-based algorithm for discovering clusters in large spatial databases with noise.  In KDD.
  * Fei-Fei et al. (2004) Fei-Fei, L., Fergus, R. and Perona, P. (2004).  Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories.  In CVPRW.
  * Fu et al. (2016) Fu, K., Cheng, D., Tu, Y. and Zhang, L. (2016).  Credit card fraud detection using convolutional neural networks.  In International Conference on Neural Information Processing.
  * Gidaris et al. (2018) Gidaris, S., Singh, P. and Komodakis, N. (2018).  Unsupervised representation learning by predicting image rotations.  In ICLR.
  * Golan and El-Yaniv (2018) Golan, I. and El-Yaniv, R. (2018).  Deep anomaly detection using geometric transformations.  In NeurIPS.
  * Goodfellow et al. (2014) Goodfellow, I. J., Shlens, J. and Szegedy, C. (2014).  Explaining and harnessing adversarial examples.  In ICLR.
  * Gu et al. (2020) Gu, X., Sun, J. and Xu, Z. (2020).  Spherical space domain adaptation with robust pseudo-label loss.  In CVPR.
  * He et al. (2016) He, K., Zhang, X., Ren, S. and Sun, J. (2016).  Deep residual learning for image recognition.  In CVPR.
  * He et al. (2003) He, Z., Xu, X. and Deng, S. (2003).  Discovering cluster-based local outliers.  Pattern Recognition Letters.
  * Hendrycks et al. (2019) Hendrycks, D., Mazeika, M. and Dietterich, T. G. (2019).  Deep anomaly detection with outlier exposure.  In ICLR.
  * Hsu et al. (2020) Hsu, Y.-C., Shen, Y., Jin, H. and Kira, Z. (2020).  Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data.  In CVPR.
  * Ioffe and Szegedy (2015) Ioffe, S. and Szegedy, C. (2015).  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In ICML.
  * Khan and Yairi (2018) Khan, S. and Yairi, T. (2018).  A review on the application of deep learning in system health management.  Mechanical Systems and Signal Processing.
  * Kim and Scott (2012) Kim, J. and Scott, C. D. (2012).  Robust kernel density estimation.  JMLR.
  * Kingma and Ba (2015) Kingma, D. P. and Ba, J. (2015).  Adam: A method for stochastic optimization.  In ICLR.
  * Krizhevsky et al. (2009) Krizhevsky, A., Hinton, G. et al. (2009).  Learning multiple layers of features from tiny images.
  * Kumagai et al. (2019) Kumagai, A., Iwata, T. and Fujiwara, Y. (2019).  Transfer anomaly detection by inferring latent domain representations.  In NeurIPS.
  * Kwon et al. (2019) Kwon, D., Kim, H., Kim, J., Suh, S. C., Kim, I. and Kim, K. J. (2019).  A survey of deep learning-based network anomaly detection.  Cluster Computing.
  * Kwon et al. (2020) Kwon, G., Prabhushankar, M., Temel, D. and AlRegib, G. (2020).  Backpropagated gradient representations for anomaly detection.  In ECCV.
  * Lai et al. (2020a) Lai, C., Zou, D. and Lerman, G. (2020a).  Robust subspace recovery layer for unsupervised anomaly detection.  In ICLR.
  * Lai et al. (2020b) Lai, C.-H., Zou, D. and Lerman, G. (2020b).  Novelty detection via robust variational autoencoding.  arXiv preprint arXiv:2006.05534.
  * Lang (1995) Lang, K. (1995).  Newsweeder: Learning to filter netnews.  In Machine Learning Proceedings.
  * Lee et al. (2018) Lee, K., Lee, K., Lee, H. and Shin, J. (2018).  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  In NeurIPS.
  * Lerman and Maunu (2018) Lerman, G. and Maunu, T. (2018).  An overview of robust subspace recovery.  Proceedings of the IEEE.
  * Liang et al. (2018) Liang, S., Li, Y. and Srikant, R. (2018).  Enhancing the reliability of out-of-distribution image detection in neural networks.  In ICLR.
  * Liu et al. (2008) Liu, F. T., Ting, K. M. and Zhou, Z.-H. (2008).  Isolation forest.  In ICDM.
  * Liu et al. (2020) Liu, W., Wang, X., Owens, J. D. and Li, Y. (2020).  Energy-based out-of-distribution detection.  arXiv preprint arXiv:2010.03759.
  * Min et al. (2017) Min, S., Lee, B. and Yoon, S. (2017).  Deep learning in bioinformatics.  Briefings in bioinformatics.
  * Nazare et al. (2018) Nazare, T. S., de Mello, R. F. and Ponti, M. A. (2018).  Are pre-trained cnns good feature extractors for anomaly detection in surveillance videos?  arXiv preprint arXiv:1811.08495.
  * Oord et al. (2018) Oord, A. v. d., Li, Y. and Vinyals, O. (2018).  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748.
  * Paffenroth et al. (2018) Paffenroth, R., Kay, K. and Servi, L. (2018).  Robust pca for anomaly detection in cyber networks.  arXiv preprint arXiv:1801.01571.
  * Pang et al. (2020a) Pang, G., Shen, C., Cao, L. and Hengel, A. v. d. (2020a).  Deep learning for anomaly detection: A review.  arXiv preprint arXiv:2007.02500.
  * Pang et al. (2020b) Pang, G., Yan, C., Shen, C., van den Hengel, A. and Bai, X. (2020b).  Self-trained deep ordinal regression for end-to-end video anomaly detection.  In CVPR.
  * Pedregosa et al. (2011) Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V. et al. (2011).  Scikit-learn: Machine learning in python.  JMLR.
  * Perera et al. (2019) Perera, P., Nallapati, R. and Xiang, B. (2019).  OCGAN: one-class novelty detection using gans with constrained latent representations.  In CVPR.
  * Pidhorskyi et al. (2018) Pidhorskyi, S., Almohsen, R. and Doretto, G. (2018).  Generative probabilistic novelty detection with adversarial autoencoders.  In NeurIPS.
  * Rajaraman and Ullman (2011) Rajaraman, A. and Ullman, J. D. (2011).  Mining of massive datasets.  Cambridge University Press.
  * Reiss et al. (2021) Reiss, T., Cohen, N., Bergman, L. and Hoshen, Y. (2021).  Panda: Adapting pretrained features for anomaly detection and segmentation.  In CVPR.
  * Rippel et al. (2020) Rippel, O., Mertens, P. and Merhof, D. (2020).  Modeling the distribution of normal data in pre-trained deep features for anomaly detection.  arXiv preprint arXiv:2005.14140.
  * Ruff et al. (2020) Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G. and Müller, K.-R. (2020).  A unifying review of deep and shallow anomaly detection.  arXiv preprint arXiv:2009.11732.
  * Sabokrou et al. (2018) Sabokrou, M., Fayyaz, M., Fathy, M., Moayed, Z. and Klette, R. (2018).  Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes.  CVIU.
  * Schirrmeister et al. (2020) Schirrmeister, R. T., Zhou, Y., Ball, T. and Zhang, D. (2020).  Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features.  arXiv preprint arXiv:2006.10848.
  * Schölkopf et al. (2000) Schölkopf, B., Williamson, R. C., Smola, A. J., Shawe-Taylor, J. and Platt, J. C. (2000).  Support vector method for novelty detection.  In NeurIPS.
  * Sehwag et al. (2021) Sehwag, V., Chiang, M. and Mittal, P. (2021).  Ssd: A unified framework for self-supervised outlier detection.  In ICLR.
  * Shen et al. (2007) Shen, A., Tong, R. and Deng, Y. (2007).  Application of classification models on credit card fraud detection.  In 2007 International conference on service systems and service management. IEEE.
  * Shyu et al. (2003) Shyu, M.-L., Chen, S.-C., Sarinnapakorn, K. and Chang, L. (2003).  A novel anomaly detection scheme based on principal component classifier.  Tech. rep., MIAMI UNIV CORAL GABLES FL DEPT OF ELECTRICAL AND COMPUTER ENGINEERING.
  * Simonyan and Zisserman (2015) Simonyan, K. and Zisserman, A. (2015).  Very deep convolutional networks for large-scale image recognition.  In ICLR.
  * Somepalli et al. (2020) Somepalli, G., Wu, Y., Balaji, Y., Vinzamuri, B. and Feizi, S. (2020).  Unsupervised anomaly detection with adversarial mirrored autoencoders.  arXiv e-prints.
  * Sultani et al. (2018) Sultani, W., Chen, C. and Shah, M. (2018).  Real-world anomaly detection in surveillance videos.  In CVPR.
  * Tack et al. (2020) Tack, J., Mo, S., Jeong, J. and Shin, J. (2020).  Csi: Novelty detection via contrastive learning on distributionally shifted instances.  In NeurIPS.
  * Tan et al. (2011) Tan, S. C., Ting, K. M. and Liu, F. T. (2011).  Fast anomaly detection for streaming data.  In IJCAI.
  * Venkataramanan et al. (2019) Venkataramanan, S., Peng, K.-C., Singh, R. V. and Mahalanobis, A. (2019).  Attention guided anomaly detection and localization in images.  arXiv preprint arXiv:1911.08616.
  * Vercruyssen et al. (2020) Vercruyssen, V., Meert, W. and Davis, J. (2020).  Transfer learning for anomaly detection through localized and unsupervised instance selection.  In AAAI.
  * Wang et al. (2019) Wang, S., Zeng, Y., Liu, X., Zhu, E., Yin, J., Xu, C. and Kloft, M. (2019).  Effective end-to-end unsupervised outlier detection via inlier priority of discriminative network.  In NeurIPS.
  * Xu et al. (2015) Xu, B., Wang, N., Chen, T. and Li, M. (2015).  Empirical evaluation of rectified activations in convolutional network.  arXiv preprint arXiv:1505.00853.
  * Yang et al. (2009) Yang, X., Latecki, L. J. and Pokrajac, D. (2009).  Outlier detection with globally optimal exemplar-based gmm.  In SDM.
  * Zenati et al. (2018) Zenati, H., Foo, C. S., Lecouat, B., Manek, G. and Chandrasekhar, V. R. (2018).  Efficient gan-based anomaly detection.  arXiv preprint arXiv:1802.06222.
  * Zong et al. (2018) Zong, B., Song, Q., Min, M. R., Cheng, W., Lumezanu, C., Cho, D. and Chen, H. (2018).  Deep autoencoding gaussian mixture model for unsupervised anomaly detection.  In ICLR.

##  Appendix A Proofs

###  A.1 Proof of Proposition 1

###### Proof.

We first prove $\mathbf{A}\mathcal{V}$ is still a linear subspace of
$\mathbb{R}^{k}$.

Suppose $\mathbf{y}_{1}$ and $\mathbf{y}_{2}$ are two vectors in
$\mathbf{A}\mathcal{V}$, then by definition there exist
$\mathbf{x}_{1},\mathbf{x}_{2}\in\mathcal{V}$ s.t.
$\mathbf{A}\mathbf{x}_{1}=\mathbf{y}_{1},\mathbf{A}\mathbf{x}_{2}=\mathbf{y}_{2}$.
Hence

$\displaystyle\mathbf{y}_{1}+\mathbf{y}_{2}$
$\displaystyle=\mathbf{A}\mathbf{x}_{1}+\mathbf{A}\mathbf{x}_{2}$
$\displaystyle=\mathbf{A}(\mathbf{x}_{1}+\mathbf{x}_{2}).$ (2)

Considering $\mathbf{x}_{1},\mathbf{x}_{2}\in\mathcal{V}$ and $\mathcal{V}$ is
a linear subspace, we have $\mathbf{x}_{1}+\mathbf{x}_{2}\in\mathcal{V}$.
Combining this with (A.1) we obtain

$\mathbf{y}_{1}+\mathbf{y}_{2}\in\mathbf{A}\mathcal{V}.$ (3)

For any $c\in\mathbb{R}$,
$c\mathbf{y}_{1}=c\mathbf{A}\mathbf{x}_{1}=\mathbf{A}(c\mathbf{x}_{1})$. As
$\mathbf{x}_{1}\in\mathcal{V}$ and $\mathcal{V}$ is a linear subspace, we have
$c\mathbf{x}_{1}\in\mathcal{V}$, therefore we get

$c\mathbf{y}_{1}\in\mathcal{V}.$ (4)

Combining (3) and (4) we conclude $\mathbf{A}\mathcal{V}$ is a linear subspace
of $\mathbb{R}^{k}$.

Next we prove $\mathbf{A}\mathcal{U}$ is still a convex set in
$\mathbb{R}^{k}$.

Suppose $\mathbf{y}^{\prime}_{1}$ and $\mathbf{y}^{\prime}_{2}$ are two
vectors in $\mathbf{A}\mathcal{U}$, then by definition there exist
$\mathbf{x}^{\prime}_{1},\mathbf{x}^{\prime}_{2}\in\mathcal{U}$ s.t.
$\mathbf{A}\mathbf{x}^{\prime}_{1}=\mathbf{y}^{\prime}_{1},\mathbf{A}\mathbf{x}^{\prime}_{2}=\mathbf{y}^{\prime}_{2}$.
Hence for any $0<\alpha<1$, we have

$\displaystyle\alpha\mathbf{y}^{\prime}_{1}+(1-\alpha)\mathbf{y}^{\prime}_{2}$
$\displaystyle=\alpha\mathbf{A}\mathbf{x}^{\prime}_{1}+(1-\alpha)\mathbf{A}\mathbf{x}^{\prime}_{2}$
$\displaystyle=\mathbf{A}\big{(}\alpha\mathbf{x}^{\prime}_{1}+(1-\alpha)\mathbf{x}^{\prime}_{2}\big{)}.$
(5)

Since $\mathcal{U}$ is a convex set in $\mathbb{R}^{d}$ and
$\mathbf{x}^{\prime}_{1},\mathbf{x}^{\prime}_{2}\in\mathcal{U}$, we have
$\alpha\mathbf{x}^{\prime}_{1}+(1-\alpha)\mathbf{x}^{\prime}_{2}\in\mathcal{U}$.
Then by (A.1) we get
$\alpha\mathbf{y}^{\prime}_{1}+(1-\alpha)\mathbf{y}^{\prime}_{2}\in\mathbf{A}\mathcal{U}$.
Thereby we proved the convexity of set $\mathbf{A}\mathcal{U}$. ∎

###  A.2 Proof of Theorem 1

###### Proof.

For notational simplity, we use $\left\|\cdot\right\|$ to denote L2 norm
$\left\|\cdot\right\|_{2}$ and we denote $\mathbf{A}^{(m)}=(a_{ij})$ in the
following. Some of the proof techniques are adapted from Johnson–Lindenstrauss
lemma (Arriaga and Vempala, 1999). We first prove property ($1$).

Consider $\forall\mathbf{v}\in\mathbb{R}^{d}$, by our definition we have
$\mathbf{v}^{(m)}=\mathbf{A}^{(m)}\mathbf{v}$, we have

$\displaystyle
P\Big{(}\left\|\mathbf{v}^{(m)}\right\|^{2}\geq(1+\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}$
$\displaystyle=$ $\displaystyle
P\Big{(}\left\|\mathbf{A}^{(m)}\mathbf{v}\right\|^{2}\geq(1+\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}$
$\displaystyle=$ $\displaystyle
P\bigg{(}\frac{\left\|\mathbf{A}^{(m)}\mathbf{v}\right\|^{2}}{\left\|\mathbf{v}\right\|^{2}}\geq(1+\epsilon)k\bigg{)}.$
(6)

Let

$\displaystyle
W\triangleq\frac{\left\|\mathbf{A}^{(m)}\mathbf{v}\right\|^{2}}{\left\|\mathbf{v}\right\|^{2}},$

and

$\displaystyle
w_{i}\triangleq\frac{\sum_{j}a_{ij}v_{j}}{\left\|\mathbf{v}\right\|},$

it is straightforward that

$\displaystyle W=\sum_{i=1}^{k}w_{i}^{2}.$

Because $a_{ij}\sim\mathcal{N}(0,1)$ and $\\{a_{ij}\\}$are i.i.d., we have
$w_{i}\sim\mathcal{N}(0,1)$ and $\\{w_{i}\\}$ are i.i.d. Hence by (A.2), for
any $\alpha>0$, we have

$\displaystyle
P\Big{(}\left\|\mathbf{v}^{(m)}\right\|^{2}\geq(1+\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}$
$\displaystyle=$ $\displaystyle P\big{(}W\geq(1+\epsilon)k\big{)}$
$\displaystyle=$ $\displaystyle P\Big{(}\mathrm{e}^{\alpha
W}\geq\mathrm{e}^{(1+\epsilon)k\alpha}\Big{)}$
$\displaystyle\mathop{\leq}^{\rm(i)}$
$\displaystyle\frac{\mathbb{E}\Big{(}\mathrm{e}^{\alpha
W}\Big{)}}{\mathrm{e}^{(1+\epsilon)k\alpha}}$
$\displaystyle\mathop{=}^{\rm(ii)}$
$\displaystyle\frac{\prod_{i=1}^{k}\mathbb{E}\big{(}\mathrm{e}^{\alpha
w_{i}^{2}}\big{)}}{\mathrm{e}^{(1+\epsilon)k\alpha}}$ $\displaystyle=$
$\displaystyle\Bigg{(}\frac{\mathbb{E}\big{(}\mathrm{e}^{\alpha
w_{1}^{2}}\big{)}}{\mathrm{e}^{(1+\epsilon)\alpha}}\Bigg{)}^{k},$ (7)

where (i) comes from Markov’s inequality and (ii) uses the fact that
$\\{w_{i}\\}$ are i.i.d.. Notice that when $0\leq\alpha<\frac{1}{2}$ we have

$\displaystyle\mathbb{E}\big{(}\mathrm{e}^{\alpha w_{1}^{2}}\big{)}$
$\displaystyle=\int_{-\infty}^{\infty}\mathrm{e}^{\alpha
w_{1}^{2}}\frac{1}{\sqrt{2\pi}}\mathrm{e}^{-\frac{w_{1}^{2}}{2}}\mathrm{d}w_{1}$
$\displaystyle=\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi}}\mathrm{e}^{-\frac{w_{1}^{2}}{2}(1-2\alpha)}\mathrm{d}w_{1}$
$\displaystyle=\frac{1}{\sqrt{1-2\alpha}}\int_{-\infty}^{\infty}\frac{\sqrt{1-2\alpha}}{\sqrt{2\pi}}\mathrm{e}^{-\frac{w_{1}^{2}}{2}(1-2\alpha)}\mathrm{d}w_{1}$
$\displaystyle\mathop{=}^{\rm(i)}\frac{1}{\sqrt{1-2\alpha}},$ (8)

where (i) uses the integral of random variable with distribution
$\mathcal{N}(0,\frac{1}{1-2\alpha})$. Now plugging (A.2) into (11), we get

$\displaystyle
P\Big{(}\left\|\mathbf{v}^{(m)}\right\|^{2}\geq(1+\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}\leq\bigg{(}\frac{\mathrm{e}^{-2(1+\epsilon)\alpha}}{1-2\alpha}\bigg{)}^{\frac{k}{2}}.$

Take $\alpha=\frac{\epsilon}{2(1+\epsilon)}$, we have

$\displaystyle
P\Big{(}\left\|\mathbf{v}^{(m)}\right\|^{2}\geq(1+\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}\leq$
$\displaystyle\big{(}(1+\epsilon)\mathrm{e}^{-\epsilon}\big{)}^{\frac{k}{2}}$
$\displaystyle=$
$\displaystyle\big{(}\mathrm{e}^{\log(1+\epsilon)-\epsilon}\big{)}^{\frac{k}{2}}$
$\displaystyle\mathop{\leq}^{\rm(i)}$
$\displaystyle\big{(}\mathrm{e}^{-(\frac{\epsilon^{2}}{2}-\frac{\epsilon^{3}}{3})}\big{)}^{\frac{k}{2}}$
$\displaystyle\leq$
$\displaystyle\mathrm{e}^{-\frac{k}{4}(\epsilon^{2}-\epsilon^{3})},$ (9)

where (i) employs inequality $\log(1+x)<x-\frac{x^{2}}{2}+\frac{x^{3}}{3}$.

Similarly, we have

$\displaystyle
P\Big{(}\left\|\mathbf{v}^{(m)}\right\|^{2}\leq(1-\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}$
$\displaystyle=$ $\displaystyle P\big{(}W\leq(1-\epsilon)k\big{)}$
$\displaystyle\leq$ $\displaystyle P\Big{(}\mathrm{e}^{-\alpha
W}\geq\mathrm{e}^{-(1-\epsilon)\alpha k}\Big{)}$ $\displaystyle\leq$
$\displaystyle\frac{\mathbb{E}\Big{(}\mathrm{e}^{-\alpha
W}\Big{)}}{\mathrm{e}^{-(1-\epsilon)\alpha W}}$ $\displaystyle=$
$\displaystyle\frac{\prod_{i=1}^{k}\mathbb{E}\big{(}\mathrm{e}^{-\alpha
w_{i}^{2}}\big{)}}{\mathrm{e}^{-(1-\epsilon)\alpha k}}$ $\displaystyle=$
$\displaystyle\bigg{(}\frac{\mathbb{E}\big{(}\mathrm{e}^{-\alpha
w_{1}^{2}}\big{)}}{\mathrm{e}^{-(1-\epsilon)\alpha}}\bigg{)}^{k}$
$\displaystyle=$
$\displaystyle\bigg{(}\frac{\mathrm{e}^{2(1-\epsilon)\alpha}}{1+2\alpha}\bigg{)}^{\frac{k}{2}}.$

Take $\alpha=\frac{\epsilon}{2(1-\epsilon)}$, we obtain

$\displaystyle
P\Big{(}\left\|\mathbf{v}^{(m)}\right\|^{2}\leq(1-\epsilon)k\left\|\mathbf{v}\right\|^{2}\Big{)}$
$\displaystyle\leq\big{(}(1-\epsilon)\mathrm{e}^{\epsilon}\big{)}^{\frac{k}{2}}$
$\displaystyle=\big{(}\mathrm{e}^{\log(1-\epsilon)+\epsilon}\big{)}^{\frac{k}{2}}$
$\displaystyle\leq\big{(}\mathrm{e}^{-\frac{\epsilon^{2}}{2}+\frac{\epsilon^{3}}{3}}\big{)}^{\frac{k}{2}}$
$\displaystyle\leq\mathrm{e}^{-\frac{k}{4}(\epsilon^{2}-\epsilon^{3})}.$ (10)

Combining (A.2) and (A.2) and setting
$\mathrm{e}^{-\frac{k}{4}(\epsilon^{2}-\epsilon^{3})}<\frac{\delta}{2}$, we
have: when $k>\frac{4\log\frac{2}{\delta}}{\epsilon^{2}-\epsilon^{3}}$, with
probability at least $1-\delta$,

$(1-\epsilon)\left\|\mathbf{v}\right\|^{2}\leq\frac{\left\|\mathbf{v}^{(m)}\right\|^{2}}{k}\leq(1+\epsilon)\left\|\mathbf{v}\right\|^{2}.$
(11)

Now for any vectors $\mathbf{v}_{i},\mathbf{v}_{j}\in\mathbb{R}^{d}$,
substituting $\mathbf{v}$ in (11) by $\mathbf{v}_{i}-\mathbf{v}_{j}$, we
hereby proved property (1).

Next we prove the property (2).

For any index pair $(i,j)$, applying (11) to vectors
$\mathbf{v}_{1}+\mathbf{v}_{2}$ and $\mathbf{v}_{1}-\mathbf{v}_{2}$, we have
when $k>\frac{4\log\frac{4}{\delta}}{\epsilon^{2}-\epsilon^{3}}$, with
probability at least $1-\delta$,

$\displaystyle k(1-\epsilon)\left\|\mathbf{v}_{i}+\mathbf{v}_{j}\right\|^{2}$
$\displaystyle\leq\left\|\mathbf{v}^{(m)}_{i}+\mathbf{v}^{(m)}_{j}\right\|^{2}\leq
k(1+\epsilon)\left\|\mathbf{v}_{i}+\mathbf{v}_{j}\right\|^{2},$ (12)
$\displaystyle k(1-\epsilon)\left\|\mathbf{v}_{i}-\mathbf{v}_{j}\right\|^{2}$
$\displaystyle\leq\left\|\mathbf{v}^{(m)}_{i}-\mathbf{v}^{(m)}_{j}\right\|^{2}\leq
k(1+\epsilon)\left\|\mathbf{v}_{i}-\mathbf{v}_{j}\right\|^{2}.$ (13)

Hence we have

$\displaystyle\mathbf{v}_{i}^{(m)}\cdot\mathbf{v}_{j}^{(m)}$
$\displaystyle=\frac{\left\|\mathbf{v}_{i}^{(m)}+\mathbf{v}_{j}^{(m)}\right\|^{2}-\left\|\mathbf{v}_{i}^{(m)}-\mathbf{v}_{j}^{(m)}\right\|^{2}}{4}$
$\displaystyle\mathop{\geq}^{\rm(i)}\frac{k(1-\epsilon)\left\|\mathbf{v}_{i}+\mathbf{v}_{j}\right\|^{2}-k(1+\epsilon)\left\|\mathbf{v}_{i}-\mathbf{v}_{j}\right\|^{2}}{4}$
$\displaystyle\mathop{=}^{\rm(ii)}\frac{k}{4}\Big{(}4\mathbf{v}_{i}\cdot\mathbf{v}_{j}-2\epsilon\big{(}\left\|\mathbf{v}_{i}\right\|^{2}+\left\|\mathbf{v}_{j}\right\|^{2}\big{)}\Big{)}$
$\displaystyle=k\Big{(}\mathbf{v}_{i}\cdot\mathbf{v}_{j}-\epsilon\Big{)},$
(14)

where (i) uses the left hand side of (12) and right hand side of (13), and
(ii) utilizes $\left\|\mathbf{v}_{i}\right\|=\left\|\mathbf{v}_{j}\right\|=1$
due to L2 normalization. Likewise, we get

$\displaystyle\mathbf{v}_{i}^{(m)}\cdot\mathbf{v}_{j}^{(m)}$
$\displaystyle=\frac{\left\|\mathbf{v}_{i}^{(m)}+\mathbf{v}_{j}^{(m)}\right\|^{2}-\left\|\mathbf{v}_{i}^{(m)}-\mathbf{v}_{j}^{(m)}\right\|^{2}}{4}$
$\displaystyle\leq\frac{k(1+\epsilon)\left\|\mathbf{v}_{i}+\mathbf{v}_{j}\right\|^{2}-k(1-\epsilon)\left\|\mathbf{v}_{i}-\mathbf{v}_{j}\right\|^{2}}{4}$
$\displaystyle=\frac{k}{4}\Big{(}4\mathbf{v}_{i}\cdot\mathbf{v}_{j}+2\epsilon\big{(}\left\|\mathbf{v}_{i}\right\|^{2}+\left\|\mathbf{v}_{j}\right\|^{2}\big{)}\Big{)}$
$\displaystyle=k\Big{(}\mathbf{v}_{i}\cdot\mathbf{v}_{j}+\epsilon\Big{)},$
(15)

Combining (A.2) and (A.2), we get as long as
$k>\frac{4\log\frac{4}{\delta}}{\epsilon^{2}-\epsilon^{3}}$, with probability
at least $1-\delta$,

$\displaystyle\mathbf{v}_{i}\cdot\mathbf{v}_{j}-\epsilon\leq\frac{\mathbf{v}_{i}^{(m)}\cdot\mathbf{v}_{j}^{(m)}}{k}\leq\mathbf{v}_{i}\cdot\mathbf{v}_{j}+\epsilon.$

∎

##  Appendix B Complete Results

In this section we provide the UAD performance comparison on image tasks using
ResNet-101 as feature extractor. As illustrated above in Fig. 4, our methods
still achieve the best performance consistently.

We also provide the detailed UAD performance results on CIFAR-10 and text
tabular datasets. Note that our reported results in the main paper is the
averaged AUROC and AUPR, as we have mentioned that in each dataset each class
serves as normal data in turn. The 20 superclasses of CIFAR-100 are ’fish’,
’flowers’, ’food containers’, ’fruit and vegetables’, ’household electrical
devices’, ’household furniture’, ’insects’, ’large carnivores’, ’large man-
made outdoor things’, ’large natural outdoor scenes’, ’large omnivores and
herbivores’, ’medium-sized mammals’, ’non-insect invertebrates’, ’people’,
’reptiles’, ’small mammals’, ’trees’, ’vehicles’ and ’vehicles 2’. The 11
selected classes of Caltech are ’watch’, ’ketch’, ’hawksbill’, ’grand piano’,
’chandelier’, ’car side’, ’bonsai’, ’airplanes’, ’motorbikes’, ’leopards’ and
’faces’. All the experients are run 5 times independently using seeds
$0,1,2,3,4$, and averaged scores$\pm$standard deviations are reported.

Dataset | p | IF | OCSVM | DAGMM | RSRAE | PANDA | SLA (ours) |  SLA2P (ours)
---|---|---|---|---|---|---|---|---
| 0.1 | 85.44 / 45.60 | 87.30 / 50.23 | 64.27 / 19.60 | 84.33 / 45.95 | 88.68 / 60.11 |  92.74 / 65.59 |  94.08 / 73.62
CIFAR-10 | 0.3 | 81.45 / 58.85 | 81.29 / 58.14 | 68.34 / 38.61 | 75.72 / 52.42 | 87.36 / 65.35 |  90.08 / 74.93 |  91.66 / 78.28
| 0.5 | 78.65 / 64.72 | 76.32 / 60.83 | 68.68 / 50.61 | 72.06 / 58.15 | 83.06 / 71.27 |  87.11 / 76.56 |  89.18 / 80.69
| 0.1 | 79.12 / 34.93 | 81.69 / 37.44 | 53.52 / 11.95 | 91.40 / 60.68 |  87.90 / 58.41 |  89.94 / 58.38 |  93.04 / 75.05
CIFAR-100 | 0.3 | 75.91 / 51.41 | 76.23 / 50.37 | 53.99 / 27.21 | 87.35 / 69.41 | 86.65 / 64.35 |  87.63 / 71.31 |  87.92 / 73.40
| 0.5 | 73.29 / 58.90 | 72.35 / 56.26 | 55.36 / 39.21 | 84.12 / 72.61 | 81.81 / 69.62 |  85.36 / 75.37 |  84.61 / 74.41
| 0.1 | 91.11 / 62.71 | 93.27 / 70.29 | 72.26 / 33.32 | 95.24 / 78.83 | 96.40 / 81.11 |  96.69 / 82.87 |  96.98 / 84.73
Caltech 101 | 0.3 | 87.41 / 70.92 | 88.60 / 74.02 | 74.01 / 51.49 | 89.47 / 73.90 | 95.16 / 88.38 |  96.23 / 89.44 |  96.48 / 90.94
| 0.5 | 84.68 / 73.63 | 83.50 / 71.99 | 70.09 / 56.17 | 84.95 / 75.30 | 94.27 / 90.34 |  94.89 / 90.40 |  97.25 / 95.76

Table 4: AUROC/AUPR (%) results on image datasets for UAD with ResNet-101 as
the feature extractor. The best in bold and the second best underlined. The
mean scores over $5$ independent runs are reported.

Inlier class name | IF | OCSVM | DAGMM | E3Outlier | RSRAE | PANDA | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---|---
airplane | 84.82$\pm$0.76 | 85.21$\pm$0.62 | 68.51$\pm$0.01 | 78.61$\pm$0.69 | 83.95$\pm$0.61 | 87.48$\pm$0.59 | 88.84$\pm$0.60 | 90.50$\pm$0.58
automobile | 92.33$\pm$0.56 | 94.68$\pm$0.40 | 66.64$\pm$0.02 | 95.47$\pm$0.65 | 93.12$\pm$0.59 | 96.34$\pm$0.31 | 96.77$\pm$0.28 | 97.21$\pm$0.30
bird | 74.39$\pm$0.88 | 72.87$\pm$1.01 | 46.31$\pm$0.01 | 79.08$\pm$0.61 | 72.90$\pm$1.63 | 77.77$\pm$0.97 | 81.30$\pm$1.19 | 83.83$\pm$1.33
cat | 75.37$\pm$2.23 | 80.87$\pm$0.45 | 74.71$\pm$0.00 | 73.92$\pm$1.23 | 67.11$\pm$3.12 | 83.58$\pm$0.46 | 86.49$\pm$0.92 | 88.59$\pm$0.81
deer | 87.01$\pm$1.24 | 87.88$\pm$0.26 | 76.41$\pm$0.04 | 84.91$\pm$1.01 | 84.94$\pm$1.57 | 90.87$\pm$0.34 | 90.65$\pm$0.38 | 92.13$\pm$0.44
dog | 71.14$\pm$4.30 | 74.42$\pm$0.41 | 47.20$\pm$0.01 | 86.76$\pm$0.97 | 80.01$\pm$0.88 | 75.38$\pm$0.96 | 87.99$\pm$0.85 | 89.86$\pm$1.04
frog | 87.02$\pm$0.60 | 86.23$\pm$0.79 | 76.09$\pm$0.02 | 87.21$\pm$0.63 | 81.87$\pm$2.14 | 86.59$\pm$0.58 | 91.68$\pm$0.37 | 93.05$\pm$0.39
horse | 79.89$\pm$2.61 | 83.68$\pm$0.81 | 47.81$\pm$0.03 | 92.19$\pm$0.93 | 88.92$\pm$1.43 | 85.16$\pm$0.29 | 93.32$\pm$0.26 | 94.16$\pm$0.29
ship | 89.47$\pm$1.31 | 91.23$\pm$0.20 | 78.24$\pm$5.61 | 92.62$\pm$0.51 | 89.91$\pm$1.71 | 90.06$\pm$0.47 | 94.64$\pm$0.50 | 95.76$\pm$0.48
truck | 90.61$\pm$1.40 | 92.90$\pm$0.49 | 62.33$\pm$3.55 | 90.90$\pm$1.04 | 93.10$\pm$0.35 | 93.60$\pm$0.32 | 96.44$\pm$0.48 | 96.98$\pm$0.31
average | 83.24$\pm$0.91 | 85.19$\pm$0.13 | 59.86$\pm$0.03 | 85.89$\pm$0.08 | 83.58$\pm$0.21 | 86.69$\pm$0.57 | 90.81$\pm$0.15 | 91.56$\pm$0.12

Table 5: AUROC (%) on CIFAR-10 for UAD using ResNet-50 as extractor when
$p=0.1$. The best performance is in bold.

Inlier class name | IF | OCSVM | DAGMM | E3Outlier | RSRAE | PANDA | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---|---
airplane | 81.51$\pm$1.50 | 80.18$\pm$0.38 | 76.62$\pm$0.03 | 76.22$\pm$0.71 | 77.04$\pm$0.75 | 83.39$\pm$0.88 | 85.45$\pm$0.27 | 86.96$\pm$0.41
automobile | 90.01$\pm$1.58 | 91.28$\pm$0.12 | 87.76$\pm$0.01 | 94.32$\pm$0.52 | 88.78$\pm$0.43 | 94.63$\pm$0.54 | 95.62$\pm$0.30 | 96.21$\pm$0.26
bird | 67.18$\pm$1.28 | 67.23$\pm$0.53 | 67.29$\pm$0.01 | 71.18$\pm$2.39 | 62.03$\pm$1.48 | 73.37$\pm$2.11 | 76.36$\pm$0.53 | 77.66$\pm$0.38
cat | 70.71$\pm$1.10 | 75.08$\pm$0.60 | 77.23$\pm$0.02 | 62.54$\pm$1.97 | 52.22$\pm$1.19 | 81.97$\pm$0.74 | 82.00$\pm$0.99 | 83.38$\pm$0.88
deer | 84.11$\pm$0.18 | 83.12$\pm$0.29 | 82.01$\pm$0.04 | 80.76$\pm$1.36 | 76.23$\pm$1.86 | 88.46$\pm$0.62 | 88.16$\pm$0.47 | 89.73$\pm$0.58
dog | 64.42$\pm$3.11 | 67.10$\pm$0.38 | 52.12$\pm$0.01 | 76.67$\pm$1.05 | 71.12$\pm$0.83 | 69.70$\pm$1.78 | 82.58$\pm$1.13 | 85.42$\pm$0.57
frog | 83.13$\pm$1.53 | 78.79$\pm$0.51 | 75.78$\pm$0.02 | 80.33$\pm$2.00 | 69.91$\pm$2.19 | 82.36$\pm$0.82 | 87.46$\pm$0.54 | 89.75$\pm$0.52
horse | 73.82$\pm$2.10 | 77.11$\pm$0.31 | 52.79$\pm$0.01 | 87.31$\pm$0.87 | 80.31$\pm$0.81 | 81.97$\pm$0.51 | 91.46$\pm$0.17 | 92.38$\pm$0.29
ship | 85.73$\pm$1.76 | 85.85$\pm$0.38 | 72.72$\pm$18.01 | 90.52$\pm$1.21 | 84.04$\pm$1.19 | 88.63$\pm$0.73 | 92.51$\pm$0.17 | 93.85$\pm$0.33
truck | 89.08$\pm$0.68 | 88.69$\pm$0.18 | 84.87$\pm$4.69 | 90.13$\pm$0.78 | 89.40$\pm$0.64 | 91.77$\pm$0.67 | 95.17$\pm$0.33 | 95.78$\pm$0.29
average | 79.36$\pm$0.81 | 79.52$\pm$0.10 | 67.04$\pm$2.19 | 80.58$\pm$0.35 | 75.05$\pm$0.32 | 83.63$\pm$0.94 | 87.68$\pm$0.25 | 87.69$\pm$0.18

Table 6: AUROC (%) on CIFAR-10 for UAD using ResNet-50 as extractor when
$p=0.3$. The best performance is in bold.

Inlier class name | IF | OCSVM | DAGMM | E3Outlier | RSRAE | PANDA | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---|---
airplane | 79.51$\pm$1.78 | 75.67$\pm$0.42 | 79.02$\pm$3.22 | 73.79$\pm$1.01 | 73.68$\pm$0.51 | 83.61$\pm$1.15 | 81.76$\pm$0.43 | 84.17$\pm$1.14
automobile | 87.57$\pm$1.39 | 86.88$\pm$0.28 | 86.11$\pm$0.02 | 93.32$\pm$0.54 | 86.02$\pm$0.87 | 92.92$\pm$0.77 | 94.38$\pm$0.44 | 95.31$\pm$0.33
bird | 65.67$\pm$2.08 | 63.29$\pm$0.43 | 72.02$\pm$2.41 | 65.53$\pm$2.19 | 57.62$\pm$0.87 | 72.25$\pm$2.37 | 72.76$\pm$0.69 | 73.20$\pm$0.79
cat | 67.61$\pm$2.60 | 71.20$\pm$0.43 | 78.35$\pm$2.03 | 55.78$\pm$1.70 | 48.51$\pm$1.41 | 79.67$\pm$1.24 | 77.74$\pm$0.40 | 79.95$\pm$0.18
deer | 82.13$\pm$1.56 | 79.14$\pm$0.15 | 77.44$\pm$2.31 | 76.01$\pm$1.41 | 72.58$\pm$1.28 | 88.24$\pm$0.96 | 85.74$\pm$0.47 | 88.04$\pm$0.33
dog | 59.58$\pm$3.01 | 62.38$\pm$0.18 | 59.76$\pm$0.28 | 72.12$\pm$3.14 | 63.19$\pm$0.76 | 70.48$\pm$2.12 | 77.76$\pm$0.85 | 81.12$\pm$1.14
frog | 80.52$\pm$1.11 | 73.02$\pm$0.45 | 85.67$\pm$7.02 | 74.59$\pm$0.73 | 64.81$\pm$1.00 | 79.89$\pm$1.18 | 81.83$\pm$0.64 | 85.26$\pm$1.69
horse | 71.08$\pm$1.37 | 72.17$\pm$0.16 | 53.93$\pm$11.51 | 85.38$\pm$1.04 | 78.03$\pm$1.12 | 80.92$\pm$0.95 | 90.16$\pm$0.26 | 91.50$\pm$0.40
ship | 83.20$\pm$1.91 | 80.62$\pm$0.33 | 84.14$\pm$1.97 | 89.51$\pm$0.58 | 80.42$\pm$0.89 | 89.22$\pm$1.38 | 90.12$\pm$0.42 | 92.38$\pm$0.41
truck | 86.32$\pm$0.89 | 83.29$\pm$0.07 | 75.03$\pm$11.55 | 87.58$\pm$0.78 | 87.02$\pm$0.44 | 90.32$\pm$0.74 | 94.17$\pm$0.25 | 95.35$\pm$0.54
average | 75.94$\pm$0.91 | 74.83$\pm$0.14 | 62.73$\pm$0.81 | 77.44$\pm$0.62 | 71.27$\pm$0.33 | 82.75$\pm$1.11 | 84.64$\pm$0.10 | 84.71$\pm$0.27

Table 7: AUROC (%) on CIFAR-10 for UAD using ResNet-50 as extractor when
$p=0.5$. The best performance is in bold.

Dataset | Inlier class name | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | alt.atheism | 55.29$\pm$3.70 | 95.59$\pm$0.00 | 59.36$\pm$3.59 | 94.52$\pm$1.24 | 96.11$\pm$0.19 | 96.20$\pm$0.34
comp.graphics | 68.14$\pm$2.53 | 65.23$\pm$0.00 | 58.41$\pm$3.04 | 84.23$\pm$2.55 | 83.73$\pm$0.71 | 88.62$\pm$0.33
comp.os.ms-windows.misc | 64.75$\pm$1.71 | 82.64$\pm$0.00 | 56.84$\pm$4.48 | 88.45$\pm$2.51 | 89.35$\pm$0.87 | 91.98$\pm$0.27
comp.sys.ibm.pc.hardware | 63.77$\pm$2.39 | 73.21$\pm$0.00 | 61.36$\pm$5.04 | 84.36$\pm$5.44 | 86.89$\pm$0.90 | 90.16$\pm$0.14
comp.sys.mac.hardware | 60.83$\pm$2.33 | 73.96$\pm$0.00 | 56.03$\pm$2.13 | 88.69$\pm$2.80 | 88.06$\pm$0.68 | 91.86$\pm$0.21
comp.windows.x | 65.17$\pm$5.17 | 69.78$\pm$0.00 | 54.70$\pm$5.14 | 86.52$\pm$1.91 | 91.21$\pm$0.50 | 94.07$\pm$0.30
misc.forsale | 67.01$\pm$1.57 | 73.47$\pm$0.00 | 58.39$\pm$5.40 | 86.99$\pm$4.28 | 85.76$\pm$0.82 | 90.74$\pm$0.53
rec.autos | 55.60$\pm$3.88 | 70.76$\pm$0.00 | 57.86$\pm$5.02 | 87.45$\pm$4.69 | 88.33$\pm$0.97 | 91.46$\pm$0.38
rec.motorcycles | 52.12$\pm$4.00 | 87.94$\pm$0.00 | 54.78$\pm$3.63 | 92.51$\pm$1.90 | 93.40$\pm$0.50 | 95.66$\pm$0.23
rec.sport.baseball | 62.46$\pm$2.82 | 82.73$\pm$0.00 | 61.20$\pm$3.65 | 91.39$\pm$1.68 | 92.91$\pm$0.37 | 95.29$\pm$0.18
rec.sport.hockey | 59.82$\pm$2.13 | 92.38$\pm$0.00 | 53.36$\pm$10.76 | 94.98$\pm$0.99 | 95.75$\pm$0.47 | 97.12$\pm$0.14
sci.crypt | 46.89$\pm$2.39 | 93.50$\pm$0.00 | 61.73$\pm$2.82 | 97.23$\pm$0.55 | 95.58$\pm$0.29 | 96.14$\pm$0.14
sci.electronics | 59.80$\pm$2.38 | 61.22$\pm$0.00 | 58.41$\pm$4.79 | 76.29$\pm$3.05 | 79.99$\pm$1.03 | 86.73$\pm$0.38
sci.med | 56.97$\pm$2.58 | 77.48$\pm$0.00 | 47.21$\pm$5.49 | 83.93$\pm$0.95 | 89.64$\pm$0.22 | 91.89$\pm$0.48
sci.space | 53.04$\pm$3.65 | 87.06$\pm$0.00 | 48.34$\pm$4.43 | 90.84$\pm$1.93 | 92.81$\pm$0.39 | 93.71$\pm$0.24
soc.religion.christian | 51.60$\pm$3.59 | 91.81$\pm$0.00 | 51.52$\pm$8.33 | 90.45$\pm$1.72 | 93.00$\pm$0.35 | 94.38$\pm$0.26
talk.politics.guns | 49.52$\pm$2.51 | 92.25$\pm$0.00 | 56.55$\pm$7.24 | 92.67$\pm$1.67 | 93.42$\pm$0.29 | 95.44$\pm$0.12
talk.politics.mideast | 43.62$\pm$3.10 | 96.00$\pm$0.00 | 45.74$\pm$6.15 | 94.92$\pm$2.20 | 96.71$\pm$0.14 | 97.53$\pm$0.12
talk.politics.misc | 43.73$\pm$2.04 | 86.44$\pm$0.00 | 48.90$\pm$6.42 | 90.04$\pm$1.24 | 91.84$\pm$0.41 | 93.20$\pm$0.29
talk.religion.misc | 48.89$\pm$5.79 | 87.45$\pm$0.00 | 46.01$\pm$5.13 | 84.17$\pm$3.43 | 90.84$\pm$0.35 | 91.89$\pm$0.17
average | 56.45$\pm$0.67 | 82.04$\pm$0.00 | 54.84$\pm$2.27 | 89.03$\pm$2.34 | 90.77$\pm$0.12 | 93.20$\pm$0.07
Reuters-21578 | exchanges | 75.79$\pm$0.94 | 96.81$\pm$0.00 | 77.35$\pm$7.09 | 96.66$\pm$1.34 | 96.89$\pm$0.25 | 98.27$\pm$0.12
organizations | 66.01$\pm$3.94 | 69.98$\pm$0.00 | 62.53$\pm$5.03 | 76.50$\pm$4.25 | 80.73$\pm$0.31 | 86.72$\pm$0.57
people | 58.43$\pm$3.70 | 97.81$\pm$0.00 | 63.61$\pm$8.67 | 95.67$\pm$0.14 | 97.77$\pm$0.08 | 98.23$\pm$0.15
places | 52.74$\pm$1.89 | 96.70$\pm$0.00 | 59.74$\pm$7.47 | 96.47$\pm$2.09 | 97.12$\pm$0.07 | 97.93$\pm$0.11
topics | 43.53$\pm$3.31 | 93.46$\pm$0.00 | 62.11$\pm$8.28 | 96.90$\pm$0.70 | 95.03$\pm$0.24 | 97.71$\pm$0.08
| average | 59.30$\pm$0.82 | 90.95$\pm$0.00 | 65.07$\pm$4.12 | 92.44$\pm$1.70 | 93.51$\pm$0.09 | 95.77$\pm$0.12

Table 8: AUROC (%) on text datasets for UAD when $p=0.1$. The best performance
is in bold.

Dataset | Inlier class name | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | alt.atheism | 12.41$\pm$0.30 | 78.86$\pm$0.00 | 13.12$\pm$1.92 | 73.39$\pm$3.24 | 79.37$\pm$1.62 | 83.86$\pm$0.93
comp.graphics | 18.61$\pm$1.73 | 18.66$\pm$0.00 | 14.40$\pm$2.31 | 36.56$\pm$5.32 | 31.62$\pm$3.30 | 63.21$\pm$1.62
comp.os.ms-windows.misc | 16.01$\pm$0.77 | 34.31$\pm$0.00 | 11.70$\pm$1.42 | 40.66$\pm$5.97 | 43.19$\pm$3.50 | 67.91$\pm$1.50
comp.sys.ibm.pc.hardware | 21.65$\pm$1.63 | 26.66$\pm$0.00 | 18.60$\pm$4.95 | 35.72$\pm$9.73 | 37.26$\pm$2.86 | 64.24$\pm$1.50
comp.sys.mac.hardware | 21.46$\pm$3.09 | 28.15$\pm$0.00 | 13.96$\pm$6.04 | 41.90$\pm$9.32 | 37.85$\pm$3.00 | 69.89$\pm$0.62
comp.windows.x | 16.19$\pm$2.55 | 19.63$\pm$0.00 | 13.50$\pm$4.18 | 36.58$\pm$2.78 | 40.45$\pm$3.01 | 66.94$\pm$0.88
misc.forsale | 23.36$\pm$1.56 | 29.37$\pm$0.00 | 15.40$\pm$3.30 | 45.05$\pm$6.36 | 35.80$\pm$3.90 | 65.81$\pm$2.33
rec.autos | 14.24$\pm$0.78 | 29.01$\pm$0.00 | 16.74$\pm$8.13 | 41.00$\pm$9.34 | 39.49$\pm$3.19 | 65.68$\pm$1.22
rec.motorcycles | 13.65$\pm$2.01 | 62.64$\pm$0.00 | 13.33$\pm$2.48 | 68.45$\pm$5.61 | 67.31$\pm$3.54 | 78.87$\pm$1.11
rec.sport.baseball | 16.72$\pm$1.20 | 50.43$\pm$0.00 | 13.77$\pm$2.66 | 54.76$\pm$3.15 | 55.98$\pm$1.14 | 76.46$\pm$1.14
rec.sport.hockey | 17.89$\pm$0.74 | 54.61$\pm$0.00 | 14.66$\pm$7.16 | 63.95$\pm$4.02 | 60.47$\pm$2.78 | 78.86$\pm$0.82
sci.crypt | 9.45$\pm$0.77 | 64.29$\pm$0.00 | 13.91$\pm$1.25 | 72.41$\pm$3.19 | 67.49$\pm$1.77 | 78.35$\pm$0.67
sci.electronics | 22.18$\pm$2.45 | 17.22$\pm$0.00 | 13.66$\pm$3.73 | 26.17$\pm$2.08 | 26.27$\pm$4.36 | 57.72$\pm$0.63
sci.med | 12.97$\pm$1.28 | 34.09$\pm$0.00 | 9.07$\pm$1.22 | 36.44$\pm$2.60 | 43.05$\pm$3.53 | 66.93$\pm$0.68
sci.space | 11.58$\pm$1.85 | 48.89$\pm$0.00 | 9.58$\pm$2.61 | 48.53$\pm$4.43 | 55.44$\pm$3.65 | 73.91$\pm$0.65
soc.religion.christian | 12.45$\pm$0.73 | 49.12$\pm$0.00 | 11.03$\pm$1.98 | 49.60$\pm$3.50 | 48.09$\pm$2.01 | 70.85$\pm$1.19
talk.politics.guns | 10.80$\pm$0.87 | 64.36$\pm$0.00 | 12.34$\pm$4.22 | 58.73$\pm$4.68 | 66.14$\pm$1.67 | 77.53$\pm$0.40
talk.politics.mideast | 9.51$\pm$1.56 | 67.09$\pm$0.00 | 9.59$\pm$2.47 | 68.82$\pm$8.68 | 70.68$\pm$1.24 | 83.68$\pm$1.30
talk.politics.misc | 9.83$\pm$0.88 | 52.73$\pm$0.00 | 8.58$\pm$1.22 | 56.43$\pm$4.94 | 53.83$\pm$2.23 | 72.10$\pm$0.55
talk.religion.misc | 10.92$\pm$1.26 | 46.30$\pm$0.00 | 8.21$\pm$0.93 | 40.69$\pm$6.71 | 51.59$\pm$1.56 | 70.65$\pm$0.77
average | 15.09$\pm$0.22 | 43.82$\pm$0.00 | 12.76$\pm$1.06 | 49.79$\pm$5.28 | 50.57$\pm$0.65 | 71.77$\pm$0.34
Reuters-21578 | exchanges | 36.21$\pm$0.64 | 71.23$\pm$0.00 | 38.72$\pm$11.34 | 73.37$\pm$9.35 | 71.77$\pm$3.63 | 87.68$\pm$1.00
organizations | 25.86$\pm$4.09 | 20.11$\pm$0.00 | 18.23$\pm$5.66 | 32.19$\pm$6.35 | 34.19$\pm$0.78 | 60.47$\pm$0.87
people | 11.94$\pm$1.39 | 71.16$\pm$0.00 | 17.14$\pm$5.29 | 67.57$\pm$4.98 | 71.39$\pm$1.83 | 85.48$\pm$0.86
places | 11.85$\pm$1.38 | 68.71$\pm$0.00 | 15.65$\pm$5.04 | 70.04$\pm$6.79 | 68.31$\pm$3.35 | 85.00$\pm$0.92
topics | 7.82$\pm$0.45 | 77.23$\pm$0.00 | 21.93$\pm$12.22 | 79.04$\pm$6.10 | 77.26$\pm$1.53 | 88.35$\pm$0.82
| average | 18.74$\pm$0.93 | 61.69$\pm$0.00 | 22.33$\pm$3.62 | 64.44$\pm$6.71 | 64.58$\pm$1.34 | 81.40$\pm$0.32

Table 9: AUPR (%) on text datasets for UAD when $p=0.1$. The best performance
is in bold.

Dataset | Inlier class name | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | alt.atheism | 56.28$\pm$1.11 | 88.77$\pm$0.00 | 58.88$\pm$8.15 | 94.79$\pm$0.83 | 94.01$\pm$0.20 | 95.36$\pm$0.19
comp.graphics | 67.45$\pm$1.50 | 64.61$\pm$0.00 | 56.44$\pm$3.78 | 81.69$\pm$1.27 | 78.01$\pm$0.84 | 85.31$\pm$0.78
comp.os.ms-windows.misc | 65.54$\pm$3.20 | 75.41$\pm$0.00 | 58.71$\pm$4.46 | 88.43$\pm$0.72 | 86.04$\pm$0.31 | 89.69$\pm$0.30
comp.sys.ibm.pc.hardware | 66.70$\pm$2.86 | 67.15$\pm$0.00 | 54.50$\pm$2.96 | 84.51$\pm$1.28 | 81.46$\pm$0.85 | 86.95$\pm$0.93
comp.sys.mac.hardware | 66.15$\pm$1.20 | 67.72$\pm$0.00 | 56.47$\pm$2.82 | 87.35$\pm$1.71 | 82.56$\pm$0.49 | 88.18$\pm$0.41
comp.windows.x | 64.89$\pm$1.87 | 63.71$\pm$0.00 | 56.31$\pm$1.03 | 84.89$\pm$0.82 | 81.89$\pm$0.53 | 87.55$\pm$0.32
misc.forsale | 66.26$\pm$2.01 | 64.54$\pm$0.00 | 59.40$\pm$4.67 | 86.67$\pm$1.01 | 78.77$\pm$0.83 | 85.26$\pm$0.74
rec.autos | 57.95$\pm$1.62 | 69.99$\pm$0.00 | 51.62$\pm$3.82 | 86.59$\pm$1.73 | 83.80$\pm$0.43 | 88.41$\pm$0.55
rec.motorcycles | 57.71$\pm$1.11 | 73.54$\pm$0.00 | 50.54$\pm$3.15 | 92.41$\pm$1.00 | 89.02$\pm$0.41 | 92.45$\pm$0.31
rec.sport.baseball | 64.57$\pm$2.06 | 72.12$\pm$0.00 | 58.86$\pm$2.88 | 90.86$\pm$1.92 | 88.19$\pm$0.59 | 91.75$\pm$0.34
rec.sport.hockey | 61.35$\pm$1.78 | 85.87$\pm$0.00 | 56.12$\pm$3.93 | 94.25$\pm$0.95 | 92.45$\pm$0.49 | 94.44$\pm$0.41
sci.crypt | 53.63$\pm$1.62 | 90.63$\pm$0.00 | 55.95$\pm$5.21 | 96.89$\pm$0.32 | 93.59$\pm$0.19 | 94.88$\pm$0.20
sci.electronics | 59.84$\pm$2.64 | 57.25$\pm$0.00 | 54.57$\pm$3.14 | 77.03$\pm$2.45 | 72.75$\pm$0.11 | 80.84$\pm$0.36
sci.med | 55.63$\pm$1.61 | 71.56$\pm$0.00 | 47.53$\pm$2.51 | 83.20$\pm$1.87 | 84.28$\pm$0.71 | 88.71$\pm$0.61
sci.space | 54.62$\pm$1.82 | 77.32$\pm$0.00 | 53.03$\pm$7.49 | 91.16$\pm$0.98 | 89.11$\pm$0.16 | 91.80$\pm$0.25
soc.religion.christian | 52.36$\pm$2.13 | 82.46$\pm$0.00 | 61.11$\pm$5.88 | 90.11$\pm$1.59 | 87.99$\pm$0.44 | 90.39$\pm$0.47
talk.politics.guns | 48.15$\pm$1.56 | 83.58$\pm$0.00 | 52.52$\pm$5.13 | 92.55$\pm$0.88 | 91.46$\pm$0.19 | 93.57$\pm$0.32
talk.politics.mideast | 47.20$\pm$3.43 | 86.40$\pm$0.00 | 43.65$\pm$5.28 | 95.30$\pm$1.19 | 93.39$\pm$0.31 | 94.94$\pm$0.21
talk.politics.misc | 47.73$\pm$3.22 | 80.08$\pm$0.00 | 47.48$\pm$4.14 | 91.81$\pm$1.58 | 89.22$\pm$0.40 | 91.84$\pm$0.34
talk.religion.misc | 52.96$\pm$1.35 | 78.64$\pm$0.00 | 47.62$\pm$2.46 | 86.63$\pm$1.93 | 87.86$\pm$0.14 | 90.87$\pm$0.21
average | 58.35$\pm$0.45 | 75.07$\pm$0.00 | 54.07$\pm$0.57 | 88.86$\pm$1.30 | 86.29$\pm$0.11 | 90.16$\pm$0.11
Reuters-21578 | exchanges | 76.45$\pm$1.50 | 90.49$\pm$0.00 | 71.75$\pm$5.28 | 95.18$\pm$0.44 | 95.27$\pm$0.15 | 97.06$\pm$0.22
organizations | 67.49$\pm$3.11 | 57.04$\pm$0.00 | 60.68$\pm$4.51 | 63.66$\pm$2.88 | 60.48$\pm$0.32 | 89.91$\pm$0.75
people | 58.40$\pm$2.89 | 89.15$\pm$0.00 | 52.18$\pm$6.94 | 93.75$\pm$1.22 | 94.72$\pm$0.16 | 96.78$\pm$0.12
places | 48.81$\pm$2.12 | 88.10$\pm$0.00 | 56.93$\pm$5.25 | 94.64$\pm$1.00 | 94.82$\pm$0.10 | 97.06$\pm$0.16
topics | 43.18$\pm$2.63 | 91.41$\pm$0.00 | 55.11$\pm$8.95 | 96.25$\pm$0.49 | 95.04$\pm$0.16 | 96.79$\pm$0.17
| average | 58.86$\pm$0.74 | 83.24$\pm$0.00 | 59.33$\pm$2.68 | 88.70$\pm$1.21 | 88.07$\pm$0.07 | 95.52$\pm$0.13

Table 10: AUROC (%) on text datasets for UAD when $p=0.3$. The best
performance is in bold.

Dataset | Inlier class name | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | alt.atheism | 32.34$\pm$1.21 | 79.18$\pm$0.00 | 30.32$\pm$7.71 | 87.05$\pm$1.62 | 85.45$\pm$1.17 | 89.27$\pm$0.87
comp.graphics | 38.58$\pm$0.91 | 32.30$\pm$0.00 | 29.75$\pm$3.33 | 51.82$\pm$1.83 | 47.93$\pm$1.08 | 64.49$\pm$1.70
comp.os.ms-windows.misc | 35.88$\pm$2.29 | 45.57$\pm$0.00 | 28.80$\pm$3.46 | 64.87$\pm$1.82 | 58.19$\pm$0.98 | 70.33$\pm$0.46
comp.sys.ibm.pc.hardware | 46.80$\pm$3.94 | 38.06$\pm$0.00 | 28.80$\pm$1.53 | 57.72$\pm$1.67 | 52.38$\pm$0.87 | 67.26$\pm$1.83
comp.sys.mac.hardware | 45.69$\pm$1.99 | 39.99$\pm$0.00 | 29.23$\pm$1.27 | 62.98$\pm$4.59 | 55.00$\pm$1.39 | 69.85$\pm$1.06
comp.windows.x | 37.76$\pm$1.25 | 34.38$\pm$0.00 | 26.95$\pm$0.41 | 56.12$\pm$2.17 | 51.28$\pm$0.98 | 66.50$\pm$0.79
misc.forsale | 44.86$\pm$2.90 | 38.19$\pm$0.00 | 32.07$\pm$3.83 | 63.22$\pm$1.39 | 50.60$\pm$1.76 | 66.44$\pm$1.98
rec.autos | 34.70$\pm$1.98 | 44.40$\pm$0.00 | 24.94$\pm$2.31 | 61.43$\pm$3.65 | 56.68$\pm$1.01 | 68.83$\pm$1.00
rec.motorcycles | 36.40$\pm$1.00 | 58.58$\pm$0.00 | 24.22$\pm$1.81 | 80.87$\pm$2.31 | 73.08$\pm$1.46 | 82.14$\pm$0.62
rec.sport.baseball | 38.35$\pm$3.57 | 51.58$\pm$0.00 | 30.98$\pm$3.30 | 74.09$\pm$4.16 | 68.20$\pm$1.85 | 77.87$\pm$0.79
rec.sport.hockey | 38.20$\pm$1.91 | 63.70$\pm$0.00 | 31.01$\pm$3.55 | 78.36$\pm$2.29 | 73.33$\pm$2.04 | 81.09$\pm$1.77
sci.crypt | 27.95$\pm$0.82 | 74.10$\pm$0.00 | 28.50$\pm$5.65 | 87.00$\pm$0.81 | 78.60$\pm$0.44 | 83.25$\pm$0.54
sci.electronics | 40.90$\pm$2.68 | 32.59$\pm$0.00 | 29.02$\pm$3.64 | 47.55$\pm$3.10 | 41.80$\pm$0.81 | 57.49$\pm$1.23
sci.med | 29.88$\pm$1.91 | 47.39$\pm$0.00 | 22.64$\pm$2.18 | 55.24$\pm$4.00 | 57.41$\pm$1.62 | 70.40$\pm$2.01
sci.space | 28.94$\pm$1.02 | 55.17$\pm$0.00 | 25.43$\pm$4.01 | 68.36$\pm$1.66 | 68.30$\pm$1.12 | 77.34$\pm$1.41
soc.religion.christian | 30.78$\pm$1.47 | 56.53$\pm$0.00 | 33.81$\pm$8.10 | 70.18$\pm$2.76 | 62.88$\pm$1.52 | 71.69$\pm$2.08
talk.politics.guns | 25.36$\pm$1.23 | 67.53$\pm$0.00 | 25.25$\pm$5.08 | 77.79$\pm$1.58 | 76.87$\pm$0.33 | 83.04$\pm$0.51
talk.politics.mideast | 23.78$\pm$1.88 | 67.66$\pm$0.00 | 20.52$\pm$2.50 | 83.25$\pm$3.11 | 77.06$\pm$0.81 | 82.88$\pm$0.67
talk.politics.misc | 25.65$\pm$1.41 | 62.98$\pm$0.00 | 22.17$\pm$3.27 | 77.70$\pm$3.04 | 70.95$\pm$1.33 | 78.71$\pm$1.49
talk.religion.misc | 28.43$\pm$1.25 | 55.68$\pm$0.00 | 23.20$\pm$2.12 | 66.59$\pm$2.39 | 65.63$\pm$0.53 | 75.39$\pm$0.70
average | 34.56$\pm$0.72 | 52.28$\pm$0.00 | 27.38$\pm$0.82 | 68.61$\pm$2.50 | 63.58$\pm$0.09 | 74.21$\pm$0.17
Reuters-21578 | exchanges | 57.39$\pm$3.64 | 71.58$\pm$0.00 | 51.14$\pm$8.66 | 83.75$\pm$2.41 | 81.89$\pm$0.87 | 90.39$\pm$0.48
organizations | 49.37$\pm$1.98 | 28.15$\pm$0.00 | 33.60$\pm$4.60 | 36.99$\pm$4.06 | 31.43$\pm$0.88 | 84.52$\pm$1.07
people | 26.28$\pm$2.34 | 69.27$\pm$0.00 | 27.53$\pm$4.12 | 78.52$\pm$2.78 | 79.64$\pm$1.30 | 90.09$\pm$0.41
places | 26.17$\pm$1.90 | 67.45$\pm$0.00 | 28.68$\pm$3.25 | 78.66$\pm$2.17 | 79.17$\pm$1.36 | 90.67$\pm$0.65
topics | 20.71$\pm$1.80 | 79.80$\pm$0.00 | 27.12$\pm$6.92 | 87.45$\pm$1.56 | 87.62$\pm$0.75 | 92.73$\pm$0.39
| average | 35.99$\pm$0.70 | 63.25$\pm$0.00 | 33.61$\pm$2.77 | 73.07$\pm$2.60 | 71.95$\pm$0.42 | 89.68$\pm$0.20

Table 11: AUPR (%) on text datasets for UAD when $p=0.3$. The best performance
is in bold.

Dataset | Inlier class name | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | alt.atheism | 55.37$\pm$2.51 | 84.37$\pm$0.00 | 56.22$\pm$6.82 | 93.74$\pm$0.65 | 92.45$\pm$0.16 | 93.29$\pm$0.10
comp.graphics | 66.44$\pm$2.36 | 55.84$\pm$0.00 | 52.66$\pm$5.06 | 77.90$\pm$1.89 | 75.69$\pm$0.61 | 79.34$\pm$0.43
comp.os.ms-windows.misc | 67.75$\pm$2.34 | 68.66$\pm$0.00 | 54.80$\pm$6.22 | 86.54$\pm$0.82 | 84.54$\pm$0.36 | 86.62$\pm$0.37
comp.sys.ibm.pc.hardware | 65.91$\pm$1.58 | 65.63$\pm$0.00 | 59.71$\pm$4.35 | 82.21$\pm$1.60 | 79.41$\pm$0.57 | 82.21$\pm$0.45
comp.sys.mac.hardware | 66.34$\pm$1.31 | 63.12$\pm$0.00 | 56.00$\pm$2.59 | 85.15$\pm$1.24 | 80.53$\pm$0.47 | 82.87$\pm$0.27
comp.windows.x | 63.82$\pm$1.15 | 61.06$\pm$0.00 | 55.17$\pm$2.02 | 81.52$\pm$1.27 | 78.96$\pm$0.18 | 81.92$\pm$0.36
misc.forsale | 67.50$\pm$1.99 | 62.82$\pm$0.00 | 52.94$\pm$2.67 | 82.16$\pm$0.72 | 76.11$\pm$0.42 | 79.49$\pm$0.40
rec.autos | 55.86$\pm$1.07 | 70.15$\pm$0.00 | 49.91$\pm$3.20 | 84.62$\pm$1.59 | 81.96$\pm$0.62 | 84.40$\pm$0.37
rec.motorcycles | 55.01$\pm$0.55 | 70.27$\pm$0.00 | 52.22$\pm$3.01 | 89.93$\pm$2.14 | 86.92$\pm$0.31 | 88.64$\pm$0.35
rec.sport.baseball | 61.85$\pm$2.65 | 69.67$\pm$0.00 | 56.25$\pm$6.17 | 88.45$\pm$1.81 | 86.01$\pm$0.27 | 87.81$\pm$0.19
rec.sport.hockey | 59.34$\pm$2.73 | 76.95$\pm$0.00 | 59.53$\pm$3.58 | 92.81$\pm$0.88 | 90.10$\pm$0.15 | 91.25$\pm$0.12
sci.crypt | 51.13$\pm$1.73 | 83.21$\pm$0.00 | 52.09$\pm$10.96 | 96.29$\pm$0.32 | 92.63$\pm$0.11 | 93.48$\pm$0.17
sci.electronics | 59.41$\pm$2.32 | 59.95$\pm$0.00 | 49.29$\pm$3.30 | 73.16$\pm$2.59 | 71.19$\pm$0.36 | 75.72$\pm$0.39
sci.med | 55.23$\pm$2.43 | 64.81$\pm$0.00 | 46.71$\pm$5.30 | 80.61$\pm$1.20 | 81.55$\pm$0.38 | 84.00$\pm$0.29
sci.space | 52.55$\pm$1.61 | 74.08$\pm$0.00 | 48.36$\pm$3.08 | 89.45$\pm$0.70 | 87.76$\pm$0.41 | 89.27$\pm$0.24
soc.religion.christian | 50.07$\pm$0.92 | 78.20$\pm$0.00 | 57.08$\pm$5.01 | 89.15$\pm$1.60 | 87.14$\pm$0.31 | 88.78$\pm$0.31
talk.politics.guns | 44.45$\pm$1.68 | 80.81$\pm$0.00 | 46.17$\pm$5.30 | 91.27$\pm$1.20 | 90.20$\pm$0.32 | 91.44$\pm$0.17
talk.politics.mideast | 46.62$\pm$2.87 | 82.79$\pm$0.00 | 46.96$\pm$4.29 | 94.02$\pm$1.08 | 91.90$\pm$0.18 | 92.80$\pm$0.13
talk.politics.misc | 45.09$\pm$2.11 | 78.98$\pm$0.00 | 40.78$\pm$2.39 | 89.48$\pm$1.47 | 88.45$\pm$0.19 | 89.48$\pm$0.23
talk.religion.misc | 50.29$\pm$2.74 | 75.32$\pm$0.00 | 45.86$\pm$6.69 | 86.11$\pm$1.46 | 86.14$\pm$0.49 | 88.13$\pm$0.38
average | 57.00$\pm$0.33 | 71.33$\pm$0.00 | 51.94$\pm$1.02 | 86.73$\pm$1.31 | 84.48$\pm$0.11 | 86.55$\pm$0.06
Reuters-21578 | exchanges | 74.68$\pm$2.24 | 80.03$\pm$0.00 | 67.68$\pm$5.85 | 92.53$\pm$1.68 | 91.76$\pm$0.23 | 93.83$\pm$0.22
organizations | 65.09$\pm$1.57 | 48.58$\pm$0.00 | 62.84$\pm$4.04 | 51.99$\pm$2.66 | 46.62$\pm$0.33 | 55.99$\pm$0.54
people | 53.49$\pm$3.23 | 79.78$\pm$0.00 | 60.71$\pm$5.49 | 89.64$\pm$3.78 | 91.05$\pm$0.15 | 93.25$\pm$0.16
places | 48.26$\pm$1.21 | 79.49$\pm$0.00 | 60.54$\pm$4.41 | 87.54$\pm$5.76 | 90.53$\pm$0.14 | 92.54$\pm$0.12
topics | 37.70$\pm$2.64 | 86.50$\pm$0.00 | 58.30$\pm$9.01 | 95.27$\pm$0.43 | 94.41$\pm$0.17 | 94.84$\pm$0.18
| average | 55.85$\pm$0.87 | 74.88$\pm$0.00 | 62.01$\pm$2.54 | 83.39$\pm$2.86 | 82.87$\pm$0.06 | 86.09$\pm$0.13

Table 12: AUROC (%) on text datasets for UAD when $p=0.5$. The best
performance is in bold.

Dataset | Inlier class name | IF | OCSVM | DAGMM | RSRAE | SLA (ours) | SLA2P (ours)
---|---|---|---|---|---|---|---
20 Newsgroups | alt.atheism | 41.80$\pm$1.28 | 79.72$\pm$0.00 | 38.23$\pm$5.56 | 89.16$\pm$0.83 | 88.14$\pm$0.76 | 90.11$\pm$0.50
comp.graphics | 50.38$\pm$1.99 | 39.35$\pm$0.00 | 36.54$\pm$3.53 | 58.07$\pm$1.96 | 57.25$\pm$1.74 | 68.75$\pm$0.90
comp.os.ms-windows.misc | 49.54$\pm$1.91 | 51.78$\pm$0.00 | 37.20$\pm$4.28 | 69.13$\pm$1.55 | 68.15$\pm$0.86 | 75.75$\pm$0.64
comp.sys.ibm.pc.hardware | 55.09$\pm$1.86 | 49.65$\pm$0.00 | 44.41$\pm$4.83 | 64.54$\pm$3.11 | 61.92$\pm$0.97 | 71.14$\pm$0.72
comp.sys.mac.hardware | 56.40$\pm$1.03 | 48.76$\pm$0.00 | 40.85$\pm$4.84 | 70.01$\pm$1.79 | 64.67$\pm$1.25 | 72.49$\pm$0.53
comp.windows.x | 48.97$\pm$0.93 | 44.21$\pm$0.00 | 38.15$\pm$1.59 | 61.97$\pm$2.37 | 60.07$\pm$1.12 | 71.15$\pm$0.69
misc.forsale | 55.78$\pm$2.54 | 46.97$\pm$0.00 | 37.19$\pm$3.33 | 66.20$\pm$1.36 | 58.45$\pm$1.07 | 69.10$\pm$0.81
rec.autos | 42.72$\pm$1.51 | 56.07$\pm$0.00 | 32.65$\pm$2.24 | 68.25$\pm$2.42 | 66.11$\pm$1.85 | 74.91$\pm$0.90
rec.motorcycles | 44.48$\pm$1.12 | 63.35$\pm$0.00 | 35.54$\pm$2.10 | 82.33$\pm$2.47 | 78.90$\pm$1.05 | 82.95$\pm$0.73
rec.sport.baseball | 46.19$\pm$1.98 | 57.15$\pm$0.00 | 39.12$\pm$6.25 | 77.70$\pm$3.58 | 74.16$\pm$1.01 | 80.34$\pm$0.42
rec.sport.hockey | 46.73$\pm$3.28 | 64.72$\pm$0.00 | 42.38$\pm$2.78 | 83.08$\pm$1.32 | 78.62$\pm$0.50 | 83.97$\pm$0.37
sci.crypt | 36.27$\pm$1.20 | 74.18$\pm$0.00 | 35.10$\pm$6.49 | 89.82$\pm$0.55 | 84.09$\pm$0.89 | 88.19$\pm$0.40
sci.electronics | 51.65$\pm$3.19 | 45.37$\pm$0.00 | 33.21$\pm$2.73 | 54.24$\pm$2.96 | 53.56$\pm$0.72 | 64.96$\pm$0.66
sci.med | 39.89$\pm$1.90 | 51.69$\pm$0.00 | 32.02$\pm$3.27 | 63.48$\pm$1.64 | 65.62$\pm$0.45 | 73.90$\pm$0.45
sci.space | 37.12$\pm$1.18 | 59.51$\pm$0.00 | 32.97$\pm$2.01 | 75.09$\pm$2.18 | 76.06$\pm$1.13 | 81.51$\pm$0.61
soc.religion.christian | 37.47$\pm$1.58 | 62.21$\pm$0.00 | 40.12$\pm$6.40 | 76.40$\pm$2.17 | 71.70$\pm$1.39 | 78.83$\pm$0.63
talk.politics.guns | 32.83$\pm$1.22 | 72.04$\pm$0.00 | 30.75$\pm$3.50 | 82.08$\pm$2.11 | 81.78$\pm$0.36 | 85.33$\pm$0.13
talk.politics.mideast | 32.75$\pm$2.12 | 71.31$\pm$0.00 | 31.91$\pm$1.96 | 85.94$\pm$2.04 | 81.87$\pm$0.33 | 86.19$\pm$0.22
talk.politics.misc | 33.10$\pm$2.06 | 69.18$\pm$0.00 | 27.86$\pm$1.10 | 80.16$\pm$2.93 | 78.86$\pm$0.40 | 82.97$\pm$0.47
talk.religion.misc | 36.14$\pm$2.19 | 62.59$\pm$0.00 | 32.33$\pm$3.58 | 73.37$\pm$2.50 | 73.69$\pm$1.48 | 80.12$\pm$0.77
average | 43.77$\pm$0.21 | 58.49$\pm$0.00 | 35.93$\pm$0.62 | 73.55$\pm$2.09 | 71.18$\pm$0.22 | 78.13$\pm$0.11
Reuters-21578 | exchanges | 62.72$\pm$2.18 | 66.11$\pm$0.00 | 55.81$\pm$7.96 | 83.71$\pm$3.17 | 80.48$\pm$0.89 | 87.02$\pm$0.48
organizations | 53.76$\pm$3.25 | 34.05$\pm$0.00 | 44.81$\pm$4.25 | 38.47$\pm$2.70 | 33.05$\pm$0.21 | 49.14$\pm$0.63
people | 34.28$\pm$2.33 | 67.09$\pm$0.00 | 44.57$\pm$7.30 | 78.33$\pm$6.13 | 80.38$\pm$0.66 | 86.85$\pm$0.40
places | 33.34$\pm$0.93 | 66.53$\pm$0.00 | 44.53$\pm$5.87 | 74.31$\pm$8.89 | 79.54$\pm$0.84 | 85.62$\pm$0.31
topics | 26.38$\pm$1.08 | 78.00$\pm$0.00 | 43.00$\pm$9.53 | 89.82$\pm$0.51 | 89.74$\pm$0.35 | 90.97$\pm$0.30
| average | 42.10$\pm$0.92 | 62.36$\pm$0.00 | 46.54$\pm$2.09 | 72.93$\pm$4.28 | 72.64$\pm$0.15 | 79.92$\pm$0.19

Table 13: AUPR (%) on text datasets for UAD when $p=0.5$. The best performance
is in bold.