# Limit Distribution Theory for the Smooth 1-Wasserstein Distance with
Applications

Ritwik<EMAIL_ADDRESS>[ Ziv
<EMAIL_ADDRESS>[ Kengo Katolabel=e3 [
<EMAIL_ADDRESS>Department of Statistics and Data Science, Cornell
University.  Department of Electrical and Computer Engineering, Cornell
Univeristy.

###### Abstract

The smooth 1-Wasserstein distance (SWD)
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ was recently proposed as a means to
mitigate the curse of dimensionality in empirical approximation, while
preserving the Wasserstein structure. Indeed, SWD exhibits parametric
convergence rates and inherits the metric and topological structure of the
classic Wasserstein distance. Motivated by the above, this work conducts a
thorough statistical study of the SWD, including a high-dimensional limit
distribution result for empirical $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$,
bootstrap consistency, concentration inequalities, and Berry-Esseen type
bounds. The derived nondegenerate limit stands in sharp contrast with the
classic empirical $\mathsf{W}_{1}$, for which a similar result is known only
in the one-dimensional case. We also explore asymptotics and characterize the
limit distribution when the smoothing parameter $\sigma$ is scaled with $n$,
converging to $0$ at a sufficiently slow rate. The dimensionality of the
sampled distribution enters empirical SWD convergence bounds only through the
prefactor (i.e., the constant). We provide a sharp characterization of this
prefactor’s dependence on the smoothing parameter and the intrinsic dimension.
This result is then used to derive new empirical convergence rates for classic
$\mathsf{W}_{1}$ in terms of the intrinsic dimension. As applications of the
limit distribution theory, we study two-sample testing and minimum distance
estimation (MDE) under $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$. We establish
asymptotic validity of SWD testing, while for MDE, we prove measurability,
almost sure convergence, and limit distributions for optimal estimators and
their corresponding $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ error. Our results
suggest that the SWD is well suited for high-dimensional statistical learning
and inference.

intrinsic dimension,

limit theorems,

multivariate homogeneity testing,

optimal transport,

smooth Wasserstein distance,

strong approximation.,

###### keywords:

, and

##  1 Introduction

Optimal transport (OT) is a versatile framework for quantifying discrepancy
between probability distributions. It has a longstanding history in
mathematics, dating back to the Monge in the 18th century [64]. Recently, OT
has emerged as a tool of choice for a myriad of machine learning tasks,
encompassing clustering [52], computer vision [75, 76, 59], generative
modeling [7, 49], natural language processing [6, 90, 48], and model
ensembling [32]. This wide adoption was driven by recent computational
advances [25, 79, 1] and the rich geometric structure OT-based discrepancy
measures, like Wasserstein distances, induce on the space of probability
measures.111The Wasserstein space, i.e., the space of probability measure
endowed with the $p$-Wasserstein distance, is a geodesic (in particular,
metric) space, with induced weak* topology. However, Wasserstein distances
suffer from the so-called curse of dimensionality (CoD) when distributions are
estimated from samples, with empirical convergence rates of $n^{-1/d}$, where
$d>3$ is the dimension. This slow rate is burdensome given the high-
dimensionality of real-world data in modern applications.

Recently, smoothing the underlying distribution via convolution with a
Gaussian kernel was proposed as a means for mitigating the CoD while
preserving the classic Wasserstein structure. Specifically, for a smoothing
parameter $\sigma>0$ and $p\geq 1$, consider
$\mathsf{W}_{p}^{\mspace{1.0mu}\sigma}(P,Q):=\mathsf{W}_{p}(P\ast\mathcal{N}_{\sigma},Q\ast\mathcal{N}_{\sigma})$,
where $\mathcal{N}_{\sigma}=\mathcal{N}(0,\sigma^{2}\mathrm{I}_{d})$ is the
$d$-dimensional isotropic Gaussian measure of parameter $\sigma$,
$P\ast\mathcal{N}_{\sigma}$ is the convolution of $P$ and
$\mathcal{N}_{\sigma}$, and $\mathsf{W}_{p}$ is the regular $p$-Wasserstein
distance. In contrast to the CoD rates, [45] showed that several popular
statistical divergences, including $\mathsf{W}_{1}$ and $\mathsf{W}_{2}$,
enjoy fast empirical convergence of $n^{-1/2}$ once distributions are smoothed
by $\mathcal{N}_{\sigma}$. The followup work [43] focused on
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ and showed that the smooth distance
inherits the metric and topological structure of its classic counterpart. That
work also established regularity properties of
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ in $\sigma$, and quantified the gap
due to smoothing as
$\big{|}\mathsf{W}_{1}(P,Q)-\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q)\big{|}=O(\sigma\sqrt{d})$.
These results were later extended to general $p\geq 1$ [68]. Motivated by the
above, herein we conduct an in-depth study of the statistical properties of
the smooth Wasserstein distance (SWD) $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$,
exploring the high dimensional limit distribution of the empirical distance,
bootstrapping, Berry-Esseen type bounds, vanishing $\sigma$ analysis, as well
as several applications.

###  1.1 Contributions

Our first main result characterizes the limit distribution of the scaled
empirical SWD $\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ in any
dimension $d$ (Theorem 1). Here, $P_{n}:=n^{-1}\sum_{i=1}^{n}\delta_{X_{i}}$
is the empirical measure based on $n$ independently and identically
distributed (i.i.d.) samples, $X_{1},\ldots,X_{n}$, from $P$. The analysis
relies on the Kantorovich-Rubinstein (KR) duality [87], which allows
representing $\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ as a
supremum of an empirical process indexed by the class of 1-Lipschitz functions
convolved with a Gaussian density. We prove that this function class is
Donsker (i.e., satisfies the uniform central limit theorem (CLT)) under a
polynomial moment condition on $P$. By the continuous mapping theorem, we then
conclude that $\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$
converges in distribution to the supremum of a tight Gaussian process. To
enable approximation of the distributional limit, we also prove consistency of
the nonparametric bootstrap (Corollary 2). Our high-dimensional SWD limit
distribution theory stands in sharp contrast to the classic $\mathsf{W}_{1}$
case, for which similar results are known only when $d=1$ [27] (follows from
the CLT in $L^{1}(\mathbb{R})$). Indeed, while 1-Lipschitz functions are
generally not Donsker for $d>1$, the smoothing introduced by the Gaussian
convolution shrinks the class and enables controlling its complexity by
uniformly bounding higher order derivatives. As a consequence, empirical SWD
lends itself well for analysis via techniques from empirical process
theory,222The reader is referred to, e.g., [57, 83, 41] as useful references
on modern empirical process theory. giving rise to our limit distribution
result.

As a corollary of the distributional limit, we bound the expected empirical
SWD as
$\mathbb{E}\big{[}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\big{]}$
$\lesssim_{d,P}\sigma^{-d/2+1}n^{-1/2}\log n$. This result is reminiscent of
[45, Proposition 1], but with a sharper dependence on $\sigma$ in terms of
$d$. In Section 5, we refine this result to show that $d$ can be replaced with
the intrinsic dimension of the population distribution, which is defined via
an entropic restriction of its support. This comes at the cost of a constant
pre-factor that is free of $\sigma$. We also undertake a Berry-Esseen type
analysis of empirical SWD, as well as its empirical bootstrap analogue
(Theorems 2 and 3 in Section 4). Specifically, we bound the absolute
difference between versions of
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ and its
distributional limit in probability, and as a corollary, obtain a bound on the
Prokhorov distance between their distributions. These results are established
under a variable smoothing factor $\sigma_{n}$, possibly vanishing as
$n\to\infty$ at a slow enough rate. The vanishing smoothing parameter analysis
is motivated by noise annealing techniques used in modern machine learning:
the opposite regime, namely $\sigma\to\infty$, has already been analyzed in
[21]. These results, in conjunction with the stability of
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ with respect to (w.r.t.) $\sigma$ [43,
Lemma 1], enable leveraging SWD to analyze classic $\mathsf{W}_{1}$, as
described subsequently.

We consider three applications of the derived theory: (i) dependence of the
classic empirical $\mathsf{W}_{1}$ on the intrinsic dimension of the support;
(ii) minimum distance estimation (MDE) with the SWD; and (iii) two-sample
testing. For the first problem, we derive a bound on
$\mathbb{E}\big{[}\mathsf{W}_{1}(P_{n},P)\big{]}$ that matches the known sharp
convergences rates from [89], while removing the compact support assumption
that previous results required [89, 16, 33]. For MDE problem, which was the
focus on an earlier work by a subset of the authors [44], we establish
measurability and strong consistency of the estimator
$\widehat{\theta}_{n}\in\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})$,
along with almost sure convergence of the associated minimal distance. Moving
to a limit distribution analysis, we characterize the high-dimensional limits
of $\sqrt{n}(\widehat{\theta}_{n}-\theta^{\ast})$ and
$\sqrt{n}\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})$,
establishing $n^{-1/2}$ convergence rates for both quantities in arbitrary
dimension. Similar MDE limit distribution results for $\mathsf{W}_{1}$ are
only known for $d=1$, naturally bottlenecked by the lack of high-dimensional
limit distribution theory for the classic distance [9, 12] (see also [11,
10]). Finally, we consider SWD two-sample testing (for equality of
distribution) and prove consistency of tests based on the bootstrap. These
results pose $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ as a potent tool for
high-dimensional learning and inference, and demonstrate its utility for
analyzing classic Wasserstein distances.

###  1.2 Related Works

The problems of empirical approximation, MDE, and limiting distributions,
studied here for $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$, have received
considerable attention under the classic 1-Wasserstein framework. Since
$\mathsf{W}_{1}$ metrizes weak convergence and convergence of first moments in
$\mathcal{P}_{1}(\mathbb{R}^{d})$, Varadarajan’s theorem (see, for example,
Theorem 11.4.1 in [34]) yields $\mathsf{W}_{1}(P_{n},P)\to 0$ as $n\to\infty$,
almost surely (a.s.), for any $P\in\mathcal{P}_{1}(\mathbb{R}^{d})$. The
convergence rate of the expected value, first studied by Dudley [33], is well
understood with sharp rates known in all dimensions333Except $d=2$, where a
log factor is possibly missing. In particular,
$\mathbb{E}\big{[}\mathsf{W}_{1}(P_{n},P)\big{]}\lesssim n^{-1/d}$ provided
that $P$ has sufficiently many moments (see [38]; see also [17, 15, 31, 16,
58] for extensions of this result to other orders of the Wasserstein distance
and general Polish spaces). Notably, this rate suffers from the CoD. Faster
rates are attainable under entropic restrictions on dimension [33, 16, 89],
albeit still with exponential dependence on the intrinsic dimension.

Despite the comprehensive account of expected $\mathsf{W}_{1}(P_{n},P)$,
limiting distribution results for a scaled version thereof are scarce. In
fact, a limit distribution characterization analogous to that presented herein
for the SWD (i.e., assuming only finiteness of moments) is known for the
classic distance only when $d=1$. Specifically, Theorem 2 in [42] yields that
$\mathsf{Lip}_{1}(\mathbb{R})$ is a $P$-Donsker class if (and only if)
$\sum_{j}P\big{(}[-j,j]^{c}\big{)}^{1/2}<\infty$. Combining with KR duality
and the continuous mapping theorem, we have
$\sqrt{n}\mathsf{W}_{1}(P_{n},P)\stackrel{{\scriptstyle
w}}{{\to}}\|G_{P}\|_{\mathsf{Lip}_{1}(\mathbb{R})}$ for some tight Gaussian
process $G_{P}$ in $\ell^{\infty}(\mathsf{Lip}_{1}(\mathbb{R}))$. An
alternative derivation of the one-dimensional limit distribution is given in
[27], based on the fact that $\mathsf{W}_{1}$ equals the $L^{1}$ distance
between cumulative distribution functions (CDFs) when $d=1$. However, these
arguments do not generalize to higher dimensions. When $d\geq 2$, the function
class $\mathsf{Lip}_{1}(\mathbb{R}^{d})$ is not Donsker; if it was, then
$\mathbb{E}[\mathsf{W}_{1}(P_{n},P)]$ would be of order $O(n^{-1/2})$,
contradicting known lower bounds [38].

For $p$-Wasserstein distances, with general $p\geq 1$, the limit distribution
of $\sqrt{n}\mathsf{W}_{p}^{p}(P_{n},P)$, when $P$ is supported on a finite or
a countable set, was derived in [80] and [82], respectively. Their proof
relies on the directional functional delta method [35]. Also related is [29],
where asymptotic normality of
$\sqrt{n}\big{(}\mathsf{W}_{2}^{2}(P_{n},Q)-\mathbb{E}\big{[}\mathsf{W}_{2}^{2}(P_{n},Q)\big{]}\big{)}$
in arbitrary dimension is derived under the assumption that $Q\neq P$.
Notably, the centering term is the expected empirical distance as opposed to
the population one. This gap was addressed in [61], where the asymptotic
normality of
$\sqrt{n}\big{(}\mathsf{W}_{2}^{2}(\widehat{P}_{n},Q)-\mathsf{W}_{2}^{2}(P,Q)\big{)}$
was established for distributions $P$ and $Q$ with sufficiently smooth
densities. Their estimator $\widehat{P}_{n}$ is the probability measure
associated with a wavelet-based density estimate constructed from samples from
$P$. The limit distribution for the (unmodified) empirical $2$-Wasserstein
distance with $Q=P$ is known only when $d=1$ [28]. The key observation there
is that when $d=1$, the empirical $2$-Wasserstein distance coincides with the
$L^{2}$ distance between the empirical and population quantile functions,
resulting in a non-Gaussian limit distribution. Another related work is [46],
where limit distribution results for smooth empirical total variation distance
and $\chi^{2}$ divergence were derived based on the CLT in Banach spaces. None
of the techniques employed in these works are applicable in our case, which
requires a different analysis as described in Section 3.

Two approaches to alleviate the CoD besides smoothing are slicing [72] and
entropic regularization [25]. Empirical convergence under sliced distances
follows their rates when $d=1$ [66], which amounts to $n^{-1/2}$ for sliced
$\mathsf{W}_{1}$ [69, 60]. A sliced 1-Wasserstein MDE analysis, covering
questions similar to those considered here, was provided in [67]. The authors
proved a limit distribution result for empirical sliced $\mathsf{W}_{1}$, but
their assumptions included a Donsker-type theorem that makes the conclusion
immediate. Entropic regularization of OT also speeds up empirical convergence
rates in some cases. Specifically, the rate of two-sample empirical
convergence under entropic OT (EOT) is $n^{-1/2}$ for smooth costs (thus
excluding entropic $\mathsf{W}_{1}$) with compactly supported distributions
[40], or quadratic cost with subGaussian distributions [63].

A CLT for empirical EOT under quadratic cost was also derived in [63] (see
also [14, 56]. This result is similar to that of [30] for the classic
$\mathsf{W}_{2}$, but is markedly different from ours, as it has the expected
empirical EOT, and not the population one, as the centering constant. While
both sliced Wasserstein distances and EOT can be efficiently computed (sliced
distances amount to simple one-dimensional formulas while EOT employs Sinkhorn
iterations [25, 5]), sliced distances offer a poor proxy of classic ones [18,
Lemma 5.1.4], while EOT forfeits the Wasserstein metric and topological
structure [37, 14]. On the other hand, $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$
preserves the Wasserstein structure and is within an additive
$2\sigma\sqrt{d}$ gap from $\mathsf{W}_{1}$ [43, Lemma 1]. Computational
aspects of the SWD, however, are still premature.

##  2 Background and Preliminaries

###  2.1 Notation

Let $\|\cdot\|$ denote the Euclidean norm, and $x\cdot y$, for
$x,y\in\mathbb{R}^{d}$, designate the inner product. The diameter of a set
$X\subset\mathbb{R}^{d}$ is $\mathsf{diam}(X):=\sup_{x,y\in X}\|x-y\|$. In a
metric space $(S,d)$, $B(x,\epsilon)$ denotes the open ball of radius
$\epsilon$ around $x$: if $x$ is the origin in $\mathbb{R}^{d}$ it is omitted.
For a subset $A$ of a metric space, we denote its $\epsilon$-fattening as
$A^{\epsilon}:=\\{x\in S:d(x,A)\leq\epsilon\\}$. We write $a\lesssim_{x}b$
when $a\leq C_{x}b$ for a constant $C_{x}$ that depends only on $x$
($a\lesssim b$ means the hidden constant is absolute), and $a_{n}\ll b_{n}$
for $\lim_{n\to\infty}a_{n}/b_{n}\to 0$. We denote by $a\vee b$ and $a\wedge
b$ the maximum and minimum of $a$ and $b$, respectively.

We denote by $(\Omega,\mathcal{A},\mathbb{P})$ the underlying probability
space on which all random variables are defined. The class of Borel
probability measures on $\mathbb{R}^{d}$ is $\mathcal{P}(\mathbb{R}^{d})$. The
subset of measures with finite first moment is denoted by
$\mathcal{P}_{1}(\mathbb{R}^{d})$, i.e., $P\in\mathcal{P}_{1}(\mathbb{R}^{d})$
whenever $P\|x\|<\infty$. The convolution of
$P,Q\in\mathcal{P}(\mathbb{R}^{d})$ is $P\ast
Q(A):=\int\int\mathds{1}_{A}(x+y)\,\mathrm{d}P(x)\,\mathrm{d}Q(y)$, where
$\mathds{1}_{A}$ is the indicator of $A$. The convolution of measurable
functions $f,g$ on $\mathbb{R}^{d}$ is $f\ast g(x)=\int
f(x-y)\,g(y)\,\mathrm{d}y$. We also recall that
$\mathcal{N}_{\sigma}:=\mathcal{N}(0,\sigma^{2}\mathrm{I}_{d})$, and use
$\varphi_{\sigma}(x)=(2\pi\sigma^{2})^{-d/2}e^{-\|x\|^{2}/(2\sigma^{2})}$,
$x\in\mathbb{R}^{d}$, for the Gaussian density. The probability law of a
random variable $X$ over $(\Omega,\mathcal{A},\mathbb{P})$ is denoted by
$\mathsf{Law}(X)$. For any measure $Q$ on a measurable space $(S,\mathcal{S})$
and a measurable function $f:S\to\mathbb{R}$, we use the notation
$Qf:=\int_{S}f\,\mathrm{d}Q$, whenever the corresponding integral exists. For
a stochastic process $G$ indexed by a set $\mathcal{F}$, we write $G(f)$ to
denote the coordinate projection of $G$ to $f\in\mathcal{F}$. We write
$\mu_{n}\stackrel{{\scriptstyle w}}{{\to}}\mu$ for a sequence of probability
measures $\mu_{n}$ converging weakly to a probability measure $\mu$.

For a non-empty set $\mathcal{T}$, let $\ell^{\infty}(\mathcal{T})$ denote the
space of all bounded functions $f:\mathcal{T}\to\mathbb{R}$, equipped with the
uniform norm $\|f\|_{\mathcal{T}}:=\sup_{t\in\mathcal{T}}\big{|}f(t)\big{|}$.
We write
$\mathsf{Lip}_{1}(\mathbb{R}^{d}):=\\{f:\mathbb{R}^{d}\to\mathbb{R}:|f(x)-f(y)|\leq\|x-y\|\,,\
\forall x,y\in\mathbb{R}^{d}\\}$ for the set of Lipschitz continuous functions
on $\mathbb{R}^{d}$ with Lipschitz constant bounded by $1$, and
$\mathsf{Lip}_{1,0}(\mathbb{R}^{d}):=\\{f\,\in\mathsf{Lip}_{1}(\mathbb{R}^{d}):f(0)=0\\}$
for the subset of $\mathsf{Lip}_{1}$ consisting of functions that are zero at
zero. When the dimension $d$ is clear from the context we use the shorthands
$\mathsf{Lip}_{1}$ and $\mathsf{Lip}_{1,0}$. We write
$N(\epsilon,\mathcal{F},d)$ for the $\epsilon$-covering number of a function
class $\mathcal{F}$ w.r.t. a metric $d$, and
$N_{[\,]}(\epsilon,\mathcal{F},d)$ for the corresponding bracketing number.
When the metric is induced by a norm $\|\cdot\|$, we replace $d$ with it. The
$L^{p}(P)$ norm of a function $f$ on $(S,\mathcal{S})$ is
$\big{(}\int_{S}|f|^{p}dP\big{)}^{1/p}$: if $P$ is omitted, it is the usual
$L^{p}$ norm w.r.t. the Lebesgue measure on the space. We denote by
$\mathds{1}_{A}$ the indicator function of the set $A$. We also denote by
$\mathcal{F}|_{A}$ the restriction of a function class $\mathcal{F}$ to subset
$A$ of its domain.

###  2.2 The 1-Wasserstein Distance

The Kantorovich OT problem [53] between $P,Q\in\mathcal{P}(\mathbb{R}^{d})$
with cost $c(x,y)$ seeks to minimize $\int c(x,y)\,\mathrm{d}\pi(x,y)$ over
$\pi\in\Pi(P,Q)$, where $\Pi(P,Q)$ is the set of couplings (or transport
plans) between $P$ and $Q$. The 1-Wasserstein distance ($\mathsf{W}_{1}$) [87,
Chapter 6] is the minimum value achieved for the Euclidean distance cost
$c(x,y)=\|x-y\|$:

$\mathsf{W}_{1}(P,Q)=\inf_{\pi\in\Pi(P,Q)}\int_{\mathbb{R}^{d}\times\mathbb{R}^{d}}\|x-y\|\
\,\mathrm{d}\pi(x,y).$

The Wasserstein space
$\big{(}\mathcal{P}_{1}(\mathbb{R}^{d}),\mathsf{W}_{1}\big{)}$ is a metric
space; in fact, $\mathsf{W}_{1}$ metrizes weak convergence together with
convergence of first moments [87]. KR duality [54] gives rise to the following
representation of the 1-Wasserstein distance

$\mathsf{W}_{1}(P,Q)=\inf_{f\in\mathsf{Lip}_{1}}\int_{\mathbb{R}^{d}}f\,\,\mathrm{d}(P-Q).$

This form is invariant to shifting the function $f$ by a constant.
Consequently we may assume without loss of generality that $f(0)=0$, and
optimize over the class $\mathsf{Lip}_{1,0}$ instead. We subsequently leverage
this dual form to derive the limit distribution theory for the smooth version
of the distance.

###  2.3 Smooth 1-Wasserstein Distances

The smooth 1-Wasserstein distance levels out local irregularities in the
distributions via convolution with the Gaussian kernel. It is defined as

$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q):=\mathsf{W}_{1}(P\ast\mathcal{N}_{\sigma},Q\ast\mathcal{N}_{\sigma}),$

where $\mathcal{N}_{\sigma}$ is the $d$-dimensional isotropic Gaussian measure
with parameter $\sigma$. [43] showed that
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ inherits the metric and topological
structure of $\mathsf{W}_{1}$. Stability w.r.t. the smoothing parameter was
also established in that work, in the sense that

$\left|\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{1}}(P,Q)-\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{2}}(P,Q)\right|\leq
2\sqrt{d|\sigma_{1}^{2}-\sigma_{2}^{2}|}.$ (1)

This enabled proving regularity properties of
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q)$ in $\sigma$, such as continuity,
monotonicity, and $\Gamma$-convergence. In addition, [45] showed that
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ alleviates the CoD, converging as
$\mathbb{E}[\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)]=O(n^{-1/2})$ in
all dimensions under a sub-Gaussian condition on $P$. In the next section, we
characterize the limit distribution of
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$, prove that this
distribution can be accurately estimated via the bootstrap, and derive
concentration inequalities. Later, we move to a vanishing $\sigma$ analysis,
Berry-Esseen type bounds, and applications.

##  3 Limit Distribution Theory for Smooth Wasserstein Distance

In this section we characterize the limit distribution of
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ in all dimensions. We
also derive consistency of the bootstrap as a means for computing the limit
distribution, and establish concentration inequalities for
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$.

###  3.1 Limit Distribution

To state the limit distribution result, some definitions are due. For
$P\in\mathcal{P}_{2}(\mathbb{R}^{d})$, let $G_{P}^{\mspace{1.0mu}\sigma}$ be a
centered Gaussian process on $\mathsf{Lip}_{1,0}$ with covariance function
$\mathbb{E}\big{[}G_{P}^{\mspace{1.0mu}\sigma}(f)\,G_{P}^{\mspace{1.0mu}\sigma}(g)\big{]}=\mathsf{Cov}_{P}(f\ast\varphi_{\sigma},g\ast\varphi_{\sigma})$,
where $f,g,\in\mathsf{Lip}_{1,0}$. One may verify that
$|f\ast\varphi_{\sigma}(x)|\leq\|x\|+\sigma\sqrt{d}$ (cf. Section 7.1), so
that $P|f\ast\varphi_{\sigma}|^{2}<\infty$, for all $f\in\mathsf{Lip}_{1,0}$
(which ensures that the said covariance function is well-defined). With that,
we are ready to state the theorem.

######  Theorem 1 (Limit distribution).

Let $\mathbb{R}^{d}=\bigcup_{j=1}^{\infty}I_{j}$ be a partition of
$\mathbb{R}^{d}$ into bounded convex sets with nonempty interior such that
$K:=\sup_{j}\operatorname{\mathsf{diam}}(I_{j})<\infty$ and
$M_{j}:=\sup_{I_{j}}\|x\|\geq 1$ for all $j$. If
$P\in\mathcal{P}_{2}(\mathbb{R}^{d})$ satisfies

$\sum_{j=1}^{\infty}M_{j}P(I_{j})^{1/2}<\infty.$ (2)

then there exists a version of $G_{P}^{\mspace{1.0mu}\sigma}$ that is tight in
$\ell^{\infty}(\mathsf{Lip}_{1,0})$, and denoting the tight version by the
same symbol $G_{P}^{\mspace{1.0mu}\sigma}$, we have
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\stackrel{{\scriptstyle
w}}{{\to}}\|G_{P}^{\mspace{1.0mu}\sigma}\|_{\mathsf{Lip}_{1,0}}=:L_{P}^{\mspace{1.0mu}\sigma}$.
In addition, for any $\alpha>d/2$, we have

$\mathbb{E}\big{[}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\big{]}\lesssim_{\alpha,d,K}\frac{\sigma^{-\alpha+1}}{\sqrt{n}}\sum_{j=1}^{\infty}M_{j}P(I_{j})^{1/2}.$
(3)

The proof, given below in Section 7.1, uses KR duality to translate the
Gaussian convolution in the measure space to the convolution of Lipschitz
functions with a Gaussian density. It is then shown that the class of
Gaussian-smoothed Lipschitz functions is $P$-Donsker by bounding the metric
entropy of the function class restricted to each $I_{j}$. As the next
corollary shows, the dependence on $\sigma$ in (3) can be tightened to
$\sigma^{-d/2+1}$ up to a $\log n$ term.

######  Corollary 1 (Expectation bound).

Under Condition (2) , we have

$\mathbb{E}\left[\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]\lesssim_{d,K}\frac{\sigma^{-d/2+1}\log
n}{\sqrt{n}}\sum_{j=1}^{\infty}M_{j}P(I_{j})^{1/2}.$ (4)

The proof of Corollary 1 is given in Appendix A.2. Combining (4) with the
stability result (1), we can recover, up to a log factor, the (correct)
$n^{-1/d}$ bound on the classic empirical $\mathsf{W}_{1}$. Specifically,
taking $\sigma=n^{-1/d}$, we have

$\begin{split}\mathbb{E}\left[\mathsf{W}_{1}(P_{n},P)\right]&=\mathbb{E}\left[\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]+\mathbb{E}\left[\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)-\mathsf{W}_{1}(P_{n},P)\right]\\\
&\lesssim_{d,P}\inf_{\sigma\in(0,1)}\frac{\sigma^{-d/2+1}\log
n}{\sqrt{n}}+\sigma\\\ &\lesssim_{d,P}n^{-1/d}\log n.\end{split}$

A bound similar to (3) and (4) was derived in [45, Proposition 1] with
explicit dependence on $\sigma$ and $d$. However, unlike the above bound,
their dependence on $\sigma$ is not sharp enough to imply the known polynomial
rates for the empirical unsmoothed distance.

######  Remark 1 (Discussion of Condition (2)).

Let $\\{I_{j}\\}$ consist of cubes with side length $1$ and integral lattice
points as vertices. One may then obtain the bound

$\sum_{j=1}^{\infty}M_{j}P(I_{j})^{1/2}\lesssim_{d}\sum_{k=1}^{\infty}k^{d}P\big{(}\|x\|_{\infty}>k\big{)}^{1/2}\lesssim\int_{1}^{\infty}t^{d}P\big{(}\|x\|_{\infty}>t\big{)}^{1/2}\,\mathrm{d}t,$

which is finite (by Markov’s inequality) if there exists $\epsilon>0$ such
that $P|x_{j}|^{2(d+1)+\epsilon}<\infty$ for all $j$. Proposition 1 in [45]
shows that
$\mathbb{E}\big{[}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\big{]}=O(n^{-1/2})$
whenever $P$ is sub-Gaussian. Theorem 1 thus relaxes this moment condition, in
addition to deriving a limit distribution.

###  3.2 Bootstrap

We next show that the cumulative distribution function (CDF) of the limiting
variable can be estimated via the bootstrap [83, Chapter 3.6]. To that end we
first need to guarantee that the distribution of
$L_{P}^{\mspace{1.0mu}\sigma}$ has a Lebesgue density that is sufficiently
regular.

######  Lemma 1 (Distribution of $L_{P}^{\mspace{1.0mu}\sigma}$).

Assume that Conditions (2) holds and that $P$ is not a point mass. Then the
distribution of $L_{P}^{\mspace{1.0mu}\sigma}$ is absolutely continuous w.r.t.
the Lebesgue measure and its density is positive and continuous on
$(0,\infty)$ except for at most countably many points.

To define the bootstrap estimate, let $X_{1}^{B},\dots,X_{n}^{B}$ be i.i.d.
from $P_{n}$ conditioned on $X_{1},\ldots,X_{n}$, and set
$P_{n}^{B}:=n^{-1}\sum_{i=1}^{n}\delta_{X_{i}^{B}}$ as the bootstrap empirical
distribution. Let $\mathbb{P}^{B}$ be the probability measure induced by the
bootstrap (i.e., the conditional probability given $X_{1},X_{2},\dots$). We
have the following bootstrap consistency.

######  Corollary 2 (Bootstrap consistency).

If Condition (2) holds and $P$ is not a point mass, then

$\sup_{t\geq
0}\left|\mathbb{P}^{B}\big{(}\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n}^{B},P_{n})\leq
t\big{)}-\mathbb{P}\big{(}L_{P}^{\mspace{1.0mu}\sigma}\leq t\big{)}\right|\to
0\quad\mbox{a.s.}$

This corollary, together with continuity of the distribution function of
$L_{P}^{\mspace{1.0mu}\sigma}$ implies that for
$\widehat{q}_{1-\alpha}:=\inf\\{t\geq
0:\mathbb{P}^{B}\big{(}\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n}^{B},P_{n})\leq
t\big{)}\geq 1-\alpha\\}$ (which can be computed numerically), we have
$\mathbb{P}\big{(}\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)>\widehat{q}_{1-\alpha}\big{)}=\alpha+o(1)$.

###  3.3 Concentration Inequalities

Next, we consider quantitative concentration inequalities for
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$, under different moment
conditions on $P$. For $\alpha>0$, let
$\|\xi\|_{\psi_{\alpha}}:=\inf\\{C>0:\mathbb{E}[e^{(|\xi|/C)^{\alpha}}]\leq
2\\}$ be the Orlicz $\psi_{\alpha}$-norm for a real-valued random variable
$\xi$ (if $\alpha\in(0,1)$, then $\|\cdot\|_{\psi_{\alpha}}$ is a quasi-norm).

######  Proposition 1 (Concentration inequalities).

Let $X\sim P$ with $\mathsf{supp}(P)=\mathcal{X}$ and suppose one of the
following conditions holds: (i) $\mathcal{X}$ is compact; (ii)
$\big{\|}\|X\|\big{\|}_{\psi_{\alpha}}<\infty$ for some $\alpha\in(0,1]$;
(iii) $P\|x\|^{q}<\infty$ for some $q\in[1,\infty)$. Then, for all
$\eta\in(0,1)$, $\delta>0$, there exist constants $C:=C_{\eta,\delta}$, such
that for all $t>0$:

$\mathbb{P}\Big{[}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)>(1+\eta)\mathbb{E}\big{[}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\big{]}+t\Big{]}\leq\exp\left(-\frac{nt^{2}}{C\big{(}P\|x\|^{2}+\sigma^{2}d\big{)}}\right)+a(n,t),$
(5)

where

$a(n,t)=\begin{cases}\exp\Big{(}-\frac{nt}{C(\operatorname{\mathsf{diam}}(\mathcal{X})+\sigma\sqrt{d})}\Big{)},&\text{
under (i),}\\\
3\exp\left(-\left(\frac{nt}{C\left((\log(1+n))^{1/\alpha}\big{\|}\|X\|\big{\|}_{\psi_{\alpha}}+\sigma\sqrt{d}\right)}\right)^{\alpha}\right),&\text{
under (ii),}\\\
\frac{C\big{(}n^{1/q}\mathbb{E}[\|X\|^{q}]+\sigma^{q}d^{q/2}\big{)}}{n^{q}t^{q}},&\text{
under (iii).}\end{cases}$

The proof of the above result can be found in Appendix A.4.

######  Remark 2 (Comparison to [43]).

Corollary 1 of [43] establishes the following Gaussian concentration of
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ when $P$ has bounded support:

$\mathbb{P}\left(\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\geq\mathbb{E}\big{[}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\big{]}+t\right)\leq
e^{-\frac{2nt^{2}}{\operatorname{\mathsf{diam}}(\mathcal{X})^{2}}},\quad\forall
t>0.$ (6)

Herein, we generalize this result, accounting also for unbounded support. The
term $a(n,t)$ characterizes the different concentration behaviours of
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ for large $t$.

###  3.4 Two-Sample Setting

Theorem 1 and Corollary 2 can be extended to the two-sample case, i.e.,
accounting for $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{m})$, where
$P_{n}$ and $Q_{m}$ correspond to possibly different population measures $P$
and $Q$. We first treat the case where $P=Q$. Let $N:=m+n$ and consider the
statistic

$W_{m,n}:=\sqrt{\frac{mn}{N}}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{m},Q_{n})=\sqrt{\frac{mn}{N}}\|P_{m}-Q_{n}\|_{\mathcal{F}_{\sigma}},$

where
$\mathcal{F}_{\sigma}:=\left\\{f\ast\varphi_{\sigma}:f\in\mathsf{Lip}_{1,0}\right\\}$
(cf. (9)), and define its bootstrap version as follows. Let $H_{N}$ be the
pooled empirical measure constructed from i.i.d. samples
$X_{1},\dots,X_{m}\sim P$ and $Y_{1},\dots,Y_{n}\sim Q$. Letting
$Z_{1},\dots,Z_{N}$ be a bootstrap sample from $H_{N}$, construct bootstrap
measures $P_{m}^{B}:=m^{-1}\sum_{i=1}^{m}\delta_{Z_{i}}$,
$Q_{n}^{B}:=n^{-1}\sum_{i=m+1}^{N}\delta_{Z_{i}}$. Now let

$W_{m,n}^{B}=\sqrt{\frac{mn}{N}}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{m}^{B},Q_{n}^{B}).$

The following corollary shows that when $P=Q$, $W_{m,n}$ converges in
distribution to $L_{P}^{\mspace{1.0mu}\sigma}$, and the bootstrap distribution
of $W_{m,n}^{B}$ is also consistent for the same limit (see Appendix A.5 for
the proof).

######  Corollary 3 (Two-sample limit distribution and bootstrap).

Let $P,Q\in\mathcal{P}(\mathbb{R}^{d})$. Assume that $P$ is non-degenerate,
$\sum_{j=1}^{\infty}M_{j}P(I_{j})^{1/2}<\infty$, and
$\sum_{j=1}^{\infty}M_{j}Q(I_{j})^{1/2}<\infty$. For $m,n\in\mathbb{N}$, set
$N=m+n$, and assume $m/N\to\lambda\in(0,1)$ as $m,n\to\infty$. The following
hold:

  1. (i)

If $P=Q$, then $W_{m,n}\stackrel{{\scriptstyle
w}}{{\to}}L_{P}^{\mspace{1.0mu}\sigma}$, and $W_{m,n}\overset{P}{\to}\infty$,
otherwise.

  2. (ii)

For $H:=\lambda P+(1-\lambda)Q$, we have
$\mathsf{Law}\big{(}W_{m,n}^{B}|X_{1},\dots,X_{m},Y_{1},\dots,Y_{n}\big{)}\stackrel{{\scriptstyle
w}}{{\to}}L_{H}^{\mspace{1.0mu}\sigma},$ a.s. When $P=Q=H$ we further have

$\sup_{t\in\mathbb{R}}\,\Big{|}\mathbb{P}^{B}\big{(}W_{m,n}^{B}\leq
t\big{)}-\mathbb{P}\big{(}L_{P}^{\mspace{1.0mu}\sigma}\leq t\big{)}\Big{|}\to
0\;\text{a.s.}$

To treat the case when $P\neq Q$, consider the centered statistic

$\overline{W}_{m,n}:=\sqrt{\frac{mn}{m+n}}\left(\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{m})-\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q)\right).$

Theorem 6.1 of [20] uses directional differentiability of the supremum norm
and the functional delta method to derive limit distributions for integral
probability metrics (IPMs) [92, 65] over Donsker classes. The
$\overline{W}_{m,n}$ statistic satisfies this condition, since, up to
centering, it is an IPM over $\mathcal{F}_{\sigma}$, which is $P$\- and
$Q$-Donsker under appropriate moment conditions. This leads to the following
result.

######  Corollary 4.

Assume the conditions of Corollary 3. Let

$d_{\sigma}(f,g):=\sqrt{P(f\ast\varphi_{\sigma}-g\ast\varphi_{\sigma})^{2}}+\sqrt{Q(f\ast\varphi_{\sigma}-g\ast\varphi_{\sigma})^{2}}.$

Then $(\mathsf{Lip}_{1,0},d_{\sigma})$ is a totally bounded pseudometric
space. Further, we have

$\overline{W}_{m,n}\stackrel{{\scriptstyle
w}}{{\to}}\|G_{P,Q,\lambda}^{\mspace{1.0mu}\sigma}\|_{\bar{M}^{+}(D,d_{\sigma})}=:L^{\mspace{1.0mu}\sigma}_{P,Q,\lambda},$
(7)

where
$G_{P,Q,\lambda}^{\mspace{1.0mu}\sigma}=\sqrt{1-\lambda}\widetilde{G}_{P}^{\mspace{1.0mu}\sigma}-\sqrt{\lambda}\widetilde{G}_{Q}^{\mspace{1.0mu}\sigma}$
and
$\bar{M}^{+}(D,d_{\sigma}):=\\{f^{*}\in\overline{\mathsf{Lip}}_{1,0}:(P-Q)\,f^{*}\ast\varphi_{\sigma}=\sup_{\mathsf{Lip}_{1,0}}(P-Q)\,f\ast\varphi_{\sigma}\\}$,
with $\overline{\mathsf{Lip}}_{1,0}$ as the $d_{\sigma}$-completion of
$\mathsf{Lip}_{1,0}$, and $\widetilde{G}_{P}^{\mspace{1.0mu}\sigma}$,
$\widetilde{G}_{Q}^{\mspace{1.0mu}\sigma}$ as extensions of
$G_{P}^{\mspace{1.0mu}\sigma}$, $G_{Q}^{\mspace{1.0mu}\sigma}$ to that
completion, respectively.

This corollary follows from a direct application of Theorem 6.1 in [20].

######  Remark 3 (Relation between Corollaries 3 and 4).

Under $P=Q$, $\widetilde{G}_{P}^{\mspace{1.0mu}\sigma}$ and
$\widetilde{G}_{Q}^{\mspace{1.0mu}\sigma}$ are independent copies of the same
Gaussian process,
$d_{\sigma}(f,g)=2d_{P}(f,g):=2\sqrt{P(f\ast\varphi_{\sigma}-g\ast\varphi_{\sigma})^{2}}$,
and $\bar{M}^{+}(D,d_{\sigma})=\overline{\mathsf{Lip}}_{1,0}$. In this case,
the resulting distribution above coincides with that of Corollary 3. When
$P\neq Q$, Corollary 3 finds the limiting distribution of $W^{B}_{m,n}$, which
is a symmetrized bootstrap version of $W_{m,n}$. For testing $H_{0}:\,P=Q$
against the alternative $H_{1}:\,P\neq Q$, the quantiles of $W_{m,n}^{B}$ can
be used for setting rejection thresholds since the resulting limit
distribution matches that of $W_{m,n}$ for $P=Q$. In contrast, Corollary 4 may
be used to find asymptotic power in specific examples under local alternatives
$Q_{n}\to P$ for such a test, assuming the limit distribution in (7) was
tractable. See [83, Example 3.10.13] for a similar result concerning goodness-
of-fit tests.

######  Remark 4 (Uniqueness of potentials).

If the set $\bar{M}^{+}(D,d_{\sigma})$ of optimal potentials for $P,Q$ is a
singleton (i.e., the optimal potential is unique), then the limit distribution
in (7) is a univariate Gaussian. However, no such uniqueness result is known
for $\mathsf{W}_{1}$ in general, even for smooth measures, since the
underlying cost function is not strictly convex. [39] outlines the
relationship between optimal potentials and transport maps for strictly convex
costs, and also derives existence and uniqueness of the Monge map in such
cases. For other IPMs, such as maximum mean discrepancy (i.e., where the
underlying function class is a unit ball inside a reproducing kernel Hilbert
space), some uniqueness results are derived in [81].

##  4 Berry-Esseen Type Bounds for Vanishing $\sigma$

The classical Berry-Esseen theorem evaluates the distance in supremum norm
between the CDF of the (scaled and centered) sample mean and its Gaussian
limit distribution. In other words, when $F_{n}$ is an empirical CDF from a
centered distribution with unit variance and finite absolute third moment
$M_{3}$, the classical Berry-Esseen Theorem states that

$\sup_{x\in\mathbb{R}}\big{|}F_{n}(x)-\Phi(x)\big{|}\leq\frac{CM_{3}}{\sqrt{n}}.$

The original result [13, 36] has been proven under various settings by many
different authors and the constant $C$ in the above inequality has been
refined over the years (see [78] for a summary, and a recent proof). It was
extended to Banach spaces by [88]. Similar approximation results for empirical
and bootstrap processes for bounded and VC-type functions classes have been
established in [62, 22, 23].

In this section, we derive analogous results for the non-asymptotic
approximation of the limiting distribution of
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ from Theorem 1 by
either the finite sample quantity
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ or the empirical
bootstrap. Proximity between the distributions is measured in the Prokhorov
distance, which is given by

$\rho(P,Q)=\inf\\{\epsilon>0\ :\ P(A)\leq Q(A^{\epsilon})+\epsilon\,,\ \forall
A\in\mathcal{B}(\mathbb{R})\\}.$

With some abuse of notation, we write $\rho(X,Y)$ instead of $\rho(P,Q)$, for
$X\sim P$ and $Y\sim Q$.

The next result provides a Berry-Esseen type bound for the distance to the
limiting variable $L_{P}^{\mspace{1.0mu}\sigma}$ for small $\sigma$ values.
The small $\sigma$ regime was motivated by noise annealing techniques used in
modern machine learning. In addition, $\mathsf{W}_{1}(P,Q)$ is bounded by
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q)+2\sigma\sqrt{d}$. If SWD is viewed
as an approximation of $W_{1}$, it is desirable to have approximations that
converge to the original quantity in the large sample setup.

To simplify notation, we introduce the following quantities: for $d\geq 2$ and
any $\alpha>d/2$, define $a:=\alpha-1,\,b:=d/(2\alpha)$.

######  Theorem 2 (Berry-Esseen type bound).

Suppose $P\|x\|_{\infty}^{l}=M<\infty$ for some $l>4\alpha-d+3$ and let
$n^{-p}\leq\sigma_{n}\leq 1$ for some $p<\frac{1-b}{8ab}$. Then:

  1. i)

there exist versions $V_{n}$ and $W_{n}$ of $L^{\mspace{1.0mu}\sigma_{n}}_{P}$
and $\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{n}}(P_{n},P)$,
respectively, on a probability space, such that
$V_{n}-W_{n}=o_{\mathbb{P}}(r_{n})$, for any sequence $r_{n}\gg
n^{-(1-b)/\left(2(3+b)\right)}\sigma_{n}^{-4ab/(3+b)}$;

  2. ii)

the Prokhorov distance is bounded as
$\rho\big{(}\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{n}}(P_{n},P),\,L_{P}^{\mspace{1.0mu}\sigma_{n}}\big{)}\lesssim_{\alpha,d,M,l}n^{-(1-b)/8}\sigma_{n}^{-ab}$.

The proof of Theorem 2 is inspired by [22], where a methodology for deriving
such results over function classes of the VC type is developed. Since our
function class is not VC type, a direct application of their results is not
possible. Instead, we restrict attention to the Prokhorov distance and
construct our proof from intermediate results.

######  Remark 5 (Fast Berry-Esseen rates).

For $\alpha=2d^{2}$, we have $r_{n}\sim O(n^{-1/6\ +\
\epsilon_{d}})\sigma_{n}^{-4(d^{2}-1)}$, where $\epsilon_{d}$ is small for
large $d$. Hence, if for instance, $\sigma_{n}$ decays at an inverse
logarithmic rate and $d$ is sufficiently large, we have
$r_{n}=o(n^{-1/6+\epsilon})$ for some small $\epsilon>0$. Consequently, when
the limiting distribution has sufficiently high moments and $\sigma_{n}$
decays slowly enough, the CoD is avoided even for Berry-Esseen type rates, in
the sense that although there are $d$-dependent constants involved in the
above bounds, they are decoupled from $n$. This is essentially the rate found
in [22], for smaller (VC-type) function classes.

######  Remark 6 (Other distances).

With further restriction on the measure $P$ (e.g., anti-concentration
assumptions such as those [24]), the bound in Claim (ii) can be derived under
a metric stronger than Prokhorov. However, such an analysis is quite technical
and does not shed new light on the problem, and so we decided to omit it.

In Corollary 2, it was shown that the empirical bootstrap is consistent in
estimating the limitng distribution of SWD. Here, we establish a analogue of
Theorem 2 for the empirical bootstrap process under slightly stronger moment
conditions.

The empirical bootstrap process on $\mathcal{F}_{\sigma}$ is defined as:

$\mathbb{G}_{n}^{*}(f)=\sqrt{n}(P_{n}^{B}-P_{n})f,\ f\in\mathcal{F}_{\sigma}.$
(8)

Recall that $P_{n}^{B}$ is the empirical measure formed from a bootstrap
sample from $P_{n}$, as defined earlier. Then,
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n}^{B},P_{n})=\|\mathbb{G}_{n}^{*}\|_{\mathcal{F}_{\sigma}}$.
We establish the following analogue of Theorem 2 for
$\|\mathbb{G}_{n}^{*}\|_{\mathcal{F}_{\sigma}}$.

######  Theorem 3 (Berry-Esseen type bound for bootstrap).

Under the conditions of Theorem 2,

  1. i)

there exist random variables $W_{n}$ and $V_{n}$ such that

$\mathsf{Law}(W_{n}|X_{1:n})=\mathsf{Law}\big{(}\|\mathbb{G}_{n}^{*}\|_{\mathcal{F}_{\sigma_{n}}}\big{|}X_{1:n}\big{)},\
\
\mathsf{Law}(V_{n}|X_{1:n})=\mathsf{Law}\big{(}L_{P}^{\mspace{1.0mu}\sigma_{n}}\big{)},\
\ |V_{n}-W_{n}|=o_{\mathbb{P}}(r_{n}),$

for any sequence $r_{n}\gg n^{-(1-b)/(2(3+b))}\sigma_{n}^{-a}$;

  2. ii)

the Prokhorov distance is bounded as

$\rho\big{(}\mathsf{Law}\left(\|\mathbb{G}_{n}^{*}\|_{\mathcal{F}_{\sigma_{n}}}\middle|\,X_{1:n}\right),\mathsf{Law}\left(L_{P}^{\sigma_{n}}\right)\big{)}=O_{\mathbb{P}}\big{(}n^{-\frac{1-b}{4(3+b)}}\sigma_{n}^{-a/2}\big{)}.$

The proof of the Theorem 2 can be found in Section 7.2. Theorem 3 can be
proved in a similar fashion with conditional probability versions of the
intermediate lemmas used there. An outline is provided in 7.3, while a
detailed proof is deferred to Appendix B.2.

######  Remark 7 (Measurability and existence of versions).

The existence of versions $W_{n}$ and $V_{n}$ is guaranteed under the
assumption that the underlying probability space is rich enough to contain a
$\mathsf{U}[0,1]$ random variable independent of the sample observations (cf.
[23], Page 3). This can be easily ensured simply by taking an appropriate
product space, but we omit the details.

##  5 Dependence on Intrinsic Dimension

The rate of convergence of the empirical 1-Wasserstein distance
$\mathsf{W}_{1}(P_{n},P)$ adapts to the intrinsic dimension of the population
distribution [33, 16, 69, 89]. These results are derived under the assumption
that $P$ is compactly supported, which is a reasonable compromise given the
complexity of the $\text{Lip}_{1,0}$ class. For the smooth distance, the
derived bounds imply that the function class in concern is much smaller.
Indeed, Theorem 1 shows that it is $P$-Donsker under mild moment assumptions.
As shown next, in this case the dependence on the intrinsic dimension is
captured by the Gaussian smoothing parameter $\sigma$. Our result allows for
unbounded support, albeit under a stronger entropy condition.

###  5.1 Empirical Convergence and Intrinsic Dimension

The notion of intrinsic dimension used in the aforementioned papers and in
this work employs covering numbers. The idea is to leverage the fact that the
$\epsilon$-covering number of a set of intrinsic dimension $s$ will scale as
$c\epsilon^{-s}$, where $c$ may depend on the size of the and the dimension of
the ambient space. Some preliminaries are needed to state our intrinsic
dimension definition. For $P\in\mathcal{P}(\mathbb{R}^{d})$, denote its
$(\epsilon,\tau)$-covering number by

$N_{\epsilon}(P,\tau):=\inf\big{\\{}N(\epsilon,S,\|\cdot\|)\ :\
S\in\mathcal{B}(\mathbb{R}^{d}),\ P(S)\geq 1-\tau\big{\\}},$

and let $d_{\epsilon}(P,\tau)=-\frac{\log N_{\epsilon}(P,\tau)}{\log\epsilon}$
be the corresponding $(\epsilon,\tau)$-dimension.

######  Definition 1 (Upper smooth 1-Wasserstein dimension).

The upper smooth 1-Wasserstein dimension of a probability measure
$P\in\mathcal{P}(\mathbb{R}^{d})$ is defined as:

$\dim^{*}_{\mathsf{SW}_{1}}(P):=\inf\left\\{s\in(2,\infty)\ :\
\limsup_{\epsilon\to
0}d_{\epsilon}\Big{(}P,\epsilon^{s^{2}/(2(s-2))}\Big{)}\leq s\right\\}.$

The above definition is stricter than regular upper 1-Wasserstein dimension
[33, 89] in the sense that for a given $\epsilon$, it considers sets that
exclude a smaller probability mass (i.e., larger sets) for determining
dimensionality. A detailed comparison between the two as well as relations to
Minkowski dimension are provided in the next subsection. The stricter notion
of dimensionality enables treating empirical convergence of distributions with
unbounded support, as stated next.

######  Theorem 4 (Empirical convergence and intrinsic dimension).

Let $P\in\mathcal{P}(\mathbb{R}^{d})$ have $P\|x\|^{s}<\infty$ and
$\dim^{*}_{\mathsf{SW}_{1}}(P)<s$. Then, for all $n>1$,

$\mathbb{E}\left[\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]\lesssim_{d,s,P}\sigma^{-s/2+1}(\log
n)^{3/2}.$

Theorem 4 follows from a stronger version that we establish in Section 7.4,
where the assumption that $\dim^{*}_{\mathsf{SW}_{1}}(P)<s$ is relaxed to a
condition based on entropy.

We conclude this subsection with a sufficient primitive condition on $P$ for
$\dim^{*}_{\mathsf{SW}_{1}}(P)\leq s$ to hold. The next lemma states that this
is the case for sub-exponential distributions supported on an affine
$s$-dimensional subspace of $\mathbb{R}^{d}$ for $d\geq s$.

######  Lemma 2 (Sufficient primitive condition).

Suppose that $P\in\mathcal{P}(\mathbb{R}^{d})$ is supported on an affine space
$H_{s}=\\{x\in\mathbb{R}^{d}\ :x-x_{0}\in V_{s}\\}$, where $V_{s}$ is an
$s$-dimensional linear subspace of $\mathbb{R}^{d}$. Also suppose that
$Z=\|X-x_{0}\|$ for $X\sim P$ has Orlicz $\psi_{\beta}$-norm
$\|Z\|_{\psi_{\beta}}<\infty$, where $\beta>0$. Then,
$\dim^{*}_{\mathsf{SW}_{1}}(P)\leq s$. In addition, we also have
$\mathbb{E}\left[\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]\lesssim_{d,s,P,\beta}\sigma^{-s/2+1}(\log
n)^{(3\beta+1)/2\beta}$.

###  5.2 Implications to Classic 1-Wasserstein Distance

Recall that in Corollary 1 we established the bound
$\mathbb{E}\big{[}\mathsf{W}_{1}(P_{n},P)\big{]}\lesssim_{d}n^{-1/d}\log n$.
We can adopt a similar proof strategy combined with Theorem 4 to get a similar
results in terms of intrinsic dimension.

######  Corollary 5 (Sharp $\mathsf{W}_{1}$ rate under low intrinsic
dimension).

Under the assumptions of Theorem 4, we have
$\mathbb{E}\big{[}\mathsf{W}_{1}(P_{n},P)\big{]}\lesssim_{d,s,P}n^{-1/s}$.

While a direct computation suggests the presence of a log factor, we can get
eliminate it at the cost of a constant dependent on
$\dim^{*}_{\mathsf{SW}_{1}}(P)$ (see Appendix C for a proof). The dependence
of the constants on $P$ is purely through moment conditions and
$\dim_{\mathsf{SW1}}^{*}(P)$. A similar bound was derived in [89, Theorem 1]
under a compact support assumption. Our bound is tight in the following sense:
if $P$ is supported on a regular set of dimension $s$ (see Section 5.3), then
Corollary 2 of [89] implies $\mathbb{E}\big{[}W_{1}(P_{n},P)\big{]}\gtrsim
n^{-1/t}$, for any $t<s=\dim^{*}_{\mathsf{SW}_{1}}(P)$. More importantly,
Corollary 5 establishes this rate even for distributions with unbounded
support.

######  Remark 8 (Berry-Essen bounds under intrinsic dimension).

Similar adaptation of Theorems 2 and 3 to the intrinsic dimension scenario are
possible. However, the resulting rate bound would have the same sample
complexity, with the only change being a faster feasible decay of the
smoothing parameter. The overall rate will remain nearly $n^{-1/6}$, and for
that reason, we omit the details of such refinements.

###  5.3 Relation to Minkowski and Wasserstein Dimension

The study of empirical convergence rates based on box-counting or Minkowski
dimension and variants thereof dates back to [33], with more recent results
available in [16, 89]. These notions of dimensionality are restricted to
compact or bounded sets. We next compare these previously considered intrinsic
dimensions to our Definition 1, under the appropriate boundedness assumptions.

######  Definition 2 (Minkowski dimension).

The Minkowski dimension $\dim_{\mathsf{M}}(S)$ of a bounded
$S\subseteq\mathbb{R}^{d}$ is

$\dim_{\mathsf{M}}(S):=\limsup_{\epsilon\to 0}\frac{\log
N(\epsilon,S,\|\cdot\|_{2})}{-\log\epsilon}.$

For a compactly supported $P\in\mathcal{P}(\mathbb{R}^{d})$, we also write
$\dim_{\mathsf{M}}(P):=\dim_{\mathsf{M}}\big{(}\mathsf{supp}(P)\big{)}.$

######  Definition 3 (Upper Wasserstein dimension [89]).

The upper 1-Wasserstein dimension for a compactly supported
$P\in\mathcal{P}(\mathbb{R}^{d})$ is

$\dim_{\mathsf{W}_{1}}^{*}(P):=\inf\left\\{s\in(2,\infty):\limsup_{\epsilon\to
0}d_{\epsilon}\left(P,\epsilon^{s/(s-2)}\right)\leq s\right\\}.$

A simple extension of [89, Proposition 2] gives the following.

######  Lemma 3 (Dimension comparison).

If $\dim_{\mathsf{M}}(P)\geq 2$, then
$\dim_{\mathsf{W}_{1}}^{*}(P)\leq\dim^{*}_{\mathsf{SW}_{1}}(P)\leq\dim_{\mathsf{M}}(P)$.

Thus, the upper smooth 1-Wasserstein dimension lies between the unsmoothed
variant and the classic Minkowski dimension. However, for probability measures
with sufficiently well behaved supports, [89, Proposition 8] shows that
$\dim_{\mathsf{W}_{1}}^{*}(P)=\dim_{\mathsf{M}}(P)$, as long as
$\dim_{\mathsf{M}}(P)\geq 2$. Such supporting sets are called regular sets, as
defined in [47, Definition 12.1]:

######  Definition 4 (Regular sets [47]).

A compact set $S\subset\mathbb{R}^{d}$ is called regular of dimension $s$ if
there exist $c>0,r_{0}>0$ such that its Hausforff measure $\mathcal{H}^{s}(S)$
satisfies $\frac{1}{c}r^{d}\leq\mathcal{H}^{s}(S\cap B(x,r))\leq cr^{d}$, for
all $x\in S$ and $r\in(0,r_{0})$.

Proposition 9 in [89] provides a list of common regular sets of dimension $s$.
This includes, in particular, compact smooth $s$-dimensional manifolds and
non-empty compact convex sets supported on an affine $s$-dimensional
subspaces. Since $\dim_{\mathsf{W}_{1}}^{*}(P)=\dim_{\mathsf{M}}(P)$, for $P$
that is supported on a regular set as above, we actually have
$\dim_{\mathsf{W}_{1}}^{*}(P)=\dim^{*}_{\mathsf{SW}_{1}}(P)=\dim_{\mathsf{M}}(P)$
in that case.

##  6 Applications

###  6.1 Two-Sample Homogeneity Testing

The nonparametric two-sample homoegeneity testing problem seeks to detect
difference between two distributions on the same measurable (usually
Euclidean) space given only samples from them. For
$P,Q\in\mathcal{P}(\mathbb{R}^{d})$, let $X_{1},\dots,X_{n}\sim P$ and
$Y_{1},\dots,Y_{m}\sim Q$ be independent samples from these distributions,
respectively. Based on these samples we wish to test whether $P$ and $Q$ are
the same (null hypothesis) or not (alternative), i.e.,

$H_{0}:P=Q\ \ \text{versus}\ \ H_{1}:P\neq Q.$

For univariate distributions, two-sample testing is executed using the
Kolmogorov-Smirnov test, which rejects the null hypothesis for large values
$\|F_{n}-G_{m}\|_{\infty}$, where $F_{n}$ and $G_{m}$ are the corresponding
empirical CDFs. Generalizations to multivariate samples take some care to
account for the dependence between coordinates, which typically removes the
distribution-free property of the limiting statistic. Herein, we propose a
multivariate test statistic based on SWD.

Consider a general sequence of 2-sample tests $\phi_{m,n}$ defined as

$\phi_{m,n}(X_{1},\dots,X_{n},Y_{1},\dots,Y_{m}):=\begin{cases}H_{1},&\text{if
}D_{m,n}>c_{m,n},\\\ H_{0},&\text{otherwise},\end{cases}$

where $D_{m,n}:=D_{m,n}(X_{1},\dots,X_{n},Y_{1},\dots,Y_{m})$ is a sequence of
statistics, and $c_{m,n}$ are real numbers. We say that $\phi_{n,m}$ is
asymptotically level $\alpha\in[0,1]$, if
$\limsup_{m,n\to\infty}\mathbb{P}(D_{m,n}>c_{m,n})\leq\alpha$, for all
possible distributions of the statistic $D_{m,n}$ under $H_{0}$. We say that
$\phi_{n,m}$ is consistent at asymptotic level $\alpha$ if it is
asymptotically level $\alpha$ and
$\liminf_{m,n\to\infty}\mathbb{P}(D_{m,n}>c_{m,n})=1$ for all possible
distributions of the statistic $D_{m,n}$ under $H_{1}$.

Corollary 3 allows us to use the bootstrap distribution of $W_{m,n}^{B}$ for
setting critical values. This results in a consistent two sample test.

######  Proposition 2 (Two-sample testing consistency).

Under the setting of Corollary 3, consider the problem of testing $H_{0}:P=Q$
versus $H_{1}:P\neq Q$ using the rule

$T_{m,n}(X_{1},\dots,X_{n},Y_{1},\dots,Y_{m}):=\begin{cases}H_{1},&\text{if
}W_{m,n}>c^{B}_{\alpha},\\\ H_{0},&\text{otherwise},\end{cases}$

where $c^{B}_{\alpha}$ is the $(1-\alpha)$-th quantile of $W_{m,n}^{B}$. Then,
this test is consistent at asymptotic level $\alpha$.

The result follows immediately from Corollary 3 and the discussion that
follows it.

######  Remark 9 (Comparison to classic Wasserstein testing).

Over the last two decades, OT distances have been considered for both one-
sample goodness-of-fit tests [28, 50] and two-sample testing [73]. These works
provide critical values for the tests only for $d=1$ due to the lack of a
multivariate limit distribution theory for $\mathsf{W}_{p}$. Here, we showed
that adopting the SWD as the figure of merit for testing overcomes this
bottleneck.

###  6.2 Minimum Smooth Wasserstein Estimation

The unsupervised learning task of generative modelling concerns learning a
parametrized model $\\{Q_{\theta}\\}_{\theta\in\Theta}$ that best approximates
a data distribution $P$, based on samples from it. Proximity between $P$ and
$Q_{\theta}$ is typically measured in terms of a chosen statistical distance,
which gives rise to the MDE problem [70]. We adopt the SWD as our figure of
merit and use the empirical distribution $P_{n}$ as an estimate for $\mu$. The
resulting minimum smooth Wasserstein estimation (M-SWE) problem is

$\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta}),$

with an optimal estimate
$\widehat{\theta}_{n}\in\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})$.
This section studies structural and statistical aspects of M-SWE.

In what follows we assume (without further mentioning) all distributions,
i.e., $P$ and $Q_{\theta}$, $\theta\in\Theta$, are in
$\mathcal{P}_{1}(\mathbb{R}^{d})$. We further assume that the parameter space
$\Theta\subset\mathbb{R}^{d_{0}}$ is compact with nonempty interior. The
boundedness assumption on $\Theta$ can be weakened with some adjustments to
the proofs of Theorems 5 and 6 below; cf. [12, Assumption 2.3]. We start by
showing that $\widehat{\theta}_{n}$ is measurable.

######  Theorem 5 (M-SWE measurability).

Assume that the map $\theta\mapsto Q_{\theta}$ is continuous relative to the
weak topology, i.e., $Q_{\theta}\stackrel{{\scriptstyle
w}}{{\to}}Q_{\overline{\theta}}$ whenever $\theta\to\overline{\theta}$ in
$\Theta$. Then, for every $n\in\mathbb{N}$, there exists a measurable function
$\omega\mapsto\widehat{\theta}_{n}(\omega)$ such that
$\widehat{\theta}_{n}(\omega)\in\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}\big{(}P_{n}(\omega),Q_{\theta}\big{)}$
for every $\omega\in\Omega$ (this also implies that
$\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}\big{(}P_{n}(\omega),Q_{\theta}\big{)}$
is nonempty).

Next, we establish consistency of the M-SWE. The proof relies on [74, Theorem
7.33]. To apply it, we verify epi-convergence of the map
$\theta\mapsto\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})$ towards
$\theta\mapsto\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})$ (after
proper extensions). See Section D.3 for details.

######  Theorem 6 (M-SWE consistency).

Assume that the map $\theta\mapsto Q_{\theta}$ is continuous relative to the
weak topology. Then the following hold:

  1. (i)

$\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})\to\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})$
a.s.;

  2. (ii)

there exists an event with probability one on which the following holds: for
any sequence $\\{\widehat{\theta}_{n}\\}_{n\in\mathbb{N}}$ of measurable
estimators such that
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\widehat{\theta}_{n}})\leq\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})+o(1)$,
the set of cluster points of $\\{\widehat{\theta}_{n}\\}_{n\in\mathbb{N}}$ is
included in
$\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})$;

  3. (iii)

in particular, if
$\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})$
is unique, i.e.,
$\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})=\\{\theta^{\ast}\\}$,
then $\widehat{\theta}_{n}\to\theta^{\ast}$ a.s.

Finally, we study the limit distributions of the M-SWE and the associated SWD.
Results are presented for the ‘well-specified’ setting, i.e., when
$P=Q_{\theta^{\ast}}$ for some $\theta^{\ast}$ in the interior of $\Theta$.
Extensions to the ‘misspecified’ case are straightforward (cf. [12, Theorem
B.8]). Our derivation uses the method of [70] for MDE analysis over normed
spaces. To make the connection, we need some definitions.

With any $Q\in\mathcal{P}_{1}(\mathbb{R}^{d})$, associate the functional
$Q^{\mspace{1.0mu}\sigma}:\mathsf{Lip}_{1,0}\to\mathbb{R}$ defined by
$Q^{\mspace{1.0mu}\sigma}(f):=Q(f\ast\varphi_{\sigma})=(Q\ast\mathcal{N}_{\sigma})(f)$.
Note that
$\left\|Q^{\mspace{1.0mu}\sigma}\right\|_{\mathsf{Lip}_{1,0}}:=\sup_{f\in\mathsf{Lip}_{1,0}}\left|Q^{\mspace{1.0mu}\sigma}(f)\right|$
is finite as $Q\in\mathcal{P}_{1}(\mathbb{R}^{d})$ and
$|(f\ast\varphi_{\sigma})(x)|\leq\|x\|+\sigma\sqrt{d}$ for any
$f\in\mathsf{Lip}_{1,0}$. Consequently,
$Q^{\mspace{1.0mu}\sigma}\in\ell^{\infty}(\mathsf{Lip}_{1,0})$ for any
$Q\in\mathcal{P}_{1}(\mathbb{R}^{d})$. Finally, observe that
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q)=\big{\|}P^{\mspace{1.0mu}\sigma}-Q^{\mspace{1.0mu}\sigma}\big{\|}_{\mathsf{Lip}_{1,0}}$,
for any $P,Q\in\mathcal{P}_{1}(\mathbb{R}^{d})$ (cf. Section 7.1).

SWD limit distribution. We start from the limit distribution of the (scaled)
infimized SWD. This result is central for deriving the limiting M-SWE
distribution (see Theorem 8 and Corollary 6 below). Theorem 7 is proven in
Appendix D.4 via an adaptation of the argument from [70, Theorem 4.2].

######  Theorem 7 (M-SWE error limit distribution).

Let $P$ satisfy the conditions of Theorem 1. In addition, suppose that:

  1. (i)

the map $\theta\mapsto Q_{\theta}$ is continuous relative to the weak
topology;

  2. (ii)

$P\neq Q_{\theta}$ for any $\theta\neq\theta^{\ast}$.

  3. (iii)

there exists a vector-valued functional
$D^{\mspace{1.0mu}\sigma}\mspace{-2.0mu}\in\mspace{-2.0mu}(\ell^{\infty}(\mathsf{Lip}_{1,0}))^{d_{0}}$
such that
$\big{\|}Q^{\mspace{1.0mu}\sigma}_{\theta}\mspace{-2.0mu}-Q^{\mspace{1.0mu}\sigma}_{\theta^{\ast}}\mspace{-2.0mu}-\mspace{-2.0mu}\langle\theta-\theta^{\ast}\mspace{-3.0mu},\mspace{-2.0mu}D^{\mspace{1.0mu}\sigma}\rangle\big{\|}_{\mathsf{Lip}_{1,0}}\mspace{-2.0mu}$
$=o(\|\theta-\theta^{\ast}\|)$ as $\theta\to\theta^{\ast}$, where $\langle
t,D^{\mspace{1.0mu}\sigma}\rangle:=\sum_{i=1}^{d_{0}}t_{i}D_{i}^{\mspace{1.0mu}\sigma}$
for $t\in\mathbb{R}^{d_{0}}$;

  4. (iv)

the derivative $D^{\mspace{1.0mu}\sigma}$ is nonsingular in the sense that
$\langle t,D^{\mspace{1.0mu}\sigma}\rangle\neq 0$, i.e., $\langle
t,D^{\mspace{1.0mu}\sigma}\rangle\in\ell^{\infty}(\mathsf{Lip}_{1,0})$ is not
the zero functional for all $0\neq t\in\mathbb{R}^{d_{0}}$.

Then,
$\sqrt{n}\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})\stackrel{{\scriptstyle
w}}{{\to}}\inf_{t\in\mathbb{R}^{d_{0}}}\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}},$
where $G_{P}^{\mspace{1.0mu}\sigma}$ is the Gaussian process from Theorem 1.

######  Remark 10 (Norm differentiability).

Condition (iii) in Theorem 7 is called ‘norm differentiability’ in [70]. In
these terms, the theorem assumes that the map $\theta\mapsto
Q^{\mspace{1.0mu}\sigma}_{\theta},\Theta\to\ell^{\infty}(\mathsf{Lip}_{1,0})$,
is norm differentiable around $\theta^{\ast}$ with derivative
$D^{\mspace{1.0mu}\sigma}$. This allows approximating the map $\theta\mapsto
Q^{\mspace{1.0mu}\sigma}_{\theta}$ by the affine function
$Q^{\mspace{1.0mu}\sigma}_{\theta^{\ast}}+\big{<}\theta-\theta^{\ast},D^{\mspace{1.0mu}\sigma}\big{>}$
near $\theta^{\ast}$. Together with the result of Theorem 1 and the right
reparameterization, norm differentiability is key for establishing the
theorem.

######  Remark 11 (Primitive conditions for norm differentiability).

Suppose that $\\{Q_{\theta}\\}_{\theta\in\Theta}$ is dominated by a common
Borel measure $\nu$ on $\mathbb{R}^{d}$, and let $q_{\theta}$ denote the
density of $Q_{\theta}$ with respect to $\nu$, i.e.,
$\,\mathrm{d}Q_{\theta}=q_{\theta}\,\mathrm{d}\nu$. Then,
$Q_{\theta}\ast\mathcal{N}_{\sigma}$ has Lebesgue density
$x\mapsto\int\varphi_{\sigma}(x-t)q_{\theta}(x)\,\mathrm{d}\nu(t)$. Assume
that $q_{\theta}$ admits the Taylor expansion
$q_{\theta}(x)=q_{\theta^{\ast}}(x)+\dot{q}_{\theta^{\ast}}(x)\cdot(\theta-\theta^{\ast})+r_{\theta}(x)\cdot(\theta-\theta^{\ast})$
with $r_{\theta}(x)=o(1)$ as $\theta\to\theta^{\ast}$. Then, one may verify
that Condition (iii) holds with $D^{\mspace{1.0mu}\sigma}(f)=\int
f(x)\int\varphi_{\sigma}(x-t)\dot{q}_{\theta^{\ast}}(t)\,\mathrm{d}\nu(t)\,\mathrm{d}x=\int(f\ast\varphi_{\sigma})(t)\dot{q}_{\theta^{\ast}}(t)\,\mathrm{d}\nu(t)$,
for $f\in\mathsf{Lip}_{1,0}$, provided that
$\int(1+\|t\|)\|\dot{q}_{\theta^{\ast}}(t)\|\,\mathrm{d}\nu(t)<\infty$ and
$\int(1+\|t\|)\|r_{\theta}(t)\|\,\mathrm{d}\nu(t)=o(1)$ (use the fact that
$|f(t)|\leq\|t\|$, for any $f\in\mathsf{Lip}_{1,0}$).

M-SWE limit distribution. We study convergence in distribution of the M-SWE.
Optimally, the limit distribution of
$\sqrt{n}(\widehat{\theta}_{n}-\theta^{\ast})$, for some
$\widehat{\theta}_{n}\in\mathop{\mathrm{argmin}}_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})$,
is the object of interest. However, a limit is guaranteed to exist only when
the (convex) function
$t\mapsto\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}$
has a unique minimum a.s. (see Corollary 6 below). To avoid this stringent
assumption, before treating $\sqrt{n}(\widehat{\theta}_{n}-\theta^{\ast})$, we
first consider the set of approximate minimizers
$\widehat{\Theta}_{n}:=\big{\\{}\theta\in\Theta:\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})\leq\inf_{\theta^{\prime}\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta^{\prime}})+\lambda_{n}/\sqrt{n}\big{\\}}$,
where $\\{\lambda_{n}\\}_{n\in\mathbb{N}}$ is an arbitrary $o_{\mathbb{P}}(1)$
sequence.

We show that $\widehat{\Theta}_{n}\subset\theta^{\ast}+n^{-1/2}K_{n}$ for some
(random) sequence of compact convex sets $\\{K_{n}\\}_{n\in\mathbb{N}}$ with
inner probability approaching one. Resorting to inner probability seems
inevitable since the event
$\\{\widehat{\Theta}_{n}\subset\theta^{\ast}+n^{-1/2}K_{n}\\}$ need not be
measurable in general (see [70, Section 7]). To define such sequence
$\\{K_{n}\\}_{n\in\mathbb{N}}$, for any
$L\in\ell^{\infty}(\mathsf{Lip}_{1,0})$ and $\beta\geq 0$, let

$K(L,\beta):=\left\\{t\in\mathbb{R}^{d_{0}}:\big{\|}L-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}\leq\inf_{t^{\prime}\in\mathbb{R}^{d}}\big{\|}L-\big{<}t^{\prime},D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}\mspace{-8.0mu}+\mspace{-2.0mu}\beta\right\\}.$

Lemma 7.1 of [70] ensures that for any $\beta\geq 0$, $L\mapsto K(L,\beta)$ is
a measurable map from $\ell^{\infty}(\mathsf{Lip}_{1,0})$ into $\mathfrak{K}$
– the class of all compact, convex, and nonempty subsets of
$\mathbb{R}^{d_{0}}$ – endowed with the Hausdorff topology, i.e., the topology
induced by the metric $d_{\mathsf{H}}(K_{1},K_{2}):=\inf\big{\\{}\delta>0:$
$K_{1}^{\delta}\supset K_{2},\ K_{2}^{\delta}\supset K_{1}\big{\\}}$, where
$K^{\delta}$ is the $\delta$-fattening of $K$.

######  Theorem 8 (M-SWE limit distribution).

Under the conditions of Theorem 7, there exists a sequence of nonnegative
reals $\beta_{n}\downarrow 0$ such that the following hold:

  1. (i)

$\mathbb{P}_{*}\big{(}\widehat{\Theta}_{n}\subset\theta^{\ast}+n^{-1/2}K(\mathbb{G}_{n}^{\mspace{1.0mu}\sigma},\beta_{n}\big{)}\big{)}\to
1$, where
$\mathbb{G}_{n}^{\mspace{1.0mu}\sigma}:=\sqrt{n}(P_{n}^{\mspace{1.0mu}\sigma}-P^{\mspace{1.0mu}\sigma})$
is the (smooth) empirical process and $\mathbb{P}_{*}$ denotes inner
probability.

  2. (ii)

$K(\mathbb{G}_{n}^{\mspace{1.0mu}\sigma},\beta_{n})\stackrel{{\scriptstyle
w}}{{\to}}K(G_{P}^{\mspace{1.0mu}\sigma},0)$ as $\mathfrak{K}$-valued random
variables.

Given Theorem 7, the proof of Theorem 8 follows by repeating of the argument
from [70, Section 7.2]; the details are therefore omitted. If
$\mathop{\mathrm{argmin}}_{t\in\mathbb{R}^{d_{0}}}\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}$
is unique a.s. (a nontrivial assumption), then Theorem 8 simplifies as
follows.444Note that
$\mathop{\mathrm{argmin}}_{t\in\mathbb{R}^{d_{0}}}\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}\neq\emptyset$
provided that $D^{\mspace{1.0mu}\sigma}$ is nonsingular, since the latter
guarantees that
$\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}\to\infty$
as $\|t\|\to\infty$.

######  Corollary 6 (Simplified M-SWE limit distribution).

Assume the conditions of Theorem 7. Let
$\\{\widehat{\theta}_{n}\\}_{n\in\mathbb{N}}$ be a sequence measurable
estimators such that
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\widehat{\theta}_{n}})\mspace{-3.0mu}\leq\mspace{-3.0mu}\inf_{\theta\in\Theta}\mspace{-2.0mu}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})+o_{\mathbb{P}}(n^{-\nicefrac{{1}}{{2}}})$.
Then, provided that
$\mathop{\mathrm{argmin}}_{t\in\mathbb{R}^{d_{0}}}\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}$
is unique a.s., we have
$\sqrt{n}(\widehat{\theta}_{n}-\theta^{\ast})\stackrel{{\scriptstyle
w}}{{\to}}\mathop{\mathrm{argmin}}_{t\in\mathbb{R}^{d_{0}}}\big{\|}G_{P}^{\mspace{1.0mu}\sigma}-\big{<}t,D^{\mspace{1.0mu}\sigma}\big{>}\big{\|}_{\mathsf{Lip}_{1,0}}$.

Corollary 6 is proven in Section D.5 using weak convergence of argmin of
random convex maps.

The last result of this section is a high probability generalization bound for
generative modeling via M-SWE, in accordance to the framework from [8, 91].
The goal is to control the gap between the
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$ loss attained by approximate, possibly
suboptimal, empirical minimizers and the population loss
$\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})$.
Upper bounding this gap by the rate of empirical convergence, the
concentration result from Proposition 1 together with the triangle inequality
imply the following.

######  Corollary 7 (M-SWE generalization error bound).

Let $P$ be compactly supported and assume $\widehat{\theta}_{n}$ is an
estimator such that
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\widehat{\theta}_{n}})\leq\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},Q_{\theta})+\epsilon$,
for some $\epsilon>0$. We have

$\mathbb{P}\left(\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\widehat{\theta}_{n}})-\inf_{\theta\in\Theta}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P,Q_{\theta})>\epsilon+t\right)\leq
Ce^{-cnt^{2}},$

for constants $C,c$ independent of $n$ and $t$.

This result is proved in Appendix D.6. As a consequence, if
$\\{\widehat{\theta}_{n}\\}_{n\in\mathbb{N}}$ attains an error of order
$n^{-1/2}$, as in Corollary 6, we obtain a high-probability $O(n^{-1/2})$
generalization bound for M-SWE.

##  7 Proofs of Main Results

This section presents proofs of our main results from Theorems 1-4.
Derivations of auxiliary lemmas used in these proofs are deferred to the
appropriate Appendix sections. Due to the technical nature and length of the
proof of Theorem 3, herein we only provide an outline of the argument and
defer the full details to the Appendix.

###  7.1 Proof of Theorem 1

Recall that $\varphi_{\sigma}$ is the density function of
$\mathcal{N}(0,\sigma^{2}\mathrm{I}_{d})$, i.e.,
$\varphi_{\sigma}(x)=(2\pi\sigma^{2})^{-d/2}e^{-\|x\|^{2}/(2\sigma^{2})}$ for
$x\in\mathbb{R}^{d}$. Since $P_{n}\ast\mathcal{N}_{\sigma}$ has a density
given by $x\mapsto n^{-1}\sum_{i=1}^{n}\varphi_{\sigma}(x-X_{i}),$ we arrive
at the expression

$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)=\sup_{f\in\mathsf{Lip}_{1}}\left[\frac{1}{n}\sum_{i=1}^{n}f\ast\varphi_{\sigma}(X_{i})-Pf\ast\varphi_{\sigma}\right].$
(9)

Observing that the RHS of (9) does not change if we replace $f$ with
$f-f(x^{\ast})$ for any fixed $x^{\ast}$, the problem boils down to showing
that the function class
$\mathcal{F}_{\sigma}:=\\{f\ast\varphi_{\sigma}:f\in\mathsf{Lip}_{1,0}\\}$ is
$P$-Donsker.

To that end we next derive a bound on the Hölder norm of
$f_{\sigma}:=f\ast\varphi_{\sigma}$, for $f\in\mathsf{Lip}_{1,0}$. The Donsker
property will follow by metric entropy bounds for Hölder balls. For a vector
$k=(k_{1},\dots,k_{d})$ of $d$ nonnegative integers, define the differential
operator

$D^{k}=\frac{\partial^{|k|}}{\partial x_{1}^{k_{1}}\cdots\partial
x_{d}^{k_{d}}},$

with $|k|=\sum_{i=1}^{d}k_{i}$. For a function $f:\mathcal{X}\to\mathbb{R}$
defined on a bounded set $\mathcal{X}\subset\mathbb{R}^{d}$, the Hölder norm
of order $\alpha>0$ is

$\|f\|_{\alpha}:=\|f\|_{\alpha,\mathcal{X}}:=\max_{0\leq|k|\leq\underline{\alpha}}\sup_{x\in\mathcal{X}}\big{|}D^{k}f(x)\big{|}+\sup_{\overset{|k|=\underline{\alpha}}{x,y\in\mathcal{X}}}\frac{\big{|}D^{k}f_{\sigma}(x)-D^{k}f_{\sigma}(y)\big{|}}{\|x-y\|^{\alpha-\underline{\alpha}}},$

where $\underline{\alpha}$ is the greatest integer strictly smaller than
$\alpha$ and the suprema are taken over the interior of $\mathcal{X}$. We have
the following lemma (see Appendix A.1 for the proof).

######  Lemma 4 (Hölder norm bound).

For any $\alpha\geq 1$, $0<\sigma\leq 1$, $f\in\mathsf{Lip}_{1,0}$, and a
bounded convex $\mathcal{X}\subset\mathbb{R}^{d}$, we have
$\|f_{\sigma}\|_{\alpha,\mathcal{X}}\lesssim_{\alpha,d}\sigma^{-\alpha+1}\big{(}1\vee\sup_{\mathcal{X}}\|x\|\big{)}$.

The following metric entropy bound from [83, Theorem 2.7.1] will be
subsequently used.

######  Lemma 5 (Metric entropy bound for Hölder ball).

Let $\mathcal{X}\subset\mathbb{R}^{d}$ be bounded and convex with nonempty
interior. For $\alpha\geq 1$ and $M>0$, let $C^{\alpha}_{M}(\mathcal{X})$ be
the set of continuous functions $f:\mathcal{X}\to\mathbb{R}$ with
$\|f\|_{\alpha}\leq M$. Then, for any $0<\epsilon\leq 1$, we have $\ \log
N\big{(}\epsilon
M,C_{M}^{\alpha}(\mathcal{X}),\|\cdot\|_{\infty}\big{)}\lesssim_{d,\alpha,\operatorname{\mathsf{diam}}(\mathcal{X})}\epsilon^{-d/\alpha}$.

We are now in position to prove Theorem 1, which applies Theorem 1.1 in [83]
to the function class $\mathcal{F}_{\sigma}$ to show that it is $P$-Donsker.
We begin with noting that the function class $\mathcal{F}_{\sigma}$ has
envelope $F(x):=\|x\|+\sigma\sqrt{d}$, which is $P$-integrable as required
under the assumed moment conditions. Indeed, for any $f\in\mathsf{Lip}_{1,0}$,
since $|f(y)|\leq|f(0)|+\|y\|=\|y\|$, we have

$\begin{split}|f_{\sigma}(x)|&\leq\int\|y\|\varphi_{\sigma}(x-y)\,\mathrm{d}y\leq\int\big{(}\|x\|+\|x-y\|\big{)}\varphi_{\sigma}(x-y)\,\mathrm{d}y\\\
&\leq\|x\|+\int\|y\|\varphi_{\sigma}(y)\,\mathrm{d}y\leq\|x\|+\left(\int_{\mathbb{R}^{d}}\|y\|^{2}\varphi_{\sigma}(y)\,\mathrm{d}y\right)^{1/2}=\|x\|+\sigma\sqrt{d}.\end{split}$

Next, for each $j$, consider the restriction of $\mathcal{F}_{\sigma}$ to
$I_{j}$, denoted as
${\mathcal{F}}_{j}=\\{f\mathds{1}_{I_{j}}:f\in{\mathcal{F}}\\}$. To invoke
[83, Theorem 1.1], we have to verify that each $\mathcal{F}_{j}$ is
$P$-Donsker and to bound $\mathbb{E}[\|\mathbb{G}_{n}\|_{{\mathcal{F}}_{j}}]$,
where $\mathbb{G}_{n}:=\sqrt{n}(P_{n}-P)$. In view of Lemma 4,
${\mathcal{F}}_{j}$ can be regarded as a subset of
$C_{M^{\prime}_{j}}^{\alpha}(I_{j})$ with $\alpha>d/2$ and
$M_{j}^{\prime}=\sigma^{-\alpha+1}C_{\alpha,d}\sup_{I_{j}}\|x\|$. Lemma 5 then
implies that the $L^{2}(Q)$-metric entropy of ${\mathcal{F}}_{j}$, for any
probability measure $Q\in\mathcal{P}(\mathbb{R}^{d})$, can be bounded as

$\log N\left(\epsilon
M_{j}^{\prime}Q(I_{j})^{1/2},{\mathcal{F}}_{j},L^{2}(Q)\right)\lesssim_{\alpha,d,K}\epsilon^{-d/\alpha},$

where $K=\sup_{j}\operatorname{\mathsf{diam}}(I_{j})<\infty$. For any choice
of $\alpha>d/2$, the square root of the RHS is integrable (w.r.t. $\epsilon$)
around $0$, so that $\mathcal{F}_{j}$ is $P$-Donsker by Theorem 2.5.2 in [83].
By Theorem 2.14.1 in [83] we further obtain

$\mathbb{E}[\|\mathbb{G}_{n}\|_{{\mathcal{F}}_{j}}]\lesssim_{\alpha,d,K}M_{j}^{\prime}P(I_{j})^{1/2}\lesssim_{\alpha,d}\sigma^{-\alpha+1}M_{j}P(I_{j})^{1/2},$
(10)

with $M_{j}=1\vee\sup_{I_{j}}\|x\|$. By assumption, the RHS is summable over
$j$.

By Theorem 1.1 in [86] we conclude that ${\mathcal{F}_{\sigma}}$ is
$P$-Donsker, which implies that there exists a tight version of $P$-Brownian
bridge process $G_{P}$ in $\ell^{\infty}({\mathcal{F}_{\sigma}})$ such that
$\big{(}\mathbb{G}_{n}(f)\big{)}_{f\in\mathcal{F}_{\sigma}}$ converges weakly
in $\ell^{\infty}({\mathcal{F}_{\sigma}})$ to $G_{P}$. Finally, the continuous
mapping theorem yields that

$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)=\|\mathbb{G}_{n}\|_{{\mathcal{F}}_{\sigma}}\stackrel{{\scriptstyle
w}}{{\to}}\|G_{P}\|_{{\mathcal{F}}_{\sigma}}=\big{\|}G_{P}^{\mspace{1.0mu}\sigma}\big{\|}_{\mathsf{Lip}_{1,0}},$

where $G_{P}^{\mspace{1.0mu}\sigma}(f):=G_{P}(f\ast\varphi_{\sigma})$. By
construction, the Gaussian process
$\big{(}G_{P}^{\mspace{1.0mu}\sigma}(f)\big{)}_{f\in\mathsf{Lip}_{1,0}}$ is
tight in $\ell^{\infty}(\mathsf{Lip}_{1,0})$. The moment bound for
$\alpha>d/2$ follows from summing up the moment bounds for
${\mathcal{F}}_{j}$.

###  7.2 Proof of Theorem 2

The statistic $\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ and its
limit $L_{P}^{\mspace{1.0mu}\sigma}$ are, respectively, the suprema of the
empirical process and its Gaussian process limit over the function class
$\mathcal{F}_{\sigma}$. Given a finite $L^{2}(P)$ $\delta$-net of functions
covering $\mathcal{F}_{\sigma}$, the difference in the distribution of these
suprema can be bound by their difference w.r.t. the $\delta$-net plus
approximation error terms. Formally, if $f_{1},\dots,f_{N_{\sigma,\delta}}$ is
a minimal $\delta$-net as above, where
$N_{\sigma,\delta}=N\big{(}\delta,\mathcal{F}_{\sigma},L^{2}(P)\big{)}$, then,
for any set $A\in\mathcal{B}(\mathbb{R})$,

$\displaystyle\mathbb{P}\Big{(}$
$\displaystyle\,\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}}\in
A\Big{)}-\mathbb{P}\Big{(}\|G_{P}\|_{\mathcal{F}_{\sigma}}\in
A^{3\eta}\Big{)}$
$\displaystyle\leq\underbrace{\mathbb{P}\Big{(}\big{|}\|G_{P}\|_{\mathcal{F}_{\sigma}}-\max_{1\leq
i\leq
N_{\sigma,\delta}}G_{P}(f_{i})\big{|}>\eta\Big{)}}_{\text{(I)}}+\underbrace{\mathbb{P}\Big{(}\big{|}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}}-\max_{1\leq
i\leq
N_{\sigma,\delta}}\mathbb{G}_{n}(f_{i})\big{|}>\eta\Big{)}}_{\text{(II)}}$
$\displaystyle\qquad\quad+\underbrace{\mathbb{P}\Big{(}\max_{1\leq i\leq
N_{\sigma,\delta}}\mathbb{G}_{n}(f_{i})\in
A^{\eta}\Big{)}-\mathbb{P}\Big{(}\max_{1\leq i\leq
N_{\sigma,\delta}}G_{P}(f_{i})\in A^{2\eta}\Big{)}}_{\text{(III)}}.$ (11)

We wish to translate this inequality into a bound on the difference
$V_{\sigma,n}-W_{\sigma,n}$, where $\big{(}V_{\sigma,n},W_{\sigma,n}\big{)}$
are a coupling of $\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}}$ and
$\|G_{P}\|_{\mathcal{F}_{\sigma}}$. Strassen’s theorem allows us to ensure the
existence of such couplings. (see [22, Lemma 4.1] for the version stated
below).

######  Lemma 6 (Strassen’s theorem).

Let $X,Y$ be real random variables. If $\mathbb{P}(X\in A)\leq\mathbb{P}(Y\in
A^{\delta})+\epsilon$ for all $A\in\mathcal{B}(\mathbb{R})$, then there exists
a coupling $(\widetilde{X},\widetilde{Y})$ of $X$ and $Y$ such that
$\mathbb{P}\big{(}|\widetilde{X}-\widetilde{Y}|>\delta\big{)}\leq\epsilon$.

In fact, we will derive an $O_{\mathbb{P}}$ rate for the difference
$|V_{n}-W_{n}|$, where $V_{n}:=V_{n,\sigma_{n},\delta_{n},\eta_{n}}$ and
$W_{n}:=W_{n,\sigma_{n},\delta_{n},\eta_{n}}$, for a smoothing parameter
$\sigma=\sigma_{n}$ that is allowed to decay at a slow enough rate. To that
end, first observe that for each fixed $\sigma>0$, the class
$\mathcal{F}_{\sigma}$ is pre-Gaussian, which implies the existence of a tight
version of the limit process $\big{(}G_{P}(f)\big{)},\
f\in\mathcal{F}_{\sigma}$ and ensures measurability of
$\|G_{P}\|_{\mathcal{F}_{\sigma}}$. Since $\mathcal{F}_{\sigma}$ is separable
w.r.t. pointwise convergence, $\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}}$ is
also measurable.

Thus, if we establish a non-asymptotic (in $n$) bound on the RHS of (7.2) and
characterizes the exact dependence on $\sigma$, we may set $\sigma=\sigma_{n}$
and still obtain $(V_{n},W_{n})$ on $(\Omega,\mathcal{A},\mathbb{P})$, whose
difference is bounded in probability. Combining the bounds for each $n$, we
get an $O_{\mathbb{P}}$ rate for $|V_{n}-W_{n}|$. The rate bound on the
Prokhorov distance follows more directly from (7.2) without going through
Lemma 6.

We begin with a few technical lemmas that will be required throughout the
proof; derivations of the lemmas are given in Appendix B.1. The first is a
bound on $L^{2}$ entropy classic of functions with unbounded support. This
provides control on $N_{\sigma,\delta}$ for a given discretization error level
$\delta$.

######  Lemma 7 (Example 19.9 in [84]).

Let $\\{I_{j}\\}_{j=1}^{\infty}$ be an enumeration of unit cubes in
$\mathbb{R}^{d}$ with vertices on integer lattice points. If $\mathcal{F}$ is
a function class such that for any $j\in\mathbb{N}$ the restriction
${\mathcal{F}}_{j}=\\{f\mathds{1}_{I_{j}}:f\in{\mathcal{F}}\\}$ belongs to the
Hölder class $C^{\alpha}_{M^{\prime}_{j}}(I_{j})$, for $\alpha>0$ and
$M_{j}>0$, i.e., $\mathcal{F}_{j}\subset C^{\alpha}_{M^{\prime}_{j}}(I_{j})$,
then

$\log
N_{[\,]}(\delta,\mathcal{F},L^{2}(P))\lesssim_{\alpha,d}\left(\frac{1}{\delta}\right)^{\frac{d}{\alpha}}\left[\sum_{j=1}^{\infty}\big{(}(M^{\prime}_{j})^{2}P(I_{j})\big{)}^{\frac{d}{d+2\alpha}}\right]^{\frac{d+2\alpha}{2\alpha}}.$
(12)

The next lemma relates the probability law of the maximum of (centered) i.i.d.
random vectors to that of the maximum of i.i.d. Gaussians with same covariance
matrices. This will enable bounding term (III) in (7.2).

######  Lemma 8 (Theorem 3.1 in [23] simplified).

Let $X_{1},\dots,X_{n}$ be i.i.d. zero mean $\mathbb{R}^{p}$-valued random
vectors $(p\geq 2)$ with covariance matrix $\Sigma$ and coordinate-wise finite
absolute third moments, i.e., $\mathbb{E}\big{[}|X_{ij}|^{3}\big{]}<\infty$,
for all $1\leq i\leq n$ and $1\leq j\leq p$. Consider the statistic
$Z:=\max_{1\leq j\leq p}\frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_{ij}$. Let
$Y\sim\mathcal{N}(0,\Sigma)$ and define $\widetilde{Z}:=\max_{1\leq j\leq
p}Y_{j}$. Then, for every $\eta>0$ and $A\in\mathcal{B}(\mathbb{R})$, we have

$\mathbb{P}(Z\in A)\leq\mathbb{P}(\widetilde{Z}\in
A^{C_{7}\eta})+\frac{C_{8}(\log
p)^{2}}{\eta^{3}\sqrt{n}}\big{(}L_{n}+M_{n,X}(\eta)+M_{n,Y}(\eta)\big{)},$

where $C_{7},C_{8}$ are universal positive constants, $L_{n}=\max_{1\leq j\leq
p}\mathbb{E}\left[|X_{ij}|^{3}\right]$, and

$\displaystyle M_{n,X}(\eta)=\mathbb{E}\left[\max_{1\leq j\leq
p}|X_{ij}|^{3}\mathds{1}_{\left\\{\max_{1\leq j\leq
p}|{X}_{ij}|>\eta\sqrt{n}/\log p\right\\}}\right],$ $\displaystyle
M_{n,Y}(\eta)=\mathbb{E}\left[\max_{1\leq j\leq
p}|Y_{j}|^{3}\mathds{1}_{\left\\{\max_{1\leq j\leq
p}|{Y}_{j}|>\eta\sqrt{n}/\log p\right\\}}\right].$

We also use a Fuk-Nagev type inequality (Theorem 2, [3]) for empirical process
concentration to bound the term (II) in (7.2).

######  Lemma 9 (Fuk-Nagev type inequality).

Let $\mathcal{F}$ be a function class $f:S\to\mathbb{R}$ with a measurable
envelope $F$, that is separable w.r.t. pointwise convergence, and let
$P\in\mathcal{P}(S)$. Suppose that $Pf=0$ for all $f\in\mathcal{F}$, and that
$F\in L^{p\vee 2}(P)$.

For $X_{1},\ldots,X_{n}$ i.i.d. according to $P$ define
$Z:=\sup_{f\in\mathcal{F}}\big{|}\sum_{i=1}^{n}f(X_{i})\big{|}$ and let
$\sigma^{2}>0$ be any positive constant such that
$\sigma^{2}\geq\sup_{f\in\mathcal{F}}Pf^{2}$. Then

$\mathbb{P}\big{(}Z\geq(1+\alpha)\mathbb{E}[Z]+x\big{)}\leq
e^{-cx^{2}/(n\sigma^{2})}+\frac{c^{\prime}\mathbb{E}[M^{p}]}{x^{p}},\quad\forall\,\alpha,x>0,$

where $M:=\max_{1\leq i\leq n}F(X_{i})$ and $c,c^{\prime}$ are positive
constants that depend only on $p$ and $\alpha$.

Finally, we also use the following relation between moments of random
variables and discrete sums approximating such moments.

######  Lemma 10 (Sufficient moment condition).

Let $\\{I_{j}\\}_{j=1}^{\infty}$ be an enumeration of unit cubes in
$\mathbb{R}^{d}$ with vertices on integer lattice points, disjointified such
that they form a partition of $\mathbb{R}^{d}$. If
$P\|x\|_{\infty}^{l}=M<\infty$ for any
$l>\frac{(d-1)(1-p_{2})+p_{1}+1}{p_{2}}$, then
$\sum_{j=1}^{\infty}\big{(}\sup_{I_{j}}\|x\|\big{)}^{p_{1}}P(I_{j})^{p_{2}}\lesssim_{l,d,p_{1},p_{2}}M^{p_{2}}<\infty$.

We are now ready to prove Theorem 2.

###### Proof of Theorem 2.

Proof of (i): First, let $\sigma,\delta\in(0,1]$ be arbitrary; these
parameters will be specified later. Recall that we fix a smoothness parameter
$\alpha>d/2$ and let $a=\alpha-1,\,b=d/(2\alpha)$. Note that, if
$f_{1},\dots,f_{N_{\sigma,\delta}}$ is a minimal $L^{2}(P)$ $\delta$-net of
$\mathcal{F}_{\sigma}$, then

$\begin{split}\left|\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}}-\max_{1\leq
i\leq
N_{\sigma,\delta}}\mathbb{G}_{n}(f_{i})\right|&\leq\sup_{\begin{subarray}{c}f,g\in\mathcal{F}_{\sigma}:\\\
\|f-g\|_{L^{2}(P)}<\delta\end{subarray}}\big{|}\mathbb{G}_{n}(f-g)\big{|},\\\
\left|\|G_{P}\|_{\mathcal{F}_{\sigma}}-\max_{1\leq i\leq
N_{\sigma,\delta}}G_{P}(f_{i})\right|&\leq\sup_{\begin{subarray}{c}f,g\in\mathcal{F}_{\sigma}:\\\
\|f-g\|_{L^{2}(P)}<\delta\end{subarray}}\big{|}G_{P}(f)-G_{P}(g)\big{|}.\end{split}$
(13)

For the second inequality, we extend $G_{P}$ to the linear span of
$\mathcal{F}_{\sigma}$ so that its sample paths are linear, and the RHS may be
written as a supremum of $|G_{P}(f-g)|$. We split the proof into the following
steps:

Step 1 (Bound on metric entropy). Set
$K=\sum_{j=1}^{\infty}\big{(}M_{j}^{2}P(I_{j})\big{)}^{d/(d+2\alpha)}\big{)}$.
Invoking Lemma 7 with
$M_{j}^{\prime}=\sigma^{-\alpha+1}C_{\alpha,d}\sup_{I_{j}}\|x\|$ (which is a
feasible choice by Lemma 4), we obtain

$\log
N\big{(}\delta,\mathcal{F}_{\sigma},L^{2}(P)\big{)}\lesssim_{\alpha,d}K\sigma^{-d\frac{\alpha-1}{\alpha}}\delta^{-\frac{d}{\alpha}}\lesssim_{\alpha,d,M,l}\sigma^{-2ab}\delta^{-2b},$
(14)

where $K\lesssim_{\alpha,d}M^{d/(d+2\alpha)}$ by Lemma 10.

Step 2 (Empirical process discretization error). Let
$\\{I_{j}\\}_{j=1}^{\infty}$ be a suitably disjointed enumeration of unit
cubes on integer lattice points in $\mathbb{R}^{d}$. Define $M_{j}$ and
$M^{\prime}_{j}$ as in Theorem 1, and let
$\mathcal{F}_{\sigma}(\delta):=\big{\\{}f-g:f,g\in\mathcal{F}_{\sigma},\,\|f-g\|_{L^{2}(P)}<\delta\big{\\}}$.
Note that
$\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}(\delta)}\big{]}\leq\sum_{j=1}^{\infty}\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{{\mathcal{F}_{j}}(\delta)}\big{]}$,
where
${\mathcal{F}_{j}}(\delta)=\\{f\mathds{1}_{I_{j}}:f\in\mathcal{F}_{\sigma}(\delta)\\}$.
Also set
${\mathcal{F}}_{j}=\\{f\mathds{1}_{I_{j}}:f\in{\mathcal{F}_{\sigma}}\\}$.
Clearly, $F_{j}(x)=M^{\prime}_{j}\mathds{1}_{I_{j}}(x)$ is an envelope
function for ${\mathcal{F}}_{j}$. Then, using Lemma 5 with $M=M^{\prime}_{j}$
and $N=\alpha$ (which are feasible choices by Lemma 4), we have $\log
N\big{(}\delta
M^{\prime}_{j}Q(I_{j})^{1/2},{\mathcal{F}}_{j},L^{2}(Q)\big{)}\lesssim_{\alpha,d}\delta^{-2b}$
for any probability measure $Q$ on a finite set. Observing that

$N\left(2\delta
M^{\prime}_{j}Q(I_{j})^{\nicefrac{{1}}{{2}}},{\mathcal{F}}_{j}(\delta),L^{2}(Q)\right)\leq
N\left(\delta
M^{\prime}_{j}Q(I_{j})^{\frac{1}{2}},{\mathcal{F}}_{j},L^{2}(Q)\right)^{2},$
(15)

we obtain $\log N\big{(}2\delta
M^{\prime}_{j}Q(I_{j})^{\frac{1}{2}},\mathcal{F}_{j}(\delta),L^{2}(Q)\big{)}\lesssim_{\alpha,d}\delta^{-2b}$.

For the difference class $\mathcal{F}_{j}(\delta)$, $2F_{j}$ is an envelope
function. Next, we compute the entropy integral
$J(\delta,{\mathcal{F}}_{j}(\delta),2F_{j})$ as

$J(\delta,\mathcal{F}_{j},2F_{j}):=\int_{0}^{\delta}\sup_{Q\in\mathcal{P}_{\mathsf{f}}(\mathbb{R}^{d})}\sqrt{\log
N\left(\epsilon\|2F_{j}\|_{L^{2}(Q)},\mathcal{F}_{j}(\delta),L^{2}(Q)\right)}\,\mathrm{d}\epsilon\lesssim_{\alpha,d}\mspace{-3.0mu}\int_{0}^{\delta}\epsilon^{-b}\,\mathrm{d}\epsilon\lesssim_{\alpha,d}\delta^{1-b},$
(16)

where
$\mathcal{P}_{\mathsf{f}}(\mathbb{R}^{d})\subset\mathcal{P}(\mathbb{R}^{d})$
denotes the set of discrete measures on finitely many points in
$\mathbb{R}^{d}$. The local maximal inequality for uniform entropy [22,
Theorem 5.2] then implies

$\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{j}(\delta)}\big{]}\lesssim_{\alpha,d}\left(M^{\prime}_{j}P(I_{j})^{\frac{1}{2}}\right)^{b}\delta^{1-b}+\frac{1}{\sqrt{n}}(M^{\prime}_{j})^{1+2b}P(I_{j})^{b}\delta^{-2b}.$
(17)

By Lemma 10,
$\sum_{j=1}^{\infty}\big{(}M_{j}P(I_{j})^{1/2}\big{)}^{b}\lesssim_{\alpha,d,l}M^{b/2}<\infty$
and $\sum_{j=1}^{\infty}M_{j}^{1+2b}P(I_{j})^{b}\lesssim_{\alpha,d,l}M^{b}$.
Hence, we sum over $j$ while using the fact that
$M^{\prime}_{j}\lesssim_{\alpha,d}\sigma^{-a}M_{j}$ to obtain

$\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}(\delta)}\big{]}\lesssim_{\alpha,d}\mspace{-3.0mu}\sum_{j=1}^{\infty}\left[\big{(}M^{\prime}_{j}P(I_{j})^{\frac{1}{2}}\big{)}^{b}\delta^{1-b}\mspace{-5.0mu}+\mspace{-3.0mu}\frac{(M^{\prime}_{j})^{1+2b}P(I_{j})^{b}}{\delta^{2b}\sqrt{n}}\right]\mspace{-5.0mu}\lesssim_{\alpha,d,M,l}\sigma^{-ab}\delta^{1-b}+\frac{\sigma^{-a(1+2b)}}{\delta^{2b}\sqrt{n}}.\mspace{-3.0mu}$
(18)

Step 3 (Gaussian process discretization error). Recall that $G_{P}$ is a mean
zero Gaussian process on $\mathcal{F}_{\sigma}$ with covariance kernel
$\mathsf{Cov}\big{(}G_{P}(f),G_{P}(g)\big{)}=\mathsf{Cov}_{P}(f,g)$. By
Dudley’s theorem (see, for e.g., Theorem 11.17, [57]), modulo arguments
relating the entropy of the difference class $\mathcal{F}_{\sigma}(\delta)$
with that of $\mathcal{F}_{\sigma}$ (as in (15)), we have

$\mathbb{E}\left[\|G_{P}\|_{\mathcal{F}_{\sigma}(\delta)}\right]\lesssim\int_{0}^{\delta}\sqrt{\log
N\big{(}\mathcal{F}_{\sigma},L^{2}(P),\epsilon\big{)}}\,\mathrm{d}\epsilon\lesssim_{\alpha,d,M,l}\sigma^{-ab}\delta^{1-b}.$
(19)

Step 4 (Comparison between $\mathbb{G}_{n}$ and $G_{P}$ after discretization).
Let $f_{1},\dots,f_{N_{\sigma,\delta}}$ be a minimal $L^{2}(P)$ $\delta$-net
of $\mathcal{F}_{\sigma}$ and define $Z=\max_{1\leq j\leq
N_{\sigma,\delta}}\mathbb{G}_{n}(f_{j})$ and $\widetilde{Z}=\max_{1\leq j\leq
N_{\sigma,\delta}}G_{P}(f_{j})$. Invoking Lemma 8, the rest of this step
focuses on bounding $L_{n}$, $M_{n,X}(\eta)$, and $M_{n,Y}(\eta)$. Let $X\sim
P$. For the first two, set $X_{ij}=f_{j}(X_{i})-Pf_{j}$ and note that
$\mathbb{E}\big{[}|X_{ij}|^{3}\big{]}\leq\mathbb{E}\big{[}(\|X\|+\sigma\sqrt{d}\,)^{3}\big{]}\leq\mathbb{E}\big{[}(\|X\|+\sqrt{d}\,)^{3}\big{]}$.

This gives us

$L_{n}\leq\max_{1\leq j\leq
N_{\sigma,\delta}}\mathbb{E}\big{[}|X_{ij}|^{3}\big{]}\leq\mathbb{E}\left[\big{(}\|X\|+\sqrt{d}\,\big{)}^{3}\right]\lesssim_{d}M^{3/l},$
(20) $M_{n,X}(\eta)\leq\mathbb{E}\left[\max_{1\leq j\leq
N_{\sigma,\delta}}|X_{ij}|^{3}\right]\leq\mathbb{E}\left[\big{(}\|X\|+\sqrt{d}\,\big{)}^{3}\right]\lesssim_{d}M^{3/l}.$
(21)

We next bound $M_{n,Y}(\eta)$. Here $Y_{j}$ are
$\mathcal{N}(0,\sigma_{j}^{2})$, where
$\sigma_{j}^{2}\leq\mathbb{E}\big{[}(\|X\|+\sigma\sqrt{d})^{2}\big{]}$, for
all $1\leq j\leq p$. Then, the following concentration inequality and moment
bound for Gaussian random variables hold [41, Example 2.1.19]:

$\displaystyle\mathbb{P}\left(\max_{1\leq j\leq
N_{\sigma,\delta}}|Y_{j}|>\mathbb{E}\left[\max_{1\leq j\leq
N_{\sigma,\delta}}|Y_{j}|\right]+t\right)$
$\displaystyle\leq\exp\left(-\frac{t^{2}}{2\mathbb{E}\left[\big{(}\|X\|+\sigma\sqrt{d}\big{)}^{2}\right]}\right),$
$\displaystyle\mathbb{E}\left[\max_{1\leq j\leq
N_{\sigma,\delta}}|Y_{j}|\right]$
$\displaystyle\leq\sqrt{2\mathbb{E}\left[\big{(}\|X\|+\sigma\sqrt{d}\big{)}^{2}\right]\log(2N_{\sigma,\delta})}.$

Setting $l(n,\eta)=\eta\sqrt{n}/\log
N_{\sigma,\delta}-\sqrt{2\mathbb{E}\big{[}(\|X\|+\sigma\sqrt{d})^{2}\big{]}\log
2N_{\sigma,\delta}}$, we have for $l(n,\eta)>0$,

$\begin{split}M_{n,Y}(\eta)&=\mathbb{E}\left[\max_{1\leq j\leq
p}|Y_{j}|^{3}\mathds{1}_{\left\\{\max_{1\leq j\leq p}|Y_{j}|>\eta\sqrt{n}/\log
N_{\sigma,\delta}\right\\}}\right]\\\
&\lesssim\int_{l(n,\eta)}^{\infty}t^{2}\exp\left(-\frac{t^{2}}{2\mathbb{E}\left[\big{(}\|X\|+\sigma\sqrt{d}\big{)}^{2}\right]}\right)\,\,\mathrm{d}t\\\
&\leq\int_{0}^{\infty}t^{2}\exp\left(-\frac{t^{2}}{2\mathbb{E}\left[\big{(}\|X\|+\sigma\sqrt{d}\big{)}^{2}\right]}\right)\,\,\mathrm{d}t\
\lesssim_{d,M}1.\end{split}$ (22)

Hence, from (20)-(22), for every $A\in\mathcal{B}(\mathbb{R})$ and every
$\eta>0$ such that $l(n,\eta)>0$, we have

$\mathbb{P}(Z\in A)<\mathbb{P}\big{(}\widetilde{Z}\in
A^{\eta}\big{)}+\frac{C_{\alpha,d,M,l}(\log
N_{\sigma,\delta})^{2}}{\eta^{3}\sqrt{n}},$ (23)

for a constant $C_{\alpha,d,M,l}$ depending only on $\alpha$, $d$, $l$, and
$M$.

Step 5 (Strassen’s theorem). By (23) and Markov’s inequality, we have (as in
(7.2)), for each $\eta\in(0,1]$ and $n$ such that $l(n,\eta)>0$,

$\displaystyle\mathbb{P}\left(\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\in
A\right)$
$\displaystyle\qquad\leq\mathbb{P}\left(L^{\mspace{1.0mu}\sigma}_{P}\in
A^{(2+C_{7})\eta}\right)+\mathbb{P}\Big{(}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}(\delta)}>\eta\Big{)}+\mathbb{P}\Big{(}\|G_{P}\|_{\mathcal{F}_{\sigma}(\delta)}>\eta\Big{)}+\frac{C_{\alpha,d,M,l}(\log
N_{\sigma,\delta})^{2}}{\eta^{3}\sqrt{n}}$ (24)
$\displaystyle\qquad\leq\mathbb{P}\Big{(}L^{\mspace{1.0mu}\sigma}_{P}\in
A^{(2+C_{7})\eta}\Big{)}+\frac{\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}(\delta)}\big{]}}{\eta}+\frac{\mathbb{E}\big{[}\|G_{P}\|_{\mathcal{F}_{\sigma}(\delta)}\big{]}}{\eta}+\frac{C_{\alpha,d,M,l}(\log
N_{\sigma,\delta})^{2}}{\eta^{3}\sqrt{n}}.$

Hence, by Strassen’s theorem (Lemma 6), for $\eta>0$ such that $l(n,\eta)>0$,
there exists a coupling
$\big{(}V_{n,\sigma,\delta,\eta},W_{n,\sigma,\delta,\eta}\big{)}$ of
$\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)$ and
$L^{\mspace{1.0mu}\sigma}_{P}$, such that

$\mathbb{P}\big{(}|V_{n,\sigma,\delta,\eta}-W_{n,\sigma,\delta,\eta}|>(2+C_{7})\eta\big{)}\leq\frac{\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma}(\delta)}\big{]}}{\eta}+\frac{C_{\alpha,d,M,l}(\log
N_{\sigma,\delta})^{2}}{\eta^{3}\sqrt{n}}+\frac{\mathbb{E}\big{[}\|G_{P}\|_{\mathcal{F}_{\sigma}(\delta)}\big{]}}{\eta}.$
(25)

Substituting the bounds (14), (18), and (19) in (25), we obtain the following
estimate

$\mathbb{P}\big{(}|V_{n,\sigma,\delta,\eta}-W_{n,\sigma,\delta,\eta}|>(2+C_{7})\eta\big{)}\lesssim_{\alpha,d,M,l}\frac{1}{\eta}\left[\sigma^{-ab}\delta^{1-b}+\frac{1}{\delta^{2b}\sqrt{n}}\sigma^{-a(1+2b)}\right]+\frac{1}{\eta^{3}}\frac{\sigma^{-4ab}}{\delta^{4b}\sqrt{n}}.$
(26)

Step 6 (Rates for $\delta$ and $\eta$). Note that (26) holds for any
$\sigma,\delta,\eta\in(0,1]$, and $n$ under the moment condition of the
theorem and $l(n,\eta)>0$. Let $\sigma=\sigma_{n}$ as in the statement of the
theorem.

Taking $\delta=n^{-1/(2(3+b))}\sigma^{-ab/(3+b)}$ and
$\eta=\eta_{n}=K_{n}n^{-(1-b)/(2(3+b))}\sigma_{n}^{-4ab/(3+b)}$ for $K_{n}\geq
1$, we see that the RHS is $O(1/K_{n})$ with $l(n,\eta_{n})>0$, for all
sufficiently large $n$. For all such $n$, the random variables
$V_{n}:=V_{n,\sigma_{n},\delta_{n},\eta_{n}}$ and
$W_{n}:=W_{n,\sigma_{n},\delta_{n},\eta_{n}}$ satisfy

$\mathbb{P}\left(|V_{n}-W_{n}|\geq
K_{n}n^{-\frac{1-b}{2(3+b)}}\sigma_{n}^{-\frac{4ab}{3+b}}\right)=\frac{1}{K_{n}}O(1).$

For $K_{n}\to\infty$ we have the claim (i).

Proof of (ii). To control
$\rho\big{(}\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{n}}(P_{n},P),\,L_{P}^{\mspace{1.0mu}\sigma_{n}}\big{)}$,
we start from (24) and bound the latter probability terms on RHS using
concentration inequalities.

Step 1 (Empirical process concentration via Fuk-Nagev). To treat the empirical
process term, we apply Lemma 9 to $\mathcal{F}_{\sigma_{n}}(\delta_{n})$.
Matching the lemma notation to our framework, we set $p=l>4\alpha-d+3$,
$\mathcal{F}=\\{f-Pf\,:\,f\in\mathcal{F}_{\sigma_{n}}(\delta_{n})\\}$,
$F(x)=\|x\|+{\sigma_{n}}\sqrt{d}$, $\alpha=1$,
$Z=\sqrt{n}\|\mathbb{G}_{n}\|_{\mathcal{F}}$, $\sigma^{2}=\delta_{n}^{2}$,
$\mathbb{E}[M^{p}]\lesssim_{d}\mathbb{E}\big{[}\max_{1\leq i\leq
n}\|X_{i}\|^{p}\big{]}\leq n\mathbb{E}\big{[}\|X_{i}\|^{p}\big{]}$, and
$x=\sqrt{n}\big{(}\eta_{n}-2\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}\big{)}$.
For
$\eta_{n}>2\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}$,
we then have, for all $n$ such that
$\eta_{n}-2\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}>0$,

$\begin{split}\mathbb{P}\Big{(}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}>\eta_{n}\Big{)}&\lesssim_{\alpha,d,M,l}\underbrace{e^{-\frac{c_{\alpha,d}}{\delta_{n}^{2}}\left(\eta_{n}-2\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}\right)^{2}}}_{\mathrm{(a)}}\\\
&\qquad+\underbrace{n^{-\frac{l-2}{2}}\Big{(}\eta_{n}-2\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}\Big{)}^{-(4\alpha-d+3)}}_{\mathrm{(b)}}.\end{split}$

Hence, if we choose $\delta_{n}$ and $\eta_{n}$ such that $\eta_{n}\geq
3\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\sigma_{n},\delta_{n}}\big{]}$, and
$\eta_{n}/\delta_{n}\gtrsim n^{p^{\prime}}$ for some $p^{\prime}>0$, where
$p^{\prime}$ depends only on $\alpha$ and $d$, then
$\text{(a)}\lesssim_{\alpha,d,M,l,p}n^{-p}$ for any $p>0$. We then also have
$\text{(b)}\lesssim_{\alpha,d,M,l}n^{-(l-2)/2}\eta_{n}^{-l}$.

Step 2 (Gaussian concentration). Under the above rate conditions, the Borel-
Sudakov-Tsirelson inequality [41, Theorem 2.2.7] implies that, for any $p>0$,

$\displaystyle\mathbb{P}\Big{(}\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}>\eta_{n}\Big{)}$
$\displaystyle=\mathbb{P}\Big{(}\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}>\mathbb{E}[\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}]+\big{(}\eta_{n}-\mathbb{E}[\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}]\big{)}\Big{)}$
$\displaystyle\leq
e^{-\frac{1}{2\delta_{n}^{2}}\big{(}\eta_{n}-\mathbb{E}\big{[}\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}\big{)}^{2}}$
$\displaystyle\lesssim_{\alpha,d,M,l,p}n^{-p},$ (27)

with the same parameters as before.

Step 3 (Combined bounds and optimal rates). Assume we have chosen $\delta_{n}$
and $\eta_{n}$ so that the rate conditions from Step 1 hold. Using the last
bounds in Steps 1 and 2 we obtain, respectively,
$\mathbb{P}(\|\mathbb{G}_{n}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}>\eta_{n})\lesssim_{\alpha,d,M,l,p}n^{-(l-2)/2}\eta_{n}^{-l}+n^{-p}$
and
$\mathbb{P}(\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}>\eta_{n})\lesssim_{\alpha,d,M,l,p}n^{-p}$,
for any $p>0$. Note that (14) further gives $\frac{C_{d,M,l}(\log
N_{\sigma_{n},\delta_{n}})^{2}}{\eta_{n}^{3}\sqrt{n}}\lesssim_{\alpha,d,M,l}\frac{1}{\eta_{n}^{3}}\frac{\sigma_{n}^{-4ab}\delta_{n}^{-4b}}{\sqrt{n}}$.
Inserting these bounds into (24), the Prokhorov distance is bounded as

$\rho\left(\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{n}}(P_{n},P),L_{P}^{\mspace{1.0mu}\sigma}\right)\lesssim_{\alpha,d,M,l}\eta_{n}\wedge\frac{1}{\eta_{n}^{3}}\frac{\sigma_{n}^{-4ab}\delta_{n}^{-4b}}{\sqrt{n}}\wedge
n^{-\frac{l-2}{2}}\eta_{n}^{-l}.$

To conclude, we set $\delta_{n}=n^{-1/8}$. Then, from (19), we can choose a
constant $C_{\alpha,d,M,l}$ such that
$\eta_{n}=C_{\alpha,d,M,l}n^{-(1-b)/8}\sigma_{n}^{-ab}\geq
3\mathbb{E}\big{[}\|G_{P}\|_{\mathcal{F}_{\sigma_{n}}(\delta_{n})}\big{]}$,
which fulfills the rate conditions from Step 1 with $p^{\prime}=b$, and yields

$\rho\left(\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma_{n}}(P_{n},P),\,L_{P}^{\mspace{1.0mu}\sigma_{n}}\right)\lesssim_{\alpha,d,M,l}n^{-(1-b)/8}\sigma_{n}^{-ab},$

as claimed.

∎

###  7.3 Proof (Outline) of Theorem 3

The subsection presents an outline of the proof, with the full details
available in Appendix B.2. The comparison between the empirical bootstrap
process and the limit Gaussian process is split into two parts. First, we
compare the limit Gaussian process with its bootstrap analogue, the multiplier
bootstrap process. Then, we related the multiplier bootstrap process to the
empirical bootstrap process.

The multiplier bootstrap process indexed by the smooth 1-Lipschitz class
$\mathcal{F}_{\sigma}$ is
$\mathbb{G}_{n}^{B}(f):=n^{-1/2}\sum_{i=1}^{n}\xi_{i}\big{(}f\left(X_{i}\right)-P_{n}f\big{)},\
{f\in\mathcal{F}_{\sigma}}$, where $\xi_{i}\overset{iid}{\sim}\mathcal{N}_{1}$
are independent of $(X_{i})_{i\in\mathbb{N}}$. We also introduce the shorthand
$X_{1:n}:=(X_{1},\dots,X_{n})$. As in (7.2), we start by discretizing the
three processes and then compare the distributions of the discretized maxima
of $\mathbb{G}_{n}^{B}$ and $G_{P}$. We have

$\displaystyle\mathbb{P}$
$\displaystyle\big{(}\|G_{n}^{*}\|_{\mathcal{F}_{\sigma}}\in
A\,\big{|}\,X_{1:n}\big{)}-\mathbb{P}\left(\|\mathbb{G}_{P}\|_{\mathcal{F}_{\sigma}}\in
A^{(3+C_{7})\eta}\right)$
$\displaystyle\leq\underbrace{\mathbb{P}\Big{(}\big{|}\|G_{P}\|_{\mathcal{F}_{\sigma}}-\max_{1\leq
j\leq
N_{\sigma,\delta}}G_{P}(f_{j})\big{|}>\eta\Big{)}}_{\text{(I)}}+\underbrace{\mathbb{P}\left(\big{|}\|\mathbb{G}_{n}^{*}\|_{\mathcal{F}_{\sigma}}-\max_{1\leq
j\leq
N_{\sigma,\delta}}\mathbb{G}_{n}^{*}(f_{j})\big{|}>\eta\,\Big{|}\,X_{1:n}\right)}_{\text{(II)}}$
$\displaystyle\qquad\quad+\underbrace{\mathbb{P}\left(\max_{1\leq j\leq
N_{\sigma,\delta}}\mathbb{G}_{n}^{*}(f_{j})\in
A^{\eta}\,\middle|\,X_{1:n}\right)-\mathbb{P}\left(\max_{1\leq j\leq
N_{\sigma,\delta}}\mathbb{G}_{n}^{B}(f_{j})\in
A^{(1+C_{7})\eta}\,\middle|\,X_{1:n}\right)}_{\text{(III)}}$
$\displaystyle\qquad\quad+\underbrace{\mathbb{P}\left(\max_{1\leq j\leq
N_{\sigma,\delta}}\mathbb{G}_{n}^{B}(f_{j})\in
A^{(1+C_{7})\eta}\middle|\,X_{1:n}\right)-\mathbb{P}\left(\max_{1\leq j\leq
N_{\sigma,\delta}}G_{P}(f_{j})\in A^{(2+C_{7})\eta}\right)}_{\text{(IV)}}.$
(28)

Term (I) can be bounded using (27), while (II) is controlled using [83, Lemma
3.6.6], with the expectation bound supplied by the local maximal inequality
for a poissonized empirical process. Term (III) is handled by Lemma 8, while a
bound on (IV) can be obtained by controlling the difference between the sample
and population covariance matrices. For the latter, we use the following
result.

######  Lemma 11 (Theorem 3.2, [23]).

Let $X=(X_{1},\dots,X_{p})^{\intercal}\sim\mathcal{N}(\mu,\Sigma^{X})$ and
$Y=(Y_{1},\dots,Y_{p})^{\intercal}\sim\mathcal{N}(\mu,\Sigma^{Y})$ be Gaussian
random vectors in $\mathbb{R}^{p}$. Define $\Delta=\max_{1\leq j,k\leq
p}\big{|}\Sigma_{j,k}^{X}-\Sigma_{j,k}^{Y}\big{|}$,
$Z\mspace{-2.0mu}=\mspace{-2.0mu}\max_{1\leq j\leq p}\mspace{-2.0mu}X_{j}$,
and $\widetilde{Z}=\max_{1\leq j\leq p}Y_{j}$. Then, for every $\eta>0$ and
$A\in\mathcal{B}(\mathbb{R})$, we have

$\mathbb{P}(Z\in A)\leq\mathbb{P}\big{(}\widetilde{Z}\in
A^{\eta}\big{)}+C\eta^{-1}\sqrt{\Delta\log p},$

where $C>0$ is an universal constant.

It thus remains to convert (7.3) to an absolute difference bound in
probability. For this step, we use a conditional version of Strassen’s theorem
to ensure the existence of couplings
$\big{(}V_{\sigma,n},W_{\sigma,n}\big{)}$, with
$\mathsf{Law}\left(V_{\sigma,n}\,\big{|}\,X_{1:n}\right)=\mathsf{Law}\left(\|\mathbb{G}_{n}^{*}\|_{\mathcal{F}_{\sigma}}\middle|\,X_{1:n}\right)$
and
$\mathsf{Law}(W_{\sigma,n})=\mathsf{Law}\left(\|G_{P}\|_{\mathcal{F}_{\sigma}}\right)$,
such that the difference $|V_{\sigma,n}-W_{\sigma,n}|$ is bounded in
probability.

######  Lemma 12 (Lemma 4.2, [23]).

Let $V$ be a real-valued random variable defined on a probability space
$(\Omega,\mathcal{A},\mathbb{P})$, and let $\mathcal{C}$ be a countably
generated sub $\sigma$-field of $\mathcal{A}$. Assume that there exists a
uniform random variable on $[0,1]$ independent of $\mathcal{C}\vee\sigma(V)$.
Let $G(\cdot|\mathcal{C})$ be a regular conditional distribution on the Borel
$\sigma$-field of $\mathbb{R}$ given $\mathcal{C}$, and suppose that for some
$\delta>0$ and $\epsilon>0$,

$\mathbb{E}\left[\sup_{A\in\mathcal{B}[\mathbb{R}]}\left(\mathbb{P}\big{(}V\in
A\,\big{|}\,\mathcal{C}\big{)}-G\big{(}A^{\delta}\,\big{|}\,\mathcal{C}\big{)}\right)\right]\leq\epsilon.$

Then there exists a random variable $W$ such that the conditional distribution
of $W$ given $\mathcal{C}$ coincides with $G(\cdot|\mathcal{C})$ and

$\mathbb{P}\big{(}|V-W|>\delta\big{)}<\epsilon.$

An exact characterization of the dependence on $\sigma$ will allow us to fix
$\sigma=\sigma_{n}$, possibly vanishing as $n$ tends to infinity, in the
bounds. The bound on Prokhorov distance will follow directly from (7.3) by
bounding the expectation of terms on the RHS.

###  7.4 Proof of Theorem 4

We prove the following stronger result, from which Theorem 4 follows.

######  Theorem 9 (Strengthening of Theorem 4 via entropy condition).

Suppose $P\in\mathcal{P}(\mathbb{R}^{d})$ satisfies

$N_{\epsilon}(P,\epsilon^{\frac{\alpha s}{s-2}})\leq
K\epsilon^{-s}\big{(}\log(1/\epsilon)\big{)}^{\beta},\quad\forall\,\epsilon>0$
(29)

and $P\|x\|^{s}=M<\infty$, for some constant $K>0$, $\beta\geq 0$ and
$\alpha>s>2$. Then, we have

$\mathbb{E}\left[\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]\lesssim_{\alpha,d,s,M,K,\beta}\sigma^{-\alpha+1}.$

If instead $\alpha=s$, then

$\mathbb{E}\left[\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]\lesssim_{d,s,M,K,\beta}\sigma^{-s/2+1}(\log
n)^{\frac{\beta+3}{2}}.$ (30)

In particular, if $\dim^{*}_{\mathsf{SW}_{1}}(P)<s$ and $P\|x\|^{s}=M<\infty$,
then (30) holds with $\beta=0$ and $\alpha=s/2$.

We again begin with a few technical lemmas that will be used in the proof. The
first provides a bound on tail expectations of random variables in the
quantile form. This result is later used to bound the $L^{2}(P)$ entropy of
$\mathcal{F}_{\sigma}$ by accounting for the probability mass excluded in
(29).

######  Lemma 13 (Tail Moment).

Let $X$ be a non-negative random variable with $\mathbb{E}[X^{p}]=M<\infty$,
for $p>1$. Then, for any $u\in(0,1)$ and any event $A$ with $\mathbb{P}(A)\leq
u$, we have $\mathbb{E}[X\mathds{1}_{A}]\leq\frac{M}{1-1/p}u^{1-1/p}$.

The next lemma gives a bound on the metric entropy of smooth function classes
defined on sufficiently small balls. This will be used in conjunction with the
entropy condition (29) to bound the entropy of the entire, unbounded, function
class, as in [86, Theorem 2.7.4].

######  Lemma 14 (Metric entropy of small balls).

For all $\epsilon\in(0,1)$, $\log
N\big{(}\epsilon,C_{1}^{\alpha}\big{(}B(\epsilon^{1/\alpha})\big{)},\|\cdot\|_{\infty}\big{)}\lesssim_{\alpha,d}\log\big{(}1+\frac{2C_{d}}{\epsilon}\big{)},$
where $C_{d}=(1+e^{d})$.

Next, we state an adaptation of [83, Lemma 2.14.3] for controlling moments of
the empirical process when the entropy of the indexing function class is not
integrable around 0.

######  Lemma 15 (Maximal inequality for non-integrable bracketing entropy).

Let $\mathcal{F}$ be a function class symmetric about $0$ and containing the
$0$ function, with a measurable envelope $F\in L^{2}(P)$. Then,

$\begin{split}\mathbb{E}\big{[}\|\mathbb{G}_{n}\|_{\mathcal{F}}\big{]}\lesssim\inf_{0<\eta<1/3}&\left\\{\eta\sqrt{n}\|F\|_{L^{2}(P)}+\|F\|_{L^{2}(P)}\int_{\eta}^{1}\sqrt{1+\log
N_{[\,]}\big{(}\epsilon\|F\|_{L^{2}(P)},\mathcal{F},L^{2}(P)\big{)}}\
\,\mathrm{d}\epsilon\right\\}\\\ &\qquad\quad\
+\sqrt{n}PF\mathds{1}_{\left\\{4F\geq\sqrt{n}\|F\|_{L^{2}(P)}\left(1+\log
N_{[\,]}\big{(}\|F\|_{L^{2}(P)}/2,\mathcal{F},L^{2}(P)\big{)}\right)^{-1/2}\right\\}}.\end{split}$

Proofs of the above lemmas are given in Appendix C. We are now ready to prove
Theorem 9. By (29), there is a set $S\in\mathcal{B}(\mathbb{R}^{d})$ such that
$P(S)>1-\epsilon^{s/(s-2)}$ and $N_{\epsilon^{1/\alpha}}(S)\leq
K\alpha^{-\beta}\epsilon^{-s/\alpha}\big{(}\log(1/\epsilon)\big{)}^{\beta}$.
Let $x_{1},\dots,x_{m}$ be a minimal $\epsilon^{1/\alpha}$-net of $S$. By
Lemma 4, there is a constant $C_{\alpha,d}$, depending only on $\alpha$ and
$d$, such that each $\mathcal{F}_{\sigma}|_{B(x_{j},\epsilon^{1/\alpha})}$ is
a subset of
$C_{M^{\prime}_{j}}^{\alpha}\big{(}B(x_{j},\epsilon^{1/\alpha})\big{)}$ for
$M^{\prime}_{j}=C_{\alpha,d}\sigma^{-\alpha+1}\big{(}1\vee\sup_{x\in
B(x_{j},\epsilon^{1/\alpha})}\|x\|\big{)}$. For every $j=1,\dots,m$, let
$f_{j,1},\dots,f_{j,p_{j}}$ be a minimal $\epsilon M^{\prime}_{j}$-net of
$\mathcal{F}_{\sigma}|_{B(x_{j},\epsilon^{1/\alpha})}$ w.r.t.
$\|\cdot\|_{\infty}$. Consider now the disjointed balls:

$B_{1}=B\big{(}x_{1},\epsilon^{1/\alpha}\big{)},\
B_{j}=B(x_{j},\epsilon^{1/\alpha})\setminus\left\\{\bigcup\nolimits_{k=1}^{j-1}B_{k}\right\\},\quad
j=2,\dots,m.$

Let $g(x)=\|x\|\wedge 1+\sqrt{d}$, $F(x)=\sigma^{-\alpha+1}g(x)$. Construct
the function brackets

$\sum_{j=1}^{m}\big{(}f_{j,i_{j}}\mathds{1}_{B_{j}}\pm\epsilon
M^{\prime}_{j}\big{)}\pm
F\mathds{1}_{\left\\{\bigcup_{j=1}^{n}B_{j}\right\\}^{c}},\quad
i_{j}=1,\dots,p_{j},\quad j=1,\ldots,m.$

These cover $\mathcal{F}_{\sigma}$, and have $L^{2}(P)$ bracket width bounded
by:

$\sqrt{4\epsilon^{2}\sum_{j=1}^{m}\big{(}M_{j}^{{}^{\prime}2}P(B_{j})\big{)}+4PF^{2}\mathds{1}_{\left\\{\bigcup_{j=1}^{n}B_{j}\right\\}^{c}}}\leq
2\epsilon\sigma^{-\alpha+1}\mspace{-1.0mu}\sqrt{Pg^{2}+1+\frac{Pg^{s}}{1-\frac{2}{s}}}\lesssim_{\alpha,M,s,d}\epsilon\|F\|_{L^{2}(P)},$

where the first inequality uses Lipschitzness of $g$ and Lemma 13, while the
second is since $P\|x\|^{s}\mspace{-4.0mu}=\mspace{-4.0mu}M$.

The total number of covering brackets is $\prod_{j=1}^{m}p_{j}$. Lemma 14
implies $\log p_{j}\lesssim_{\alpha,d}\log\big{(}1+2C_{d}/\epsilon\big{)}$ for
each $j=1,\dots,m$. Since (29) implies $m\leq
K\alpha^{-\beta}\epsilon^{-s/\alpha}\big{(}\log(1/\epsilon)\big{)}^{\beta}$,
we further have

$\log
N_{[\,]}\big{(}C\epsilon\|F\|_{L^{2}(P)},\mathcal{F}_{\sigma},L^{2}(P)\big{)}\leq\sum_{j=1}^{m}\log
p_{j}\lesssim_{\alpha,M,s,d,\beta}K\epsilon^{-s/\alpha}\log\left(1+\frac{2C_{d}}{\epsilon}\right)\big{(}\log(1/\epsilon)\big{)}^{\beta}.$

For $\alpha>s/2$, the square root of the right hand side is integrable around
0, so by Theorem 2.14.1 in [83], we get the first conclusion of Theorem 4. For
$\alpha=s/2$, we have

$\log
N_{[\,]}\big{(}C\epsilon\|F\|_{L^{2}(P)},\mathcal{F}_{\sigma},L^{2}(P)\big{)}\lesssim_{\alpha,s,d,M}K\epsilon^{-2}\big{(}\log(1/\epsilon)\big{)}^{1+\beta},$

which is not integrable around 0; this is where Lemma 15 is needed. Observing
that

$\sqrt{n}PF\mathds{1}_{\\{F>\sqrt{n}C\|F\|_{L^{2}(P)}\\}}\leq
PF^{2}\mathds{1}_{\\{F>\sqrt{n}C\|F\|_{L^{2}(P)}\\}}/(C\|F\|_{L^{2}(P)})\leq\|F\|_{L^{2}(P)}/C,$

where $C=\sqrt{1+\log
N_{[\,]}\big{(}\|F\|_{L^{2}(P)}/2,\mathcal{F}_{\sigma},L^{2}(P)\big{)}}\geq
1$, the lemma implies that

$\mathbb{E}\left[\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\right]\lesssim\inf_{0<\gamma\leq
1/3}\|F\|_{L^{2}(P)}\mspace{-3.0mu}\left(\mspace{-2.0mu}1\mspace{-2.0mu}+\mspace{-2.0mu}\gamma\sqrt{n}\mspace{-3.0mu}+\mspace{-1.0mu}\int_{\gamma}^{1}\mspace{-5.0mu}\sqrt{1+\log
N_{[\,]}(\epsilon\|F\|_{L^{2}(P)},\mathcal{F}_{\sigma},L^{2}(P))}\,\mathrm{d}\epsilon\mspace{-2.0mu}\right)\mspace{-4.0mu}.$

Choosing $\gamma=1/\sqrt{9n}$ yields
$\mathbb{E}\big{[}\sqrt{n}\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}(P_{n},P)\big{]}\lesssim_{\alpha,s,d,\beta}\|F\|_{L^{2}(P)}+\|F\|_{L^{2}(P)}(\log
n)^{(\beta+3)/2}$, and the second conclusion follows by noting that
$\|F\|_{L^{2}(P)}\lesssim_{\alpha,d,M}\sigma^{-\alpha+1}$.

For the last statement, observe that if $\dim^{*}_{\mathsf{SW}_{1}}(P)<s$,
then $\limsup_{\epsilon\to
0}d_{\epsilon}\Big{(}P,\epsilon^{s^{2}/(2(s-2))}\Big{)}\leq s$ by definition.
This implies, for all small enough $\epsilon>0$, that
$N_{\epsilon}\big{(}P,\epsilon^{s^{2}/(2(s-2))}\big{)}\lesssim_{P}\epsilon^{-s}$,
which is precisely condition (29) with $\alpha=s/2$ and $\beta=0$.

##  8 Concluding Remarks

In this work, we established limit distribution, bootstrap consistency,
concentration, and strong approximation results for the empirical SWD.
Furthermore, we showed that its rate of convergence in expectation adapts to
the intrinsic dimension through the smoothing parameter. The two-sample limit
distribution and bootstrap results were applied to obtain a consistent
homogeneity test. Additionally, we explored minimum SWD estimation on
parametric families and established measurability, consistency, and limit
distributions. The dimension adaptability of SWD was used to derive sharp
rates of convergence for empirical $\mathsf{W}_{1}$ (up to log factors) when
the population distribution has unbounded support—a case that remained open
until now.

This work focused on statistical aspects of empirical SWD. An important future
objective is to develop an efficient algorithm for computing
$\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$, tailored to exploit the Gaussian
convolution structure. Other possible future directions include evaluating
asymptotic power and relative efficiency for two-sample homogeneity testing
based on $\mathsf{W}_{1}^{\mspace{1.0mu}\sigma}$, and modifying the statistic
$W_{m,n}$ to test for independence of paired observations. While this paper
extensively exploits the dual form of the smoothed transport problem, it would
also be interesting to study the rate of convergence of optimal empirical
couplings, defined in an appropriate sense, to the set of optimal couplings of
the population distributions. Properties of such smoothed couplings between
measures, such as cross-correlations between the involved variables, are also
of interest.

## Acknowledgements

Z. Goldfeld is supported by the NSF CRII Grant CCF-1947801, the NSF CAREER
Award under Grant CCF-2046018, and the 2020 IBM Academic Award. K. Kato is
partially supported by NSF grants DMS-1952306 and DMS-2014636.

## References

  * [1] [author] A. Genevay, M. CuturiM. C., Peyré, G.G. and Bach, F.F. (2017). Stochastic optimization for large-scale optimal transport. In Advances in Neural Information Processing Systems (NeurIPS-2016) 3440–3448.
  * [2] [author] Adamczkak, R.R. (2008). A tail inequality for suprema of unbounded empirical processes with applications to Markov chains. Electronic Journal of Probability 34 1000-1034.
  * [3] [author] Adamczkak, R.R. (2010). A Few Remarks on the Operator Norm of Random Toeplitz Matrices. Journal of Theoretical Probability 23 85-108.
  * [4] [author] Aliprantis, C. D.C. D. and Border, K. C.K. C. (2006). Infinite Dimensional Analysis: A Hitchhiker’s Guide. Springer.
  * [5] [author] Altschuler, J.J., Niles-Weed, J.J. and Rigollet, P.P. (2017). Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS-2017) 1964–1974.
  * [6] [author] Alvarez-Melis, D.D. and Jaakkola, T. S.T. S. (2018). Gromov-Wasserstein alignment of word embedding spaces. arXiv preprint arXiv:1809.00013.
  * [7] [author] Arjovsky, M.M., Chintala, S.S. and Bottou, L.L. (2017). Wasserstein generative adversarial networks. In International Conference on Machine Learning (ICML-2017) 214–223.
  * [8] [author] Arora, S.S., Ge, R.R., Liang, Y.Y., Ma, T.T. and Zhang, Y.Y. (2017). Generalization and Equilibrium in Generative Adversarial Nets (GANs). In Proceedings of the International Conference on Machine Learning (ICML-2017) 224-232.
  * [9] [author] Bassetti, F.F., Bodini, A.A. and Regazzini, E.E. (2006). On minimum Kantorovich distance estimators. Statistics & Probability Letters 76 1298–1302.
  * [10] [author] Bassetti, F.F. and Regazzini, E.E. (2006). Asymptotic properties and robustness of minimum dissimilarity estimators of location-scale parameters. Theory of Probability & Its Applications 50 171–186.
  * [11] [author] Belili, N.N., Bensaï, A.A. and Heinich, H.H. (1999). An estimate based Kantorovich functional and the Levy distance. Comptes Rendus de l’Academie des Sciences Series I Mathematics 5 423–426.
  * [12] [author] Bernton, E.E., Jacob, P. E.P. E., Gerber, M.M. and Robert, C. P.C. P. (2019). On parameter estimation with the Wasserstein distance. Information and Inference: A Journal of the IMA.
  * [13] [author] Berry, A. C.A. C. (1941). The accuracy of the Gaussian approximation to the sum of independent variates. Transactions of the American Mathematical Society 49 122–136.
  * [14] [author] Bigot, J.J., Cazelles, E.E. and Papadakis, N.N. (2019). Central limit theorems for entropy-regularized optimal transport on finite spaces and statistical applications. arXiv preprint arXiv:1711.08947.
  * [15] [author] Boissard, E.E. (2011). Simple Bounds for the Convergence of Empirical and Occupation Measures in 1-Wasserstein Distance. Electron. J. Probab. 16 2296–2333.
  * [16] [author] Boissard, E.E. and Gouic, T. LeT. L. (2014). On the mean speed of convergence of empirical and occupation measures in Wasserstein distance. Annales de l’Institut Henri Poincaré Probabilités et Statistiques 50 539–563.
  * [17] [author] Bolley, F.F., Guillin, A.A. and Villani, C.C. (2007). Quantitative concentration inequalities for empirical measures on non-compact spaces. Probab. Theory Related Fields 137 541-593.
  * [18] [author] Bonnotte, N.N. (2013). Unidimensional and evolution methods for optimal transportation, PhD thesis, Paris-Sud University.
  * [19] [author] Brown, L. D.L. D. and Purves, R.R. (1973). Measurable selections of extrema. Annals of Statistics 1 902–912.
  * [20] [author] Cárcamo, J.J., Cuevas, A.A. and Rodríguez, L. A.L. A. (2020). Directional differentiability for supremum-type functionals: statistical applications. Bernoulli 26 2143–2175.
  * [21] [author] Chen, H. B.H. B. and Niles-Weed, J.J. (2020). Asymptotics of smoothed Wasserstein distances.
  * [22] [author] Chernozhukov, V.V., Chetverikov, D.D. and Kato, K.K. (2014). Gaussian approximation of Suprema of empirical processes. Annals of Statistics 42 1564-1597.
  * [23] [author] Chernozhukov, V.V., Chetverikov, D.D. and Kato, K.K. (2016). Empirical and multiplier bootstraps for suprema of empirical processes of increasing complexity, and related Gaussian couplings. Stochastic Processes and their Applications 126 3632 - 3651. https://doi.org/10.1016/j.spa.2016.04.009
  * [24] [author] Chernozhukov, V.V., Chetverikov, D.D., Kato, K.K. et al. (2014). Anti-concentration and honest, adaptive confidence bands. Annals of Statistics 42 1787–1818.
  * [25] [author] Cuturi, M.M. (2013). Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems (NeurIPS-2013) 2292–2300.
  * [26] [author] Davydov, Y. A.Y. A., Lifschits, M. A.M. A. and Smorodina, N. V.N. V. (1998). Local Properties of Distributions of Stochastic Functionals. Translation of Mathematical Monographs. American Mathematical Society.
  * [27] [author] del Barrio, E.E., Giné, E.E. and Matrán, C.C. (1999). Central limit theorems for the Wasserstein distance between the empirical and the true distributions. Annals of Probability 27 1009–1071.
  * [28] [author] del Barrio, E.E., Giné, E.E. and Utzet, F.F. (2005). Asymptotics for $L_{2}$ functionals of the empirical quantile process, with applications to tests of fit based on weighted Wasserstein distances. Bernoulli 11 131–189.
  * [29] [author] del Barrio, E.E. and Loubes, J. M.J. M. (2019). Central limit theorems for empirical transportation cost in general dimension. Annals of Probability 47 926–951.
  * [30] [author] del Barrio, E.E. and Loubes, J. M.J. M. (2019). Central limit theorems for empirical transportation cost in general dimension. Annals of Probability 47 926-951.
  * [31] [author] Dereich, S.S., Scheutzow, M.M. and Schottstedt, R.R. (2013). Constructive quantization: Approximation by empirical measures. Annales de l’Institut Henri Poincaré Probabilités et Statistiques 49 1183–1203.
  * [32] [author] Dognin, P.P., Melnyk, I.I., Mroueh, Y.Y., Ross, J.J., Santos, C. D.C. D. and Sercu, T.T. (2019). Wasserstein barycenter model ensembling. In Proceedings of the International Conference on Learning Representations (ICLR-2019).
  * [33] [author] Dudley, R. M.R. M. (1969). The speed of mean Glivenko-Cantelli convergence. Ann. Math. Stats. 40 40–50.
  * [34] [author] Dudley, R. M.R. M. (2002). Real Analysis and Probability. Cambridge University Press.
  * [35] [author] Dümbgen, L.L. (1993). On nondifferentiable functions and the bootstrap. Probability Theory and Related Fields 95 125-140.
  * [36] [author] Esseen, C. G.C. G. (1942). On the Liapunov limit error in the theory of probability. Arkiv för matematik, astronomi och fysik 28 1–19.
  * [37] [author] Feydy, J.J., Séjourné, T.T., Vialard, F. X.F. X., Amari, S.S., Trouvé, A.A. and Peyré, G.G. (2018). Interpolating between optimal transport and MMD using Sinkhorn divergences. arXiv preprint arXiv:1810.08278.
  * [38] [author] Fournier, N.N. and Guillin, A.A. (2015). On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields 162 707-738.
  * [39] [author] Gangbo, W.W. and McCann, R. J.R. J. (1996). The geometry of optimal transportation. Acta Mathematica 177 113–161.
  * [40] [author] Genevay, A.A., Chizat, L.L., Bach, F.F., Cuturi, M.M. and Peyré, G.G. (2019). Sample Complexity of Sinkhorn Divergences. In International Conference on Artificial Intelligence and Statistics (AISTATS-2019) 1574–1583.
  * [41] [author] Giné, E.E. and Nickl, R.R. (2016). Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge University Press.
  * [42] [author] Giné, E.E. and Zinn, J.J. (1986). Empirical processes indexed by Lipschitz functions. Annals of Probability 14 1329-1338.
  * [43] [author] Goldfeld, Z.Z. and Greenewald, K.K. (2020). Gaussian-Smoothed Optimal Transport: Metric Structure and Statistical Efficiency. In International Conference on Artificial Intelligence and Statistics (AISTATS-2020).
  * [44] [author] Goldfeld, Z.Z., Greenewald, K.K. and Kato, K.K. (2020). Asymptotic guarantees for generative modeling based on the smooth Wasserstein distance. In Proceedings of the Annual Conference on Advances in Neural Information Processing Systems (NeurIPS-2020).
  * [45] [author] Goldfeld, Z.Z., Greenewald, K.K., Polyanskiy, Y.Y. and Weed, J.J. (2020). Convergence of smoothed empirical measures with applications to entropy estimation. IEEE Transactions on Information Theory.
  * [46] [author] Goldfeld, Z.Z. and Kato, K.K. (2020). Limit distribution for smooth total variation and $\chi^{2}$-divergence in high dimensions. In IEEE International Symposium on Information Theory (ISIT-2020).
  * [47] [author] Graf, S.S. and Luschgy, H.H. (2007). Foundations of quantization for probability distributions. Springer.
  * [48] [author] Grave, E.E., Joulin, A.A. and Berthet, Q.Q. (2019). Unsupervised Alignment of Embeddings with Wasserstein Procrustes. In International Conference on Artificial Intelligence and Statistics (AISTATS-2019) 1880–1890.
  * [49] [author] Gulrajani, I.I., Ahmed, F.F., Arjovsky, M.M., Dumoulin, V.V. and Courville, A. C.A. C. (2017). Improved training of Wasserstein GANs. In Advances in Neural Information Processing Systems (NeurIPS-2017) 5767–5777.
  * [50] [author] Hallin, M.M., Mordant, G.G. and Segers, J.J. (2021). Multivariate goodness-of-fit tests based on Wasserstein distance. Electronic Journal of Statistics 15 1328–1371.
  * [51] [author] Hjort, N. I.N. I. and Pollard, D.D. (1993). Asymptotics for minimizers of convex processes. Unpublished manuscript.
  * [52] [author] Ho, N.N., Nguyen, X. L.X. L., Yurochkin, M.M., Bui, H. H.H. H., Huynh, V.V. and Phung, D.D. (2017). Multilevel clustering via Wasserstein means. In International Conference on Machine Learning (ICML-2017) 1501–1509.
  * [53] [author] Kantorovich, Leonid VL. V. (1942). On the translocation of masses. In Dokladi Academii Nauk 37 199–201. (russian).
  * [54] [author] Kantorovitch, L. V.L. V. and Rubinštein, G. S.G. S. (1958). On a space of completely additive functions (in Russian). Vestnik Leningrad Univ. 13 52–59.
  * [55] [author] Kato, K.K. (2009). Asymptotics for argmin processes: Convexity arguments. Journal of Multivariate Analysis 100 1816-1829.
  * [56] [author] Klatt, M.M., Tameling, C.C. and Munk, A.A. (2020). Empirical regularized optimal transport: Statistical theory and applications. SIAM Journal on Mathematics of Data Science 2 419–443.
  * [57] [author] Ledoux, M.M. and Talagrand, M.M. (1991). Probability in Banach Spaces: Isoperimetry and Processes. Springer. New York.
  * [58] [author] Lei, J.J. (2020). Convergence and concentration of empirical measures under Wasserstein distance in unbounded functional spaces. Bernoulli 26 767–798.
  * [59] [author] Li, P.P., Wang, Q.Q. and Zhang, L.L. (2013). A novel earth mover’s distance methodology for image matching with gaussian mixture models. In IEEE International Conference on Computer Vision (ICCV-2013) 1689–1696.
  * [60] [author] Lin, T.T., Zheng, Z.Z., Chen, E.E., Cuturi, M.M. and Jordan, M.M. (2021). On projection robust optimal transport: Sample complexity and model misspecification. In International Conference on Artificial Intelligence and Statistics (AISTATS-2021) 130 262–270.
  * [61] [author] Manole, T.T., Balakrishnan, S.S., Niles-Weed, J.J. and Wasserman, L.L. (2021). Plugin Estimation of Smooth Optimal Transport Maps.
  * [62] [author] Massart, P.P. (1989). Strong approximation for multivariate empirical and related processes, via KMT constructions. Annals of Probability 266–291.
  * [63] [author] Mena, G.G. and Niles-Weed, J.J. (2019). Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS-2019) 4541–4551.
  * [64] [author] Monge, G.G. (1781). Mémoire sur la théorie des déblais et des remblais. Histoire de l’Académie Royale des Sciences de Paris.
  * [65] [author] Müller, A.A. (1997). Integral probability metrics and their generating classes of functions. Advances in Applied Probability 29 429–443.
  * [66] [author] Nadjahi, K.K., Durmus, A.A., Chizat, L.L., Kolouri, S.S., Shahrampour, S.S. and Şimşekli, U.U. (2020). Statistical and Topological Properties of Sliced Probability Divergences. Advances in Neural Information Processing Systems 33\.
  * [67] [author] Nadjahi, K.K., Durmus, A.A., Şimşekli, U.U. and Badeau, R.R. (2019). Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance. In Advances in Neural Information Processing Systems (NeurIPS-2019).
  * [68] [author] Nietert, S.S., Goldfeld, Z.Z. and Kato, K.K. (2021). Smooth $p$-Wasserstein Distance: Structure, Empirical Approximation, and Statistical Applications. In Proceedings of the 38th International Conference on Machine Learning (ICML-2021) 8172–8183.
  * [69] [author] Niles-Weed, J.J. and Rigollet, P.P. (2019). Estimation of Wasserstein distances in the spiked transport model. arXiv preprint arXiv:1909.07513.
  * [70] [author] Pollard, D.D. (1980). The minimum distance method of testing. Metrika 27 43–70.
  * [71] [author] Pollard, D.D. (1991). Asymptotics for least absolute deviation regression estimators. Econometric Theory 7 186-199.
  * [72] [author] Rabin, J.J., Peyré, G.G., Delon, J.J. and Bernot, M.M. (2011). Wasserstein barycenter and its application to texture mixing. In Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision (SSVM-2011) 435–446.
  * [73] [author] Ramdas, A.A., Trillos, N. G.N. G. and Cuturi, M.M. (2017). On Wasserstein Two-Sample Testing and Related Families of Nonparametric Tests. Entropy 19\.
  * [74] [author] Rockafellar, R. T.R. T. and Wets, R. J. B.R. J. B. (2009). Variational Analysis 317\. Springer Science & Business Media.
  * [75] [author] Rubner, Y.Y., Tomasi, C.C. and Guibas, L. J.L. J. (2000). The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision 40 99–121.
  * [76] [author] Sandler, R.R. and Lindenbaum, M.M. (2011). Nonnegative matrix factorization with earth mover’s distance metric for image analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 33 1590–1602.
  * [77] [author] Santambrogio, F.F. (2015). Optimal Transport for Applied Mathematicians. Birkhäuser.
  * [78] [author] Shevtsova, I.I. (2011). On the absolute constants in the Berry-Esseen type inequalities for identically distributed summands. arXiv preprint arXiv:1111.6554.
  * [79] [author] Solomon, J.J., Goes, F. DeF. D., Gabriel, G. PeyréG. P., Cuturi, M.M., Butscher, A.A., Nguyen, A.A., Du, T.T. and Guibas, L.L. (2015). Convolutional Wasserstein distances: Efficient optimal transportation on geometric domains. ACM Transactions on Graphics (TOG) 34 66\.
  * [80] [author] Sommerfeld, M.M. and Munk, A.A. (2018). Inference for empirical Wasserstein distances on finite spaces. Journal of Royal Statistical Society Series B 80 219-238.
  * [81] [author] Sriperumbudur, B. K.B. K., Fukumizu, K.K., Gretton, A.A., Schölkopf, B.B. and Lanckriet, G. R. G.G. R. G. (2009). On integral probability metrics, $\phi$-divergences and binary classification. arXiv preprint arXiv:0901.2698.
  * [82] [author] Tameling, C.C., Sommerfeld, M.M. and Munk, A.A. (2019). Empirical optimal transport on countable metric spaces: distributional limits and statistical applications. Annals of Applied Probability 29 2744-2781.