between the true and the approximate posterior, but not a direct bound on the
test misclassification rate. They discuss how some of the tightest PAC-
Bayesian generalization bounds (which directly bound the test
misclassification rate) naturally incorporate a temperature parameter, that
trades off the effect of the prior compared to the training data.

Despite the aforementioned research, the cold and tempered posterior effect
has still not been completely explained, posing interesting and fruitful
questions for the Bayesian deep learning community.

####  5.1.4 Deep ensembles

Lakshminarayanan et al., (2017) suggest using an ensemble of networks for
uncertainty estimation, which does not suffer from mode collapse but is still
computationally expensive. Neural network ensembles are multiple MAP estimates
of the deep neural network weights. The predictions of these MAP estimates are
then averaged to make an ensemble prediction. Subsequent methods such as
snapshot ensembling (Huang et al.,, 2017), fast geometric ensembling (FGE:
Garipov et al.,, 2018), stochastic weight averaging (SWA: Izmailov et al.,,
2019), SWA-Gaussian (SWAG: Maddox et al.,, 2019), greatly reduce the
computation cost but at the price of a lower predictive performance (Ashukha
et al.,, 2020). While Lakshminarayanan et al., (2017) frame ensemble
approaches as an essentially non-Bayesian technique, they can also be cast as
a Bayesian model averaging technique (Wilson and Izmailov,, 2020; Pearce et
al.,, 2020), and can even asymptotically converge to true posterior samples
when adding repulsion (D’Angelo and Fortuin,, 2021). Specifically they can be
seen as performing a very rough Monte Carlo estimate of the posterior
distribution over weights. Ensembles are both cheap, but more importantly,
typically outperform Bayesian approaches that have been carefully crafted
(Ashukha et al.,, 2020). This has been empirically explained as resulting from
the increased functional diversity of different modes of the loss landscape
(Fort et al.,, 2019). These are sampled by definition using deep ensembles,
and this sampling is hard to beat using Bayesian inference.

###  5.2 Performance certificates

TL;DR Bayesian inference is renowned for its ability to provide guarantees on
accurate inference of the true posterior distribution given a sufficient
amount of data. However, such guarantees pertain to the accurate estimation of
the posterior distribution itself, rather than ensuring performance on out-of-
sample data. To address the latter, it becomes necessary to rely on
generalization bounds, such as the PAC-Bayes framework. Within this framework,
model comparison utilizing the marginal likelihood offers guarantees on the
performance of the selected model on out-of-sample data, provided that the
inference process has been conducted accurately.

####  5.2.1 Frequentist validation of the posterior

Recent works address generalization and approximation errors for the
estimation of smooth functions in a nonparametric regression framework using
sparse deep NNs and study their posterior mass concentration depending on data
sample size. Schmidt-Hieber, (2020) shows that sparsely connected deep neural
network with ReLU activation converges at near-minimax rates when estimating
Hölder-smooth functions, preventing the curse of dimensionality. Based on this
work, Polson and Rockova, (2018) introduce a Spike-and-Slab prior for deep
ReLU networks which induces a specific regularization scheme in the model
training. The obtained posterior in such neural networks concentrates around
smooth functions with near-minimax rates of convergence. Further, Kohler and
Langer, (2021) extend the consistency guarantees for Hölder-smooth functions
of Schmidt-Hieber, (2020) and Polson and Rockova, (2018) to fully connected
neural networks without the sparsity assumption. Alternatively, Suzuki, (2018)
provides generalization error bounds for more general functions in Besov
spaces and variants with mixed smoothness.

One of the ways to visualize the obtained uncertainty is using credible sets
around some parameter estimator, where the credible region contains a large
fraction of the posterior mass (Szabó et al.,, 2015). Hadji and Szabó, (2021)
study the uncertainty resulting from using Gaussian process priors. Franssen
and Szabó, (2022) provide Bayesian credible sets with frequentist coverage
guarantees for standard neural networks trained with gradient descent. Only
the last layer is assigned a prior distribution on the parameters and the
output obtained from the previous layer is used to compute the posterior.

####  5.2.2 Posterior concentration and generalization to out-of-sample data

It is interesting to take a step back and evaluate the difference in _goals_
between the frequentist and Bayesian approaches to machine learning. The
Bayesian approach emphasizes that the posterior concentrates around the true
parameter as we increase the training set size, see the previous section. The
primary goal of the frequentist approach is the performance on out-of-sample
data, i.e., generalization, see Section 2.4. This performance is quantified
with validation and test sets. These two goals frequently align, although
posterior concentration guarantees and performance on out-of-sample data are
typically not mathematically equivalent problems.

When the number of parameters is smaller than the number of samples $n$,
typically in parametric models, the posterior concentrates on the true set of
parameters when $n$ approaches to infinity. In such cases, the posterior tends
to a Dirac delta mass centered on the true parameters. In this setting, we can
then argue that we are making predictions using the true predictive
distribution, and frequentist and Bayesian goals align. We have inferred the
true predictor (according to Bayesian goals) and can be sure that we cannot
improve the predictor loss on new out-of-sample data, such as validation and
test sets (according to the frequentist approach priorities).

However, neural networks do not operate in this regime. They are heavily
overparametrized, so that Bayesian model averaging always occurs empirically.
Usually, we are not interested in the proposed model itself but in its
predictions based on new data. Also, due to misspecification, we cannot even
assume that we are concentrating around the true predictor. At this point, the
frequentist and Bayesian goals diverge. But it is clear that in a non-
asymptotic setting and where performance on out-of-sample data is crucial, we
need a more detailed description of the predictor’s loss on new data.

One way to approach this problem is through generalization bounds (Vapnik,,
1999) which directly link the empirical loss on the training set with the loss
on new data. Of particular interest are PAC-Bayes generalization bounds
(McAllester,, 1999; Germain et al.,, 2016; Dziugaite and Roy,, 2017; Dziugaite
et al.,, 2021), which directly bound the true risk of a stochastic predictor.
Minimizing the ELBO objective in variational inference corresponds to
minimizing a PAC-Bayes bound (Dziugaite and Roy,, 2017), and thus a bound on
the true risk. If alternatively one samples _exactly_ from the Gibbs posterior
(for example using MCMC), then one is still minimizing a PAC-Bayes bound on
the true risk (Germain et al.,, 2016). Furthermore, in this setting,
maximizing the _marginal likelihood_ of the model is equivalent to minimizing
a PAC-Bayes bound (Germain et al.,, 2016) and it has been shown that PAC-Bayes
bounds can be used to meta-learn better priors for BNNs (Rothfuss et al.,,
2021, 2022).

Of particular interest in this discussion is that performing Bayesian
inference is equivalent to minimizing _some_ PAC-Bayes bound and not
necessarily _the tightest_ bound. PAC-Bayes bounds typically include a
temperature parameter that trades-off the empirical risk with the KL
complexity term, and plays a crucial role in the bound tightness (see Section
5.1.3). An interesting open question is whether this temperature parameter
provides a justification for the _cold posterior effect_ , with a number of
works providing evidence to support this view (Grünwald,, 2012; Pitas and
Arbel,, 2022).

####  5.2.3 Marginal likelihood and generalization

The marginal likelihood (MacKay,, 2003) has been explored for model selection,
architecture search and hyperparameter learning for deep neural networks.
While estimating the marginal likelihood and computing its gradients is
relatively straightforward for simple models such as Gaussian processes
(Bishop and Nasrabadi,, 2006), deep neural networks often require to resort to
approximations.

One approach is the Laplace approximation as previously discussed in Section
4.2.2. Daxberger et al., 2021a ; Immer et al., 2021a ; Immer et al., (2022)
use the Laplace approximation to the marginal likelihood to select the best-
performing model on out-of-sample data. They also use the marginal likelihood
to learn hyperparameters, in particular the prior variance and the softmax
temperature parameter. For the case of the Laplace approximation, the marginal
likelihood of training data $\mathcal{D}$ given the deep neural network
architecture $\mathcal{M}$ can be written as

$\displaystyle\log p(\mathcal{D}|\mathcal{M})$ $\displaystyle=\log
p(\mathcal{D}|\hat{\bm{w}}_{\text{MAP}},\mathcal{M})+\log
p(\hat{\bm{w}}_{\text{MAP}}|\mathcal{M})$ $\displaystyle+\frac{d}{2}\log
2\pi+\frac{1}{2}\log\left|\bm{\Lambda}_{\hat{\bm{w}}_{\text{MAP}}}\right|,$
(11)

where $d$ is the number of weights of the neural network,
$\hat{\bm{w}}_{\text{MAP}}$ is a MAP estimate of the network parameters, and
$\bm{\Lambda}_{\hat{\bm{w}}_{\text{MAP}}}$ is the precision matrix of the
Gaussian posterior distribution under the Laplace approximation. Similarly to
the discussion in Section 4.2.2, the primary computational problem is forming
the precision matrix and estimating its determinant. Again the generalized
Gauss–Newton approximation and the Empirical Fisher approximation to the
Hessian (and correspondingly to the precision matrix) are the most common and
efficient approximations, and are the ones used in Daxberger et al., 2021a ;
Immer et al., 2021a . On a conceptual level, a main criticism of the Laplace
approximation for the marginal likelihood of deep neural networks is that it
is unimodal while the loss landscape of deep neural networks has multiple
minima (Lotfi et al.,, 2022). This might severely underestimate the volume of
good solutions with respect to bad solutions given the prior, which is
essentially what the marginal likelihood estimates. A further criticism is
that this approximation to the marginal likelihood is sensitive to the prior
variance. Indeed for a fixed prior variance across different neural network
architectures, Lotfi et al., (2022) show that the marginal likelihood performs
poorly for model selection. However optimizing a common prior covariance
across layers, or optimizing different prior variances for different layers,
results in a better empirical correlation of the marginal likelihood with out-
of-sample performance. Overall, the marginal likelihood provides reasonable
predictive power for out-of-sample performance for deep neural networks, and
as such constitutes a reasonable approach to model selection.

A different approach is to resort to the product decomposition of the marginal
likelihood as

$\displaystyle\log p(\mathcal{D}|\mathcal{M})$
$\displaystyle=\log\prod_{i=1}^{n}p(\mathcal{D}_{i}|\mathcal{D}_{<i},\mathcal{M})$
(12)
$\displaystyle=\sum_{i=1}^{n}\log[\bm{\mathrm{E}}_{p(\theta|\mathcal{D}_{<i})}p(\mathcal{D}_{i}|\theta,\mathcal{M})]$

which measures how good the model is at predicting each data point
$\mathcal{D}_{i}$ in sequence given every data point before it,
$\mathcal{D}_{<i}$. Based on this observation, Lyle et al., (2020); Ru et al.,
(2021) propose the sum of losses of the different batches across an epoch as
an approximation to the marginal likelihood. Then, they use this as a measure
of the ability of a model to generalize to out-of-sample data. They also
propose different heuristics, such as taking the average of the sum of the
losses over multiple epochs. A further heuristic is keeping only the last
epochs of training while rejecting the sum of the losses of the first epochs.
Finally the authors propose to train the neural network for a limited number
of epochs, for example only half of the number of epochs that would be
typically used to train to convergence. As such the approach is
computationally efficient, requiring only partial convergence of the deep
neural network and a calculation of the training losses over batches, which
are efficient to estimate.

Ru et al., (2021) compare their approach to the task of architecture search to
other common approaches. These approaches are a mixture of heuristics and
frequentist statistics. The first is the sum of validation losses up to a
given epoch. The second is the validation accuracy at an early epoch, which
corresponds to the early-stopping practice whereby the user estimates the
final test performance of a network using its validation accuracy at an early
epoch. The third is the learning curve extrapolation method, which was
proposed in Baker et al., (2017) and which trains a regression model on
previously evaluated architecture data to predict the final test accuracy of
new architectures. The inputs for the regression model comprise architecture
meta-features and learning curve features up to a given epoch. They also
compare to zero-cost baselines: an estimator based on input Jacobian
covariance (JavCov, Mellor et al.,, 2021) and two adapted from pruning
techniques (SNIP and SynFlow, Abdelfattah et al.,, 2021). The authors
demonstrate significantly better rank-correlation in neural architecture
search (NAS) for the marginal likelihood approach compared to the baselines.
These results have been further validated in Lotfi et al., (2022).

Ru et al., (2021) have however been criticized for using the term “training
speed” (as in the number of steps needed to reach a certain training error) to
describe their approach. In short, they claim that Equation (12) corresponds
to some measure of training speed, and thus they claim that _training faster
corresponds to better generalization_. This however is not generally true as
pointed out in Lotfi et al., (2022). The marginal likelihood can be _larger_
for a model that converges _in more steps_ (than another model) if the
marginal likelihood at step $i=1$ in decomposition (12) is higher.

There is a debate as to whether the marginal likelihood is appropriate for
model selection at all. Lotfi et al., (2022) make a distinction between the
question “what is the probability that a prior model generated the training
data?” and the question “how likely is the posterior, conditioned on the
training data, to have generated withheld points drawn from the same
distribution?”. They claim that the marginal likelihood answers the first
question and not the second. However, high marginal likelihood also provides
frequentist guarantees on out-of-sample performance through PAC-Bayesian
theorems (Germain et al.,, 2016). If one selects a model based on the marginal
likelihood and also performs Bayesian inference correctly, then the resulting
model and its posterior over parameters are guaranteed to result in good
performance on out-of-sample data. Overall, the debate is far from concluded,
and in light of the good empirical performance of the marginal likelihood,
more research is warranted in its direction.

###  5.3 Benchmarking

BNNs present unique challenges in terms of their evaluation and benchmarking.
Two main challenges are the choice of the evaluation _datasets_ and _metrics_
that do not have a consensus in the society. Non-consensus reflects a
difficulty with clearly defining the goals of Bayesian deep learning in a
field traditionally viewed through a _frequentist_ lens, and more specifically
through performance on out-of-sample data.

TL;DR When evaluating Bayesian deep learning approaches, it is customary to
employ standard datasets like MNIST, FMNIST, and CIFAR-10 as benchmark
datasets. In contrast, the adoption of Imagenet is relatively infrequent due
to computational limitations associated with its larger scale. The primary
advantage of Bayesian methods lies in their ability to provide reliable
calibration metrics, such as the Expected Calibration Error (ECE) and the
Thresholded Adaptive Calibration Error (TACE). However, caution must be
exercised when comparing models that utilize temperature scaling, as different
practitioners may perceive it either as an integral component of the model or
as part of the evaluation metric.

####  5.3.1 Evaluation datasets

There is no single dataset benchmark specifically tailored to Bayesian deep
learning models. The significant computational burden imposed by Bayesian deep
learning has limited the dataset choice to smaller datasets comparatively with
how deterministic neural networks are evaluated. The majority of papers, for
example Khan et al., (2018); Khan and Swaroop, (2021); Maddox et al., (2019);
Gal and Ghahramani, (2016); Izmailov et al., 2021b ; Wenzel et al., (2020),
use the MNIST, CIFAR-10, CIFAR-100, and Imagenet datasets combined with
different metrics. Also popular are the UCI regression datasets (Dua and
Graff,, 2017) such as Concrete, Energy, Kin8nm, Naval, Power, Wine, Yacht.
However, these datasets are smaller than what would be a typical benchmark for
deep learning models such as modern CNNs architectures.

The Imagenet dataset is notably absent from most papers, as benchmarking on
Imagenet for Bayesian deep learning is computationally prohibitive for most
practitioners. Both the CIFAR-10 dataset and the CIFAR-100 dataset have 50K
training samples, while Imagenet has 1.3M training samples. Correspondingly, a
ResNet for CIFAR-10 might typically have 20 layers while 50 or more layers are
typically required for Imagenet. Many Bayesian deep learning approaches incur
linear increases in computational cost with respect to the training set size
when estimating the Hessian, and quadratic increases in storage cost with
respect to the number of parameters, potentially resulting in prohibitive
costs.

There have been some efforts to create a standardized benchmarking task that
reflects the complexities and challenges of safety-critical real-world tasks
while adequately accounting for the reliability of the models predictive
uncertainty estimates. Band et al., (2021) undertake this issue applying it to
the _diabetic retinopathy_ dataset. The dataset is made of retina images
associated with diabetic retinopathy, a medical condition considered as the
leading cause of vision impairment and blindness. Unlike in other works on
diabetic retinopathy detection, the benchmarking tasks presented in Band et
al., (2021) are specifically designed to assess the reliability of machine
learning models and the quality of their predictive uncertainty estimates
using both aleatoric and epistemic uncertainty estimates. Examples of tasks
are _selective prediction_ and _expert referral_. These mirror real-world
scenarios of predictive uncertainty estimates to identify data points where
the likelihood of an incorrect prediction is particularly high and refer for
further review to experts, in this case, doctors. Better uncertainty estimates
would lead to lower error rates on this task. However, the work of Band et
al., (2021) has not been widely adopted.

####  5.3.2 Evaluation metrics-tasks

For most popular machine learning tasks, the community has reached a consensus
on the appropriate evaluation metric of choice, such as the mean-squared error
(MSE) for regression and zero-one loss for classification. In the case of
Bayesian deep learning, there is not yet a clear choice. Should the Bayesian
approach improve on frequentist metrics such as misclassification rate on
held-out data? Should it provide solutions to known issues of traditional
approaches, such as improved robustness to adversarial and non-adversarial
noise? Or should Bayesian approaches be evaluated on different metrics
altogether or on metrics that capture _uncertainty_?

Standard losses. Practitioners propose several metrics (and corresponding
tasks) for the evaluation of Bayesian deep learning approaches. By far the
most popular choice is to evaluate frequentist metrics on held-out data that
are the MSE for regression and the zero-one loss for classification (Khan et
al.,, 2018; Khan and Swaroop,, 2021; Gal and Ghahramani,, 2016; Izmailov et
al., 2021b, ; Wenzel et al.,, 2020). The intuition behind this choice is that
the posterior predictive distribution should improve upon deterministic
predictions as multiple predictions from the posterior are averaged. For
example, in the case of classification, the posterior predictive is meant to
better approximate the _probability_ that a given class is correct.

One problem with this approach is that Bayesian approaches have typically
provided inconsistent gains for this task-metric combination. For example,
sometimes Bayesian approaches improve upon a deterministic neural network and
sometimes provide worse results. See for example Figure 5 in Izmailov et al.,
2021b  where the MSE is evaluated on UCI regression tasks. Similarly, Figure
4.a. in Daxberger et al., 2021a  shows that the Laplace approximation to a DNN
posterior does not improve upon the MAP solution.

Wenzel et al., (2020) point out that one can improve upon deterministic neural
networks by using heuristics such as cold posteriors which however deviate
from the Bayesian paradigm. One common switch away from MSE and zero-one loss
consists in evaluating the (negative) log-likelihood of the test data. Here,
Bayesian approaches often outperform frequentist ones, but exceptions remain
(Wenzel et al.,, 2020).

Calibration. By far, the metric on which Bayesian neural networks consistently
outperform is the _calibration_ for a classification task, i.e., if a
classifier has $x\%$ confidence when classifying samples from a sample set, it
should also be correct $x\%$ of the time. The two most popular metrics for
evaluating calibration are the _expected calibration error_ (ECE: DeGroot and
Fienberg,, 1983), and the _thresholded adaptive calibration error_ (TACE:
Nixon et al.,, 2019). For this type of task-metric combination Bayesian and
Bayesian-like approaches such as ensembles (see Section 5.1.4) consistently
outperform deterministic neural networks (Izmailov et al., 2021b, ; Daxberger
et al., 2021a, ; Maddox et al.,, 2019). Ashukha et al., (2020) provide a
detailed discussion on evaluation metrics for uncertainty estimation as well
as common pitfalls. They argue that for a given metric one should always
compare a Bayesian method to an ensemble. Ensembles provide good gains in
different uncertainty metrics for each new ensemble member. Bayesian methods,
often do not result in the same gains for each new sample from the posterior.

Other methods for evaluating calibration include reliability diagrams
(Vaicenavicius et al.,, 2019) and calibration curves (Maddox et al.,, 2019). A
strength of these metrics is that they are generally clear, direct and
intuitive. One weakness of them is that like other visual methods, they are
subject to misinterpretation. For example, calibration curves provide a simple
and intuitive way to determine which classifier is better calibrated than
others when the difference between the classifiers is large. However, when the
difference is small or the classifier is miscalibrated only for certain
confidence levels, then deriving reliable conclusions becomes more tedious.
One caveat is that a classifier that is guessing completely at random and
assigns the marginal class frequencies as predictive probabilities to each
data point would trivially achieve a perfect ECE of 0 (Gruber and Buettner,,
2022). Moreover, it has been argued that while many of these metrics measure
marginal uncertainties on single data points, joint uncertainties across many
points might be more relevant in practice, e.g., for sequential decision-
making (Osband et al.,, 2022).

Robustness. There are many works that explored robustness to adversarial noise
(Louizos and Welling,, 2017; Rawat et al.,, 2017; Liu et al.,, 2019; Grosse et
al.,, 2019; Bekasov and Murray,, 2018) and to non-adversarial noise (Gal and
Ghahramani,, 2016; Daxberger et al., 2021b, ; Dusenberry et al.,, 2020;
Daxberger et al., 2021a, ; Izmailov et al., 2021a, ), including Gaussian
noise, image rotations, among others. Band et al., (2021) analyze a form of
distribution shift whereby classifiers are trained on a set of images for
which diabetic retinopathy exists at moderate levels. Then, the evaluation of
the classifiers is assessed on a test set where diabetic retinopathy is more
severe. The intuition is that Bayesian approaches should correctly classify
these corrupted samples and assign low confidence in their predictions. The
results in these tasks-metrics are mixed. In the adversarial setting, BNNs are
typically far from the state-of-the-art defenses against adversarial attacks.
In the non-adversarial setting, some works show _improved_ robustness
(Daxberger et al., 2021b, ), while others show _reduced_ robustness (Izmailov
et al., 2021a, ).

####  5.3.3 Output interpretation

We conclude by analyzing the output of BNNs with the question of its
probabilistic interpretation and its relation to evaluation metrics. We
restrict the discussion to classification models, though the discussion for
other tasks is similar. Both frequentist and Bayesian practitioners recognize
that the outputs of a deep neural network classifier often do not accurately
reflect the probability of choosing the correct class. That is, the NNs are
not well calibrated. However, frequentist and Bayesian communities propose
different solutions. The frequentist solution is to transform the outputs of
the classifier through a post-processing step to obtain well-calibrated
outputs. Common approaches include _histogram binning_ (Zadrozny and Elkan,,
2001), _isotonic regression_ (Zadrozny and Elkan,, 2002), _Bayesian binning
into quantiles_ (Naeini et al.,, 2015) as well as _Platt scaling_ (Platt,,
1999).

In a Bayesian setting, the predictive distribution has a clear interpretation,
that is the confidence of the model in each class for a given input signal.
Confusion can arise from the fact that scaling is sometimes considered part of
an evaluation metric. For example, Guo et al., (2017) consider _Platt scaling_
as a post-processing step (therefore it defines a new model), while Ashukha et
al., (2020) propose that it be incorporated into a new evaluation metric. The
choice of which of the two is true is important as the impact of recalibration
methods can be significant in improving the calibration of a model. Thus, if
one considers recalibration as defining a new model as in Ashukha et al.,
(2020), then a K-FAC Laplace BNN outperforms its corresponding frequentist one
significantly in calibration. If recalibration is part of the evaluation
metric, then the gains become marginal.

##  6 Conclusion

The present review encompasses various topics, such as the selection of prior
(Section 3.2), computational methods (Section 3.3), and model selection
(Section 3.4), which pertain to Bayesian problems in a general sense as well
as Bayesian neural networks specifically. This comprehensive perspective
enables the contextualization of the diverse inquiries that emerge within the
Bayesian deep learning community.

Despite the growing interest and advancements in inference techniques for
Bayesian deep learning models, the considerable computational burden
associated with Bayesian deep learning approaches remains a primary hindrance.
Consequently, the community dedicated to Bayesian deep learning remains
relatively small, and the adoption of these approaches in the industry remains
limited.

The establishment of a consensus regarding evaluation metrics and benchmarking
datasets for Bayesian deep learning has yet to be attained. The lack of
consensus stems from the challenge of precisely defining the objectives of
Bayesian deep learning within a domain traditionally perceived through a
_frequentist_ framework, particularly emphasizing performance on out-of-sample
data.

This review provides readers with a thorough exposition of the challenges
intrinsic to Bayesian deep learning, while also shedding light on avenues that
warrant additional exploration and enhancement. With this cohesive resource,
our objective is to empower statisticians and machine learners alike,
facilitating a deeper understanding of Bayesian neural networks (BNNs) and
promoting their wider practical implementation.

## References

  * Abdar et al., (2021) Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., and Acharya, U. R. (2021).  A review of uncertainty quantification in deep learning: Techniques, applications and challenges.  Information Fusion.
  * Abdelfattah et al., (2021) Abdelfattah, M. S., Mehrotra, A., Dudziak, Ł., and Lane, N. D. (2021).  Zero-cost proxies for lightweight NAS.  arXiv preprint arXiv:2101.08134.
  * (3) Adlam, B., Lee, J., Xiao, L., Pennington, J., and Snoek, J. (2020a).  Exploring the uncertainty properties of neural networks’ implicit priors in the infinite-width limit.  In International Conference on Learning Representations.
  * (4) Adlam, B., Snoek, J., and Smith, S. L. (2020b).  Cold posteriors and aleatoric uncertainty.  ICML Workshop on Uncertainty & Robustness in Deep Learning.
  * Ahn et al., (2012) Ahn, S., Korattikara, A., and Welling, M. (2012).  Bayesian posterior sampling via stochastic gradient fisher scoring.  In International Conference on Machine Learning.
  * Aitchison, (2020) Aitchison, L. (2020).  Why bigger is not always better: on finite and infinite neural networks.  In International Conference on Machine Learning.
  * Aitchison, (2021) Aitchison, L. (2021).  A statistical theory of cold posteriors in deep neural networks.  In International Conference on Learning Representations.
  * Antognini, (2019) Antognini, J. M. (2019).  Finite size corrections for neural network Gaussian processes.  ICML Workshop on Theoretical Physics for Deep Learning.
  * Antorán et al., (2022) Antorán, J., Janz, D., Allingham, J. U., Daxberger, E., Barbano, R. R., Nalisnick, E., and Hernández-Lobato, J. M. (2022).  Adapting the linearised Laplace model evidence for modern deep learning.  In International Conference on Machine Learning.
  * Arora et al., (2019) Arora, S., Du, S. S., Hu, W., Li, Z., Salakhutdinov, R., and Wang, R. (2019).  On exact computation with an infinitely wide neural net.  In Advances in Neural Information Processing Systems.
  * Arora et al., (2018) Arora, S., Ge, R., Neyshabur, B., and Zhang, Y. (2018).  Stronger generalization bounds for deep nets via a compression approach.  In International Conference on Machine Learning.
  * Ashukha et al., (2020) Ashukha, A., Lyzhov, A., Molchanov, D., and Vetrov, D. (2020).  Pitfalls of in-domain uncertainty estimation and ensembling in deep learning.  International Conference on Learning Representations.
  * Bachmann et al., (2022) Bachmann, G., Noci, L., and Hofmann, T. (2022).  How Tempering Fixes Data Augmentation in Bayesian Neural Networks.  Advances in Neural Information Processing Systems.
  * Baker et al., (2017) Baker, B., Gupta, O., Raskar, R., and Naik, N. (2017).  Accelerating neural architecture search using performance prediction.  arXiv preprint arXiv:1705.10823.
  * Band et al., (2021) Band, N., Rudner, T. G., Feng, Q., Filos, A., Nado, Z., Dusenberry, M. W., Jerfel, G., Tran, D., and Gal, Y. (2021).  Benchmarking bayesian deep learning on diabetic retinopathy detection tasks.  In NeurIPS Workshop on Distribution Shifts: Connecting Methods and Applications.
  * Barber and Bishop, (1998) Barber, D. and Bishop, C. M. (1998).  Ensemble learning in Bayesian neural networks.  In Generalization in Neural Networks and Machine Learning, pages 215–237. Springer-Verlag.
  * Barron, (1994) Barron, A. R. (1994).  Approximation and estimation bounds for artificial neural networks.  Machine Learning, 14(1):115–133.
  * Bartlett et al., (2017) Bartlett, P. L., Foster, D. J., and Telgarsky, M. J. (2017).  Spectrally-normalized margin bounds for neural networks.  In Advances in Neural Information Processing Systems.
  * Bekasov and Murray, (2018) Bekasov, A. and Murray, I. (2018).  Bayesian adversarial spheres: Bayesian inference and adversarial examples in a noiseless setting.  NeurIPS Workshop on Bayesian Deep Learning.
  * Belkin et al., (2019) Belkin, M., Hsu, D., Ma, S., and Mandal, S. (2019).  Reconciling modern machine-learning practice and the classical bias–variance trade-off.  National Academy of Sciences, 116(32):15849–15854.
  * Bibi et al., (2018) Bibi, A., Alfadly, M., and Ghanem, B. (2018).  Analytic expressions for probabilistic moments of PL-DNN with Gaussian input.  In Computer Vision and Pattern Recognition.
  * Bingham et al., (2019) Bingham, E., Chen, J. P., Jankowiak, M., Obermeyer, F., Pradhan, N., Karaletsos, T., Singh, R., Szerlip, P., Horsfall, P., and Goodman, N. D. (2019).  Pyro: Deep universal probabilistic programming.  Journal of Machine Learning Research, 20(1):973–978.
  * Bishop and Nasrabadi, (2006) Bishop, C. M. and Nasrabadi, N. M. (2006).  Pattern recognition and machine learning, volume 4.  Springer.
  * Blaas and Roberts, (2021) Blaas, A. and Roberts, S. J. (2021).  The effect of prior Lipschitz continuity on the adversarial robustness of Bayesian neural networks.  arXiv preprint arXiv:2101.02689.
  * Blei et al., (2017) Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017).  Variational inference: A review for statisticians.  Journal of the American Statistical Association, 112(518):859–877.
  * Blumer et al., (1989) Blumer, A., Ehrenfeucht, A., Haussler, D., and Warmuth, M. K. (1989).  Learnability and the vapnik-chervonenkis dimension.  Journal of the ACM, 36(4):929–965.
  * Blundell et al., (2015) Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015).  Weight uncertainty in neural networks.  International Conference on Machine Learning.
  * Candès, (1998) Candès, E. J. (1998).  Ridgelets: theory and applications.  PhD thesis, Stanford University.
  * Carpenter et al., (2017) Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. (2017).  Stan: A probabilistic programming language.  Journal of Statistical Software, 76(1):1–32.
  * (30) Chatziafratis, V., Nagarajan, S. G., and Panageas, I. (2020a).  Better depth-width trade-offs for neural networks through the lens of dynamical systems.  In International Conference on Machine Learning.
  * (31) Chatziafratis, V., Nagarajan, S. G., Panageas, I., and Wang, X. (2020b).  Depth-width trade-offs for ReLU networks via Sharkovsky’s theorem.  International Conference on Learning Representations.
  * Chen et al., (2014) Chen, T., Fox, E., and Guestrin, C. (2014).  Stochastic gradient hamiltonian monte carlo.  In International Conference on Machine Learning.
  * Cho and Saul, (2009) Cho, Y. and Saul, L. K. (2009).  Kernel methods for deep learning.  In Advances in Neural Information Processing Systems.
  * Cox, (1961) Cox, R. T. (1961).  Algebra of probable inference.  JHU Press.
  * Cui et al., (2021) Cui, T., Havulinna, A., Marttinen, P., and Kaski, S. (2021).  Informative Bayesian Neural Network Priors for Weak Signals.  Bayesian Analysis, 1(1):1–31.
  * Cybenko, (1989) Cybenko, G. (1989).  Approximation by superpositions of a sigmoidal function.  Mathematics of Control, Signals and Systems, 2(4):303–314.
  * Damianou and Lawrence, (2013) Damianou, A. and Lawrence, N. D. (2013).  Deep Gaussian processes.  In International Conference on Artificial Intelligence and Statistics.
  * D’Angelo and Fortuin, (2021) D’Angelo, F. and Fortuin, V. (2021).  Repulsive deep ensembles are Bayesian.  Advances in Neural Information Processing Systems, 34:3451–3465.
  * d’Ascoli et al., (2021) d’Ascoli, S., Touvron, H., Leavitt, M., Morcos, A., Biroli, G., and Sagun, L. (2021).  ConViT: Improving vision transformers with soft convolutional inductive biases.  International Conference on Machine Learning.
  * Davenport and Kalakota, (2019) Davenport, T. and Kalakota, R. (2019).  The potential for artificial intelligence in healthcare.  Future healthcare journal, 6(2):94.
  * (41) Daxberger, E., Kristiadi, A., Immer, A., Eschenhagen, R., Bauer, M., and Hennig, P. (2021a).  Laplace Redux-effortless Bayesian deep learning.  Advances in Neural Information Processing Systems.
  * (42) Daxberger, E., Nalisnick, E., Allingham, J. U., Antorán, J., and Hernández-Lobato, J. M. (2021b).  Bayesian deep learning via subnetwork inference.  In International Conference on Machine Learning.
  * de Valpine et al., (2017) de Valpine, P., Turek, D., Paciorek, C. J., Anderson-Bergman, C., Lang, D. T., and Bodik, R. (2017).  Programming with models: writing statistical algorithms for general model structures with NIMBLE.  Journal of Computational and Graphical Statistics, 26(2):403–413.
  * DeGroot and Fienberg, (1983) DeGroot, M. H. and Fienberg, S. E. (1983).  The comparison and evaluation of forecasters.  Journal of the Royal Statistical Society: Series D (The Statistician), 32(1-2):12–22.
  * Der Kiureghian and Ditlevsen, (2009) Der Kiureghian, A. and Ditlevsen, O. (2009).  Aleatory or epistemic? Does it matter?  Structural safety, 31(2):105–112.
  * Dillon et al., (2017) Dillon, J. V., Langmore, I., Tran, D., Brevdo, E., Vasudevan, S., Moore, D., Patton, B., Alemi, A., Hoffman, M., and Saurous, R. A. (2017).  Tensorflow distributions.  arXiv preprint arXiv:1711.10604.
  * Dosovitskiy et al., (2021) Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021).  An image is worth 16x16 words: Transformers for image recognition at scale.  International Conference on Learning Representations.
  * Dua and Graff, (2017) Dua, D. and Graff, C. (2017).  UCI machine learning repository.
  * Dusenberry et al., (2020) Dusenberry, M., Jerfel, G., Wen, Y., Ma, Y., Snoek, J., Heller, K., Lakshminarayanan, B., and Tran, D. (2020).  Efficient and scalable Bayesian neural nets with rank-1 factors.  In International Conference on Machine Learning.
  * Duvenaud et al., (2014) Duvenaud, D., Rippel, O., Adams, R., and Ghahramani, Z. (2014).  Avoiding pathologies in very deep networks.  In International Conference on Artificial Intelligence and Statistics.
  * Dyer and Gur-Ari, (2020) Dyer, E. and Gur-Ari, G. (2020).  Asymptotics of wide networks from Feynman diagrams.  In International Conference on Learning Representations.
  * Dziugaite et al., (2021) Dziugaite, G. K., Hsu, K., Gharbieh, W., Arpino, G., and Roy, D. (2021).  On the role of data in pac-bayes bounds.  In International Conference on Artificial Intelligence and Statistics.
  * Dziugaite and Roy, (2017) Dziugaite, G. K. and Roy, D. M. (2017).  Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data.  arXiv preprint arXiv:1703.11008.
  * Erhan et al., (2010) Erhan, D., Courville, A., Bengio, Y., and Vincent, P. (2010).  Why does unsupervised pre-training help deep learning?  In International Conference on Artificial Intelligence and Statistics.
  * Favaro et al., (2020) Favaro, S., Fortini, S., and Stefano, P. (2020).  Stable behaviour of infinitely wide deep neural networks.  In International Conference on Artificial Intelligence and Statistics.
  * Fawzi et al., (2016) Fawzi, A., Moosavi-Dezfooli, S.-M., and Frossard, P. (2016).  Robustness of classifiers: from adversarial to random noise.  Advances in Neural Information Processing Systems.
  * Flam-Shepherd et al., (2017) Flam-Shepherd, D., Requeima, J., and Duvenaud, D. (2017).  Mapping Gaussian process priors to Bayesian neural networks.  In NeurIPS Workshop on Bayesian Deep Learning.
  * Flam-Shepherd et al., (2018) Flam-Shepherd, D., Requeima, J., and Duvenaud, D. (2018).  Characterizing and warping the function space of Bayesian neural networks.  In NeurIPS Workshop on Bayesian Deep Learning.
  * Folgoc et al., (2021) Folgoc, L. L., Baltatzis, V., Desai, S., Devaraj, A., Ellis, S., Manzanera, O. E. M., Nair, A., Qiu, H., Schnabel, J., and Glocker, B. (2021).  Is mc dropout bayesian?  arXiv preprint arXiv:2110.04286.
  * Foong et al., (2019) Foong, A. Y., Burt, D. R., Li, Y., and Turner, R. E. (2019).  Pathologies of factorised Gaussian and MC dropout posteriors in Bayesian neural networks.  Stat, 1050:2.
  * Foret et al., (2021) Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. (2021).  Sharpness-aware minimization for efficiently improving generalization.  In International Conference on Learning Representations.
  * Fort et al., (2019) Fort, S., Hu, H., and Lakshminarayanan, B. (2019).  Deep ensembles: A loss landscape perspective.  arXiv preprint arXiv:1912.02757.
  * Fortuin, (2021) Fortuin, V. (2021).  On the Choice of Priors in Bayesian Deep Learning.  PhD thesis, ETH Zurich.
  * Fortuin, (2022) Fortuin, V. (2022).  Priors in Bayesian deep learning: A review.  International Statistical Review.
  * Fortuin et al., (2021) Fortuin, V., Garriga-Alonso, A., Wenzel, F., Rätsch, G., Turner, R., van der Wilk, M., and Aitchison, L. (2021).  Bayesian neural network priors revisited.  Symposium on Advances in Approximate Bayesian Inference.
  * Frankle and Carbin, (2019) Frankle, J. and Carbin, M. (2019).  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  International Conference on Learning Representations.
  * Franssen and Szabó, (2022) Franssen, S. and Szabó, B. (2022).  Uncertainty quantification for nonparametric regression using empirical bayesian neural networks.  arXiv preprint arXiv:2204.12735.
  * Funahashi, (1989) Funahashi, K.-I. (1989).  On the approximate realization of continuous mappings by neural networks.  Neural Networks, 2(3):183–192.
  * Gal, (2016) Gal, Y. (2016).  Uncertainty in deep learning.  PhD thesis, University of Cambridge.
  * Gal and Ghahramani, (2016) Gal, Y. and Ghahramani, Z. (2016).  Dropout as a Bayesian approximation: Representing model uncertainty in deep learning.  In International Conference on Machine Learning.
  * Garipov et al., (2018) Garipov, T., Izmailov, P., Podoprikhin, D., Vetrov, D., and Wilson, A. G. (2018).  Loss surfaces, mode connectivity, and fast ensembling of DNNs.  In Advances in Neural Information Processing Systems.
  * Garriga-Alonso et al., (2019) Garriga-Alonso, A., Rasmussen, C. E., and Aitchison, L. (2019).  Deep convolutional networks as shallow Gaussian processes.  In International Conference on Learning Representations.
  * Gawlikowski et al., (2021) Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., and Roscher, R. (2021).  A survey of uncertainty in deep neural networks.  arXiv preprint arXiv:2107.03342.
  * Geirhos et al., (2019) Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W. (2019).  ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.  International Conference on Learning Representations.
  * Gelman and Rubin, (1992) Gelman, A. and Rubin, D. B. (1992).  Inference from iterative simulation using multiple sequences.  Statistical Science, 7(4):457–472.
  * Gelman et al., (2020) Gelman, A., Vehtari, A., Simpson, D., Margossian, C. C., Carpenter, B., Yao, Y., Kennedy, L., Gabry, J., Bürkner, P.-C., and Modrák, M. (2020).  Bayesian workflow.  arXiv preprint arXiv:2011.01808.
  * Germain et al., (2016) Germain, P., Bach, F., Lacoste, A., and Lacoste-Julien, S. (2016).  PAC-Bayesian theory meets Bayesian inference.  Advances in Neural Information Processing Systems.
  * Ghahramani, (2015) Ghahramani, Z. (2015).  Probabilistic machine learning and artificial intelligence.  Nature, 521(7553):452–459.
  * Glorot and Bengio, (2010) Glorot, X. and Bengio, Y. (2010).  Understanding the difficulty of training deep feedforward neural networks.  In International Conference on Artificial Intelligence and Statistics.
  * Goan and Fookes, (2020) Goan, E. and Fookes, C. (2020).  Bayesian neural networks: An introduction and survey.  In Case Studies in Applied Bayesian Data Science, pages 45–87. Springer.
  * Golowich et al., (2017) Golowich, N., Rakhlin, A., and Shamir, O. (2017).  Size-independent sample complexity of neural networks.  arXiv preprint arXiv:1712.06541.
  * Graves, (2011) Graves, A. (2011).  Practical variational inference for neural networks.  Advances in Neural Information Processing Systems.
  * Graves et al., (2013) Graves, A., Mohamed, A., and Hinton, G. E. (2013).  Speech recognition with deep recurrent neural networks.  In International Conference on Acoustics, Speech and Signal Processing.
  * Grosse et al., (2019) Grosse, K., Pfaff, D., Smith, M. T., and Backes, M. (2019).  The limitations of model uncertainty in adversarial settings.  NeurIPS Workshop on Bayesian Deep Learning.
  * Gruber and Buettner, (2022) Gruber, S. and Buettner, F. (2022).  Better uncertainty calibration via proper scores for classification and beyond.  Advances in Neural Information Processing Systems, 35:8618–8632.
  * Grünwald, (2012) Grünwald, P. (2012).  The safe Bayesian.  In International Conference on Algorithmic Learning Theory.
  * Gunasekar et al., (2018) Gunasekar, S., Lee, J., Soudry, D., and Srebro, N. (2018).  Implicit bias of gradient descent on linear convolutional networks.  In Advances in Neural Information Processing Systems.
  * Guo et al., (2017) Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. (2017).  On calibration of modern neural networks.  In International Conference on Machine Learning.
  * Gurbuzbalaban et al., (2021) Gurbuzbalaban, M., Şimşekli, U., and Zhu, L. (2021).  The heavy-tail phenomenon in SGD.  In International Conference on Machine Learning.
  * Hadji and Szabó, (2021) Hadji, A. and Szabó, B. (2021).  Can we trust Bayesian uncertainty quantification from Gaussian process priors with squared exponential covariance kernel?  Journal on Uncertainty Quantification, 9(1):185–230.
  * Hayou, (2022) Hayou, S. (2022).  On the infinite-depth limit of finite-width neural networks.  arXiv preprint arXiv:2210.00688.
  * Hayou et al., (2019) Hayou, S., Doucet, A., and Rousseau, J. (2019).  On the impact of the activation function on deep neural networks training.  In International Conference on Machine Learning.
  * He et al., (2015) He, K., Zhang, X., Ren, S., and Sun, J. (2015).  Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification.  In International Conference on Computer Vision.
  * (94) He, K., Zhang, X., Ren, S., and Sun, J. (2016a).  Deep residual learning for image recognition.  In Computer Vision and Pattern Recognition.
  * (95) He, K., Zhang, X., Ren, S., and Sun, J. (2016b).  Identity mappings in deep residual networks.  In European Conference on Computer Vision.
  * Hein et al., (2019) Hein, M., Andriushchenko, M., and Bitterwolf, J. (2019).  Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem.  In Computer Vision and Pattern Recognition.
  * Hernández-Lobato and Adams, (2015) Hernández-Lobato, J. M. and Adams, R. (2015).  Probabilistic backpropagation for scalable learning of Bayesian neural networks.  In International Conference on Machine Learning.
  * Hinton and Van Camp, (1993) Hinton, G. E. and Van Camp, D. (1993).  Keeping the neural networks simple by minimizing the description length of the weights.  In Conference on Computational Learning Theory.
  * Hochreiter and Schmidhuber, (1997) Hochreiter, S. and Schmidhuber, J. (1997).  Long short-term memory.  Neural Computation, 9(8):1735–1780.
  * Hornik et al., (1989) Hornik, K., Stinchcombe, M., and White, H. (1989).  Multilayer feedforward networks are universal approximators.  Neural Networks, 2(5):359–366.
  * (101) Hron, J., Bahri, Y., Novak, R., Pennington, J., and Sohl-Dickstein, J. (2020a).  Exact posterior distributions of wide Bayesian neural networks.  ICML Workshop on Uncertainty and Robustness in Deep Learning.
  * (102) Hron, J., Bahri, Y., Sohl-Dickstein, J., and Novak, R. (2020b).  Infinite attention: NNGP and NTK for deep attention networks.  In International Conference on Machine Learning.
  * Hron et al., (2018) Hron, J., Matthews, A., and Ghahramani, Z. (2018).  Variational Bayesian dropout: pitfalls and fixes.  In International Conference on Machine Learning.
  * Huang et al., (2017) Huang, G., Li, Y., Pleiss, G., Liu, Z., Hopcroft, J. E., and Weinberger, K. Q. (2017).  Snapshot ensembles: Train 1, get M for free.  International Conference on Learning Representations.
  * Huang and Yau, (2020) Huang, J. and Yau, H.-T. (2020).  Dynamics of deep neural networks and neural tangent hierarchy.  In International Conference on Machine Learning.
  * Huang et al., (2023) Huang, Y., Chouzenoux, E., Elvira, V., and Pesquet, J.-C. (2023).  Efficient bayes inference in neural networks through adaptive importance sampling.  Journal of the Franklin Institute.
  * (107) Immer, A., Bauer, M., Fortuin, V., Rätsch, G., and Emtiyaz, K. M. (2021a).  Scalable marginal likelihood estimation for model selection in deep learning.  In International Conference on Machine Learning.
  * (108) Immer, A., Korzepa, M., and Bauer, M. (2021b).  Improving predictions of bayesian neural nets via local linearization.  In International Conference on Artificial Intelligence and Statistics.
  * Immer et al., (2022) Immer, A., van der Ouderaa, T. F., Fortuin, V., Rätsch, G., and van der Wilk, M. (2022).  Invariance learning in deep neural networks with differentiable laplace approximations.  arXiv preprint arXiv:2202.10638.
  * Ioffe and Szegedy, (2015) Ioffe, S. and Szegedy, C. (2015).  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In International Conference on Machine Learning.
  * (111) Izmailov, P., Nicholson, P., Lotfi, S., and Wilson, A. G. (2021a).  Dangers of Bayesian model averaging under covariate shift.  Advances in Neural Information Processing Systems.
  * Izmailov et al., (2019) Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., and Wilson, A. G. (2019).  Averaging weights leads to wider optima and better generalization.  Uncertainty in Artificial Intelligence.
  * (113) Izmailov, P., Vikram, S., Hoffman, M. D., and Wilson, A. G. (2021b).  What are Bayesian neural network posteriors really like?  In International Conference on Machine Learning.
  * Jacot et al., (2018) Jacot, A., Gabriel, F., and Hongler, C. (2018).  Neural tangent kernel: Convergence and generalization in neural networks.  In Advances in Neural Information Processing Systems.
  * Jordan et al., (1999) Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. (1999).  An introduction to variational methods for graphical models.  Machine Learning, 37(2):183–233.
  * Jospin et al., (2022) Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., and Bennamoun, M. (2022).  Hands-on bayesian neural networks—a tutorial for deep learning users.  IEEE Computational Intelligence Magazine, 17(2):29–48.
  * Kääriäinen and Langford, (2005) Kääriäinen, M. and Langford, J. (2005).  A comparison of tight generalization error bounds.  In International Conference on Machine Learning.
  * Kapoor et al., (2022) Kapoor, S., Maddox, W. J., Izmailov, P., and Wilson, A. G. (2022).  On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification.  Advances in Neural Information Processing Systems.
  * Kass and Raftery, (1995) Kass, R. E. and Raftery, A. E. (1995).  Bayes factors.  Journal of the American Statistical Association, 90(430):773–795.
  * Khan et al., (2018) Khan, M., Nielsen, D., Tangkaratt, V., Lin, W., Gal, Y., and Srivastava, A. (2018).  Fast and scalable Bayesian deep learning by weight-perturbation in ADAM.  In International Conference on Machine Learning.
  * Khan et al., (2017) Khan, M. E., Liu, Z., Tangkaratt, V., and Gal, Y. (2017).  Vprop: Variational inference using rmsprop.  arXiv preprint arXiv:1712.01038.
  * Khan and Rue, (2021) Khan, M. E. and Rue, H. (2021).  The Bayesian learning rule.  arXiv preprint arXiv:2107.04562.
  * Khan and Swaroop, (2021) Khan, M. E. and Swaroop, S. (2021).  Knowledge-adaptation priors.  Advances in Neural Information Processing Systems.
  * Khan et al., (2019) Khan, M. E. E., Immer, A., Abedi, E., and Korzepa, M. (2019).  Approximate inference turns deep networks into Gaussian processes.  Advances in neural information processing systems, 32.
  * Kingma et al., (2015) Kingma, D. P., Salimans, T., and Welling, M. (2015).  Variational dropout and the local reparameterization trick.  In Advances in Neural Information Processing Systems.
  * Kingma and Welling, (2014) Kingma, D. P. and Welling, M. (2014).  Auto-encoding variational Bayes.  International Conference on Learning Representations.
  * Koh and Liang, (2017) Koh, P. W. and Liang, P. (2017).  Understanding black-box predictions via influence functions.  In International Conference on Machine Learning.
  * Kohler and Langer, (2021) Kohler, M. and Langer, S. (2021).  On the rate of convergence of fully connected deep neural network regression estimates.  The Annals of Statistics, 49(4):2231–2249.
  * Korattikara Balan et al., (2015) Korattikara Balan, A., Rathod, V., Murphy, K. P., and Welling, M. (2015).  Bayesian dark knowledge.  Advances in Neural Information Processing Systems.
  * Kristiadi et al., (2020) Kristiadi, A., Hein, M., and Hennig, P. (2020).  Being Bayesian, even just a bit, fixes overconfidence in ReLU networks.  In International Conference on Machine Learning, pages 5436–5446.
  * Krizhevsky et al., (2012) Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012).  ImageNet classification with deep convolutional neural networks.  In Advances in Neural Information Processing Systems.
  * Lakshminarayanan et al., (2017) Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017).  Simple and scalable predictive uncertainty estimation using deep ensembles.  Advances in Neural Information Processing Systems.
  * Langford, (2005) Langford, J. (2005).  Tutorial on practical prediction theory for classification.  Journal of Machine Learning Research, 6(Mar):273–306.
  * LeCun et al., (1989) LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. (1989).  Backpropagation applied to handwritten zip code recognition.  Neural Computation, 1(4):541–551.
  * Lee et al., (2020) Lee, J., Schoenholz, S., Pennington, J., Adlam, B., Xiao, L., Novak, R., and Sohl-Dickstein, J. (2020).  Finite versus infinite neural networks: an empirical study.  In International Conference on Neural Information Processing Systems.
  * (136) Lee, J., Sohl-Dickstein, J., Pennington, J., Novak, R., Schoenholz, S., and Bahri, Y. (2018a).  Deep neural networks as Gaussian processes.  In International Conference on Learning Representations.
  * Lee et al., (2019) Lee, J., Xiao, L., Schoenholz, S., Bahri, Y., Novak, R., Sohl-Dickstein, J., and Pennington, J. (2019).  Wide neural networks of any depth evolve as linear models under gradient descent.  Advances in Neural Information Processing Systems, 32:8572–8583.
  * (138) Lee, K., Lee, H., Lee, K., and Shin, J. (2018b).  Training confidence-calibrated classifiers for detecting out-of-distribution samples.  International Conference on Learning Representations.
  * Li et al., (2016) Li, C., Chen, C., Carlson, D., and Carin, L. (2016).  Preconditioned stochastic gradient langevin dynamics for deep neural networks.  In Conference on Artificial Intelligence.
  * Li et al., (2021) Li, Y., Yu, Q., Tan, M., Mei, J., Tang, P., Shen, W., Yuille, A., and Xie, C. (2021).  Shape-texture debiased neural network training.  International Conference on Learning Representations.
  * Lim et al., (2021) Lim, B., Arık, S. Ö., Loeff, N., and Pfister, T. (2021).  Temporal fusion transformers for interpretable multi-horizon time series forecasting.  International Journal of Forecasting, 37(4):1748–1764.
  * Lipton, (2018) Lipton, Z. C. (2018).  The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.  Queue, 16(3):31–57.
  * Liu et al., (2019) Liu, X., Li, Y., Wu, C., and Hsieh, C.-J. (2019).  Adv-BNN: Improved adversarial defense through robust Bayesian neural network.  International Conference on Learning Representations.
  * Lloyd et al., (2014) Lloyd, J., Duvenaud, D., Grosse, R., Tenenbaum, J., and Ghahramani, Z. (2014).  Automatic construction and natural-language description of nonparametric regression models.  In International Conference on Artificial Intelligence and Statistics.
  * Lotfi et al., (2022) Lotfi, S., Izmailov, P., Benton, G., Goldblum, M., and Wilson, A. G. (2022).  Bayesian model selection, the marginal likelihood, and generalization.  arXiv preprint arXiv:2202.11678.
  * Louizos and Welling, (2016) Louizos, C. and Welling, M. (2016).  Structured and efficient variational deep learning with matrix Gaussian posteriors.  In International Conference on Machine Learning.
  * Louizos and Welling, (2017) Louizos, C. and Welling, M. (2017).  Multiplicative normalizing flows for variational Bayesian neural networks.  In International Conference on Machine Learning.
  * Lu et al., (2020) Lu, L., Shin, Y., Su, Y., and Karniadakis, G. E. (2020).  Dying ReLU and initialization: Theory and numerical examples.  Communications in Computational Physics, 28(5):1671–1706.
  * Lyle et al., (2020) Lyle, C., Schut, L., Ru, R., Gal, Y., and van der Wilk, M. (2020).  A Bayesian perspective on training speed and model selection.  Advances in Neural Information Processing Systems.
  * Ma et al., (2015) Ma, Y.-A., Chen, T., and Fox, E. (2015).  A complete recipe for stochastic gradient mcmc.  Advances in Neural Information Processing Systems, 28.
  * MacKay, (1992) MacKay, D. J. (1992).  A practical Bayesian framework for backpropagation networks.  Neural Computation, 4(3):448–472.
  * MacKay, (2003) MacKay, D. J. (2003).  Information theory, inference and learning algorithms.  Cambridge university press.
  * Maddox et al., (2019) Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., and Wilson, A. G. (2019).  A simple baseline for Bayesian uncertainty in deep learning.  Advances in Neural Information Processing Systems.
  * Mahoney and Martin, (2019) Mahoney, M. and Martin, C. (2019).  Traditional and heavy-tailed self-regularization in neural network models.  In International Conference on Machine Learning.
  * Malach et al., (2020) Malach, E., Yehudai, G., Shalev-Schwartz, S., and Shamir, O. (2020).  Proving the lottery ticket hypothesis: Pruning is all you need.  In International Conference on Machine Learning.
  * Mandt et al., (2017) Mandt, S., Hoffman, M. D., and Blei, D. M. (2017).  Stochastic gradient descent as approximate Bayesian inference.  Journal of Machine Learning Research, 18:1–35.
  * Martens and Grosse, (2015) Martens, J. and Grosse, R. (2015).  Optimizing neural networks with kronecker-factored approximate curvature.  In International Conference on Machine Learning.
  * Matsubara et al., (2021) Matsubara, T., Oates, C. J., and Briol, F.-X. (2021).  The Ridgelet prior: A covariance function approach to prior specification for Bayesian neural networks.  Journal of Machine Learning Research, 22:1–57.
  * Matthews et al., (2018) Matthews, A. G. d. G., Rowland, M., Hron, J., Turner, R. E., and Ghahramani, Z. (2018).  Gaussian process behaviour in wide deep neural networks.  In International Conference on Learning Representations.
  * McAllester, (1999) McAllester, D. A. (1999).  Some PAC-Bayesian theorems.  Machine Learning, 37(3):355–363.
  * Mellor et al., (2021) Mellor, J., Turner, J., Storkey, A., and Crowley, E. J. (2021).  Neural architecture search without training.  In International Conference on Machine Learning.
  * Mianjy et al., (2018) Mianjy, P., Arora, R., and Vidal, R. (2018).  On the implicit bias of dropout.  In International Conference on Machine Learning.
  * Mikkola et al., (2023) Mikkola, P., Martin, O. A., Chandramouli, S., Hartmann, M., Pla, O. A., Thomas, O., Pesonen, H., Corander, J., Vehtari, A., Kaski, S., Bürkner, P.-C., and Klami, A. (2023).  Prior knowledge elicitation: The past, present, and future.  Bayesian Analysis.
  * Minderer et al., (2021) Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D., and Lucic, M. (2021).  Revisiting the calibration of modern neural networks.  Advances in Neural Information Processing Systems.
  * Mishkin et al., (2018) Mishkin, A., Kunstner, F., Nielsen, D., Schmidt, M., and Khan, M. E. (2018).  Slang: Fast structured covariance approximations for Bayesian deep learning with natural gradient.  Advances in Neural Information Processing Systems.
  * Mitros and Mac Namee, (2019) Mitros, J. and Mac Namee, B. (2019).  On the validity of Bayesian neural networks for uncertainty estimation.  arXiv preprint arXiv:1912.01530.
  * Modas et al., (2021) Modas, A., Rade, R., Ortiz-Jiménez, G., Moosavi-Dezfooli, S.-M., and Frossard, P. (2021).  PRIME: A few primitives can boost robustness to common corruptions.  arXiv preprint arXiv:2112.13547.
  * Mohri et al., (2018) Mohri, M., Rostamizadeh, A., and Talwalkar, A. (2018).  Foundations of machine learning.  MIT press.
  * Moins et al., (2023) Moins, T., Arbel, J., Dutfoy, A., and Girard, S. (2023).  On the use of a local $\hat{R}$ to improve MCMC convergence diagnostic.  Bayesian Analysis.
  * Molchanov et al., (2017) Molchanov, D., Ashukha, A., and Vetrov, D. (2017).  Variational dropout sparsifies deep neural networks.  In International Conference on Machine Learning.
  * Möllenhoff and Khan, (2022) Möllenhoff, T. and Khan, M. E. (2022).  Sam as an optimal relaxation of bayes.  In International Conference on Learning Representations.
  * Montavon et al., (2018) Montavon, G., Samek, W., and Müller, K.-R. (2018).  Methods for interpreting and understanding deep neural networks.  Digital Signal Processing, 73:1–15.
  * Moosavi-Dezfooli et al., (2017) Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., and Frossard, P. (2017).  Universal adversarial perturbations.  In Conference on Computer Vision and Pattern Recognition.
  * Moosavi-Dezfooli et al., (2016) Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. (2016).  Deepfool: a simple and accurate method to fool deep neural networks.  In Conference on Computer Vision and Pattern Recognition.
  * Morrison et al., (2021) Morrison, K., Gilby, B., Lipchak, C., Mattioli, A., and Kovashka, A. (2021).  Exploring corruption robustness: Inductive biases in vision transformers and MLP-mixers.  arXiv preprint arXiv:2106.13122.
  * Murphy, (2012) Murphy, K. P. (2012).  Machine learning: a probabilistic perspective.  MIT press.
  * Nabarro et al., (2021) Nabarro, S., Ganev, S., Garriga-Alonso, A., Fortuin, V., van der Wilk, M., and Aitchison, L. (2021).  Data augmentation in Bayesian neural networks and the cold posterior effect.  arXiv preprint arXiv:2106.05586.
  * Naeini et al., (2015) Naeini, M. P., Cooper, G., and Hauskrecht, M. (2015).  Obtaining well calibrated probabilities using Bayesian binning.  In Conference on Artificial Intelligence.
  * Nakkiran et al., (2021) Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., and Sutskever, I. (2021).  Deep double descent: Where bigger models and more data hurt.  Journal of Statistical Mechanics: Theory and Experiment, 2021(12):124003.
  * Nalisnick et al., (2021) Nalisnick, E., Gordon, J., and Hernández-Lobato, J. M. (2021).  Predictive complexity priors.  In International Conference on Artificial Intelligence and Statistics.
  * Nalisnick et al., (2019) Nalisnick, E., Hernández-Lobato, J. M., and Smyth, P. (2019).  Dropout as a structured shrinkage prior.  In International Conference on Machine Learning.
  * Nalisnick, (2018) Nalisnick, E. T. (2018).  On priors for Bayesian neural networks.  PhD thesis, University of California, Irvine.
  * Naveh et al., (2020) Naveh, G., Ben-David, O., Sompolinsky, H., and Ringel, Z. (2020).  Predicting the outputs of finite networks trained with noisy gradients.  arXiv preprint arXiv:2004.01190.
  * Neal, (1992) Neal, R. M. (1992).  Bayesian training of backpropagation networks by the hybrid Monte Carlo method.  Technical report, Citeseer.
  * Neal, (1993) Neal, R. M. (1993).  Bayesian learning via stochastic dynamics.  In Advances in Neural Information Processing Systems.
  * Neal, (1996) Neal, R. M. (1996).  Bayesian learning for neural networks.  Springer Science & Business Media.
  * Neelakantan et al., (2016) Neelakantan, A., Vilnis, L., Le, Q. V., Sutskever, I., Kaiser, L., Kurach, K., and Martens, J. (2016).  Adding gradient noise improves learning for very deep networks.  International Conference on Learning Representations Workshop.
  * Neiswanger et al., (2014) Neiswanger, W., Wang, C., and Xing, E. P. (2014).  Asymptotically exact, embarrassingly parallel mcmc.  In Conference on Uncertainty in Artificial Intelligence.
  * Neyshabur et al., (2017) Neyshabur, B., Bhojanapalli, S., McAllester, D., and Srebro, N. (2017).  A pac-bayesian approach to spectrally-normalized margin bounds for neural networks.  arXiv preprint arXiv:1707.09564.
  * Nixon et al., (2019) Nixon, J., Dusenberry, M. W., Zhang, L., Jerfel, G., and Tran, D. (2019).  Measuring calibration in deep learning.  In CVPR Workshop.
  * Noci et al., (2021) Noci, L., Bachmann, G., Roth, K., Nowozin, S., and Hofmann, T. (2021).  Precise characterization of the prior predictive distribution of deep ReLU networks.  International Conference on Learning Representations.
  * Novak et al., (2020) Novak, R., Xiao, L., Bahri, Y., Lee, J., Yang, G., Hron, J., Abolafia, D. A., Pennington, J., and Sohl-dickstein, J. (2020).  Bayesian deep convolutional networks with many channels are Gaussian processes.  In ICML Workshop on Uncertainty and Robustness in Deep Learning.
  * Osawa et al., (2019) Osawa, K., Swaroop, S., Khan, M. E. E., Jain, A., Eschenhagen, R., Turner, R. E., and Yokota, R. (2019).  Practical deep learning with Bayesian principles.  Advances in Neural Information Processing Systems.
  * Osband et al., (2022) Osband, I., Wen, Z., Asghari, S. M., Dwaracherla, V., Lu, X., Ibrahimi, M., Lawson, D., Hao, B., O’Donoghue, B., and Van Roy, B. (2022).  The neural testbed: Evaluating joint predictions.  Advances in Neural Information Processing Systems, 35:12554–12565.
  * Ovadia et al., (2019) Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., and Snoek, J. (2019).  Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift.  Advances in Neural Information Processing Systems.
  * Pearce et al., (2020) Pearce, T., Leibfried, F., and Brintrup, A. (2020).  Uncertainty in neural networks: Approximately Bayesian ensembling.  In International Conference on Artificial Intelligence and Statistics.
  * Pitas and Arbel, (2022) Pitas, K. and Arbel, J. (2022).  Cold Posteriors through PAC-Bayes.  arXiv preprint arXiv:2206.11173.
  * Platt, (1999) Platt, J. (1999).  Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods.  Advances in Large Margin Classifiers.
  * Polson and Rockova, (2018) Polson, N. and Rockova, V. (2018).  Posterior concentration for sparse deep learning.  In Advances in Neural Information Processing Systems.
  * Poole et al., (2016) Poole, B., Lahiri, S., Raghu, M., Sohl-Dickstein, J., and Ganguli, S. (2016).  Exponential expressivity in deep neural networks through transient chaos.  In International Conference on Neural Information Processing Systems.
  * Rasmussen and Williams, (2006) Rasmussen, C. E. and Williams, C. K. (2006).  Gaussian processes for machine learning.  MIT press Cambridge, MA.
  * Rawat et al., (2017) Rawat, A., Wistuba, M., and Nicolae, M.-I. (2017).  Adversarial phenomenon in the eyes of Bayesian deep learning.  arXiv preprint arXiv:1711.08244.
  * Ritter et al., (2018) Ritter, H., Botev, A., and Barber, D. (2018).  A scalable Laplace approximation for neural networks.  In International Conference on Learning Representations.
  * Robbins and Monro, (1951) Robbins, H. and Monro, S. (1951).  A stochastic approximation method.  The Annals of Mathematical Statistics, pages 400–407.
  * Robert and Casella, (2004) Robert, C. P. and Casella, G. (2004).  Monte Carlo statistical methods.  Springer-Verlag, New York.
  * Rosenblatt, (1958) Rosenblatt, F. (1958).  The perceptron: a probabilistic model for information storage and organization in the brain.  Psychological review, 65(6):386.
  * Rothfuss et al., (2021) Rothfuss, J., Fortuin, V., Josifoski, M., and Krause, A. (2021).  PACOH: Bayes-optimal meta-learning with PAC-guarantees.  In International Conference on Machine Learning, pages 9116–9126. PMLR.
  * Rothfuss et al., (2022) Rothfuss, J., Josifoski, M., Fortuin, V., and Krause, A. (2022).  PAC-Bayesian meta-learning: From theory to practice.  arXiv preprint arXiv:2211.07206.
  * Ru et al., (2021) Ru, R., Lyle, C., Schut, L., Fil, M., van der Wilk, M., and Gal, Y. (2021).  Speedy performance estimation for neural architecture search.  Advances in Neural Information Processing Systems.
  * Rumelhart et al., (1986) Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986).  Learning representations by back-propagating errors.  Nature, 323(6088):533–536.
  * Salvatier et al., (2016) Salvatier, J., Wiecki, T. V., and Fonnesbeck, C. (2016).  Probabilistic programming in python using PyMC3.  PeerJ Computer Science, 2:e55.
  * Sarker, (2021) Sarker, I. H. (2021).  Deep learning: a comprehensive overview on techniques, taxonomy, applications and research directions.  SN Computer Science, 2(6):1–20.
  * Savage, (1972) Savage, L. J. (1972).  The foundations of statistics.  Courier Corporation.
  * Schmidt-Hieber, (2020) Schmidt-Hieber, J. (2020).  Nonparametric regression using deep neural networks with ReLU activation function.  Annals of Statistics, 48(4).
  * Schoenholz et al., (2017) Schoenholz, S. S., Gilmer, J., Ganguli, S., and Sohl-Dickstein, J. (2017).  Deep information propagation.  In International Conference on Learning Representations.
  * Selvaraju et al., (2017) Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017).  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In International Conference on Computer Vision.
  * Silver et al., (2016) Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., and Lanctot, M. (2016).  Mastering the game of Go with deep neural networks and tree search.  Nature, 529(7587):484–489.
  * Sokolić et al., (2016) Sokolić, J., Giryes, R., Sapiro, G., and Rodrigues, M. R. (2016).  Robust large margin deep neural networks.  IEEE Transactions on Signal Processing.
  * Soudry et al., (2018) Soudry, D., Hoffer, E., Nacson, M. S., Gunasekar, S., and Srebro, N. (2018).  The implicit bias of gradient descent on separable data.  Journal of Machine Learning Research, 19(1):2822–2878.
  * Springer and Thompson, (1970) Springer, M. D. and Thompson, W. E. (1970).  The distribution of products of beta, gamma and Gaussian random variables.  SIAM Journal on Applied Mathematics, 18(4):721–737.
  * Srivastava et al., (2014) Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014).  Dropout: A simple way to prevent neural networks from overfitting.  Journal of Machine Learning Research, 15(1):1929–1958.
  * (222) Sun, S., Cao, Z., Zhu, H., and Zhao, J. (2019a).  A survey of optimization methods from a machine learning perspective.  IEEE Transactions on Cybernetics, 50(8):3668–3681.
  * Sun et al., (2017) Sun, S., Chen, C., and Carin, L. (2017).  Learning structured weight uncertainty in Bayesian neural networks.  In International Conference on Artificial Intelligence and Statistics.
  * (224) Sun, S., Zhang, G., Shi, J., and Grosse, R. (2019b).  Functional variational Bayesian neural networks.  International Conference on Learning Representations.
  * Sun et al., (2018) Sun, S., Zhang, G., Wang, C., Zeng, W., Li, J., and Grosse, R. (2018).  Differentiable compositional kernel learning for Gaussian processes.  In International Conference on Machine Learning.
  * Sundararajan et al., (2017) Sundararajan, M., Taly, A., and Yan, Q. (2017).  Axiomatic attribution for deep networks.  In International Conference on Machine Learning.
  * Suzuki, (2018) Suzuki, T. (2018).  Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality.  In International Conference on Learning Representations.
  * Szabó et al., (2015) Szabó, B., van der Vaart, A. W., and van Zanten, J. (2015).  Frequentist coverage of adaptive nonparametric Bayesian credible sets.  The Annals of Statistics, 43(4):1391–1428.
  * Telgarsky, (2016) Telgarsky, M. (2016).  Benefits of depth in neural networks.  In Conference on Learning Theory.
  * Tibshirani, (1996) Tibshirani, R. (1996).  Regression shrinkage and selection via the Lasso.  Journal of the Royal Statistical Society. Series B (Methodological), pages 267–288.
  * Tran et al., (2020) Tran, B.-H., Rossi, S., Milios, D., and Filippone, M. (2020).  All you need is a good functional prior for Bayesian deep learning.  arXiv preprint arXiv:2011.12829.
  * Ulyanov et al., (2018) Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2018).  Deep image prior.  In Computer Vision and Pattern Recognition.
  * Vaicenavicius et al., (2019) Vaicenavicius, J., Widmann, D., Andersson, C., Lindsten, F., Roll, J., and Schön, T. (2019).  Evaluating model calibration in classification.  In International Conference on Artificial Intelligence and Statistics.
  * Vapnik, (1999) Vapnik, V. N. (1999).  An overview of statistical learning theory.  IEEE Transactions on Neural Networks, 10(5):988–999.
  * Vaswani et al., (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017).  Attention is all you need.  In Advances in Neural Information Processing Systems.
  * Vehtari et al., (2021) Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., and Bürkner, P.-C. (2021).  Rank-normalization, folding, and localization: An improved $\widehat{R}$ for assessing convergence of MCMC (with discussion).  Bayesian Analysis, 16(2):667–718.
  * (237) Vladimirova, M., Arbel, J., and Girard, S. (2021a).  Bayesian neural network unit priors and generalized Weibull-tail property.  In Asian Conference on Machine Learning.
  * (238) Vladimirova, M., Arbel, J., and Girard, S. (2021b).  Dependence between Bayesian neural network units.  In NeurIPS Workshop on Bayesian Deep Learning.
  * Vladimirova et al., (2020) Vladimirova, M., Girard, S., Nguyen, H., and Arbel, J. (2020).  Sub-Weibull distributions: Generalizing sub-Gaussian and sub-exponential properties to heavier tailed distributions.  Stat, 9(1):e318.
  * Vladimirova et al., (2019) Vladimirova, M., Verbeek, J., Mesejo, P., and Arbel, J. (2019).  Understanding priors in Bayesian neural networks at the unit level.  In International Conference on Machine Learning.
  * Wager et al., (2013) Wager, S., Wang, S., and Liang, P. S. (2013).  Dropout training as adaptive regularization.  Advances in Neural Information Processing Systems.
  * Wang et al., (2015) Wang, Y.-X., Fienberg, S., and Smola, A. (2015).  Privacy for free: Posterior sampling and stochastic gradient monte carlo.  In International Conference on Machine Learning.
  * Weiss et al., (2016) Weiss, K., Khoshgoftaar, T. M., and Wang, D. (2016).  A survey of transfer learning.  Journal of Big Data, 3(1):1–40.
  * Welling and Teh, (2011) Welling, M. and Teh, Y. W. (2011).  Bayesian learning via stochastic gradient Langevin dynamics.  In International Conference on Machine Learning.
  * Wen et al., (2018) Wen, Y., Vicol, P., Ba, J., Tran, D., and Grosse, R. (2018).  Flipout: Efficient pseudo-independent weight perturbations on mini-batches.  arXiv preprint arXiv:1803.04386.
  * Wenzel et al., (2020) Wenzel, F., Roth, K., Veeling, B., Swiatkowski, J., Tran, L., Mandt, S., Snoek, J., Salimans, T., Jenatton, R., and Nowozin, S. (2020).  How good is the Bayes posterior in deep neural networks really?  In International Conference on Machine Learning.
  * Wilson, (2020) Wilson, A. G. (2020).  The case for Bayesian deep learning.  arXiv preprint arXiv:2001.10995.
  * Wilson et al., (2016) Wilson, A. G., Hu, Z., Salakhutdinov, R., and Xing, E. P. (2016).  Deep kernel learning.  In International Conference on Artificial Intelligence and Statistics.
  * Wilson and Izmailov, (2020) Wilson, A. G. and Izmailov, P. (2020).  Bayesian deep learning and a probabilistic perspective of generalization.  International Conference on Neural Information Processing Systems.
  * Wolinski and Arbel, (2023) Wolinski, P. and Arbel, J. (2023).  Gaussian Pre-Activations in Neural Networks: Myth or Reality?  Arxiv Preprint, Submitted.
  * Wolinski et al., (2020) Wolinski, P., Charpiat, G., and Ollivier, Y. (2020).  Interpreting a penalty as the influence of a Bayesian prior.  arXiv preprint arXiv:2002.00178.
  * Wolpert, (1996) Wolpert, D. H. (1996).  The lack of a priori distinctions between learning algorithms.  Neural Computation, 8(7):1341–1390.
  * Yaida, (2020) Yaida, S. (2020).  Non-Gaussian processes and neural networks at finite widths.  In Mathematical and Scientific Machine Learning.
  * (254) Yang, G. (2019a).  Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation.  arXiv preprint arXiv:1902.04760.
  * (255) Yang, G. (2019b).  Tensor programs i: Wide feedforward or recurrent neural networks of any architecture are gaussian processes.  Advances in Neural Information Processing Systems.
  * Zadrozny and Elkan, (2001) Zadrozny, B. and Elkan, C. (2001).  Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers.  In International Conference on Machine Learning.
  * Zadrozny and Elkan, (2002) Zadrozny, B. and Elkan, C. (2002).  Transforming classifier scores into accurate multiclass probability estimates.  In International Conference on Knowledge Discovery and Data Mining.
  * Zavatone-Veth et al., (2021) Zavatone-Veth, J. A., Canatar, A., and Pehlevan, C. (2021).  Asymptotics of representation learning in finite Bayesian neural networks.  arXiv preprint arXiv:2106.00651.
  * Zavatone-Veth and Pehlevan, (2021) Zavatone-Veth, J. A. and Pehlevan, C. (2021).  Exact priors of finite neural networks.  International Conference on Learning Representations.
  * (260) Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2017a).  Understanding deep learning requires rethinking generalization.  International Conference on Learning Representations.
  * (261) Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2017b).  Understanding deep learning requires rethinking generalization.  International Conference on Learning Representations.
  * Zhang et al., (2021) Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2021).  Understanding deep learning (still) requires rethinking generalization.  Communications of the ACM, 64(3):107–115.
  * Zhang et al., (2018) Zhang, G., Sun, S., Duvenaud, D., and Grosse, R. (2018).  Noisy natural gradient as variational inference.  In International Conference on Machine Learning.
  * Zhang et al., (2019) Zhang, R., Li, C., Zhang, J., Chen, C., and Wilson, A. G. (2019).  Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning.  In International Conference on Learning Representations.