# Variational Causal Networks: Approximate Bayesian Inference over Causal
Structures

Yashas Annadani 1, Jonas Rothfuss 1, Alexandre Lacoste 3, Nino Scherrer 1,
Anirudh Goyal 2, Yoshua Bengio 2, Stefan Bauer 4,5
1 ETH Zurich, 2 Mila, Université de Montréal, 3 ElementAI/ ServiceNow, 4 MPI
for Intelligent Systems 5 CIFAR Azrieli Global Scholar Work done during an
internship at Mila.

###### Abstract

Learning the causal structure that underlies data is a crucial step towards
robust real-world decision making. The majority of existing work in causal
inference focuses on determining a single _directed acyclic graph (DAG)_ or a
Markov equivalence class thereof. However, a crucial aspect to acting
intelligently upon the knowledge about causal structure which has been
inferred from finite data demands reasoning about its uncertainty. For
instance, planning _interventions_ to find out more about the causal
mechanisms that govern our data requires quantifying _epistemic uncertainty_
over DAGs. While Bayesian causal inference allows to do so, the posterior over
DAGs becomes intractable even for a small number of variables. Aiming to
overcome this issue, we propose a form of variational inference over the
graphs of Structural Causal Models (SCMs). To this end, we introduce a
parametric variational family modelled by an autoregressive distribution over
the space of discrete DAGs. Its number of parameters does not grow
exponentially with the number of variables and can be tractably learned by
maximising an Evidence Lower Bound (ELBO). In our experiments, we demonstrate
that the proposed variational posterior is able to provide a good
approximation of the true posterior.

Figure 1: Schematic Diagram of Variational Inference on SCMs with VCN.

Moving from learning correlation and association in data to causation is a
critical step towards increased robustness, interpretability and real-world
decision-making [27, 34]. Doing so entails learning the causal structure
underlying the data generating process. Causal inference is concerned with
determining the causal structure of a set of random variables from data,
commonly represented as a directed acycilc graph (DAG) [29]. While Structural
Causal Models (SCMs) provide a generative model over the data, they are hard
to learn from data due to the non-identifiability of the causal models without
interventional data [7] and the combinatorial nature of the space of DAGs
[15]. Even with infinite amount of data, recovering the causal structure is
intrinsically hard since a DAG is only identifiable up to its _Markov
equivalence class_ (MEC) and the space of possible DAGs grows super-
exponentially with the number of variables. While the majority of work on
causal inference [6, 5, 38] deals with getting a single underlying causal
structure without a probabilistic treatment, quantifying the epistemic
uncertainty in case of non-identifiability is crucial and is not possible in
these approaches.

In this work, we take a Bayesian approach to causal structure learning. Given
only finite observational data, a Bayesian approach allows us to quantify the
uncertainty in the causal structure of the data generating process, even
before performing interventions. Having such a framework over causal
structures can help further downstream tasks on graph learning and causal
inference. For example, we can leverage the model’s uncertainty to select
informative interventions and discover the full graph with minimal amount of
interventions [2]. Also, for estimating the causal effect of certain
variables, it is often not required to know the full graph. Hence, by
marginalising the posterior we can uncover the confidence we have about a
specific causal effect, which may already fall below a specified tolerance
level. Having such a model is very desirable as interventions are hard to
perform, sometimes unethical and even impossible [29].

A key component in learning causal structures is the identifiability under the
availability of observational data. While some assumptions are always required
to say anything about the underlying causal process, additional assumptions
are sometimes made to especially make the causal model identifiable from
(observational) data. These additional assumptions are not necessarily part of
the data generating process, and hence the recovered causal structure may be
incorrect due to model misspecification. In addition, identifiability results
are usually asymptotic in the number of samples. Causal discovery in a limited
sample regime calls for active learning to perform interventions to improve
identifiability. Such setups benefit from probabilistic reasoning about
unknown causal structures. We are interested in such settings and we propose a
Bayesian framework for linear SCM’s with additive noise which can quantify the
uncertainty in the learned causal structure.

Our contributions are as follows:

  * •

We perform Bayesian inference over the unknown causal structures. The
posterior over this distribution is intractable. Therefore, we perform
variational inference and demonstrate how we can model the variational family
over this distribution.

  * •

The key contribution is to model distributions over adjacency matrices of the
causal structures with an autoregressive distribution using an LSTM.

  * •

Empirically demonstrate the performance of our modeling choice.

  * •

Evaluating Bayesian causal inference techniques are hard in practice. We
discuss the difficulty entailed in evaluation, as well as provide insights
which alleviate this problem.

##  1 Problem Setting

##### Causal modeling

Consider a set of $d$ random variables $\mathbf{X}:=\\{X_{1},\dots,X_{d}\\}$.
A Structural Causal Model (SCM) [29] over $\mathbf{X}$ is defined as set of
structural assignments

$X_{i}:=f_{i}(\mathbf{x}_{\pi_{\mathcal{G}}(i)},\epsilon_{i})~{},i=1,...,d$
(1)

corresponding to a Directed Acyclic Graph (DAG) $\mathcal{G}$ with vertices
$\mathbf{X}$. In here, $\pi_{\mathcal{G}}(i)$ are the parents of $X_{i}$ in
$\mathcal{G}$, $\epsilon_{i}$ are independent noise variables with probability
density $P_{\epsilon_{i}}$ and the $f_{i}$’s are (potentially non-linear)
functions. The SCM entails a joint probability distribution $P_{\mathbf{x}}$
over the random variables. In this work, we assume that all the endogenous
variables are observed, that is, there are no hidden confounders.

A popular instantiation of this generic framework are linear SCMs with
additive noise, given by:

$x_{i}:=\bm{\theta}_{i}^{T}\mathbf{x}_{\pi_{\mathcal{G}}(i)}+\epsilon_{i}~{}~{}~{}~{}\bm{\theta}_{i}\in\mathbb{R}^{|\pi_{\mathcal{G}}(i)|}$
(2)

wherein $\bm{\theta}_{i}$ are parameters (edge weights) of the linear
functions $f_{i}$. Alternatively, this can be written as

$x_{i}:=\bm{\theta}_{i}^{T}\left(\mathbf{X}\circ\mathbf{A}_{\mathcal{G}_{i}}\right)+\epsilon_{i}~{}~{}~{}~{}\bm{\theta}_{i}~{},~{}\mathbf{A}_{\mathcal{G}_{i}}\in\mathbb{R}^{d}~{},$
(3)

where $\circ$ corresponds to elementwise product,
$\mathbf{A}_{\mathcal{G}_{i}}$ is the $i$th row of
$\mathbf{A}_{\mathcal{G}}\in\mathbb{R}^{d\times d}$, the $(0,1)$-adjacency
matrix of $\mathcal{G}$. We restrict our further exposition to continuous
random variables. However, the framework presented in the remainder of the
paper applies to discrete random variables as well. We focus on linear SCMs
with Gaussian variables, as they are non-identifiable in this setting and
hence obtaining uncertainty estimates is useful. Non-linear additive noise
SCM’s with Gaussian variables are identifiable, and hence a single point
estimate suffices.

##### NOTEARS - Causal inference as continuous optimization problem

In practice, we often have data while the causal structure that underlies the
data generating process is unknown to us. Causal discovery is concerned with
recovering this causal structure, e.g. in the form of an SCM, from
observational data or interventional data (or both). Assuming a linear SCM,
this coincides with estimating the graph $\mathcal{G}$, i.e. its adjacency
matrix $\mathbf{A}$, and its corresponding edge weights
$\bm{\theta}=\\{\bm{\theta}_{1},...,\bm{\theta}_{d}\\}$, given data
${\mathcal{D}}:=\\{\mathbf{x}_{1},...,\mathbf{x}_{n}\\}\in{\mathcal{X}}^{n}$.

In score-based causal discovery, we use a score function $F:\\{0,1\\}^{d\times
d}\times{\mathcal{X}}^{n}\mapsto\mathbb{R}$ to find the graph that best
corresponds to the data as follows:

$\operatorname*{arg\,min}_{\mathbf{A}\in\\{0,1\\}^{d\times
d}}F(\mathbf{A},{\mathcal{D}})\quad\text{s.t.}~{}~{}{\mathcal{G}}(\mathbf{A})\in\operatorname{DAG}(d)$
(4)

where $\mathbf{A}$ is the $\\{0,1\\}$ adjacency matrix of graph
${\mathcal{G}}$.

However, since the search space of DAGs of this combinatorial optimization
problem grows in the order of $\mathcal{O}(2^{d^{2}})$, solving (4) becomes
infeasible even for a relatively small number of variables $d$.

Addressing this issue, Zheng et al. [38] propose an alternative formulation
that converts the combinatorial problem into the following continuous program:

$\operatorname*{arg\,min}_{{\bm{W}}\in\mathbb{R}^{d\times
d}}F({\bm{W}},{\mathcal{D}})\quad\text{s.t.}~{}~{}\text{tr}\left(e^{{\bm{W}}}\right)-d=0~{}.$
(5)

that can be solved with standard tools of constrained optimization such as the
Lagrange method. In here, we have converted the adjacency matrix
$\mathbf{A}\in\\{0,1\\}^{d\times d}$ into a weighted adjacency matrix
${\bm{W}}\in\mathbb{R}^{d\times d}$ such that $a_{i,j}=1\Leftrightarrow
w_{i,j}\neq 0$.

##### Bayesian inference over DAGs

An important aspect towards acting intelligently upon the knowledge about the
causal structure which has been inferred from finite data demands reasoning
about its uncertainty. For instance, in scientific inquiry, planning
_interventional experiments_ to find out more about the causal mechanisms that
govern a data-generating system of interest requires quantifying _epistemic
uncertainty_ over DAGs. Bayesian inference gives us a principled framework for
the treatment of such uncertainty (Figure 1).

In the Bayesian framework, we encode our prior structural knowledge in form of
a _prior_ $p({\mathcal{G}})$ over DAGs. By combining the prior with the
_likelihood_ $p({\mathcal{D}}|{\mathcal{G}})$ through Bayes’ rule, we obtain a
_posterior distribution_ over DAGs:

$p(\mathcal{G}|{\mathcal{D}})=\frac{p({\mathcal{D}}|\mathcal{G})p(\mathcal{G})}{p({\mathcal{D}})}$
(6)

where
$p({\mathcal{D}})=\sum_{\mathcal{G}}\int_{\theta}p(\mathbf{x}|\mathcal{G},\bm{\theta})p(\bm{\theta}|\mathcal{G})p(\mathcal{G})d\bm{\theta}$
is the _model evidence_. A key issue with (6) is the intractable evidence term
in the denominator. Even if the integral over $\bm{\theta}$ can be solved
analytically such as in the case of linear structural equations with Gaussian
prior and likelihood with standard parameter priors [10], the sum over
possible DAGs grows in the order of ${\mathcal{O}}(2^{d^{2}})$, making its
computation infeasible even for a small number of variables $d$.

##  2 Variational Causal Networks

Figure 2: Schematic Diagram of Variational Causal Networks

In this section, we present our variational inference (VI) framework for
approximating Bayesian posteriors over causal structures (Equation 6). First,
we derive the evidence lower bound and sketch out how to perform VI on DAGs.
Unlike the variational inference in a latent variable model [4], the evidence
lower bound is still intractable in the case of causal models due to the
superexponential number of graph configurations. Therefore, we introduce a
novel parametric variational family which makes the intractable ELBO into a
tractable one while still having the flexibility to model complex
distributions over DAGs. Such a variational family can be modelled using
autoregressive models like LSTM [17]. This particular instantiation of
approximate Bayesian inference for SCMs with the presented variational family
is called Variational Causal Networks (VCN).

###  2.1 Variational Inference for Causal Structures

In variational inference [4], we aim to approximate the Bayesian posterior
$p(\mathcal{G}|{\mathcal{D}})$ by a variational distribution
$q_{\phi}({\mathcal{G}})$, parameterized by $\phi$, that has a tractable
density. To learn the parameters of our variational distribution, we minimize
the KL-Divergence between $q_{\phi}(\mathcal{G})$ and the true posterior
$p(\mathcal{G}|{\mathcal{D}})$ which is equivalent to maximising the _Evidence
Lower Bound (ELBO)_ , given by the following proposition.

######  Proposition 1.

(ELBO) Let $q_{\phi}(\mathcal{G})$ be the variational posterior over causal
structures. Then the evidence lower bound (ELBO) is given by:

$\begin{split}\log
p({\mathcal{D}})\geq\mathcal{L}(\phi;{\mathcal{D}})=\mathop{{}\mathbb{E}}_{q_{\phi}(\mathcal{G})}\left[\log
p({\mathcal{D}}|\mathcal{G})\right]-\operatorname{D_{KL}}\left(q_{\phi}(\mathcal{G})||p(\mathcal{G})\right)\end{split}$
(7)

To maximise the ELBO, we estimate the gradients
$\nabla_{(\phi)}\mathcal{L}(\phi;{\mathcal{D}})$ to employ a form of gradient
ascent on the objective.

###  2.2 A Variational Family for Causal Structures

A key challenge to variational inference over DAGs concerns the choice of the
variational distribution $q_{\phi}(\mathcal{G})$ over discrete DAGs
${\mathcal{G}}$. For instance, if we choose $q_{\phi}({\mathcal{G}})$ naively
as categorical distribution over the space of DAGs, the dimensionality of
$\phi$ grows super-exponentially in the number of variables $d$ (i.e.
$\mathcal{O}(2^{d^{2}})$), rendering this choice impractical. Hence, we need
to find a way of representing $q_{\phi}({\mathcal{G}})$ that is not
combinatorial in its nature while ensuring a rich family of distributions over
graphs. In addition, to be able to compute Monte Carlo gradient estimates of
the ELBO in (7), $q_{\phi}(\mathcal{G})$ needs to have a tractable probability
density.

Aiming to fulfil these requirements, we represent graphs by their adjacency
matrix $\mathbf{A}\in\\{0,1\\}^{d\times d}$ and model $q_{\phi}$ as a discrete
distribution over such $\\{0,1\\}$-matrices. While Ke et al. [19] represent
each entry of the adjacency matrix as independent Bernoulli variable, such
approach is insufficient to capture any dependencies between the entries of
the adjacency matrix which are necessary to ensure that $q(\mathbf{A})$ only
assigns positive probabilities to adjacency matrices corresponding to DAGs.
For instance, to avoid cycles of length two, $(\mathbf{A})_{i,j}$ needs to
have a strong negative dependency $(\mathbf{A})_{j,i}$, i.e., if there is a
directed edge $i\rightarrow j$ there must be no edge $j\rightarrow i$, and
reverse. In addition, a factorisable distribution can only capture unimodal
distributions while the posterior over causal structures in the non-
identifiable case could have exponential number of modes. Hence, the
variational distribution needs to be parameterised such that multiple modes
could be captured. In order to address these issues, we model this discrete
distribution as an autoregressive distribution over entries of the adjacency
matrix using an LSTM [17].

Let $q_{\phi}(\mathcal{G})=q_{\phi}(\mathbf{A}_{\mathcal{G}})$ be defined in
an autoregressive manner as follows:

$\begin{split}q_{\phi}(\mathbf{A}_{\mathcal{G}})&=\prod_{i=1}^{d(d-1)}q_{\phi}(a_{\mathcal{G}_{i}}|a_{\mathcal{G}_{1:i-1}})\\\
s.t.~{}~{}~{}~{}q_{\phi}(a_{\mathcal{G}_{i}}|a_{\mathcal{G}_{1:i-1}})&:=\text{Bernoulli}\left(a_{\mathcal{G}_{i}};f_{\phi}\left(a_{\mathcal{G}_{1:i-1}}\right)\right),~{}~{}\mathbf{A}_{\mathcal{G}}=\\{a_{\mathcal{G}_{i}}\\}_{i=1}^{d(d-1)}\end{split}$

where only the non-diagonal elements of the adjacency matrix are modelled and
$f_{\phi}$ is a function which predicts the Bernoulli parameter based on the
previous realisations of the entries of the adjacency matrix, thus making the
distribution autoregressive. We model this function using an LSTM [17].

Note that using an autoregressive distribution on the entries of the adjacency
matrix helps to implicitly keep a distribution over super-exponential number
of graphs, and also be able to sample from that distribution.

##### Prior over graph structures

Choosing the appropriate prior $p(\mathcal{G})$ over graph structures is
important to learn good approximation of the posterior. In causal discovery,
it is usually a requirement that the search space of all graphs is a DAG.
However, in the parameterisation described above, the discrete distribution is
defined over all possible graphs, including the one which contains cycles.
Having support over graphs with cycles and using maximum likelihood estimation
results in high probability mass corresponding to a fully connected graph in
the approximate posterior. Therefore, appropriate DAG regularizers are
required. We employ the result of NOTEARS [38] to define a Gibbs distribution
as the prior which helps us to limit the support of graphs to just DAGs. We
can also encode additional assumptions about the data generating process such
as sparsity. The Gibbs distribution in this case could be defined as:

$p(\mathcal{G})\propto\exp(-\lambda_{t}g(\mathbf{A}_{\mathcal{G}})-\lambda_{s}\norm{\mathbf{A}_{\mathcal{G}}}_{1})$
(8)

where $g(\mathbf{A}_{\mathcal{G}})$ is the DAG constraint given by the matrix
exponential [38], i.e
$g(\mathbf{A}_{\mathcal{G}})=\tr\left[e^{\mathbf{A}_{\mathcal{G}}}\right]-d$.
Matrix binomial can also be used to define $g$, as given in [37]. The second
term in the Gibbs distribution corresponds to the sparsity constraint.
$\lambda_{t}$ and $\lambda_{s}$ are tunable hyperparameters. Typically,
$\lambda_{t}\to\infty$ if graphs with cycles should have zero probability mass
under the posterior. However, any large value should still give a reasonable
prior with very low probabilities for cyclic graphs.

##### Marginal Likelihood

We use standard parameter priors which ensure marginal likelihood $\log
p({\mathcal{D}}|\mathcal{G})$ is ”score-equivalent”, i.e. all the DAGs which
are in the same MEC have the same marginal likelihood. This can be ensured
with a Gaussian-Wishart prior over the parameters $\bm{\theta}$ and the
marginalisation can be done in closed form [10, 23] (called as the _BGe
score_). Using this parameter prior to calculate marginal likelihood in the
ELBO makes the ELBO score-equivalent. Details are given in Appendix B.3.

###  2.3 Estimating Gradients

Since the KL regulariser is defined on a discrete distribution with many
parameters, obtaining a Monte Carlo estimate of this term (and hence the
corresponding ELBO) requires unbiased gradients of $\phi$. Therefore, we use
the score-function gradient estimator [36] with exponential moving average
baseline for obtaining the gradients. As the score function estimator does not
require the estimand function to be differentiable, we can perform closed form
marginalisation of the parameters with the standard parameter priors like that
of Gaussian-Wishart.

The detailed algorithm is given in Algorithm 1.

Algorithm 1 VCN Algorithm

1:$\phi\leftarrow$ initialise parameters

2:repeat

3: Sample $L$ graphs $\\{\mathcal{G}^{(i)}\\}_{i=1}^{L}$ autoregressively from
$q_{\phi}(\mathbf{A}_{\mathcal{G}})$ using an LSTM.

4: Calculate the per-sample marginal log-likelihood $\log
p({\mathcal{D}}|\mathcal{G}^{(i)})$ by marginalising $\bm{\theta}$ to obtain
the BGe Score [10, 23].

5: $g\leftarrow\frac{1}{L}\sum_{i=1}^{L}\nabla_{\phi}\left[\log
p({\mathcal{D}}|\mathcal{G}^{(i)})-\left[\log q_{\phi}(\mathcal{G}^{(i)})-\log
p(\mathcal{G}^{(i)})\right]\right]$

6: $\phi\leftarrow$ Update parameters using gradients $g$.

7:until $\phi\leftarrow$ converge.

##  3 Related Work

Causal Discovery: Broadly, causal structure learning approaches can be mainly
categorised to two different paradigms with regards to kind of data used:
methods involving learning structure from purely observational data and
methods involving learning structure from both observational and
interventional data. Both these approaches strongly rely on the
identifiability of SCM. Zheng et al. [38], Yu et al. [37], Lachapelle et al.
[24] use a score-based objective with global search over DAG space to learn
the structure. Lachapelle et al. [24] extend the linear model of [38] to non-
linear case using neural networks. GES [6, 31] greedily searches over the
space of CPDAG’s and maximize a score based on Bayesian Information Criterion
(BIC). LiNGAM [33] uses an Independent Component Analysis (ICA) based approach
by making the assumption that either the variables or one of the noise
variables are non-Gaussian. PC [34] algorithm uses a constraint based approach
for causal discovery.

Different from the above approaches, a few approaches use a combination of
observational and interventional data to learn the causal structure [13]. ICP
[28] assumes a linear SCM and resorts to conditional independence testing to
test the hypothesis of invariance, a concept wherein the plausible causal
predictors of a given variable are stable across different interventions. [16]
extend ICP to non-linear SCMs. These methods do not infer the complete graph
but instead just the causal parents of a particular variable. The difficulty
of these approaches rest in the fact that conditional independence tests are
hard to perform. More recently, [3, 20] and [19] use a meta-learning objective
to learn the causal structure using interventional data.

Bayesian Approach to Structure Learning: Existing approaches for Bayesian
learning of causal graphs are mainly concerned with efficient sampling of
graph structures from the posterior distribution. A popular choice for such an
approach is the Markov Chain Monte Carlo (MCMC) with suitably chosen energy
functions and heuristics [8, 26, 25, 22, 14, 9]. While Niinimäki et al. [26]
is concerned with causal discovery by sampling graph structures from partial
orders of edge orientation, Madigan et al. [25] proposes an approach for
learning probabilistic graphical models with discrete variables called as
structure MCMC. Kuipers and Moffa [22], Grzegorczyk and Husmeier [12] and
Ellis and Wong [8] propose improved MCMC techniques for sampling graph
structures by restricting graph space to certain topology and sampling from a
restricted space. Heckerman et al. [14] use a BIC based scoring function to
learn a Bayesian network by sampling from Metropolis-Hastings under different
energy configurations. An estimate of MAP is obtained using Maximum Likelihood
to get the final graph structure. Agrawal et al. [2] use DAG bootstrapping to
estimate the posterior over graphs and use this for budgeted experimental
design for causal discovery. While all these techniques are intended for a
Bayesian approach to structure learning, they face the problem of efficiently
approximating the posterior over graph structures and hence resort to either
heuristics or restricted graph structures. However, we do not make any
simplifying assumptions about the graph structures and we can efficiently
approximate the posterior while being able to sample exactly from them.

##  4 Experiments

To validate the modelling choice presented in the previous section, we perform
experiments mainly focusing on the following aspects: (1) since we approximate
the posterior, we evaluate how close is the approximation to the true
posterior in lower dimensional settings ($\leq 4$) where enumeration of true
posterior is possible. (2) We outline the difficulty involved in evaluating
the Bayesian Causal inference models in higher dimensions where enumeration is
not possible and suggest possible metrics which alleviate the problem and
quantify how well the true posterior is approximated. We further evaluate our
model on these metrics.

##### Key Findings

We summarise the experimental results as follows: (1) The proposed
autoregressive approach of VCN performs better than a factorised distribution
[19] in all the considered settings. This is due to the ability of an
autoregressive distribution to model multi-modal posteriors as compared to the
unimodal factorised distribution. (2) In higher dimensions, the proposed
approach compares favourably to competitive baselines on various metrics. (3)
VCN achieves a good approximation vs run-time trade-off, thus making it a
suitable approach for Bayesian causal inference.

##### Experimental Settings

We evaluate our method on both synthetic and real datasets. For generating
synthetic data, we follow the procedure of NOTEARS [38]. We sample a DAG at
random from an Erdos-Renyi (ER) model with expected number of edges equal to
$d$. Each reported result is over 20 different random graphs. The models were
trained with a learning rate of $1e-2$ using the Adam optimiser [21] for 30k
epochs. We consider the settings when we have $n=10,100$ samples. For taking
the Monte Carlo estimate of the ELBO, we take $L=1000$ samples. $\lambda_{s}$
is fixed to 0.01 and $\lambda_{t}$ is annealed from 10 to 1000 with an
exponential annealing schedule.

##### Baselines

Parameterising the posterior with a factorised distribution [19] is our main
baseline as it involves differentiable causal learning based techniques
similar to our approach. We also compare with DAG Bootstrap [2] where the
posterior is estimated by bootstrapping the data with any causal discovery
algorithm and then forming an empirical estimate of the posterior based on
frequency count. We use LiNGAM [33] and NOTEARS [38] as underlying causal
discovery algorithms for DAG Bootstrap. They are denoted by Boot Lingam and
Boot Notears respectively. In addition, we compare with the MCMC approach of
Minimal IMAP MCMC [1]. Details of experimental settings of each of these
techniques is given in Appendix B.4.

###  4.1 Evaluation Metrics

For low dimensional variables $(d\leq 4)$, we can enumerate the true posterior
for all graphs. Therefore, in this setting we compute the distance between the
variational posterior learned from data and the true posterior.

For higher dimensional variables, where enumerating the true posterior is not
possible, evaluating Bayesian causal inference algorithms is not
straightforward. Directly comparing the likelihoods do not necessarily ensure
that the posterior learned is good, as graphs with more edges always have
higher likelihoods. Instead, we employ the following metrics:

##### Expected Structural Hamming Distance
($\mathop{{}\mathbb{E}}[\text{SHD}]$)

Given that the true posterior has modes over all the graphs inside the MEC
corresponding to the ground truth graph, we can sample the graphs from the
model and then compute the Structural Hamming Distance (SHD) between these
samples and the ground truth. We can then compute the empirical mean of these
SHDs as a metric. As all the graphs inside the MEC have the same number of
edges, this metric indicates how well the approximated posterior is close to
the ground truth on average. If $\mathcal{G}_{\text{GT}}$ is the ground truth
data generating graph, then $\mathop{{}\mathbb{E}}[\text{SHD}]$ is given by

$\mathop{{}\mathbb{E}}_{q_{\phi}(\mathcal{G})}[\text{SHD}]\approx\frac{1}{T}\sum_{i=1}^{T}\left[\text{SHD}(\mathcal{G}^{(i)},\mathcal{G}_{\text{GT}})\right]~{}~{}~{}~{}\mathcal{G}^{(i)}\sim
q_{\phi}(\mathcal{G})$ (9)

##### Area Under Reciever Operating Curve (AUROC)

If we care for just feature probabilities of quantities like the presence of
an edge, we can compute edge beliefs and compute the Receiver Operating Curve
(ROC). The area under this curve can be treated as a metric for evaluating the
Bayesian model. However, it does not necessarily give a full picture of the
multiple modes of the posterior and whether the sampled graph is far away
/close to the MEC of the true graph. Nevertheless, it can still be informative
of the edge beliefs of the learned approximation. Details of computing this is
give in [30].

We would like to note that though these metrics capture to a reasonable extent
how well the true posterior is approximated, they are still imperfect. Since
the true posterior has mass over all the graphs corresponding to MEC of the
ground truth graph, the true posterior does not necessarily evaluate to zero
on $\mathop{{}\mathbb{E}}[\text{SHD}]$ and one on AUROC. Hence, evaluating the
learned model on these metrics only partially indicates how well the true
posterior is approximated.

###  4.2 Results

Figure 3: Hellinger distance of the full posterior of the approximation with
the true posterior.

Figure 4: Average runtime (in seconds, log scale) of different Bayesian causal
inference approaches.

####  4.2.1 Estimation of True Posterior

As indicated before, when the number of variables in the graph is small, we
can enumerate the true posterior and compute the divergence/distance between
the approximation and the true posterior. Figure 4 presents the Hellinger
distance of the approximation up to four nodes. It can be seen that the
autoregressive distribution of VCN reconstructs the true posterior much better
than the factorised distribution, mainly due to the non-identifiability of
graphs and the fact that the true posterior is multimodal.

####  4.2.2 Evaluation in higher dimensions

Figure 5 reports the $\mathop{{}\mathbb{E}}[\text{SHD}]$ of the approximation
for different number of nodes. It can be seen that the autoregressive
distribution of VCNs gives better performance than the factorised distribution
in all the settings. In addition, the autoregressive distribution gives better
results as compared to all the competitive baselines when there are 10
samples, a regime common in biological applications. When the number of
samples are increased to 100, the proposed approach compares favourably to the
baselines which do not involve differentiable learning based techniques, while
being much better than the factorised distribution. Figure 6 presents the
AUROC of these approaches. The autoregressive distribution of VCN performs
better than the learning based method of factorised distribution, while
comparing favourably to IMAP MCMC. We found that Boot Notears usually performs
well on this metric. However, DAG Bootstrap in general has some limitations.
Though DAG Bootstrap can capture multiple modes, its support is limited to the
DAGs estimated using the bootstrap procedure and hence does not necessarily
have full support. In addition, it can be significantly slower than VCN
(Figure 4). Therefore, the proposed approach has a good tradeoff of runtime
versus performance against the evaluated metrics and is strictly favourable in
terms of purely learning based approaches.

Figure 5: $\mathop{{}\mathbb{E}}\left[\text{SHD}\right]$ for $d=10$ and $d=20$
node ER random graphs (lower is better). Results obtained using 20 different
random graphs.

Figure 6: AUROC for $d=10$ and $d=20$ node ER random graphs (higher is
better). Results obtained using 20 different random graphs.

####  4.2.3 Real Dataset

Table 1: Results on the Dream4 gene expression dataset in terms of AUROC. | AUROC
---|---
IMAP MCMC | 0.438 $\pm$ 0.02
Boot Notears | 0.330
Boot Lingam | 0.689
Factorised | 0.489 $\pm$ 0.01
Autoregressive (VCN) | 0.519 $\pm$ 0.03

We evaluate our method in terms of AUROC on edge feature probabilities against
all the baselines on the _Dream4_ in-silico network challenge on gene
regulation. As the typical metric for this challenge is the AUROC, we restrict
our focus to this metric on this dataset. In particular, we examine the
multifactorial dataset consisting of ten nodes and ten observations. Table 1
reports the AUROC of all the methods including the baselines. As the table
suggests, our method achieves favourable performance while still being faster.
In addition, we note that the underlying true graph corresponding to this
dataset need not be a DAG, therefore introducing model misspacification.

##  5 Conclusion

We present a variational inference approach to perform Bayesian inference over
the unknown causal structure in the structural causal model. Since the
posterior over graph structures is multimodal in nature due to non-
identifiability, we parameterise the variational family as an autoregressive
distribution and model it using an LSTM. While we explore getting uncertainty
estimates for linear models, it could be relevant to extend this to the case
of non-linear models. One of the main limitations of this approach is the
scalability of this method to larger dimensions. While a score function
estimator might be limiting in higher dimensions due to higher variance,
advanced variance reduction techniques could be used [11]. Further
improvements to the proposed method can be obtained by combining with
normalising flows for discrete variables [35, 18]. Nevertheless, the
uncertainty estimates obtained in this framework can be useful for selecting
the most informative interventions, performing budgeted interventional
experiments as well as representation learning of high-dimensional signals
like natural images [3].

## References

  * Agrawal et al. [2018] Raj Agrawal, Caroline Uhler, and Tamara Broderick.  Minimal i-map mcmc for scalable structure discovery in causal dag models.  In _International Conference on Machine Learning_ , pages 89–98. PMLR, 2018.
  * Agrawal et al. [2019] Raj Agrawal, Chandler Squires, Karren Yang, Karthik Shanmugam, and Caroline Uhler.  Abcd-strategy: Budgeted experimental design for targeted causal structure discovery.  _arXiv preprint arXiv:1902.10347_ , 2019.
  * Bengio et al. [2019] Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Rosemary Ke, Sébastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal.  A meta-transfer objective for learning to disentangle causal mechanisms.  _arXiv preprint arXiv:1901.10912_ , 2019.
  * Blei et al. [2017] David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe.  Variational Inference: A Review for Statisticians.  _Journal of the American Statistical Association_ , 112(518):859–877, Apr 2017.  ISSN 1537-274X.  doi: 10.1080/01621459.2017.1285773.  URL http://dx.doi.org/10.1080/01621459.2017.1285773.
  * Bühlmann et al. [2014] Peter Bühlmann, Jonas Peters, Jan Ernest, et al.  Cam: Causal additive models, high-dimensional order search and penalized regression.  _The Annals of Statistics_ , 2014.
  * Chickering [2002] David Maxwell Chickering.  Optimal structure identification with greedy search.  _Journal of machine learning research_ , 2002.
  * Eberhardt and Scheines [2007] Frederick Eberhardt and Richard Scheines.  Interventions and causal inference.  _Philosophy of science_ , 74(5):981–995, 2007\.
  * Ellis and Wong [2008] Byron Ellis and Wing Hung Wong.  Learning causal bayesian network structures from experimental data.  _Journal of the American Statistical Association_ , 2008.
  * Friedman and Koller [2003] Nir Friedman and Daphne Koller.  Being bayesian about network structure. a bayesian approach to structure discovery in bayesian networks.  _Machine learning_ , 50(1):95–125, 2003.
  * Geiger et al. [2002] Dan Geiger, David Heckerman, et al.  Parameter priors for directed acyclic graphical models and the characterization of several probability distributions.  _The Annals of Statistics_ , 2002.
  * Grathwohl et al. [2017] Will Grathwohl, Dami Choi, Yuhuai Wu, Geoffrey Roeder, and David Duvenaud.  Backpropagation through the void: Optimizing control variates for black-box gradient estimation.  _arXiv preprint arXiv:1711.00123_ , 2017.
  * Grzegorczyk and Husmeier [2008] Marco Grzegorczyk and Dirk Husmeier.  Improving the structure mcmc sampler for bayesian networks by introducing a new edge reversal move.  _Machine Learning_ , 2008.
  * Hauser and Bühlmann [2012] Alain Hauser and Peter Bühlmann.  Characterization and greedy learning of interventional markov equivalence classes of directed acyclic graphs.  _Journal of Machine Learning Research_ , 2012.  URL http://jmlr.org/papers/v13/hauser12a.html.
  * Heckerman et al. [1999] David Heckerman, Christopher Meek, and Gregory Cooper.  A bayesian approach to causal discovery.  _Computation, causation, and discovery_ , 1999.
  * Heinze-Deml et al. [2018a] Christina Heinze-Deml, Marloes H Maathuis, and Nicolai Meinshausen.  Causal structure learning.  _Annual Review of Statistics and Its Application_ , 5:371–391, 2018a.
  * Heinze-Deml et al. [2018b] Christina Heinze-Deml, Jonas Peters, and Nicolai Meinshausen.  Invariant causal prediction for nonlinear models.  _Journal of Causal Inference_ , 2018b.
  * Hochreiter and Schmidhuber [1997] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  _Neural computation_ , 1997.
  * Hoogeboom et al. [2019] Emiel Hoogeboom, Jorn WT Peters, Rianne van den Berg, and Max Welling.  Integer discrete flows and lossless compression.  _arXiv preprint arXiv:1905.07376_ , 2019.
  * Ke et al. [2019] Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Chris Pal, and Yoshua Bengio.  Learning neural causal models from unknown interventions.  _arXiv preprint arXiv:1910.01075_ , 2019.
  * Ke et al. [2020] Nan Rosemary Ke, Jane Wang, Jovana Mitrovic, Martin Szummer, Danilo J Rezende, et al.  Amortized learning of neural causal representations.  _arXiv preprint arXiv:2008.09301_ , 2020.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kuipers and Moffa [2017] Jack Kuipers and Giusi Moffa.  Partition mcmc for inference on acyclic digraphs.  _Journal of the American Statistical Association_ , 2017.
  * Kuipers et al. [2014] Jack Kuipers, Giusi Moffa, David Heckerman, et al.  Addendum on the scoring of gaussian directed acyclic graphical models.  _Annals of Statistics_ , 42(4):1689–1691, 2014\.
  * Lachapelle et al. [2019] Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, and Simon Lacoste-Julien.  Gradient-based neural dag learning.  _arXiv preprint arXiv:1906.02226_ , 2019.
  * Madigan et al. [1995] David Madigan, Jeremy York, and Denis Allard.  Bayesian graphical models for discrete data.  _International Statistical Review/Revue Internationale de Statistique_ , 1995.
  * Niinimäki et al. [2016] Teppo Niinimäki, Pekka Parviainen, and Mikko Koivisto.  Structure discovery in bayesian networks by sampling partial orders.  _The Journal of Machine Learning Research_ , 2016.
  * Pearl [2009] Judea Pearl.  _Causality_.  Cambridge university press, 2009.
  * Peters et al. [2016] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen.  Causal inference by using invariant prediction: identification and confidence intervals.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 2016.
  * Peters et al. [2017] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf.  _Elements of causal inference_.  The MIT Press, 2017.
  * Prill et al. [2010] Robert J Prill, Daniel Marbach, Julio Saez-Rodriguez, Peter K Sorger, Leonidas G Alexopoulos, Xiaowei Xue, Neil D Clarke, Gregoire Altan-Bonnet, and Gustavo Stolovitzky.  Towards a rigorous assessment of systems biology models: the dream3 challenges.  _PloS one_ , 2010.
  * Ramsey et al. [2017] Joseph Ramsey, Madelyn Glymour, Ruben Sanchez-Romero, and Clark Glymour.  A million variables and more: the fast greedy equivalence search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images.  _International journal of data science and analytics_ , 2017.
  * Schaffter et al. [2011] Thomas Schaffter, Daniel Marbach, and Dario Floreano.  Genenetweaver: in silico benchmark generation and performance profiling of network inference methods.  _Bioinformatics_ , 2011.
  * Shimizu [2014] Shohei Shimizu.  LiNGAM: Non-Gaussian methods for estimating causal structures.  _Behaviormetrika_ , 2014.
  * Spirtes et al. [2000] Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman.  _Causation, prediction, and search_.  MIT press, 2000.
  * Tran et al. [2019] Dustin Tran, Keyon Vafa, Kumar Agrawal, Laurent Dinh, and Ben Poole.  Discrete flows: Invertible generative models of discrete data.  In _Advances in Neural Information Processing Systems_ , 2019.
  * Williams [1992] Ronald J Williams.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  _Machine learning_ , 1992.
  * Yu et al. [2019] Yue Yu, Jie Chen, Tian Gao, and Mo Yu.  Dag-gnn: Dag structure learning with graph neural networks.  _arXiv preprint arXiv:1904.10098_ , 2019.
  * Zheng et al. [2018] Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing.  Dags with no tears: Continuous optimization for structure learning.  In _Advances in Neural Information Processing Systems_ , pages 9472–9483, 2018.

##  Appendix A Proof of Proposition 1

###### Proof.

$\begin{split}&\operatorname*{arg\,min}_{\phi}\operatorname{D_{KL}}{\left(q_{\phi}(\mathcal{G})||p(\mathcal{G}|{\mathcal{D}})\right)}\\\
&\stackrel{{\scriptstyle
Eq.\ref{eq:true_posterior}}}{{=}}\operatorname*{arg\,min}_{\phi}\mathop{{}\mathbb{E}}_{q_{\phi}(\mathcal{G})}[\log
p({\mathcal{D}}|\mathcal{G})+\log p({\mathcal{D}})+\log
q_{\phi}(\mathcal{G})-\log p(\mathcal{G})]\\\\[4.30554pt]
&=\operatorname*{arg\,min}_{\phi}\mathop{{}\mathbb{E}}_{q_{\phi}(\mathcal{G})}\left[-\log
p({\mathcal{D}}|\mathcal{G})\right]+\log
p({\mathcal{D}})+\operatorname{D_{KL}}\left(q_{\phi}(\mathcal{G})||p(\mathcal{G})\right)\\\\[4.30554pt]
\end{split}$

which gives us a lower bound on the marginal log-likelihood of the data.

$\begin{split}\log
p({\mathcal{D}})\geq\mathop{{}\mathbb{E}}_{q_{\phi}(\mathcal{G})}\left[\log
p({\mathcal{D}}|\mathcal{G})\right]-\operatorname{D_{KL}}\left(q_{\phi}(\mathcal{G})||p(\mathcal{G})\right)\\\
\end{split}$

∎

##  Appendix B Detailed Experimental Setup

###  B.1 Synthetic Data Generation

For generating synthetic data, we follow the procedure of NOTEARS [38]. We
sample a DAG at random from an Erdos-Renyi model with expected number of edges
equal to $d$. We only select the DAG if it has atleast two graphs inside the
MEC. The edge weights are sampled independently at random from a Gaussian
prior with a mean of $2$ and variance of $1$.The noise variables of the SCM
are Gaussian mean 0 and variance of 1.

###  B.2 Experimental Details

Each model was then trained 20 times with different graphs (and hence
datasets) for $n=10$ and $n=100$ data samples. For the plots of the true
posterior comparison, we focus on $n=10$ samples. In addition, we also take
the Hellinger distance instead of other divergences as the probabilities of
many graph configurations are almost 0 and the log probabilities become
numerically infeasible to tractably compute. For the factorised distribution
and VCN, the models were trained with a learning rate of $1e-2$ with Adam
optimiser [21] for 30k epochs where the Gibbs temperature of the DAG
constraint $\lambda_{t}$ is annealed from 10 to 1000 with the following
annealing schedule:

$\lambda_{t}(i)=\text{min\\_temp}+10^{\frac{-2\max(0,k-1.1i)}{k}}(\text{max\\_temp}-\text{min\\_temp})$
(10)

where $k$ is the total number of epochs and $i$ is the current epoch. We set
$\lambda_{s}$ to 0.01. We also note that $Z_{d}$ is the partition function of
the Gibbs distribution which in practice cannot be computed easily for data
with dimensionality $d\geq 5$. However, we note that computing this is not
required for the optimisation of the ELBO (Eq. 7) as it turns out to be a
constant.

###  B.3 BGe score

To compute the BGe score, the parameters $\theta$ have the following prior:

$\begin{split}\bm{\theta}&\sim\mathcal{N}(\bm{\theta}|\mu,W^{-1})\\\
p(\mu|\gamma,W,\alpha_{w})&=\mathcal{N}(\mu|\gamma,(\alpha_{\mu}W)^{-1})\\\
p(W|\alpha_{\lambda},T)&:=\text{Wishart}(W|\alpha_{\lambda},T)\end{split}$
(11)

We set $\gamma$ to 2, $\alpha_{\lambda}$ to 10 if $d\leq 5$ else we set it to
1000. $T$ is set to
$\alpha_{\mu}\frac{\alpha_{\lambda}-d-1)}{\alpha_{\mu}+1}\cdot\mathbb{I}^{d\times
d}$. The posterior for the above equation can be obtained with the same
distribution family in close form and is given by:

$\begin{split}p(\mu|W,{\mathcal{D}})&:=\mathcal{N}(\mu|\gamma^{\prime},W^{\prime})\\\
\gamma^{\prime}&=\frac{\alpha_{\mu}\gamma+n\bar{x}_{n}}{\alpha_{\mu}+n}\\\
W^{\prime}&=(\alpha_{\mu}+n)W\end{split}$ (12)
$\begin{split}p(W|{\mathcal{D}})&:=\text{Wishart}(\alpha_{w}^{\prime},T^{\prime})\\\
\alpha_{w}^{\prime}&=\alpha_{w}+n\\\
T^{\prime}&=T+S_{n}+\frac{\alpha_{\mu}n}{\alpha_{\mu}+n}(\gamma-\bar{x}_{n})(\gamma-\bar{x}_{n})^{T}\end{split}$
(13)

where $\bar{x}_{n}=\sum_{i=1}^{n}x^{(i)}$ is the sample mean and
$S_{n}=\sum_{i=1}^{n}(x^{(i)}-\bar{x}_{n})(x^{(i)}-\bar{x}_{n})^{T}$ is the
sample covariance.

###  B.4 Baselines

Here we review the baselines which have been used for comparison and outline
the hyperparameters and training procedures which have been used.

####  B.4.1 DAG Bootstrap

For the DAG bootstrap method, we ran Lingam [33] and NOTEARS [38] by
resampling (with replacement) 50 points at random for 1000 different times for
$n=100$ and 252 times with 5 points for $n=10$. An empirical estimate of the
posterior
$P(\mathcal{G}|\mathbf{x})=w_{\mathcal{G},\mathbf{x}}\mathbbm{1}(\mathcal{G}\in\mathcal{G}_{t})$
is obtained where $\mathcal{G}_{t}$ is the set of all graphs predicted during
the Bootstrap procedure and $w_{\mathcal{G},\mathbf{x}}$ is the empirical
normalised frequency count.

####  B.4.2 IMAP MCMC

We follow the default hyperparameters given in the IMAP MCMC paper [1]. In
particular, we run the MCMC chain for $1e5$ iterations with a burn-in period
of $2e4$ iterations. We use a thinning factor of 100 and use the same
parameters for BGe score calculation as that for our approach.

####  B.4.3 Open Source Codes for Baselines

We use the following open source implementations for the baselines:

  * •

NOTEARS: https://github.com/xunzheng/notears with Apache-2.0 License.

  * •

Lingam: https://github.com/cdt15/lingam with MIT License.

  * •

IMAP MCMC: https://github.com/miraep8/Minimal_IMAP_MCMC.

###  B.5 Real Dataset

For the Dream4 [32] gene-expression dataset, we use one of the gene
expressions from the multifactorial dataset with ten nodes and ten
observations. We also note that all the graphs in Dream4 do not necessarily
correspond to DAGs, thereby introducing a crucial model misspacification in
many of the models presented. We hypothesize that the inferior results of many
of the techniques including the proposed approach suffer from this because of
the non-DAGness.


# How Fragile is Relation Extraction under Entity Replacements?

Yiwei Wang1 Bryan Hooi2 Fei Wang3 Yujun Cai4 Yuxuan Liang2
Wenxuan Zhou3 Jing Tang5 Manjuan Duan1 Muhao Chen3
1 Amazon Inc. 2 National University of Singapore
3 University of Southern California 4 Nanyang Technological University
5 The Hong Kong University of Science and Technology (Guangzhou)
<EMAIL_ADDRESS>The work was done prior to joining Amazon.

###### Abstract

Relation extraction (RE) aims to extract the relations between entity names
from the textual context. In principle, textual context determines the ground-
truth relation and the RE models should be able to correctly identify the
relations reflected by the textual context. However, existing work has found
that the RE models memorize the entity name patterns to make RE predictions
while ignoring the textual context. This motivates us to raise the question:
“are RE models robust to the entity replacements?” In this work, we operate
the random and type-constrained entity replacements over the RE instances in
TACRED and evaluate the state-of-the-art RE models under the entity
replacements. We observe the 30% - 50% F1 score drops on the state-of-the-art
RE models under entity replacements. These results suggest that we need more
efforts to develop effective RE models robust to entity replacements. We
release the source code at https://github.com/wangywUST/RobustRE.

##  1 Introduction

Recent literature has shown that the sentence-level relation extraction (RE)
models may overly rely on entity names for RE instead of reasoning from the
textual context Peng et al. (2020); Wang et al. (2022). This problem is also
known as _entity bias_ Longpre et al. (2021); Qian et al. (2021); Xu et al.
(2022); Wang et al. (2022): the spurious correlation between entity names and
relations. This motivates us to raise a question: “how robust are RE models
under entity replacements?”

Entity bias degrades the RE models’ generalization, such that the entity names
can mislead the models to make wrong predictions. However, a seemingly
conflicting phenomenon is that RE models exhibit high (in-distribution)
accuracy on standard benchmarks, such as TACRED. In our work, we find that
these benchmarks are prone to have shortcuts from entity names to ground-truth
relations (see Fig. 2), low entity diversity, and a large portion of incorrect
entity annotations. These issues suggest that, given the presence of entity
bias, the current benchmarks are not challenging enough to evaluate the
generalization of RE in practice.

Figure 1:  The performance of state-of-the-art RE models drop a lot under
entity replacements.

Most existing methods for evaluating the generalizability of NLP focus on
sentence classification Jin et al. (2020); Li et al. (2020); Minervini and
Riedel (2018) and question answering Jia and Liang (2017); Ribeiro et al.
(2018); Gan and Ng (2019), but these methods lack special designs to seize on
the entity bias in RE. In this work, we propose a type-constrained and random
entity replacement method: ENTRE. Type-constrained means we replace the named
entity in the type [PERSON] or [ORGANIZATION] with the new entity belonging to
the same type as the original entity. Random means we randomly select the
entity names from a Wikipedia entity lexicon that consists of 24,933
organizations and 902,007 person entities for replacements. These two
principles guarantee the effectiveness of entity replacement to produce valid
and diverse RE instances.

We apply ENTRE to TACRED to produce ENTRED, a challenging RE benchmark with
fewer shortcuts and higher entity diversity. We evaluate the RE models on the
instances with replaced entity names produced by ENTRE.

We analyze the RE models under entity replacements in order to answer four
research questions: (Q1) Does ENTRE reduce prediction shortcuts from entity
names to the ground-truth relations? (Q2) Does ENTRE improve the entity
diversity? (Q3) How do the strong RE models perform under entity replacements?
(Q4) How to improve the generalization of RE?

We observe several key findings. First, ENTRE reduces the shortcuts by more
than 50% on many relations, and improves the subject name diversity by more
than 25 times compared to TACRED. Second, the strong RE models LUKE Yamada et
al. (2020) and IRE Zhou and Chen (2021) tend to memorize entity-relation
patterns to infer the relation instead of reasoning based on the textual
context that actually describes the relation. This phenomenon causes the model
to be brittle to entity replacements, resulting in a significant performance
drop of 30% - 50% in terms of the F1 score. Third, the recent causal inference
approach CoRE Wang et al. (2022) improves the robustness at a higher magnitude
than other methods. We believe the proposed benchmark ENTRED and ENTRE will
benefit future research toward improving the robustness of RE.

Figure 2:  TACRED offers many shortcuts from entity names to ground-truth
relations in the test set, where the model predicts the correct relation even
when only given the entity names, despite all textual context being removed.
As a result, it is not challenging enough to measure the generalization under
entity bias.

##  2 Analysis of Entity Names in TACRED

Before building ENTRED, we first analyze the existing popular RE datasets. Our
analysis is focused on the following three perspectives: 1) the correctness of
entity name annotations; 2) the diversity of entity names; 3) the prediction
shortcuts from entity names to the ground-truth relations.

In the popular TACRED Zhang et al. (2017), TACREV Alt et al. (2020), and Re-
TACRED Stoica et al. (2021) datasets, we find that: first, there exist some
portion of incorrect entity name annotations; second, many entity names are
reused more than one hundred times across instances; third, the entity names
in more than 70% of the instances act as shortcuts to the ground-truth
relations. We introduce the details as follows.

###  2.1 Incorrect Entity Annotations

In the TACRED Zhang et al. (2017), TACREV Alt et al. (2020), and Re-TACRED
Stoica et al. (2021) datasets, there exist quite a few incorrect entity
annotations. To detect these incorrect entity annotations, we use a BERT based
NER model Devlin et al. (2019) to automatically annotate the subject and
object entity names in the TACRED dataset. Then, we conduct manual
investigation on the entities where the NER annotations are different the
original TACRED annotations. We find that more than 10% of the test instances
contain incorrect entity annotations.111Including both incorrect span and type
annotations. We present two examples in Fig. 3. Using these mistaken entity
annotations to evaluate the RE models compromise our goal of correctly
measuring RE performance.

Figure 3:  Two examples of incorrect entity annotations in TACRED.  Figure 4:
The number of different subject entity names (red) is much lower than the
number of instances (blue) in the test sets of the TACRED, TACREV, and Re-
TACRED datasets. In other words, the diversity of entity names in these
datasets’ test sets is limited.

###  2.2 Diversity of Entity Names

The TACRED, TACREV, and Re-TACRED datasets have low diversity of entity names:
most entity names repeatedly appear in a large portion of instances (see Fig.
4). In the TACRED datasets, there are only 420 entity names repeatedly
appearing as 15509 instances’ subjects. For example, “ShopperTrak”, as the
subject, has repeatedly appeared as the subject entity in 270 instances. This
heavily repeated use of entity names increases the risk that RE relies on
entity bias to make RE predictions. Also, with these benchmarks, it is
impossible to comprehensively evaluate the generalization of RE models on a
diverse set of entity names to imitate real-world scenarios.

###  2.3 Causal Inference for Entity Bias

We follow the prior work Wang et al. (2022) to analyze the entity bias based
on causal inference. Wang et al. (2022) builds the causal graph of RE as a
directed acyclic graph: $(E,X)\rightarrow Y$ in Figure 5. $X$ is the input
text, $E$ denotes the entity mentions, and $Y$ is the relation extraction
result. On the edges $(X,E)\rightarrow Y$, the RE model encodes $E$ and $X$ to
predict the relation $Y$.

Figure 5:  The original causal graph of RE models (left) together with its
counterfactual alternatives for the entity bias (right). The shading indicates
the mask of corresponding variables.  Figure 6:  The ratio of instances with
shortcuts (the entity bias is as same as the ground truth relation) in the
TACRED test set.

Based on the causal graph displayed in Figure 5, we can diagnose whether the
entities have shortcuts to relation. Wang et al. (2022) distill the entity
bias by counterfactual analysis, which assigns the hypothetical combination of
values to variables in a way that is counter to the empirical evidence
obtained from data. We mask the tokens in $X$ to conduct the intervention
$X=\bar{x}$ on $X$, while keeping the variable $E$ as the original entity
mentions $e$. In this way, the textual context is removed and the entity
information is maintained. Accordingly, the counterfactual prediction is
denoted as $Y_{\bar{x},e}$ (see Figure 2). $Y_{\bar{x},e}$ refers to the
output, i.e., a probability distribution or a logit vector, where only the
entity mentions are given.

###  2.4 Shortcuts to the Ground-Truth Relations

Existing work has found that the popular RE benchmarks’ test sets provide
abundant shortcuts from entity names to ground-truth relations Wang et al.
(2022); Peng et al. (2020). In other words, on many instances, the model need
not “extract” the relation from the textual context but can infer the correct
prediction directly through shortcuts from entities.

To verify these observations, we conduct a preliminary study of the shortcuts
using the strong RE model LUKE Yamada et al. (2020) on the TACRED dataset. We
first compute the instance-wise relation extraction result in the TACRED’s
test set. Then, we analyze the shortcuts from entity names to the relations
based on causal inference (see details in Sec. 2.3). We find that there exists
a large portion of instances having shortcuts from entity names to the ground-
truth relations. We visualize the ratio of instances that present shortcuts in
different relations in Fig. 6. Last but not least, we observe similar
phenomena on other models and TACREV, Re-TACRED datasets as well.

The analyses suggest that these benchmarks do not accurately evaluate the
“extraction” capability of RE models without the shortcuts from entity names.
In our words, the standard RE benchmarks are not challenging enough to
evaluate whether the RE models can extract the correct relations from the
textual context. In our work, we replace the entity names to reduce the
shortcuts, to mitigate the possibility that RE models rely on the shortcut of
entity bias to achieve over-optimistically high RE performance. Our ENTRED is
able to better simulate real-world scenarios with fewer shortcuts and higher
entity diversity, which is a better evaluation of the generalization of RE
models.

##  3 Entity Replacement for RE

We present ENTRE: a simple yet effective procedure to generate high-quality RE
instances with entity replacements. ENTRE replaces entity names in the RE
instances in a random and type-constrained manner. We apply ENTRE to the test
set of TACRED to evaluate the state-of-the-art RE models’ robustness under
entity replacements.

###  3.1 Targetting the Instances for Replacements

We desire entity replacements to not affect the soundness of language. As we
have analyzed in Sec. 2.1, there exist a significant amount of incorrect
entity annotations. To handle these incorrect entity annotations, we use a
BERT based NER model Devlin et al. (2019) to re-annotate the entities in the
TACRED datasets. Then, we further conduct manual investigation over the entity
annotations. We filter out incorrect entity annotation instances and only
replace the tokens that belong to named entities. This ensures that our entity
name replacements do not alter the ground-truth relation labels.

Besides the incorrect entity annotations, there are also some entities for
which replacement may inevitably cause noise. For example, some entities
belong to the [MISC] (miscellenous) class. If we replace a [MISC] entity with
another [MISC] one, it is likely that we will break the semantics of the
original sentence. In contrast, replacing the [PERSON] and [ORGANIZATION]
entities with those belonging to the same type generally do not affect the
ground-truth relations. We notice that all the instances in TACRED have a
[PERSON] or [ORGANIZATION] entity as the subject or object. Therefore, in our
work, we focus on replacing the [PERSON] and [ORGANIZATION] entities.

###  3.2 A Large Lexicon of Entities

We set the standards for the new entity names selected for replacements:

  1. 1.

The new entity belong to the same type as the replaced one.

  2. 2.

The new entity names are more diverse.

These two principles contribute to making the resulting instances natural –
i.e., containing real, valid entities that are of the same class as the
original entities, and are linguistically sound; challenging – i.e., the new
entities may not offer shortcuts to the model, which cannot easily get the
correct extraction result by seeing only the entity names and comprehensive –
i.e., the robustness of RE is evaluated on a more diverse set of entities.

To satisfy the above principles, we first build up a large entity name lexicon
to provide the new entity names for replacements. The size of the entity name
lexicon determines the diversity of entity names in our new RE benchmark
ENTRED. Also, a larger entity name lexicon can help us to evaluate the
generalization of RE models on more out-of-domain entity names in test time.
Therefore, in addition to the entity names appearing in the TACRED, we collect
the entity names from Wikipedia belonging to the category of person and
organization to enrich the entity name corpus. Overall, we collect 24,933
organization and 902,007 person names from
Wikipedia.222https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-
articles.xml.bz2

###  3.3 Entity Replacements

Based on the constructed entity lexicon, we propose ENTRE: a type-constrained
and random entity replacement method. Type-constrained means we replace the
named entity in the type [PERSON] or [ORGANIZATION] with the new entity
belonging to the same type as the original entity. Random means we randomly
select the entity names from our entity lexicon that consist of 24,933
organizations and 902,007 person entities for replacements. These two
principles guarantee the effectiveness of entity replacement to produce valid
RE instances. We iterate over TACRED instances and replace the entity names.
We summarize ENTRE as the following pipeline:

  1. 1.

Collecting the instances with predictions as same as the ground-truth
relation.

  2. 2.

Replace the entity names for the collected entities in Step 1. Repeat step 1.

The above steps can be repeated for many times, and a higher repetition time
leads to a higher level of the adversary. We can stop the repeating until all
the entities in the lexicon have been used. But that will induce too long
running time. Therefore, in our work, we set the maximum number of repetitions
as 200.

Step 1 requires the inference on many test instances, which is time-consuming.
Considering that the F1 score’s calculation of RE takes the “no_relation” as
the background class, we can alternatively collect the instances not belonging
to the “no_relation” class in Step 1. We denote such an alternate as ENTRE-
fast, which saves 90% evaluation time in the experiments.

Benchmark | TACRED | ENTRED
---|---|---
$\\#$ Sentences | 15,509 | 12,419
$\\#$ Tokens | 539,306 | 457,121
Table 1: Statistics of the TACRED and ENTRED benchmarks.

We create the challenging RE benchmark ENTRED based on the public benchmark
TACRED by applying ENTRE on the test set of TACRED. The overall statistics of
ENTRED are shown in Table 3, alongside the statistics of the original TACRED
dataset. The number of sentences in ENTRED is slightly smaller than that in
TACRED because we filter out the instances having incorrect entity
annotations. We showcase ENTRE using TACRED in this paper because of its
popularity on evaluating RE models and comprehensive relation-type coverage.
However, our ENTRE can be applied to other RE datasets.

Figure 7:  ENTRED significantly reduces the ratio of instances with shortcuts
(the entity bias is as same as the ground truth relation) compared with
TACRED.  Figure 8:  The number of subject entity names, person entity names,
and organization entity names in the test set of TACRED (red) and ENTRED
(blue).

##  4 Experiments

In this section, we investigate ENTRE and use it to evaluate the robustness of
the strong RE models LUKE Yamada et al. (2020), IRE Zhou and Chen (2021), and
other methods that can improve the robustness of RE. Our experimental settings
closely follow those of previous work Zhang et al. (2017); Zhou and Chen
(2021); Nan et al. (2021) to ensure a fair comparison. We organize our results
and analysis as four main research questions and their answers.

[couleur= msftBlue!15, arrondi=0, logo=, marge=2, couleurBord=msftBlue!10,
sousTitre =Q1: Does ENTRE reduce shortcuts?]

####  ENTRE leads to fewer shortcuts from entity names to ground-truth
relations

We perform causal inference over ENTRED to analyze how many instances have
shortcuts from entity names to the ground-truth relations after the entity
replacements. We present the comparison of the shortcut ratio on ENTRED and
TACRED on different relations in Fig. 7. We observe that ENTRED greatly
reduces the shortcuts for more than 50% instances on most relations. As a
result, when being evaluated using ENTRED, RE models have to extract the
informative signals describing the ground-truth relations from the textual
context, rather than rely on the shortcuts from the entity names.

[couleur= msftBlue!15, arrondi=0, logo=, marge=2, couleurBord=msftBlue!10,
sousTitre =Q2: Does ENTRE improve diversity?]

#### Comparison between ENTRED and existing benchmarks.

As we have analyzed in Sec. 2.1, the diversity of entity names in the existing
benchmarks TACRED, TACREV and Re-TACRED are rather limited. These limitations
hinder the evaluation of the generalization and generalization of RE. In our
work, thanks to our larger lexicon built from the Wikipedia entity names, our
ENTRED have much higher diversity than the TACRED and Re-TACRED, as shown in
Fig. 8. With these diverse entity names, ENTRED is able to evaluate the
performance of RE models on a larger scale of diverse entities, which better
imitates the real–world scenario.

Method | TACRED | ENTRED (Ours) | $\Delta$
---|---|---|---
LUKE Yamada et al. (2020) | 72.7 | 45.0 | $\downarrow 44\%$
w/ Resample Burnaev et al. (2015) | 73.1 | 45.8 | $\downarrow 37\%$
w/ Entity Mask (w/o name, w/o type) Zhang et al. (2017) | 21.3 | 21.0 | $\downarrow 1\%$
w/ Entity Mask (w/o name, w/ type) Zhang et al. (2017) | 44.9 | 45.9 | $\uparrow 2\%$
w/ Entity Mask (w/ name, w/ type) Zhang et al. (2017) | 72.3 | 61.2 | $\downarrow 15\%$
w/ Focal Lin et al. (2017) | 72.9 | 47.1 | $\downarrow 35\%$
w/ CoRE Wang et al. (2022) | 74.6 | 61.7 | $\downarrow 17\%$
Table 2: F1 scores (%) and the performance dropping of RE on the test sets of
TACRED and our ENTRED. The best results in each column are highlighted in bold
font. We additionally report the performance drop (%) compared with the
performance on the original TACRED dataset.

[couleur= msftBlue!15, arrondi=0, logo=, marge=2, couleurBord=msftBlue!10,
sousTitre =Q3: How robust is RE under entity replacements?]

#### Main Results

We evaluate the robustness of the state-of-the-art RE models LUKE Yamada et
al. (2020) and IRE Zhou and Chen (2021) under entity replacements. Our
experimental settings closely follow those of previous work Zhang et al.
(2017); Zhou and Chen (2021); Nan et al. (2021) to ensure a fair comparison.
We visualize the empirical results in Fig. 1. We observe that the 30% - 50%
drops in terms of F1 scores happen on the state-of-the-art RE models after
entity replacements. These results suggest that there remains a large gap
between the current research and the really effective RE models robust to
entity replacements.

We show the F1 scores on ENTRED in Table 2. We can see that the state-of-the-
art LUKE has a significant performance drop in our challenging ENTRED; there
is a 44% relative decrease (in the models’ F1) in ENTRED as compared to their
results before entity replacements.

Original Instance Original Prediction New Entity Names New Prediction Finance
Ministry spokesperson Chileshe Kandeta who confirmed this on Sunday said
Magande signed a loan agreement of 31 million dollars with the ADF for the
country ’s Poverty Reduction Budget Support. no_relation ✓ American
Association of University Women, Willingboro Chapter members ✗ John Graham , a
55-year-old man from Canada , is accused of shooting Aquash in the head and
leaving her to die on the Pine Ridge reservation in South Dakota.
stateorprovince_of_death ✓ Liu Shaozhuo, South Dakota no_relation ✗ After the
staffing firm Hollister Inc lost 20 of its 85 employees, it gave up nearly a
third of its 3,750-square-foot Burlington office, allowing the property owner
to put up a dividing wall to create a space for another tenant.
number_of_employees/members ✓ Yoruba Academy, 85 alternate_names ✗ Kercher ’s
mother, Arline Kercher, tells court in emotional testimony that she will never
get over her daughter ’s brutal death. children ✓ Sanju Yadav, Matti Koistinen
no_relation✗ Lt. Assaf Ramon, the son of Israel ’s first astronaut, Col. Ilan
Ramon, who died in the space shuttle Columbia disaster in 2003, was killed
Sunday when an F16-A plane he was piloting crashed in the hills south of
Hebron in the West Bank. children ✓ Aaron Morgan, Ángel Guillermo Heredia
Hernández no_relation ✗ Police have released scant information about the
killing of 61-year-old Carol Daniels, whose body was found Sunday inside the
Christ Holy Sanctified Church, a weather-beaten building on a rundown block
near downtown Anadarko in southwest Oklahoma. stateorprovince_of_death ✓ Mao
Weiming, Oklahoma no_relation ✗

Table 3: A case study for LUKE on the relation extraction benchmark TACRED and
our ENTRED. Underlines and wavy lines highlight the subject and object
entities respectively. We report the original prediction, the new entity names
for replacements and the prediction in ENTRED.

#### Case Study

We conduct case studies to empirically examine the effects of our entity
replacements of ENTRE. Table 3 gives a qualitative comparison example between
the RE results on Re-TACRED and our ENTRED. The results show that our ENTRE
misleads the strong RE model LUKE to predict incorrect relations. For example,
given the TACRED instance “Finance Ministry spokesperson Chileshe Kandeta who
confirmed this on Sunday said Magande signed a loan agreement of 31 million
dollars with the ADF for the country ’s Poverty Reduction Budget Support.”,
there is no relation between the subject and object existing in the text.
After the entity replacement, LUKE believes that the relation between them is
“members”.

The entity bias can account for this result, where given only the entity
mentions American Association of University Women and Willingboro Chapter, the
RE model returns the relation “members” without any textual context. This
implies that the model makes the prediction for the original input relying on
the entity mentions, which leads to the wrong RE prediction. In our work, we
replace the original entities with the new ones that convey the entity bias
different from the ground-truth label to test the generalization of RE models
under entity bias.

#### Memorizing or Reasoning?

We propose ENTRE to test the ability to use the textual context to infer the
relations. As the entity replacements of ENTRE do not affect the ground-truth
relations, RE models should be robust against entity name changes. However, we
observe the large performance drops from our entity replacements.

Therefore, we conclude that the strong RE model LUKE is apt to memorize the
entity name patterns for predicting relations and is more brittle when the
entities that convey the biases different from the ground-truth relations
existing in the input text. To make RE models more robust, we believe an
important future direction is to develop context-based reasoning approaches,
taking advantage of inductive biases on the textual context that determines
the relations.

[couleur= msftBlue!15, arrondi=0, logo=, marge=2, couleurBord=msftBlue!10,
sousTitre =Q4: How to improve the generalization?]

#### Methods

In our work, we consider the following methods to improve the generalization
of RE: (1) Focal Lin et al. (2017) adaptively reweights the losses of
different instances so as to focus on the hard ones. (2) Resample Burnaev et
al. (2015) up-samples rare categories by the inversed sample fraction during
training. (3) Entity Mask Zhang et al. (2017): masks the entity mentions with
special tokens to reduce the over-fitting on entities. (4) CoRE Wang et al.
(2022) is a causal inference based method that mitigates entity bias.

#### Results & Analysis

The results of the above methods on the RE model are shown in Table 2. The
recently proposed causal inference based debiasing method CoRE offers the best
improvements against our entity replacements (($45.0\%\rightarrow 61.7\%$)).
We conjecture that this is because it mitigates the biasing signals from
entity names, which enhances its entity-level generalization ability and makes
RE models focus more on the textual context for inference, resulting in a
better generalization under entity name replacements. Other methods, however,
lead to lower improvements for LUKE, potentially because they cannot
effectively capture the biased patterns between relations and entity names.

##  5 Related Work

Relation extraction is a sub-task of information extraction that aims to
identify semantic relations between entities from natural language text Zhang
et al. (2017). It is an effective way to automatically acquire important
knowledge and plays a vital role in Natural Language Processing (NLP).
Relation Extraction is the key component for building relation knowledge
graphs, and it is of crucial significance to natural language processing
applications such as structured search, sentiment analysis, question
answering, and summarization Huang and Wang (2017).

Early research efforts Nguyen and Grishman (2015); Wang et al. (2016); Zhang
et al. (2017) train RE models from scratch based on lexicon-level features.
The recent RE work fine-tunes pretrained language models (PLMs; Devlin et al.
2019; Liu et al. 2019). For example, K-Adapter Wang et al. (2020) fixes the
parameters of the PLM and uses feature adapters to infuse factual and
linguistic knowledge. Recent work focuses on utilizing the entity information
for RE Zhou and Chen (2021); Yamada et al. (2020), but this leaks superficial
and spurious clues about the relations Zhang et al. (2018). Despite the biases
in existing RE models, scarce work has discussed the spurious correlation
between entity mentions and relations that causes such biases. Our work builds
an automated pipeline to generate natural instances with fewer shortcuts and
large coverage at scale to reflect the serious effects of entity bias on the
RE models.

There are also work in other domains aiming to evaluate models’ generalization
to perturbed inputs. For example, Jia and Liang (2017) attack reading
comprehension models by adding word sequences to the input. Gan and Ng (2019)
and Iyyer et al. (2018) paraphrase the input to test models’ over-sensitivity.
Jones et al. (2020) target adversarial typos. Si et al. (2021) propose a
benchmark for reading comprehension with diverse types of test-time
perturbation. These works focus on different domains than our research does,
and they do not consider the composition of RE examples. Little attention is
drawn to the entities in the sentences, and many attacks (e.g. character
swapping, word injection) may make the perturbed sentences invalid. To the
best of our knowledge, this work is among the first to propose a
straightforward, dedicated pipeline for generating natural adversarial
examples for the RE task, which takes into account the serious effects of
entity bias in RE models.

##  6 Conclusion

Our contributions in this paper are three-fold. 1) Methodology-wise: we
propose ENTRE, an end-to-end entity replacement method that reduces the
shortcuts from entity names to ground-truth relations. 2) Resource-wise: we
develop ENTRED, a straightforward method for generating natural and
counterfactual entity replacements for RE, which produces ENTRED, a benchmark
for auditing the generalization of RE models under entity bias. 3) Evaluation-
wise: our experimental results and analysis provide answers to four main
research questions on the generalization of RE. We believe ENTRED and the
entity replacement method ENTRE can benefit the community working to increase
the RE models’ generalization under entity bias.

## References

  * Alt et al. (2020) Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020.  TACRED revisited: A thorough evaluation of the TACRED relation extraction task.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1558–1569, Online. Association for Computational Linguistics.
  * Burnaev et al. (2015) Evgeny Burnaev, Pavel Erofeev, and Artem Papanov. 2015.  Influence of resampling on accuracy of imbalanced classification.  In _Eighth international conference on machine vision (ICMV 2015)_ , volume 9875, page 987521. International Society for Optics and Photonics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Gan and Ng (2019) Wee Chung Gan and Hwee Tou Ng. 2019.  Improving the robustness of question answering systems to question paraphrasing.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6065–6075, Florence, Italy. Association for Computational Linguistics.
  * Huang and Wang (2017) Yi Yao Huang and William Yang Wang. 2017.  Deep residual learning for weakly-supervised relation extraction.  _arXiv preprint arXiv:1707.08866_.
  * Iyyer et al. (2018) Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018.  Adversarial example generation with syntactically controlled paraphrase networks.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1875–1885, New Orleans, Louisiana. Association for Computational Linguistics.
  * Jia and Liang (2017) Robin Jia and Percy Liang. 2017.  Adversarial examples for evaluating reading comprehension systems.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2021–2031, Copenhagen, Denmark. Association for Computational Linguistics.
  * Jin et al. (2020) Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020.  Is BERT really robust? A strong baseline for natural language attack on text classification and entailment.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 8018–8025. AAAI Press.
  * Jones et al. (2020) Erik Jones, Robin Jia, Aditi Raghunathan, and Percy Liang. 2020.  Robust encodings: A framework for combating adversarial typos.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2752–2765, Online. Association for Computational Linguistics.
  * Li et al. (2020) Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020.  BERT-ATTACK: Adversarial attack against BERT using BERT.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6193–6202, Online. Association for Computational Linguistics.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017\.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2980–2988.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Longpre et al. (2021) Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021.  Entity-based knowledge conflicts in question answering.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7052–7063.
  * Minervini and Riedel (2018) Pasquale Minervini and Sebastian Riedel. 2018.  Adversarially regularising neural NLI models to integrate logical background knowledge.  In _Proceedings of the 22nd Conference on Computational Natural Language Learning_ , pages 65–74, Brussels, Belgium. Association for Computational Linguistics.
  * Nan et al. (2021) Guoshun Nan, Jiaqi Zeng, Rui Qiao, Zhijiang Guo, and Wei Lu. 2021.  Uncovering main causalities for long-tailed information extraction.  _arXiv preprint arXiv:2109.05213_.
  * Nguyen and Grishman (2015) Thien Huu Nguyen and Ralph Grishman. 2015.  Relation extraction: Perspective from convolutional neural networks.  In _Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing_ , pages 39–48, Denver, Colorado. Association for Computational Linguistics.
  * Peng et al. (2020) Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2020.  Learning from context or names? an empirical study on neural relation extraction.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3661–3672.
  * Qian et al. (2021) Kun Qian, Ahmad Beirami, Zhouhan Lin, Ankita De, Alborz Geramifard, Zhou Yu, and Chinnadhurai Sankar. 2021.  Annotation inconsistency and entity bias in MultiWOZ.  In _Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue_ , pages 326–337, Singapore and Online. Association for Computational Linguistics.
  * Ribeiro et al. (2018) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018.  Semantically equivalent adversarial rules for debugging NLP models.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 856–865, Melbourne, Australia. Association for Computational Linguistics.
  * Si et al. (2021) Chenglei Si, Ziqing Yang, Yiming Cui, Wentao Ma, Ting Liu, and Shijin Wang. 2021\.  Benchmarking robustness of machine reading comprehension models.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 634–644, Online. Association for Computational Linguistics.
  * Stoica et al. (2021) George Stoica, Emmanouil Antonios Platanios, and Barnabás Póczos. 2021.  Re-tacred: Addressing shotcomings of the tacred dataset.  In _Proceedings of the Thirty-fifth AAAI Conference on Aritificial Intelligence 2021_.
  * Wang et al. (2016) Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. 2016.  Relation classification via multi-level attention CNNs.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1298–1307, Berlin, Germany. Association for Computational Linguistics.
  * Wang et al. (2020) Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao, Daxin Jiang, Ming Zhou, et al. 2020.  K-adapter: Infusing knowledge into pre-trained models with adapters.  _arXiv preprint arXiv:2002.01808_.
  * Wang et al. (2022) Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Dayiheng Liu, Baosong Yang, Juncheng Liu, and Bryan Hooi. 2022.  Should we rely on entity mentions for relation extraction? debiasing relation extraction with counterfactual analysis.  _arXiv preprint arXiv:2205.03784_.
  * Xu et al. (2022) Nan Xu, Fei Wang, Bangzheng Li, Mingtao Dong, and Muhao Chen. 2022.  Does your model classify entities reasonably? diagnosing and mitigating spurious correlations in entity typing.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 8642–8658, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Yamada et al. (2020) Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. 2020\.  LUKE: Deep contextualized entity representations with entity-aware self-attention.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6442–6454, Online. Association for Computational Linguistics.
  * Zhang et al. (2018) Yuhao Zhang, Peng Qi, and Christopher D. Manning. 2018.  Graph convolution over pruned dependency trees improves relation extraction.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2205–2215, Brussels, Belgium. Association for Computational Linguistics.
  * Zhang et al. (2017) Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017\.  Position-aware attention and supervised data improve slot filling.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 35–45.
  * Zhou and Chen (2021) Wenxuan Zhou and Muhao Chen. 2021.  An improved baseline for sentence-level relation extraction.  _arXiv preprint arXiv:2102.01373_.