# Hadronic resonance production with ALICE at the LHC

Sergey Kiselev∗ for the ALICE Collaboration  ∗NRC ”Kurchatov Institute”,
Moscow, Russia<EMAIL_ADDRESS>
###### Abstract

We present recent results on short-lived hadronic resonances obtained by the
ALICE experiment at LHC energies. Results include system-size and collision-
energy evolution of transverse momentum spectra, yields and ratios of
resonance yields to those of longer lived particles, and nuclear modification
factors. The results are compared with model predictions and measurements at
lower energies.

Hadronic resonance production plays an important role both in elementary and
in nucleus-nucleus collisions. In heavy-ion collisions, since the lifetimes of
short-lived resonances are comparable with the lifetime of the late hadronic
phase, regeneration and rescattering effects become important and ratios of
yields of resonances relative to those of longer lived particles can be used
to estimate the time interval between the chemical and kinetic freeze-out. The
measurements in pp and p–Pb collisions constitute a reference for nuclear
collisions and provide information for tuning event generators inspired by
Quantum Chromodynamics.

Results on short-lived mesonic $\rho(770)^{0}$, $\mathrm{K}^{*}(892)^{0}$,
$\mathrm{K}^{*}(892)^{\pm}$, $f_{0}(980)$, $\phi(1020)$ as well as baryonic
$\Sigma(1385)^{\pm}$, $\Lambda(1520)$ and $\Xi(1530)^{0}$ resonances
(hereafter $\rho^{0}$, $\mathrm{K}^{*0}$, $\mathrm{K}^{*\pm}$, $f_{0}$,
$\phi$, $\Sigma^{*\pm}$, $\Lambda^{*}$, $\Xi^{*0}$) have been obtained by the
ALICE experiment. The resonances are reconstructed in their hadronic decay
channels and have very different lifetimes as shown in Table 1.

Table 1: Reconstructed decay mode and lifetime values [1] for hadronic resonances. | $\rho^{0}$ | $\mathrm{K}^{*0}$ | $\mathrm{K}^{*\pm}$ | $f_{0}$ | $\Sigma^{*\pm}$ | $\Lambda^{*}$ | $\Xi^{*0}$ | $\phi$
---|---|---|---|---|---|---|---|---
decay channel | $\pi\pi$ | $\mathrm{K}\pi$ | $\mathrm{K^{0}_{S}}\pi$ | $\pi\pi$ | $\Lambda\pi$ | $p\mathrm{K}$ | $\Xi\pi$ | $\mathrm{K}\mathrm{K}$
lifetime (fm/c) | 1.3 | 4.2 | 4.2 |  $\sim$ 5 | 5-5.5 | 12.6 | 21.7 | 46.2
ALICE papers | [2] |  [3]-[12] | [13] | [14] |  [15]-[17] |  [18]-[19] |  [15]-[16] |  [3]-[12]
|  |  |  |  |  |  |  |

Here we present recent results obtained for $\mathrm{K}^{*0}$ and $\phi$ in
p–Pb at $\sqrt{s_{\rm NN}}$ = 8.16 TeV [11], pp and Pb–Pb at 5.02 TeV [10],
for $\mathrm{K}^{*\pm}$ in pp at 5.02, 8, 13 TeV [13] and Pb–Pb at 5.02 TeV,
for $f_{0}$ in pp [14] and p–Pb at 5.02 TeV, for $\Sigma^{*\pm}$ in Pb–Pb at
5.02 TeV [17], for $\Lambda^{*}$ in pp at 5.02, 13 TeV and Pb–Pb at 5.02 TeV.

Figure 1 shows the transverse momentum spectra for $\Sigma^{*-}$ in Pb–Pb
collisions at $\sqrt{s_{NN}}$ = 5.02 TeV (left) and ratios of the measured
spectra to model predictions (right).

Figure 1: (color online) Left: Transverse momentum spectra for $\Sigma^{*-}$
in different centrality classes of Pb–Pb collisions at $\sqrt{s_{NN}}$ = 5.02
TeV. Right: Ratios of the measured spectra to model predictions from
PYTHIA8/Angantyr [20] and EPOS [21] with and without the UrQMD afterburner.
From [17].

The spectra have been measured for different centralities up to
$p_{\mathrm{T}}=9$ GeV/$c$. Models do not fully describe data.

The mean transverse momenta of $\Lambda^{*}$ as a function of the charged-
particle multiplicity density are shown in Fig. 2 (left).

Figure 2: (color online) Left: The mean transverse momentum of $\Lambda^{*}$
as a function of the charged-particle multiplicity density in Pb–Pb collisions
at $\sqrt{s_{NN}}$ = 5.02 TeV with model predictions from Blast-Wave [22] and
MUSIC [23] with and without the hadronic phase (SMASH off). Results are also
shown for pp at $\sqrt{s}$ = 7 TeV [19], p–Pb at $\sqrt{s_{\rm NN}}$ = 5.02
TeV [19] and Pb–Pb at $\sqrt{s_{\rm NN}}$ = 2.76 TeV with EPOS3 predictions
[18]. Right: The multiplicity-scaled $p_{T}$-integrated yield (a, d) and mean
transverse momentum (c, d) of $\mathrm{K}^{*0}$(a, c) and $\phi$(b, d) as
functions of the charged-particle multiplicity density in p–Pb collisions at
$\sqrt{s_{NN}}$ = 8.16 TeV with model predictions from EPOS-LHC [24], DPMJET
[25] and HIJING [26]. The measurements in pp at $\sqrt{s}$ = 7 [3], 13 [8]
TeV, p–Pb at $\sqrt{s_{\rm NN}}$ = 5.02 TeV [5] and Pb–Pb at $\sqrt{s_{\rm
NN}}$ = 2.76 [4], 5.02 [7] TeV are also shown. From [11].

New data from Pb–Pb collisions at $\sqrt{s_{NN}}$ = 5.02 TeV show larger
values than at $\sqrt{s_{NN}}$ = 2.76 TeV and predictions from the Blast-Wave
model. Predictions from the MUSIC+SMASH model [23], which includes modeling of
the hadronic phase, are consistent with the data in central collisions but
underestimate them in peripheral ones. Models without hadronic afterburner
underestimate the measurements at both energies. Figure 2 (right, bottom)
shows the mean transverse momenta of $\mathrm{K}^{*0}$(c) and $\phi$(d) as a
function of the charged-particle multiplicity density. New data values for
p–Pb at $\sqrt{s_{NN}}$ = 8.16 TeV are close to the values for other light
systems. Among the models, EPOS-LHC [24] gives the best agreement with the
data. In pp and p–Pb collisions, the $\langle p_{\mathrm{T}}\rangle$ rises
faster with multiplicity than in Pb–Pb collisions. An analogous behavior has
been observed in [27] for charged particles and can be understood as the
effect of color reconnection between strings produced in multi-parton
interactions.

Normalized $p_{T}$-integrated yields of $\mathrm{K}^{*0}$ and $\phi$ as a
function of the charged-particle multiplicity density are presented in Fig. 2
(right, top). New data for p–Pb collisions at $\sqrt{s_{NN}}$ = 8.16 TeV
follow the general trend: yields are independent of collision system and
energy and appear to be driven by the event multiplicity. EPOS-LHC predictions
agree with the data.

Figure 3 (left) shows the particle yield ratios
$\mathrm{K}^{*\pm}/\mathrm{K^{0}_{S}}$ as a function of the charged-particle
multiplicity density.

Figure 3: (color online) Particle yield ratios
$\mathrm{K}^{*\pm}/\mathrm{K^{0}_{S}}$ (left) and $\Lambda^{*}/\Lambda$
(right) as a function of the charged-particle multiplicity density in pp, p–Pb
and Pb–Pb collisions with model predictions from PYTHIA6 [28], PYTHIA8 [29],
EPOS-LHC [24], DIPSY [30], $\gamma_{S}$-CSM [31], HRG-PCE [32], MUSIC [23],
THERMUS [33], GSI-Heidelberg [34] and SHARE3 [35]. STAR data [36] are also
shown for $\Lambda^{*}/\Lambda$ ratios.

The ratio is significantly suppressed, by about 55%, going from peripheral to
central Pb–Pb collisions. This suppression is consistent with rescattering of
$\mathrm{K}^{*\pm}$ decay products in the hadronic phase of central collisions
as the dominant effect. Models with rescattering effect (MUSIC+SMASH and HRG-
PCE) qualitatively describe the data. There is a hint of decrease of the ratio
with increasing multiplicity in pp collisions. Among the models, EPOS-LHC
gives the best agreement with the data. The $\mathrm{K}^{*\pm}$ measurement is
consistent with previous results for $\mathrm{K}^{*0}$ [8]. It is worth noting
that the systematic uncertainties of charged $\mathrm{K}^{*}$ are smaller than
those of neutral $\mathrm{K}^{*}$.

The particle yield ratios $\Lambda^{*}/\Lambda$ are illustrated in Fig. 3
(right). The new measurements in Pb–Pb collisions at $\sqrt{s_{NN}}$ = 5.02
TeV follow the suppression trend observed for Pb–Pb at $\sqrt{s_{NN}}$ = 2.76
TeV [18]. The suppression, of about 70%, is larger than for
$\mathrm{K}^{*\pm}/\mathrm{K^{0}_{S}}$ although $\tau(\Lambda^{*})\approx
3\tau(\mathrm{K}^{*\pm})$. The difference might also be explained by a
difference in the interaction cross sections of the resonance decay products
with pions, the most abundant species in the hadron gas. MUSIC with SMASH
afterburner reproduces the suppression trend. All thermal models overestimate
the ratio in central Pb–Pb collisions. A multiplicity-dependent suppression is
not observed in the new data for pp collisions at $\sqrt{s}$ = 5.02 and 13
TeV. ALICE results confirm the trend seen by STAR at 200 GeV.

Figure 4 (left) shows the particle yield ratios $\Sigma^{*\pm}/\pi$ as a
function of the charged-particle multiplicity density in pp, p–Pb and Pb–Pb
collisions.

Figure 4: (color online) Left: Particle yield ratios $\Sigma^{*\pm}/\pi$ as a
function of the charged-particle multiplicity density in pp, p–Pb and Pb–Pb
collisions with model predictions from PYTHIA/Angantyr [20], EPOS [21] and
GSI-Heidelberg [34]. STAR data [36] are also shown. From [17]. Right: Double
ratio of particle yields to pion yields as a function of multiplicity in p–Pb
collisions at $\sqrt{s_{\mathrm{NN}}}$ = 5.02 TeV.

New results for Pb–Pb at $\sqrt{s_{NN}}$ = 5.02 TeV do not exhibit particular
trend with multiplicity [17]. There is a hint of some suppression at the
highest multiplicity but future higher precision measurements are needed. EPOS
with UrQMD and PYTHIA/Angantyr models reproduce qualitatively multiplicity
dependence. The GSI-Heidelberg thermal model overestimates the ratio in
central Pb–Pb collisions. In pp and p–A collisions, ALICE results are close to
STAR data.

To investigate the structure of $f_{0}$, the new ALICE measurements of the
$f_{0}$ yields are compared to those of other hadrons and resonances with
similar mass. Figure 4 (right) shows the double ratio of particle yields to
pion yields as a function of multiplicity in p–Pb collisions at
$\sqrt{s_{\mathrm{NN}}}$ = 5.02 TeV. Strangeness enhancement with multiplicity
could explain the increase with multiplicity of the $\phi/\pi$ ratios. The
$\mathrm{K}^{*0}/\pi$ ratios demonstrate the competition between strangeness
enhancement and rescattering effect. The $f_{0}/\pi$ suppression shows that
rescattering is the effect that dominantly affects the yield at low $p_{T}$.
Notably, the $\gamma_{S}-$CSM model prediction for the $f_{0}/\pi$ ratio
assuming zero net strangeness of $f_{0}$ is consistent with the data within
1.9$\sigma$ [14].

Figure 5 presents the nuclear modification factor $R_{AA}$ of
$\mathrm{K}^{*0}$ and $\phi$ for Pb–Pb collisions. New results at
$\sqrt{s_{NN}}$ = 5.02 TeV are comparable to the corresponding measurements at
$\sqrt{s_{NN}}$ = 2.76 TeV [6].

Figure 5: (color online) The nuclear modification factor $R_{AA}$ for
$\mathrm{K}^{*0}$ (top) and $\phi$ (bottom) as a function of transverse
momentum in Pb–Pb collisions in different centralities. From [10].

Significant energy dependence of $R_{AA}$ is not observed.

In summary, recent results on short-lived hadronic resonances obtained by the
ALICE experiment in pp, p–Pb and Pb–Pb collisions at the LHC energies have
been presented. In pp and p–Pb collisions the $\langle p_{\mathrm{T}}\rangle$
values for the $\mathrm{K}^{*0}$ and $\phi$ resonances rise faster with
multiplicity than in Pb–Pb collisions. One possible explanation could be the
effect of color reconnection between strings produced in multi-parton
interactions. Yields of $\mathrm{K}^{*0}$ and $\phi$ are independent of the
collision system and energy and appear to be driven by the event multiplicity.
The $\mathrm{K}^{*\pm}/\mathrm{K^{0}_{S}}$, and $\Lambda^{*}/\Lambda$ ratios
exhibit a significant suppression going from peripheral to central Pb–Pb
collisions, consistent with rescattering of the decay products of the short-
lived resonances in the hadronic phase. There is a hint of suppression for the
$\Sigma^{*\pm}/\pi$ ratio but future higher precision measurements are needed.
The suppression is qualitatively described by models with rescattering. For
light systems there are hints of suppression for
$\mathrm{K}^{*\pm}/\mathrm{K^{0}_{S}}$ and $f_{0}/\pi$ ratios and no
suppression for $\Lambda^{*}/\Lambda$. No significant energy dependence of
$R_{AA}$ for $\mathrm{K}^{*0}$ and $\phi$ in Pb–Pb collisions is observed.

## References

## References

  * [1] Workman R L et al. (Particle Data Group) 2022 Prog. Theor. Exp. Phys. 2022 083C01
  * [2] Acharya S et al. (ALICE Collaboration) 2019 Phys. Rev. C 99 064901
  * [3] Abelev B et al. (ALICE Collaboration) 2012 Eur. Phys. J. C 72 2183
  * [4] Abelev B et al. (ALICE Collaboration) 2015 Phys. Rev. C 91 024609
  * [5] Adam J et al. (ALICE Collaboration) 2016 Eur. Phys. J. C 76 245
  * [6] Adam J et al. (ALICE Collaboration) 2017 Phys. Rev. C 95 064606
  * [7] Acharya S et al. (ALICE Collaboration) 2020 Phys. Lett. B 802 135225
  * [8] Acharya S et al. (ALICE Collaboration) 2020 Phys. Lett. B 807 135501
  * [9] Acharya S et al. (ALICE Collaboration) 2020 Phys. Rev. C 102 024912
  * [10] Acharya S et al., (ALICE Collaboration), arXiv:2106.13113 [nucl-ex]
  * [11] Acharya S et al., (ALICE Collaboration), arXiv:2110.10042 [nucl-ex]
  * [12] ALICE Collaboration, arXiv:2204.10263 [nucl-ex]
  * [13] Acharya S et al. (ALICE Collaboration) 2022 Phys. Lett. B 828 137013
  * [14] ALICE Collaboration, arXiv:2206.06216 [nucl-ex]
  * [15] Abelev B et al. (ALICE Collaboration) 2015 Eur. Phys. J. C 75 1
  * [16] Adamova D et al. (ALICE Collaboration) 2017 Eur. Phys. J. C 77 389
  * [17] ALICE Collaboration, arXiv:2205.13998 [nucl-ex]
  * [18] Acharya S et al. (ALICE Collaboration) 2019 Phys. Rev. C 99 024905
  * [19] Acharya S et al. (ALICE Collaboration) 2020 Eur. Phys. J. C 80 160
  * [20] Bierlich C, Gustafson G, Lönnblad L, and Shah H 2018 J. High Energy Phys. 10 134
  * [21] Knospe A G et al. 2016 Phys. Rev. C 93 014911
  * [22] Abelev B et al. (ALICE Collaboration) 2014 Phys. Lett. B 728 25
  * [23] Oliinychenko D and Shen C arXiv:2105.07539 [hep-ph]
  * [24] Pierog T. et al. 2016 Phys. Rev. C 92 034906
  * [25] Roesler S, Engel R, and Ranft J arXiv:0012252 [hep-ph]
  * [26] Gyulassy M and Wang X-N 1994 Comput. Phys. Commun. 83 307
  * [27] Abelev B et al. (ALICE Collaboration) 2013 Phys. Lett. B 727 371
  * [28] Skands P Z 2010 Phys. Rev. D 82 074018
  * [29] Skands P Z et al. 2014 Eur. Phys. J. C 74 3024
  * [30] Flensburg C et al. 2011 J. High Energy Phys. 8 103
  * [31] Vovchenko V, Donigus B, and Stoecker H 2019 Phys. Rev. C 100 054906
  * [32] Motornenko A et al. 2020 Phys. Rev. C 102 024909
  * [33] Wheaton S et al. 2009 Comput. Phys. Commun. 180 84\.
  * [34] Stachel J et al. 2014 J. Phys.: Conf. Ser. 509 012019\.
  * [35] Petran M et al. 2014 Comput. Phys. Commun. 185 2056.
  * [36] Abelev B et al., (STAR Collaboration) 2008 Phys. Rev. C 78 044906\.


# Flexible Instance-Specific Rationalization of NLP Models

George Chrysostomou, Nikolaos Aletras

###### Abstract

Recent research on model interpretability in natural language processing
extensively uses feature scoring methods for identifying which parts of the
input are the most important for a model to make a prediction (i.e.
explanation or rationale). However, previous research has shown that there is
no clear best scoring method across various text classification tasks while
practitioners typically have to make several other ad-hoc choices regarding
the length and the type of the rationale (e.g. short or long, contiguous or
not). Inspired by this, we propose a simple yet effective and flexible method
that allows selecting optimally for each data instance: (1) a feature scoring
method; (2) the length; and (3) the type of the rationale. Our method is
inspired by input erasure approaches to interpretability which assume that the
most faithful rationale for a prediction should be the one with the highest
difference between the model’s output distribution using the full text and the
text after removing the rationale as input respectively. Evaluation on four
standard text classification datasets shows that our proposed method provides
more faithful, comprehensive and highly sufficient explanations compared to
using a fixed feature scoring method, rationale length and type. More
importantly, we demonstrate that a practitioner is not required to make any
ad-hoc choices in order to extract faithful rationales using our
approach.111Code for experiments available at:
https://github.com/GChrysostomou/instance-specific-rationale

##  1 Introduction

Large pre-trained transformer-based language models such as BERT (Devlin et
al. 2019; Bommasani et al. 2021), currently dominate performance across
language understanding benchmarks (Wang et al. 2019). These developments have
opened up new challenges on how to extract faithful explanations (i.e.
rationales222We use these terms interchangeably throughout the paper.), which
accurately represent the true reasons behind a model’s prediction when adapted
to downstream tasks (Jacovi and Goldberg 2020).

Recent studies use feature scoring (i.e. attribution) methods such as gradient
and attention-based scores (Arras et al. 2016; Sundararajan, Taly, and Yan
2017; Jain and Wallace 2019; Chrysostomou and Aletras 2021b) to identify
important (i.e. salient) segments of the input to subsequently extract them as
rationales (Jain et al. 2020; Treviso and Martins 2020). However, a single
feature scoring method is typically applied across the whole dataset (i.e.
globally). This might not be optimal for individual instances resulting into
less faithful explanations (Jacovi and Goldberg 2020; Atanasova et al. 2020).
Additionally, rationales are usually extracted using a pre-defined fixed
length (i.e. the ratio of a rationale compared to the full input sequence) and
type (i.e. top $k$ terms or contiguous) globally. We hypothesize that using a
fixed length or type for different instances could result into shorter (i.e.
not sufficient for explaining a model’s prediction) or longer than needed
rationales reducing rationale faithfulness, whilst finding the explanation
length is an open problem (Zhang et al. 2021). Moreover to extract rationales,
practitioners are currently required to make assumptions for the rationale
parameters (i.e. feature scoring method, length and type), whilst different
choice of parameters might substantially affect the faithfulness of the
rationales.

In this paper, we propose a simple yet effective method that operates at
instance-level and mitigates the a priori selection of a specific (1) feature
scoring method; (2) length and (3) type when extracting faithful rationales.
Our proposed method is flexible and allows the automatic selection of some of
these instance-specific parameters or all. Inspired by erasure methods, it
functions by computing differences between a model’s output distributions
obtained using the full input sequence and the input without the rationale
respectively. We base this on the assumption that by removing important tokens
from the sequence, we should observe large divergences in the model’s
predicted distribution (Nguyen 2018; Serrano and Smith 2019; DeYoung et al.
2020) resulting into more faithful rationales (Atanasova et al. 2020; Chen and
Ji 2020). The contributions of our work are thus as follows:

  * •

To the best of our knowledge, we are the first to propose a method for
instance-specific faithful rationale extraction;

  * •

We empirically demonstrate that rationales extracted with instance-specific
flexible feature scoring method, length and type using our proposed method are
more comprehensive than rationales with fixed, pre-defined parameters;

  * •

We show that our method results in consistently highly sufficient rationales,
mitigating the variability in faithfulness of different feature scoring
methods across datasets when used globally, i.e. the same for all instances
(Atanasova et al. 2020).

##  2 Background and Related Work

### Rationale Extraction

Given a trained model $\mathcal{M}$, an input
$\mathbf{x}=[x_{1},\ldots,x_{T}]$ and a predicted distribution over classes
$\mathcal{Y}$, rationale extraction methods seek to identify the most
important subset $\mathcal{R}\in\mathbf{x}$ of the input for explaining the
model’s prediction.

There are two common approaches for extracting rationales. The first consists
of two modules jointly trained on an end-task, e.g. sentiment analysis (Lei,
Barzilay, and Jaakkola 2016; Bastings, Aziz, and Titov 2019). The first module
extracts the rationale (i.e. typically by learning to select which inputs
should be masked) and the second module is trained using only the rationale.
The second approach consists of using feature scoring (or attribution) methods
(i.e. salience metrics) to first identify important parts of the input and
then extract the rationales from $\mathcal{M}$ (Jain et al. 2020; Treviso and
Martins 2020; DeYoung et al. 2020). A limitation of the first approach is that
the models are hard to train compared to the latter and often do not reach
high accuracy (Jain et al. 2020). Regarding the latter approach, a limitation
is that the same feature scoring method is applied to all instances in a given
dataset, irrespective of whether a feature scoring method is not the best for
a particular instance (Atanasova et al. 2020; Jacovi and Goldberg 2020) while
finding a suitable explanation length is an open problem (Zhang et al. 2021).

### Computing Input Importance

Feature scoring methods $\Omega$ compute input importance scores
$\boldsymbol{\omega}$ for each token in the sequence $\mathbf{x}$, such that
$\boldsymbol{\omega}=\Omega(\mathcal{M},\mathbf{x},\mathcal{Y})$. High scores
indicate that the associated tokens contributed more towards a model’s
prediction. Subsequently, $\mathcal{R}$ is extracted by selecting the $K$
highest scored tokens (or $K$-gram for contiguous) in a sequence (DeYoung et
al. 2020; Jain et al. 2020).

A common approach to computing $\boldsymbol{\omega}$ is by calculating the
gradients of the prediction with respect to the input (Kindermans et al. 2016;
Li et al. 2016; Arras et al. 2016; Sundararajan, Taly, and Yan 2017; Bastings
and Filippova 2020). Jain et al. (2020) use attention weights to attribute
token importance for rationale extraction, while Treviso and Martins (2020)
propose sparse attention. Li et al. (2016) compute input importance scores by
measuring the difference in a model’s prediction between keeping and omitting
each token, with Kim et al. (2020) also suggesting input marginalization as an
alternative to token omission. Another way is using sparse linear meta-models
that are easier to interpret (Ribeiro, Singh, and Guestrin 2016). Atanasova et
al. (2020) however show that sparse linear meta-models are not as faithful as
gradient-based approaches for interpreting large language models.

### Evaluating Rationale Faithfulness

Having extracted $\mathcal{R}$, we typically need to evaluate how faithful
that explanation is for a model’s prediction. Several studies evaluate the
faithfulness of explanations by training a separate classifier on an end-task
using only the rationales as input (Jain et al. 2020; Treviso and Martins
2020). These classifiers are inherently faithful, as they are trained only on
the rationales (Jain et al. 2020). Other studies compare the ability of
different feature scoring methods to identify important tokens by using word
erasure, i.e. masking (Samek et al. 2017; Serrano and Smith 2019; Atanasova et
al. 2020; Chen and Ji 2020; DeYoung et al. 2020; Zhang et al. 2021;
Chrysostomou and Aletras 2021a). The intuition is that by removing the most
important tokens, it should result in a larger difference in the output
probabilities, compared to removing a less important token which will also
lead to drops in classification accuracy (Robnik-Šikonja and Kononenko 2008;
Nguyen 2018; Atanasova et al. 2020). DeYoung et al. (2020) use erasure to
evaluate the comprehensiveness and sufficiency of rationales. Carton, Rathore,
and Tan (2020) suggest normalizing these metrics using the predictions of the
model with a baseline input, to allow for a fairer comparison across models
and datasets.

##  3 Instance-Specific Rationale Extraction

Our aim is to address the _“one-size-fits-all”_ ad-hoc approach of previous
work on rationale extraction with feature scoring methods that typically
extracts rationales using the same feature scoring method, length and type
across all instances in a dataset. Inspired by word erasure approaches (Nguyen
2018; Serrano and Smith 2019; DeYoung et al. 2020) we mask the tokens that
constitute a rationale and record the difference $\delta$ in a model’s output
distribution by using the full text and the reduced input. Our main assumption
is that a sufficiently faithful rationale is the one that will result into the
largest $\delta$ (Atanasova et al. 2020; Chen and Ji 2020; DeYoung et al.
2020). Following this assumption, we can extract rationales by selecting for
each instance a specific (1) _feature scoring method_ ; (2) _length_ ; and (3)
_type_.333Similar to Jain et al. (2020), we consider two rationale types: (a)
TopK tokens ranked by a feature scoring method, treating each word in the
input sequence independently; and (b) Contiguous span of input tokens of
length _K_ with the highest overall score computed by a feature scoring
method.

### Instance-level Feature Scoring Selection

Given a set of $M$ feature scoring methods
$\\{\Omega_{1},\ldots,\Omega_{M}\\}$, we extract a rationale $\mathcal{R}$ as
follows:

  1. 1.

For each $\Omega_{i}$ in the set we compute input importance scores
$\boldsymbol{\omega}_{i}=\Omega_{i}(\mathcal{M},\mathbf{x},\mathcal{Y})$;

  2. 2.

We subsequently select the $K$ highest scored tokens (TopK) or the highest
$K$-gram (Contiguous) to form a rationale $\mathcal{R}_{i}$, where $K$ is the
rationale length;

  3. 3.

For each rationale we compute the difference $\delta_{i}$, between the
reference model output (using full text input) and the model output having
masked the rationale, such that:

$\delta_{i}=\Delta(\mathcal{Y},\mathcal{Y}^{m}_{i})=\Delta(\mathcal{M}(\mathbf{x}),\mathcal{M}(\mathbf{x}_{\backslash\mathcal{R}_{i}}))$

where $\Delta$ is the function used to compute the difference between the two
outputs;

  4. 4.

We select the rationale $\mathcal{R}$ with the highest difference
$\delta_{max}=max(\\{\delta_{1},\ldots,\delta_{i},\ldots,\delta_{M}\\})$.

For computing $\delta$, we experiment with the following divergence metrics
($\Delta$): (a) Kullback-Leibler (KL); (b) Jensen-Shannon divergence (JSD);
(c) Perplexity (Perp.) and (d) Predicted Class Probability (ClassDiff).444We
describe the metrics in detail in App. B.

### Instance-level Rationale Length Selection

For computing at instance-level the rationale length $k$ and extracting the
rationale $R$ using a single feature scoring method $\Omega$, we propose the
following steps:

  1. 1.

Given $\Omega$, we first compute input importance scores
$\boldsymbol{\omega}=\Omega(\mathcal{M},\mathbf{x},\mathcal{Y})$;

  2. 2.

We then iterate over the sequence such that $k=range(1,N)$, where $N$ is the
fixed, pre-defined rationale length and $k$ the possible rationale length at
the current iteration. We set $N$ as the upper bound rationale length for our
approach to make results comparable with fixed length rationales.

  3. 3.

At each iteration we begin by masking the top $k$ tokens (as indicated by
$\omega$) to form a candidate rationale $\mathcal{R}_{k}$. When using TopK we
mask the $k$ highest scored tokens, whilst with Contiguous we mask the highest
scored $k$-gram;

  4. 4.

We compute the difference $\delta_{k}$ between the reference model output
$\mathcal{Y}$ and the model output having masked the candidate rationale
$\mathcal{Y}^{m}_{k}=\mathcal{M}(\mathbf{x}_{\backslash\mathcal{R}_{k}})$;

  5. 5.

We record every $\delta$ until $k=N$ and extract the rationale $\mathcal{R}$
with the highest difference
$\delta_{max}=max(\\{\delta_{1},\ldots,\delta_{k},\ldots,\delta_{N}\\})$,
where $k$ at $\delta_{max}$ is the computed rationale length.555We also
experimented with early stopping, whereby the difference between $\delta_{k}$
and the $\delta_{max}$ until $k$ are under a specified threshold, however this
resulted in reduced performance (we included a more thorough analysis in the
App. D).

### Instance-level Rationale Type Selection

In a similar way to selecting a feature scoring method, our approach can also
be used to select between different rationale types (i.e. Contiguous or TopK)
for each instance in the dataset.

Finally, our approach is flexible and can be easily modified to support
selecting any of these parameters while keeping the rest fixed (i.e. feature
scoring method, rationale length and rationale type) or by selecting any
combination of them. An important benefit of our approach is that we extract
rationales with different settings for each instance rather than using uniform
settings globally (i.e. across the whole dataset), which we empirically
demonstrate to be beneficial for faithfulness (§5).

##  4 Experimental Setup

### Tasks

For our experiments we use the following datasets (details in Table 1):

  * •

SST: Binary sentiment classification without neutral sentences (Socher et al.
2013).

  * •

AG: News articles categorized in Science, Sports, Business, and World topics
(Corso, Gulli, and Romani 2005).

  * •

Evidence Inference (Ev.Inf.): Abstract-only biomedical articles describing
randomized controlled trials. The task is to infer the relationship between a
given intervention and comparator with respect to an outcome (Lehman et al.
2019).

  * •

MultiRC (M.Rc): A reading comprehension task with questions having multiple
correct answers that depend on information from multiple sentences (Khashabi
et al. 2018). Following DeYoung et al. (2020) and Jain et al. (2020), we
convert this to a binary classification task where each
rationale/question/answer triplet forms an instance and each candidate answer
is labeled as True/False

### Models

Similar to Jain et al. (2020), we use BERT (Devlin et al. 2019) for SST and
AG); SciBERT (Beltagy, Lo, and Cohan 2019) for Ev.Inf. and Roberta (Liu et al.
2019) for M.RC. See App. A for hyperparameters.

Data | $|W|$ | C |  | Splits
---
Train/Dev/Test
F1 | $N$
SST | 18 | 2 | 6,920 / 872 / 1,821 | 90.1 $\pm$ 0.2 | 20%
AG | 36 | 4 | 102,000 / 18,000 / 7,600 | 93.5 $\pm$ 0.2 | 20%
Ev.Inf. | 363 | 3 | 5,789 / 684 / 720 | 83.0 $\pm$ 1.6 | 10%
M.RC | 305 | 2 | 24,029 / 3,214 / 4,848 | 73.2 $\pm$ 1.7 | 20%
Table 1: Dataset statistics including average words at instance ($|W|$),
number of classes (C), data splits, F1 macro performance and the fixed, pre-
defined rationale ratio across all instances ($N$).

### Feature Scoring Methods

We use a random baseline and six other feature scoring methods (to compute
input importance scores) similar to Jain et al. (2020) and Serrano and Smith
(2019).

  * •

Random (Rand):  Random allocation of token importance.

  * •

Attention ($\boldsymbol{\alpha}$): Token importance corresponding to
normalized attention scores (Jain et al. 2020).

  * •

Scaled Attention ($\boldsymbol{\alpha}\nabla\boldsymbol{\alpha}$): Scales the
attention scores $\alpha_{i}$ with their corresponding gradients
$\nabla\alpha_{i}=\frac{\partial\hat{y}}{\partial\alpha_{i}}$ (Serrano and
Smith 2019) .

  * •

InputXGrad ($\mathbf{x}\nabla\mathbf{x}$): Attributes input importance by
multiplying the gradient of the input by the input with respect to the
predicted class, where $\nabla x_{i}=\frac{\partial\hat{y}}{\partial x_{i}}$
(Kindermans et al. 2016; Atanasova et al. 2020) .

  * •

Integrated Gradients ($\mathbf{IG}$): Ranking words by computing the integral
of the gradients taken along a straight path from a baseline input (zero
embedding vector) to the original input (Sundararajan, Taly, and Yan 2017).

  * •

DeepLift: Ranking words according to the difference between the activation of
each neuron to a reference activation (Shrikumar, Greenside, and Kundaje
2017).

  * •

LIME: Ranking words by learning an interpretable model locally around the
prediction (Ribeiro, Singh, and Guestrin 2016).

### Evaluating Explanation Faithfulness

  * •

F1 macro: Similar to Arras et al. (2017) we measure the F1 macro performance
of model $\mathcal{M}$ when masking the rationale in the original input
($\mathbf{x}_{\setminus\mathcal{R}}$). A key difference in our approach is
that we use the predicted labels of the model with full input as gold labels,
as we are interested in the faithfulness of explanations for the predictions
of the model. Larger drops in F1 scores indicate that the extracted rationale
is more faithful.666We also conducted experiments using the dataset gold
labels with results being comparable. We include all results in App. G.

  * •

Normalized Sufficiency (NormSuff):  We measure the degree to which the
extracted rationales are sufficient for a model to make a prediction (DeYoung
et al. 2020). Similar to Carton, Rathore, and Tan (2020) we bind sufficiency
between 0 and 1 and use the reverse difference so that higher is better. We
modify this metric and measure the normalized sufficiency (Carton, Rathore,
and Tan 2020) such that:

$\displaystyle\text{Suff}(\mathbf{x},\hat{y},\mathcal{R})=1-max(0,p(\hat{y}|\mathbf{x})-p(\hat{y}|\mathcal{R}))$
(1)
$\displaystyle\text{NormSuff}(\mathbf{x},\hat{y},\mathcal{R})=\frac{\text{Suff}(\mathbf{x},\hat{y},\mathcal{R})-\text{Suff}(\mathbf{x},\hat{y},0)}{1-\text{Suff}(\mathbf{x},\hat{y},0)}$

where $\text{Suff}(\mathbf{x},\hat{y},0)$ is the sufficiency of a baseline
input (zeroed out sequence) and $\hat{y}$ the model predicted class using the
full text $\mathbf{x}$ as input, such that $\hat{y}=\text{arg
max}(\mathcal{Y})$.

  * •

Normalized Comprehensiveness (NormComp):  We measure the extent to which a
rationale is needed for a prediction (DeYoung et al. 2020). For an explanation
to be highly comprehensive, the model’s prediction when masking the rationale
should have a high difference between the model’s prediction with full text.
Similarly to Carton, Rathore, and Tan (2020) we bind this metric between 0 and
1 and normalize it. We compute it by:

$\displaystyle\text{Comp}(\mathbf{x},\hat{y},\mathcal{R})=max(0,p(\hat{y}|\mathbf{x})-p(\hat{y}|\mathbf{x}_{\backslash\mathcal{R}}))$
(2)
$\displaystyle\text{NormComp}(\mathbf{x},\hat{y},\mathcal{R})=\frac{\text{Comp}(\mathbf{x},\hat{y},\mathcal{R})}{1-\text{Suff}(\mathbf{x},\hat{y},0)}$

We do not conduct human experiments to evaluate explanation faithfulness since
that is only relevant to explanation plausibility (i.e. how understandable by
humans a rationale is (Jacovi and Goldberg 2020)) and in practice faithfulness
and plausibility do not correlate (Atanasova et al. 2020). Finally we do not
compare with select-then-predict methods (Lei, Barzilay, and Jaakkola 2016;
Jain et al. 2020), as we are interested in faithfully explaining the model
$\mathcal{M}$ and not forming inherently faithful classifiers.

### Performance-Time Trade-off

Input erasure approaches typically require $N$ forward passes to compute a
rationale length (see §3) when removing one token at a time. Similar to Nguyen
(2018); Atanasova et al. (2020), we expedite this process when selecting a
rationale length by “skipping” every $X$% of tokens. For our work, we use a 2%
skip rate which led to a seven-fold reduction in the time required to compute
rationales for datasets comprising of long sequences, such as MRc and EvInf,
with comparable performance in faithfulness to the slower process of removing
one token at a time. We include in App. E the performance/skip-rate trade-off
when (1) we do not use a skip-rate; (2) at 2% and (3) at 5%.

##  5 Results

(a) F1 macro

(b) NormSuff

(c) NormComp

Figure 1: F1 macro (lower is better), mean NormSuff (higher is better) and
mean NormComp (higher is better), when using any single feature scoring method
across all instances in a dataset and our proposed method of selecting a
feature scoring method for each instance (OURS) for TopK rationale types.

### Selecting Instance-specific Feature Scoring

Figure 1 compares the faithfulness of extracted rationales when using our
proposed method for selecting an instance-specific feature scoring method
(OURS) and our baselines, that use a single fixed pre-defined feature scoring
method globally (i.e. across all instances in a dataset). We measure
faithfulness using F1 macro (lower is better), mean NormSuff and mean NormComp
(higher is better respectively). For clarity we show results using the TopK
rationale type, with results for Contiguous included in App. G.777Also for
clarity, all results presented in this work are using JSD for $\Delta$. The
other divergence functions performed comparably and we include a comparison
between them in App. F.

Overall, results demonstrate that rationales extracted with our proposed
approach are highly sufficient and comprehensive. In fact, our approach
results in more sufficient rationales against all single feature scoring
methods in AG and is comparable with the best NormSuff scores in the remainder
of the datasets. This suggests that even when rationales with our proposed
method are not the most sufficient, they are consistently highly sufficient
(i.e. rationales extracted with our approach are significantly more sufficient
than fixed, pre-defined feature scoring methods in 18 out of 24 test cases).
Compared to our six baselines, the rationales extracted with our approach are
significantly more comprehensive across all four datasets (Wilcoxon Rank Sum,
$p<.05$). Additionally, the larger drops in F1 macro performance demonstrate
that rationales extracted with our proposed approach are more necessary for a
model to make a prediction compared to a globally used, pre-defined feature
scoring approach.

Our results strengthen the hypothesis that whilst some feature scoring methods
are better than others globally, they might not be optimal for all instances
in a dataset (Jacovi and Goldberg 2020) and our approach helps mitigate that.
Similar to (Atanasova et al. 2020), we observe that the faithfulness
performance of single feature scoring methods varies across datasets. For
example LIME returns more comprehensive rationales than $\alpha\nabla\alpha$
in MultiRC, however is outperformed by the latter in SST. By returning
consistently highly comprehensive and sufficient rationales, our propose
method helps reducing the variability in faithfulness performance observed
when using any single feature scoring method across datasets.

### Selecting Instance-specific Rationale Length

Table 2 shows the Relative Improvement (R.I.) ratio in mean NormSuff and
NormComp ($>$1.0 is better) between rationales extracted using a fixed pre-
defined length (see $N$ in Table 1) and rationales extracted using our method
with instance-specific length across feature scoring methods and datasets. For
brevity the detailed results, including F1 macro where we make similar
observations to comprehensiveness, are included in App. G. Overall, rationales
extracted using our approach are on average shorter than fixed length
rationales. Specifically, rationale length drops from 20% to 16% on average in
SST, AG; from 20% to 15% in M.Rc and from 10% to 7% in Ev.Inf..888We include
the computed rationale length results in App. C and the same evaluation under
2 $\times N$ in App. G.

NormSuff scores indicate that our shorter on average rationales are overall
less but comparably sufficient with longer, fixed-length rationales. For
example with SST rationales with instance-specific length are 0.9-1.0 times
less sufficient that rationale with pre-defined length. We find this
particularly evident in datasets such as M.Rc and Ev.Inf., where our
rationales are on average 4-5% shorter (approximately 15 tokens shorter on
average for $\alpha$ in M.Rc) but still retain comparable sufficiency, while
in some cases improving it (e.g. 1.2 R.I. in Ev.Inf. with DeepLift).

We also note that rationales extracted with instance-specific length are more
comprehensive in most cases, despite being shorter on average compared to
fixed-length rationales. For example in Ev.Inf., Contiguous rationales with
I.G. are 1.4 times more comprehensive when we select their length at instance-
level. Results also indicate that using our proposed method benefits more
Contiguous rationales compared to TopK for comprehensiveness, leading to
increased R.I. in the majority of cases. Overall, findings support our initial
hypothesis that in certain cases a rationale with longer than needed length
might contain unnecessary information and adversely impact its
comprehensiveness.

|  | NormSuff | NormComp
---|---|---|---
| Feat | SST | M.Rc | AG | Ev.Inf. | SST | M.Rc | AG | Ev.Inf.
TopK | DeepLift | 0.9 | 0.8 | 0.8 | 1.1 | 0.8 | 1.1 | 1.0 | 1.0
LIME | 1.0 | 0.7 | 0.9 | 0.9 | 0.9 | 1.1 | 1.0 | 1.0
$\boldsymbol{\alpha}$ | 0.9 | 0.9 | 0.7 | 0.8 | 0.8 | 1.1 | 0.9 | 1.2
$\boldsymbol{\alpha\nabla\alpha}$ | 0.9 | 0.9 | 0.8 | 0.9 | 1.0 | 1.1 | 0.9 | 1.0
IG | 0.9 | 0.9 | 0.8 | 0.9 | 0.9 | 1.1 | 1.0 | 1.1
$\boldsymbol{\mathbf{x}\nabla\mathbf{x}}$ | 1.0 | 0.8 | 0.7 | 0.8 | 0.9 | 1.1 | 0.9 | 1.2
Contiguous | DeepLift | 0.9 | 0.9 | 0.8 | 1.2 | 0.9 | 1.1 | 1.3 | 1.5
LIME | 0.9 | 0.7 | 0.8 | 0.9 | 1.0 | 1.1 | 1.2 | 1.3
$\boldsymbol{\alpha}$ | 0.9 | 0.9 | 0.7 | 0.9 | 0.7 | 1.1 | 1.0 | 1.2
$\boldsymbol{\alpha\nabla\alpha}$ | 0.9 | 0.8 | 0.8 | 0.9 | 1.0 | 1.1 | 1.1 | 1.1
IG | 0.9 | 0.8 | 0.8 | 1.0 | 1.0 | 1.2 | 1.2 | 1.4
$\boldsymbol{\mathbf{x}\nabla\mathbf{x}}$ | 0.9 | 0.8 | 0.7 | 1.0 | 1.0 | 1.1 | 1.0 | 1.3
Table 2: Relative Improvement (R.I.) ratios for mean NormSuff and mean NormComp between fixed length rationales (see $N$ in Table 1) extracted using our method and rationales with instance-specific length ($>$1.0 is better). |  |  | NormSuff | NormComp
---|---|---|---|---
Type | Len | Feat | SST | M.Rc | AG | Ev.Inf. | SST | M.Rc | AG | Ev.Inf.
TopK | Fix | Fix | .68 | .12 | .37 | .43 | .54 | .42 | .28 | .80
I-L | Fix | .61 | .11 | .30 | .37 | .52 | .46 | .27 | .82
Fix | I-L | .63 | .09 | .44 | .38 | .57 | .59 | .41 | .84
| I-L | I-L | .59 | .07 | .38 | .36 | .55 | .62 | .39 | .86
Cont. | Fix | Fix | .71 | .07 | .41 | .85 | .46 | .47 | .17 | .55
I-L | Fix | .63 | .06 | .33 | .78 | .47 | .54 | .19 | .62
Fix | I-L | .67 | .07 | .42 | .82 | .46 | .60 | .22 | .59
| I-L | I-L | .61 | .05 | .33 | .76 | .48 | .65 | .24 | .67
I-L | I-L | I-L | .60 | .06 | .39 | .49 | .57 | .69 | .41 | .88
Table 3: Mean NormSuff and NormComp scores when we select at instance-level
(I-L) a combination of the: (1) rationale length (Len); (2) feature scoring
method (Feat.); and (3) rationale type (Type). {Type}-Fix-Fix and {Type}-I-L-
Fix values are from the highest scoring feature scoring method (see Figure 1).
Bold values denote the highest performing combination in column-wise (higher
is better).

(a) F1 macro

(b) NormSuff

(c) NormComp

Figure 2: F1 macro (lower is better), mean NormSuff and mean NormComp (higher
is better), when extracting rationales with our approach given decreasing
numbers of feature scoring methods.

### Selecting Instance-specific Feature Scoring, Length and Type

Table 3 shows mean NormSuff and NormComp scores when using our proposed method
to select at instance-level (I-L) a combination of: (1) the feature scoring
method (Feat); (2) the rationale length (Len); and (3) the rationale type
(Type). For comparison, we also show scores of the best performing fixed (Fix)
feature scoring function, rationale type and length (see Figure 1). For
brevity we include results with F1 macro in App. G, observing similar trends.

We first observe that the highest NormSuff scores across three datasets (SST,
MRc, EvInf), are from the best performing fixed scoring method with fixed
length and rationale type. Additionally, the best performing combination of
our proposed approach for sufficiency is when we only select the feature
scoring method keeping the length and type fixed. This combination results in
the highest NormSuff scores in AG (.44 with TopK type compared to .42, which
is the second best with Contiguous) and competitive NormSuff scores with the
highest scoring combination (e.g. .82 in Ev.Inf. and Contiguous compared to
.85). We assume that using combinations which include instance-specific
lengths do not perform as well for sufficiency due to the shorter rationale
length, which we have previously shown to partially degrade rationale
sufficiency.

Finally, our results demonstrate that we obtain highly comprehensive
rationales when selecting at instance level all parameters (Feat. \+ Len \+
Type) using our approach. In fact, this results in higher NormComp scores
compared to any other setting combination across all datasets. For example in
M.Rc., selecting all parameters results in a NormComp score of .69 which is
.22 units higher than the rationales extracted with fixed feature scoring
method and length and type. This highlights the efficacy of our approach in
extracting highly comprehensive rationales, without requiring strong a priori
assumptions about rationale parameters.

Example 1 Data.:AG Id: test_4614
---
[Fixed-Len + $\alpha$]: … game last Friday night will stand , the CFL
announced yesterday. While a review … [I-L-Len + $\alpha$ (Ours)]: … game last
Friday night will stand , the CFL announced yesterday. While a review …
[Predicted Topic $||$ True Topic]: Decreased significantly $||$ Decreased
significantly
Example 2 Data.:Ev.Inf. Id: 3162205_2
[Fixed-Len + $\alpha\nabla\alpha$]: … computed tomography ( 3D - CT  ) scans .
ABSTRACT.RESULTS : The control sides treated with an autograft showed
significantly better Lenke scores than the study sides treated with
$\boldsymbol{\beta}$ \- CPP at 3 and 6 months postoperatively , but there was
no difference between the two sides at 12 months . The fusion ..
[I-L-Len + $\alpha\nabla\alpha$ (Ours)]: … computed tomography ( 3D - CT )
scans . ABSTRACT.RESULTS : The control sides treated with an autograft showed
significantly better Lenke scores than the study sides treated with
$\boldsymbol{\beta}$ \- CPP at 3 …
[Predicted Relationship $||$ True Relationship]: Increased significantly $||$
No significant difference
Example 3 Data.:SST Id: test_694
[Fixed-Len + $\alpha$]: … Frontal is the antidote for Soderbergh fans who
think he s gone too commercial …
[I-L-Len + I-L-Feat (Ours)]: … Frontal is the antidote for Soderbergh fans who
think he s gone too commercial …
[Predicted Sentiment $||$ True Sentiment]: Negative $||$ Positive
Example 4 Data.:SST Id: test_1039
[Fixed-Len + $\alpha$]: It ’s just incredibly dull.
[I-L-Len + I-L-Feat (Ours)]: It ’s just incredibly dull.
[Predicted Sentiment $||$ True Sentiment]: Negative $||$ Negative
Table 4: Examples when using our approach (Ours) to select at instance-level
(I-L) a combination of the: (1) rationale length (Len); (2) feature scoring
method (Feat) against our baseline of fixed-length rationales from a fixed
feature scoring method.

### Ablation Study

We finally perform an ablation study to examine the behavior and effectiveness
of our approach by sequentially removing one feature scoring method at a time
to measure changes in F1 macro, NormSuff and NormComp. The intuition is that
we should observe drops in faithfulness scores when removing feature
attribution methods for our approach to be effective (i.e. we should extract
more faithful rationales when having more feature scoring options to choose
from). Figure 2 shows the results.

We first observe that removing one feature scoring method at a time results in
increases in F1 macro (lower is better) and drops in NormComp scores (higher
is better). This demonstrate that the faithfulness of the rationales extracted
with our approach deteriorates as the number of feature scoring methods
becomes smaller highlighting the efficacy of our proposed approach. For
example, in Ev.Inf. by removing $\alpha\nabla\alpha$ results in a drop of .14
in mean NormComp (.84 when including $\alpha\nabla\alpha$ compared to .70
without it). On the other hand, we also observe that our method can still
benefit from feature scoring methods that achieve low NormComp scores when
used standalone, resulting in improvements in comprehensiveness and drops in
F1 macro (e.g. $\alpha$ in SST). This indicates that our approach steadily
improves rationale faithfulness for model’s predictions given a larger pool of
available feature scoring methods.

Results show a deterioration in NormSuff scores as the number of feature
scoring methods becomes smaller, showing that our method results in more
sufficient rationales when presented with a larger list of available feature
scoring methods in the majority of the datasets. We hypothesize that this is
not true for MultiRC due to the already low NormSuff scores of the rationales
(e.g. no more than 0.12). By using all six feature scoring methods, our
approach produces highly sufficient rationales and is comparable to the set
achieved the highest sufficiency. For example in Ev.Inf. using all feature
scoring methods results to a NormSuff score of approximately .38 compared to
the highest scoring feature scoring set (all except LIME) and the lowest
scoring ($\mathbf{x}\nabla\mathbf{x}$) which achieved .39 and .15
respectively.

We also tested different combinations of feature scoring methods with similar
observations (see App. G). Finally, we experimented with doubling the upper
bound of the rationale length (from $N$ to $2\times N$) for both fixed length
rationales and our proposed approach. Our approach still yielded more
comprehensive rationales compared to the fixed-length ones that were also
highly sufficient (results included in App. G).

##  6 Qualitative Analysis

Table 4 shows examples of the qualitative comparison between our approach
(Ours) for selecting at instance-level (I-L) a combination of the: (1)
rationale length (Len); (2) feature scoring method (Feat against our baseline
of fixed-length rationales from a fixed feature scoring method.

#### Concise rationales:

Example 1 presents an instance from AG. Our approach extracts a rationale that
is six tokens shorter than the one with fixed length while also achieving a
higher NormComp score. However, the fixed length rationale scores higher in
NormSuff. We can assume from this that sufficiency positively correlates with
rationale length.

#### Error analysis:

Our assumption is that if a model makes a wrong prediction, we should be able
to extract the rationale that better demonstrates what led to a wrong
prediction. Example 2 shows an instance from Ev.Inf., where the model has
wrongly predicted that “Lenke scores at 12 months” have ‘increased
significantly’ instead of the correct ‘no significant difference’.
Surprisingly, both rationales recorded maximum scores (1.0) in NormSuff and
NormComp. We observe that the correct answer is included in the fixed length
rationale, however the model made a wrong prediction. On the contrary, our
rationale highlights something directly related to its prediction.

Example 3 presents an instance from SST, where the fixed-length rationale and
the instance-specific rationale (ours) attend at different sections of the
text. Our rationale scored lower for NormSuff, however we observe that it
aligns more closely with the predicted sentiment.

#### When using a fixed pre-defined length is not sufficient:

Example 4 presents a different scenario, where the fixed-length rationale for
SST is at 20% whilst the upper bound $N$ for our rationale is at 40%. The
intuition is that in certain cases a fixed rationale length might not be
sufficient for all instances to explain a prediction. We argue that our
approach highlighted something more informative for the task (“incredibly
dull” compared to “incredibly”), due to removing the restriction of a pre-
defined fixed length.

##  7 Conclusions

We have proposed a simple yet effective approach for selecting at instance-
level (1) feature scoring method; (2) length; and (3) type of the rationale.
We empirically demonstrated that rationales extracted with our approach are
significantly more comprehensive and highly sufficient, while being shorter
compared to rationales extracted with a fixed feature scoring method, length
and type. Finally, we consider our work an important step towards instance-
level faithful rationalization while finding the most sufficient rationale, an
interesting direction for future work.

## Acknowledgments

NA is supported by EPSRC grant EP/V055712/1, part of the European Commission
CHIST-ERA programme, call 2019 XAI: Explainable Machine Learning-based
Artificial Intelligence.

## References

  * Arras et al. (2016) Arras, L.; Horn, F.; Montavon, G.; Müller, K.-R.; and Samek, W. 2016.  Explaining Predictions of Non-Linear Classifiers in NLP.  In _Proceedings of the 1st Workshop on Representation Learning for NLP_ , 1–7. Berlin, Germany: Association for Computational Linguistics.
  * Arras et al. (2017) Arras, L.; Montavon, G.; Müller, K.-R.; and Samek, W. 2017.  Explaining Recurrent Neural Network Predictions in Sentiment Analysis.  In _Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_ , 159–168. Copenhagen, Denmark: Association for Computational Linguistics.
  * Atanasova et al. (2020) Atanasova, P.; Simonsen, J. G.; Lioma, C.; and Augenstein, I. 2020.  A Diagnostic Study of Explainability Techniques for Text Classification.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 3256–3274. Online: Association for Computational Linguistics.
  * Bastings, Aziz, and Titov (2019) Bastings, J.; Aziz, W.; and Titov, I. 2019.  Interpretable Neural Predictions with Differentiable Binary Variables.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 2963–2977. Florence, Italy: Association for Computational Linguistics.
  * Bastings and Filippova (2020) Bastings, J.; and Filippova, K. 2020.  The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?  In _Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP_ , 149–155. Online: Association for Computational Linguistics.
  * Beltagy, Lo, and Cohan (2019) Beltagy, I.; Lo, K.; and Cohan, A. 2019.  SciBERT: A Pretrained Language Model for Scientific Text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 3615–3620. Hong Kong, China: Association for Computational Linguistics.
  * Bommasani et al. (2021) Bommasani, R.; Hudson, D. A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M. S.; Bohg, J.; Bosselut, A.; Brunskill, E.; et al. 2021.  On the Opportunities and Risks of Foundation Models.  _arXiv preprint arXiv:2108.07258_.
  * Carton, Rathore, and Tan (2020) Carton, S.; Rathore, A.; and Tan, C. 2020.  Evaluating and Characterizing Human Rationales.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 9294–9307. Online: Association for Computational Linguistics.
  * Chen and Ji (2020) Chen, H.; and Ji, Y. 2020.  Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 4236–4251. Online: Association for Computational Linguistics.
  * Chrysostomou and Aletras (2021a) Chrysostomou, G.; and Aletras, N. 2021a.  Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , 8189–8200. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics.
  * Chrysostomou and Aletras (2021b) Chrysostomou, G.; and Aletras, N. 2021b.  Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 477–488. Online: Association for Computational Linguistics.
  * Corso, Gulli, and Romani (2005) Corso, G. M. D.; Gulli, A.; and Romani, F. 2005.  Ranking a stream of news.  In Ellis, A.; and Hagino, T., eds., _Proceedings of the 14th international conference on World Wide Web, WWW 2005, Chiba, Japan, May 10-14, 2005_ , 97–106. ACM.
  * Devlin et al. (2019) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 4171–4186. Minneapolis, Minnesota: Association for Computational Linguistics.
  * DeYoung et al. (2020) DeYoung, J.; Jain, S.; Rajani, N. F.; Lehman, E.; Xiong, C.; Socher, R.; and Wallace, B. C. 2020.  ERASER: A Benchmark to Evaluate Rationalized NLP Models.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 4443–4458. Online: Association for Computational Linguistics.
  * Jacovi and Goldberg (2020) Jacovi, A.; and Goldberg, Y. 2020.  Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 4198–4205. Online: Association for Computational Linguistics.
  * Jain and Wallace (2019) Jain, S.; and Wallace, B. C. 2019.  Attention is not Explanation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 3543–3556. Minneapolis, Minnesota: Association for Computational Linguistics.
  * Jain et al. (2020) Jain, S.; Wiegreffe, S.; Pinter, Y.; and Wallace, B. C. 2020.  Learning to Faithfully Rationalize by Construction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 4459–4473. Online: Association for Computational Linguistics.
  * Khashabi et al. (2018) Khashabi, D.; Chaturvedi, S.; Roth, M.; Upadhyay, S.; and Roth, D. 2018.  Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , 252–262. New Orleans, Louisiana: Association for Computational Linguistics.
  * Kim et al. (2020) Kim, S.; Yi, J.; Kim, E.; and Yoon, S. 2020.  Interpretation of NLP models through input marginalization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 3154–3167. Online: Association for Computational Linguistics.
  * Kindermans et al. (2016) Kindermans, P.-J.; Schütt, K.; Müller, K.-R.; and Dähne, S. 2016.  Investigating the influence of noise and distractors on the interpretation of neural networks.  _arXiv preprint arXiv:1611.07270_.
  * Lehman et al. (2019) Lehman, E.; DeYoung, J.; Barzilay, R.; and Wallace, B. C. 2019.  Inferring Which Medical Treatments Work from Reports of Clinical Trials.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 3705–3717. Minneapolis, Minnesota: Association for Computational Linguistics.
  * Lei, Barzilay, and Jaakkola (2016) Lei, T.; Barzilay, R.; and Jaakkola, T. 2016.  Rationalizing Neural Predictions.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , 107–117. Austin, Texas: Association for Computational Linguistics.
  * Li et al. (2016) Li, J.; Chen, X.; Hovy, E.; and Jurafsky, D. 2016.  Visualizing and Understanding Neural Models in NLP.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , 681–691. San Diego, California: Association for Computational Linguistics.
  * Liu et al. (2019) Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _CoRR_ , abs/1907.11692.
  * Nguyen (2018) Nguyen, D. 2018.  Comparing Automatic and Human Evaluation of Local Explanations for Text Classification.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , 1069–1078. New Orleans, Louisiana: Association for Computational Linguistics.
  * Ribeiro, Singh, and Guestrin (2016) Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016.  ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier.  In Krishnapuram, B.; Shah, M.; Smola, A. J.; Aggarwal, C. C.; Shen, D.; and Rastogi, R., eds., _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016_ , 1135–1144. ACM.
  * Robnik-Šikonja and Kononenko (2008) Robnik-Šikonja, M.; and Kononenko, I. 2008.  Explaining classifications for individual instances.  _IEEE Transactions on Knowledge and Data Engineering_ , 20(5): 589–600.
  * Samek et al. (2017) Samek, W.; Binder, A.; Montavon, G.; Lapuschkin, S.; and Müller, K. 2017\.  Evaluating the Visualization of What a Deep Neural Network Has Learned.  _IEEE Transactions on Neural Networks and Learning Systems_ , 28(11): 2660–2673.
  * Serrano and Smith (2019) Serrano, S.; and Smith, N. A. 2019.  Is Attention Interpretable?  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 2931–2951. Florence, Italy: Association for Computational Linguistics.
  * Shrikumar, Greenside, and Kundaje (2017) Shrikumar, A.; Greenside, P.; and Kundaje, A. 2017.  Learning Important Features Through Propagating Activation Differences.  In Precup, D.; and Teh, Y. W., eds., _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , 3145–3153. PMLR.
  * Socher et al. (2013) Socher, R.; Perelygin, A.; Wu, J.; Chuang, J.; Manning, C. D.; Ng, A.; and Potts, C. 2013.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , 1631–1642. Seattle, Washington, USA: Association for Computational Linguistics.
  * Sundararajan, Taly, and Yan (2017) Sundararajan, M.; Taly, A.; and Yan, Q. 2017.  Axiomatic Attribution for Deep Networks.  In Precup, D.; and Teh, Y. W., eds., _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_ , volume 70 of _Proceedings of Machine Learning Research_ , 3319–3328. PMLR.
  * Treviso and Martins (2020) Treviso, M.; and Martins, A. F. T. 2020.  The Explanation Game: Towards Prediction Explainability through Sparse Communication.  In _Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP_ , 107–118. Online: Association for Computational Linguistics.
  * Wang et al. (2019) Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. R. 2019.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Wiegreffe and Pinter (2019) Wiegreffe, S.; and Pinter, Y. 2019.  Attention is not not Explanation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 11–20. Hong Kong, China: Association for Computational Linguistics.
  * Wolf et al. (2020) Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; Davison, J.; Shleifer, S.; von Platen, P.; Ma, C.; Jernite, Y.; Plu, J.; Xu, C.; Le Scao, T.; Gugger, S.; Drame, M.; Lhoest, Q.; and Rush, A. 2020.  Transformers: State-of-the-Art Natural Language Processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , 38–45. Online: Association for Computational Linguistics.
  * Zhang et al. (2021) Zhang, W.; Huang, Z.; Zhu, Y.; Ye, G.; Cui, X.; and Zhang, F. 2021.  On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 5399–5411. Online: Association for Computational Linguistics.

##  Appendix A Model Hyperparameters

Table 5 presents the hyper-parameters used to train the models across
different datasets, along with F1 macro performance on the development set.
Models where finetuned across 3 runs for 5 epochs. We implement our models
using the Huggingface library (Wolf et al. 2020) and use default parameters of
the AdamW optimizer apart from the learning rates. We use a linear scheduler
with 10% of the steps in the first epoch as warmup steps. Experiments are run
on a single Nvidia Tesla V100 GPU.

Dataset | Model | $lr^{m}$ | $lr^{c}$ | F1
---|---|---|---|---
SST | bert-base | 1e-5 | 1e-4 | 90.7 $\pm$ 0.2
AG | bert-base | 1e-5 | 1e-4 | 93.3 $\pm$ 0.0
Ev.Inf. | scibert | 1e-5 | 1e-4 | 82.5 $\pm$ 0.9
M.RC | roberta-base | 1e-5 | 1e-4 | 76.3 $\pm$ 0.2
Table 5: Model and their hyper-parameters for each dataset, including learning
rate for the model ($lr^{m}$) and the classifier layer ($lr^{c}$) and F1 macro
scores on the development set across three runs.

##  Appendix B Divergence metrics ($\boldsymbol{\Delta}$)

To compute the value $\delta$ for how much $\mathcal{Y}^{m}$ differs from
$\mathcal{Y}$, we consider four divergence measures ($\Delta$), also
previously used in literature (Robnik-Šikonja and Kononenko 2008; Jain and
Wallace 2019; Wiegreffe and Pinter 2019):

#### Kullback Leibler (KL)

: A non-symmetric divergence measure of how a particular distribution
divergences from a reference distribution:

$KL(\mathcal{Y}||\mathcal{Y^{*}})=\mathcal{Y}(\log(\mathcal{Y}-\log(\mathcal{Y}^{m}))$
(3)

#### Jensen-Shannon (JSD)

: A symmetric divergence metric based on the KL divergence of two
distributions from their mean:

$JSD(\mathcal{Y}||\mathcal{Y}^{m})=\frac{1}{2}(KL(\mathcal{Y}||\mathbf{\mu})+\frac{1}{2}(KL(\mathcal{Y}^{m}||\mu))$
(4)

where $\mu$ is the average distribution of $\mathcal{Y}$ and
$\mathcal{Y}^{m}$.

#### Perplexity (Perp.)

: A measure of how well a model can predict a sample, where:

$PERP(\mathcal{Y}||\mathcal{Y}^{m})=\exp^{\mathcal{H(\mathcal{Y},\mathcal{Y}}^{m})}$
(5)

where we consider $\mathcal{Y}$ as the ground truth and
$H(\mathcal{Y}||\mathcal{Y}^{m})$ is the Cross Entropy Loss.

#### Class Difference (ClassDiff)

: The direct difference between the predicted class probability from the model
with full text ($\mathbf{x}$) and the same class probability with reduced text
($\mathbf{x}_{\setminus\mathcal{R}}$) :

$CLASSDIFF(\mathcal{Y}||\mathcal{Y}^{m})=p(\hat{y}|\mathbf{x})-p(\hat{y}|\mathbf{x}_{\backslash\mathcal{R}})$
(6)

where $\hat{y}=\text{arg max}(\mathcal{Y})$.

##  Appendix C Computed Instance-Specific Lengths

% |  | $\mathbf{x}\nabla\mathbf{x}$ | IG | DeepLift | LIME | $\alpha$ | $\alpha\nabla\alpha$ | Avg.
---|---|---|---|---|---|---|---|---
TopK | SST | 15.8 | 16.3 | 15.7 | 16.9 | 15.5 | 16.6 | 16.1
AG | 14.5 | 16.5 | 15.2 | 16.3 | 16.3 | 16.3 | 15.8
Ev.Inf. | 7.7 | 7.7 | 8.4 | 7.4 | 6.6 | 7.0 | 7.5
MultiRC | 14.1 | 14.6 | 16.1 | 13.4 | 15.9 | 16.0 | 15.0
Cont. | SST | 15.5 | 15.6 | 15.4 | 15.9 | 14.7 | 15.7 | 15.5
AG | 14.0 | 15.7 | 14.9 | 15.0 | 14.5 | 15.1 | 14.9
Ev.Inf. | 7.2 | 6.9 | 7.6 | 7.3 | 6.6 | 6.9 | 7.1
MultiRC | 14.0 | 15.1 | 15.9 | 13.6 | 16.1 | 16.1 | 15.1
Table 6: Average instance-specific rationale lengths (as a percentage %) computed using JSD (as $\Delta$), across instances for TopK and Contiguous rationale types. |  | @Token | @2% | @5%
---|---|---|---|---
|  | F1 | Suff. | Comp. | Rat-Len. | F1 | Suff. | Comp. | Rat-Len. | F1 | Suff. | Comp. | Rat-Len.
TopK | SST | 57.22 | 0.59 | 0.55 | 17.15 | 57.22 | 0.59 | 0.55 | 17.15 | 57.34 | 0.59 | 0.55 | 17.23
AG | 69.90 | 0.37 | 0.39 | 17.05 | 69.93 | 0.37 | 0.39 | 17.06 | 70.12 | 0.38 | 0.39 | 17.52
Ev.Inf. | 21.09 | 0.35 | 0.87 | 6.52 | 21.74 | 0.36 | 0.86 | 7.25 | 23.22 | 0.37 | 0.84 | 8.28
MultiRC | 47.63 | 0.07 | 0.63 | 13.59 | 48.76 | 0.07 | 0.62 | 14.31 | 49.92 | 0.08 | 0.61 | 15.37
Cont. | SST | 66.43 | 0.61 | 0.48 | 16.01 | 66.43 | 0.61 | 0.48 | 16.01 | 66.48 | 0.61 | 0.48 | 16.21
AG | 85.27 | 0.33 | 0.24 | 15.04 | 85.27 | 0.33 | 0.24 | 15.06 | 85.42 | 0.34 | 0.24 | 16.00
Ev.Inf. | 37.03 | 0.68 | 0.70 | 6.12 | 40.66 | 0.76 | 0.67 | 7.08 | 46.20 | 0.84 | 0.62 | 8.37
MultiRC | 43.38 | 0.05 | 0.66 | 12.29 | 45.12 | 0.05 | 0.65 | 13.22 | 47.87 | 0.06 | 0.63 | 14.42
Table 7: F1 macro (lower is better), NormSuff (higher is better) and NormComp
(higher is better) for our rationales with instance-specific length and
feature scoring method at each instance, when computing $\delta$ at: (1) each
token (@Token); (2) at every 2% (@ 2%) and at every 5% (@ 5%). We also include
average rationale lengths for helping with the analysis.

In Table 6 we present the average computed rationale lengths using JSD across
each feature scoring method, for each dataset and rationale type. To extract
rationales we use an upper bound $N$, our pre-defined fixed rationale length
(as indicated in Table 1 and as defined by (Jain et al. 2020)), to make it
comparable with our baseline.

We first observe that rationales extracted with our proposed approach are on
average shorter than the a priori set rationale ratio. In datasets such as SST
and AG where we have short sequences on average (See Table 1), these
differences are negligible, as a 4% decrease translates to approximately
having a single token less in the rationale. This strengthens our initial
hypothesis that certain instances might not require as many tokens to
successfully explain a prediction.

In datasets with longer sequences lengths on average (Ev.Inf. and M.RC) such
differences are more evident. For example with LIME and Contiguous rationales
in M.RC, our proposed approach results in rationales which are on average 6%
shorter than the fixed length rationales. This translates to approximately 20
less tokens to form a rationale. What is particularly interesting is that by
observing the standard deviations for M.RC and Ev.Inf., we notice that the
vast majority of instances does not exhaust the upper bound $N$ to form a
rationale. We consider this particularly important for longer sequences, as
often when acquiring an explanation for a model’s prediction it is desirable
to avoid a noisy interpretation of why a model predicted a particular class.

##  Appendix D Alternative Formulation of Instance-Specific Length Rationales

We also examined introducing a thresholding approach in early experimentation
when extracting instance-specific length rationales, whereby
$\delta_{prev}-\delta_{max}<thresh$, similar to early stopping to avoid
exhausting the upper bound $N$. Early results suggested that for datasets with
shorter length sequences (SST and AG) this approach performed worse albeit
comparably. In datasets with longer sequence lengths (Ev.Inf. and M.Rc), the
threshold approach performed poorly. On a closer inspection this was
attributed to finding a threshold too early in the sequence thus not capturing
all the necessary information. We experimented with JSD and the following
thresholds : {1e-4, 1e-3, 1e-2}.

We considered using patience to avoid such naive thresholding, however the
computation time would increase significantly. The reason being that we
conduct experiments at instance-level, in batches. Introducing patience, would
entail that we conduct it using a single instance at a time thus increasing
computations by the batch size. As such we have not conducted this approach as
it would be computationally expensive.

|  | @Token | @2% | @5%
---|---|---|---|---
|  | (s) | (s) | R.I. | (s) | R.I.
TopK | SST | 0.05 | 0.05 | 1.0 | 0.05 | 1.0
AG | 0.29 | 0.29 | 1.0 | 0.15 | 1.9
Ev.Inf. | 1.99 | 0.26 | 7.7 | 0.11 | 18.1
MultiRC | 3.07 | 0.51 | 6.0 | 0.21 | 14.6
Cont. | SST | 0.06 | 0.06 | 1.0 | 0.06 | 1.0
AG | 0.37 | 0.38 | 1.0 | 0.19 | 1.9
Ev.Inf. | 2.59 | 0.33 | 7.8 | 0.13 | 19.9
MultiRC | 3.72 | 0.65 | 5.7 | 0.26 | 14.3
Table 8: Average time taken (s) to extract a rationales of instance-specific
length per instance, when computing $\delta$ at: (1) each token (@Token); (2)
at every 2% (@ 2%) and at every 5% (@ 5%), where lower time is better. We also
denote relative improvements (R.I.) where higher is better.

##  Appendix E Reducing Time Complexities

Albeit significantly faster to computing LIME scores, selecting a rationale
length at each instance in a dataset can be computationally expensive when we
compute $\delta$ for every token, being similar to counting decision flips
(Nguyen 2018; Serrano and Smith 2019; Atanasova et al. 2020). This takes into
consideration that we have to perform a forward pass for every token until we
reach $N$ tokens, for each feature attribution approach $\Omega$. In the
following segments we describe approaches to reduce computational times.

#### Reducing search granularity:

Similar to Atanasova et al. (2020), we can reduce significantly computation
times by reducing the granularity of our search. In our implementation we
describe masking each token or n-gram sequentially, which can be altered to
skip tokens. For example, consider a sequence with 200 tokens and an upper-
bound $N=$ 20% and as such $N_{t}=40$. Instead of computing $\delta$ for each
token, we can compute it for every 5 tokens and as such reducing complexity by
5. Similarly we can compute $\delta$ at every 2% of the sequence until we
reach $N$. For example for Ev.Inf., where $N=$10% we compute $\delta$ at every
{2%, 4%, $\ldots$ 10%} thus keeping the forward passes constant across
instances.

(a) F1 macro

(b) Sufficiency

(c) Comprehensiveness

Figure 3: F1 macro (lower is better), mean NormSuff (higher is better) and
mean NormComp (higher is better) when using any single feature scoring method
across all instances in a dataset and our proposed method of selecting a
feature scoring method for each instance (OURS).

In Table 8, we present average time taken (in seconds (s)) to extract a
rationale of instance-specific length for each instance,when computing
$\delta$ at: (1) each token (@Token); (2) at every 2% (@ 2%) and at every 5%
(@ 5%) using JSD. We observe that in datasets with a short average length of
instance when moving from @Token to @2% does not reduce time, but results in
significant reductions in computations with MultiRc ($\sim$6x R.I.) and Evinf
($\sim$8x R.I.). As expected, these are further reduced when reducing the
granularity to searching @5%, with AG recording a $\sim$2x R.I., MultiRC
$\sim$14.5 R.I. and Ev.Inf. $\sim$19% R.I.

In Table 7 we present the faithfulness performance as we reduce the
granularity of our search. Results suggest that by reducing granularity,
NormComp scores reduce whilst F1 macro performances increase, suggesting a
reduction in faithfulness. However, we observe that moving from @Token to @2%
this reduction is negligible considering the significantly improved
computational times. However, as expected moving from @Token to @5%
performance degrades rapidly and as such 2% seems like a more appropriate step
to consider. Unsurprisingly, with increasing step-size we observe increases in
the computed rationale lengths, which leads to an increase in sufficiency.

#### Combining feature scoring rankings:

We considered further reducing our computation time by merging importance
scores from all feature scoring methods. The intuition is that we obtain a
combined ranking and avoid selecting the best feature scoring method at each
instance and computing a rationale length for all feature scoring methods. We
attempted this by averaging the normalized importance scores for each sequence
from all the feature scoring methods, however as expected results where not
comparable (63.2 average F1 macro compared to 54.4) with our proposed approach
or even our best performing baseline.

##  Appendix F Comparing divergence metrics

We first compare the effectiveness of divergence metrics in computing a
rationale with instance-specific length and selecting the best feature scoring
method at instance level. Table 9 presents F1 macro (lower is better),
NormSuff and NormComp (higher is better) macro scores for our proposed
instance-specific length rationales from the best feature scoring method at
instance level.

Results demonstrate that all divergence metrics perform comparably with the
exception of perplexity. The remainder of the metrics result in similar scores
for NormSuff, NormComp and F1 macro, with JSD and ClassDiff having a slight
edge over KLDIV.

| $\Delta$ | NormSuff | NormComp | F1
---|---|---|---|---
TopK | JSD | 0.38 | 0.64 | 46.03
KLDIV | 0.38 | 0.63 | 46.40
ClassDiff | 0.38 | 0.63 | 45.02
Perp. | 0.35 | 0.55 | 52.22
TopK | JSD | 0.38 | 0.64 | 46.03
KLDIV | 0.38 | 0.63 | 46.40
ClassDiff | 0.38 | 0.64 | 45.02
Perp. | 0.35 | 0.55 | 52.22
Table 9: NormSuff (higher is better), NormComp (higher is better) and F1 macro
(lower is better) when using different divergence metrics to select the
rationale length and feature scoring method($\Delta$)

(a) F1 macro

(b) NormSuff

(c) NormComp

Figure 4: F1 macro (lower is better), mean NormSuff and mean NormComp (higher
is better), when extracting rationales with our approach given decreasing
numbers of feature scoring methods.

##  Appendix G Additional Results

### Selecting a Feature Scoring Method

Figure 3 shows F1 macro (lower is better), mean NormSuff and mean NormComp
(higher is better), when using a single feature scoring method globally
(across all instances in a dataset) to extract rationales and when we use our
proposed approach to select a feature scoring method for each instance (OURS)
for contiguous type rationales. Overall, results are similar to those observed
using TopK, with our approach resulting to rationales with significantly
higher NormComp scores and lower F1 macro scores, whilst still retaining high
sufficiency as demonstrated by the high NormSuff scores. Overall, with
Contiguous rationales our approach results in slight improvements in NormSuff
scores over TopK when using our proposed method.

### Different Sets of Feature Scoring Methods

Figure 4 shows F1 macro (lower is better), mean NormSuff and mean NormComp
(higher is better) scores, when using our proposed method to select from
different sets of feature scoring methods.

Overall, we observe similar results to the analysis in §5 with our approach
reaching peak performance in NormComp (highest) and F1 macro (lowest) with the
set including all the feature scoring methods. Also, similar to the analysis
in the main body, we observe that again our approach results in highly
sufficient rationales when presented with the largest set of feature scoring
methods.

What is demonstrated from this more detailed analysis, is that certain feature
scoring methods contribute more to our approach compared to others. For
example LIME and $\alpha\nabla\alpha$ in Ev.Inf. is enough to reach a NormComp
score of .80 compared to the best of .84, so only .04 points behind. We find a
correlation between the feature scoring methods when they are performing well
globally (see Fig 1) and when they are improving drastically our proposed
approach, as they are essentially giving a good head-start to our approach.

### When $N$ is Double

We hypothesize that the information a rationale holds, increases with
increasing rationale lengths similar to Jain et al. (2020). We therefore
evaluate the effectiveness of our approach, when doubling the upper-bound of
the maximum allowed rationale length $N$ (see §3). We assume that this should
result into better rationale comprehensiveness and sufficiency.

In Table 10 we present the computed rationale lengths when we double $N$. As
we observe our rationales are still shorter compared to $N\times 2$, with
certain cases resulting in significant reductions. For example in Ev.Inf. with
$\alpha$, contiguous rationales extracted with our approach are on average
7.5% shorter than fixed length rationales (approximately 27 tokens shorter).

Figure 5 shows F1 macro, NormSuff and NormComp scores of rationales, when we
increase $N$ (1x) to $2\times N$ (2x). For brevity we show results from
rationales extracted with our best performing method (i.e. using instance-
level feature scoring method, rationale length and type). Results highlight
that our approach successfully scales with increasing rationale lengths,
resulting in more sufficient and comprehensive rationales with average length
shorter than the $2\times N$ upper-bound.

|  | $\mathbf{x}\nabla\mathbf{x}$ | IG | DeepLift | LIME | $\alpha$ | $\alpha\nabla\alpha$ | Avg.
---|---|---|---|---|---|---|---|---
TopK | SST | 29.5 | 30.5 | 30.3 | 31.0 | 28.4 | 30.3 | 30.0
AG | 32.4 | 34.8 | 32.4 | 32.8 | 33.6 | 32.9 | 33.2
Ev.Inf. | 14.8 | 15.3 | 16.9 | 15.3 | 11.9 | 14.0 | 14.7
MultiRC | 26.9 | 28.5 | 31.7 | 25.4 | 31.1 | 32.1 | 29.3
Cont. | SST | 28.3 | 28.7 | 28.8 | 28.4 | 27.0 | 29.3 | 28.4
AG | 31.7 | 32.2 | 31.8 | 30.6 | 31.4 | 31.4 | 31.5
Ev.Inf. | 13.4 | 13.6 | 15.3 | 14.4 | 12.5 | 13.0 | 13.7
MultiRC | 26.4 | 28.6 | 30.0 | 25.1 | 30.1 | 30.0 | 28.4
Table 10: Average instance-specific rationale lengths computed using JSD,
across instances for TopK and Contiguous rationale types when we double $N$ to
$2\times N$.

(a) F1

(b) NormSuff

(c) NormComp

Figure 5: F1 macro (lower is better), mean NormSuff and NormComp scores (higher is better) extracted rationales when . |  | SST | MultiRC | AG | Ev.Inf.
---|---|---|---|---|---
TopK | DeepLift | 1.0 | 1.0 | 1.0 | 1.0
LIME | 1.1 | 0.9 | 1.0 | 1.0
$\boldsymbol{\alpha}$ | 1.1 | 1.0 | 1.0 | 0.8
$\boldsymbol{\alpha\nabla\alpha}$ | 1.0 | 1.0 | 1.0 | 0.8
IG | 1.0 | 0.9 | 1.0 | 1.0
$\boldsymbol{\mathbf{x}\nabla\mathbf{x}}$ | 1.0 | 0.9 | 1.0 | 0.9
Contiguous | DeepLift | 1.0 | 0.9 | 1.0 | 0.9
LIME | 1.0 | 0.8 | 1.0 | 0.9
$\boldsymbol{\alpha}$ | 1.0 | 1.0 | 1.0 | 0.8
$\boldsymbol{\alpha\nabla\alpha}$ | 1.0 | 0.9 | 1.0 | 0.8
IG | 1.0 | 0.9 | 1.0 | 0.9
$\boldsymbol{\mathbf{x}\nabla\mathbf{x}}$ | 1.0 | 0.9 | 1.0 | 0.9
Table 11: Relative Improvement (R.I.) for F1 macro, when moving from fixed
length rationales (see $N$ in Table 1) to rationales with instance-specific
length from our proposed approach ($<$1.0 is better).

### Instance-specific Rationale Length

Table 11 presents the Relative Improvement (R.I.) for F1 macro, when moving
from fixed length rationales (see $N$ in Table 1) to rationales with instance-
specific length from our proposed approach ($<$1.0 is better).

Similarly to NormComp scores, we observe that datasets MultiRc and Ev.Inf.
benefit the most from our proposed approach, resulting in relatively lower F1
macro scores irrespective of the rationale type. We consider this important,
as despite of the shorter rationales, a model finds more necessary for a
prediction the instance-specific length rationales extracted with our proposed
method compared to the fixed-length longer ones.

### Instance-specific Feature Scoring, Length and Type

Table 12 shows F1 macro performance (lower is better) when we select at
instance-level (I-L) a combination of: (1) the rationale length (Len); (2) the
feature scoring method (Feat.); and (3) the rationale type (Type). For
reference we also show the highest scoring fixed (Fix) feature scoring
function, fixed rationale type and fixed and instance-specific rationale
length.

Results show that we can obtain highly faithful rationales when selecting at
instance level all parameters (Feat. \+ Len \+ Type) using our proposed
approach. Selecting at instance-level all rationale settings results in lower
F1 macro performance compared to any combination across all datasets. For
example with MRc, F1 performance drops to just 38.00 when we select all
parameters compared to 45.1 with the second best (7 F1 point difference). This
highlights the efficacy of our approach in extracting rationales that are
necessary for a model to make a prediction, without requiring any a priori
assumptions about any of the rationale parameters.

|  |  | F1 macro
---|---|---|---
Type | Len | Feat. | SST | MRc | AG | EvInf
TopK | Fix | Fix | 63.26 | 67.33 | 78.80 | 32.59
I-L | Fix | 64.41 | 62.74 | 79.52 | 25.26
Fix | I-L | 56.40 | 52.40 | 68.80 | 26.10
| I-L | I-L | 57.20 | 48.80 | 69.90 | 21.70
Cont. | Fix | Fix | 70.89 | 68.96 | 89.18 | 55.32
I-L | Fix | 68.80 | 57.07 | 87.70 | 45.69
Fix | I-L | 69.80 | 55.00 | 86.90 | 50.50
| I-L | I-L | 66.40 | 45.10 | 85.30 | 40.70
I-L | I-L | I-L | 56.70 | 38.00 | 69.60 | 19.80
Table 12: F1 macro when we select at instance-level (I-L) a combination of:
(1) the rationale length (Len); (2) the feature scoring method (Feat.); and
(3) the rationale type (Type). {Type}-Fix-Fix and {Type}-I-L-Fix values are
from the best performing feature scoring method. Bold values denote the
highest performing combination in column-wise (lower is better).