The problem of interpreting the decisions of machine learning is a well-researched and important.
We are interested in a specific type of machine learning model that deals with graph data called graph neural networks.
Evaluating interpretability approaches for graph neural networks (GNN) specifically are known to be challenging due to the lack of a commonly accepted benchmark.
Given a GNN model, several interpretability approaches exist to explain GNN models with diverse (sometimes conflicting) evaluation methodologies.
In this paper, we propose a benchmark for evaluating the explainability approaches for GNNs called .
In , we firstly propose four diverse GNN explanation evaluation regimes –
1) faithfulness,
2) sparsity,
3) correctness, and 4) plausibility.
We reconcile multiple evaluation metrics in the existing literature and cover diverse notions for a holistic evaluation.
Our graph datasets range from citation networks, document graphs, to graphs from molecules and proteins.
We conduct an extensive empirical study on four GNN models and nine post-hoc explanation approaches for node and graph classification tasks.
We open both the benchmarks and reference implementations and make them available at <https://github.com/Mandeep-Rathee/Bagel-benchmark>.

§ INTRODUCTION

Graph neural networks (GNNs) [34, 13, 14, 37, 8] are representation learning techniques that encode structured information into low dimensional space using a feature aggregation mechanism over the node neighborhoods. GNNs have shown state-of-the-art performance across many scientific fields in various important downstream applications, such as molecular data analysis, drug discovery, toxic molecule detection, and community clustering [4, 7, 39].

There have been benchmarks and datasets for interpretability of machine learning models [36, 17]. The rising number of applications of GNNs in several sensitive domains like medicine and healthcare [4, 18] necessitates the need to explain their decision-making process. GNNs are inherently black-box and non-interpretable. Moreover, due to the complex interplay of node features and neighborhood structure in the decision-making process, general explanation approaches [19, 23, 30] cannot be trivially applied for graph models.
Consequently, several explanation techniques [38, 6, 35, 40] have been proposed for GNNs in the last few years.
Unlike standard machine learning models, in GNNs, the explanation methods return a combination of feature and data attributions as explanations given a trained GNN model.
For example, in a node classification task, an explanation of a class prediction for a certain node could be a subset of the responsible features in addition to a subset of responsible nodes/edges in its computational graph.
A known challenge in developing explanation techniques is that of evaluation of the quality of explanations.
This challenge also extends to the evaluation of explainability approaches for GNNs and is the focus of this work.

Existing approaches usually focus on a certain aspect of evaluation, sometimes even performed on synthetic datasets. For example, some works employ synthetic datasets with an already-known subgraph (sometimes referred to as the ground truth reason or simply the ground truth).
Explanations are then evaluated based on their agreement with the ground truth.
Such an evaluation is sometimes flawed as one cannot always guarantee that the GNN has used in the first place the seeded subgraph for its decision-making process [5].
Besides, there is no standardized procedure for comparing different GNN explanations. For example, feature attribution methods can generate soft masks (feature importance as a distribution) or hard masks (boolean selections) over features. Comparing hard and soft mask explanations needs a common and standardized protocol.
Finally, the check for human plausibility and correctness have been ignored in the evaluation of GNN explainers.
Human plausibility checks if a model predicts right for the right reason. On the other hand, the correctness of an explanation checks if the is able to isolate spurious correlations and biases that are intentionally added to the training data as a proxy for biases present in real-world data.

An overview of the  benchmark.

To address the issues of a holistic evaluation and contribute a resource to the growing community on GNN explainability, we developed , a benchmark platform for evaluating explanation approaches for graph neural networks or GNNs.  as depicted in Figure <ref> covers diverse datasets [28, 2, 1, 42] from the literature, a range of standardized metrics, and a modular, extendable framework for execution and evaluation of GNN explanation approaches, along with initial implementations of recent popular explanation methods.  includes:

$\circ$  Four diverse evaluation notions that evaluate the faithfulness, sparsity, correctness, and plausibility of GNN explanations on real-world datasets.
While the first three metrics focus on evaluating the explainers, plausibility checks for explanations to be human congruent.

$\circ$ Besides the widely used datasets for measuring faithfulness of explanations,  consists of new datasets for the plausibility of explanation approaches in our benchmark datasets.

$\circ$ We unify multiple evaluations, metrics, domains, and datasets into an easy-to-use format that reduces the entry barrier for evaluating new approaches to explain GNNs. Additionally, we provide an extendable library to implement and evaluate GNN explainers.

$\circ$ We conduct an extensive evaluation of GNNExplainer() [38], PGM-Explainer(PGM) [35], Zorro [6], Grad [29], GradInput[29], Integrated Gradient(IG) [33], SmoothGrad [31], CAM [22] and GradCAM [22] existing GNN in .

We show that there is no clear winner in GNN explanation methods showing nuanced interpretations of the GNN explanation methods using the multiple metrics considered. We finally note that evaluating the effectiveness of explanations is an intrinsically human-centric task that ideally requires human studies.
However, the goal of  is to provide a fast and accurate evaluation strategy that is often desirable to develop new explainability techniques using empirical evaluation metrics before the human trial stage. The code and the datasets used in our benchmark are available at <https://github.com/Mandeep-Rathee/Bagel-benchmark>.

§ BACKGROUND AND PRELIMINARIES

Graph Neural Networks. Let $\mathcal{G(V,E)}$ be a graph with $\mathcal{V}$ is a set of nodes and $\mathcal{E}$ is a set of edges. Let $\mathcal{A}\in\{0,1\}^{(n,n)}$  be the adjacency matrix of the graph where $n$ is the number of nodes in the graph with $\mathcal{A}_{ij}=1$ if there is an edge between node $i$ and $j$ and $0$ otherwise. Let $\mathcal{X}\in\mathbf{R}^{(n,d)}$ be the features matrix where $d$ is the number of features. For a given node $v\in \mathcal{V}$, $\mathbf{x}_{v}$ denotes its features vector and $\mathcal{N}_{v}$ is a set of its neighbors. We denote the trained GNN model as $f$ on the given graph. For each layer $\ell$, the representation of node $v$ is obtained by aggregating and transforming the representations of its neighboring nodes at layer $\ell-1$

\begin{align}
\boldsymbol{h}_{v}^{(\ell)}=&\operatorname{AGG}\left(\left\{\mathbf{x}_{v}^{(\ell-1)},\left\{\mathbf{x}_{u}^{(\ell-1)} \mid u \in{\mathcal{N}}_{v}\right\}\right\}\right), \quad
\mathbf{x}_{v}^{(\ell)}= \operatorname{TRANSFORM}
\left(\boldsymbol{h}_{v}^{(\ell)}, W^{(\ell)}\right),
\end{align}

where $W^{(\ell)}$ represents the weight matrix at layer $\ell$.
The aggregation function $\operatorname{AGG}$ function is dependent on the GNN model. For example, graph convolution network (GCN) [13] uses a degree weighted aggregation of neighborhood features, whereas graph attention network (GAT) [34] learns neighborhood weights via an attention mechanism. The prediction can be obtained at the final layer using a softmax function. An additional pooling layer is applied for graph classification tasks before applying softmax function.

§.§ Post-hoc explanations and evaluation for GNNs

Post-hoc explainers for GNNs produce feature and local structure attributions where a combination of a masked set of nodes, edges, and features is retrieved as an explanation. To compute the explanation for a $k$-layer GNN, the $k$-hop neighborhood (also referred to as the node's computational graph) of the node is utilized. currently consists of 3 classes of post-explanation techniques: gradient based, perturbation based and surrogate model approaches.
The gradient-based methods [29, 33, 31, 22]
are the simplest approaches for generating the explanation for any differentiable trained model.
In these approaches, the importance scores for the explanation are usually computed using gradients of the input. The perturbation-based approaches [6, 38, 20, 41, 27] learns the important features and structural information by observing the predictive power of the model when noise is added to the input. The surrogate-based approaches [35, 10, 43] learns a simple interpretable model for the local neighborhood of a query node and its prediction.
The explanations generated by this simple model are treated as the explanations of the original model. We note that is, in general, applicable for any explainer which returns binary (hard) or continuous (soft) importance scores (as depicted in Figure <ref>) for the input features/nodes/edges as an explanation.

§.§ Related work on evaluation for post-hoc explanations

Evaluation of explanation methods for any predictive model is inherently tricky.
Specifically, when evaluating already trained models, we are faced with the lack of true explanations. Collecting true explanations (sometimes referred to as ground truth) for GNNs is even more challenging due to the abstract nature of the input graphs. Moreover, depending on the explanation collection method, it is not always clear if the model used the ground truth in its decision-making process.

Nevertheless, some current works employ small synthetic datasets seeded with a ground truth subgraph. Consequently, metrics such as explanation accuracy [26, 38] were proposed, which measure the agreement of found explanation with that of ground truth. Observing the false optimism of accuracy metric for small explanation subgraphs [6] proposed the use of Precision instead of accuracy.

Another important notion is faithfulness where the key idea is to measure how much the explanation characterizes the model's working. To measure faithfulness [26] degrade model performance by damaging the training dataset and measuring how each explanation method responds.
The lack of ground truth again limits such a measure. [22] proposed to compute faithfulness as the difference of accuracy (or predicted probability) between the original predictions and the new predictions after masking out the input features found by the explanation. This was called Fidelity in their work. As the features cannot be removed in entirety to measure their impact [6] proposed  based on rate distortion theory defined as the expected predictive score of an explanation over all possible configurations of the non-explanation features.

An important criterion to measure the goodness of an explanation is its size. For example, the full input is also a faithful explanation. However, humans find shorter explanations easier to analyze and reason. Works such as [22] measure the sparsity of an explanation as the fraction of features selected by the explainer. Noting that this definition is not directly applicable for softmask approaches, [6] proposes to quantify sparsity as entropy over the normalized distribution of explanation masks.

The authors in [26] argued that the explanation should be stable under input perturbations. In particular, for graph classification, they perturbed test graphs by adding a few nodes/edges such that the final prediction remains the same as that for an unperturbed graph. Lower the change in explanation under perturbations better the stability. A challenge here is that there is no principled way to find the perturbations. For example, a part of the explanation might be altered under random perturbations even if the prediction is unchanged.

§ : A UNIFIED FRAMEWORK FOR EVALUATING EXPLANATIONS

We now present our framework  for evaluating GNN explanations. Specifically, unifies existing and our proposed notions into 2 main classes. In the first class of measures we aim at evaluating the explanation methods in the sense that whether they are truly describing the model's workings. The first category includes three metrics: faithfulness, sparsity and correctness. Faithfulness determines if an explanation alone can replicate the model's behavior. Sparsity focuses on rewarding shorter explanations. Correctness determines if the explanation model is able to detect any injected correlations responsible for altering model's behavior.
The metrics in the second class are aimed at evaluating the GNN model itself. Here we propose plausibility which measures how close is the decision making process of the trained model (as revealed by explanations) to human rationales.

§.§ Faithfulness: Can explanations approximate model's behavior?

The key idea here to evaluate the ability of the explanation to characterize model's working. Unlike previous works we argue that there is not a single measure for faithfulness which can be effectively used for all kinds of datasets and explanations. Consequently we propose a set of two measures to quantify faithfulness depending on the dataset/explanation type.

* Rate distortion based fidelity.
The fidelity of an explanation is usually measured by the ability of an explanation to  approximate the model behavior [23].
For explanations which contain the feature attributions with or without structure attributions,
we propose the rate distortion theory based metric proposed in [6] to measure the fidelity of an explanation.
In short, a subgraph of the node's computational graph and its set of features
are relevant for a classification decision if the expected classifier score remains nearly the same when randomizing the remaining features.

The  of explanation $\mathcal{S}$ corresponding to explanation mask $M(\mathcal{S})$ with respect to the GNN $f$, input $X$ and the noise distribution $\mathcal{N}$ is given by

\begin{equation}
\label{eq:fidelity}
\mathcal{F}(\mathcal{S}) = \mathbb{E}_{Y_{\mathcal{S}}|Z\sim \mathcal{N}} \left[\mathbbm{1}_{f\left(X\right)=f(Y_{\mathcal{S}})}\right].
\end{equation}

where the perturbed input is given by

\begin{equation}
Y_{\mathcal{S}} = X\odot M(\mathcal{S}) + Z\odot(\mathbbm{1} - M(\mathcal{S})), Z\sim \mathcal{N},
\label{eq:noisy_features_matrix}
\end{equation}

where $\odot$ denotes an element-wise multiplication, and $\mathbbm{1}$ a matrix of ones with the corresponding size and $\mathcal{N}$ is a noise distribution. We choose the noise distribution to the global empirical distribution of the features. This makes sure that the perturbed data points are still in the same distribution as the original data  [9].

Connection to explanation stability. As shown in [6] explanations with high RDT-fidelity are highly stable. High fidelity score implies that the explanation has high predictive power under perturbations of the rest of the input. Unlike the strategy of [26] to evaluate explanation stability, it is here ensured that the explanation itself is never altered.

The special case of dense feature representations. For some datasets it is more appropriate to consider only structure based explanations. For example, when features themselves are dense representations extracted using some black-box embedding method, feature explanations as well as feature perturbations might not make much sense. It is then more appropriate to check the abilities of the explanation with the rest of nodes/edges removed and keeping the features intact. Towards that we employ the following measures of comprehensiveness and sufficiency also used in [3].

* Comprehensiveness and Sufficiency. For explanations which contain only nodes or/and edges we adapt the comprehensiveness and sufficiency measures of [3] for GNNs.
Let $\mathcal{G}$ is the graph and $\mathcal{G^\prime}\subseteq \mathcal{G}$ is the explanation graph with important (attribution) nodes/edges. Let $f$ be the trained GNN model and $f(\mathcal{G})_j$ be the prediction made by GNN for $j^{th}$ class. We measure fidelity by comprehensiveness (which answers the question if  all nodes/edges in the graph needed to make a prediction were selected?) and sufficiency (if the extracted nodes/edges are sufficient to come up the original prediction?)

\begin{equation}
\textit{sufficiency}=f\left(\mathcal{G}\right)_{j}-f\left(\mathcal{G^\prime}\right)_{j}, \quad \textit{comprehensiveness}=f\left(\mathcal{G}\right)_{j}-f\left(\mathcal{G} \backslash \mathcal{G^\prime}\right)_{j}
\end{equation}

The high comprehensiveness value shows that the prediction is most likely because of the explanation $\mathcal{G^\prime}$ and low comprehensiveness value shows that $\mathcal{G^\prime}$ is mostly not responsible for the prediction. Since most of the explainers retrieve soft masks we employ aggregated comprehensiveness and sufficiency measures. In particular, we divide the soft masks into $|\mathcal{B}|=5$ bins by using top $k\in \mathcal{B} = \{1\%, 5\%, 10\%, 20\%, 50\% \}$ of the explanation with respect to the soft masks values [25]. The aggregated sufficiency is defined as:
$\frac{1}{|\mathcal{B}|}\left(\sum_{k=1}^{|\mathcal{B}|} f\left(\mathcal{G}\right)_{j}-f\left(\mathcal{G^\prime}_{k}\right)_{j}\right).$ The aggregated comprehensiveness is defined in similar fashion.

§.§ Sparsity: Are the explanations non trivial?

High faithfulness ensures that the explanation approximates the model behavior well. However, the complete input completely determines the model behavior. Thus explanation sparsity is an important criteria for evaluation.
Let $p$ be the normalized distribution of explanation (feature) masks. Then sparsity of an explanation is given by the entropy $H(p)$ and is bounded from above by $\log(|M|)$ where $M$ corresponds to a complete set of features or nodes. While an entire input can be a faithful explanation it is important to evaluate an explanation with respect to its size. A shorter explanation is easier to analyse and is more human understandable. We adopt the entropy based definition of sparsity as in [6] because of its applicability to both soft and hard explanation masks.
In particular, let $p$ denote the normalized distribution of node/edge/feature masks. We compute sparsity of an explanation as the entropy over the mask distribution:
$H(p)= -\sum_{\phi\in M} p(\phi) \log p(\phi).$

§.§ Correctness: Can the explanations detect externally injected correlations?

While the above measures are essential that the given explanation is predictive certain applications might need explanations for model debugging, for example to detect any spurious correlations picked up by model thereby increasing model bias. Towards that we measure correctness of an explanation in terms of its ability to recognize the externally injected correlations which alters the model decision. A switch in model decision is an evidence of the use of these injected correlations in the actual decision making process.

In particular, we first choose a set of  incorrectly labelled nodes, $V$. To each such node $v$, we add edges to the nodes in the training data which have the same label as $v$. We call such edges decoys. We retrain the GNN model with the perturbed data. We measure the correctness of explanation $\mathcal{S}$ for nodes in $V$ which are now correctly predicted in terms of precision and recall of the decoys in the returned explanation:
Precision_C = {N_{de} \over N_e}, ~~~~ Recall_C ={N_{de} \over N_d},
where $N_{de}$ is the number of decoys in the obtained explanation, $N_{d}$ total number of decoys injected and $N_{e}$ is the size of the retrieved explanation. Note that our proposed approach of using injected correlations is different from using a synthetic graph with seeded ground truth. In particular, for seeded graph approach it is not always clear if the ground truth is actually picked up by the model to make its decision.

§.§ Plausibility: How close is the model's decision process to humans rationals?

Human congruence or plausibility [16, 15, 32] tries to establish how close or congruent is the trained model to human rationales for solving a predictive task.
Trained models often exhibit the clever-hans effect, that is predictive models can adopt spurious correlations in the training data or due to misplaced inductive biases that have right results for the wrong reasons.
Towards this, data is collected from humans for perceptive tasks where humans explicitly provide their rationales.
These human rationales are in used as ground truth for evaluating if trained models are right for the right reasons.

For applications where obtaining human rationales is indeed possible we propose the use of token-level F1  for binary explanation masks and area under precision recall curve (AUPRC) for soft masks. The reader might have noticed that this metric is similar to the explanation accuracy in earlier works. We argue against the use of term accurate to measure plausibility as similarity to human rationale does not always guarantee that the model has learnt an explanation which contains the reasoning of the model itself and not only of the humans.

§ EXPERIMENTAL SETUP

Models and Explainers. We demonstrate the use and advantage of the proposed framework by evaluating $9$ explanation methods over $6$ datasets and $4$ GNN models.  Currently our benchmark consists of these GNN models:
graph convolutional networks () [13], graph attention network () [34], the approximation of personalized propagation of neural predictions () [14], and graph isomorphism network () [37]. The models were chosen based on their differences in (i) exploiting inductive biases (based on different feature aggregation strategies), (ii) test performance (see tables <ref> and <ref> in the Appendix) and (iii) response to injected correlations (see Table <ref> and the corresponding discussion).

We perform experiments with perturbation based approaches like GNNExplainer () [38] and Zorro [6], surrogate methods like PGM-Explainer (PGM) [35], and gradient-based approaches like Grad [29], GradInput[29], Integrated Gradient (IG) [33], SmoothGrad [31], CAM [22] and GradCAM [22].  returns soft feature masks and edge masks. We transform the edge masks into node masks, in which we equally distribute the edge importance score to both nodes sharing the edge. As already mentioned is extendable, and more approaches and explainers can be easily added.

§.§ Datasets

We now describe new and existing datasets used in our evaluation framework and the corresponding rationale.

New Dataset for Plausibility. To measure the plausibility of an explanation, we first require the corresponding human rationales. Since the existing graph datasets do not have such annotated information, we transform a text sentiment prediction task into a graph classification task. Specifically, we adopt the Movie Reviews dataset [42] from the ERASER benchmark [3].
The task is the binary classification, which is the differentiation between positive and negative movie reviews. A detailed description of dataset construction is given in Appendix <ref>.

Dataset to measure Comprehensiveness and Sufficiency. Note that the measures comprehensiveness and sufficiency are only applicable for node/edge explanations. We use the above-described Movie Review dataset for these two measures too. The rationale is that the node features are generated using Glove and are not human-understandable. In this case, a structure-based explanation would be more meaningful than a feature-based one.

Datasets for Correctness.
We employ two citation datasets Cora [28] and CiteSeer [28].
After injecting correlations/decoys corresponding to incorrectly labeled nodes as described in Section <ref>, we re-train the GNN model. The rationale behind adding homophily increasing correlations is the observation from previous works [12, 44] that GNN's performance increases with higher homophily. A model will have picked up these correlations if the previous incorrect nodes are now correctly predicted. We further evaluate the correctness of the explanation only for newly correctly predicted nodes.

Datasets for RDT-Fidelity.

We perform the RTD-Fidelity evaluation on both node classification and graph classification tasks. At node level, we use Citation datasets. Table <ref> shows the dataset statistics and GNNs performances.
For node classification task, we select 300 nodes for Cora and CiteSeer and 700 nodes for PubMed randomly. For graph classification task, we select 50 graphs for both MUTAG [2] and PROTEINS [1] datasets.

§ RESULT ANALYSIS

§.§ Faithfulness

Results for RDT-Fidelity for node classification.

2*Mask     2*Methods     4cCora     4cCiteSeer     4cPubMed

GCN    GAT     GIN     APPNP     GCN     GAT     GIN     APPNP     GCN     GAT     GIN     APPNP

2*Hard     Zorro         0.97      0.97      0.96      0.97     0.97      0.97      0.97      0.96       0.96      0.97      0.97     0.96

PGM       0.84      0.77      0.60      0.89     0.92     0.93      0.73      0.95       0.78      0.69      0.74      0.96

6*Soft          0.71      0.66      0.52      0.65         0.68      0.69      0.51      0.62       0.67      0.73      0.67      0.72

Grad                0.15      0.18      0.19      0.17         0.17      0.19      0.28      0.18       0.37      0.43      0.42      0.37

GradInput           0.15      0.18      0.18      0.16         0.16      0.18      0.26      0.17       0.36      0.42      0.42      0.36

SmoothGrad        0.44      0.42     0.38     0.50      0.54      0.57      0.45     0.62       0.52      0.53    0.67     0.59

IG          0.45     0.47     0.26     0.51      0.53      0.70      0.45     0.62     0.52     0.56      0.68     0.59

Empty Expl.           0.15      0.18      0.18      0.16         0.16      0.18      0.26      0.17       0.36      0.42      0.42      0.36

In Table <ref> we compare the RDT-Fidelity scores of various explanation methods. As a trivial baseline, we include an empty explanation baseline, i.e., all features are set randomly using the noise distribution. A common feature of and is that they both learn the explanations from a sampled local dataset. The local dataset is created by perturbing the features of nodes from the computational graph (neighborhood of query nodes). While they employ different optimization strategies to find explanations, the result is a stable explanation that also reflects our results. The gradient-based explanations achieve the lowest fidelity.

For the graph classification task (see Table <ref> in Appendix), all methods, including the gradient-based approaches, perform relatively well except for the model. shows more consistent performance across all models and datasets. We leave out as it is not applicable for the graph classification task.

Faithfulness as comprehensiveness and sufficiency measured for dataset. Low sufficiency and high comprehensiveness indicates high faithfulness.

2cGCN       2cGAT      2cGIN     2cAPPNP

Suff.     Comp.    Suff.   Comp.    Suff.    Comp.    Suff.    Comp.

0.56    -0.01     0.47    0.03      0.24    0.32     0.48    0.07

Grad              0.02     0.39      0.15    0.16     0.25    0.31      0.11    0.20

GradInput         0.07    0.36     0.14    0.16     0.22     0.33      0.12    0.24

SmoothGrad     0.08     0.34     0.15    0.23      0.25    0.27      0.14    0.23

IG         0.11    0.38      0.16    0.21     0.23    0.27     0.16    0.24

CAM         0.41    0.01     0.11    0.14     0.26    0.26     0.39    0.05

GradCAM         0.02     0.29      0.06    0.27     0.21    0.27      0.07    0.28

Comprehensiveness and Sufficiency. We evaluate faithfulness for explanations for the dataset using aggregated comprehensiveness and sufficiency measures. The results for soft-mask explanations are shown in Table <ref>. has the lowest sufficiency which suggests that the explanations are sufficient to mimic the prediction of GNN models. On the other hand, the explanations generated by with and suggest that there still exists important part of the input outside of the explanations which are required to approximate the GNN's prediction. On the other hand, , which so far outperformed gradient-based explanations for the node classification task, shows the worst sufficiency and comprehensiveness. Even if we use the complete feature set and only the node masks to evaluate explanations, the node masks for are learned together with the feature explanations. This differs from gradient-based approaches, which ignore feature and structure explanation trade-offs. The current performance of indicates that it might not be appropriate to use entangled features and structure explanations independently.

§.§ Sparsity

As already mentioned, a complete input could already do well on all faithfulness measures. Therefore, we further look for sparser explanations. The results for node sparsity for explanations in node classification task are provided in Table <ref>.
For the hard masking approaches (Zorro and PGM), Zorro outperforms PGM with all GNN models except for GIN. Conversely, there is no clear winner for the soft mask approach. The high sparsity for soft-masking approaches implies a near-uniform node attribution and consequently lower interpretability. In general, faithfulness and sparsity of an explanation should be analyzed together. A uniformly distributed explanation mask could already provide an explanation with high faithfulness as it leads to using the complete input as an explanation.

Results for sparsity (computed as entropy over mask distribution) for node classification. The lower the score sparser is the explanation.

2*Mask     2*Methods     4cCora     4cCiteSeer     4cPubMed

GCN    GAT     GIN     APPNP     GCN     GAT     GIN     APPNP     GCN     GAT     GIN     APPNP

2*Hard     Zorro       1.58      1.59      2.17      1.48         1.26      1.09      1.58      1.07       1.51      1.31      2.18      1.25

PGM         2.06     1.82      1.66     1.99     1.47      1.59     1.10     1.54     1.64      1.16     1.62      2.93

5*Soft           2.48      2.49      2.56      2.51         1.67      1.67      1.70      1.68        2.70      2.71      2.71      2.71

Grad                 2.48      2.34      2.25      2.35         1.70      1.61      1.55      1.60       2.91      2.76      3.11      2.73

GradInput             2.53      2.43      2.23      2.41         1.61      1.58      1.54      1.52       3.02      2.94      3.41      2.81

SmoothGrad    2.48     2.52     2.91     2.31     1.77    1.77    1.93     1.66     2.89     3.02      3.23     2.54

IG     2.49    2.50      2.84    2.31     1.76    1.77    1.91     1.66     2.84     2.89      3.06     2.58

§.§ Correctness

The number of incorrectly labelled nodes () decreases after addition of decoys. The number of new correctly labelled nodes after injecting decoys is listed under .

2*Model      3cCora      3cCiteSeer

$\uparrow$(%)               $\uparrow$(%)

GCN       88     79    89.7     329      229     69.6

GAT       86     85     98.8     311      301     96.7

GIN       6     6     100     56      56     100

APPNP      73     70     95.8     280     252    90.0

The correctness results corresponding to different models and explainers are reported for (in
Table <ref>) and (in Table <ref>). We report precision, recall and F1 score by choosing top k nodes for the soft explanations. For hard masked approaches the number of returned nodes is listed under $|\mathcal{S}|$. Note that the number of decoys added per node is $10$. For Table <ref> and Table <ref> we use $k=20$.

In Table <ref>, the effect of decoys can be seen where most of the earlier incorrectly classified nodes are now correctly classified except for GCN on CiteSeer.
We also observe that number of selected nodes for GIN is very low for Cora dataset (i.e., only a few nodes were initially incorrectly labeled).
outperforms all other based explainers in detecting the injected correlations for both Cora and CiteSeer (detailed results moved to Table <ref> in the Appendix due to space constraints).

Comparing soft mask and hard mask approaches in this setting is tricky as for some approaches like , we cannot control the explanation size. For example, for retrieved an explanation of size 40. A precision of 0.25 shows that it found all 10 injected correlations. Lack of feature ranking, as in soft mask approaches, makes it difficult to evaluate hard mask approaches for Correctness.
For fairer evaluation, we further plot the performance of soft mask approaches with different $k$ in Appendix <ref>. For example, the  shows high improvement when we increase the size of the explanation to 15. It is not surprising to see the performance degrades when we increase the size of the explanation further since it already had captured all injected decoys. Now it returns some irrelevant nodes in the explanation. Furthermore, in Table <ref> and <ref>, we use mean as a threshold to generate hard masks. As the mean threshold turns out to be very low for all approaches, almost all nodes of the computational graph are selected as the explanation. Consequently, we observe a very low correctness score (when measured in terms of precision).

Correctness  of the explanation for node classification on Cora dataset. We use $k=20$.

3*Mask   3*Methods      16cCora

4cGCN     4cGAT     4cGIN     4cAPPNP

P@k     R@k    F1     |$\mathcal{S}$|    P@k     R@k    F1     |$\mathcal{S}$|     P@k     R@k    F1     |$\mathcal{S}$|     P@k     R@k    F1     |$\mathcal{S}$|

2*Hard        0.19     0.80     0.30     45     0.25    0.83      0.37    40     0.26     0.45    0.27     33      0.22     0.79     0.33     38

PGM          0.11   0.22    0.15     20      0.18   0.36   0.24     20      0.18   0.36    0.25     20     0.19   0.38    0.25   20

5*Soft          0.42   0.84   0.56    20      0.44   0.88   0.59    20     0.50   1.00   0.67    20      0.34   0.67    0.58    20

Grad       0.23   0.46    0.31      20    0.29   0.58     0.39     20     0.30   0.60     0.40    20     0.33   0.67    0.45    20

GradInput      0.16   0.32    0.21     20     0.28   0.56     0.34    20      0.30   0.60    0.40    20     0.28   0.56     0.38    20

SmoothGrad       0.12    0.25     0.16    20      0.24   0.48    0.32    20    0.50   1.00     0.67    20     0.22   0.43     0.29    20

IG        0.16   0.32    0.22    20     0.24   0.49    0.33    20      0.50   1.00    0.67    20     0.28   0.55    0.37    20

§.§ Plausibility

Table <ref> shows the Plausibility scores computed for explanations of different GNN models. Recall that we compare explanations with human rationales to compute plausibility. The average size of human rationales over the test dataset is 165. To compute token level F1 score, we use mean as a threshold to generate hard masks from soft masks.

We observe that all explainers assign the best plausibility scores to . obtains the second-best plausibility scores. We also observe that the overall difference in the plausibility scores over models is relatively small, with some exceptions like the combination of and . The corresponding explanation also has the largest size. This further highlights the issues of soft-hard mask conversion. AUPRC scores which directly use the soft masks are more stable.

One surprising fact in these results is that even though other GNN models achieve higher test accuracy than (see Table <ref> in the Appendix). Overall, their explanations have similar plausibility as for except for . In such cases, an application user might want to look in more detail at specific correctly labeled instances to check if the model imitates human reasoning.

Plausibility for movie review dataset measured by auprc and F1 score (macro). |$\mathcal{S}$| represents the average size of the explanations generated by the explainers.

2*Mask     2*Methods
3cGCN         3cGAT         3cGIN        3cAPPNP

auprc     F1    |$\mathcal{S}$|      auprc     F1    |$\mathcal{S}$|        auprc    F1    |$\mathcal{S}$|       auprc    F1    |$\mathcal{S}$|

Hard     PGM       —      0.42     25       —       0.43     25       —     0.43    25        —    0.43    25

6*Soft        0.46      0.54    168        0.43    0.54   149        0.45    0.35    410       0.45     0.53    158

Grad      0.44     0.52     265       0.38     0.51   158        0.40    0.52    156        0.38     0.50   255

GradInput     0.39    0.51    221        0.37     0.50   154        0.39    0.51    154        0.37     0.50    227

SmoothGrad      0.40     0.52    219        0.37    0.50   154        0.40     0.52    172        0.38    0.50    221

IG      0.37     0.49   225       0.37     0.50    188        0.39     0.51    186        0.38     0.50   219

CAM     0.54      0.61   224        0.40     0.51   177       0.44       0.55   156        0.44     0.53   195

GradCAM     0.67     0.34   175        0.67    0.35    191        0.67     0.34   166        0.67    0.34   188

§ CONCLUSION

We develop a unified, modular, extendable benchmark called to evaluate GNN explanations on four diverse axes: 1) faithfulness, 2) sparsity, 3) correctness, and 4) plausibility.
We summarize the key insights of our experiments on  here:

* Faithfulness measured via RDT-Fidelity can be employed for a wide set of tasks and datasets. The only exception is when the input features are themselves non-interpretable. In that case, one should prefer structure-based explanations.

* High RDT-Fidelity also implies high explanation stability.

* The  comprehensiveness and sufficiency measures should be used to evaluate the faithfulness of structure-based explanations where perturbing features might not be feasible.

* It is important to measure the sparsity of the explanation to avoid the extreme case of using the whole input as an explanation.

* Correctness should be used with care, as injecting appropriate correlations to change a model's decision is not always straightforward. Approaches based on adversarial attacks to manipulate the GNN's prediction [11] can be adopted.

* Plausibility measures the joint utility of the explanation method and the trained GNN model with respect to human rationales.
This means that the loss of plausibility can be either due to human-incongruent correlations or the explainer, and it is impossible to disentangle them.

[1]
K. M. Borgwardt, C. S. Ong, S. Schönauer, S. Vishwanathan, A. J. Smola, and
H.-P. Kriegel.
Protein function prediction via graph kernels.
Bioinformatics, 21(suppl_1):i47–i56, 2005.

[2]
A. K. Debnath, R. L. Lopez de Compadre, G. Debnath, A. J. Shusterman, and
C. Hansch.
Structure-activity relationship of mutagenic aromatic and
heteroaromatic nitro compounds. correlation with molecular orbital energies
and hydrophobicity.
Journal of medicinal chemistry, 34(2):786–797, 1991.

[3]
J. DeYoung, S. Jain, N. F. Rajani, E. Lehman, C. Xiong, R. Socher, and B. C.
Eraser: A benchmark to evaluate rationalized nlp models.
arXiv preprint arXiv:1911.03429, 2019.

[4]
T. N. Dong, S. Mucke, and M. Khosla.
Mucomid: A multitask graph convolutional learning framework for
mirna-disease association prediction.
IEEE/ACM Transactions on Computational Biology and
Bioinformatics, 2022.

[5]
L. Faber, A. K. Moghaddam, and R. Wattenhofer.
When comparing to ground truth is wrong: On evaluating gnn
explanation methods.
In Proceedings of the 27th ACM SIGKDD Conference on Knowledge
Discovery & Data Mining, pages 332–341, 2021.

[6]
T. Funke, M. Khosla, M. Rathee, and A. Anand.
Zorro: Valid, sparse, and stable explanations in graph neural
arXiv preprint arXiv:2105.08621, 2021.

[7]
T. Gaudelet, B. Day, A. R. Jamasb, J. Soman, C. Regep, G. Liu, J. B. R. Hayter,
R. Vickers, C. Roberts, J. Tang, D. Roblin, T. L. Blundell, M. M. Bronstein,
and J. P. Taylor-King.
Utilizing graph machine learning within drug discovery and
Briefings in Bioinformatics, 22(6), 05 2021.

[8]
W. L. Hamilton, R. Ying, and J. Leskovec.
Inductive representation learning on large graphs.
In NIPS, 2017.

[9]
S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim.
A benchmark for interpretability methods in deep neural networks.
In Advances in Neural Information Processing Systems, pages
9737–9748, 2019.

[10]
Q. Huang, M. Yamada, Y. Tian, D. Singh, D. Yin, and Y. Chang.
Graphlime: Local interpretable model explanations for graph neural
arXiv:2001.06216, 2020.

[11]
W. Jin, Y. Li, H. Xu, Y. Wang, S. Ji, C. Aggarwal, and J. Tang.
Adversarial attacks and defenses on graphs.
ACM SIGKDD Explorations Newsletter, 22(2):19–34, 2021.

[12]
M. Khosla, V. Setty, and A. Anand.
A comparative study for unsupervised network representation learning.
TKDE, 2019.

[13]
T. N. Kipf and M. Welling.
Semi-supervised classification with graph convolutional networks.
In ICLR, 2017.

[14]
J. Klicpera, A. Bojchevski, and S. Günnemann.
Predict then propagate: Graph neural networks meet personalized
International Conference on Learning Representations (ICLR),

[15]
I. Lage, E. Chen, J. He, M. Narayanan, B. Kim, S. Gershman, and F. Doshi-Velez.
An evaluation of the human-interpretability of explanation.
arXiv preprint arXiv:1902.00006, 2019.

[16]
T. Lei, R. Barzilay, and T. Jaakkola.
Rationalizing neural predictions.
arXiv preprint arXiv:1606.04155, 2016.

[17]
Y. Liu, S. Khandagale, C. White, and W. Neiswanger.
Synthetic benchmarks for scientific research in explainable machine
arXiv preprint arXiv:2106.12543, 2021.

[18]
H. Lu and S. Uddin.
A weighted patient network-based framework for predicting chronic
diseases using graph neural networks.
Scientific reports, 11(1):1–12, 2021.

[19]
S. M. Lundberg and S.-I. Lee.
A unified approach to interpreting model predictions.
In Advances in neural information processing systems, pages
4765–4774, 2017.

[20]
D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang.
Parameterized explainer for graph neural network.
arXiv preprint arXiv:2011.04573, 2020.

[21]
J. Pennington, R. Socher, and C. D. Manning.
Glove: Global vectors for word representation.
In Proceedings of the 2014 conference on empirical methods in
natural language processing (EMNLP), pages 1532–1543, 2014.

[22]
P. E. Pope, S. Kolouri, M. Rostami, C. E. Martin, and H. Hoffmann.
Explainability methods for graph convolutional neural networks.
In Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, pages 10772–10781, 2019.

[23]
M. T. Ribeiro, S. Singh, and C. Guestrin.
" why should i trust you?" explaining the predictions of any
In Proceedings of the 22nd ACM SIGKDD international conference
on knowledge discovery and data mining, pages 1135–1144, 2016.

[24]
F. Rousseau and M. Vazirgiannis.
Graph-of-word and tw-idf: new approach to ad hoc ir.
In Proceedings of the 22nd ACM international conference on
Information & Knowledge Management, pages 59–68, 2013.

[25]
W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K.-R. Müller.
Evaluating the visualization of what a deep neural network has
IEEE transactions on neural networks and learning systems,
28(11):2660–2673, 2016.

[26]
B. Sanchez-Lengeling, J. Wei, B. Lee, E. Reif, P. Wang, W. W. Qian,
K. McCloskey, L. Colwell, and A. Wiltschko.
Evaluating attribution for graph neural networks.
Advances in neural information processing systems, 33, 2020.

[27]
M. S. Schlichtkrull, N. De Cao, and I. Titov.
Interpreting graph neural networks for nlp with differentiable edge
arXiv preprint arXiv:2010.00577, 2020.

[28]
P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad.
Collective classification in network data.
AI magazine, 29(3):93–93, 2008.

[29]
K. Simonyan, A. Vedaldi, and A. Zisserman.
Deep inside convolutional networks: Visualising image classification
models and saliency maps.
arXiv preprint arXiv:1312.6034, 2013.

[30]
J. Singh and A. Anand.
Model agnostic interpretability of rankers via intent modelling.
In Conference on Fairness, Accountability, and Transparency,

[31]
D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg.
Smoothgrad: removing noise by adding noise.
arXiv preprint arXiv:1706.03825, 2017.

[32]
J. Strout, Y. Zhang, and R. J. Mooney.
Do human rationales improve machine explanations?
arXiv preprint arXiv:1905.13714, 2019.

[33]
M. Sundararajan, A. Taly, and Q. Yan.
Axiomatic attribution for deep networks.
In Proceedings of the 34th International Conference on Machine
Learning-Volume 70, pages 3319–3328. JMLR. org, 2017.

[34]
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò,
and Y. Bengio.
Graph Attention Networks.
ICLR, 2018.

[35]
M. N. Vu and M. T. Thai.
Pgm-explainer: Probabilistic graphical model explanations for graph
neural networks.
In Advances in Neural Information Processing Systems 33: Annual
Conference on Neural Information Processing Systems 2020, NeurIPS 2020,

[36]
S. Wiegreffe and A. Marasović.
Teach me to explain: A review of datasets for explainable nlp.
arXiv preprint arXiv:2102.12060, 2021.

[37]
K. Xu, W. Hu, J. Leskovec, and S. Jegelka.
How powerful are graph neural networks?
In International Conference on Learning Representations, 2019.

[38]
R. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec.
Gnn explainer: A tool for post-hoc explanation of graph neural
arXiv preprint arXiv:1903.03894, 2019.

[39]
R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec.
Graph convolutional neural networks for web-scale recommender
In Proceedings of the 24th ACM SIGKDD international conference
on knowledge discovery & data mining, pages 974–983, 2018.

[40]
H. Yuan, J. Tang, X. Hu, and S. Ji.
Xgnn: Towards model-level explanations of graph neural networks.
In KDD '20, page 430–438. Association for Computing
Machinery, 2020.

[41]
H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji.
On explainability of graph neural networks via subgraph explorations.
arXiv preprint arXiv:2102.05152, 2021.

[42]
O. Zaidan and J. Eisner.
Modeling annotators: A generative approach to learning from annotator
In Proceedings of the 2008 conference on Empirical methods in
natural language processing, pages 31–40, 2008.

[43]
Y. Zhang, D. Defazio, and A. Ramesh.
Relex: A model-agnostic relational model explainer.
In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics,
and Society, pages 1042–1049, 2021.

[44]
J. Zhu, Y. Yan, L. Zhao, M. Heimann, L. Akoglu, and D. Koutra.
Beyond homophily in graph neural networks: Current limitations and
effective designs.
Advances in Neural Information Processing Systems,
33:7793–7804, 2020.

§ APPENDIX

§ MORE DETAILS ON DATASETS

In this section we describe in detail the tasks and datasets used in our framework.

§.§ Datasets

For node classification task, we use citation datasets  [28], namely, Cora, CiteSeer and PubMed. In these citation datasets, each paper represents node and there is an edge between two papers if one cites other. Each node has its input features vectors and a label.
We use the semi-supervised setting for data spilt where only 20 nodes per label are used in training and 500 nodes for validation and 1000 nodes for test data. The details are available in Table <ref>. For correctness experiment, we use the fully supervised split where all the nodes except validation and test data are in training data.

For graph classification task, we use molecules datasets like PROTEINS [1] and MUTAG [2] and text dataset like Movie Reviews [42]. In PROTEINS dataset, each graph represents protein and task is to classify the protein into enzymes or non-enzymes. The MUTAG dataset contains 188 chemical compounds and task is to classify whether the compound has a mutagenic effect on a bacterium. The is a text dataset which we transform into a graph dataset.  In the transformed  dataset each graph represents a movie review and the task is to classify the sentiment of the review as Positive or Negative. The data statistics and the model accuracy are reported in Table <ref>. We now describe the construction of dataset.

Construction of Movie Reviews Dataset. Each input instance or review is a passage of text, typically with multiple sentences. Each input review is annotated by humans that reflects the actual "human" reasons for predicting the sentiment of the review. These annotations are are extractive pieces of text and we call them human rationales. We transform sentences into graphs using the graph-of-words approach [24].
As a pre-processing step, we remove stopwords, such as "the" or "a". The complete list of used stopwords is included in our repository. Each word is represented as a node and all words within a sliding window of three are connected via edges. As features, we use the output of a pre-trained Glove model [21]. We then train a 2-layer GNN model with global average/attention pooling on the defined training set and record in Table <ref> the achieved performance. Further details about dataset statistics and GNNs performances are available in Section <ref>. Figure <ref> provides an example of a graph from dataset.

An example for text to graph generation. The graph is corresponding to input sentences "? romeo and juliet ' , and ? the twelfth night ' . it is easier for me to believe that he had a wet dream and that 's how all his plays develop , but please spare me all of this unnecessary melodrama."

§.§ Dataset statistics and Model Performance

Datasets statistics and model performance for node classification.

4c Test Accuracy

Dataset     Class     $d$     $|V|$     $|E|$     GCN     GAT     GIN     APPNP

Cora     7     1433     2708     10556     $0.794$     $0.791$     $0.679$     $0.799$

CiteSeer     6     3703     3327     9104     $0.675$     $0.673$     $0.480$     $0.663$

PubMed     3     500     19717     88648     $0.782$     $0.765$     $0.590$     $0.782$

Datasets statistics and model performance for Graph classification.

4c Test Accuracy

Dataset     #Graphs     Class     $d$     Avg. $|V|$     Avg. $|E|$     GCN     GAT     GIN     APPNP

MUTAG    188      2   7     17.9     39.6     0.76     0.76    0.79     0.76

PROTEINS    1113    2     3     39.1     145.6     0.66     0.58     0.69     0.58

Movie Reviews    2000    2     300     500.8    1997.5    0.85    0.85     0.78     0.82

§ DETAILS OF EXPLAINERS

In the following section, we  summarize the GNN explanation approaches currently implemented in .

§.§ Grad

For a given node $v$ and the trained GNN model $f(\mathbf{x}_v, \mathcal{G},\phi)$, where $\phi$ is set of parameters , the gradient based approach, Grad[29], generate an explanation by assigning importance scores to the input features. The score corresponding to a feature reveals the importance of this feature in making prediction. The high score corresponds to high importance. The explanation for node $v$ is given by

\begin{equation}
\mathcal{S}(\mathbf{x}_v)=  \frac{\partial f}{\partial \mathbf{x}_v}
\end{equation}

the gradients of $f$ with respect to input features $\mathbf{x}_v$.

§.§ GradInput

GradInput explanations are transformed version of Gradient based explanation by doing element-wise multiplication with input features. The explanation for node $v$ is given by:

\begin{equation}
\mathcal{S}(\mathbf{x}_v)=  \frac{\partial f}{\partial \mathbf{x}_v}\odot{\mathbf{x}_{v}}
\end{equation}

§.§ Integrated Gradient (IG)
The gradient based explanations are often noisy and can sometimes be insensitive with respect to input, for example if the learnt function $f$ is a straight line then the gradients are same with respect to different inputs. IG [33] proposed the interpolation based gradient method which cumulates the gradients along straight path between input features vector $\mathbf{x}_v$ and a baseline vector $\mathbf{x}^\prime$ of all zeros or ones. The explanation for node $v$ is given by:

\begin{equation}
\mathcal{S}(\mathbf{x}_v) =\left(\mathbf{x}_v-\mathbf{x}^{\prime}\right) \times \int_{\alpha=0}^{1} \frac{\partial f\left(\mathbf{x}^{\prime}+\alpha \times\left(\mathbf{x}_v-\mathbf{x}^{\prime}\right)\right)}{\partial \mathbf{x}_v} d \alpha
\end{equation}

§.§ SmoothGrad

The SmoothGrad [31] claims that the gradients of model $f$ may saturate, which means the importance score of a feature cause a strong effect globally but it shows very low effect locally. Also, the gradient based explanations are noisy. It proposed the mechanism of smoothing the noisy gradients by adding noise to the input. This process generate extra samples for training and the explanations are more robust to the noise. The explanations of SmoothGrad are given by:

\begin{equation}
\hat{\mathcal{S}}(\mathbf{x}_{v})=\frac{1}{n} \sum_{1}^{n} \mathcal{S}\left(\mathbf{x}_{v}+\mathcal{N}\left(0, \sigma^{2}\right)\right)
\end{equation}

where $n$ is the number of samples and $\mathcal{N}\left(0, \sigma^{2}\right)$ is the noise distribution.

§.§ CAM

CAM [22] explains the graph classification tasks only. For graph classification tasks, we use an additional global average pooling (GAP) layer to generate the representation of the graph. For a graph $\mathcal{G}$, let $f_\mathcal{G}$ be the trained model for graph classification task which can be represented as:

\begin{equation}
f_{\mathcal{G}}(\mathcal{A}, \mathcal{X}) = \sigma(\operatorname{GAP}(\mathbf{x}^{(L)}_{u} \mid u \in \mathcal{G}), \textit{w})
\end{equation}

where $L$ represents the final layer of GNN, $\mathbf{x}_{u}^{(L)}$ is the representation of node $u$ at layer $L$ and $\textit{w}$ represents the trainable parameters for the classification layer and $\sigma$ is the activation function. CAM assumes that the final layer representations encode the input feature behavior corresponds to prediction and the input feature importance is given by weighted sum of different features. The explanation of the graph $\mathcal{G}$ is given by:

\begin{equation}
\mathcal{S(G)}=\operatorname{ReLU}\left(\sum_{k} w_{k} \mathcal{X}_{k}^{(L)}\right)
\end{equation}

where $\mathcal{X}_{k}^{(L)}$ represents the $k^{th}$ feature at layer $L$.

§.§ GradCAM

GradCAM [22] is an extended version of CAM designed for graph classification based GNN models which does not need the global average pooling (GAP) layer. It also uses the final representation to generate the importance score. It assigns the gradient based weights to the representation.  The weight for $k^{th}$ feature at layer $l$ is computed as:

\begin{equation}
\alpha_{k}^{(l)}=\frac{1}{N} \sum_{n=1}^{N} \frac{\partial f_\mathcal{G}}{\partial \mathcal{X}_{k}^{(l)}}
\end{equation}

where $N$ is the number of nodes in graph $\mathcal{G}$. Finally the explanation of the graph is given by:

\begin{equation}
\mathcal{S(G)}=\operatorname{ReLU}\left(\sum_{k} \alpha_{k}^{l} \mathcal{X}_{k}^{(l)})\right)
\end{equation}

§.§ Zorro

Zorro[6] is a greedy-combinatorial algorithm, which optimizes RDT-Fidelity, to receive valid, sparse, and stable explanations. Zorro iteratively extends the explanation with the feature or node, which yields the highest increase in RDT-Fidelity. The addition of features and nodes is stopped, when a pre-defined threshold is reached. Hence, it automatically determines the explanation size. For the formulation and details of RDT-Fidelity see Section <ref>. The authors in [6] evaluated two different RDT-Fidelity thresholds. Here, we performed the experiments using the higher threshold of $\tau=0.98$. Zorro is designed for node classification and is model agnostic. The explanations consists of hard feature and hard node masks.

§.§ PGM

PGM-Explainer () [35] is a surrogate based method which fits a simple and interpretable model on a sampled local dataset. The first step is data generation which is collection of perturbed input for a given target node and its prediction. Secondly the variable selector removes the irrelevant samples from the generated data and finally a Bayesian network is trained on the sampled data and the explanations generated by Bayesian network are treated as explanations of GNN model for the target node. retrieves hard masks over the nodes. explains both node and graph classification tasks.

For a given graph $\mathcal{G}$,  [38] learns subgraph $\mathcal{G^\prime}\subseteq \mathcal{G}$ which contains the important graph structure (mainly edges) and import features which are responsible for the prediction. For the subgraph, learns the soft masks over edges and features which are optimized using the mutual information between the prediction of GNN on $\mathcal{G}^{\prime}$ and the prediction ($Y$) of GNN on $\mathcal{G}$. Mathematically,

\begin{equation}
\max _{\mathcal{G}^{\prime}} M I\left(Y,\mathcal{G}^{\prime}\right) =
H(Y)-H\left(Y \mid \mathcal{G}=\mathcal{G}^{\prime}\right)
\end{equation}

where $H(.)$ is the entropy. learns masks locally and applicable to all graph based tasks like graph classification, node classification and link prediction.

§ RESULTS OF RDT-FIDELITY ON GRAPH CLASSIFICATION

In Table <ref>, we report RDT-Fidelity for graph classification task.

RDT-Fidelity for graph classification.

2*Mask     2*Methods     4cMUTAG     4cPROTEINS

GCN    GAT     GIN     APPNP     GCN     GAT     GIN     APPNP

1*Hard    PGM           0.58     1.00      0.60     1.00         0.97     1.00     0.89     1.00

7*Soft    GNNExp        0.73      0.96     0.75      0.99     0.38    0.46     0.23      1.00

Grad               0.39     0.86      0.58      0.96      0.93      0.90      0.90     0.94

GradInput          0.38     0.87      0.58      0.96       0.93      0.91      0.90      0.94

SmoothGrad           0.38      0.86      0.59      0.96       0.93      0.90      0.90      0.94

IG           0.38      0.85      0.59     0.96     0.93      0.92      0.90      0.94

CAM           0.39      0.85      0.59      0.97      0.93     0.91      0.90      0.95

GradCAM          0.38      0.85      0.58     0.96     0.93      0.90      0.89      0.95

§ RESULTS OF CORRECTNESS ON

We report correctness for dataset in Tablw <ref>.

Correctness  of the explanation for node classification on CiteSeer dataset. We use $k=20$.

3*Mask   3*Methods      16cCiteSeer

4cGCN     4cGAT     4cGIN     4cAPPNP

P@k     R@k    F1     |$\mathcal{S}$|     P@k     R@k    F1     |$\mathcal{S}$|     P@k     R@k    F1     |$\mathcal{S}$|     P@k     R@k    F1     |$\mathcal{S}$|

2*Hard     ZORRO        0.17    0.59    0.24      36     0.31   0.41    0.33      14      0.19     0.53     0.25      35    0.20    0.61    0.25      40

PGM          0.17   0.34      0.23     20     0.18   0.36     0.24      20      0.19    0.39    0.25      20      0.17   0.34      0.23      20

5*Soft         0.47   0.94      0.63      20     0.46   0.92      0.62      20     0.47   0.95      0.63     20      0.46   0.92     0.61      20

Grad        0.04   0.08      0.06      20    0.28   0.56       0.38      20     0.23   0.46      0.31     20     0.23   0.46     0.31      20

GradInput     0.02   0.04      0.02      20     0.25   0.50      0.29      20    0.26   0.51     0.34      20     0.09   0.17      0.12      20

IG        0.02   0.04    0.03      20     0.23   0.46      0.31      20     0.37   0.73     0.49      20     0.13   0.25     0.17      20

SmoothGrad       0.01   0.02    0.02      20     0.32   0.64     0.43     20      0.29   0.58     0.38     20      0.06   0.12      0.08      20

§ DIFFERENT STRATEGIES TO COMPUTE HARD MASK FROM SOFT MASK TO MEASURE CORRECTNESS

We report correctness for and in Table <ref> and <ref> with mean as threshold for generating hard masks.

Correctness  of the explanation on Cora dataset. We use mean as threshold to generate hard masks. |$\mathcal{S}$| represents size of the explanations.

3*Mask   3*Methods      12cCora

3cGCN     3cGAT    3cGIN     3cAPPNP

Pre.     Rec.   |$\mathcal{S}$|     Pre.     Rec.   |$\mathcal{S}$|     Pre.     Rec.   |$\mathcal{S}$|     Pre.    Rec.    |$\mathcal{S}$|

2*Hard     ZORRO       0.19     0.80     45     0.25    0.83      40     0.26     0.45     33      0.22     0.79     38

PGM          0.11   0.22    20    0.18   0.36   20    0.18   0.36    20    0.19   0.38   20

5*Soft     GNNExplainer          0.11       1.00     108     0.11     1.00    109   0.26     1.00   39    0.11      1.00    107

Grad         0.12      0.99    94    0.12       1.00   91    0.24     0.90   38      0.12      0.99   94

GradInput       0.11       1.00   98      0.11     1.00   106    0.26    1.00   39     0.11      1.00    99

IG           0.11      1.00    98     0.11      1.00   100    0.26     1.00    39    0.11      1.00   101

SmoothGrad         0.11      1.00   104     0.11     1.00    102.   0.26     1.00    39     0.11     1.00   102

Correctness  of the explanation for CiteSeer dataset. We use mean as threshold to generate hard masks. |$\mathcal{S}$| represents size of the explanations.

3*Mask   3*Methods      12cCiteSeer

3cGCN     3cGAT     3cGIN     3cAPPNP

Pre.     Rec.   |$\mathcal{S}$|     Pre.     Rec.   |$\mathcal{S}$|     Pre.     Rec.   |$\mathcal{S}$|     Pre.    Rec.    |$\mathcal{S}$|

2*Hard     ZORRO        0.17    0.59     36     0.31   0.41   14      0.19   0.53   35     0.20   0.61   40

PGM          0.17   0.34    20   0.18   0.36     20     0.17   0.34     20     0.17   0.34     20

5*Soft     GNNExplainer         0.11      1.00    99    0.10      1.00   111    0.19    1.00   60     0.10     1.00   106

Grad          0.11       0.97   92      0.11     0.99   99    0.19     0.98   58     0.11      0.99   97

GradInput        0.11      1.00   95      0.10     1.00   106    0.19     1.00   59     0.11     1.00   102

IG          0.11       1.00   95      0.10      1.00   106    0.19    1.00   59     0.11     1.00   101

SmoothGrad         0.11     1.00    99     0.10     1.00   110    0.19   1.00   60      0.10     1.00   106

In Figure <ref>, we report token level macro-F1 (ma-F1) and micro-F1 (mi-F1) scores with different $topk\in[5,10,15,20,25]$ with different GNN models. Figure <ref> and <ref> show the correctness for and respectively with different topk.

§ LIMITATIONS

The major limitation of this work is that only focuses on the post-hoc explanation approaches. This work does not cover interpretable-by-design and counter factual based explanation methods.
Though we tried our best to use datasets with varying graph properties and distributions, we believe this benchmark has a large scope to expand to multiple graph datasets with varying graph properties.
We strive to increase the number of more varied datasets in in future.

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = ma-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/ma-f1_GCN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = mi-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/mi-f1_GCN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = ma-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/ma-f1_GAT.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = mi-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/mi-f1_GAT.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = ma-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/ma-f1_GIN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = mi-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/mi-f1_GIN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
xlabel = topk,
ylabel = ma-F1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/ma-f1_APPNP.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
xlabel = topk,
ylabel = mi-f1,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

[color=cyan, mark=triangle, line width = 1pt] table [x index=0, y index=6, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

[color=magenta, mark=diamond, line width = 1pt] table [x index=0, y index=7, col sep = comma] plots/movie_reviews/mi-f1_APPNP.txt;

Plausibility for . GNNExp <ref>  Grad <ref>  GradInput <ref>  SmoothGrad <ref>  IG <ref>  CAM <ref>  GradCAM <ref>

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_cora_GCN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_cora_GCN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_cora_GCN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_cora_GCN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_cora_GCN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_cora_GCN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_cora_GCN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_cora_GCN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_cora_GCN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_cora_GCN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_cora_GAT.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_cora_GAT.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_cora_GAT.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_cora_GAT.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_cora_GAT.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_cora_GAT.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_cora_GAT.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_cora_GAT.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_cora_GAT.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_cora_GAT.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_cora_GIN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_cora_GIN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_cora_GIN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_cora_GIN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_cora_GIN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_cora_GIN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_cora_GIN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_cora_GIN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_cora_GIN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_cora_GIN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
xlabel = topk,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_cora_APPNP.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_cora_APPNP.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_cora_APPNP.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_cora_APPNP.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_cora_APPNP.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
xlabel = topk,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_cora_APPNP.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_cora_APPNP.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_cora_APPNP.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_cora_APPNP.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_cora_APPNP.txt;

Correctness for . GNNExp <ref>  Grad <ref>  GradInput <ref>  SmoothGrad <ref>  IG <ref>

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_citeseer_GCN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_citeseer_GCN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_citeseer_GCN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_citeseer_GCN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_citeseer_GCN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_citeseer_GCN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_citeseer_GCN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_citeseer_GCN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_citeseer_GCN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_citeseer_GCN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_citeseer_GAT.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_citeseer_GAT.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_citeseer_GAT.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_citeseer_GAT.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_citeseer_GAT.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_citeseer_GAT.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_citeseer_GAT.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_citeseer_GAT.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_citeseer_GAT.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_citeseer_GAT.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_citeseer_GIN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_citeseer_GIN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_citeseer_GIN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_citeseer_GIN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_citeseer_GIN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_citeseer_GIN.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_citeseer_GIN.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_citeseer_GIN.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_citeseer_GIN.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_citeseer_GIN.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
xlabel = topk,
ylabel = Precision,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,

[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/precision_citeseer_APPNP.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/precision_citeseer_APPNP.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/precision_citeseer_APPNP.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/precision_citeseer_APPNP.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/precision_citeseer_APPNP.txt;

width  = ,
height = 0.8,
major x tick style = transparent,
grid = major,
grid style = dashed, gray!20,
xlabel = topk,
ylabel = Recall,
symbolic x coords=5, 10, 15, 20, 25,
xtick = data,
enlarge x limits=0.25,
[color=black, mark=*, line width = 1pt] table [x index=0, y index=1, col sep = comma] plots/Correctness/recall_citeseer_APPNP.txt;

[color=red, mark=square*, line width = 1pt] table [x index=0, y index=2, col sep = comma] plots/Correctness/recall_citeseer_APPNP.txt;

[color=green, mark=diamond*, line width = 1pt] table [x index=0, y index=3, col sep = comma] plots/Correctness/recall_citeseer_APPNP.txt;

[color=brown, mark=triangle*, line width = 1pt] table [x index=0, y index=4, col sep = comma] plots/Correctness/recall_citeseer_APPNP.txt;

[color=blue, mark=square, line width = 1pt] table [x index=0, y index=5, col sep = comma] plots/Correctness/recall_citeseer_APPNP.txt;

Correctness for . GNNExp <ref>  Grad <ref>  GradInput <ref>  SmoothGrad <ref>  IG <ref>