# Federated Unlearning via
Class-Discriminative Pruning

Junxiao Wang†, Song Guo†, Xin Xie†, Heng Qi‡
Hong Kong Polytechnic University†, Dalian University of Technology‡
{junxiao.wang, song.guo<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We explore the problem of selectively forgetting categories from trained CNN
classification models in federated learning (FL). Given that the data used for
training cannot be accessed globally in FL, our insights probe deep into the
internal influence of each channel. Through the visualization of feature maps
activated by different channels, we observe that different channels have a
varying contribution to different categories in image classification.

Inspired by this, we propose a method for scrubbing the model cleanly of
information about particular categories. The method does not require
retraining from scratch, nor global access to the data used for training.
Instead, we introduce the concept of Term Frequency Inverse Document Frequency
(TF-IDF) to quantize the class discrimination of channels. Channels with high
TF-IDF scores have more discrimination on the target categories and thus need
to be pruned to unlearn. The channel pruning is followed by a fine-tuning
process to recover the performance of the pruned model.

Evaluated on CIFAR10 dataset, our method accelerates the speed of unlearning
by 8.9× for the ResNet model, and 7.9× for the VGG model under no degradation
in accuracy, compared to retraining from scratch. For CIFAR100 dataset, the
speedups are 9.9× and 8.4×, respectively. We envision this work as a
complementary block for FL towards compliance with legal and ethical criteria.

##  1 Introduction

Edge devices such as mobile phones and IoT devices have become the primary
computing platform today. These devices generate a tremendous amount of
valuable data while providing hidden insights about the human world. In this
case, AI can hugely help evaluate the data and make sense of it. By training
machine learning models that simulate intelligent behaviours to make well-
informed decisions with little or no human intervention. However, analyzing
large amounts of data with complex machine learning algorithms requires
significant computational capabilities. Thus, users have to upload their data
to a cloud server to train a satisfactory model, which is undesirable from a
privacy, security, regulatory or economic perspective.

As an emerging and promising distributed machine learning paradigm, federated
learning (FL) allows multiple devices to jointly train a shared model without
direct access to sensitive training data [1, 2]. In FL, devices train on an
initial model locally with their own data, and upload their model updates to
the federated server to build a shared global model [3, 4]. However, in
dynamic edge environments, data distribution is often not as static. To allow
intelligent applications to adapt to the current environment with limited
resources consumption, we need to eliminate specific categories of data from
the trained models. Moreover, recent privacy legislation such as the European
Union’s _General Data Protection Regulation_ (GDPR) and the former _Right to
be Forgotten_ [5] also give users the right to eliminate specific categories
of data from the trained model. These practical needs call for efficient
techniques that enable FL models to unlearn, or to forget what has been
learned from the categories to be removed (referred to as the target
categories).

Figure 1: Overview and workflow of the proposed unlearning method. Given the
GDPR request to remove a specific category, as first, each online FL device
downloads an unlearning program from the federated server; Following the
program, the local trained CNN model takes the private images as input and
generates a feature map score between each channel and category; These feature
map scores are then communicated to the federated server and are aggregated as
the global feature map scores; The server uses TF-IDF to evaluate the relevant
score between the channels and categories, and builds a pruner to execute
pruning on the most discriminative channels of the target category; Upon
notification of the pruning complete, each online FL device downloads the
pruned model from the federated server, and conducts normal federated training
program with target category-excluded training data to achieve fine-tuning.

A naive way to make such models provably forgotten is to retrain them from
scratch. However, the computational and time overhead associated with fully
retraining models affected by training data erasure can be prohibitively
expensive [6]. Approximate unlearning methods produce ML models that are an
approximation of the fully retrained models. Most of them can be categorized
into one of three groups. Given the category $\bm{i}$ to be unlearned from the
trained model. The _first group_ updates the trained ML model using the data
$\bm{D_{i}}$ of $\bm{i}$ to perform a corrective Newton step; it is defined in
[7]. The _second group_ updates the trained ML model using the remaining data
$\bm{D}$-$\bm{D_{i}}$ to perform a corrective Newton step; it follows [8, 9,
10]. The _third group_ updates the trained ML model by correcting the SGD
steps that led to the trained model; it follows the method defined in [11,
12]. Unfortunately, current centralized unlearning methods are not suited to
handle the systems challenges that arise in federated learning, including lack
of direct data access, communication cost, and non-IID data. Addressing these
challenges is therefore a key contribution of our work.

Through the visualization of feature maps activated by different channels, we
observe that different channels have a varying contribution to different
categories in image classification. Inspired by this, we propose a pruning-
based method for scrubbing the model cleanly of information about particular
categories, as shown in Figure 1. The proposed method introduces the Term
Frequency Inverse Document Frequency (TF-IDF) to quantize the class
discrimination of channels and prune the most relevant channel of the target
category to unlearn its contribution to the model. TF-IDF is a statistical
measure that evaluates how relevant a word is to a document in a set of
documents. Specifically, our unlearning method considers the output of a
channel as a word, the feature map of a category as a document, and uses TF-
IDF to evaluate the relevant scores between the channels and the categories.
The channels with high TF-IDF scores have more class discrimination and thus
need to be pruned to eliminate the contribution of the target class. The
pruning process does not require any iterative training or searching, and thus
is computationally efficient. Finally, a fine-tuning process is conducted to
recover the performance of the pruned model, and to provide a better utility
than the retrained model.

Our approach achieves practical unlearning in FL settings thanks to the
following three designs: We primarily consider the _category-level unlearning_
, rather than sample-level unlearning. Due to the lack of global access to the
raw data, it is impractical for the federated server to evaluate the
contribution of each erased data sample. Instead, we will try to evaluate the
approximation features aggregated from a small set of participants. Next, we
measure the relationship between channels and categories using a _high-level
feature map_ , rather than raw data. The high-level feature maps contain more
information about the image category. The non-IID raw training data in the
same category should also share similar high-level features, thus the class
discrimination of the feature map can be learned through collaboration between
a small set of participants. Thirdly, to achieve channel pruning in the
federated server, each participant client only needs to upload its _local
representation score_ that describes the correlation between channels and
categories. Such a method avoids raw data leakage and massive communication
between client and server, thereby better adapting to the distributed and
large scale settings of the FL.

To the best of our knowledge, this is the first time that CNN channel pruning
has been used to guide the machine unlearning process. We also envision our
work as a complementary block for FL towards compliance with legal and ethical
criteria. Extensive experiments on several representative image classification
tasks demonstrate the superiority of our unlearning method. Compared to
retraining from scratch, the time required to achieve unlearning is
significantly decreased by our method with no degradation in accuracy: on the
CIFAR10 dataset, we observe that the speedup is 8.9× for the ResNet model, and
7.9× for the VGG model; on the CIFAR100 dataset, the speedups are 9.9× and
8.4×, respectively.

Figure 2: Visualization of images from different categories (first row) along
with the feature map of the 52-th and 127-th channels (second and third row,
respectively) in the 13-th convolutional layer of VGG16 trained on ImageNet.
As seen, with different images as input, head and textual information are
always activated by these two channels respectively.

##  2 Preliminaries

For sake of completeness, we now briefly review the federated learning (FL),
the correspondent problem of target category unlearning, existing studies on
unlearning in the context of ML (a.k.a. machine unlearning) and their
limitations in the FL settings, due to the inherent distinction in the way how
FL and ML learn from data.

###  2.1 Federated Learning

The term _federated learning_ was introduced by [3]. Federated learning (FL)
is a machine learning setting where a set of $\bm{m}$ clients (_e.g._ , mobile
devices) collaboratively train a model under the orchestration of a federated
server (_e.g._ , service provider), while the training data of clients is
stored locally and not exchanged [2]. The federated server orchestrates the
collaborative training process, by repeating the following steps until
training is converged:

1) Client selection. Given the unstable client availability, for the round
$\bm{t}$ of federated learning, the federated server selects $\bm{n}$ clients
meeting eligibility requirements to participate in the learning.

2) Local training. Upon notification of being selected at the round $\bm{r}$,
each selected client downloads the current parameters $\bm{w}$ of global model
and a training program from the federated server. Each selected client locally
computes an update to the global model on its local training data by executing
the training program.

3) Global aggregation. Upon having received local updates from $\bm{n}$
clients, the federated server aggregates these updates and updates its global
model, and initiates next round of learning.

###  2.2 General Centralized Unlearning

Recently introduced legislation, e.g., the General Data Protection Regulation
(GDPR) in the European Union [13], introduces a right for individuals to have
their data erased [14]. To remove the $\bm{i}$-th category from a model
trained with an integrated dataset $\bm{D}$, a vanilla way would be to retrain
the model from scratch on the remaining data points $\bm{D}$-$\bm{D_{i}}$.
However, the time, computation, and energy costs of model retraining can be
quite costly. To address this problem, machine unlearning has been recently
studied, which typically produces an approximation of the fully retrained
model at low cost. We coarsely characterize these methods into three groups:

1) Fisher unlearning method. The _first group_ uses the Fisher information
[15] of the remaining data $\bm{D}$-$\bm{D_{i}}$ and injects optimal noise in
order to unlearn the target category [8, 9, 10]. The unlearning algorithm
takes as input the currently trained model, the subset of the training data
$\bm{D_{i}}\in\bm{D}$, and outputs an updated model given by

$\displaystyle\bm{w}^{\prime}=\bm{w}-\bm{F}^{-1}\bm{\Delta}+\bm{\sigma}\bm{F}^{-\frac{1}{4}}\bm{b}$
(1)

where $\bm{\Delta}$ is the gradient of the loss function, computed on the
remaining training data $\bm{D}$-$\bm{D_{i}}$. $\bm{F}$ is the Fisher matrix,
also computed on the remaining training data $\bm{D}$-$\bm{D_{i}}$. The first
term corresponds to the corrective Newton step. The second term corresponds to
noise injection, and adds standard normal noise $\bm{b}$ to the updated model
in the direction of the Fisher matrix.

2) Influence unlearning method. The _second group_ [7, 16] is based on the
Influence theory [17]. The unlearning algorithm is performed by computing the
influence of the data $\bm{D_{i}}$ on the trained ML model then updating the
model to remove that influence as

$\displaystyle\bm{w}^{\prime}=\bm{w}+\bm{I}^{-1}\bm{\nabla}$ (2)

where $\bm{\nabla}$ is the gradient of the loss function, computed on the data
$\bm{D_{i}}$. $\bm{I}$ is the second derivative of the loss function, computed
on the remaining training data $\bm{D}$-$\bm{D_{i}}$. The second term is known
as the influence function of the data $\bm{D_{i}}$ on the trained model. The
loss function incorporates noise injection to achieve unlearning.

3) Gradient unlearning method. The _third group_ focuses on approximating the
SGD steps as if full retraining was performed [11, 12, 18, 19]. To aid in a
history of accurate computations to produce an effective approximation, the
unlearning method periodically computes the exact gradient after some
iterations. The exact gradient is computed on the remaining training data
$\bm{D}$-$\bm{D_{i}}$.

###  2.3 Federated Unlearning

We primarily consider a scenario where a service provider is required by most
users to delete the target data category from their model to protect the
user’s privacy and avoid legal risks. We frame the problem of target category
unlearning in FL as follows. Suppose train a shared model on data points
$\bm{D}$ distributed across $\bm{m}$ devices, where $\bm{m}\gg\bm{n}$ (the
number of participant devices). Let $\bm{U}$ $=$
$\\{u_{1},\cdots,u_{i},\cdots,u_{|\bm{U}|}\\}$ be the classification space
that data space can mapping to, $\bm{w}$ be the model in hypothesis space
after pre-training. To unlearn the $\bm{i}$-th category from the trained
model, we need to update the model to make it operate as if the training data
$\bm{D_{i}}$ in $\bm{i}$-th category had never been observed.

######  Definition 2.1 (Federated Unlearning).

We define the algorithm $\mathcal{L}$: $\bm{U}$ $\to$ $\bm{w}$ be a learning
process which maps classification space $\bm{U}$ into the hypothesis space of
model $\bm{w}$. $u^{\prime}$ be the category that most users require to
delete. We define an unlearning process $\mathcal{L}^{-}$:
$\mathcal{L}(\bm{U})$ $\otimes$ $\bm{U}$ $\otimes$ $u^{\prime}$ $\to$
$w^{\prime}$, which takes an input classification space $\bm{U}$, a learned
model $\mathcal{L}(\bm{U})$ and the category $u^{\prime}$ that required to be
forgotten. Thus the objective of federated unlearning can be described as

$\Phi[\mathcal{L}(\bm{U}\backslash
u^{\prime})]=\Phi[\mathcal{L}^{-}(\mathcal{L}(\bm{U}),\bm{U},u^{\prime})],$
(3)

where $\bm{U}\backslash u^{\prime}$ defines the classification space without
target category $u^{\prime}$. $\Phi[\cdot]$ indicates the probability
distribution of the output.

###  2.4 Challenges

The existing centralized unlearning methods always assume that they have
access to the relevant global training data (e.g., $\bm{D}$ and $\bm{D_{i}}$).
However, such assumptions are broken in the FL setting, since the number of
participant devices $\bm{n}$ is usually much smaller than the total devices
$\bm{m}$ [3], Moreover, the non-IID training data across different
participants [1] makes it even more challenging to find the appropriate
updating direction for the global models. Thus, with incomplete and severely
biased local training data, the centralized unlearning method can only offer
inaccurate model approximations.

##  3 Pathway to Channel Pruning

While the data used for training are impossible to access globally in FL,
insights may still be gleaned by probing deep into the internal influence of
each channel. Imagine such a possibility: if we can quantify the category
information learned by each channel without globally accessing the data, then
we have a chance to forget special categories by pruning the channels whose
class discrimination is the most. Assuming this idea is feasible, the
federated unlearning will usher a novel form of implementation, on which the
existing practical limitations are well adapted to. To study the pathway to
the channel pruning based solution, we start by investigating the class
discrimination of channels in CNNs.

###  3.1 Class Discrimination of Channels

As exploited in [20], the feature map of each channel has the locality that a
particular area in one feature map is activated. Inspired by this, we
visualize the feature map generated by VGG16 trained on ImageNet to explore
the local information in the internal 13-th convolutional layer. As can be
seen from Figure 2, the 52-th channel in the 13-th convolutional layer always
generates a feature map that highlights the head information while the 127-th
channel always highlights the textual information. Even though there is no
explicitly labeled head or text, the class-discriminative channels
automatically learn to extract partial information to make better decisions,
which exactly meets human intuition when classifying an image. That is, head
information extracted by the 52-th channel helps the model to identify
animals, and textual information extracted by the 127-th channel contributes
to classifying categories with texts such as airliners and digital watches. On
the other hand, some local features may not be beneficial to identifying all
categories. For example, the 127-th channel always chooses to deactivate most
of the pixels when processing images with no textual semantics like cats and
dogs (see the first and fourth columns).

Such local representation on the internal layer shows that different channels
have varying contributions to different categories in image classification,
which motivates us to rethink the importance of class-discriminative channels
on federated unlearning. By evaluating the class-discriminative information,
we aim to find channels that have the greatest separability with respect to
the target categories, which fits seamlessly with the goal of class
forgetting.

###  3.2 TD-IDF based Channel Scoring

We introduce the Term Frequency Inverse Document Frequency (TF-IDF) to
quantize the class discrimination of channels. TF-IDF [21] is a statistical
measure that evaluates how relevant a word is to a document in a set of
documents. It is used in many scenarios of document search and information
retrieval, most importantly in automated text analysis, and is very useful for
scoring words in machine learning algorithms for Natural Language Processing
(NLP) [22]. TF-IDF defines a quantification of the relevance between a word
and a document: the relevance increases proportionally to the number of times
the word appears in the document, but is offset by the number of documents
that contain the word. Therefore, words that are common in every document,
rank low even though they may appear many times, since they mean few to that
document in particular. Instead, if a word appears many times in a document,
while not appearing many times in others, it probably means that the word is
very relevant.

Concretely, TF-IDF for a word in a document is quantized by multiplying two
different metrics: (1) The term frequency (TF) of the word in the document;
(2) The inverse document frequency (IDF) of the word across a collection of
documents. For word $\bm{t}$ in document $\bm{e}$ from the set $\bm{E}$ of all
documents, TF is usually calculated with a count of instances of $\bm{t}$
appearing in $\bm{e}$. Then, the frequency is divided by the length of
$\bm{e}$. The IDF can be calculated by taking the total volume of $\bm{E}$,
dividing it by the number of documents that contain $\bm{t}$, and calculating
the logarithm. Multiplying the TF and IDF results in the TF-IDF score of
$\bm{t}$ in $\bm{e}$.

Determining how discriminative a channel is to a category can use TF-IDF in a
similar way. Specifically, our unlearning method regards the output of a
channel as a word and the feature map of a category as a document. Hence, this
TF-IDF variant can evaluate the relevant score between the channels and the
categories. Channels with high TF-IDF scores have more class discrimination
and thus need to be pruned.

###  3.3 Class-Discriminative Channel Pruning

Channel pruning targets at removing the specific channels in each layer and
shrinking a CNN model into thinner one [23]. As a type of structured
simplification, the advantage of channel-level pruning is removing the entire
channels directly, which can be well supported by general-purpose hardware and
high-efficiency Basic Linear Algebra Subprograms (BLAS) libraries [24].

Moreover, existing methods of class-discriminative channel pruning have not
well studied the computational effectiveness. For example, the recent work
[25] uses a closed-form function and the work [26] inserts a cross-entropy
type loss. They both require iterative optimization steps or heavy matrix
operation like matrix inversion to obtain the discrimination score, which can
hinder these methods to be applied to large-scale FL settings. Compared to
existing methods of class-discriminative channel pruning, our TF-IDF guided
channel pruning does not require any iterative training or searching, which is
much more lightweight in terms of computational overhead.

##  4 Federated Unlearning Framework

Our framework is applicable to FL models that pre-trained in a wide range of
ways, unlike existing unlearning methods that require the training to be
conducted in a special manner. Given the request to remove a specific category
from a pre-trained model stored in both local and global, the workflow
depicted in Figure 1 is composed of two key components running respectively in
the FL client and the federated server: participant clients transform their
private images to generate local representations which are communicated to the
server, in order to build a pruner based on the TF-IDF score between the
target category and channels.

###  4.1 Local Processing in FL Clients

At each FL client, local_proc in the unlearning program is conducted with
local private images, in order to generate a local representation between
channels and classes.

Considering the local trained model as an $L$-layer CNN model, whose kernel
weights can be represented as $\bm{w}$ =
$\left\\{\bm{w}_{1},\bm{w}_{2},\dots,\bm{w}_{L}\right\\}$. The kernel in the
$l$-th layer is denoted as $\bm{w}_{l}\in\mathcal{R}^{C_{out}^{l}\times
C_{in}^{l}\times K_{l}\times K_{l}}$, where $C_{out}^{l}$, $C_{in}^{l}$,
$K_{l}$ denote the numbers of output channels and input channels, and the
kernel size, respectively. Let $\bm{X}_{l}\in\mathcal{R}^{N\times
C_{in}^{l}\times H_{l}\times W_{l}}$ be the input of the $l$-th layer where
$N$ is the batch size of input images, and $H_{l}$, $W_{l}$ are the height and
width of the input respectively. Therefore, the local set of private images
can be denoted as $\bm{X}_{1}$. The output feature map of the $l$-th layer is
calculated as

$\displaystyle\bm{O}_{l}=\bm{X}_{l}\circledast\bm{w}_{l}$ (4)

where $\circledast$ is the convolutional operation,
$\bm{O}_{l}\in\mathcal{R}^{N\times C_{in}^{l+1}\times H_{l+1}\times W_{l+1}}$,
and $C_{in}^{l+1}$=$C_{out}^{l}$. As a first step, the client will record the
feature map generated by local model in each layer. Then, we apply ReLU [27]
followed by an average pooling operation over the feature map. For the
channels at $l$-th layer, the activation of output feature map is denoted by
$\bm{A}_{l}$, can be calculated by

$\displaystyle\bm{A}_{l}=\text{AvgPooling}(\text{ReLU}(\bm{O}_{l}))$ (5)

where $\bm{O}_{l}$ is the output feature map generated by $l$-th layer, and
the feature map of each channel is resized from $H_{l+1}$$\times$$W_{l+1}$ to
1$\times$1\. Therefore, $\bm{A}_{l}\in\mathcal{R}^{N\times C_{out}^{l}}$ is a
local representation between images and channels. Finally, the representation
$\bm{A}_{l}$ will be averaged across classes and re-stacking each channel, as
$\bm{A}^{\prime}_{l}\in\mathcal{R}^{|\bm{U}|\times C_{out}^{l}}$, where
$|\bm{U}|$ is the number of classes. Upon the unlearning program is conducted
completely in the client, the representation $\bm{A}^{\prime}_{l}$ is uploaded
to the federated server for further processing.

###  4.2 Processing in the Federated Server

At the federated server, server_proc is conducted with the local uploaded
representations between classes and channels. Upon having received local
representations from all participant clients, the server first aggregates
these local representations in average to generate a global representation
between classes and channels.

Given a global representation $\bm{A}^{*}\in\mathcal{R}^{|\bm{U}|\times
C_{out}^{l}}$ in $l$-th layer, the term frequency (TF) can be calculated by

$\displaystyle\text{TF}_{l}^{u^{\prime}}=\frac{{\bm{A}^{*}}_{l}^{u^{\prime}}}{{\bm{\sum}}_{j=0}^{C_{out}^{l}}{\bm{A}^{*}}_{l}^{u^{\prime},j}}$
(6)

where TF${}_{l}^{u^{\prime}}\in\mathcal{R}^{C_{out}^{l}}$ represents the
contribution of each word (channel) to a specific document (class
$u^{\prime}$). Note that, some channels that have high scores in TF may also
have a contribution to other categories outside the target category. In order
to obtain the most discriminative channels of the target category, the inverse
document frequency (IDF) in $l$-th layer can be calculated by

$\displaystyle\text{IDF}_{l}^{j}=\log\frac{1+|\bm{U}|}{1+\left|\big{\\{}u_{i}\in\bm{U}:{\bm{A}^{*}}_{l}^{u_{i},j}\geq\text{Avg}\big{(}{\bm{A}^{*}}_{l}^{u_{i}}\big{)}\big{\\}}\right|}$
(7)

where IDF${}_{l}^{j}\in\mathcal{R}^{1}$ represents how common or rare
contribution of a specific word (channel $j$) is in the entire document set
(classes). The closer it is to 0, the more common contribution of a channel
is. Hence, if contribution of a channel is very common and appears in many
classes, its IDF will approach 0. Otherwise, it will approach 1. Multiplying
TF${}_{l}^{u^{\prime}}$ and IDFl results in the TF-IDF${}_{l}^{u^{\prime}}$
score of channels for the target class in $l$-th layer, as

$\displaystyle\text{TF-
IDF}_{l}^{u^{\prime}}=\text{TF}_{l}^{u^{\prime}}*\\{\text{IDF}_{l}^{1},\cdots,\text{IDF}_{l}^{C_{out}^{l}}\\}$
(8)

where TF-IDF${}_{l}^{u^{\prime}}\in\mathcal{R}^{C_{out}^{l}}$ represents the
relevant score between the channels and categories. The higher the score, the
more relevant that channel is in that particular class.

Based on the calculated TF-IDF score between channels and categories, the
server then builds a pruner to execute pruning on the most discriminative
channels of the target category. Pruning is a common technique to compress
neural network models. It prunes specific weights from models, thereby their
values are zeroed and we make sure they do not take part in the back-
propagation process. We adopt a one-shot pruning to prune the channels whose
TF-IDF score is beyond a pre-defined percentage $R$. Since some channels in
$l$+1-th layer are removed, the convolution filters that generate these
channels in $l$-th layer will be removed accordingly. The pruned kernel
$\hat{w}_{l}\in\mathcal{R}^{\hat{C}_{out}^{l}\times\hat{C}_{in}^{l}\times
K_{l}\times K_{l}}$ is obtained under the constraints of
$\hat{C}_{in}^{l}$$\leqslant$$C_{in}^{l}$ and
$\hat{C}_{out}^{l}$$\leqslant$$C_{out}^{l}$. We have the Equation 4 in the
pruned model $\hat{\bm{w}}_{l}$ as

$\displaystyle\hat{\bm{O}}_{l}=\hat{\bm{X}}_{l}\circledast\hat{\bm{w}}_{l}$
(9)

In the case of multi-class removal, the pruning process is executed multiple
times, removing one class each time. Finally, upon the pruning complete, the
federated server will notify each participant FL client to download the pruned
model from it. A fine-tuning process is then conducted to achieve the after-
pruned model with a target accuracy.

###  4.3 Fine-tuning Processing

After pruning the most discriminative channels of the target category, the
accuracy degradation should be compensated by retraining the pruned model. Our
fine-tuning process is the same as the normal training procedure of federated
learning, without introducing additional regularization. To reduce the
unlearning time, we apply the _prune once and retrain_ strategy: prune
channels of multiple layers at once and retrain them until the target accuracy
is restored. We find for our unlearning method, the _prune once and retrain_
strategy can be used to prune away significant portions of the model
corresponding to the target class, and any loss in accuracy can be regained by
retraining for a short period of time (significantly less than retraining from
scratch).

###  4.4 Discussion

Feasibility and limitations of applying our method to centralized learning.
Although specifically designed for FL, the class-discriminative channel
pruning method can definitely be applied in centralized learning, since the
representative feature distribution between channels and categories can easily
be obtained with global access to the data. However, the advantages of class-
discriminative pruning are significantly diminished in centralized scenarios.
With full access to the raw data, the central server can accurately evaluate
the individual contributions of each data point. Additionally, data privacy
protection and communication overhead optimization is no longer required. This
will lead to a far greater diversity of class-unlearning designs, allowing the
central server to obtain a better approximation to the target unlearned model
in terms of accuracy or overhead.

Challenges on sample-level unlearning. This study focuses on class-level
unlearning problems in FL scenarios, while the sample-level unlearning problem
is a more challenging issue that remains to be studied in future work. The
sample-level unlearning task requires the model to remove specific data
samples from the model while still maintaining the accuracy of the model. To
achieve such an ambitious goal, it requires a more elaborate design since the
data point contributions to the model are difficult to evaluate without access
to the raw data, and we are working on a solution to the problem in the near
future.

##  5 Experimental Evaluation

In this section, we empirically evaluate the performance of the proposed
unlearning method in the FL settings. We report experiments on CIFAR10 and
CIFAR100 datasets [28] with ResNet and [29] VGG [30] models. We choose these
datasets and models because they are sufficiently standard and general in
image classification tasks, thus proving that our method is general instead of
only valid for specific image tasks, such as facial image classification.
Specifically, CIFAR10 dataset consists of 60000 32x32 colour images in 10
classes, with 6000 images per class. CIFAR100 dataset is just like the
CIFAR10, except it has 100 classes containing 600 images each.

###  5.1 FL Settings

Note that, the characteristic of FL is that only a portion of total clients
are sampled as participant, and the participant clients have Non-IID local
training data [3, 31, 1]. Therefore, the participant data are incomplete and
biased. We assume the participant data have a distribution different from the
overall training data distribution of the learning task. In particular, we
assume the participant data are biased towards certain classes.

Figure 3: Federated unlearning the last category of CIFAR10/CIFAR100 from pre-trained models. | CIFAR10 | CIFAR100
---|---|---
| Rounds of training | Test accuracy on U/R-set | Rounds of training | Test accuracy on U/R-set
Bias probability | 0.10 | 0.45 | 1.00 | 0.10 | 0.45 | 1.00 | 0.01 | 0.35 | 1.00 | 0.01 | 0.35 | 1.00
Our method | 113 | 135 | 181 | 00.00/80.13% | 00.00/74.45% | 00.00/66.87% | 110 | 163 | 235 | 00.00/50.34% | 00.00/46.99% | 00.00/39.45%
Fisher method | 610 | 750 | 1110 | 22.47/80.00% | 28.54/73.79% | 19.10/66.04% | 700 | 820 | 1190 | 15.33/49.86% | 14.71/45.30% | 17.09/38.32%
Table 1: Impact of different bias probabilities (unlearning the last category
from ResNet20 model).

We follow Fang et al. [32] to distribute the training data points in a dataset
among clients. Assuming there are $M$ classes in a dataset. We randomly split
the clients into $M$ groups. A training data point with label $m$ is assigned
to group $m$ with probability $q$¿0 and to any other group with probability
(1-$q$)/($M$-1). Within the same group, data are uniformly distributed to each
client. $q$, as a bias probability, controls the distribution difference of
the local training data. If $q$=1/$M$, the local training data are independent
and identically distributed (IID), otherwise the local training data are Non-
IID. A larger $q$ denotes a higher level of Non-IID among the local training
data. We will set $q$¿1/$M$ to simulate the Non-IID settings. Moreover, we
assume the training data are distributed among 100 clients, and the number of
clients participating in each round is 25. The bias probability $q$ is set
from 0.1 to 1.0 for CIFAR10, and is set from 0.01 to 1.0 for CIFAR100. On both
CIFAR10 and CIFAR100, we compare our unlearning method against two baselines.

Baselines. (1) retraining the entire data (without the points to be unlearned)
from scratch; (2) forgetting the target category based on the Fisher
unlearning method [8, 9, 10]. As introduced in Section 2.2, this method uses
the Fisher information of the participant data and injects optimal noise in
order to achieve unlearning.

###  5.2 Experimental Results

As shown in Figure 3, we unlearn the last category of CIFAR10 and CIFAR100
from different pre-trained models, and compare the speedup with fully
retraining. The learning method is FedSGD, learning rate is set 0.1, and bias
probability is 0.1(0.01). For full retrain, the x-axis represents the number
of training rounds from scratch. For our method, the x-axis represents the
number of training rounds in fine-tuning process. The y-axis represents the
accuracy of the unlearned models on the testset (without the category to be
forgotten). We show the impact of different bias probabilities in Table 1. The
learning method is FedAVG, local epoch is 5, learning rate is 0.1. The model
accuracy is calculated respectively on the testset only containing the target
category (i.e., unlearned testset, or U-set) and the testset containing the
rest categories (i.e., rest testset, or R-set). We set the hyper-parameter
$R$=0.05 for CIFAR10 ResNet; 0.1 for CIFAR10 VGG; 0.1 for CIFAR100 ResNet;
0.15 for CIFAR100 VGG.

Unlearning speedup. Obviously, the speedups produced by our method compared to
fully retraining is very significant: on CIFAR10 dataset, we observe the
speedup is 8.9$\sim$11.0× for the ResNet model, and 7.9$\sim$9.1× for the VGG
model; on CIFAR100 dataset, the speedups are 9.9× and 8.4×, respectively. On
the one hand, as the depth of the network increases, the effect of
acceleration will get better. On the other hand, as the total number of
categories increases, the effect of unlearning acceleration will also become
better. These results show that our method can be well adapted to real large-
scale scenarios with a large number of categories and a deep network.

Compared to the baseline method using the Fisher information, our method
performs significantly better because our method is insensitive to the
distribution of training data, thereby can achieve a accurate unlearning even
with severely biased participant data. As another way to represent category
information in the unlearning task, the Fisher method heavily depends on
accurate and global access to all training data, and depicts a linear
corrective Newton step to achieve unlearning by injecting optimal noise. Once
the training data cannot be globally accessed (i.e., with higher bias in our
settings), the corrective step cannot be guided in precise. Therefore, a large
value of noise will indiscriminately remove information from the model which
in turn drastically reduces the effectiveness of further fine-tuned training.
Moreover, even the case of no bias, deep and nonlinear neural networks are
great challenges for the linear corrective method.

Information erasure. As shown in Table 2, we compare the model accuracy on
R/U-set respectively for our method and full retrain, which serves as a
befitting manner to measure the quality of unlearning (i.e., assess how much
information about the target category is still contained in the unlearned
model). It can be observed that, first of all, our method has no loss of
precision on R-set. On this basis, the accuracy of our method on U-set is the
same as full retrain, and both are 0%. These results prove that our method
achieves resemble effect of information erasure on the pre-trained model as
the full retrain does, without the expense of accuracy.

It is worth noting that, as the total number of categories increases, the
accuracy of the after-pruned model on R-set is get lower (e.g., closing to 0%
for CIFAR100). However, the speedup of unlearning is still kept as can be seen
from Figure 3. This proves from the side that our method can indeed cut the
channels most relevant to the target class from the trained model. In the
fine-tuning process, less interference (on forward and backward propagation)
comes from the channels related to the target category, thereby can
effectively accelerate the recovery of accuracy on R-set.

Multi-class removal. As shown in Table 3, the performance in the case of
multi-class removal is still advanced (with the speedups comparable to the
case of single-class removal). After pruning for the second class, the
accuracy of the after-pruned model on U-set gets lower (i.e., to 0%), which
shows an ideal information erasure.

| CIFAR10 | CIFAR100
---|---|---
| Raw model | After-pruned | Fine-tuned | Re-trained | Raw model | After-pruned | Fine-tuned | Re-trained
Accuracy | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set
ResNet20 | 91.50% | 83.33% | 00.00% | 20.79% | 00.00% | 86.40% | 00.00% | 86.33% | 54.00% | 50.01% | 00.00% | 05.38% | 00.00% | 57.17% | 00.00% | 57.11%
ResNet32 | 94.20% | 83.71% | 00.00% | 11.58% | 00.00% | 86.40% | 00.00% | 86.14% | 52.00% | 51.67% | 00.00% | 01.06% | 00.00% | 59.62% | 00.00% | 59.42%
ResNet44 | 89.90% | 83.94% | 00.00% | 22.19% | 00.00% | 86.48% | 00.00% | 86.34% | 48.00% | 53.25% | 00.00% | 01.22% | 00.00% | 60.41% | 00.00% | 59.85%
ResNet56 | 93.10% | 84.02% | 00.00% | 11.11% | 00.00% | 86.42% | 00.00% | 86.38% | 44.00% | 52.91% | 00.00% | 01.32% | 00.00% | 59.60% | 00.00% | 59.28%
VGG11 | 88.20% | 84.72% | 00.00% | 18.29% | 00.00% | 87.24% | 00.00% | 87.13% | 50.00% | 53.55% | 00.00% | 01.28% | 00.00% | 59.25% | 00.00% | 58.20%
VGG13 | 91.50% | 84.19% | 00.00% | 15.17% | 00.00% | 89.18% | 00.00% | 89.09% | 60.00% | 51.88% | 00.00% | 03.82% | 00.00% | 59.65% | 00.00% | 59.27%
VGG16 | 91.60% | 84.38% | 00.00% | 17.79% | 00.00% | 89.20% | 00.00% | 89.30% | 44.00% | 50.34% | 00.00% | 01.46% | 00.00% | 59.72% | 00.00% | 59.57%
VGG19 | 88.80% | 83.53% | 00.00% | 11.11% | 00.00% | 89.72% | 00.00% | 89.62% | 52.00% | 52.15% | 00.00% | 01.02% | 00.00% | 58.78% | 00.00% | 58.96%
Table 2: Information forgotten effect compared to fully retraining (unlearning the last category). | Raw model | First class pruned | Second class pruned | Fine-tuned | Re-trained
---|---|---|---|---|---
Model Accuracy | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | Speedup
Delete class 0+9 from [0-9] | 88.70% | 83.01% | 02.10% | 24.57% | 00.00% | 32.41% | 00.00% | 87.12% | 00.00% | 87.29% | ×8.71
Delete class 0+5 from [0-9] | 81.90% | 84.71% | 00.25% | 25.04% | 00.00% | 26.74% | 00.00% | 89.62% | 00.00% | 89.75% | ×10.62
Delete class 5+9 from [0-9] | 84.70% | 84.01% | 02.10% | 31.74% | 00.00% | 37.71% | 00.00% | 88.37% | 00.00% | 88.21% | ×8.92
Table 3: Delete two categories of CIFAR10 simultaneously from ResNet20 model.

| ResNet20 | ResNet56 | VGG11 | VGG19
---|---|---|---|---
Delete class 0 from [0-9] | 00.35% | 00.47% | 00.39% | 00.40%
Delete class 9 from [0-9] | 00.46% | 00.50% | 00.28% | 00.37%
Delete class 0+9 from [0-9] | 00.51% | 00.63% | 00.49% | 00.56%
Table 4: Difference of MIA success rate between fully retraining and our
method under CIFAR10.

| ResNet20 | ResNet56 | VGG19
---|---|---|---
Delete class 0 from [0-9] | 00.33%/3.01e-6 | 00.46%/4.50e-6 | 00.35%/3.65e-6
Delete class 9 from [0-9] | 00.27%/2.58e-6 | 00.43%/4.84e-6 | 00.32%/3.87e-6
Delete class 0+9 from [0-9] | 00.36%/2.63e-6 | 00.59%/5.27e-6 | 00.41%/3.55e-6
Table 5: Distance of classification accuracy distribution between fully
retraining and our method under CIFAR10.

Membership inference attack. We launch a membership inference attack (MIA)
introduced in literature [33]. Given a sample of target category and the model
after unlearning, the attack aims to infer whether the sample was used to
train the original model. In other words, the attack aims to know that the
target sample is in the training dataset of the original model. While the goal
of unlearning is to protect privacy information about the target sample, a
successful attack considered here can show unlearning instead leaks the target
sample’s privacy. We measure the success rate of attacks to evaluate how much
information about the target category is still contained in the model after
unlearning. Seen from results under CIFAR10 (as referred to Table 5), no
significant difference is found in the attack success rate between fully
retraining and our method (with average deviation less than 0.63%).

Classification accuracy distribution. We investigate how are the
classification accuracies distributed among all classes after unlearning and
whether it is an ideal distribution approximate that of retraining from
scratch. As shown in Table 5, the class-wise accuracy difference is only 0.59%
at most. Moreover, we use the KL divergence to evaluate the distance between
our distribution and the ideal distribution. Seen from results under CIFAR10,
the KL divergence between fully retraining and our method is very close to 0
(less than 5.27e-6).

##  6 Related Work

Machine unlearning as the field most related to this paper, its notion was
first proposed by Cao et al. [34] in the context of statistical query
learning. Different from the class-level unlearning we defined, the goal of
machine unlearning is more strict as it forgets a particular subset of samples
within a class. To unlearn a data sample, Cao et al. [34] transforms the
learning algorithms into a summation form that follows statistical query
learning, breaking down the dependencies of training data. However, the method
in [34] is not applicable to learning algorithms that cannot be transformed
into summation form, such as neural networks.

Thereafter, machine unlearning for different ML models have been explored.
Ginart et al. [5] studied the problem of data removal for variants of k-means
clustering, but cannot be applied to supervised learning. The works [8, 9, 10,
7, 16, 11, 12, 18, 19] introduced in Section 2.2 focus on supervised linear
classification models, such as logistic regression, but not specializing for
more complex models, such as neural networks. There are other approximate
unlearning methods that fit for specific ML models, such as Brophy et al. [35]
for random forest models, or Nguyen et al. [36] for bayesian learning.
Bourtoule et al. [6] proposed a more general algorithm named SISA. The main
idea of SISA is to split the training data into several disjoint shards, with
each shard training one sub-model. To remove a specific sample, the algorithm
only needs to retrain the sub-model that contains this sample. Nevertheless,
existing machine unlearning works focus on ML models in traditional
centralized settings, i.e., the training data assumed to be globally accessed,
which is ineligible for unlearning in the FL settings. Liu et al. [37] studied
client-level unlearning in FL, i.e., to eliminate the influence of data of a
FL device on the global model, and proposed an algorithm named FedEraser. The
main idea of FedEraser is to unlearn model by leveraging the historical
parameter updates of clients that have been retained at the federated server
during the training process. However, for convolutional architectures, [37]
show that unlearning is possible for only shallow networks, such as for a
2-layer CNN followed by 2 fully-connected (FC) layers. How much the amount of
stored information in the federated server also leads to a trade-off between
the accuracy and scalability.

##  7 Conclusion

We propose a definition of selectively forgetting categories for the trained
CNNs classification model in the federated learning (FL) settings. Based on
this definition, we propose a scrubbing procedure that cleans information
about particular categories from the trained model, without the need to
globally access the training data, nor to retrain from scratch. Compared to
the gold standard model trained from scratch without any knowledge of the
target category data, our method demonstrates its superiority in terms of
unlearning speed under no degradation in accuracy. We envision this work as an
early step towards federated unlearning, which contributes as a complementary
block for compliance with legal and ethical criteria. In future work, we will
investigate a more challenging fine-grained unlearning task in FL settings,
where not only does an entire class need to be forgotten, but rather a
particular subset of samples within a class needs to be removed, while still
maintaining output knowledge of that class.

## References

  * [1] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra.  Federated learning with non-iid data.  arXiv preprint arXiv:1806.00582, 2018.
  * [2] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.  Advances and open problems in federated learning.  arXiv preprint arXiv:1912.04977, 2019.
  * [3] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
  * [4] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong.  Federated machine learning: Concept and applications.  ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019.
  * [5] Antonio Ginart, Melody Y Guan, Gregory Valiant, and James Zou.  Making ai forget you: Data deletion in machine learning.  In Proceedings of Conference on Neural Information Processing Systems (NeurIPS), 2019.
  * [6] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot.  Machine unlearning.  In Proceedings of IEEE Symposium on Security and Privacy (SP), 2021\.
  * [7] Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten.  Certified data removal from machine learning models.  In Proceedings of International Conference on Machine Learning (ICML), 2020.
  * [8] Aditya Golatkar, Alessandro Achille, and Stefano Soatto.  Eternal sunshine of the spotless net: Selective forgetting in deep networks.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [9] Aditya Golatkar, Alessandro Achille, and Stefano Soatto.  Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations.  In Proceedings of European Conference on Computer Vision (ECCV), 2020.
  * [10] Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto.  Mixed-privacy forgetting in deep networks.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  * [11] Yinjun Wu, Edgar Dobriban, and Susan Davidson.  Deltagrad: Rapid retraining of machine learning models.  In Proceedings of International Conference on Machine Learning (ICML), 2020.
  * [12] Yinjun Wu, Val Tannen, and Susan B Davidson.  Priu: A provenance-based approach for incrementally updating regression models.  In Proceedings of ACM SIGMOD International Conference on Management of Data, 2020.
  * [13] Alessandro Mantelero.  The eu proposal for a general data protection regulation and the roots of the ‘right to be forgotten’.  Computer Law & Security Review, 29(3):229–235, 2013.
  * [14] Supreeth Shastri, Melissa Wasserman, and Vijay Chidambaram.  The seven sins of personal-data processing systems under $\\{$GDPR$\\}$.  In Proceedings of USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), 2019.
  * [15] James Martens.  New insights and perspectives on the natural gradient method.  Journal of Machine Learning Research, 21(146):1–76, 2020.
  * [16] Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou.  Approximate data deletion from machine learning models.  In Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), 2021.
  * [17] Pang Wei Koh and Percy Liang.  Understanding black-box predictions via influence functions.  In Proceedings of International Conference on Machine Learning (ICML), 2017.
  * [18] Laura Graves, Vineel Nagisetty, and Vijay Ganesh.  Amnesiac machine learning.  In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
  * [19] Seth Neel, Aaron Roth, and Saeed Sharifi-Malvajerdi.  Descent-to-delete: Gradient-based methods for machine unlearning.  In Proceedings of International Conference on Algorithmic Learning Theory (ALT), 2021.
  * [20] Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson.  Understanding neural networks through deep visualization.  arXiv preprint arXiv:1506.06579, 2015.
  * [21] Jiaul H Paik.  A novel tf-idf weighting scheme for effective ranking.  In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval, 2013.
  * [22] Inbal Yahav, Onn Shehory, and David Schwartz.  Comments mining with tf-idf: the inherent bias and its removal.  IEEE Transactions on Knowledge and Data Engineering (TKDE), 31(3):437–450, 2018.
  * [23] Yihui He, Xiangyu Zhang, and Jian Sun.  Channel pruning for accelerating very deep neural networks.  In Proceedings of IEEE International Conference on Computer Vision (ICCV), 2017.
  * [24] Mingbao Lin, Rongrong Ji, Yuxin Zhang, Baochang Zhang, Yongjian Wu, and Yonghong Tian.  Channel pruning via automatic structure search.  In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), 2020.
  * [25] Sun-Yuan Kung, Zejiang Hou, and Yuchen Liu.  Methodical design and trimming of deep learning networks: Enhancing external bp learning with internal omnipresent-supervision training paradigm.  In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019.
  * [26] Zhuangwei Zhuang, Mingkui Tan, Bohan Zhuang, Jing Liu, Yong Guo, Qingyao Wu, Junzhou Huang, and Jinhui Zhu.  Discrimination-aware channel pruning for deep neural networks.  In Proceedings of Conference on Neural Information Processing Systems (NeurIPS), 2018.
  * [27] Vinod Nair and Geoffrey E Hinton.  Rectified linear units improve restricted boltzmann machines.  In Proceedings of International Conference on Machine Learning (ICML), 2010.
  * [28] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of IEEE/CVF conference on Computer Vision and Pattern Recognition, 2016.
  * [30] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [31] Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon.  Federated learning: Strategies for improving communication efficiency.  arXiv preprint arXiv:1610.05492, 2016.
  * [32] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong.  Local model poisoning attacks to byzantine-robust federated learning.  In Proceedings of USENIX Security Symposium (Security), 2020.
  * [33] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In Proceedings of IEEE Symposium on Security and Privacy (SP), pages 3–18, 2017.
  * [34] Yinzhi Cao and Junfeng Yang.  Towards making systems forget with machine unlearning.  In Proceedings of IEEE Symposium on Security and Privacy (SP), 2015\.
  * [35] Jonathan Brophy and Daniel Lowd.  Machine unlearning for random forests.  In Proceedings of International Conference on Machine Learning (ICML), 2021.
  * [36] Quoc Phong Nguyen, Bryan Kian Hsiang Low, and Patrick Jaillet.  Variational bayesian unlearning.  In Proceedings of Conference on Neural Information Processing Systems (NeurIPS), 2020.
  * [37] Gaoyang Liu, Xiaoqiang Ma, Yang Yang, Chen Wang, and Jiangchuan Liu.  Federaser: Enabling efficient client-level data removal from federated learning models.  In Proceedings of IEEE/ACM International Symposium on Quality of Service (IWQoS), 2021.

##  Appendix A Appendices

###  A.1 Multi-class removal: extensive cases

In the case of multi-class removal, Table 3 has shown the effect of deleting
two categories of CIFAR10 simultaneously from ResNet20. For a more
comprehensive investigation, we conduct similar experiments on CIFAR10 dataset
with more pre-trains models (ResNet20/56, VGG11/19), as shown in Figure 4 and
Table 6. The learning method is FedSGD, learning rate is set 0.1, and bias
probability is 0.1(0.01). As seen, the performance of our method in unlearning
acceleration and information erasure is satisfactory.

###  A.2 Term frequency inverse document frequency (TF-IDF): formal definition

As introduced in Section 3.2, TF-IDF for a word in a document is quantized by
multiplying TF and IDF. TF-IDF multiplies the frequency of term (TF) appears
in a document and the inverse document frequency (IDF) of the term across a
collection of documents. In particular, TF-IDF score for a term $\bm{t}$ in
document $\bm{e}$ from the document set
$\bm{E}$={$\bm{e}_{1}$,$\bm{e}_{2}$,$\dots$,$\bm{e}_{n}$} is calculated:

$\displaystyle\text{TF-
IDF}(\bm{t},\bm{e},\bm{E})=\text{TF}(\bm{t},\bm{e})*\text{IDF}(\bm{t},\bm{E})$
(10)

where TF(·) and IDF(·) are defined to calculate as:

$\displaystyle\text{TF}(\bm{t},\bm{e})$
$\displaystyle=\texttt{freq}(\bm{t},\bm{e})/|\bm{e}|$ (11)
$\displaystyle\text{IDF}(\bm{t},\bm{E})=\log[(|\bm{E}|+1)$
$\displaystyle/(\texttt{count}(\bm{e},\bm{e}\in\bm{E}\cap\bm{t}\in\bm{e})+1)]$
(12)

TF equals that the number of occurrences $\texttt{freq}(\bm{t},\bm{e})$ in
document $\bm{e}$ divided by the length $|\bm{e}|$ of this document, which
eliminates the impact of the length of the document on the term frequency. IDF
equals the length of the set $\bm{E}$ of all documents, divided by the length
$\texttt{count}(\bm{e},\bm{e}\in\bm{E}\cap\bm{t}\in\bm{e})$ of the subset of
documents term $\bm{t}$ appears in each of these documents.

###  A.3 ReLU activation and avg_pool2d operation: more details in Section 4.1

At each client, local_proc in the unlearning program is conducted with local
private images, in order to generate a local representation between channels
and categories, and facilitate measuring the class-wise discrimination of
different channels.

As a first step, the client will record the feature map generated by local
model in each layer. Then, for $l$-th layer, we reduce the dimension of
feature map tensor $\bm{O}_{l}^{i}$ of $i$-th category image by a ReLU layer
and an average pool layer as:

$\displaystyle{\bm{A}^{\prime}}_{l}^{i}=\left[\sum_{1}^{H_{l+1}}\sum_{1}^{W_{l+1}}\max(0,\bm{O}_{l}^{i})\right]/(H_{l+1}*W_{l+1})$
(13)

where $\bm{O}_{l}^{i}\in\mathcal{R}^{C_{in}^{l+1}\times H_{l+1}\times
W_{l+1}}$, and $C_{in}^{l+1}$=$C_{out}^{l}$. We refer
${\bm{A}^{\prime}}_{l}^{i}\in\mathcal{R}^{C_{out}^{l}\times 1}$ as the
category activation value of channels, and we denote the local representation
between classes and channels as

$\displaystyle\bm{A}^{\prime}_{l}=\left\\{{\bm{A}^{\prime}}_{l}^{1},{\bm{A}^{\prime}}_{l}^{2},\dots,{\bm{A}^{\prime}}_{l}^{|\bm{U}|}\right\\}\in\mathcal{R}^{|\bm{U}|\times
C_{out}^{l}}$ (14)

The global representation ${\bm{A}^{*}}_{l}^{i}$ of $i$-th category is
calculated by averaging local representation $\bm{A}^{\prime}_{l}$ for all
$i$-th class images. we denote the global representation between classes and
channels as

$\displaystyle\bm{A}^{*}_{l}=\left\\{{\bm{A}^{*}}_{l}^{1},{\bm{A}^{*}}_{l}^{2},\dots,{\bm{A}^{*}}_{l}^{|\bm{U}|}\right\\}\in\mathcal{R}^{|\bm{U}|\times
C_{out}^{l}}$ (15)

###  A.4 Algorithm of the framework

The procedure steps in the framework (as referred to Section 4) can be found
from Algorithm 3.

###  A.5 Algorithm of local_proc

The procedure steps in local_proc (as referred to Section 4.1) can be found
from Algorithm 1.

Algorithm 1 Procedure steps in local_proc.

0: the collection of local private images, $\bm{X}_{1}$;the local FL model
that has been pre-trained, $\bm{w}$;

0: local representation between classes and channels, $\bm{A}^{\prime}$;

1: the client conducts the unlearning program in its local;

2: for each model layer $l$ in $L$ do

3: record output feature map in $l$-th layer
$\bm{O}_{l}$=$\bm{X}_{l}\circledast\bm{w}_{l}$;

4: $\bm{O}^{\prime}_{l}\leftarrow$ apply ReLU activation to output feature map
$\bm{O}_{l}$;

5: $\bm{A}_{l}\leftarrow$ apply avg_pool2d to after-activated
$\bm{O}^{\prime}_{l}$;

6: $\bm{A}^{\prime}_{l}\leftarrow$ impose $\bm{A}_{l}$ to be averaged by
class-wise;

7: end for

8: return
$\bm{A}^{\prime}$=$\\{\bm{A}^{\prime}_{1},\bm{A}^{\prime}_{2},\dots,\bm{A}^{\prime}_{L}\\}$;

###  A.6 Algorithm of server_proc

The procedure steps in server_proc (as referred to Section 4.2) can be found
from Algorithm 2.

Algorithm 2 Procedure steps in server_proc.

0: the class to be forgotten, $u^{\prime}$;the set of total available classes,
$\bm{U}$;the collection of participant FL clients, $\bm{n}$;the global FL
model that has been pre-trained, $\bm{w}$;local representations between
classes and channels, $\bm{A}^{\prime}$;hyper-parameter to determine threshold
of pruning $R$;

0: FL model with discriminative channels pruned, $\hat{\bm{w}}$;

1: $\bm{A}^{*}\leftarrow$ aggregate $\bm{A}^{\prime}_{i}$ in average for each
$i\in\bm{n}$;

2: for each model layer $l$ in $L$ do

3: calculate TFl($\bm{A}^{*}$, $u^{\prime}$), IDFl($\bm{A}^{*}$, $\bm{U}$),
TF-IDFl(TFl, IDFl)/* according to Equation (6), (7) and (8) */

4: TF-IDF $\leftarrow$concatnate(TF-IDFl);

5: end for

6: $\hat{\bm{w}}\leftarrow$channel_pruner($\bm{w}$, TF-IDF, $R$);

7: return $\hat{\bm{w}}$;

Figure 4: Extensive experiments: removing multiple categories of CIFAR10 from
pre-trained models.

ResNet20 CIFAR10 | Raw model | First class pruned | Last class pruned | Fine-tuned | Re-trained
---|---|---|---|---|---
Model Accuracy | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | Speedup
Delete class 0+9 from [0-9] | 88.70% | 83.01% | 02.10% | 24.57% | 00.00% | 32.41% | 00.00% | 87.12% | 00.00% | 87.29% | ×8.71
Delete class 0+5 from [0-9] | 81.90% | 84.71% | 00.25% | 25.04% | 00.00% | 26.74% | 00.00% | 89.62% | 00.00% | 89.75% | ×10.62
Delete class 5+9 from [0-9] | 84.70% | 84.01% | 02.10% | 31.74% | 00.00% | 37.71% | 00.00% | 88.37% | 00.00% | 88.21% | ×8.92
Delete class 0+5+9 from [0-9] | 85.10% | 83.74% | 01.57% | 28.01% | 00.00% | 30.00% | 00.00% | 89.62% | 00.00% | 89.23% | ×8.45

ResNet56 CIFAR10 | Raw model | First class pruned | Last class pruned | Fine-tuned | Re-trained
---|---|---|---|---|---
Model Accuracy | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | Speedup
Delete class 0+9 from [0-9] | 91.75% | 83.23% | 01.20% | 12.50% | 00.00% | 19.38% | 00.00% | 87.22% | 00.00% | 86.38% | ×10.33
Delete class 0+5+9 from [0-9] | 85.57% | 84.66% | 03.82% | 14.29% | 00.00% | 33.33% | 00.00% | 87.10% | 00.00% | 87.23% | ×9.66

VGG11 CIFAR10 | Raw model | First class pruned | Last class pruned | Fine-tuned | Re-trained
---|---|---|---|---|---
Model Accuracy | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | Speedup
Delete class 0+5+9 from [0-9] | 83.60% | 85.70% | 00.63% | 17.77% | 00.00% | 20.61% | 00.00% | 88.40% | 00.00% | 88.34% | ×10.77

VGG19 CIFAR10 | Raw model | First class pruned | Last class pruned | Fine-tuned | Re-trained
---|---|---|---|---|---
Model Accuracy | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | U-set | R-set | Speedup
Delete class 0+5+9 from [0-9] | 84.87% | 83.71% | 05.60% | 24.13% | 00.00% | 34.17% | 00.00% | 89.44% | 00.00% | 89.40% | ×9.29

Table 6: Extensive experiments: delete multiple categories of CIFAR10
simultaneously from pre-trained models. Algorithm 3 Framework of federated
unlearning.

0: the collection of participant FL clients, $\bm{n}$;the FL model that has
been pre-trained, $\bm{w}$;the set of classes to be forgotten,
$\bm{U}^{\prime}$;

0: the FL model with target classes unlearned, $\bm{w}^{\prime}$;

1: if there is a request to remove $\bm{U}^{\prime}$ from $\bm{w}$ then

2: the federated server notifies all clients in $\bm{n}$;

3: for each client $i$ in $\bm{n}$ do

4: download unlearning program from the server;

5: calculate local_proc($\bm{w}$, private images);/* defined in Section 4.1 */

6: upload calculated representation to the server;

7: end for

8: if the server got uploads from all clients in $\bm{n}$ then

9: pruned $\hat{\bm{w}}\leftarrow$ server_proc($\bm{w}$,
$u^{\prime}\in\bm{U}^{\prime}$, aggregation);/* defined in Section 4.2 */

10: end if

11: $\bm{w}^{\prime}\leftarrow$ finetuning_proc($\hat{\bm{w}}$, target
accuracy);

12: return $\bm{w}^{\prime}$;

13: end if


00footnotetext: Licensed under a Creative Commons Attribution 4.0
International License (CC BY 4.0). Copyright remains with the author(s).

# Augmenting the Author: Exploring the Potential of AI Collaboration in
Academic Writing

Joseph Tu<EMAIL_ADDRESS>0000-0002-7703-6234 Stratford School of
Interaction Design and Business, University of Waterloo200 University Ave
WCanada ,  Hilda Hadan<EMAIL_ADDRESS>0000-0002-5911-1405 Stratford
School of Interaction Design and Business, University of
WaterlooWaterlooCanada ,  Derrick M. Wang<EMAIL_ADDRESS>0000-0003-3564-2532 Stratford School of Interaction Design and Business,
University of WaterlooWaterlooCanada ,  Sabrina A. Sgandurra
<EMAIL_ADDRESS>0000-0003-3187-263X Stratford School of Interaction
Design and Business, University of WaterlooWaterlooCanada ,  Reza Hadi Mogavi
<EMAIL_ADDRESS>0000-0002-4690-2769 Stratford School of Interaction
Design and Business, University of WaterlooWaterlooOntarioCanada and  Lennart
E. Nacke<EMAIL_ADDRESS>https://orcid.org/0000-0003-4290-8829
Stratford School of Interaction Design and Business, University of
WaterlooWaterlooCanada

(2024; 26 February 2024; 13 March 2024)

###### Abstract.

This workshop paper presents a critical examination of the integration of
Generative AI (Gen AI) into the academic writing process, focusing on the use
of AI as a collaborative tool. It contrasts the performance and interaction of
two AI models, Gemini and ChatGPT, through a collaborative inquiry approach
where researchers engage in facilitated sessions to design prompts that elicit
specific AI responses for crafting research outlines. This case study
highlights the importance of prompt design, output analysis, and recognizing
the AI’s limitations to ensure responsible and effective AI integration in
scholarly work. Preliminary findings suggest that prompt variation
significantly affects output quality and reveals distinct capabilities and
constraints of each model. The paper contributes to the field of Human-
Computer Interaction by exploring effective prompt strategies and providing a
comparative analysis of Gen AI models, ultimately aiming to enhance AI-
assisted academic writing and prompt a deeper dialogue within the HCI
community.

Generative AI, AI, Natural Language Processing, HCI Research Tools

††conference: ; ; ††copyright: acmlicensed††journalyear: 2024††doi:
10.17605/OSF.IO/FVTMQ††conference: GenAI Workshop ’24; May 11–16, 2024;
Honolulu, Hawaii††booktitle: GenAI Workshop ’24 May 11–16, 2024 Honolulu,
Hawaii††isbn: 10.17605/OSF.IO/FVTMQ††ccs: Computing methodologies Artificial
intelligence††ccs: Human-centered computing Human computer interaction (HCI)

##  1\. Introduction

In the realm of scholarly investigation, traversing the preliminary stages of
drafting a research manuscript can pose challenges (Sedlmair et al., 2012).
Some of the most common struggles when writing research papers are the tight
schedule, lack of inspiration (for writing), and the complex process of
organizing ideas and creating an outline (Juzwik et al., 2006; AlMarwani,
2020). This issue is particularly salient for early career academics and
individuals whose first language is not English (Zhao, 2023; Fitria, 2021). We
ask ourselves, could Artificial Intelligence (AI) help?

The purpose of our workshop paper is to explore the potential of using AI to
address challenges faced by the academic research community. Through an
exploratory research approach, we aim to gather intellectual feedback and
initiate a dialogue within the HCI community. Instead of presenting a
traditional or formal research paper, our goal is to open a discussion on this
topic. We will be sharing our combined reflections on the use of AI, focusing
on two popular Generative AI models - ChatGPT111ChatGPT.
https://chat.openai.com/share/bf65d447-70c8-4117-87ef-f258265e3e43 and
Gemini222Gemini. https://gemini.google.com/share/259d4d8f69f8.

However, the application of AI as a writing assistant is often impeded by a
dearth of transparency in researchers’ access and utilization (Rai, 2020;
Balasubramaniam et al., 2023). This lack of transparency raises concerns about
the reliability and credibility of AI-generated text (Ali et al., 2020;
Baidoo-Anu and Ansah, 2023). Hence, comparative analysis between ChatGPT and
Gemini as writing assistants for research draft outlines becomes crucial in
understanding their capabilities and potential limitations.

At its core, our research emphasizes the importance of continual reflection
and validation throughout the entire process of interacting with Gen AI. This
includes thoughtful, prompt design, comprehensive output analysis, and an
awareness of potential limitations. We contend that prioritizing this process-
oriented approach is crucial for responsibly integrating Gen AI into academic
research, leading to more accurate and reliable outcomes for researchers.

For this workshop paper, our discussion will revolve around two key aspects:

  * •

Exploring Effective Prompt Strategies: In this section, our attention is
directed towards the realm of prompts that researchers themselves generate
(the authors), as well as the pivotal inquiries that guide the outputs of Gen
AI models, including ChatGPT and Gemini. Drawing upon the valuable insights
acquired from our weekly deliberations, we aim to distinguish and articulate
efficient strategies for crafting prompts.

  * •

A Comparative Perspective on Results: We compare two Gen AI models by giving
them the same prompts and asking them to generate outlines for a specific
research topic. Our collaborative inquiry approach will reveal the nuanced
differences in their responses.

As there is growing research in prompt engineering, we aim to expand upon
existing contributions, including recent works from (Ekin, 2023; Rodriguez et
al., 2023; Wang et al., 2023), by undertaking a thorough examination of the
merits and drawbacks of different models in their handling of various prompt
methods. Furthermore, we present a detailed account of the collaborative
process with AI, providing the researcher’s perspective and promoting
transparency and accountability in assessing the efficacy and limitations of
ChatGPT and Gemini as writing aids. We focus not on formal and statistical
comparisons, as we acknowledge potential subjective biases and confounding
factors. Instead, we aim to offer a firsthand experience that can serve as a
starting point for meaningful discussions within the HCI community,
potentially leading to further and more comprehensive investigations.

This position paper aims to make two significant contributions to the field of
HCI. Firstly, it seeks to delineate effective prompt engineering strategies
that enhance the collaborative dynamic between human researchers and AI,
thereby optimizing the integration of AI in the research outline development
process. Secondly, the paper aims to provide a nuanced comparative analysis of
the capabilities and limitations of two prevalent generative AI models,
ChatGPT and Gemini, grounded in real-world application scenarios that could
serve as a foundational framework for future research and development in AI-
assisted academic writing.

##  2\. Research Approach

We used a collaborative inquiry approach (Bray, 2000), focusing on
facilitating structured and inclusive discussions and deliberations among
ourselves as researchers; and culminating in us collectively reaching
thoughtful judgments aligned with a specific goal or purpose, such as
formulating recommendations or crafting a research outline. This case study
aimed to evaluate the strengths and limitations of large language models
(LLM), particularly Gemini and ChatGPT. Researchers (4—6) participated in
weekly sessions, utilizing Otter.Ai for transcription. Discussions focused on
the strengths and limitations of the models, guided by carefully crafted
prompts. The strategic use of AI models in research outline development was
evaluated based on predefined criteria. AI-generated prompts further
facilitated critical thinking and reflection (condensed for a workshop see
Table 1).

Table 1. An overview of our collaborative inquiry approach

Approach |  Description
---|---
A. Weekly Collaborative Inquiry Sessions |  Format and Frequency: Engaging in regular weekly collaborative inquiry sessions, initiated on January 19th (still on-going as of this submission), where 4-6 researchers convene for one-hour discussions. These sessions provide a structured platform for collective exploration and dialogue. Transcription: We used Otter.Ai as a recording tool during these sessions, fostering collaborative transcription and documentation. This software not only ensures accuracy in capturing discussions but also facilitates easy reference and analysis of key points for us.
B. Collaborative Inquiry Prompts and Guiding Questions |  Designed Prompts: The research design incorporates carefully crafted prompts. These prompts are strategically formulated to extract nuanced insights from the collective group, emphasizing the strengths and limitations of each AI model. Guidance in Collaborative Deliberations: These prompts play a pivotal role in steering collaborative discussions. By focusing on specific aspects, the prompts encourage a detailed examination of the AI models, fostering a deeper understanding of their capabilities in the context of outlines for research papers.
C. Collaborative Research Outline Development |  Strategic Utilization of AI Models: At the core of the collaborative inquiry process is the strategic use of both Gemini and ChatGPT in generating research outlines. We actively explore how these AI models collectively structure and organize their ideas coherently, aiming to streamline the initial stages of the research process. Criteria for Collaborative Evaluation: The effectiveness of each AI model in research outline development is collaboratively evaluated based on predefined criteria. These criteria include the coherence of the generated outlines, alignment with research objectives, and overall recommendations and critical questions planned for the research phase.
D. Collaborative Critical Thinking Facilitation |  Collaborative Process of AI-Generated Prompts: To enhance critical thinking among the research team. We asked the tools to provide us with critical thinking questions and suggestions to enhance the writing process. These prompts are specifically designed to stimulate collective reflection and provoke thoughtful analysis of the research topics under consideration. Objectives of Collaborative Analysis: The deliberate facilitation of critical thinking through AI prompts aims to assess the intellectual stimulation provided by Gemini and ChatGPT. It also guides researchers collaboratively in refining their research objectives through a more thoughtful and analytical approach.
E. Collaborative Perspective Elicitation |  Individual Insights and Their Reflections: The research methodology includes individual insights as a means to gain diverse perspectives on Gemini and ChatGPT. These reflections serve as a targeted approach to understanding how each researcher collectively perceives the strengths and limitations of the respective AI models.

##  3\. Preliminary Analysis of ChatGPT and Gemini: Our Collective Reflection.

Through a series of comparisons and observations, we identified key
differences in their approach, information handling, and potential
applications within the research process. Exploring the capabilities of
ChatGPT and Gemini in various tasks prompts a critical examination of their
strengths and limitations within the context of research outline development.
While ChatGPT consistently fulfills tasks based on explicit instructions,
concerns arise due to its tendency to fabricate responses, raising questions
about the reliability of generated content (OpenAI, 2023). We often noted that
the model’s word-by-word rephrasing approach, coupled with a reluctance to
restructure sentences autonomously, emphasizes its dependence on precise
directives. Both models occasionally struggle with LaTeX formatting, which
introduces complexities that may impact their usability in crafting research
outlines (at the time of this submission). On the other hand, Gemini proves to
be a dynamic counterpart, not only completing tasks but also providing
valuable suggestions and rationales. Its ability to guide users through
reference searches enhances the reliability of information retrieval. Gemini’s
capacity to offer multiple word choice suggestions, each accompanied by
detailed explanations, adds sophistication to its linguistic capabilities. In
tasks requiring paragraph improvement based on references, both models
demonstrate a dependence on explicit prompts as outlined by Lo (2023),
highlighting the importance of precise instruction in research outline
development.

Table 2. A preliminary summary of our collective reflection findings for
ChatGPT and Gemini

Category |  ChatGPT Preliminary Findings |  Gemini Preliminary Findings
---|---|---
Strengths |  Consistent adherence to explicit instructions, precision, and methodical approach. |  Dynamic, provides additional suggestions and rationales. Proficient in guiding through reference searches. Offers multiple word choice suggestions with detailed explanations.
Limitations |  Tendency to fabricate responses, reliance on word-by-word rephrasing, occasional struggle with LaTeX formatting. |  Reliance on explicit prompts in tasks requiring paragraph improvement based on a reference. Limited customization options compared to ChatGPT.
Articulating AI Potential |  More straightforward outlines. |  Excels in comprehensive summaries with self-promotion.
Handling Mistakes |  May fabricate information, raising concerns about reliability. |  Prioritizes transparency, explains confusing sources, and guides on independent information retrieval.
Human-AI Collaboration |  Ongoing exploration and adaptation required for maximizing benefits and mitigating limitations in the evolving dynamics. |  Ongoing exploration and adaptation are required to maximize benefits and mitigate evolving dynamics’ limitations.
Common Sense Integration |  Deficiency in common sense could be addressed through customization using its panel. |  Undetermined
Proactivity |  Undetermined |  Demonstrates initiative and understanding of broader context, delivering optimal outcomes without waiting for specific directives.
AI’s Role in Education |  Reflects on AI’s potential evolution into a personalized learning tool that adapts to individual learning styles.

In terms of articulating the potential of generative AI for research outlines,
Gemini distinguishes itself with comprehensive summaries that include self-
promotion. In contrast, ChatGPT tends to provide more straightforward
summaries without articulating into additional insights. We also identified
ChatGPT’s potential limitations in discerning the best method or meaning, and
found its potential deficiency in common sense. This suggest that enhancing
ChatGPT’s performance could be achieved through greater customization using
its customization panel, potentially infusing a form of common sense into its
responses, particularly when generating content for research outlines.

In addition, Gemini impresses with its proactive and initiative-taking nature.
It demonstrates a keen understanding of the broader context and consistently
suggests new phrasings or alternative options without explicit requests,
showcasing its potential to aid research outline development without waiting
for specific directives. A notable distinction lies in how these models handle
mistakes. While ChatGPT at times resorts to fabricating information, raising
concerns about reliability and inadvertent alterations of intended meaning,
Gemini prioritizes being helpful over maintaining an appearance of
infallibility. Gemini transparently addresses limitations, enhancing its
credibility and transparency compared to ChatGPT, particularly in the context
of research outline development.

Researcher’s Antidote on Certain Concerns. Participant 1 (P1) quotes: “This
worries me as a reviewer, am I a human validating the author’s work or AI’s
work?” One notable concern is the potential shift in the role of reviewers,
who may unintentionally become the primary validators of AI-generated content
rather than the authors themselves. The human touch, creativity, and critical
thinking are essential components of writing. If AI takes a prominent role in
generating content, there is a risk of diminishing these human qualities. We
suggest authors utilizing AI in their writing should prioritize transparency
by clearly disclosing the involvement of AI tools in the creation process.

P6 quotes: “After using these tools for a while, I have a bias towards a
certain writing style. I can tell that AI writes this or that, but what
happens if this is just their writing style? Personally, English as a second
language, I was taught to write in a formal construct.” As we explore the
potential of AI writing tools, we acknowledge concerns surrounding the
negative perception of their generated styles, particularly within academia.
We understand the apprehension that might stem from unfamiliarity with this
format, potentially leading to bias against its lack of established
conventions. Additionally, we recognize the fears surrounding the potential
for both quality concerns and ethical misuse of these tools. These concerns
could translate into various challenges. We might encounter resistance towards
adopting AI writing tools, creating an uneven playing field for those with
access. This further highlights the need for clear guidelines to ensure
responsible and ethical usage within academic settings (Hadi Mogavi et al.,
2024). While addressing these concerns is crucial for successful integration,
we believe that continuous development and increasing familiarity with AI
writing might eventually alleviate the initial negative connotations.

###### Acknowledgements.

This workshop was supported by the SSHRC INSIGHT Grant (grant
number:435-2022-0476), NSERC Discovery Grant (grant number: RGPIN-2023-03705)
and CFI John R. Evans Leaders Fund or CFI JELF (grant number: 41844). Special
thanks to the members of the Games Institute at the University of Waterloo for
all their support.

## References

  * (1)
  * Ali et al. (2020) Muhammad Yousuf Ali, Salaman Bin Naeem, and Rubina Bhatti. 2020.  Artificial intelligence tools and perspectives of university librarians: An overview.  _Business Information Review_ 37, 3 (2020), 116–124.
  * AlMarwani (2020) Manal AlMarwani. 2020.  Academic writing: Challenges and potential solutions.  _Arab World English Journal (AWEJ) Special Issue on CALL_ 6 (2020).
  * Baidoo-Anu and Ansah (2023) David Baidoo-Anu and Leticia Owusu Ansah. 2023.  Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning.  _Journal of AI_ 7, 1 (2023), 52–62.
  * Balasubramaniam et al. (2023) Nagadivya Balasubramaniam, Marjo Kauppinen, Antti Rannisto, Kari Hiekkanen, and Sari Kujala. 2023.  Transparency and explainability of AI systems: From ethical guidelines to requirements.  _Information and Software Technology_ 159 (2023), 107197.
  * Bray (2000) John N Bray. 2000.  _Collaborative inquiry in practice: Action, reflection, and making meaning_.  Sage.
  * Ekin (2023) Sabit Ekin. 2023.  Prompt engineering for ChatGPT: A quick guide to techniques, tips, and best practices.  _Authorea Preprints_ (2023).
  * Fitria (2021) Tira Nur Fitria. 2021.  Grammarly as AI-powered English writing assistant: Students’ alternative for writing English.  _Metathesis: Journal of English Language, Literature, and Teaching_ 5, 1 (2021), 65–78.
  * Hadi Mogavi et al. (2024) Reza Hadi Mogavi, Chao Deng, Justin Juho Kim, Pengyuan Zhou, Young D. Kwon, Ahmed Hosny Saleh Metwally, Ahmed Tlili, Simone Bassanelli, Antonio Bucchiarone, Sujit Gujar, Lennart E. Nacke, and Pan Hui. 2024.  ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptions.  _Computers in Human Behavior: Artificial Humans_ 2, 1 (Jan. 2024), 100027.
  * Juzwik et al. (2006) Mary M Juzwik, Svjetlana Curcic, Kimberly Wolbers, Kathleen D Moxley, Lisa M Dimling, and Rebecca K Shankland. 2006.  Writing into the 21st century: An overview of research on writing, 1999 to 2004.  _Written Communication_ 23, 4 (2006), 451–476.
  * Lo (2023) Leo S Lo. 2023.  The CLEAR path: A framework for enhancing information literacy through prompt engineering.  _The Journal of Academic Librarianship_ 49, 4 (2023), 102720.
  * OpenAI (2023) OpenAI. 2023.  _Gpt-4 technical report_.  Technical Report. OpenAI.  arXiv preprint arXiv:2303.08774.
  * Rai (2020) Arun Rai. 2020.  Explainable AI: From black box to glass box.  _Journal of the Academy of Marketing Science_ 48 (2020), 137–141.
  * Rodriguez et al. (2023) Alberto D Rodriguez, Katherine R Dearstyne, and Jane Cleland-Huang. 2023.  Prompts Matter: Insights and Strategies for Prompt Engineering in Automated Software Traceability. In _2023 IEEE 31st International Requirements Engineering Conference Workshops (REW)_. IEEE, 455–464.
  * Sedlmair et al. (2012) Michael Sedlmair, Miriah Meyer, and Tamara Munzner. 2012.  Design study methodology: Reflections from the trenches and the stacks.  _IEEE transactions on visualization and computer graphics_ 18, 12 (2012), 2431–2440.
  * Wang et al. (2023) Mo Wang, Minjuan Wang, Xin Xu, Lanqing Yang, Dunbo Cai, and Minghao Yin. 2023.  Unleashing ChatGPT’s Power: A Case Study on Optimizing Information Retrieval in Flipped Classrooms via Prompt Engineering.  _IEEE Transactions on Learning Technologies_ (2023).
  * Zhao (2023) Xin Zhao. 2023.  Leveraging artificial intelligence (AI) technology for English writing: Introducing wordtune as a digital writing assistant for EFL writers.  _RELC Journal_ 54, 3 (2023), 890–894.