# Sentiment Analysis with R

a Working Paper and Tutorial by Dennis Klinkhammer

###### Abstract

Sentiment analysis is a sub-discipline in the field of natural language
processing and computational linguistics and can be used for automated or
semi-automated analyses of text documents. One of the aims of these analyses
is to recognize an expressed attitude as positive or negative as it can be
contained in comments on social media platforms or political documents and
speeches as well as fictional and nonfictional texts. Regarding analyses of
comments on social media platforms, this is an extension of the previous
tutorial on semi-automated screenings of social media network data. A
longitudinal perspective regarding social media comments as well as cross-
sectional perspectives regarding fictional and nonfictional texts, e.g. entire
books and libraries, can lead to extensive text documents. Their analyses can
be simplified and accelerated by using sentiment analysis with acceptable
inter-rater reliability. Therefore, this tutorial introduces the basic
functions for performing a sentiment analysis with R and explains how text
documents can be analysed step by step - regardless of their underlying
formatting. All prerequisites and steps are described in detail and associated
codes are available on GitHub. A comparison of two political speeches
illustrates a possible use case.

### Keywords

Sentiment Analysis, Natural Language Processing, Computational Linguistics,
Qualitative Data

## Introduction to Sentiment Analysis and this Tutorial

Sentiment analysis – as the name suggests - can be used to capture the
sentiment in qualitative data, such as text documents. Text documents can
contain different types of content and information, e.g. comments on social
media platforms or political documents and speeches as well as fictional and
nonfictional texts up to entire libraries. Usually, these text documents come
in different formats, e.g. PDF format, HTML format and many more, depending on
the medium the text is located. Since most formats can be converted into a
simpler format, such as the TXT format, this tutorial uses the TXT format as
default. For example, if someone wants to convert a text document from PDF
format to TXT format, the copy and paste function would be sufficient for this
tutorial.

Within this TXT format the polarity can be classified word by word as positive
or negative and in some cases neutral via basic sentiment analysis. In
addition, different types of emotional states can be classified via advanced
sentiment analysis and by using the NRC Word-Emotion Association Lexicon, the
first word-emotion lexicon with eight basic emotional states (Mohammad 2020).
Despite positive and negative sentiments, anger, fear, anticipation, trust,
surprise, sadness, joy, and disgust can be classified as well. Furthermore, a
semi-automated sentiment analysis has a sufficient inter-rater-reliability
with less time requirements. The inter-rater-reliability is a degree of
agreement among independent observers who rate, code, or assess the same
phenomenon within a text document. While scientist usually achieve an inter-
rater-reliability up to 80% the semi-automated ones can achieve up to 70%.
This appears to be an acceptable value, because even if different types of
semi-automated sentiment analyses would agree up to 100%, research indicates
that scientists would still disagree by 20% (Ogneva 2010). Therefore,
sentiment analyses can be found in a broad application context, of which a few
will be presented in this tutorial.

Since this is an addition to the previous tutorial on semi-automated
screenings of social media network data (Klinkhammer 2020), the analysis of
comments on social media platforms should not go unmentioned. Especially
social media platforms offer several users a low-threshold opportunity to
exchange opinions and experiences. For example, these opinions and experiences
can affect various areas of society, such as political and economical ones.
From a methodological point of view, the question whether one technique is
equally suitable for entire books as well as short comments on social media
platforms seems relevant. Research indicates that, for example, comments on
social media platforms can be used to capture social issues like
radicalisation and extemism (Tanoli et al. 2022), sexuality (Wood et al.
2017), side effects of medication and drugs (Korkontzelos et al. 2016) as well
as for the reflection of the offline political landscape (Tumasjan et al.
2010). A well known use-case is the political campaign of former U.S.
president Barack Obama, who used sentiment analysis back in 2012. These
methodological approaches hardly differ from the analyses of entire books.
There are many possible use-cases, but also numerous challenges in the
application of sentiment analysis. Accordingly, research in this area
continues (Hamborg & Donnay 2021) and this tutorial refers to a semi-automated
sentiment analysis rather than automated ones.

This contribution includes all software requirements, a full disclosure of
codes for the R programming language, the entire process of accessing and pre-
precessing text documents and how to perform basic and advances sentiment
analyses. This tutorial is mainly supposed to be a methodological tutorial for
students, and researchers.

## Software Requirements

Written in _R Markdown_ , this tutorial refers to the R programming language.
A free software environment for using R is available for Linux, macOS by Apple
and Windows by Microsoft. The main purpose of R is statistical computing and
it is both used for manual quantitative and qualitative analyses as well as
automated or semi-automated analyses. When it comes to Big Data, R can also be
used for unsupervised and supervised Machine Learning.

In detail, R is an object based programming language. Therefore, datasets,
variables, cases, values as well as functions can be applied as a combination
of objects. All commands, as combinations of functions, datasets, variables,
cases, values and functions will be highlighted, so that they can be used as
step by step tutorial. The commands have to be entered directly into the R
terminal, which is available after downloading, installing and starting the R
software environment.

## Preparations: Attaching necessary Packages

This tutorial requires six additional packages in order to expand the range of
basic R functions. All packages can be installed by using the _install(…)_
command and attached via the _library(…)_ command by typing the following
commands directly into the R terminal.

Since the analysis of extensive text documents requires a focus on every
single element that is to be analysed, it is necessary to break down the
underlying data structure into manageable little pieces. A package that is
specifically designed to do so is called _dplyr_. It can split, apply and
combine data for further analytical steps (Wickham 2022). The package _dplyr_
can be installed and attached as follows:

install.packages("dplyr", dependencies=TRUE)

library(dplyr)

The second package is called _stringr_. Since qualitative data, like the text
in social media comments, is represented by character variables in R, a
package that can process and - if necessary - manipulate individual characters
within the strings of a character variable is required (Wickham 2019); A
string is marked either by single quote signs or double quote signs. In order
to install and attach the _stringr_ package, following commands can be typed
in the R terminal:

install.packages("stringr", dependencies=TRUE)

library(stringr)

Another necessary package is called _textdata_. It contains several words as
references and sentiment libraries, such as the NRC Word-Emotion Association
Lexicon. So it does not only needed to be installed by the
_install.packages(…)_ command and attached by the subsequent _library(…)_
command, but the NRC Word-Emotion Association Lexicon must be installed for
advanced sentiment analysis as well. In addition, a second lexicon will be
installed to implement basic sentiment analysis: The Bing Sentiment Lexicon.
This can be done via the _get_sentiments(…)_ command:

install.packages("textdata", dependencies=TRUE)

library(textdata)

get_sentiments("nrc")

get_sentiment("bing")

Sentiment analysis is a text mining technique and the package _tidytext_ is
required in order to convert conventional text documents into tidy formats,
such as single words without punctuation or spaces (De Queiroz et al. 2022).
This allows scientists to focus on paragraphs or otherwise separated content
word by word. As a result, the tidy text format lists and counts all words
individually and assigns them a numbered line according to their paragraph or
other used methods of content separation. Again, this package can be installed
and attached as follows:

install.packages("tidytext", dependencies=TRUE)

library(tidytext)

The package _tidytext_ provides the connection between the packages _dplyr_
and _ggplot2_ by using their basic formulas and commands. The latter is
responsible for the detailed visualisation of sentiments and other types of
results, based on “The Grammer of Graphics” (Wickham et al. 2022). In
particular, defining the details of a visualisation enables scientists to
create informative as well as attractive plots. Therefore, the package
_ggplot2_ requires several dependencies in order to carry out this task:

install.packages("ggplot2", dependencies=TRUE)

library(ggplot2)

Finally, the package _gridExtra_ enables scientists to arrange multiple
visualisations at once and to create dashboards for an intuitive display of
relevant information (Auguie & Antonov 2017). Following commands must be typed
in the R terminal in order to make the package _gridExtra_ work:

install.packages("gridExtra", dependencies=TRUE)

library(gridExtra)

Additional note: If not already pre-installed, the package _magrittr_ is also
required in order to make use of the forward-pipe operator for more elegant
coding. It is possible that further packages have to be installed in the R
environment. This will be automatically checked via the extension of the
_install (…)_ command with _dependencies = TRUE_ that will install additional
packages, if necessary. In a freshly created R environment (based upon version
4.1.2 of R), the six packages listed above have been running sufficient on
Ubuntu 22.04 LTS (Linux Kernel 5.15.0-37), macOS Monterey (12.4) and Windows
11 (22000.708), each brought to application on RStudio.

## Data Pre-Processing: Getting and Cleaning Text Documents

Data pre-processing is used to check datasets for irrelevant and redundant
information present or noisy and unreliable data. In a first step, an external
text document is imported into the R working environment by using the
_read.delim(…)_ command and saving this text document as a new object called
_imported_text_. The external text document is TXT formatted and can be
accessed by entering the associated directory and file name. In this case, a
chapter from one of the Sherlock Holmes novels will be imported (thanks to the
great work of Arthur Conan Doyle):

imported_text <- read.delim("your_unformatted_document.txt", header=F,
sep="\t")

The subsequent _dim(…)_ command indicates the number of paragraphs within this
text document, which is necessary for the next step, as well as the number of
variables within the _imported_text_. As a result, _229_ paragraphs and one
Variable, named _V1_ , can be processed further:

dim(imported_text)

    ## [1] 229   1

The next step creates tibbles out of the _imported_text_. Tibbles are data
frames that are considered plain and simple. In fact, they are so plain and
simple, that the number of lines needs to be entered manually in order to
define the structure of the data frame. In this case, the number of lines
within the tibble is supposed to be the exact number of paragraphs of the
_imported_text_ , hence _229_. As a result, the _tibble(…)_ command leads to a
new object called _text_df_ and the first six lines can be inspected by using
the _head(…)_ command:

text_df <- tibble(line=1:229, text=imported_text$V1)

head(text_df)

    ## # A tibble: 6 x 2
    ##    line text
    ##   <int> <chr>
    ## 1     1 "To Sherlock Holmes she is always the woman. I have seldom heard him"
    ## 2     2 "     mention her under any other name. In his eyes she eclipses and"
    ## 3     3 "     predominates the whole of her sex. It was not that he felt any"
    ## 4     4 "     emotion akin to love for Irene Adler. All emotions, and that one"
    ## 5     5 "     particularly, were abhorrent to his cold, precise but admirably"
    ## 6     6 "     balanced mind. He was, I take it, the most perfect reasoning and"

Although the object _text_df_ is plain and simple formatted, it needs to be
pre-processed further. Sentiment analysis requires a tidy data set, which can
be generated by using the object _text_df_ combined with the
_unnest.tokens(…)_ command. In this case, each word within the text has to be
separated, indicated by _word_ and _text_. For the first time in this tutorial
the connection between an object and command will be established via the
forward-pipe operator _% >%_. Later on, this technique allows to combine
several commands regarding one object and - in this case - to generate the new
object _text_tidy_ :

text_tidy <- text_df %>%

unnest_tokens(word, text)

This object is just a listing of all the words in the different paragraphs.
This allows the sentiment analysis to be performed word by word and line by
line. Since non-essential words may be included in this list, they can be
eliminated by using the _stop_words_ data via the _data(…)_ command. Again,
the forward-pipe operator combines the object _text_tidy_ with the new command
_anti_join(…)_ in order to exclude the specified words:

data(stop_words)

text_tidy <- text_tidy %>%

anti_join(stop_words)

    ## Joining, by = "word"

Now the object _text_tidy_ is prepared for sentiment analysis. In addition,
custom words can also be excluded from this object. They can be generated by
using the _tibble(…)_ command and repeating the previous step of the pre-
processing.

custom_stop_words <- c("custom_word")

custom_stop_words <- tibble(1:1, word=custom_stop_words)

Finally, the _head(…)_ command generates an output of the first six lines of
the pre-processed object _text_tidy_. This object includes only words that are
relevant for an understanding of the original text document and the lines -
respectively paragraphs - they have been mentioned within the text document:

head(text_tidy)

    ## # A tibble: 6 x 2
    ##    line word
    ##   <int> <chr>
    ## 1     1 sherlock
    ## 2     1 holmes
    ## 3     1 woman
    ## 4     1 seldom
    ## 5     1 heard
    ## 6     2 mention

## Analysis - 1: Identification of Common Words and their Sentiment

The first three analytical steps refer to the the NRC Word-Emotion Association
Lexicon, which can be specified as criteria for sentiment analysis via the
_inner_join(…)_ command. Since the first analytical step of this tutorial
relates to the frequency of the words used within the text document and their
sentiments as well as the expressed emotional states, the _count(…)_ command
counts and sorts each _word_ and its _sentiment_ regarding the underlying
object _text_tidy_. The result of this step will be saved as a new object
_nrc_word_counts_ :

# Positive und negative Wörter auflisten

nrc_word_counts <- text_tidy %>%

inner_join(get_sentiments("nrc")) %>%

count(word, sentiment, sort = TRUE) %>%

ungroup()

    ## Joining, by = "word"

This new object can be converted into a corresponding graphic. The words shall
be listed in descending order according to their frequency, where the minimum
number is specified as _n > 2_ within the _filter(…)_ command. Larger text
documents can contain multiple repetitions of the same words, so that the
minimum number has to be adapted accordingly. It is advisable to try different
numbers if necessary. The order is specified by using the _mutate(…)_ command
and the _ggplot(…)_ command combines the words to be highlighted with their
sentiment and emotional state, where each is represented by an individual
color, specified within the _geom_col(…)_ command. Since the sentiments and
emotional states are to be shown as horizontal bar charts, the command
_coord_flip_ is also used. The plot also requires a label that has to be
assigned to the Y-axis. This is done by using the _labs(…)_ command. A title
will be added via the _ggtitle(…)_ command:

nrc_word_counts %>%

filter(n > 2) %>%

mutate(word = reorder(word, n)) %>%

ggplot(aes(word, n, fill = sentiment)) +

geom_col() +

coord_flip() +

labs(y = "sentiment (n)") +

ggtitle("Common Words & Sentiments (Frequency)")

The result is a graphical representation of the most common words in
descending order (Y-axis) and the frequency of associated sentiments and
emotional states (X-axis). For example, the word _doubt_ is the third most
common word but it represents the emotional states trust, sadness and fear as
well as a negative sentiment within different paragraphs of the text document.

## Analysis - 2: Distribution of Sentiments

In the second step of the analysis the total number of each sentiment and
emotional state within the text document shall be focused in order to
highlight their distribution. Here the object _nrc_word_counts_ is used again,
which can be directly specified and plotted via the pipe-forward operator.
This time the words will not be specified within the _ggplot(…)_ command, but
the sentiments and emotional states via _sentiment_ :

nrc_word_counts %>%

inner_join(get_sentiments("nrc")) %>%

count(word, sentiment) %>%

ggplot(aes(sentiment, n, fill = sentiment)) +

geom_col(show.legend = FALSE) +

ggtitle("Sentiments (Distribution)") +

coord_flip()

    ## Joining, by = c("word", "sentiment")

This output highlights the total number (X-axis) of each sentiment and
emotional state (Y-axix) within the entire chapter of the Sherlock Holmes
novel. Overall, the chapter is dominated by a positive sentiment. Furthermore,
trust and anticipation outweigh fear, joy and sadness. Regular readers of
Sherlock Holmes novels should recognise the basic elements that are inherent
for the work of Arthur Conan Doyle.

## Analysis - 3: Intertemporal Use of Sentiments (Conditional Mean)

While the first two analytical steps related to the sentiments and emotional
states of words, the third step is about the conditional means of their use
over time - as known as the intertemporal use of them. Therefore, the
_count(…)_ command needs to be adjusted in order to count every _sentiment_
used in a _line_. ´

nrc_word_counts <- text_tidy %>%

inner_join(get_sentiments("nrc")) %>%

count(line, sentiment, sort = TRUE) %>%

ungroup()

    ## Joining, by = "word"

Subsequently, object _nrc_word_counts_ is a tibble, indicating not only the
sentiments and emotional states within a _line_ but also the total number _n_
of them and how often they are used within that line. For example, line _33_
contains the sentiments negative and the emotional states fear and sadness. In
total, three sentiments and emotional states are used in that line, but each
of them appears only once. As a result, the mean value for line _33_ would be
_1_. Each line can be inspected by using the _subset(…)_ command:

subset(nrc_word_counts, line==33)

    ## # A tibble: 3 x 3
    ##    line sentiment     n
    ##   <int> <chr>     <int>
    ## 1    33 fear          1
    ## 2    33 negative      1
    ## 3    33 sadness       1

In order to highlight the dynamics regarding the appearance of sentiments and
emotional states over time, a smoothed slope will be plotted. Smoothed slopes
do not represent the actual values, like mean values, but they do represent
estimated values, in this case conditional means, to indicate the development.
Thus, primarily the course of the slope should be interpreted and not
individual data points. For example, line _133_ contains a positive sentiment
and three emotional states \- trust, anticipation and joy - whereas two of
them appear eight times. This paragraph (here: line) is about a narcissistic
character, who is talking about himself and people he would like to meet.
Hence, the high number of positive sentiments and emotional states. As a
result, the mean value for line _133_ should be much higher in respect to the
mean value of line _33_ , but the plotted slope will not reach out to that
specific mean value, it would only indicate a higher point in the course of
the slope. The _subset(…)_ command highlights line _133_ before plotting the
slope:

subset(nrc_word_counts, line==133)

    ## # A tibble: 4 x 3
    ##    line sentiment        n
    ##   <int> <chr>        <int>
    ## 1   133 positive         8
    ## 2   133 trust            8
    ## 3   133 anticipation     2
    ## 4   133 joy              1

The intertemporal use of sentiments and emotional states can be plotted by
using the _ggplot(…)_ command, again. The _span_ operator within the
_geom_smooth(…)_ command specifies the smoothness of the plotted slope and can
be adjusted for a more detailed representation:

ggplot(data = nrc_word_counts, mapping = aes(x = line, y = n)) +

geom_smooth(method="loess", formula="y~x", span=0.2) +

xlab("document (line)") +

ylab("sentiment (conditional mean)") +

ggtitle("Intertemporal Use of Sentiments (Conditional Mean)")

The plot inidcates that fewer sentiments and emotional states (Y-axis) are
used at the beginning, the center and the end of this chapter than in between
(represented by lines on the X-axis). Other chapters of the Sherlock Holmes
novel could be analysed accordingly. However, this plot just highlights the
intertemporal use of sentiments and emotional states as conditional means,
regardless of whether they are positive or negative. This aspect will be taken
into account in the next step of this tutorial.

## Analysis - 4: Intertemporal Use of Sentiments (Score)

Another way of analysing the intertemporal use of sentiments and emotional
states is based upon scores. A sentiment score results from the sum of
positive (1) and negative (-1) values in respect to the underlying sentiment.
Since the NRC Word-Emotion Association Lexicon differentiates between positive
and negative sentiments as well as eight emotional states, a lexicon is
required that differentiates only between positive and negative sentiments:
The Bing Sentiment Lexicon, written by distinguished professor Bing Liu from
the department of computer science at the University of Illinois (Chicago), is
suited for that task. By specifying his lexicon within the _inner_join_
command, a new object called _bing_word_counts_ can be generated and inspected
by using the _head(…)_ command:

bing_word_counts <- bind_rows(

text_tidy %>%

inner_join(get_sentiments("bing")) %>%

mutate(method = "Bing et al."))

    ## Joining, by = "word"

head(bing_word_counts)

    ## # A tibble: 6 x 4
    ##    line word      sentiment method
    ##   <int> <chr>     <chr>     <chr>
    ## 1     4 love      positive  Bing et al.
    ## 2     5 cold      negative  Bing et al.
    ## 3     5 precise   positive  Bing et al.
    ## 4     5 admirably positive  Bing et al.
    ## 5     6 balanced  positive  Bing et al.
    ## 6     6 perfect   positive  Bing et al.

Within this object a sentiment is designated to each word and the _ifelse(…)_
command assigns values of _1_ and _-1_ accordingly. A few commands are
required in order to generate a data frame that aggregates these values for
each line. Therefore, after the _cbind(…)_ command has focused the relevant
variables but eliminated their labels, the lines will be labeled _var1_ and
the sum of the values _var2_. After transformation into a data frame by using
the _as.data.frame(…)_ command the _aggregate(…)_ command generates the
necessary object _sentiment_sum_df_ , which can be sorted in respect to the
lines within _var1_ :

sentiment_sum <- ifelse(bing_word_counts$sentiment == "positive", 1, -1)

sentiment_sum_df <- cbind(bing_word_counts$line, sentiment_sum)

colnames(sentiment_sum_df) <- c('var1', 'var2')

sentiment_sum_df <- as.data.frame(sentiment_sum_df)

sentiment_sum_df <- aggregate(sentiment_sum_df$var2,

by=list(line=sentiment_sum_df$var1), FUN=sum)

The _head(…)_ command provides insights into the data frame called
_sentiment_sum_df_. For each line with sentiments the score of these
sentiments (x) is assigned as sum of all positive (1) and negative (-1)
sentiments:

head(sentiment_sum_df)

    ##   line  x
    ## 1    4  1
    ## 2    5  1
    ## 3    6  2
    ## 4    7  1
    ## 5    8  0
    ## 6    9 -1

The related plot is called via the _ggplot(…)_ command as in the previous
steps of this tutorial. Again, the X-axis relates to the lines in the text
document. This time, the Y-axis relates to the previously generated sentiment
score. The result is an intertemporal representation of these scores line by
line:

ggplot(data = sentiment_sum_df, mapping = aes(x = line, y = x)) +

geom_smooth(method="loess", formula="y~x", span=0.2) +

xlab("document (line)") +

ylab("sentiment (score)") +

ggtitle("Intertemporal Use of Sentiments (Score)")

As a result, this Sherlock Holmes chapter seems to have positive sentiments at
the beginning and an increasing sentiment at the end, while there appears to
be a dramaturgical downscaling during the chapter to highlight the tensions in
solving the crime.

## Application Example: Comparing Political Speeches

The last chapter of this tutorial is about comparing two political speeches.
For that purpose, English-language versions of two simultaneous speeches on a
specific issue relating the war in Ukraine were selected. One speech is from
Jens Stoltenberg (Secretary General of NATO) and the other speech - that is
supposed to be an answer to the first one - is from Sergey Viktorovich Lavrov
(Minister of Foreign Affairs, Russian Federation). Basic and advanced
sentiment analyses enable a comparison of the linguistic composition of both
speeches.

All commands can be used as in the previous steps of this tutorial. In order
to systematically summarise the graphical findings, a dashboard with all four
plots at once is generated by using the previously installed package
_gridExtra_ and the _grid.arrange(…)_ command. This requires that all previous
plots be created as new objects, for example: _plot1 <\- ggplot(…)_. The
assignment arrow should always be on the first line of the commands that
contain the _ggplot(…)_ command and each new object should be the first word
in that command. This makes it possible to display all plots from _plot1_ to
_plot4_ in one dashboard:

grid.arrange(plot1, plot2, plot3, plot4)

The corresponding output for both speeches is plotted on the next page:

The dashboard regarding the speech of Jens Stoltenberg:

In comparison, the dashboard regarding the speech of Sergey Viktorovich
Lavrov:

The plots above indicate that Jens Stoltenberg focuses the readiness and the
alliance of NATO states by expressing trust and anticipation as emotional
states (Frequency). There is also a strong use of positive sentiments within
his speech (Distribution). Overall, his speech is characterised by an
estimated use of three sentiments and emotional states per paragraph, while
his speech begins with an estimate of only two sentiments and emotional states
(Conditional Mean). The beginning of his speech can be characterised as
negative, which changes to a positive sentiment score as the speech progresses
(Score). Finally, his speech ends with a neutral sentiment score.

Sergey Viktorovich Lavrov, on the other hand, refers to a threat for the
Russian Federation and his trust in the president of the Russian Federation
(Frequency). The threat is expressed with the emotional states fear and anger,
thus a negative sentiment is inherent. When it comes to the distribution of
sentiments and emotional states, positive and negative sentiments seem to be
equally used, while trust and anticipation counterbalance the emotional states
fear and anger (Distribution). His speech begins with an estimate of four
sentiments and emotional states but he seems to use fewer sentiments and
emotional states at the end of his speech (Conditional Mean). In total, his
sentiment score remains negative with a slight increase at the center of his
speech (Score).

Based on such findings, it seems necessary to inspect the corresponding
paragraphs (here: lines) which include the highlighted words, their sentiments
and emotional states, but also the intertemporal use of them. For example,
each turning point and saddle point of the slopes could be inspected line by
line to validate these findings. Finally, this is primarily a technique in
order to speed up the process of analysing text documents and qualitative
data, rather than a possibility of reaching to final conclusions.

## Sources

  * •

Auguie, Baptise & Anton Antonov (2017): “gridExtra: Miscellaneous Functions
for”Grid” Graphics”. Online: https://cran.r-project.org/package=gridExtra

  * •

Bache, Stefan Milton & Hadley Wickham (2020): “magrittr: A Forward-Pipe
Operator for R”. Online: https://cran.r-project.org/package=magrittr

  * •

De Queiroz, Gabriela; Fay, Colin; Hvitfeldt, Emil; Keyes, Os; Misra, Kanishka;
Mastny, Tim; Erickson, Jeff; Robinson, David & Julia Silge (2022): “tidytext:
Text Mining using”dplyr”, “ggplot2”, and Other Tidy Tools”. Online:
https://cran.r-project.org/package=tidytext

  * •

Hamborg, Felix & Karsten Donnay (2021): “NewsMTSC: A Dataset for
(Multi-)Target-dependent Sentiment Classification in Political News Articles”.
Proceedings of the 16th Conference of the European Chapter of the Association
for Computational Linguistics.

  * •

Klinkhammer, Dennis (2020): “Analysing Social Media Network Data with R: Semi-
Automated Screening of Users, Comments and Communication Patterns”. Online:
https://arxiv.org/abs/2011.13327

  * •

Korkontzelos, Ioannis; Nikfarjam, Azadeh; Shardlow, Matthew; Sarker, Abeed;
Ananiadou, Sophia & Graciela Gonzalez (2016): “Analysis of the effect of
sentiment analysis on extracting adverse drug reactions from tweets and forum
posts”. Journal of Biomedical Informatics (62): 148–158.

  * •

Mohammad, Saif M. (2020): “Sentiment Analysis: Automatically Detecting
Valence, Emotions, and Other Affectual States from Text”. Online:
https://arxiv.org/abs/2005.11882

  * •

Ogneva, Maria (2010): “How Companies Can Use Sentiment Analysis to Improve
Their Business”. Online: https://mashable.com/archive/sentiment-analysis

  * •

Tanoli, Irfan; Pais, Sebastiao; Cordeiro, Joao & Muhammad Luqman Jamil (2022):
“Detection of Radicalisation and Extremism Online: A Survey”. Online:
https://assets.researchsquare.com/files/rs-1185415/v1_covered.pdf

  * •

Tumasjan, Andranik; Sprenger, Timm O.; Sandner, Philipp G. & Isabelle M. Welpe
(2010): “Predicting Elections with Twitter: What 140 Characters Reveal about
Political Sentiment”. Proceedings of the Fourth International AAAI Conference
on Weblogs and Social Media.

  * •

Wickham, Hadley (2019): “stringr: Simple, Consistent Wrappers for Common
String Operations”. Online: https://cran.r-project.org/package=stringr

  * •

Wickham, Hadley; François, Roman & Kirill Müller (2022): “dplyr: A Grammmar of
Data Manipulation”. Online: https://cran.r-project.org/package=dplyr

  * •

Wickham, Hadley; Chang, Winston; Henry, Lionel; Lin Pedersen, Thomas;
Takahashi, Kohske; Wilke, Claus; Woo, Kara; Yutani, Hiroaki & Dewey Dunnington
(2022): “ggplot2: Create Elegant Data Visualisations Using the Grammar of
Graphics”. Online: https://cran.r-project.org/package=ggplot2

  * •

Wood, Ian B.; Varela, Pedro Leal; Bollen, Johan; Rocha, Luis M. & Joana
Gonçalves-Sá (2017): “Human Sexual Cycles are Driven by Culture and Match
Collective Moods”. Online: https://arxiv.org/abs/1707.03959

## Author, Affiliations and Materials on GitHub

Dennis Klinkhammer is Professor for Empirical Research at the FOM University
of Applied Sciences. He advises public as well as governmental organisations
on the application of multivariate statistics and limitations of artificial
intelligence by providing introductions to Python and R:
https://www.statistical-thinking.de

All codes required for the basic and advanced Sentiment Analysis with R can be
accessed on GitHub: https://github.com/statistical-thinking/sentiment-analysis


# Parameter-efficient Modularised Bias Mitigation via AdapterFusion

Deepak Kumar1, Oleg Lesota1, George Zerveas3, Daniel Cohen3,4,
Carsten Eickhoff3,5, Markus Schedl1,2, Navid Rekabsaz1,2
1Johannes Kepler University Linz, Austria
2Linz Institute of Technology, AI Lab
<EMAIL_ADDRESS>
3AI Lab, Brown University, USA<EMAIL_ADDRESS>
4Dataminr, USA<EMAIL_ADDRESS>
5University of Tübingen, Germany<EMAIL_ADDRESS>
###### Abstract

Large pre-trained language models contain societal biases and carry along
these biases to downstream tasks. Current in-processing bias mitigation
approaches (like adversarial training) impose debiasing by updating a model’s
parameters, effectively transferring the model to a new, irreversible
_debiased_ state. In this work, we propose a novel approach to develop stand-
alone debiasing functionalities separate from the model, which can be
integrated into the model on-demand, while keeping the core model untouched.
Drawing from the concept of AdapterFusion in multi-task learning, we introduce
_DAM (Debiasing with Adapter Modules)_ – a debiasing approach to first
encapsulate arbitrary bias mitigation functionalities into separate adapters,
and then add them to the model on-demand in order to deliver fairness
qualities. We conduct a large set of experiments on three classification tasks
with gender, race, and age as protected attributes. Our results show that DAM
improves or maintains the effectiveness of bias mitigation, avoids
catastrophic forgetting in a multi-attribute scenario, and maintains on-par
task performance, while granting parameter-efficiency and easy switching
between the original and debiased models.

##  1 Introduction

Large pre-trained language models (PLM) and their variations fine-tuned on
downstream tasks encode societal biases and stereotypes. A large body of
research shows the existence of these biases Zhao et al. (2019); Sheng et al.
(2019); Rekabsaz et al. (2021b), and discuss their potential harms Blodgett et
al. (2020) in various tasks such as occupation prediction from biographies De-
Arteaga et al. (2019), information retrieval (IR) Rekabsaz et al. (2021a);
Rekabsaz and Schedl (2020), and machine translation Stanovsky et al. (2019). A
common approach to mitigating these biases is to adjust model parameters
according to specific fairness criteria, achieved using optimization methods
such as adversarial training Elazar and Goldberg (2018); Barrett et al.
(2019); Han et al. (2021b); Rekabsaz et al. (2021a).

Figure 1: A transformer block in DAM to debias arbitrarily many protected
attributes in different modules.

The resulting debiased models significantly diminish the effect of protected
attributes (e. g. gender, race, etc.) on model’s predictions. However,
depending on the usage context, characteristics of the input or fairness-
utility trade-off considerations, system designers or end-users might in
practice still prefer to instead use the original (potentially biased) model
for processing some requests. As an example, while a fairness-aware document
retrieval model should tease out gender attributes when processing bias-
sensitive queries Krieg et al. (2022) (like _how to become CEO?_), other
common queries (like _earliest pregnancy symptoms_) may specifically require
gender information to obtain satisfactory retrieval results. Previous studies
also show that one may need to occasionally accept slight performance
degradation in exchange for realizing higher fairness Zerveas et al. (2022);
Biega et al. (2018); Rekabsaz et al. (2021a). We argue that on-demand control
over the strength of bias mitigation is crucial for the broad adoption of bias
mitigation models. Using existing approaches, this would require maintaining
and deploying multiple large parallel models for every protected attribute, in
practice resulting in overly complex and resource-heavy pipelines and
increased latency.

To reduce this untenable burden, we introduce _Debiasing with Adapter Modules
( DAM)_. In our approach, bias mitigation functionalities can be trained in
fully decoupled modules from the building blocks solving the main task (see
Figure 1). These debiasing modules are learned independently, and can be
selectively inserted into the model to deliver debiasing qualities. Our method
draws from the recent success of Adapter Rebuffi et al. (2017); Houlsby et al.
(2019) and AdapterFusion Pfeiffer et al. (2021) networks, originally
introduced in the context of multi-task learning to enable parameter-efficient
training and to avoid catastrophic forgetting. Specifically in multi-task
learning with AdapterFusion, each task is encoded in specific parameter-
efficient adapter networks attached to the transformer blocks Vaswani et al.
(2017) of a PLM, and the knowledge across the tasks is shared through a
FusionAdapter layer with an attention mechanism. DAM extends this principle
idea to bias mitigation by viewing the objective of “removing each protected
attribute” as a stand-alone task, learned by an independent debiasing adapter
module. These debiasing adapters are then integrated into the core model via a
fusion layer to mitigate biases. This modular architecture of DAM enables on-
demand debiasing, where by simply removing the debiasing adapters the model
would return to its original state. The modularity and parameter efficiency of
DAM also enables creating and sharing stand-alone debiasing solutions (in form
of debiasing adapters), which can be applied to the underlying models on-
demand. So, if one uses task adapter’s output in DAM, they get biased
embedding else they can take output of fusion layer for debiased embedding.

We evaluate DAM in extensive experiments covering various bias mitigation
applications in three classification tasks: We study occupation prediction
from biographies De-Arteaga et al. (2019), mention prediction Pardo et al.
(2016), and hate speech detection Founta et al. (2018), involving the
protected attributes of gender, age, and dialect-based race. Among these, the
mention prediction task provides two protected attributes (gender, age),
enabling the study of adapter-based bias mitigation on multiple factors. Our
evaluation results show that DAM (1) consistently improves bias mitigation in
terms of lower leakage of protected attribute in comparison with fully fine-
tuned models, and (2) better avoids catastrophic forgetting Parisi et al.
(2019); Chen et al. (2020) in a multi-attribute debiasing scenario, while (3)
maintaining on-par task performance. Our results show the benefits of DAM in
supporting fairness, and reveal the challenges in the scenarios inherently
bounded by trade-offs.

Our contribution is three-fold:

  * •

Introducing DAM, a novel, parameter-efficient approach for on-demand and
modularized bias mitigation.

  * •

Conducting a large set of experiments on three datasets and three
classification tasks, showing the on-par or better performance of DAM in
comparison with strong baselines.

  * •

Examining the confounding cross-attribute effects in a multi-attribute dataset
and demonstrating the benefits of our modularized bias mitigation approach.

The remainder of the paper is organized as follows: Section 2 reviews the
relevant literature in adapter networks and bias mitigation domains. In
Section 3, we explain the architecture and training strategy of DAM. Section 4
describes the setup of the experiments, whose results are reported and
discussed in Section 5. Finally, Sections 6 and 7, respectively, summarize and
discuss limitations of our work. Our code and trained resources are available
in https://github.com/CPJKU/ModularizedDebiasing.

##  2 Related Work

###  2.1 Adapter Networks

Adapter networks have been introduced in the context of multi-task learning
Rebuffi et al. (2017). Houlsby et al. (2019), and later Stickland and Murray
(2019) attach the adapter module to the transformer blocks of a PLM, and learn
a task by only updating the adapter parameters, while keeping the PLM’s
parameters unchanged. These studies show that the highly parameter-efficient
adapter approach in general performs on par with fine-tuning all parameters of
a BERT model Devlin et al. (2019) for many tasks.

Other studies investigate various characteristics of adapter-based models such
as the parameter efficiency, architectural variations, and transfer learning
across tasks. Rücklé et al. (2021) show the training efficiency of adapter
models in comparison with full model finetuning ($\sim\\!60\%$). Han et al.
(2021a) examine the robustness to initialization seeds and training stability
of the adapter approach. Mahabadi et al. (2021b) propose more parameter-
efficient models by sharing adapter parameters across layers, followed by
Mahabadi et al. (2021a), who introduce even more compact adapter modules. Poth
et al. (2021) investigate the potential knowledge transfer across tasks.
Recently, Pfeiffer et al. (2021) introduce AdapterFusion, where a fusion layer
is defined on top of pre-trained adapter modules, and learns to combine
information of adapters via an attention mechanism. This approach, avoiding
the common pitfalls of catastrophic forgetting Parisi et al. (2019), enables
an effective knowledge transfer across tasks. Our work contributes to this
direction by extending the concept of AdapterFusion to the task of bias
mitigation via supervised interaction between the downstream task and the
protected attribute.

###  2.2 Fairness & Bias Mitigation in NLP

The existence of biases and stereotypes in PLMs has been reported and
discussed in several studies Nadeem et al. (2021); Bhardwaj et al. (2021);
Liang et al. (2021); Kirk et al. (2021); Vig et al. (2020). PLMs may even
exacerbate these biases in downstream tasks as shown e. g. in the context of
IR Rekabsaz and Schedl (2020). To reduce biases, in-processing methods – the
focus of this work – aim at reducing the (cor)relation between the model’s
internal representations and the protected attributes Ganhör et al. (2022).
Debiasing PLMs has been approached for instance by linearly projecting
embeddings to a new space that removes correlations to protected attributes
Kaneko and Bollegala (2021); Bolukbasi et al. (2016). In a similar spirit, Guo
et al. (2022) introduce a distribution alignment loss to force the model’s
outputs to become independent of the protected attribute. Schick et al. (2021)
recently show that the encoded information in models can be exploited to spot
the biases and hence to penalize them.

Adversarial training is a commonly used method to learn representations
invariant to a specific factor of variation. Xie et al. (2017) and later
Madras et al. (2018) introduce adversarial learning to the context of fair
representation learning, where an adversary network learns to predict the
protected attribute, and exploits the gradient of this prediction to remove
the protected information using a gradient reversal layer Ganin and Lempitsky
(2015). Several works further investigate the use of adversarial training for
removing demographic information from neural/PLM-based text classifiers Elazar
and Goldberg (2018); Barrett et al. (2019); Wang et al. (2021). Notably, Han
et al. (2021b) show the benefit of having an ensemble of orthogonal
adversaries. Beyond classification, Rekabsaz et al. (2021a) show that by
applying adversarial training in the context of IR, one can achieve a more
balanced retrieval of documents with respect to the presence of protected
attributes in their contents.

As alternatives to adversarial debiasing, other works approach bias mitigation
by minimizing the approximated upper bound of mutual information between task
and protected attribute Cheng et al. (2020b); Colombo et al. (2021),
contrastive learning for fair representation disentanglement Cheng et al.
(2020a); Zhang et al. (2021), and introducing list-wise fairness
regularization Zerveas et al. (2022). While the mentioned methods are applied
to whole a model, Iterative Nullspace Projection (INLP) Ravfogel et al. (2020)
achieves debiased representations by finding a linear mapping applied to
output embeddings. The linear mapping of the INLP method and similarly the one
of Ravfogel et al. (2022) offer effective bias mitigation methods particularly
designed for the models with a linear decoder, but are not necessarily suited
for the cases with non-linear decoders (e.g., a language generation decoder,
or non-linear attackers).

Regarding adapter-based debiasing and closely related to our work, Lauscher et
al. (2021) recently utilize adapters for debiasing PLMs by training an adapter
which removes a protected attribute using counterfactual augmented data. When
applied to a downstream task, another adapter is added on top of the debiasing
adapter, and trained according to the task’s objective. While shown effective
in practice, in this stacking architecture the adapters in the higher levels
inherently depend on the ones in the lower levels. They can not be learned
stand-alone. In contrast, the fusion-based approach of DAM enables control by
learning modular and independent debiasing adapters, which supports flexible
plug-in/plug-out on demand.

##  3 Bias Mitigation with DAM

We approach the scenario in which the aim is to maximize a task’s core
objective (such as occupation prediction or hate speech prediction), while
reducing the effects of the biases caused by $k$ protected attributes. The
crux of the proposed DAM approach is to first learn a task adapter and $k$
bias removal adapters all independently, and then combine them through a
fusion module. Once a model has been trained with DAM, one can easily switch
between the original model by only using the task adapter, or alternatively
impose debiasing by “plugging in” the learned debiasing adapters as well as
the fusion module. In what follows, we explain the architecture of DAM
followed by the optimization procedure.

###  3.1 Model Architecture

DAM consists of three main parts: task adapter, $k$ debiasing adapters, and
fusion. Following Pfeiffer et al. (2021), and as shown in Figure 1, these
parts extend the architecture of a transformer block by being added to its
last layer.

#### Adapters

Each adapter is defined as a multilayer perceptron (one hidden layer and
$\text{tanh}(x)$ in our experiments), where the hidden layer typically has the
same or a smaller dimension than the input. In DAM, the task adapter and the
$k$ debiasing adapters receive the output vector of the preceding transformer
components, denoted as ${\bm{u}}$, and apply the corresponding transformations
to output the vectors ${\bm{v}}_{t}$, and
${\bm{v}}_{b_{1}},...,{\bm{v}}_{b_{k}}$, respectively.

#### Fusion

The fusion module combines the outputs of the task and debiasing adapters
through the attention mechanism to produce the final output vector. This
module receives
$\left[{\bm{v}}_{t},{\bm{v}}_{b_{1}},...,{\bm{v}}_{b_{k}}\right]$ as keys and
values, and the ${\bm{u}}$ vector as the query of a single-head multiplicative
attention network, and calculates the output as the weighted sum of the value
vectors. These weights are calculated as attention scores and form a
probability distribution over value vectors. Essentially, the fusion module
learns to perform a _linear combination_ of the embedding containing
information for the task with the embeddings of the debiasing adapters, to
provide the final, debiased embedding.

Figure 2: Schematic view of applying DAM to adversarial bias removal.

###  3.2 Learning On-demand Bias Mitigation

DAM introduces a generic approach to bias mitigation and representation
disentanglement, and can, in principle, be integrated with any bias removal
optimization method, provided that the method defines separate losses for the
task and each of the protected attributes. Irrespective of the optimization
method, the training procedure of DAM is the following: first, the task
adapter is trained according to the task’s objective. Then, one debiasing
adapter for each protected attribute is trained using only its own debiasing
objective (see next paragraph), without involving the task loss. Finally, all
adapters’ parameters are kept frozen and the fusion layer is trained using a
combination of the task objective and debiasing objectives. Throughout
training, the parameters of the underlying PLM remain frozen. In what follows,
we describe using DAM together with the adversarial bias mitigation method as
depicted in Figure 2.

#### Adversarial Training

Adversarial bias mitigation aims to make the output embedding of a model
invariant to the protected attribute, by removing information that allows
predicting protected attributes from the latent representation. In this sense,
adversarial training follows a _fairness through blindness_ Barocas et al.
(2019) approach, where the system is made agnostic of the variations in
underlying protected attributes.

To apply adversarial debiasing with DAM (see Figure 2) in the context of text
classification, the model first encodes the input sequence $x$ in the latent
vector ${\bm{z}}$, based on which the corresponding class is predicted using a
task classification head. The loss of this prediction, denoted as
$\mathcal{L}_{t}$, is defined as cross entropy, and its gradient is used to
update the parameters of the task adapter. Similarly, a dedicated
classification head is defined for each protected attribute $b_{i}$, which
also receives ${\bm{z}}$ as input, predicts the corresponding protected
attribute, and calculates the cross entropy loss function
$\mathcal{L}_{b_{i}}$. A common approach to removing the information of
$b_{i}$ encoded in ${\bm{z}}$ is gradient resversal layer (GRL) Ganin and
Lempitsky (2015) added before the debiasing head. GRL multiplies the gradient
of $\mathcal{L}_{b_{i}}$ with a factor of $-\gamma_{i}$, and thereby
simplifies the learning process to a standard gradient-based optimization. In
DAM, this reversed gradient of $\mathcal{L}_{b_{i}}$ is used to learn the
parameters of the debiasing adapter corresponding to the protected attribute
$b_{i}$. Once adapters are trained, the fusion layer is learned jointly
according to all task and debiasing objectives, as formulated below:

$\mathcal{L}_{\text{Fusion}}^{\text{Adv}}=\mathcal{L}_{t}+\sum_{i=1}^{k}{\mathcal{L}_{b_{i}}}$
(1)

Note that, while no weights are used to directly scale the individual loss
functions, the effects of bias mitigation losses on model parameters are
adjusted via their corresponding $\gamma_{i}$ hyperparameters.

##  4 Experiment Setup

We evaluate DAM on three classification tasks and compare the results with a
set of strong baselines.

#### Tasks and Data

We conduct our experiments on three classification tasks. The first task is
biography classification using the BIOS De-Arteaga et al. (2019) dataset. The
objective of the task is to predict a person’s job given their biography,
where the name and any indication of the person’s gender (such as pronouns) in
the biography is omitted. The protected attribute is the person’s gender. The
BIOS dataset contains around 430K data points with 28 occupations as the core
task’s classes, and two protected attribute classes (female/male).

The second task is hate speech detection, a sensitive task, as its use in any
automated system with biases towards any societal group can be extremely
delicate (e. g. in automatic social media moderation). Xia et al. (2020) show
the existence of a strong correlation between dialect-based racial bias and
hate speech detection. In our experiments, we use FDCL18 Founta et al. (2018),
a dataset of tweets annotated with four hate speech labels: _normal_ , _spam_
, _abusive_ , and _hateful_. Following previous works (Sap et al., 2019;
Ravfogel et al., 2020; Zhang et al., 2021), we assign artificial race labels
of _African American_ , and _White American_ to FDCL18 (more details in
Appendix A.1). In this setting, the main task is hate speech classification,
and the protected attribute is the dialect-based racial identity of the tweet
author. Our final dataset contains approximately 62K data points.

Model | # Params
---|---
BERT-Mini | Ft | $11,237,380$
Ft-Debias | $11,568,910$
Adp | $330,500$
Adp-Debias | $432,206$
DAM | $1,186,830$
BERT-Base | Ft | $110,075,908$
Ft-Debias | $113,036,558$
Adp | $14,767,876$
Adp-Debias | $4,448,846$
DAM | $24,806,414$

Table 1: Number of trainable parameters. The adapter-based models contain
significantly fewer trainable parameters in comparison with fully fine-tuned
models.

While the first two tasks contain one protected attribute, in our third
classification task, we aim to explore the effect of more than one biases on
our models. We therefore opt for a subset of the PAN16 dataset Pardo et al.
(2016). This dataset has been designed for the task of mention detection,
namely to predict whether a tweet includes the mention of other users, and the
protected attributes are gender and age of the tweet’s author. The dataset
contains approximately 200K data points with binary task classes (_mention_ ,
_no mention_), as well as two gender labels and five age groups. Further
details on the three datasets are provided in Appendix A.1.

|  | BIOS (gender) | FDCL18 (race)
---|---|---|---
Model |  | Task$\uparrow$ | Attacker$\downarrow$ | Task$\uparrow$ | Attacker$\downarrow$
BERT-Mini | Ft | $82.81^{\clubsuit}_{0.1}$ | $65.10_{0.3}$ | $81.83^{\clubsuit}_{0.4}$ | $92.72_{1.1}$
Ft-Debias | $81.52_{0.2}$ | $57.34^{\clubsuit}_{0.8}$ | $79.15_{0.8}$ | $76.14_{1.8}$
INLP | $71.08_{0.2}$ | $59.83_{0.9}$ | $73.60_{1.6}$ | $71.75^{\clubsuit}_{0.4}$
INLP-NonLin | $54.17_{2.1}$ | $59.90_{1.3}$ | $51.49_{0.2}$ | $92.19_{0.0}$
2-6 | Adp | $81.82_{0.1}$ | $66.59_{0.2}$ | $81.16_{0.2}$ | $93.69_{0.5}$
| Adp-Debias | $81.71_{0.1}$ | $62.97_{0.3}$ | $\textbf{81.46}_{0.1}$ | $81.96_{0.9}$
| DAM | $\textbf{81.97}_{0.1}$ | $\textbf{57.78}_{0.3}$ | $81.06_{0.2}$ | $\textbf{77.52}_{1.4}$
BERT-Base | Ft | $85.07^{\clubsuit}_{0.1}$ | $67.15_{0.4}$ | $84.83^{\clubsuit}_{0.5}$ | $95.87_{0.6}$
Ft-Debias | $84.21_{0.2}$ | $58.35_{1.1}$ | $84.15_{0.8}$ | $72.95^{\clubsuit}_{0.8}$
INLP | $66.25_{0.4}$ | $50.51^{\clubsuit}_{0.3}$ | $71.63_{2.2}$ | $69.87_{0.2}$
INLP-NonLin | $24.44_{3.3}$ | $54.81_{0.9}$ | $42.23_{0.4}$ | $91.67_{0.3}$
2-6 | Adp | $84.84_{0.1}$ | $67.80_{0.2}$ | $82.25_{0.4}$ | $95.89_{0.4}$
| Adp-Debias | $84.40_{0.1}$ | $60.94_{0.4}$ | $83.30_{0.3}$ | $77.97_{2.0}$
| DAM | $\textbf{84.86}_{0.1}$ | $\textbf{58.93}_{0.2}$ | $\textbf{84.76}_{0.4}$ | $\textbf{73.54}_{0.4}$

Table 2: Results on BIOS and FDCL18 datasets for the task classifiers
(accuracy) and protected attributes’ attackers (balanced accuracy), reported
as the mean and standard deviation (in subscript) over three runs. Arrows
indicate the direction of better results. On each evaluation metric and each
PLM, the best results among adapter-based models are shown in bold, and over
all models by symbol $\clubsuit$.

Model |  | Task$\uparrow$ | Attackers$\downarrow$
---|---|---|---
|  |  | Gender | Age
BERT-Mini | Ft | $90.43_{0.5}$ | $64.36_{0.6}$ | $30.77_{0.9}$
$\textsc{Ft-Debias}_{G}$ | $91.89^{\clubsuit}_{0.3}$ | $54.60_{0.4}$ | $25.50_{0.6}$
$\textsc{Ft-Debias}_{A}$ | $91.06_{0.1}$ | $57.27_{1.3}$ | $22.29_{0.6}$
$\textsc{Ft-Debias}_{G+A}$ | $90.74_{0.5}$ | $59.13_{0.5}$ | $24.93_{0.3}$
$\textsc{INLP}_{G}$ | $69.26_{0.1}$ | $60.06_{0.3}$ | $29.21_{0.8}$
$\textsc{INLP-NonLin}_{G}$ | $67.00_{0.1}$ | $60.35_{0.2}$ | $30.03_{0.1}$
$\textsc{INLP}_{A}$ | $66.72_{1.8}$ | $60.62_{0.1}$ | $25.63_{0.2}$
$\textsc{INLP-NonLin}_{A}$ | $62.21_{6.6}$ | $60.39_{0.3}$ | $25.51_{0.3}$
2-5 | Adp | $89.97_{0.2}$ | $68.68_{0.3}$ | $35.97_{6.8}$
| $\textsc{Adp-Debias}_{G}$ | $88.53_{0.2}$ | $56.92_{1.4}$ | $29.58_{1.1}$
| $\textsc{Adp-Debias}_{A}$ | $89.90_{0.1}$ | $62.32_{0.6}$ | $24.75_{0.2}$
| $\textsc{Adp-Debias}_{G+A}$ | $88.87_{0.1}$ | $56.99_{0.4}$ | $26.72_{0.1}$
| $\textsc{DAM}_{G}$ | $\textbf{90.52}_{0.2}$ | $54.35_{0.6}$ | $26.09_{0.6}$
| $\textsc{DAM}_{A}$ | $90.31_{0.2}$ | $61.78_{0.2}$ | $21.97_{0.1}$
| $\textsc{DAM}_{G+A}$ | $90.19_{0.2}$ | $\textbf{53.28}^{\clubsuit}_{0.6}$ | $\textbf{21.91}^{\clubsuit}_{0.3}$
BERT-Base | Ft | $92.17_{0.5}$ | $71.10_{1.8}$ | $42.29_{7.4}$
$\textsc{Ft-Debias}_{G}$ | $92.80_{0.3}$ | $53.80_{0.4}$ | $26.92_{2.3}$
$\textsc{Ft-Debias}_{A}$ | $93.79^{\clubsuit}_{0.1}$ | $56.26_{0.7}$ | $21.89_{0.5}$
$\textsc{Ft-Debias}_{G+A}$ | $92.39_{0.4}$ | $51.96_{0.1}$ | $27.43_{5.4}$
$\textsc{INLP}_{G}$ | $69.37_{0.1}$ | $54.40_{0.1}$ | $25.53_{0.2}$
$\textsc{INLP-NonLin}_{G}$ | $65.42_{0.1}$ | $54.11_{0.1}$ | $26.98_{0.2}$
$\textsc{INLP}_{A}$ | $49.82_{0.8}$ | $54.61_{0.6}$ | $27.55_{0.2}$
$\textsc{INLP-NonLin}_{A}$ | $50.18_{5.3}$ | $54.44_{0.4}$ | $33.66_{0.0}$
2-5 | Adp | $92.34_{0.4}$ | $70.00_{0.3}$ | $42.93_{1.9}$
| $\textsc{Adp-Debias}_{G}$ | $\textbf{93.32}_{0.2}$ | $54.17_{0.2}$ | $29.67_{0.3}$
| $\textsc{Adp-Debias}_{A}$ | $88.88_{0.2}$ | $54.48_{0.6}$ | $21.96_{0.6}$
| $\textsc{Adp-Debias}_{G+A}$ | $89.91_{1.8}$ | $52.92_{0.8}$ | $21.79_{0.2}$
| $\textsc{DAM}_{G}$ | $92.32_{0.3}$ | $52.75_{0.6}$ | $28.12_{0.3}$
| $\textsc{DAM}_{A}$ | $92.06_{0.4}$ | $59.18_{0.4}$ | $\textbf{21.24}^{\clubsuit}_{0.2}$
| $\textsc{DAM}_{G+A}$ | $92.44_{0.4}$ | $\textbf{51.58}^{\clubsuit}_{0.5}$ | $21.95_{0.5}$

Table 3: Results of the PAN16 mention detection task with gender ($G$) and age
($A$) as protected attributes on the task classifiers (accuracy) and protected
attributes’ attackers (balanced accuracy). The mean and standard deviation (in
subscript) over three runs are reported. On each evaluation metric and each
PLM, the best results among adapter-based models are shown in bold, and over
all models by symbol $\clubsuit$.

#### Models and Baselines

We conduct the experiments on the following models: Ft: fully fine-tuning the
PLM on the task. Ft-Debias: fully fine-tuning the PLM on the task as well as
on the corresponding adversarial debiasing objective(s). INLP: using the
implementation and suggested hyperparameter setting by Ravfogel et al. (2020),
in particular with a linear task classifier (Logistic Regressor). INLP-NonLin:
the INLP model with the same setting, but instead of Logistic Regressor, INLP-
NonLin uses a non-linear task classifier: a two-layer feed-forward layer with
a tanh activation. This setting follows the same configuration of all other
baselines/models with respect to the task classifier. Adp: adding one adapter
to the PLM and training it only through the task objective. Adp-Debias: adding
one adapter to the PLM and training it through the task objective as well as
the adversarial debiasing objective(s). Finally, our proposed DAM model, which
consists of one task adapter and separate debiasing adapters for each
protected attribute, all combined with a fusion module. DAM uses the same task
adapter trained by Adp and further learns the debiasing adapters and fusion
layer. We should note that, since through adapter-based training the task
performance of Adp might be different from the one of Ft, a fair comparison of
the core task performance should be between the results of Adp and its
respective DAM model. We emphasize that we are especially interested in
adapters because they preserve the original model and can be disengaged at
will, to trade off debiasing for utility/performance.

As the PLM encoder for all models, we conduct our experiments using two
versions of BERT Devlin et al. (2019) with different sizes, namely BERT-Mini
Turc et al. (2019) and BERT-Base in order to provide a more comprehensive
picture regarding the effect of encoder size and number of involved
parameters. The number of trainable parameters of the models is reported in
Table 1. The complete details of our hyperparameters setting and training
procedure are explained in Appendix A.2 and Appendix A.3, respectively.

#### Evaluation and Attackers

To evaluate the performance of the classifiers on the core task, we use the
accuracy metric. To evaluate bias mitigation, following previous works Elazar
and Goldberg (2018); Barrett et al. (2019), we report the leakage of a
protected attribute in terms of accuracy and balanced accuracy of an
_attacker_. To train an attacker, we freeze the model’s parameters, and train
a new classification head (two-layer feed-forward layer with a
$\text{tanh}(x)$ activation) to predict the protected attribute from the
${\bm{z}}$ vector. We train an ensemble of 5 attackers with the same
configuration and training procedure, and report the results of the best
performing one. We report the performance of the attacker in terms of accuracy
and also balanced/macro accuracy (average of per-class accuracy scores).
Balanced accuracy has the benefit of better reflecting the performance of the
methods when considering sub- and minority groups. We believe this is
particularly important in this setting, since the datasets are unbalanced over
both task and protected labels (see Appendix A.1). To account for possible
variabilities in training, we repeat every experiment three times and report
the mean as well as standard deviation of the achieved results.

##  5 Results and Discussion

In this section, we first discuss the results on the datasets with one
protected attribute (BIOS and FDCL18), followed by investigating the effect of
DAM on PAN16 with two protected attributes.

#### Single-attribute bias mitigation

The results on BIOS and FDCL18 datasets are reported in Table 4. We start with
the results of BERT-Mini on both datasets. As shown, consistent with previous
studies the task performance of Adp is slightly worse then the full model
fine-tuning Ft. In both Ft and Adp, we observe that the adversarial debiasing
(Ft-Debias and Adp-Debias respectively) significantly reduces leakage in terms
of Acc and BAcc of attackers, while task performance remains on par with the
corresponding baseline models. Overall, adversarial methods outperform
respective INLP models, particularly in respect to maintaining task
performance during bias mitigation.

Compared to Adp and Adp-Debias, we observe that DAM improves Adp-Debias in
terms of bias mitigation, while maintaining or even slightly improving
performance on the task. The difference between DAM and Adp-Debias indicates
the benefit of learning separate adapters for performing the task and bias
mitigation, and then combining them. Also comparing DAM with Ft-Debias,
overall DAM performs on par in terms of task performance and leakage, while
DAM provides the additional benefits of selectively engaging bias mitigation
and significantly fewer trainable parameters.

Comparing the results between the core PLMs, we see that the above
observations for BERT-Mini also hold true for BERT-Base on the BIOS dataset
and the FDCL18 dataset.

#### Multi/Two-attribute bias mitigation

Table 4 reports the results on PAN16 with two protected attributes, gender
($G$) and age ($A$). To observe the effect of single- versus multiple-
attribute debiasing, we train the debiasing models (Ft-Debias, Adp-Debias, and
DAM) in three variations denoted with subscripts $G$, $A$, and
$G\\!\\!+\\!\\!A$, indicating the debiasing optimization on only gender, only
age, and both gender and age, respectively.111We run INLP and INLP-NonLin on
the two variations of $G$ and $A$, but we are not aware of a principled way
for this method to simultaneously debias more than one protected attribute.

Observing similar patterns to the previous experiments, $\textsc{DAM}_{G}$,
$\textsc{DAM}_{A}$, and $\textsc{DAM}_{G+A}$ provide a significant improvement
in bias mitigation (lower leakage) in comparison with their respective
debiasing baseline variations. Also similar to single-attribute datasets, Adp
performs slightly worse than Ft in respect to task performance, but DAM models
remain on par with Adp or slightly improve it. We observe these patterns
consistently on both BERT-Mini and BERT-Base. Overall, DAM provides equal or
better bias mitigation in comparison with Ft-Debias and INLP, while offering
on-demand debiasing and parameter efficiency.

Particularly with respect to multi-attribute bias mitigation, the results show
the benefits of $\textsc{DAM}_{A+G}$ in separately learning multiple debiasing
adapters and combining them, when compared with $\textsc{Ft-Debias}_{G+A}$,
which simultaneously learns both debiasing factors. In particular, looking at
the attacker’s BAcc, while $\textsc{Ft-Debias}_{G+A}$ is not able to debias
the age attribute (in comparison to Ft), our $\textsc{DAM}_{G+A}$ provides
strongly improved bias mitigation for age (and also gender), with similar or
better results to only optimizing for gender or age ($\textsc{DAM}_{G}$ and
$\textsc{DAM}_{A}$, respectively). This highlights the effectiveness of DAM in
preventing catastrophic forgetting of debiasing functionalities in multi-
attribute scenarios.222Our experiments using DAM also indicate a possible
correlation between Gender and Age, as debiasing Gender ($\textsc{DAM}_{G}$)
leads to slight reduction in Age leakage, while similarly, debiasing Age
($\textsc{DAM}_{A}$) results in a slight removal of gender information. We
further provide an investigation of the attention distributions of DAM’s
fusion module in Appendix B.

##  6 Conclusion

We propose a novel bias mitigation approach which enables flexible switching
between the original and debiased state of a model. Our proposed DAM method
extends the idea of multi-task learning using AdapterFusion to bias
mitigation, by first learning the main task and the debiasing objectives as
separate adapters, and then leveraging the attention-based fusion approach to
merge these adapters and deliver debiased results. Our experiments on three
classification tasks show that, beside flexible switching, DAM improves bias
mitigation, provides on par classification performance to vanilla adversarial
debiasing, and addresses the issue of catastrophic forgetting in multi-
attribute bias mitigation.

##  7 Limitations

In the experiments, gender is considered as a binary construct due to
practical constraints. In particular, in BIOS and PAN16 the gender is provided
only in the form of male and female. We are however fully aware that a gender
binary model is not representative of all individuals; yet working with in-
the-wild data (as in our case) entails an unavoidable caveat, derived from the
still predominant belief that human beings can be sorted into two discrete
categories Hyde et al. (2019). However, the proposed method can still be
defined for generic non-binary settings and can be applied to any sensitive
attribute with more than two categories, as exemplified by our consideration
of age classes.

We should also highlight two limitations of this study in respect to the model
and optimization. First, adversarial approaches in general (including our
proposed approach) aims to reduce the correlations in the model to the
protected attribute based on the observed data. This approach, like other
data-oriented bias mitigation methods, might lack effective generalization,
particularly when the model is evaluated on other domains or out-of-
distribution data. The second limitation regarding the discussed bias
mitigation methods based on adversarial training (also involving our proposed
approach) is that, while the method aims to make the model prediction agnostic
to protected attributes, it does not directly account for a balanced treatment
of subpopulations in regards to the utility metrics and quality of the
received service.

##  8 Acknowledgment

This work received financial support by the Austrian Science Fund (FWF):
P33526 and DFH-23; and by the State of Upper Austria and the Federal Ministry
of Education, Science, and Research, through grants LIT-2020-9-SEE-113 and
LIT-2021-YOU-215.

## References

  * Barocas et al. (2019) Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2019.  _Fairness and Machine Learning_.  fairmlbook.org.  http://www.fairmlbook.org.
  * Barrett et al. (2019) Maria Barrett, Yova Kementchedjhieva, Yanai Elazar, Desmond Elliott, and Anders Søgaard. 2019.  Adversarial removal of demographic attributes revisited.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6331–6336.
  * Bhardwaj et al. (2021) Rishabh Bhardwaj, Navonil Majumder, and Soujanya Poria. 2021.  Investigating gender bias in BERT.  _Cognitive Computation_ , 13(4):1008–1018.
  * Biega et al. (2018) Asia J Biega, Krishna P Gummadi, and Gerhard Weikum. 2018.  Equity of attention: Amortizing individual fairness in rankings.  In _The 41st international ACM SIGIR Conference on Research & Development in Information Retrieval_, pages 405–414.
  * Blodgett et al. (2020) Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020.  Language (technology) is power: A critical survey of “bias” in nlp.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5454–5476.
  * Blodgett et al. (2016) Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016.  Demographic dialectal variation in social media: A case study of African-American English.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1119–1130, Austin, Texas. Association for Computational Linguistics.
  * Bolukbasi et al. (2016) Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  _Advances in Neural Information Processing Systems_.
  * Chen et al. (2020) Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, and Xiangzhan Yu. 2020\.  Recall and learn: Fine-tuning deep pretrained language models with less forgetting.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7870–7881, Online. Association for Computational Linguistics.
  * Cheng et al. (2020a) Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2020a.  Fairfil: Contrastive neural debiasing method for pretrained text encoders.  In _International Conference on Learning Representations_.
  * Cheng et al. (2020b) Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, and Lawrence Carin. 2020b.  Improving disentangled text representation learning with information-theoretic guidance.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7530–7541, Online. Association for Computational Linguistics.
  * Colombo et al. (2021) Pierre Colombo, Pablo Piantanida, and Chloé Clavel. 2021.  A novel estimator of mutual information for learning to disentangle textual representations.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6539–6550.
  * De-Arteaga et al. (2019) Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019.  Bias in bios: A case study of semantic representation bias in a high-stakes setting.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , pages 120–128.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , volume 1, pages 4171–4186. Association for Computational Linguistics.
  * Elazar and Goldberg (2018) Yanai Elazar and Yoav Goldberg. 2018.  Adversarial removal of demographic attributes from text data.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_ , pages 11–21.
  * Founta et al. (2018) Antigoni Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018.  Large scale crowdsourcing and characterization of twitter abusive behavior.  In _Twelfth International AAAI Conference on Web and Social Media_.
  * Ganhör et al. (2022) Christian Ganhör, David Penz, Navid Rekabsaz, Oleg Lesota, and Markus Schedl. 2022.  Unlearning protected user attributes in recommendations with adversarial training.  In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , SIGIR ’22, page 2142–2147, New York, NY, USA. Association for Computing Machinery.
  * Ganin and Lempitsky (2015) Yaroslav Ganin and Victor Lempitsky. 2015.  Unsupervised domain adaptation by backpropagation.  In _International conference on machine learning_ , pages 1180–1189. PMLR.
  * Guo et al. (2022) Yue Guo, Yi Yang, and Ahmed Abbasi. 2022.  Auto-debias: Debiasing masked language models with automated biased prompts.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1012–1023.
  * Han et al. (2021a) Wenjuan Han, Bo Pang, and Ying Nian Wu. 2021a.  Robust transfer learning with pretrained language models through adapters.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 854–861.
  * Han et al. (2021b) Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021b.  Diverse adversaries for mitigating bias in training.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2760–2765, Online. Association for Computational Linguistics.
  * Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019.  Parameter-efficient transfer learning for NLP.  In _International Conference on Machine Learning_ , volume 97, pages 2790–2799. Proceedings of Machine Learning Research.
  * Hyde et al. (2019) Janet Shibley Hyde, Rebecca S Bigler, Daphna Joel, Charlotte Chucky Tate, and Sari M van Anders. 2019.  The future of sex and gender in psychology: Five challenges to the gender binary.  _American Psychologist_ , 74(2):171–193.
  * Kaneko and Bollegala (2021) Masahiro Kaneko and Danushka Bollegala. 2021.  Debiasing pre-trained contextualised embeddings.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1256–1266.
  * Kirk et al. (2021) Hannah Rose Kirk, Filippo Volpin, Haider Iqbal, Elias Benussi, Frederic Dreyer, Aleksandar Shtedritski, Yuki Asano, et al. 2021.  Bias out-of-the-box: An empirical analysis of intersectional occupational biases in popular generative language models.  _Advances in Neural Information Processing Systems_ , 34.
  * Krieg et al. (2022) Klara Krieg, Emilia Parada-Cabaleiro, Gertraud Medicus, Oleg Lesota, Markus Schedl, and Navid Rekabsaz. 2022.  Grep-biasir: A dataset for investigating gender representation-bias in information retrieval results.  _arXiv preprint arXiv:2201.07754_.
  * Lauscher et al. (2021) Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021.  Sustainable modular debiasing of language models.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 4782–4797, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Liang et al. (2021) Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021\.  Towards understanding and mitigating social biases in language models.  In _International Conference on Machine Learning_ , pages 6565–6576. PMLR.
  * Madras et al. (2018) David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 2018.  Learning adversarially fair and transferable representations.  In _Proceedings of the International Conference on Machine Learning_ , pages 3384–3393. PMLR.
  * Mahabadi et al. (2021a) Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. 2021a.  Compacter: Efficient low-rank hypercomplex adapter layers.  In _Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual_ , pages 1022–1035.
  * Mahabadi et al. (2021b) Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, and James Henderson. 2021b.  Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , pages 565–576. Association for Computational Linguistics.
  * Nadeem et al. (2021) Moin Nadeem, Anna Bethke, and Siva Reddy. 2021.  StereoSet: Measuring stereotypical bias in pretrained language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5356–5371, Online. Association for Computational Linguistics.
  * Pardo et al. (2016) Francisco Manuel Rangel Pardo, Paolo Rosso, Ben Verhoeven, Walter Daelemans, Martin Potthast, and Benno Stein. 2016.  Overview of the 4th author profiling task at pan 2016: Cross-genre evaluations.  In _CLEF_.
  * Parisi et al. (2019) German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. 2019.  Continual lifelong learning with neural networks: A review.  _Neural Networks_ , 113:54–71.
  * Pfeiffer et al. (2021) Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021.  Adapterfusion: Non-destructive task composition for transfer learning.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 487–503.
  * Poth et al. (2021) Clifton Poth, Jonas Pfeiffer, Andreas Rücklé, and Iryna Gurevych. 2021.  What to pre-train on? efficient intermediate task selection.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10585–10605.
  * Ravfogel et al. (2020) Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020\.  Null it out: Guarding protected attributes by iterative nullspace projection.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7237–7256.
  * Ravfogel et al. (2022) Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan D Cotterell. 2022.  Linear adversarial concept erasure.  In _International Conference on Machine Learning_ , pages 18400–18421. PMLR.
  * Rebuffi et al. (2017) Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017.  Learning multiple visual domains with residual adapters.  In _Advances in Neural Information Processing Systems (NeurIPS)_ , volume 30.
  * Rekabsaz et al. (2021a) Navid Rekabsaz, Simone Kopeinik, and Markus Schedl. 2021a.  Societal biases in retrieved contents: Measurement framework and adversarial mitigation of BERT rankers.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 306–316.
  * Rekabsaz and Schedl (2020) Navid Rekabsaz and Markus Schedl. 2020.  Do neural ranking models intensify gender bias?  In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 2065–2068.
  * Rekabsaz et al. (2021b) Navid Rekabsaz, Robert West, James Henderson, and Allan Hanbury. 2021b.  Measuring societal biases in text corpora via first-order co-occurrence.  _Proceedings of the International AAAI Conference on Web and Social Media (ICWSM)_.
  * Rücklé et al. (2021) Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2021.  AdapterDrop: On the efficiency of adapters in transformers.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7930–7946.
  * Sap et al. (2019) Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019.  The risk of racial bias in hate speech detection.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1668–1678, Florence, Italy. Association for Computational Linguistics.
  * Schick et al. (2021) Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021.  Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP.  _Transactions of the Association for Computational Linguistics_ , 9:1408–1424.
  * Sheng et al. (2019) Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019.  The woman worked as a babysitter: On biases in language generation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3398–3403.
  * Stanovsky et al. (2019) Gabriel Stanovsky, Noah A Smith, and Luke Zettlemoyer. 2019.  Evaluating gender bias in machine translation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1679–1684.
  * Stickland and Murray (2019) Asa Cooper Stickland and Iain Murray. 2019.  BERT and PALs: Projected attention layers for efficient adaptation in multi-task learning.  In _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 5986–5995. PMLR.
  * Turc et al. (2019) Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Well-read students learn better: On the importance of pre-training compact models.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30, pages 6000––6010. Curran Associates, Inc.
  * Vig et al. (2020) Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020.  Investigating gender bias in language models using causal mediation analysis.  _Advances in Neural Information Processing Systems_ , 33:12388–12401.
  * Wang et al. (2021) Liwen Wang, Yuanmeng Yan, Keqing He, Yanan Wu, and Weiran Xu. 2021.  Dynamically disentangling social bias from task-oriented representations with adversarial attack.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3740–3750.
  * Xia et al. (2020) Mengzhou Xia, Anjalie Field, and Yulia Tsvetkov. 2020.  Demoting racial bias in hate speech detection.  In _Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media_ , pages 7–14, Online. Association for Computational Linguistics.
  * Xie et al. (2017) Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. 2017.  Controllable invariance through adversarial feature learning.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pages 585–596.
  * Zerveas et al. (2022) George Zerveas, Navid Rekabsaz, Daniel Cohen, and Carsten Eickhoff. 2022.  Mitigating bias in search results through contextual document reranking and neutrality regularization.  In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , SIGIR ’22, page 2532–2538, New York, NY, USA. Association for Computing Machinery.
  * Zhang et al. (2021) Xiongyi Zhang, Jan-Willem van de Meent, and Byron C Wallace. 2021.  Disentangling representations of text by masking transformers.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 778–791.
  * Zhao et al. (2019) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019.  Gender bias in contextualized word embeddings.  In _Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 629–634.

##  Appendix A Experiment Settings – Additional Details

###  A.1 Datasets

In FDCL18 dataset, we use the TwitterAAE model Blodgett et al. (2016) to
assign racial dialect classes. The TwiiterAAE model predicts four racial
classes, _African American_ , _White American_ , _Hispanic_ , and _Others_. We
labeled a tweet as _African American_ or _White American_ if the prediction
score was greater than $0.5$. For PAN16 dataset, following Sap et al. (2019)
we balanced the task labels and sampled 200K data. The age groups of this
dataset are 18-24, 25-34, 35-49, 50-64, and 65+. The proportions of data items
regarding the labels of the task and protected attributes in the three dataset
are as follows:

  * •

BIOS dataset: Task (dentist: 0.03, professor: 0.27, teacher: 0.04,
psychologist: 0.04, nurse: 0.07, poet: 0.01, photographer: 0.06, journalist:
0.04, filmmaker: 0.02, physician: 0.08, composer: 0.2, attorney: 0.08, model:
0.03, painter: 0.02, accountant: 0.01, pastor: 0.01, comedian: 0.01, surgeon:
0.05, architect: 0.03, paralegal: 0.01, dj: 0.01, chiropractor: 0.01, software
engineer: 0.02, dietitian: 0.02, rapper: 0.01, personal trainer: 0.003, yoga
teacher: 0.01, interior designer: 0.01); Gender (Female: 0.5, Male: 0.5)

  * •

FDCL18 dataset: Task (normal: 0.73, spam: 0.12, abusive: 0.12, hateful: 0.04);
Race (White: 0.5, AA: 0.5)

  * •

PAN16 dataset: Task (Mention: 0.5, Not Mention: 0.5); Gender (male: 0.54,
female: 0.46); Age ( 3: 0.18, 1: 0.34, 2: 0.40, 0: 0.07, 4: 0.01)

###  A.2 Hyperparameter setting

Across experiments, we keep specific hyperparameters consistent. Batch size is
64, learning rate is 2e-5 (except for training task and debiasing adapter as
explained below), training epochs is 20, dropout probability is 0.3, and
adversarial debiasing coefficient is 1 for all models (when applicable).

For task adapter and debiasing adapters, we tune the learning rate and the
hidden layer dimension of adapter. We conduct brute search over the learning
rate values of [1e-5, 1e-4,1e-3,1e-2], and hidden layer dimension with a
division factor of [1,1/2,1/4,1/8,1/16]. For INLP we use 10 iterations and 10
classifiers to learn null space while for INLP-NonLin we same setting (300
iterations and 300 classifiers) as in Ravfogel et al. (2020).

###  A.3 Training procedure

We randomly split the dataset into train, validation, and test set with the
proportions 63:12:15 for BIOS, 63:12:15 for FDCL18, and 80:5:15 on PAN16. We
use the validation set for hyperparameter tuning, and the best result on the
validation set is evaluated on test set for the final results. The validation
and test sets in all datasets follow the same distribution as the whole
dataset. To address the unbalancedness of the dataset and the potential
problems in adversarial learning, we apply upsampling only on the _training
sets_ of BIOS and FDCL18 datasets, to balance the protected attribute labels
within each task label. For instance, genders are balanced in the dentist
class by repeating the data items of the minority subgroup.

##  Appendix B Fusion Attention Analysis

We investigate the attention distribution of the fusion network and observe
the weights it gives to the adapters. Figures 3(a), 3(b), and 4 depict the
attention scores of each adapter averaged over all fusion layer in DAM for
BIOS, and FDCL18, and PAN16 datasets, respectively. To avoid confusion in
visualization, we only used 4% of data points randomly sampled from test set.
As shown, the task adapter has weights close to 1, while debiasing adapters
are assigned attention scores slightly higher than 0. The top three outliers
in BIOS with the highest attentions on the debiasing adapter are reported in
Table 4.

| Text
---|---
1 |  passionately promotes healthy dietary and lifestyle choices to prevent disease and achieve optimal health
2 |  practices include clayton heights family dental street panorama family dental avenue and surrey family dental avenue
3 |  primary research interests include collective security and global health in an international relations and international political economy perspective
Table 4:

(a)

(b)

Figure 3: Attention distribution of a set of sampled data points in the fusion
module of DAM. Figure 4: Attention distribution of a set of sampled data
points in the fusion module of DAM in PAN16 dataset.