# Data-Efficient Strategies for Expanding Hate Speech Detection into Under-
Resourced Languages

Paul Röttger University of Oxford  Debora Nozza Bocconi University  Federico
Bianchi Stanford University  Dirk Hovy Bocconi University

###### Abstract

Hate speech is a global phenomenon, but most hate speech datasets so far focus
on English-language content. This hinders the development of more effective
hate speech detection models in hundreds of languages spoken by billions
across the world. More data is needed, but annotating hateful content is
expensive, time-consuming and potentially harmful to annotators. To mitigate
these issues, we explore data-efficient strategies for expanding hate speech
detection into under-resourced languages. In a series of experiments with
mono- and multilingual models across five non-English languages, we find that
1) a small amount of target-language fine-tuning data is needed to achieve
strong performance, 2) the benefits of using more such data decrease
exponentially, and 3) initial fine-tuning on readily-available English data
can partially substitute target-language data and improve model
generalisability. Based on these findings, we formulate actionable
recommendations for hate speech detection in low-resource language settings.

Content warning: This article contains illustrative examples of hateful
language.

##  1 Introduction

Hate speech is a global phenomenon, but most hate speech datasets so far focus
on English-language content (Vidgen and Derczynski, 2020; Poletto et al.,
2021). This hinders the development of effective models for detecting hate
speech in other languages. As a consequence, billions of non-English speakers
across the world are less protected against online hate, and even giant social
media platforms have clear language gaps in their content moderation systems
(Simonite, 2021; Marinescu, 2021).

Figure 1: Overview of our experimental setup. We use ISO 639-1 codes to denote
the different languages. MHC is Multilingual HateCheck (Röttger et al., 2022).

Zero-shot cross-lingual transfer, where large multilingual language models are
fine-tuned on one source language and then applied to another target language,
may appear like a potential solution to the issue of language-specific
resource scarcity. However, while this approach performs well on some tasks
(Conneau et al., 2020; Barbieri et al., 2022), it fails on many others
(Lauscher et al., 2020; Hu et al., 2020). For hate speech detection in
particular, cross-lingual performance in zero-shot settings is lacking
(Stappen et al., 2020; Leite et al., 2020). For example, zero-shot cross-
lingual transfer cannot account for language-specific taboo expressions that
play a key role in classification (Nozza, 2021). Conversely, hate speech
detection models trained or fine-tuned directly on the target language, i.e.
in few- and many-shot settings, are consistently found to perform best (Aluru
et al., 2020; Pelicon et al., 2021).

So, how do we build hate speech detection models for hundreds more languages?
We need at least some labelled data in the target language to make models
effective, but data annotation is difficult, time-consuming and expensive. It
requires resources that are often very limited for non-English languages.
Annotating hateful content in particular also risks exposing annotators to
harm in the process (Vidgen et al., 2019; Derczynski et al., 2022).

In this article, we explore strategies for hate speech detection in under-
resourced languages that make efficient use of labelled data, to build
effective models while also minimising annotation cost and risk of harm to
annotators. For this purpose, we conduct a series of experiments using mono-
and multilingual models fine-tuned on differently-sized random samples of
labelled hate speech data in English as well as Arabic, Spanish, Hindi,
Portuguese and Italian. Our key findings are:

  1. 1.

A small amount of labelled target-language data is needed to achieve strong
performance on held-out test sets.

  2. 2.

The benefits of using more such data decrease exponentially.

  3. 3.

Initial fine-tuning on readily-available English data can partially substitute
target-language data and improve model generalisability.

Based on these findings, we formulate and discuss five recommendations for
expanding hate speech detection into under-resourced languages:

  1. 1.

Collect and label target-language data.

  2. 2.

Start by labelling a small set, then iterate.

  3. 3.

Use diverse data collection methods to increase marginal benefits of
annotation.

  4. 4.

Use multilingual models for unlocking readily-available data in high-resource
languages for initial fine-tuning.

  5. 5.

Evaluate out-of-domain performance to reveal potential weaknesses in
generalisability.

With these recommendations, we hope to facilitate the development of new hate
speech detection models for yet-unserved languages.111We make all data and
code to reproduce our experiments available on GitHub.

##### Definition of Hate Speech

Definitions of hate speech vary across cultural and legal settings. Following
Röttger et al. (2021), we define hate speech as abuse that is targeted at a
protected group or at its members for being a part of that group. Protected
groups are based on characteristics such as gender identity, race or religion,
which broadly reflects Western legal consensus, particularly the US 1964 Civil
Rights Act, the UK’s 2010 Equality Act and the EU’s Charter of Fundamental
Rights. Based on these definitions, we approach hate speech detection as the
binary classification of content as either hateful or non-hateful.

##  2 Experiments

All our experiments follow the setup described in Figure 1. We start by
loading a pre-trained mono- or multilingual transformer model for sequence
classification. For multilingual models, there is an optional first phase of
fine-tuning on English data. This is to simulate using readily-available data
from a high-resource language that is not the target language. For all models,
there is then a second phase of fine-tuning on differently-sized random
samples of data in the target language. This is to simulate using scarce data
from an under-resourced language. Finally, all models are evaluated on the
held-out test set corresponding to the target-language dataset they were fine-
tuned on as well as the target-language test suite from Multilingual HateCheck
(Röttger et al., 2022).

###  2.1 Data

For all our experiments, we use hate speech datasets from hatespeechdata.com,
which was first introduced by Vidgen and Derczynski (2020) and is now the
largest public repository of datasets annotated for hate, abuse and offensive
language. At the time of our review in May 2022, the site listed 53 English
datasets as well as 57 datasets in 24 other languages. From these datasets, we
select those for our experiments that a) contain explicit labels for hate, and
b) use a definition of hate for data annotation that aligns with our own (§1).

####  2.1.1 Fine-Tuning 1: English

For the optional first phase of fine-tuning in English, we use one of three
English datasets. Dyn21_En by Vidgen et al. (2021) contains 41,255 entries, of
which 53.9% are labelled as hateful. The entries were hand-crafted by
annotators to be challenging to hate speech detection models, using the
Dynabench platform (Kiela et al., 2021). Fou18_En by Founta et al. (2018)
contains 99,996 tweets, of which 4.97% are labelled as hateful. Ken20_En by
Kennedy et al. (2020) contains 39,565 comments from Youtube, Twitter and
Reddit, of which 29.31% are labelled as hateful.

From each of these three English datasets, we sample 20,000 entries for the
optional first phase of fine-tuning, plus another 500 entries for development
and 2,000 for testing. To align the proportion of hate across English
datasets, we use random sampling for Dyn21_En, while for Ken20_En and Fou18_En
we retain all hateful entries and then sample from non-hateful entries, so
that the proportion of hate in the two datasets increases to 50.0% and 22.0%,
respectively.

####  2.1.2 Fine-Tuning 2: Target Language

For fine-tuning in the target language, we use one of five datasets in five
different target languages. Bas19_Es compiled by Basile et al. (2019) for
SemEval 2019 contains 4,950 Spanish tweets, of which 41.5% are labelled as
hateful. For19_Pt by Fortuna et al. (2019) contains 5,670 Portuguese tweets,
of which 31.5% are labelled as hateful. Has21_Hi compiled by Modha et al.
(2021) for HASOC 2021 contains 4,594 Hindi tweets, of which 12.3% are labelled
as hateful. Ous19_Ar by Ousidhoum et al. (2019) contains 3,353 Arabic tweets,
of which 22.5% are labelled as hateful. San20_It compiled by Sanguinetti et
al. (2020) for EvalIta 2020 contains 8,100 Italian tweets, of which 41.8% are
labelled as hateful.

From each of these five target-language datasets, we randomly sample
differently-sized subsets for target-language fine-tuning. Like in English, we
set aside 500 entries for development and 2,000 for testing.222Due to limited
dataset size, we only set aside 300 dev and 1,000 test entries for Ous19_Ar
(n=3,353). From the remaining data, we sample subsets in 12 different sizes –
10, 20, 30, 40, 50, 100, 200, 300, 400, 500, 1,000 and 2,000 entries – so that
we are able to evaluate the effects of using more or less labelled data within
and across different orders of magnitude.333There is at least one hateful
entry in every sample. Zhao et al. (2021) show that there can be large
sampling effects when fine-tuning on small amounts of data. To mitigate this
issue, we use 10 different random seeds for each sample size, so that in total
we have 120 different samples in each language, and 600 samples across the
five non-English languages.

###  2.2 Models

##### Multilingual Models

We fine-tune and evaluate XLM-T (Barbieri et al., 2022), an XLM-R model
(Conneau et al., 2020) pre-trained on an additional 198 million Twitter posts
in over 30 languages. XLM-R is a widely-used architecture for multilingual
language modelling, which has been shown to achieve near state-of-the-art
performance on multilingual hate speech detection (Banerjee et al., 2021;
Modha et al., 2021). We chose XLM-T because it strongly outperformed XLM-R
across our target language test sets in initial experiments.

##### Monolingual Models

For each of the five target languages, we also fine-tune and evaluate a
monolingual transformer model from HuggingFace. For Spanish, we use RoBERTuito
(Pérez et al., 2021). For Portuguese, we use BERTimbau (Souza et al., 2020).
For Hindi, we use Hindi BERT. For Arabic, we use AraBERT v2 (Antoun et al.,
2020). For Italian, we use UmBERTo. Details on model training can be found in
Appendix A.

##### Model Notation

We denote all models by an additive code. The first part is either M for a
monolingual model or X for XLM-T. For XLM-T, the second part of the code is
DEn, FEn or KEn, for models fine-tuned on 20,000 entries from Dyn21_En,
Fou18_En or Ken20_En. For all models, the final part of the code is Es, Pt,
Hi, Ar or It, corresponding to the target language that the model was
finetuned on. For example, M+It denotes the monolingual Italian model,
UmBERTo, fine-tuned on San20_It, and X+KEn+Ar denotes an XLM-T model fine-
tuned first on 20,000 English entries from Ken20_En and then on Ous19_Ar.

###  2.3 Evaluation Setup

##### Held-Out Test Sets + MHC

We test all models on the held-out test sets corresponding to their target-
language fine-tuning data, to evaluate their in-domain performance (§2.4). For
example, we test X+KEn+It, which was fine-tuned on San20_It data, on the
San20_It test set. Additionally, we test all models on the matching target-
language test suite from Multilingual HateCheck (MHC). MHC is a collection of
around 3,000 test cases for different kinds of hate as well as challenging
non-hate in each of ten different languages (Röttger et al., 2022). We use MHC
to evaluate out-of-domain generalisability (§2.5).

##### Evaluation Metrics

We use macro F1 to evaluate model performance because most of our test sets as
well as MHC are imbalanced. To give context for interpreting performance, we
show baseline model results in all figures: macro F1 for always predicting the
hateful class ("always hate"), for never predicting the hateful class ("never
hate") and for predicting both classes with equal probability ("50/50"). We
also show bootstrapped 95% confidence intervals around the average macro F1,
which is calculated across the 10 random seeds for each sample size. These
confidence intervals are expected to be wider for models fine-tuned on less
data because of larger sampling effects.

Figure 2:  Macro F1 on target-language held-out test sets across models fine-
tuned on up to N=2,000 target-language entries. Model notation as described in
§2.2. Confidence intervals and model baselines as described in §2.4. We
provide larger versions of all five graphs in Appendix B.

| Bas19_Es | For19_Pt | Has21_Hi | Ous19_Ar | San20_It
---|---|---|---|---|---
N | 20 | 200 | 2,000 | 20 | 200 | 2,000 | 20 | 200 | 2,000 | 20 | 200 | 2,000 | 20 | 200 | 2,000
M | 0.50 | 0.72 | 0.84 | 0.46 | 0.67 | 0.73 | 0.47 | 0.55 | 0.60 | 0.48 | 0.66 | 0.72 | 0.40 | 0.73 | 0.79
X | 0.48 | 0.70 | 0.81 | 0.42 | 0.67 | 0.73 | 0.47 | 0.47 | 0.56 | 0.43 | 0.69 | 0.70 | 0.40 | 0.70 | 0.78
X+DEn | 0.66 | 0.75 | 0.82 | 0.63 | 0.70 | 0.72 | 0.52 | 0.56 | 0.60 | 0.52 | 0.66 | 0.70 | 0.63 | 0.73 | 0.77
X+FEn | 0.59 | 0.68 | 0.80 | 0.66 | 0.71 | 0.73 | 0.55 | 0.56 | 0.59 | 0.61 | 0.68 | 0.70 | 0.66 | 0.73 | 0.76
X+KEn | 0.61 | 0.70 | 0.79 | 0.65 | 0.69 | 0.71 | 0.52 | 0.56 | 0.60 | 0.60 | 0.67 | 0.69 | 0.64 | 0.71 | 0.76

Table 1: Macro F1 on respective held-out test sets for models fine-tuned on N
target-language entries, averaged across 10 random seeds for each N. Best
performance for a given N in bold. Results across all N in Appendix B.

###  2.4 Testing on Held-Out Test Sets

When evaluating our mono- and multilingual models on their corresponding
target-language test sets, we find a set of consistent patterns in model
performance. We visualise overall performance in Figure 2 and highlight key
data points in Table 1.

First, there is an enormous benefit from even very small amounts of target-
language fine-tuning data N. Model performance increases sharply across models
and held-out test sets up to around N=200. For example, the performance of
X+DEn+Es increases from around 0.63 at N=10 to 0.70 at N=50, to 0.75 at N=200.

On the other hand, larger amounts of target-language fine-tuning data
correspond to much less of an improvement in model performance. Across models
and held-out test sets, there is a steep decrease in the marginal benefits of
increasing N. M+It, for example, improves by 0.33 macro F1 from N=20 to N=200,
and by just 0.06 from N=200 to N=2,000. X+Pt improves by 0.25 macro F1 from
N=20 to N=200, and by just 0.06 from N=200 to N=2,000. We analyse these
decreasing marginal benefits using linear regression in §2.6.

Further, there is a clear benefit to a first phase of fine-tuning on English
data, when there is limited target-language data. Absolute performance differs
across test sets, but multilingual models with initial fine-tuning on English
data (i.e. X+DEn, X+FEn and X+KEn) perform substantially better than those
without (i.e. M and X), up to around N=200. At N=20, there is up to 0.26 macro
F1 difference between the former and the latter. Conversely, models without
initial fine-tuning on English data need substantially more target-language
data to achieve the same performance. For example, X+DEn+Pt at N=100 performs
as well as M+Pt at N=300 on For19_Pt.

Figure 3:  Macro F1 on target-language tests from MHC across models fine-tuned
on up to N = 2,000 target-language entries. Model notation as described in
§2.2. Confidence intervals and model baselines as described in §2.4. We
provide larger versions of all five graphs in Appendix C.

| HateCheck_Es | HateCheck_Pt | HateCheck_Hi | HateCheck_Ar | HateCheck_It
---|---|---|---|---|---
N | 20 | 200 | 2,000 | 20 | 200 | 2,000 | 20 | 200 | 2,000 | 20 | 200 | 2,000 | 20 | 200 | 2,000
M | 0.42 | 0.38 | 0.64 | 0.30 | 0.54 | 0.59 | 0.23 | 0.23 | 0.23 | 0.25 | 0.23 | 0.24 | 0.29 | 0.43 | 0.48
X | 0.30 | 0.41 | 0.63 | 0.27 | 0.54 | 0.62 | 0.23 | 0.23 | 0.24 | 0.23 | 0.23 | 0.24 | 0.30 | 0.50 | 0.51
X+DEn | 0.83 | 0.82 | 0.80 | 0.76 | 0.77 | 0.76 | 0.57 | 0.53 | 0.33 | 0.60 | 0.50 | 0.31 | 0.79 | 0.80 | 0.75
X+FEn | 0.60 | 0.63 | 0.64 | 0.57 | 0.58 | 0.60 | 0.39 | 0.34 | 0.27 | 0.46 | 0.34 | 0.26 | 0.55 | 0.55 | 0.54
X+KEn | 0.54 | 0.53 | 0.61 | 0.62 | 0.62 | 0.62 | 0.36 | 0.30 | 0.24 | 0.39 | 0.30 | 0.26 | 0.53 | 0.56 | 0.56

Table 2: Macro F1 on target-language HateCheck for models fine-tuned on N
target-language entries, averaged across 10 random seeds for each N. Best
performance for a given N in bold. Results across all N in Appendix C.

Relatedly, there are clear differences in model performance based on which
English dataset was used in the first fine-tuning phase, when there is limited
target-language data. Among the three English datasets we evaluated, X+DEn
performs best for up to around N=200 on Bas19_Es, whereas X+FEn wins out for
the other four test sets.

After the strong initial divergence for low amounts of target-language fine-
tuning data, performance tends to align across models, regardless of whether
they were first fine-tuned on English data or only fine-tuned on the target
language. From around N=200 onwards, for a given test set, all models we
evaluate have broadly comparable performance. For example, all models score
0.71 to 0.72 macro F1 on For19_Pt for N=1,000.

Finally, we find that if there is some divergence in model performance on the
held-out test sets for larger amounts of target-language fine-tuning data,
then monolingual models tend to outperform multilingual models. For example,
despite the disadvantage of monolingual models for N$\leq$200, we find that
M+Es achieves 0.84 macro F1 on Bas19_Es for N=2,000, compared 0.82 macro F1
for X+DEn+Es. This difference is visible across test sets and significant at
95% confidence for the Spanish, Arabic and Italian test sets.

###  2.5 Testing on MHC

Performance on the target-language test suites from MHC, which none of the
models saw during fine-tuning, differs strongly and systematically from
results on the held-out test sets (Figure 3). Key data points are highlighted
in Table 2.

First, as for the held-out test sets, there is an initial benefit to a first
phase of fine-tuning on English data, up to around N=500 for MHC. However,
there are clear performance differences depending on which English dataset was
used. Models fine-tuned on Dyn21_En perform best by far across MHC for any N.
On Italian HateCheck for example, X+DEn achieves 0.79 macro F1 at N=20
compared to 0.55 for X+FEn+It and 0.29 for the monolingual model M+It.

Second, for Spanish, Portuguese and Italian, there is little to no benefit to
fine-tuning on more target-language data, as measured by performance on MHC.
Only those models that were not first fine-tuned on English data (i.e. M and
X) are improved by fine-tuning on more target-language data, up to around
N=500. None of the models benefit from even larger N. On Portuguese HateCheck
for example, X+Ken+Pt scores 0.62 macro F1 at N=500, N=1,000 and N=2,000.

Third, for Hindi and Arabic, training on more target-language data actively
harms performance on MHC. For larger N, all models degrade towards only
predicting the non-hateful class. On Arabic HateCheck, for example, X+FEn+Ar
achieves 0.26 macro F1 at N=2,000, which is only slightly better than the
“never hate” baseline at 0.23 macro F1. This is despite the exact same model
scoring 0.70 macro F1 on Ous19_Ar at N=2,000. We further discuss this finding
in §3.

Lastly model performance on MHC is much more volatile across random seeds
compared to performance on the held-out test sets. This can be seen from the
wider confidence intervals in Figure 3 compared to Figure 2.

###  2.6 Regression Analysis

When evaluating models on their corresponding held-out test sets, we saw
decreasing benefits to adding more target-language fine-tuning data (§2.4). To
quantify this observation and evidence its significance, we fit ordinary least
squares (OLS) linear regressions for each model on each test set. OLS is
commonly used in economics and the social sciences to analyse empirical
relationships between an outcome variable and one or more regressor variables.
In our case, the outcome is model performance as measured by macro F1 and the
regressor is the amount of target-language fine-tuning data N. Since the
relationship between the two variables is clearly non-linear and plausibly
exponential, we regress macro F1 on the natural logarithm of N instead of just
N, so that our regression equation for a given random seed $i$ takes the form
of

$F_{i}=\beta_{0}+\beta_{1}ln(N_{i})+\varepsilon_{i}$

where $F_{i}$ is the observed macro F1 and $\varepsilon_{i}$ the error term.
$\beta_{0}$, the intercept, can be interpreted as the expected zero-shot macro
F1, i.e. at N=0. $\beta_{1}$, the slope, can be interpreted as the expected
increase in macro F1 from increasing the amount of target-language fine-tuning
data N by 1%, due to our log-linear regression setup. In addition to reporting
regression coefficients, we also report adjusted $R^{2}$ as a goodness-of-fit
measure, which is calculated as the percentage of variance in macro F1
explained by our regression. Lastly, we report the expected percentage-point
increase in macro F1 from doubling N, denoted as $2N$, which is calculated as
$\beta_{1}ln(2)$. Since we saw that variance in macro F1 changes with N, we
use heteroskedasticity-robust HC3 standard errors (Hayes and Cai, 2007).

Model: M
---
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.2879 | 0.0792 | 91.97 | 5.49
For19_Pt | 0.2628 | 0.0670 | 71.23 | 4.65
Has21_Hi | 0.2998 | 0.0607 | 69.11 | 4.21
Ous19_Ar | 0.2092 | 0.0866 | 70.08 | 6.00
San20_It | 0.3784 | 0.0294 | 77.68 | 2.04
Model: X+DEn
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.5543 | 0.0363 | 87.75 | 2.52
For19_Pt | 0.5756 | 0.0205 | 63.66 | 1.42
Has21_Hi | 0.4691 | 0.0163 | 43.91 | 1.13
Ous19_Ar | 0.4320 | 0.0391 | 64.99 | 2.71
San20_It | 0.5724 | 0.0272 | 69.42 | 1.89

Table 3: OLS results for M and X+DEn. All regression coefficients $\beta_{0}$
and $\beta_{1}$ are significant at $p<0.001$. We show results for all models
in Appendix D.

We find that the natural logarithm of the amount of target-language fine-
tuning data N provides a strikingly good explanation for model performance as
measured by macro F1 (Table 3). $R^{2}$ for our regressions, where N is the
sole explanatory variable, is generally very high, up to 91.97%, which
indicates near-perfect goodness-of-fit. This strongly suggests that there is
an exponential decrease in the benefits of using more target-language fine-
tuning data. Conversely, there is a constant benefit to doubling the amount of
target-language fine-tuning data: We expect the same increase in macro F1
whether we increase N ten-fold from 10 to 20 or from 1,000 to 2,000.

Further, we can confirm our earlier findings that a) initial fine-tuning on
English data improves performance for low N, while b) all models perform
similarly well for larger N. In our regression, this is visible in monolingual
M models having much lower estimated zero-shot performance $\beta_{0}$ paired
with much larger $\beta_{1}$ and $2N$ effects compared to the X+DEn models.
For example, doubling the amount of Spanish fine-tuning data for M
($\beta_{0}$=0.29) corresponds to an expected increase in macro F1 of 5.49
points on Bas19_Es, whereas for X+DEn ($\beta_{0}$=0.55) we only expect an
increase of 2.52 points.

##  3 Discussion and Recommendations

Based on the sum of our experimental results, we formulate five
recommendations for the data-efficient development of effective hate speech
detection models for under-resourced languages.

Collect and label target-language data.

We found enormous benefits from fine-tuning models on even small amounts of
target-language data. Zero-shot cross-lingual transfer performs worse than
naive baselines, but few shots go a long way.

Do not annotate all the data you collect right away. Instead, start small and
iterate.

We found that the benefits of using more target-language fine-tuning data
decrease exponentially (Table 3), which suggests that the data in each dataset
is not diverse enough to warrant full annotation. The datasets we used in our
experiments contained several thousand entries, but most of the useful signal
in them can be learned from just a few dozen or hundred randomly-sampled
entries.444Future work could explore more deliberate selection strategies,
like active learning, which would likely achieve even better performance with
fewer target-language entries (see for example Markov et al., 2022). Thus,
most of the annotation cost and the potential harm to annotators in compiling
these datasets created very little performance benefit. Assuming constant cost
per additional annotation, improving F1 by just one percentage point grows
exponentially more expensive. Our results suggest that dataset creators should
at first annotate smaller subsets of the data they collected and iteratively
evaluate what marginal benefits additional annotations are likely to bring
compared to their costs.

Use diverse data collection strategies to minimise redundant data annotation.

We do not recommend against annotating target-language data. Instead, we
recommend against annotating too much data that was collected using the same
method. The Has21_Hi dataset, for example, was collected using trending
hashtags related to Covid and Indian politics (Modha et al., 2021), and
Ousidhoum et al. (2019) used topical keywords related to Islamic inter-sect
hate for Ous19_Ar. As a consequence, these datasets necessarily have a narrow
topical focus as well as limited linguistic diversity. This kind of selection
bias has been confirmed for several other hate speech datasets (Wiegand et
al., 2019; Ousidhoum et al., 2020). MHC, on the other hand, contains diverse
statements against seven different target groups in each language (Röttger et
al., 2022). Our results show that very quickly additional entries from a given
dataset carry little new and useful information (Figure 2), and that fine-
tuning on particularly narrow datasets like Has21_Hi and Ous19_Ar can even
harm model generalisability as measured by MHC (Figure 3). Conversely, more
diverse strategies for data collection, such as sampling from multiple
platforms, sampling from a diverse set of users, or using a wider variety of
topical keywords, are likely to result in data that is more worth annotating.

Use multilingual models to unlock readily-available data in high-resource
languages for initial fine-tuning.

We found that an initial phase of fine-tuning on English data increases model
performance on held-out test sets when there is little target-language fine-
tuning data (Figure 2). Thus, English data can partly substitute target-
language data in few-shot settings. It also substantially improves out-of-
domain generalisability to the MHC test suites (Figure 3), with benefits
varying by which English dataset is used. Only multilingual models can access
these benefits, because they can be fine-tuned on languages other than just
the target language. In our experiments, we used only three of the many
English hate speech datasets that have been published to date. Of them, we use
only one at a time, and only a subset of the full data. Future work could
explore the benefits from initial fine-tuning on larger and more diverse
datasets, or a combination of datasets in English and other languages that may
be more available than the target language.

Do not rely on just one held-out test set to evaluate model performance in the
target language.

Monolingual models appear strong when evaluated on held-out test sets, and
sometimes even outperform multilingual models when there is relatively much
target-language fine-tuning data (Figure 2). However, this hides clear
weaknesses in their generalisability as measured by MHC (Figure 3). These
findings suggest that monolingual models are more at risk of overfitting to
dataset-specific features than multilingual models fine-tuned first on data
from another language. In higher-resource settings, monolingual models are
still likely to outperform multilingual models (Martin et al., 2020; Nozza et
al., 2020; Armengol-Estapé et al., 2021).

##  4 Related Work

##### Cross-Lingual Transfer Learning

Only a very small number of the over 7,000 languages in the world are well-
represented in natural language processing resources (Joshi et al., 2020).
This has motivated much research into cross-lingual transfer, where large
multilingual language models are fine-tuned on one (usually high-resource)
source language and then applied to another (usually under-resourced) target
language. Cross-lingual transfer has been systematically evaluated for a wide
range of NLP tasks and for varying amounts of shots, i.e. target-language
training data. Evidence on the efficacy of zero-shot transfer is mixed, across
tasks and for transfer between different language pairs (Artetxe and Schwenk,
2019; Pires et al., 2019; Wu and Dredze, 2019; Lin et al., 2019; Hu et al.,
2020; Liang et al., 2020; Ruder et al., 2021, inter alia). In comparison, few-
shot approaches are generally found to be more effective (Lauscher et al.,
2020; Hedderich et al., 2020; Zhao et al., 2021; Hung et al., 2022). In many-
shot settings, multilingual models are often outperformed by their language-
specific monolingual counterparts (Martin et al., 2020; Nozza et al., 2020;
Armengol-Estapé et al., 2021). In this paper, we confirmed and expanded on key
results from this body of prior work, such as the effectiveness of few-shot
learning, specifically for the task of hate speech detection. We also
introduced OLS linear regression as an effective method for quantifying data
efficiency, i.e. dynamic cost-benefit trade-offs in data annotation.

##### Cross-Lingual Hate Speech Detection

Research on cross-lingual hate speech detection has generally had a more
narrow scope than ours. Nozza et al. (2020) demonstrate weaknesses of zero-
shot transfer between English, Italian and Spanish. Leite et al. (2020) find
similar results for Brazilian Portuguese, Bigoulaeva et al. (2021) for German,
and Stappen et al. (2020) for Spanish, while also providing initial evidence
for decreasing returns to fine-tuning on more hate speech data (see also Aluru
et al., 2020). Pelicon et al. (2021) and Ahn et al. (2020), like Stappen et
al. (2020), find that intermediate training on other languages is effective
when little target-language data is available. By comparison, we covered a
wider range of languages, with two datasets per language and a consistent
definition of hate across datasets. We provided a more systematic evaluation
of marginal benefits to additional target-language fine-tuning data based on
OLS linear regression, as well as a first assessment of out-of-domain
performance using the recently-released Multilingual HateCheck test suites. We
are also the first to formulate concrete recommendations for expanding hate
speech detection into more under-resourced languages.

##  5 Conclusion

In this paper, we explored strategies for hate speech detection in under-
resourced language settings that are effective while also minimising
annotation cost and risk of harm to annotators.

We conducted a series of experiments using mono- and multilingual language
models fine-tuned on differently-sized random samples of labelled hate speech
data in English as well as Arabic, Spanish, Hindi, Portuguese and Italian. We
evaluated all models on held-out test sets as well as out-of-domain target-
language test suites from Multilingual HateCheck, and used OLS linear
regression to quantify the marginal benefits of target-language fine-tuning
data. In our experiments, we found that 1) a small amount of target-language
fine-tuning data is needed to achieve strong performance on held-out test
sets, 2) the benefits of using more such data decrease exponentially, and 3)
initial fine-tuning on readily-available English data can partially substitute
target-language data and improve model generalisability. Based on our
findings, we formulated five actionable recommendations for expanding hate
speech detection into under-resourced languages.

Most hate speech research and resources so far have focused on English-
language content. However, hate speech is a global phenomenon, and hate speech
detection models are needed in hundreds more languages. With our findings and
recommendations, we hope that we can facilitate the development of models for
yet-unserved languages and thus help to close language gaps that right now
leave billions of non-English speakers world less protected against online
hate.

## Acknowledgments

This article was completed during Paul Röttger’s 2022 research visit to Dirk
Hovy’s research group at Bocconi University in Milan. Paul Röttger was funded
by the German Academic Scholarship Foundation. Debora Nozza received funding
from Fondazione Cariplo (grant No. 2020-4288, MONICA). Federico Bianchi was a
member of the Bocconi Institute for Data Science and Analysis (BIDSA) when
parts of this work were completed. Dirk Hovy received funding from the
European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation program (grant agreement No. 949944). Debora Nozza and
Dirk Hovy are members of the Data and Marketing Insights (DMI) Unit of the
Bocconi Institute for Data Science and Analysis (BIDSA). We thank the MilaNLP
Lab and the Pierrehumbert Language Modelling Group for their helpful comments,
and all reviewers for their constructive feedback.

## Limitations

Our analysis is necessarily constrained by the availability of suitable hate
speech datasets, particularly for non-English languages. The datasets we used
are idiosyncratic, not just because their language differs but also because
they were collected using different methods at different times. This is why we
focus on analysing general patterns across datasets in different languages,
rather than attempting like-for-like comparisons between individual languages.
As new datasets are created, future work could expand our analysis to cover
more languages as well as more datasets within each language.

Further, our experiments and the recommendations we formulate based on their
results presuppose the availability of at least some models tailored to the
target language. Multilingual models like XLM-R and XLM-T are respectively
pre-trained on 100 and on 30 languages (Conneau et al., 2020; Barbieri et al.,
2022), and there are active efforts to expand language model coverage to
hundreds of more languages (Wang et al., 2022). However, even the most
multilingual models available today cover only a fraction of the world’s
languages. Similarly, there is a growing number of monolingual transformer
models for different languages (Nozza et al., 2020) but most languages are
missing such a model.

Also, we focus on varying amounts of fine-tuning data while assuming the
existence of held-out development and test sets to evaluate our experiments.
However large or small these sets are, labelling them also creates costs and
risks of harm to annotators. This needs to be factored into cost assessments
for expanding hate speech detection into an under-resourced language.

For reasons of scope and prioritisation, we did not explore the full range of
possibilities for improving cross-lingual model performance. For example, we
chose English data for the initial fine-tuning because it is most readily
available, but English may not be the optimal choice (Lin et al., 2019).
Further, prior work on hate speech detection has found some benefits to data
augmentation based on machine translation (Pamungkas et al., 2021; Wang and
Banko, 2021) or semi-supervised learning (Zia et al., 2022), which could be
explored in future research.

Lastly, our definition of hate speech (§1) is based on a Western legal
consensus, in which the presence of hate speech hinges on the target being a
protected group. This definition is consistent throughout the datasets we use
for our experiments. However, which groups are considered protected may differ
across legal and cultural settings. The United Arab Emirates, for example, do
not consider sexual orientation a protected characteristic (UAE, 2022). To
avoid Eurocentrism, potential differences across legal and cultural settings
need to be accounted for when expanding hate speech detection into new
languages and when creating new hate speech datasets.

## Ethical Considerations

##### Language Representation

Besides English, our experiments cover a limited set of five languages:
Arabic, Spanish, Hindi, Portuguese and Italian. This is due to the very
scarcity of suitable non-English datasets for hate speech detection that our
work seeks to address. While certainly under-resourced and under-represented
in the context of hate speech detection, the five non-English languages we
consider are widely spoken, and they have received at least some attention in
hate speech research. There is a clear need for creating hate speech datasets
in even more under-resourced languages, to facilitate the creation of hate
speech detection models in these languages and also to enable even more
comprehensive analyses of the kind we conducted here.

##### Biases in Multilingual Models

Zhao et al. (2020) show that multilingual models exhibit similar biases as
their monolingual counterparts. Therefore, there is a risk that the models we
trained, especially those trained on fewer target-language entries, reflect
the biases present in their training and fine-tuning data. Our analyses
focused on general model efficacy rather than bias evaluation. Future work
could for example make use of Multilingual HateCheck to evaluate model
performance across different targeted groups.

##### Environmental Impact

We fine-tuned 3,000 models for our experiments: five model types for each of
five target languages with 12 differently-sized samples for 10 random seeds in
each language. However, each model fine-tuning step was very computationally
inexpensive, especially because we used very small amounts of fine-tuning
data. We were able to run all our experimentation on the University of
Oxford’s Advanced Research Computing cloud CPUs. Fine-tuning 120 models for a
given language took only around four hours for monolingual models and around
seven hours for XLM-T. Relative to the concerns raised around the
environmental costs of pre-training large language models (Strubell et al.,
2019; Henderson et al., 2020; Bender et al., 2021), or even larger-scale fine-
tuning with hyperparameter tuning, we therefore consider the environmental
costs of our work to be relatively minor. Further, our findings enable the
training more effective models with less data and may thus reduce the
environmental impact of future work.

## References

  * Ahn et al. (2020) Hwijeen Ahn, Jimin Sun, Chan Young Park, and Jungyun Seo. 2020.  NLPDove at SemEval-2020 task 12: Improving offensive language detection with cross-lingual transfer.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 1576–1586, Barcelona (online). International Committee for Computational Linguistics.
  * Aluru et al. (2020) Sai Saketh Aluru, Binny Mathew, Punyajoy Saha, and Animesh Mukherjee. 2020.  A deep dive into multilingual hate speech classification.  In _Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part V_ , page 423–439, Berlin, Heidelberg. Springer-Verlag.
  * Antoun et al. (2020) Wissam Antoun, Fady Baly, and Hazem Hajj. 2020.  AraBERT: Transformer-based model for Arabic language understanding.  In _Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection_ , pages 9–15, Marseille, France. European Language Resource Association.
  * Armengol-Estapé et al. (2021) Jordi Armengol-Estapé, Casimiro Pio Carrino, Carlos Rodriguez-Penagos, Ona de Gibert Bonet, Carme Armentano-Oller, Aitor Gonzalez-Agirre, Maite Melero, and Marta Villegas. 2021.  Are multilingual models the best choice for moderately under-resourced languages? A comprehensive assessment for Catalan.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4933–4946, Online. Association for Computational Linguistics.
  * Artetxe and Schwenk (2019) Mikel Artetxe and Holger Schwenk. 2019.  Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond.  _Transactions of the Association for Computational Linguistics_ , 7:597–610.
  * Banerjee et al. (2021) Somnath Banerjee, Maulindu Sarkar, Nancy Agrawal, Punyajoy Saha, and Mithun Das. 2021.  Exploring transformer based models to identify hate speech and offensive content in English and Indo-Aryan languages.  _arXiv preprint arXiv:2111.13974_.
  * Barbieri et al. (2022) Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. 2022.  XLM-T: Multilingual language models in Twitter for sentiment analysis and beyond.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 258–266, Marseille, France. European Language Resources Association.
  * Basile et al. (2019) Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019\.  SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter.  In _Proceedings of the 13th International Workshop on Semantic Evaluation_ , pages 54–63, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
  * Bender et al. (2021) Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021.  On the dangers of stochastic parrots: Can language models be too big? In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’21, page 610–623, New York, NY, USA. Association for Computing Machinery.
  * Bigoulaeva et al. (2021) Irina Bigoulaeva, Viktor Hangya, and Alexander Fraser. 2021.  Cross-lingual transfer learning for hate speech detection.  In _Proceedings of the First Workshop on Language Technology for Equality, Diversity and Inclusion_ , pages 15–25, Kyiv. Association for Computational Linguistics.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451, Online. Association for Computational Linguistics.
  * Derczynski et al. (2022) Leon Derczynski, Hannah Rose Kirk, Abeba Birhane, and Bertie Vidgen. 2022.  Handling and presenting harmful text.  _arXiv preprint arXiv:2204.14256_.
  * Fortuna et al. (2019) Paula Fortuna, João Rocha da Silva, Juan Soler-Company, Leo Wanner, and Sérgio Nunes. 2019.  A hierarchically-labeled Portuguese hate speech dataset.  In _Proceedings of the Third Workshop on Abusive Language Online_ , pages 94–104, Florence, Italy. Association for Computational Linguistics.
  * Founta et al. (2018) Antigoni Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018.  Large scale crowdsourcing and characterization of Twitter abusive behavior.  _Proceedings of the International AAAI Conference on Web and Social Media_ , 12(1).
  * Hayes and Cai (2007) Andrew F Hayes and Li Cai. 2007.  Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation.  _Behavior research methods_ , 39(4):709–722.
  * Hedderich et al. (2020) Michael A. Hedderich, David Adelani, Dawei Zhu, Jesujoba Alabi, Udia Markus, and Dietrich Klakow. 2020.  Transfer learning and distant supervision for multilingual transformer models: A study on African languages.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2580–2591, Online. Association for Computational Linguistics.
  * Henderson et al. (2020) Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020.  Towards the systematic reporting of the energy and carbon footprints of machine learning.  _Journal of Machine Learning Research_ , 21(248):1–43.
  * Hu et al. (2020) Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020.  Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation.  In _International Conference on Machine Learning_ , pages 4411–4421. PMLR.
  * Hung et al. (2022) Chia-Chien Hung, Anne Lauscher, Ivan Vulić, Simone Ponzetto, and Goran Glavaš. 2022.  Multi2WOZ: A robust multilingual dataset and conversational pretraining for task-oriented dialog.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3687–3703, Seattle, United States. Association for Computational Linguistics.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020.  The state and fate of linguistic diversity and inclusion in the NLP world.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6282–6293, Online. Association for Computational Linguistics.
  * Kennedy et al. (2020) Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. 2020.  Contextualizing hate speech classifiers with post-hoc explanation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5435–5442, Online. Association for Computational Linguistics.
  * Kiela et al. (2021) Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. 2021.  Dynabench: Rethinking benchmarking in NLP.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4110–4124, Online. Association for Computational Linguistics.
  * Lauscher et al. (2020) Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020.  From zero to hero: On the limitations of zero-shot language transfer with multilingual Transformers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4483–4499, Online. Association for Computational Linguistics.
  * Leite et al. (2020) João Augusto Leite, Diego Silva, Kalina Bontcheva, and Carolina Scarton. 2020\.  Toxic language detection in social media for Brazilian Portuguese: New dataset and multilingual analysis.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pages 914–924, Suzhou, China. Association for Computational Linguistics.
  * Liang et al. (2020) Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, and Ming Zhou. 2020.  XGLUE: A new benchmark datasetfor cross-lingual pre-training, understanding and generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6008–6018, Online. Association for Computational Linguistics.
  * Lin et al. (2019) Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. 2019.  Choosing transfer languages for cross-lingual learning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3125–3135, Florence, Italy. Association for Computational Linguistics.
  * Marinescu (2021) Delia Marinescu. 2021.  Facebook’s content moderation language barrier.  _New America_.
  * Markov et al. (2022) Todor Markov, Chong Zhang, Sandhini Agarwal, Tyna Eloundou, Teddy Lee, Steven Adler, Angela Jiang, and Lilian Weng. 2022.  A holistic approach to undesired content detection in the real world.  _arXiv preprint arXiv:2208.03274_.
  * Martin et al. (2020) Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020.  CamemBERT: a tasty French language model.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7203–7219, Online. Association for Computational Linguistics.
  * Modha et al. (2021) Sandip Modha, Thomas Mandl, Gautam Kishore Shahi, Hiren Madhu, Shrey Satapara, Tharindu Ranasinghe, and Marcos Zampieri. 2021.  Overview of the hasoc subtrack at fire 2021: Hate speech and offensive content identification in english and indo-aryan languages and conversational hate speech.  In _Forum for Information Retrieval Evaluation_ , FIRE 2021, page 1–3, New York, NY, USA. Association for Computing Machinery.
  * Nozza (2021) Debora Nozza. 2021.  Exposing the limits of zero-shot cross-lingual hate speech detection.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 907–914, Online. Association for Computational Linguistics.
  * Nozza et al. (2020) Debora Nozza, Federico Bianchi, and Dirk Hovy. 2020.  What the [MASK]? making sense of language-specific BERT models.  _arXiv preprint arXiv:2003.02912_.
  * Ousidhoum et al. (2019) Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019\.  Multilingual and multi-aspect hate speech analysis.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4675–4684, Hong Kong, China. Association for Computational Linguistics.
  * Ousidhoum et al. (2020) Nedjma Ousidhoum, Yangqiu Song, and Dit-Yan Yeung. 2020.  Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2532–2542. Association for Computational Linguistics.
  * Pamungkas et al. (2021) Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2021.  A joint learning approach with knowledge injection for zero-shot cross-lingual hate speech detection.  _Information Processing & Management_, 58(4):102544.
  * Pelicon et al. (2021) Andraž Pelicon, Ravi Shekhar, Blaž Škrlj, Matthew Purver, and Senja Pollak. 2021\.  Investigating cross-lingual training for offensive language detection.  _PeerJ Computer Science_ , 7:e559.  Publisher: PeerJ Inc.
  * Pires et al. (2019) Telmo Pires, Eva Schlinger, and Dan Garrette. 2019.  How multilingual is multilingual BERT? In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
  * Poletto et al. (2021) Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021.  Resources and benchmark corpora for hate speech detection: a systematic review.  _Language Resources and Evaluation_ , 55(2):477–523.
  * Pérez et al. (2021) Juan Manuel Pérez, Damián A. Furman, Laura Alonso Alemany, and Franco Luque. 2021\.  RoBERTuito: a pre-trained language model for social media text in Spanish.  _arXiv preprint arXiv:2111.09453_.
  * Röttger et al. (2022) Paul Röttger, Haitham Seelawi, Debora Nozza, Zeerak Talat, and Bertie Vidgen. 2022.  Multilingual HateCheck: Functional tests for multilingual hate speech detection models.  In _Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH)_ , pages 154–169, Seattle, Washington (Hybrid). Association for Computational Linguistics.
  * Röttger et al. (2021) Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Talat, Helen Margetts, and Janet Pierrehumbert. 2021.  HateCheck: Functional tests for hate speech detection models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 41–58, Online. Association for Computational Linguistics.
  * Ruder et al. (2021) Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Dan Garrette, Graham Neubig, and Melvin Johnson. 2021\.  XTREME-R: Towards more challenging and nuanced multilingual evaluation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10215–10245, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Sanguinetti et al. (2020) Manuela Sanguinetti, Gloria Comandini, Elisa Di Nuovo, Simona Frenda, Marco Stranisci, Cristina Bosco, Tommaso Caselli, Viviana Patti, and Irene Russo. 2020\.  Haspeede 2 @ EVALITA2020: overview of the EVALITA 2020 hate speech detection task.  In _Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020), Online event, December 17th, 2020_ , volume 2765 of _CEUR Workshop Proceedings_. CEUR-WS.org.
  * Simonite (2021) Tom Simonite. 2021.  Facebook is everywhere; its moderation is nowhere close.  _Wired_.
  * Souza et al. (2020) Fábio Souza, Rodrigo Nogueira, and Roberto Lotufo. 2020.  BERTimbau: Pretrained BERT models for Brazilian Portuguese.  In _Intelligent Systems_ , pages 403–417, Cham. Springer International Publishing.
  * Stappen et al. (2020) Lukas Stappen, Fabian Brunn, and Björn W. Schuller. 2020.  Cross-lingual zero- and few-shot hate speech detection utilising frozen transformer language models and AXEL.  _CoRR_ , abs/2004.13850.
  * Strubell et al. (2019) Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019.  Energy and policy considerations for deep learning in NLP.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3645–3650, Florence, Italy. Association for Computational Linguistics.
  * UAE (2022) UAE. 2022.  Facebook to pay $52m to content moderators over ptsd.  _u.ae_.
  * Vidgen and Derczynski (2020) Bertie Vidgen and Leon Derczynski. 2020.  Directions in abusive language training data, a systematic review: Garbage in, garbage out.  _PLOS ONE_ , 15(12):e0243300.
  * Vidgen et al. (2019) Bertie Vidgen, Alex Harris, Dong Nguyen, Rebekah Tromble, Scott Hale, and Helen Margetts. 2019.  Challenges and frontiers in abusive content detection.  In _Proceedings of the Third Workshop on Abusive Language Online_ , pages 80–93, Florence, Italy. Association for Computational Linguistics.
  * Vidgen et al. (2021) Bertie Vidgen, Tristan Thrush, Zeerak Talat, and Douwe Kiela. 2021.  Learning from the worst: Dynamically generated datasets to improve online hate detection.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1667–1682, Online. Association for Computational Linguistics.
  * Wang and Banko (2021) Cindy Wang and Michele Banko. 2021.  Practical Transformer-based Multilingual Text Classification.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers_ , pages 121–129, Online. Association for Computational Linguistics.
  * Wang et al. (2022) Xinyi Wang, Sebastian Ruder, and Graham Neubig. 2022.  Expanding pretrained models to thousands more languages via lexicon-based adaptation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 863–877, Dublin, Ireland. Association for Computational Linguistics.
  * Wiegand et al. (2019) Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019.  Detection of abusive language: The problem of biased datasets.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 602–608, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Wu and Dredze (2019) Shijie Wu and Mark Dredze. 2019.  Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 833–844, Hong Kong, China. Association for Computational Linguistics.
  * Zhao et al. (2020) Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Hassan Awadallah. 2020.  Gender bias in multilingual embeddings and cross-lingual transfer.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2896–2907, Online. Association for Computational Linguistics.
  * Zhao et al. (2021) Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, and Hinrich Schütze. 2021.  A closer look at few-shot crosslingual transfer: The choice of shots matters.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5751–5767, Online. Association for Computational Linguistics.
  * Zia et al. (2022) Haris Bin Zia, Ignacio Castro, Arkaitz Zubiaga, and Gareth Tyson. 2022.  Improving zero-shot cross-lingual hate speech detection with pseudo-label fine-tuning of transformer language models.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 16, pages 1435–1439.

##  Appendix A Details on Model Training

##### Parameters

We implemented XLM-T as well as the five language-specific monolingual models
(§2.2) using the HuggingFace transformers library (Wolf et al., 2020).
Training batch size was 16. Maximum sequence length was 128 tokens. Otherwise,
we used default parameters. The optional first phase of fine-tuning on English
data was for three epochs. Fine-tuning on the target language was for five
epochs, with subsequent best epoch selection based on macro F1 on the held-out
development set.

##### Computation

We ran all experiments on our institution’s 16-core cloud CPUs. Fine-tuning
120 models for a one of the five target languages took around four hours for
monolingual models and around seven hours for XLM-T.

##  Appendix B Macro F1 on Held-Out Test Sets

For larger versions of the graphs in Figure 2 from the main body, see Figure 4
on the next page. See Table 5 for a version of Table 1 across all N.

##  Appendix C Macro F1 on MHC

For larger versions of the graphs in Figure 3 from the main body, see Figure 5
below. Similarly, see Table 6 for a version of Table 2 across all N.

##  Appendix D Regression Results for All Models

See Table 4 to the right.

Model: M
---
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.2879 | 0.0792 | 91.97 | 5.49
For19_Pt | 0.2628 | 0.0670 | 71.23 | 4.65
Has21_Hi | 0.2998 | 0.0607 | 69.11 | 4.21
Ous19_Es | 0.2092 | 0.0866 | 70.08 | 6.00
San20_It | 0.3784 | 0.0294 | 77.68 | 2.04
Model: X
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.2107 | 0.0856 | 75.39 | 5.93
For19_Pt | 0.1727 | 0.0808 | 79.17 | 5.6
Has21_Hi | 0.1727 | 0.0808 | 79.17 | 5.6
Ous19_Es | 0.1852 | 0.0869 | 69.09 | 6.02
San20_It | 0.4004 | 0.019 | 40.73 | 1.32
Model: X+DEn
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.5543 | 0.0363 | 87.75 | 2.52
For19_Pt | 0.5756 | 0.0205 | 63.66 | 1.42
Has21_Hi | 0.4691 | 0.0163 | 43.91 | 1.13
Ous19_Es | 0.4320 | 0.0391 | 64.99 | 2.71
San20_It | 0.5724 | 0.0272 | 69.42 | 1.89
Model: X+FEn
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.4347 | 0.0489 | 93.95 | 3.39
For19_Pt | 0.6167 | 0.0148 | 60.96 | 1.02
Has21_Hi | 0.5684 | 0.0192 | 45.73 | 1.33
Ous19_Es | 0.6256 | 0.0191 | 63.71 | 1.32
San20_It | 0.5227 | 0.008 | 7.81 | 0.56
Model: X+KEn
Test Set | $\beta_{0}$ | $\beta_{1}$ | $R^{2}$ | $2N$
Bas19_Es | 0.5011 | 0.0381 | 93.59 | 2.64
For19_Pt | 0.5797 | 0.0182 | 72.81 | 1.26
Has21_Hi | 0.5545 | 0.0195 | 40.66 | 1.35
Ous19_Es | 0.5712 | 0.026 | 82.84 | 1.8
San20_It | 0.4444 | 0.0198 | 42.5 | 1.37

Table 4: OLS results for all models in our experiments, expanding on Table 3
from the main body. All regression coefficients $\beta_{0}$ and $\beta_{1}$
are significant at $p<0.001$. Figure 4:  Macro F1 on target-language held-out
test sets across models trained on up to N=2,000 target-language entries.
Model notation as described in §2.2. Confidence intervals and model baselines
as described in §2.4.

| Bas19_Es
---|---
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.46 | 0.50 | 0.56 | 0.57 | 0.59 | 0.66 | 0.72 | 0.77 | 0.79 | 0.80 | 0.83 | 0.84
X | 0.41 | 0.48 | 0.49 | 0.46 | 0.55 | 0.60 | 0.70 | 0.74 | 0.76 | 0.77 | 0.79 | 0.81
X+DEn | 0.63 | 0.66 | 0.68 | 0.69 | 0.70 | 0.72 | 0.75 | 0.76 | 0.78 | 0.78 | 0.81 | 0.82
X+FEn | 0.56 | 0.59 | 0.60 | 0.61 | 0.62 | 0.65 | 0.68 | 0.71 | 0.74 | 0.75 | 0.78 | 0.80
X+KEn | 0.61 | 0.61 | 0.63 | 0.63 | 0.63 | 0.67 | 0.70 | 0.72 | 0.74 | 0.74 | 0.77 | 0.79
| For19_Pt
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.45 | 0.46 | 0.44 | 0.46 | 0.50 | 0.59 | 0.67 | 0.68 | 0.70 | 0.69 | 0.71 | 0.73
X | 0.40 | 0.42 | 0.41 | 0.42 | 0.45 | 0.51 | 0.67 | 0.68 | 0.70 | 0.70 | 0.71 | 0.73
X+DEn | 0.62 | 0.63 | 0.64 | 0.64 | 0.65 | 0.68 | 0.70 | 0.70 | 0.70 | 0.70 | 0.72 | 0.72
X+FEn | 0.66 | 0.66 | 0.66 | 0.67 | 0.66 | 0.69 | 0.71 | 0.70 | 0.71 | 0.71 | 0.72 | 0.73
X+KEn | 0.62 | 0.65 | 0.64 | 0.64 | 0.64 | 0.66 | 0.69 | 0.69 | 0.69 | 0.70 | 0.71 | 0.71
| Has21_Hi
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.47 | 0.47 | 0.47 | 0.47 | 0.47 | 0.50 | 0.55 | 0.55 | 0.56 | 0.57 | 0.58 | 0.60
X | 0.47 | 0.47 | 0.47 | 0.47 | 0.47 | 0.47 | 0.47 | 0.47 | 0.50 | 0.54 | 0.56 | 0.56
X+DEn | 0.52 | 0.52 | 0.53 | 0.53 | 0.52 | 0.53 | 0.56 | 0.56 | 0.56 | 0.57 | 0.59 | 0.60
X+FEn | 0.55 | 0.55 | 0.56 | 0.55 | 0.55 | 0.54 | 0.56 | 0.57 | 0.57 | 0.57 | 0.58 | 0.59
X+KEn | 0.50 | 0.52 | 0.51 | 0.51 | 0.49 | 0.52 | 0.56 | 0.56 | 0.58 | 0.55 | 0.58 | 0.60
| Ous19_Ar
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.48 | 0.48 | 0.46 | 0.51 | 0.53 | 0.53 | 0.66 | 0.64 | 0.70 | 0.71 | 0.72 | 0.72
X | 0.42 | 0.43 | 0.43 | 0.43 | 0.43 | 0.49 | 0.69 | 0.69 | 0.69 | 0.69 | 0.71 | 0.70
X+DEn | 0.54 | 0.52 | 0.53 | 0.58 | 0.59 | 0.63 | 0.66 | 0.68 | 0.68 | 0.69 | 0.68 | 0.70
X+FEn | 0.61 | 0.61 | 0.63 | 0.65 | 0.64 | 0.67 | 0.68 | 0.69 | 0.69 | 0.68 | 0.70 | 0.70
X+KEn | 0.60 | 0.60 | 0.62 | 0.62 | 0.62 | 0.65 | 0.67 | 0.67 | 0.67 | 0.68 | 0.69 | 0.69
| San20_It
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.44 | 0.40 | 0.44 | 0.50 | 0.56 | 0.69 | 0.73 | 0.75 | 0.75 | 0.76 | 0.78 | 0.79
X | 0.45 | 0.40 | 0.35 | 0.49 | 0.56 | 0.63 | 0.70 | 0.73 | 0.74 | 0.74 | 0.77 | 0.78
X+DEn | 0.64 | 0.63 | 0.67 | 0.67 | 0.68 | 0.71 | 0.73 | 0.73 | 0.74 | 0.74 | 0.76 | 0.77
X+FEn | 0.68 | 0.66 | 0.69 | 0.69 | 0.71 | 0.72 | 0.73 | 0.74 | 0.75 | 0.74 | 0.76 | 0.76
X+KEn | 0.64 | 0.64 | 0.66 | 0.65 | 0.67 | 0.68 | 0.71 | 0.72 | 0.73 | 0.74 | 0.75 | 0.76

Table 5: Macro F1 on respective held-out test sets for models fine-tuned on N
target-language entries, averaged across 10 random seeds for each N. Best
performance for a given N in bold. Figure 5:  Macro F1 on target-language
tests from MHC across models trained on up to N = 2,000 target-language
entries. Model notation as described in §2.2. Confidence intervals and model
baselines as described in §2.4.

| HateCheck_Es
---|---
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.43 | 0.42 | 0.42 | 0.33 | 0.32 | 0.40 | 0.38 | 0.48 | 0.56 | 0.52 | 0.60 | 0.64
X | 0.35 | 0.30 | 0.31 | 0.25 | 0.28 | 0.34 | 0.41 | 0.48 | 0.56 | 0.52 | 0.61 | 0.63
X+DEN | 0.83 | 0.83 | 0.83 | 0.82 | 0.82 | 0.82 | 0.82 | 0.81 | 0.80 | 0.81 | 0.80 | 0.80
X+FEN | 0.59 | 0.60 | 0.61 | 0.61 | 0.60 | 0.59 | 0.63 | 0.62 | 0.62 | 0.63 | 0.64 | 0.64
X+KEN | 0.56 | 0.54 | 0.53 | 0.51 | 0.47 | 0.51 | 0.53 | 0.58 | 0.56 | 0.59 | 0.59 | 0.61
| HateCheck_Pt
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.38 | 0.30 | 0.30 | 0.31 | 0.37 | 0.44 | 0.54 | 0.57 | 0.58 | 0.58 | 0.57 | 0.59
X | 0.27 | 0.27 | 0.23 | 0.27 | 0.28 | 0.33 | 0.54 | 0.56 | 0.58 | 0.60 | 0.60 | 0.62
X+DEN | 0.76 | 0.76 | 0.77 | 0.77 | 0.78 | 0.78 | 0.77 | 0.78 | 0.78 | 0.76 | 0.75 | 0.76
X+FEN | 0.57 | 0.57 | 0.57 | 0.57 | 0.57 | 0.59 | 0.58 | 0.59 | 0.59 | 0.60 | 0.61 | 0.60
X+KEN | 0.60 | 0.62 | 0.62 | 0.62 | 0.62 | 0.62 | 0.62 | 0.63 | 0.64 | 0.62 | 0.62 | 0.62
| HateCheck_Hi
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23
X | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.24 | 0.24
X+DEN | 0.62 | 0.57 | 0.60 | 0.53 | 0.47 | 0.46 | 0.53 | 0.48 | 0.44 | 0.43 | 0.37 | 0.33
X+FEN | 0.43 | 0.39 | 0.42 | 0.37 | 0.33 | 0.35 | 0.34 | 0.32 | 0.31 | 0.31 | 0.29 | 0.27
X+KEN | 0.38 | 0.36 | 0.35 | 0.31 | 0.29 | 0.29 | 0.30 | 0.28 | 0.30 | 0.27 | 0.27 | 0.24
| HateCheck_Ar
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.31 | 0.25 | 0.24 | 0.25 | 0.24 | 0.23 | 0.23 | 0.24 | 0.23 | 0.24 | 0.24 | 0.24
X | 0.29 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.23 | 0.24 | 0.24 | 0.25 | 0.24
X+DEN | 0.67 | 0.60 | 0.60 | 0.60 | 0.56 | 0.49 | 0.50 | 0.44 | 0.44 | 0.46 | 0.37 | 0.31
X+FEN | 0.51 | 0.46 | 0.46 | 0.45 | 0.39 | 0.35 | 0.34 | 0.32 | 0.31 | 0.31 | 0.28 | 0.26
X+KEN | 0.43 | 0.39 | 0.37 | 0.35 | 0.32 | 0.31 | 0.30 | 0.31 | 0.31 | 0.29 | 0.26 | 0.26
| HateCheck_It
N | 10 | 20 | 30 | 40 | 50 | 100 | 200 | 300 | 400 | 500 | 1000 | 2,000
M | 0.32 | 0.29 | 0.31 | 0.25 | 0.24 | 0.33 | 0.43 | 0.39 | 0.42 | 0.45 | 0.48 | 0.48
X | 0.38 | 0.30 | 0.29 | 0.34 | 0.36 | 0.42 | 0.50 | 0.54 | 0.54 | 0.51 | 0.54 | 0.51
X+DEN | 0.81 | 0.79 | 0.80 | 0.79 | 0.80 | 0.80 | 0.80 | 0.79 | 0.78 | 0.78 | 0.77 | 0.75
X+FEN | 0.56 | 0.55 | 0.55 | 0.55 | 0.55 | 0.56 | 0.55 | 0.57 | 0.56 | 0.55 | 0.56 | 0.54
X+KEN | 0.52 | 0.53 | 0.54 | 0.54 | 0.54 | 0.54 | 0.56 | 0.56 | 0.57 | 0.57 | 0.55 | 0.56

Table 6: Macro F1 on target-language HateCheck for models fine-tuned on N
target-language entries, averaged across 10 random seeds for each N. Best
performance for a given N in bold.


# Extending the scope and understanding of all-optical magnetization switching
in Gd-based alloys by controlling the underlying temperature transients

Maxime Verges, Wei Zhang, Quentin Remy, Yann Le-Guen, Jon Gorchon, Gregory
Malinowski, Stephane Mangin, Michel Hehn, Julius Hohlfeld  Institut Jean
Lamour, CNRS UMR 7198, Université de Lorraine, Vandoeuvre-lès-Nancy F-54506,
France

###### Abstract

We use the thickness of Cu layers to control all-optical switching of
magnetization in adjacent Gd24(Fe90Co${}_{10})_{76}$ films. While increasing
the Cu thickness from $5$ to $900\,$nm has no effect on the switching
threshold, it significantly enlarges the fluence and pulse duration at which
multiple domains emerge. Having shown that thermally activated multi-domain
formation limits the maximum fluence and pulse duration for controlled
switching, we demonstrate that continuous magnetization reversal precedes
multi-domain formation in Gd18Dy4Co78 films excited with fluences slightly
larger than the multi-domain threshold.

Optical excitation of ferrimagnetic rare earth/transition metal (RE/TM) alloys
and multi-layers by a single, arbitrary polarized laser pulse can lead to
ultrafast magnetization reversal when the fluence and duration of this pulse
stay within mutually dependent bounds [1, 2, 3, 4, 5, 6, 7, 8]. In Gd-based
samples, this so-called all-optical helicity independent switching (AO-HIS)
(also known as temperature induced magnetization switching (TIMS)) results
from transient spin transfer between the two magnetic subsystems by angular
momentum conserving exchange scattering that originates from their different
de- and remagnetization rates in response to laser pulse induced changes of
the electron temperature. This understanding is the essence of so many
experimental and theoretical investigations that it is impossible to do
justice to each single one of them. Four experiments, showing the appearance
of a transient ferromagnetic state during the reversal [6] and magnetization
reversal due to picosecond heat pulses resulting from current pulse induced
joule heating [9] and from fast heat diffusion through thick gold films [10,
11], stand out as key demonstrations of the exchange and temperature driven
nature of the magnetization reversal in GdFeCo, respectively.

Theoretical works have identified the dominance of spin transfer between the
magnetic subsystems over spin transfer to the lattice as prerequisite for AO-
HIS [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22], derived preconditions for
the magnetization magnitudes of both subsystems from this switching criterion
[23], and qualitatively reproduced the impact of Gd-concentration, laser
fluence and initial temperature on AO-HIS [24, 15]. However, the questions if
the fluence thresholds for the appearance of controlled AO-HIS and of
stochastic multi-domain formation are linked to thresholds for the maximum
electron temperature [25] or not [1, 8] and if the maximum pulse duration
leading to AO-HIS is limited by a breakdown of the underlying switching
mechanism [2], or by thermal erasure at longer time delays [3] are still
matter of debate.

In this letter, we use the thickness of Cu heat sinks, $t_{\rm Cu}$, to
control the transient laser pulse induced increase of the electron temperature
within adjacent $10\,$nm thick GdFeCo films, $\Delta T_{\rm e}(t)$, that
results from their excitation by short laser pulses of various fluence,
$F_{\rm las}$, and pulse duration, $\tau_{\rm las}$. This allows us to use
slight adjustments of the laser fluence as means to tailor $\Delta T_{\rm
e}(t)$ for any given laser pulse duration such, that identical heating
dynamics are followed by faster and stronger cooling for thicker heat sinks
(cf. Fig.1).

Figure 1:  Transients of $\Delta T_{\rm e}/\Delta T_{\rm e,max}$ within GdFeCo
predicted by the TTM for $\tau_{\rm las}$ and $t_{\rm Cu}$ given in the
legend.

In this way we are able to disentangle the impact of $\Delta T_{\rm e}$ at
short times, where it reaches its maximum, and at longer times, where it is in
quasi-equilibrium with the elevated phonon temperature, on the fluence
thresholds for controlled switching, $F_{\rm SW}$, and of stochastic multi-
domain formation, $F_{\rm MD}$. We derive both thresholds from domain images
acquired via static linear magneto-optical microscopy.

The investigated samples are grown by dc magnetron sputtering onto glass
substrates in the following multilayered structure:
glass/Ta(3)/Cu(5)/Gd24(Fe90Co${}_{10})$(10)/Cu($t_{\rm Cu}$)/Pt(5), where the
numbers in brackets give the layer thickness in nm. The bottom Ta- and Cu-
layers improve adhesion of the structure to the glass substrate and ensure
that the GdFeCo-layer exhibits perpendicular magnetic anisotropy, respectively
[26]. They are thin enough to allow probing of the magnetic state of the
GdFeCo layer via measurements of the linear magneto-optical Kerr effect from
the substrate side. The Pt-capping layer prevents oxidation of the sample. The
thickness of the upper Cu heat sink wedges, $t_{\rm Cu}$, varies linearly
across the 2.5 cm wide samples over the following ranges (in nm): [5,10],
[10,15], [10,20], [20,30], [35,70], [70,105], [50,100], [100,150], [100,200],
[200,300], [300,600], and [600,900]. Great care was taken to ensure identical
compositions/magnetic properties of all GdFeCo layers, which was confirmed by
the demonstration of identical hysteresis loops and equal fluence thresholds
for different samples with equal $t_{\rm Cu}$.

The fluence thresholds $F_{\rm SW}$ and $F_{\rm MD}$ are derived from the size
and quality of written domains imaged as function of $F_{\rm las}$, $\tau_{\rm
las}$, and $t_{\rm Cu}$ by magneto-optical microscopy (cf. insets of Fig. 2).
We first calibrate this derivation by simultaneously fitting the radii
$r_{x/y}$ of the switched domains observed in samples with particular heat
sink thicknesses after their excitation by $p$-polarized, $800\,$nm, $150\,$fs
laser pulses of various pulse energies, $E_{\rm las}$, to

$r_{x/y}=\sigma_{x/y}\sqrt{\ln\left(\frac{E_{\rm
las}}{\pi\sigma_{x}\sigma_{y}F_{\rm SW}}\right)}\,\,\,,$ (1)

where $x$ and $y$ are orthogonal axis defining the sample plane and
$\sigma_{x/y}$ denote the corresponding widths of our slightly elliptical,
Gaussian laser beam on the sample. The laser spot area obtained from these
fits is then used to convert the measured pulse energy thresholds for the
appearance of reversed domains at other values of $t_{\rm Cu}$, and of non-
reversed domains within the written ones into $F_{\rm SW}$ and $F_{\rm MD}$,
respectively.

The corresponding results, shown in figure 2 for $t_{\rm Cu}\in[5,300]\,$nm,
reveal that $F_{\rm SW}$ is independent of $t_{\rm Cu}$, but that $F_{\rm MD}$
strongly increases with $t_{\rm Cu}$ in a way that resembles the increase of
femtosecond laser pulse damage thresholds of gold films with their thickness
[27]. In analogy to the fact that the damage threshold of Au is governed by
$T_{\rm e,eq}=T_{\rm melt}$, with $T_{\rm e,eq}$ and $T_{\rm melt}$ denoting
the electron temperature at the time when they reach thermal equilibrium with
the phonons and the melting point, respectively, this resemblance indicates
that multi-domain formation is also governed by a threshold condition for
$T_{\rm e}$ at longer time delays.

Figure 2:  $F_{\rm SW}$ and $F_{\rm MD}$ vs the heat sink thickness, $t_{\rm
Cu}$, for s-polarized, $750\,$nm laser pulses of $150\,$fs duration incident
at $\approx 5\,$deg. Open triangles represent results of the fits used to
determine the beam size. Insets show typical domains written at $F_{\rm SW}$
and $F_{\rm MD}$ for $t_{\rm Cu}=5$ and $300\,$nm, respectively.

We repeated the procedure outlined above to determine the impact of the laser
pulse duration on $F_{\rm SW}$ and $F_{\rm MD}$ for selected heat sink
thicknesses shown in figure 3. Throughout the investigated pulse duration
range from $250\,$fs to $10\,$ps, we find that the values of $F_{\rm SW}$
monotonously increase with $\tau_{\rm las}$ but are independent of $t_{\rm
Cu}$ (cf. panels a) and c)). A comparison of these findings to TTM
calculations with typical material parameters as published in [1, 28, 29]
shows, that both results contradict the hypothesis of a $T_{\rm
e,max}$-defined switching criterion for AO-HIS. In order to induce identical
values of $T_{\rm e,max}$ in samples with $t_{\rm Cu}=5\,$nm, the fluence of
$5\,$ps pulses would have to be about four times larger than the one of
$250\,$fs pulses – about twice as large as the measured ratio. Moreover, in
order to compensate for cooling of the electrons due to heat diffusion into
the heat sink during the excitation, $F_{\rm SW}$ would have to increase with
$t_{\rm Cu}$ if the appearance of switching would be related to $T_{\rm
e,max}$. According to the TTM, about $40\,$% larger fluences are needed to
reach the same $T_{\rm e,max}$ with $5\,$ps pulses in samples with $t_{\rm
Cu}=200\,$nm as in those with $t_{\rm Cu}=5\,$nm. In line with the work of
Gorchon et al. [1], the incompatibility of our data and TTM calculations
strongly suggests that $F_{\rm SW}$ is not directly related to $T_{\rm
e,max}$.

Figure 3:  a) $F_{\rm SW}(\tau_{\rm las})$ (open symbols/lines) and $F_{\rm
MD}(\tau_{\rm las})$ (closed symbols) for different $t_{\rm Cu}$ given in the
legend. The dashed line indicates $\tau_{\rm max}(t_{\rm Cu}=5\,$nm). b)
$F_{\rm MD}$ vs $t_{\rm Cu}$ for various $\tau_{\rm las}$ given in the legend
in ps. c) Same as b) but for $F_{\rm SW}$. All data are for $p$-polarized,
$800\,$nm pulses incident at $\approx 56\,$deg.

Now we draw our attention to the impact of $t_{\rm Cu}$ and $\tau_{\rm las}$
on $F_{\rm MD}$. As shown in Fig. 3 b), we find that $F_{\rm MD}$ increases
with $t_{\rm Cu}$ in a way that resembles exponential saturation functions,
$F_{\rm MD}=F_{0}+\Delta F\cdot(1-\exp(-t_{\rm Cu}/\lambda_{\rm Cu}))$ with
characteristic amplitudes, $\Delta F$, and lengths, $\lambda_{\rm Cu}$, that
decrease and increase with $\tau_{\rm las}$, respectively. The behavior of
$\Delta F$ is qualitatively reproduced by our TTM calculations and reflects
the changing character of heat diffusion with increasing pulse duration. While
short pulses lead to thermal diffusion by hot electrons that rapidly spreads
the absorbed energy over a range of $\lambda_{\rm th,e}\approx 230\,$nm before
electrons and phonons reach thermal equilibrium, long pulses are followed by
much slower phonon dominated diffusion that spreads the energy only over
$\lambda_{\rm th,ph}\approx 60\,$nm within $10\,$ps [27, 30]. In contrast, the
increase of $\lambda_{\rm Cu}$ with $\tau_{\rm las}$ (related to the slight
uptick of $F_{\rm MD}$ with $t_{\rm Cu}\in[300,900]\,$nm observed for long
pulse durations) cannot be explained in terms of temperature dynamics alone
and might be caused by magnetic contributions to the formation of multiple
domains [31, 32].

The demonstration of a strong increase of the multi-domain threshold with the
heat sink thickness constitutes the most important result of this work. It
proves, independent of any model, that values of $F_{\rm MD}$ reported for
non-heat sunk samples do not reflect a breakdown of the material specific
reversal mechanism due to an exceedingly high maximum electron temperature.
Instead, the temperature of insufficiently cooled electrons becomes so large
at $F_{\rm MD}$, that it leads to complete demagnetization and/or activates
dipolar field driven multi-domain formation at times longer than the electron-
phonon equilibration time [31, 32]. Moreover, it reveals that the maximum
pulse duration applicable for controlled AO-HIS, $\tau_{\rm max}$, is not
limited by accelerated demagnetization of Gd [2] but by thermally activated
multi-domain formation. As obvious from figure 3 a), $\tau_{\rm max}$ is
defined by $F_{\rm SW}(\tau_{\rm max})=F_{\rm MD}(\tau_{\rm max})$, so that an
increase of $F_{\rm MD}$ with $t_{\rm Cu}$ enlarges $\tau_{\rm max}$. The
impact of $F_{\rm MD}(t_{\rm Cu})$ on $\tau_{\rm max}$ is so strong, that
$\tau_{\rm max}$ already exceeds our experimental pulse length limit of
$10\,$ps for $t_{\rm Cu}=50\,$nm. Linear data extrapolation suggests, that
pulses as long as $25\,$ps could be used for AO-HIS in samples with $t_{\rm
Cu}=900\,$nm.

So far, we used static images of magnetization patterns resulting from the
illumination of multiple samples (with distinct $t_{\rm Cu}$) by laser pulses
of various fluences to prove that $F_{\rm MD}$ is not governed by a breakdown
of the reversal mechanism of AO-HIS. Now, we show that investigations of the
spatiotemporal magnetization changes within a single sample induced by laser
pulses of one fixed fluence might prove this fact as well. To provide such
proof, the time-resolved images need to demonstrate that laser pulse
excitation with $F_{\rm las}\geq F_{\rm MD}$ are followed by spatially
continuous magnetization reversal before the magnetization breaks up into
multiple domains. Consequently, the time-resolved approach works only for
samples where large dipolar fields and/or small magneto-crystalline
anisotropies lead to the formation multi-domains at temperatures so far below
the Curie point, that the laser heated electrons at no time cause complete
demagnetization. This implies that the multi-domain thresholds for femtosecond
laser pulse excitation of suitable samples do not exceed $F_{\rm SW}$ by much,
if at all. Since this criterion is not fulfilled for the samples investigated
so far (they exhibit $F_{\rm MD}/F_{\rm SW}\geq 3$), we demonstrate the
viability of the time-resolved approach by example of a
glass/Ta(5)/Pt(3)/$\alpha$-Gd18Dy4Co78(10)/Pt(5) sample whose multi-domain
threshold is just $\approx 29\,$% larger than $F_{\rm SW}=4.6\,$mJ/cm2 for
$s$-polarized $150\,$fs, $800\,$nm pulses incident at $\approx 5\,$deg.

Figure 4:  a) Magnetization dynamics in $\alpha$-Gd18Dy4Co78 induced by
s-polarized, $150\,$fs, $800\,$nm laser pulses of various fluences (given in
multiples of $F_{\rm MD}=5.9\,$mJ/cm2) incident at $\approx 5\,$deg. Panels b)
and c) show normalized magnetization profiles $5\,$ps and $1\,$s after the
excitation, respectively.

Both, the fluence dependent magnetization dynamics shown in figure 4a), and
the magnetization patterns measured at short and long delays given in figures
4b) and c), respectively, clearly attest that the excitation of this sample by
$150\,$fs laser pulses with $F_{\rm las}=1.12\cdot F_{\rm MD}$ is followed by
ultrafast, continuous magnetization reversal before elevated electron
temperatures in quasi-equilibrium with the phonon temperature activate the
dipolar field driven formation of multiple domains.

In conclusion we have used the thickness of Cu heat sinks to control the
electron temperature dynamics following optical excitation of adjacent GdFeCo
films by laser pulses of various duration and fluence. This enabled us to
separately study the impact of the electron temperature at short times, where
it reaches its maximum, and at longer times, where it is in quasi-equilibrium
with the phonon temperature, on the fluence thresholds for the onset of
controlled temperature induced switching and of stochastic multi-domain
formation. The measured variations of these thresholds with the heat sink
thickness and laser pulse duration evidence that neither one is solely
governed by $T_{\rm e,max}$. We have shown that $F_{\rm MD}$ corresponds to
the fluence where the temperature of insufficiently cooled electrons becomes
so large that it causes complete demagnetization and/or activates dipolar
field driven multi-domain formation at times longer than the electron-phonon
equilibration time. Based on this new insight we predict that excitation of
samples whose multi-domain threshold does not exceed the switching threshold
by much, if at all, with laser fluences slightly larger than $F_{\rm MD}$ lead
to controlled switching before the magnetization breaks up into multiple
domains. We confirmed this prediction by example of Gd18Dy4Co78, with $F_{\rm
MD}/F_{\rm SW}\approx 1.29$ and $F_{\rm las}=1.12\cdot F_{\rm MD}$. Last, but
not least, we have shown that any increase of $F_{\rm MD}$ with the heat sink
thickness lengthens the maximum pulse duration applicable for AO-HIS and
derived an upper limit of $\tau_{\rm max}\approx 25\,$ps for
Gd24(Fe90Co${}_{10})_{76}$ from linear extrapolation of our data. We expect
that thermal management by metallic heat sinks will not only pave the way for
future technological applications of AO-HIS with laser pulses emitted by cheap
laser diodes, but will also lead to the demonstration of AO-HIS in many
ferrimagnetic alloys and multi-layers that were believed to show only multi-
domain formation as already demonstrated for Gd13Dy7Co80 in [33].

## References

  * Gorchon  _et al._ [2016] J. Gorchon, R. B. Wilson, Y. Yang, A. Pattabi, J. Y. Chen, L. He, J. P. Wang, M. Li, and J. Bokor, Phys. Rev. B 94, 184406 (2016).
  * Davies  _et al._ [2020] C. Davies, T. Janssen, J. Mentink, A. Tsukamoto, A. Kimel, A. van der Meer, A. Stupakiewicz, and A. Kirilyuk, Phys. Rev. Appl. 13, 024064 (2020).
  * Wei  _et al._ [2021] J. Wei, B. Zhang, M. Hehn, W. Zhang, G. Malinovski, Y. Xu, W. Zhao, and S. Mangin, Phys. Rev. Appl. 15, 054065 (2021).
  * Graves  _et al._ [2013] C. E. Graves, A. H. Reid, T. Wang, B. Wu, S. de Jong, K. Vahaplar, I. Radu, D. P. Bernstein, M. Messerschmidt, and L. Müller et al., Nat. Mater. 12, 293 (2013).
  * Lalieu  _et al._ [2017] M. L. M. Lalieu, M. J. G. Peeters, S. R. R. Haenen, R. Lavrijsen, and B. Koopmans, Phys. Rev. B 96, 220411 (2017).
  * Radu  _et al._ [2011] I. Radu, K. Vahaplar, C. Stamm, T. Kachel, N. Pontius, H. A. Dürr, T. A. Ostler, J. Barker, R. F. L. Evans, and R. W. Chantrell et al., Nature 472, 205 (2011).
  * Steil  _et al._ [2011] D. Steil, S. Alebrand, A. Hassdenteufel, M. Cinchetti, and M. Aeschlimann, Phys. Rev. B 84, 224408 (2011).
  * Jakobs  _et al._ [2021] F. Jakobs, T. Ostler, C.-H. Lambert, Y. Yang, S. Salahuddin, R. B. Wilson, J. Gorchon, J. Bokor, and U. Atxitia, Phys. Rev. B 103, 104422 (2021).
  * Yang  _et al._ [2017] Y. Yang, R. B. Wilson, J. Gorchon, C.-H. Lambert, S. Salahuddin, and J. Bokor, Sci. Adv. 3, e1603117 (2017).
  * [10] Y. Xu, M. Deb, G. Malinowski, M. Hehn, W. Zhao, and S. Mangin, Adv. Mater. 29, 1703474.
  * Wilson  _et al._ [2017] R. B. Wilson, J. Gorchon, Y. Yang, , C.-H. Lambert, S. Salahuddin, and J. Bokor, Phys. Rev. B 95, 180409(R) (2017).
  * Atxitia  _et al._ [2013] U. Atxitia, T. Ostler, J. Barker, R. Evans, R. Chantrell, and O. Chubykalo-Fesenko, Phys. Rev. B 87, 224417 (2013).
  * Beens  _et al._ [2019] M. Beens, M. L. M. Lalieu, A. J. M. Deenen, R. A. Duine, and B. Koopmans, Phys. Rev. B 100, 220409(R) (2019).
  * Bar’yakhtar  _et al._ [2013] V. G. Bar’yakhtar, V. I. Butrium, and B. A. Ivanov, JETP Lett. 98, 289 (2013).
  * Barker  _et al._ [2013] J. Barker, U. Atxitia, T. A. Ostler, O. Hovorka, O. Chubykalo-Fesenko, and R. W. Chantrell, Sci. Rep. 3, 3262 (2013).
  * Baral and Schneider [2015] A. Baral and H. C. Schneider, Phys. Rev. B 91, 100402(R) (2015).
  * Gerlach  _et al._ [2017] S. Gerlach, L. Oroszlany, D. Hinzke, S. Sievering, S. Wiendholdt, L. Szunyogh, and U. Nowak, Phys. Rev. B 95, 224435 (2017).
  * Gridnev [2018] V. N. Gridnev, Phys. Rev. B 98, 014427 (2018).
  * Kalashnikova and Kozub [2016] A. M. Kalashnikova and V. I. Kozub, Phys. Rev. B 93, 054424 (2016).
  * Mentink  _et al._ [2012] J. H. Mentink, J. Hellsvik, D. V. Afanasiev, B. A. Ivanov, A. Kirilyuk, A. V. Kimel, O. Eriksson, M. I. Katsnelson, and T. Rasing, Phys. Rev. Lett. 108, 057202 (2012).
  * Schellekens and Koopmans [2013] A. J. Schellekens and B. Koopmans, Phys. Rev. B 87, 020407(R) (2013).
  * Wienholdt  _et al._ [2013] S. Wienholdt, D. Hinzke, K. Carva, P. M. Oppeneer, and U. Nowak, Phys. Rev. B 88, 020406(R) (2013).
  * Jakobs and Atxitia [2022] F. Jakobs and U. Atxitia, Phys. Rev. Lett. 129, 037203 (2022).
  * Atxitia  _et al._ [2014] U. Atxitia, J. Barker, R. Chantrell, and O. Chubykalo-Fesenko, Phys. Rev. B 89, 2244 (2014).
  * Raposo  _et al._ [2022] V. Raposo, F. Garcia-Sánchez, U. Atxitia, and E. Martinez, Phys. Rev. B 105, 104432 (2022).
  * Bello  _et al._ [2022] J.-L. Bello, D. Lacour, S. Migot, J. Chanbaja, S. Mangin, and M. Hehn, Appl. Phys. Lett. 121, 212402 (2022).
  * Wellershoff  _et al._ [1999] S.-S. Wellershoff, J. Hohlfeld, J. Güdde, and E. Matthias, Appl. Phys. A 69, 505 (1999).
  * Hohlfeld  _et al._ [2000] J. Hohlfeld, S.-S. Wellershoff, J. Güdde, U. Conrad, V. Jähnke, and E. Matthias, Chem. Phys. 251, 237 (2000).
  * Hopkins  _et al._ [2012] P. E. Hopkins, M. Ding, and J. Poon, J. Appl. Phys. 111, 103533 (2012).
  * Corkum  _et al._ [1988] P. B. Corkum, F. Brunel, N. K. Sherman, and T. Srinivasan-Rao, Phys. Rev. Lett. 61, 2886 (1988).
  * Ogasawana  _et al._ [2009] T. Ogasawana, N. Iwata, Y. Murakami, H. Okamoto, and Y. Tokura, Appl. Phys. Lett. 94, 162507 (2009).
  * Shieh and Kryder [1986] H.-P. D. Shieh and M. H. Kryder, Appl. Phys. Lett. 49, 473 (1986).
  * [33] W. Zhang, J. Hohlfeld, T. X. Huang, J. X. Lin, M. Hehn, Y. Le Guen, G. Malinowski, W. S. Zhao, and S. Mangin, submitted to Adv. Mater. .