# Towards Coherent Visual Storytelling with Ordered Image Attention

Tom Braude
The Interdisciplinary Center
Hertzliya, Israel
<EMAIL_ADDRESS>
Idan Schwartz
Technion
Haifa, Israel
<EMAIL_ADDRESS>
Alexander Schwing
University of Illinois at Urbana-Champaign
Champaign, IL, US
<EMAIL_ADDRESS>
Ariel Shamir
The Interdisciplinary Center
Hertzliya, Israel
<EMAIL_ADDRESS>

###### Abstract

We address the problem of visual storytelling, _i.e_., generating a story for
a given sequence of images. While each sentence of the story should describe a
corresponding image, a coherent story also needs to be consistent and relate
to both future and past images. To achieve this we develop ordered image
attention (OIA). OIA models interactions between the sentence-corresponding
image and important regions in other images of the sequence. To highlight the
important objects, a message-passing-like algorithm collects representations
of those objects in an order-aware manner. To generate the story’s sentences,
we then highlight important image attention vectors with an Image-Sentence
Attention (ISA). Further, to alleviate common linguistic mistakes like
repetitiveness, we introduce an adaptive prior. The obtained results improve
the METEOR score on the VIST dataset by 1%. In addition, an extensive human
study verifies coherency improvements and shows that OIA and ISA generated
stories are more focused, shareable, and image-grounded.

##  1 Introduction

Visual Storytelling (VST) [24, 14] – the task of generating a story based on a
sequence of images – goes beyond a basic understanding of visual scenes and
can be applied in many real-world scenarios, _e.g_., to support the visually
impaired. Moreover, VST reflects on the creative ability of intelligent
systems. Although similar in concept to other cognitive tasks such as image
captioning and visual question answering, VST differs as it requires to reason
over a _sequence_ of images while simultaneously ensuring coherence across
multiple generated sentences. To achieve this, VST methods need to address two
major challenges: the first is visual and relates to grounding the story’s
text to the images. The second is linguistic and relates to the quality of the
story. Both challenges can be described in terms of coherency: the story
should be coherent by itself, and coherent with the images.

Prior research on VST started to address the aforementioned challenges. Early
works expand captioning [33, 38, 6], focusing sentence generation mainly on
the current image [9, 34]. This limits the ability to incorporate complex
semantic information, which is necessary for visual reasoning. Prior work also
makes limited use of temporal dependence and history, _e.g_., sentences that
have already been generated are not used. Consequently, the output lacks
narrative consistency and is prone to linguistic errors such as
_repetitiveness_ and _incoherence_ [22]. To mitigate these issues, later works
strive to generate more meaningful stories via adversarial and reinforcement
learning [36, 13], which remain delicate to train.

Figure 1: We propose Ordered Image Attention (OIA) to form the structure of a
sentence and to encourage coherency. Each row shows the spatial attention of
the five images created when generating a specific sentence. We find important
objects by collecting directional interactions. The relative order to the
sentence-corresponding image determines the connection type, illustrated as
the blue and orange edges for preceding and proceeding connections. The
attended images’ border indicates the image attention importance formed by the
Image-Sentence Attention (ISA). _E.g_., red indicates a high attention score,
meaning the image is essential for generating that sentence. Our model
performs this step for all five images in parallel, creating a total of 25
spatial attention maps, that are fed into the decoder to create the sentences
in order.

Importantly, images are not independent. For example, if the first image in a
sequence shows a protest, the model may want to focus on signs in later
images. Conversely, if the last image shows a ring on a finger, then the model
should pay attention to wedding-related objects and activities in the
preceding images. This is important for VST because sentences are created per
image but are part of a story. Hence, objects that the model is focusing on in
one image should be conditioned on the selection in other images.

To do this we develop a novel model which (1) implicitly reasons over objects,
activities, and their temporal dependencies in each image; and which (2)
improves the coherency of the narrative. To reason over objects and activities
in each image, _i.e_., to understand their dependencies and their temporal
ordering, we introduce _ordered image attention_ (OIA). As illustrated in Fig.
1, for each image, OIA accumulates representation information from objects
detected within the corresponding image into an attended image representation.
Importantly, accumulation factors depend on whether the image precedes or
succeeds the image for which we are currently generating the sentence, which
permits to establish an order. The attended image representations are
subsequently summarized into a context embedding via an Image-Sentence
Attention (ISA) unit, before being used for sentence decoding.

In addition, to alleviate common linguistic mistakes like repetitiveness and
to promote coherence in the story, we incorporate information from the story
generated up to the current sentence into the sentence generation decoder.
Specifically, the decoding strategy decays the probability of a word if it has
already been used in the story. The decoder also maintains a separate prior
over the output probability distribution, independent from the language
generation unit. This prior is based on counts of the words that were already
predicted in the story. Both the prior, and the Recurrent Neural Net (RNN)
decoder output are combined to predict the next word in the sentence.

Empirical results on the challenging VIST dataset [14] demonstrate that the
proposed method generates stories with an improved narrative quality. The
method outperforms prior state-of-the-art by 1% on the METEOR score. Examples
of stories generated by the approach are shown in Fig. 1. We also present a
user study demonstrating the advantage of the model in terms of coherency.

##  2 Related Work

Vision+Language has been an active area of research for many years, addressing
tasks such as image/video captioning, paragraph generation, and visual
question answering. We briefly review those related areas in the following.

###  2.1 Image Captioning

Bernard _et al_. [3] first explored annotating images with text. Since then,
image/video captioning has seen a surge of research activity. Initial work
utilized pre-trained image embeddings from a CNN network. The success of
attention mechanisms for language translation quickly transferred to image
captioning as well [38]. Later work leveraged advances in object detection and
proposed a bottom-up/top-down attention approach to attend to specific objects
in the image instead of fixed spatial regions [1]. Different from image
captioning, for visual storytelling, both story coherency and visual grounding
are important.

Figure 2: Our architecture for Visual Storytelling synthesis.

###  2.2 Multimodal Attention

Multimodal problems are characterized by input data that comes from different
domains, _e.g_., visual and linguistic. This raises two challenges: 1) how to
model interactions between different domains, and 2) how to manage the large
input data. Considering those challenges, attention has been a prominent tool
as it models interactions to select the important elements. In early work, Xu
_et al_. [38] used interaction-based attention with the image at each caption
generation step. This idea was later extended to visual question answering
[37]. To imitate multi-step reasoning, Yang _et al_. [40] stacked attention
modules sequentially. Later, many works concentrated on better vector-fusion
modeling [7, 16, 4, 42]. Importantly, Lu _et al_. [21] suggested attending to
the visual and textual modalities separately. Afterward, Kim _et al_. [15]
proposed a bilinear module that efficiently generates attention for every
pair. Following Lu _et al_. [21], Schwartz _et al_. [27, 28] suggested a
general framework that extends attention to any number of utilities via local
and interaction-based factors. We improve upon those ideas by suggesting an
ordered attention. This ensures that interaction modeling is affected by the
image position in a sequence.

###  2.3 Visual Storytelling

Huang _et al_. [14] introduced the Visual Storytelling task. Initially,
Gonzalez _et al_. [9] adapted work by Vinyals _et al_. [33] used for
captioning. Kim _et al_. [17] presented a Seq2Seq [29] approach with a
decoding sampling strategy aimed to reduce the amount of repetition based on a
word list. We improve their strategy by using a data-driven approach,
penalizing each word differently based on its average counts. Wang _et al_.
[36] employ adversarial learning to improve output stories. Huang _et al_.
[13] utilize a reinforcement learning (RL) approach based on inter-image
relations. Later works by Li _et al_. [19] and Zhang _et al_. [43] rely on
preprocessing the data to better ground visual elements to the text while Yang
_et al_. [39] and Hsu _et al_. [12] enrich the data with an external word
common-sense knowledge graph. Our approach captures inter-image relations via
ordered attention and is trained in an end-to-end manner alleviating the
computational drawbacks of preprocessing or RL. Recently, state-of-the-art
results were obtained by generating scene graphs for each image in the
sequence [35]. Conversely, our image representations are dependant on all the
images in the sequence.

##  3 Method

The goal of visual storytelling is to generate a story, composed of $N$
_ordered_ sentences $\\{y_{s}|1\leq s\leq N\\}$, given an _ordered_ sequence
of images $I=\\{I_{s}|1\leq s\leq N\\}$. Each sentence
$y_{s}=(y_{s,0},\dots,y_{s,t},\dots)$ is composed of words $y_{s,t}\in{\cal
Y}$ from vocabulary ${\cal Y}$.

The order in which the images are given is essential as it defines the plot
line of the story. The story should be focused, _i.e_., each sentence should
be related to the remainder of the story. Importantly, the sentences should
form a coherent body of text describing the set of images, and not only a set
of related information. For instance, the story “The church was beautiful. The
bride and groom walk down the aisle. The cake was amazing.” is less coherent
than: “We went to the church for the wedding today. The bride and groom were
excited for the day. Both cut the cake together.”

Overview: To address this challenge, we develop the model illustrated in Fig.
2. It infers conditional probabilities $p^{\prime}(y_{s,t}|y_{s,t-1},c_{s})$
for the $t$-th word $y_{s,t}\in{\cal Y}$ in sentence $y_{s}$ given the
previous word $y_{s,t-1}$ and the context embedding $c_{s}$ for sentence $s$.
The context embedding $c_{s}$ summarizes region representations $r_{i,k}$ of
all $K$ object regions across all $N$ images $I_{i}$ ($i\in[1,N]$,
$k\in[1,K]$) via Ordered Image Attention (OIA) (Sec. 3.1) and Image-Sentence
Attention (ISA) (Sec. 3.2). Specifically, when generating sentence $s$, OIA
computes an attended image representation $a_{i}^{s}$ for every image $I_{i}$
by attending to the $K$ region representations $r_{i,k}$ (Sec. 3.1). These
attended image representations $a_{i}^{s}$ are subsequently summarized into
the context embedding $c_{s}$ via an image-sentence attention (Sec. 3.2).

Below we first discuss computation of the attended image representation
$a_{i}^{s}$ (Sec. 3.1), before detailing computation of the context embedding
$c_{s}$ (Sec. 3.2) and computation of the conditional probabilities
$p^{\prime}(y_{s,t}|y_{s,t-1},c_{s})$ (Sec. 3.3).

Figure 3: Illustration of Ordered Image Attention. Each node represents an
image attention belief. For each sentence, we connect all the images with the
sentence-corresponding image. The relative position to this image determines
whether the connection is modeled with the $\Psi_{\text{bwd}}$ factor (for
preceding images) or the $\Psi_{\text{fwd}}$ factor (for subsequent images).
We infer the attention belief by collecting interactions and local object
information within the image. We use scalars to calibrate the importance of
each factor. In total, we generate 25 attention maps, one per image for every
sentence.

###  3.1 Ordered Image Attention (OIA)

Ordered Image Attention (OIA) is designed to 1) form a structure across
ordered images and to 2) select the relevant objects per image. For this we
model preceding and proceeding interactions separately using different
attention factors. We calibrate each factor’s importance with trainable
scalars, which forms a graph of dependencies between the images. For each
sequence of $N$ images, the model infers a total of $N^{2}$ attention maps,
one per image for each sentence. We detail this module next.

####  3.1.1 Attention Belief

For each image $I_{i}=\\{r_{i,1},\ldots r_{i,K}\\}$ we consider a set of $K$
regions, represented by their feature vectors $r_{i,k}\in\mathbb{R}^{d}$,
where $d$ is the objects’ embedding dimension. Suppose we are currently
generating sentence $y_{s}$ ($1\leq s\leq N$). To do this we first compute an
attended image representation $a_{i}^{s}$ as follows

$\displaystyle a_{i}^{s}=\sum_{k=1}^{K}b_{i,k}^{s}r_{i,k},$ (1)

where $b_{i,k}^{s}\geq 0$ is the attention belief highlighting the importance
of the $k$-th object in the $i$-th image when generating the $s$-th sentence.
Importantly, for every image $I_{i}$ we require $b_{i,k}^{s}$ to be a valid
probability distribution, _i.e_., we also enforce
$\sum_{k=1}^{K}b_{i,k}^{s}=1$ $\forall s,i$.

The object attention belief $b_{i,k}^{s}$ is dependent on all the input data,
_i.e_., other objects and images. To avoid complex computation, we factorize
the belief $b_{i,k}^{s}$ into two pairwise dependencies that preserve the
order, and a local term. For the pairwise terms we use $\mu_{j\to
i}^{\text{bwd}}$, which is a message from a preceding image $I_{j}$, or
$\mu_{j\to i}^{\text{fwd}}$, which is a message from a subsequent image
$I_{j}$. We also use $\mu_{i\to i}$ for self-messages. Additionally, we
include a local factor $\Psi_{i}(r_{i,k})$ that considers the object
representation. Unlike the messages mentioned before, the local factor does
not rely on interactions with other objects. We aggregate all the messages
along with the local factor as illustrated in Fig. 3. For normalization we
employ a softmax.

Formally we compute the attention belief $b_{i,k}^{s}$ by distinguishing three
cases. If $i=s$ we have

$\displaystyle b_{i,k}^{s}$ $\displaystyle\propto$
$\displaystyle\exp(\alpha_{i}^{s}\Psi_{i}(r_{i,k})+\alpha_{i,i}^{s}\mu_{i\to
i}(r_{i,k})+$ $\displaystyle\sum_{j<i}\alpha_{i,j}^{s}\mu_{j\to
i}^{\text{bwd}}(r_{i,k})+\sum_{j>i}\alpha_{i,j}^{s}\mu_{j\to
i}^{\text{fwd}}(r_{i,k})).$

If $i<s$ we use

$\displaystyle b_{i,k}^{s}$ $\displaystyle\propto$
$\displaystyle\exp(\alpha_{i}^{s}\Psi_{i}(r_{i,k})+$
$\displaystyle\alpha_{i,i}^{s}\mu_{i\to i}(r_{i,k})+\alpha_{i,s}^{s}\mu_{s\to
i}^{\text{bwd}}(r_{i,k})).$

If $i>s$ we obtain

$\displaystyle b_{i,k}^{s}$ $\displaystyle\propto$
$\displaystyle\exp(\alpha_{i}^{s}\Psi_{i}(r_{i,k})+$
$\displaystyle\alpha_{i,i}^{s}\mu_{i\to i}(r_{i,k})+\alpha_{i,s}^{s}\mu_{s\to
i}^{\text{fwd}}(r_{i,k})).$

In all three cases
$\alpha_{i}^{s},\alpha_{i,i}^{s},\alpha_{i,j}^{s}\in\mathbb{R}$ are scalars
used to calibrate the importance of different messages for a given sentence.
These scalars form a dependency structure between images for each of the
generated sentence indices. Intuitively, when we generate the first sentence,
the attention belief might depend more on subsequent images, to correctly
identify the story event, _e.g_., a wedding, a parade, _etc_. Thus, the
scalars will promote interaction with later images. An analysis of these
scalars is provided in the appendix. Next, we define the different types of
messages.

Figure 4: Illustration of ISA. The attention selects the attended image
representation per sentence. We model interactions between attended images of
the same sentence to compute each image’s importance. Note, each node
represents a sentence attention belief over the attended images.

####  3.1.2 Pairwise Messages and Factors

A message aggregates interaction scores from an image to an object. The three
messages $\mu_{j\to i}^{\text{bwd}},\mu_{j\to i}^{\text{fwd}}$ and $\mu_{i\to
i}(r_{i,k})$ are computed as follows:

$\mu_{j\to
i}^{\text{bwd}}(r_{i,k})=\sum_{k^{\prime}=1}^{K}\Psi_{\text{bwd}}(r_{i,k},r_{j,k^{\prime}}),$
(5) $\mu_{j\to
i}^{\text{fwd}}(r_{i,k})=\sum_{k^{\prime}=1}^{K}\Psi_{\text{fwd}}(r_{i,k},r_{j,k^{\prime}}),~{}\text{and}$
(6) $\mu_{i\to
i}(r_{i,k})=\sum_{k^{\prime}=1}^{K}\Psi_{i,i}(r_{i,k},r_{i,k^{\prime}}).$ (7)

Importantly, these messages collect three different types of order-dependent
interaction factors: (1) A backward image interaction, namely
$\Psi_{\text{bwd}}(r_{i,k},r_{j,k^{\prime}})$. This interaction models
relations to the preceding $j$-th image in the sequence. (2) A forward image
interaction, namely $\Psi_{\text{fwd}}(r_{i,k},r_{j,k^{\prime}})$. This
interaction models relations to the subsequent $j$-th image in the sequence.
(3) The self interaction factor, namely
$\Psi_{i,i}(r_{i,k},r_{i,k^{\prime}})$, which takes into account interactions
between objects within the image. We formally define the different factors
next.

Interaction factors: A commonly used practice to capture interactions across
attention mechanisms is to first embed the elements into a joint Euclidean
space followed by a dot-product [31, 27, 8, 28]. While we follow the same
practice, we define three types of interaction factors to preserve the order.
Consider two objects, $r_{i,k}\in I_{i}$ from the sentence-corresponding image
and $r_{j,k^{\prime}}\in I_{j}$ from the interacting image. We describe three
types of interactions: for interactions with subsequent images (_i.e_., $j>i$)
we use

$\Psi_{\text{fwd}}(r_{i,k},r_{j,k^{\prime}})\\!=\\!\left(\frac{L_{\text{fwd}}r_{i,k}}{\|L_{\text{fwd}}r_{i,k}\|_{2}}\right)^{\\!\\!\top}\\!\\!\left(\frac{R_{\text{fwd}}r_{j,k^{\prime}}}{\|R_{\text{fwd}}r_{j,k^{\prime}}\|_{2}}\right).$
(8)

For interactions with preceding images (_i.e_., $j<i$) we use

$\Psi_{\text{bwd}}(r_{i,k},r_{j,k^{\prime}})\\!=\\!\left(\\!\frac{L_{\text{bwd}}r_{i,k}}{\|L_{\text{bwd}}r_{i,k}\|_{2}}\\!\right)^{\\!\\!\top}\\!\\!\left(\\!\frac{R_{\text{bwd}}r_{j,k^{\prime}}}{\|R_{\text{bwd}}r_{j,k^{\prime}}\|_{2}}\\!\right).$
(9)

For interactions within the image (_i.e_., $j=i$) we have

$\Psi_{i,i}(r_{i,k},r_{i,k^{\prime}})\\!=\\!\left(\frac{L_{i,i}r_{i,k}}{\|L_{i,i}r_{i,k}\|_{2}}\right)^{\\!\\!\top}\\!\\!\left(\frac{R_{i,i}r_{i,k^{\prime}}}{\|R_{i,i}r_{i,k^{\prime}}\|_{2}}\right).$
(10)

Note,
$L_{\text{fwd}},R_{\text{fwd}},L_{\text{bwd}},R_{\text{bwd}},L_{i,i},R_{i,i}\in\mathbb{R}^{d\times
d}$ are trainable shared weights across the entire image sequence. Also, the
object from the sentence-corresponding image will always be on the left side
of the factor equation. Thus, the factor embeddings preserve the order.

Local factor: Differently from the previous interactions the following factor
captures how important an object is based solely on the object representation.
Given an object $r_{i,k}\in I_{i}$, we define the local factor as,

$\displaystyle\Psi_{i}(r_{i,k})=v^{\top}\operatorname{ReLU}(Vr_{i,k}),$ (11)

where $v\in\mathbb{R}^{d},V\in\mathbb{R}^{d\times d}$ are trainable weights.

###  3.2 Image-Sentence Attention (ISA)

In a next step we summarize the attended image representations $a_{i}^{s}$
produced by OIA to compute the context embedding $c_{s}$ for the sentence $s$
that we wish to generate. For this we use the Image-Sentence Attention (ISA)
unit. It picks the relevant image context for generating the specific
sentence. Formally we obtain the context embedding via

$c_{s}=\sum_{i=1}^{N}\hat{b}_{s,i}a_{i}^{s},$ (12)

where attention factors

$\displaystyle\hat{b}_{s,i}\propto\exp\left(\hat{\alpha}_{s}\hat{\Psi}_{i}(a_{i}^{s})+\hat{\alpha}_{s,s}\hat{\mu}_{s\to
s}(a_{i}^{s})\right),$ (13)

and where $\hat{\alpha}_{s},\hat{\alpha}_{s,s}\in\mathbb{R}$ are scalars. To
avoid spurious correlations between sentences, we consider only self
interactions and a local factor. This is illustrated in Fig. 4. The self-
message of the attended image representation $a_{i}^{s}$ is

$\displaystyle\hat{\mu}_{s\to
s}(a_{i}^{s})=\sum_{j=1}^{N}\hat{\Psi}(a_{i}^{s},a_{j}^{s}).$ (14)

Finally, the self and local factors are defined with a different set of
weights following Eq. (10) and Eq. (11) respectively.

Method | M | B-1 | B-2 | B-3 | B-4 | R | C | Img Feat
---|---|---|---|---|---|---|---|---
seq2seq [14] | 31.4 | - | - | - | 3.5 | - | 6.84 | FC
h-attn-rank [41] | 33.9 | - | 29.8 | - | - | 29.8 | 7.4 | FC
Contextualize, Show & Tell [9] | 34.4 | 60.1 | 36.5 | 21.1 | 12.7 | 29.2 | 7.1 | FC
AREL [36] | 35.0 | 63.8 | 39.1 | 23.2 | 14.1 | 29.5 | 9.4 | FC
KnowledgeableStoryteller [39] | 35.2 | 66.4 | 39.2 | 23.1 | 12.8 | 29.9 | 12.1 | FC
HSRL [13] | 35.2 | - | - | - | 12.3 | 29.5 | 8.4 | Spatial
StoryAnchor [43] | 35.5 | 65.1 | 40.0 | 23.4 | 14.0 | 30.0 | 9.9 | FC
SGVST [35] | 35.8 | 65.1 | 40.1 | 23.8 | 14.7 | 29.9 | 9.8 | F-RCNN
Ours (ResNet) | 36.3 | 66.3 | 41.5 | 23.7 | 14.5 | 30.0 | 9.8 | Spatial
Ours |  36.8$\pm$0.1 |  68.4$\pm$0.7 |  42.7$\pm$0.3 |  25.2$\pm$0.2 |  15.3$\pm$0.2 |  30.2$\pm$0.1 | 10.1$\pm$0.2 | F-RCNN

Table 1: Quantitative results on the VIST dataset for METEOR, BLEU-1$\ldots$4,
ROUGE-L and CIDEr. The primary metric is METEOR. The ‘Img Feat’ column
describes the pretrained image features. All models utilize a ResNet [10]
backbone except CS&T which employs an Inception v3 model [30]. FC and Spatial
refer to features extracted from the penultimate layer and the preceding one
accordingly. F-RCNN are bottom up features [1].

###  3.3 Story Decoding

The goal at each timestep of decoding is to compute the conditional
probability $p(y_{s,t}|y_{s,t-1},c_{s})$ where $y_{s,t}\in{\cal Y}$ is the
$t$-th word in sentence $y_{s}$, ${\cal Y}$ is the vocabulary and $c_{s}$ is
the context embedding detailed in Sec. 3.2. For this we use a GRU recurrent
unit, tasked with generating probabilities over the vocabulary conditioned on
the context embedding $c_{s}$ and the previously generated token $y_{s,t-1}$:

$\displaystyle p(y_{s,t}=w|y_{s,t-1},c_{s})$ $\displaystyle\\!\\!\\!\propto$
$\displaystyle\\!\\!\\!\exp(\beta_{s,t}\cdot g_{w}(y_{s,t-1},h_{s,t-1},c_{s})$
(15) $\displaystyle+(1-\beta_{s,t})\cdot f_{w}(\phi_{s,t})),$

where $g_{w}$ is the output of a GRU unit for the word $w$. We set the GRU
hidden dimension to $d$. $h_{s,t-1}\in\mathbb{R}^{d}$ is the hidden state at
timestep $t-1$ for sentence $s$. $f:\mathbb{R}^{|{\cal
Y}|}\to\mathbb{R}^{|{\cal Y}|}$ is a learned prior over the vocabulary based
on a bag-of-words prior histogram $\phi_{s,t}$, which we describe in the next
paragraph. The purpose of $f$ is to reduce text repetitions. $f_{w}$ denotes
the value of $f$ for a word $w$. We also incorporate a calibration gate
$\beta_{s,t}:\mathbb{R}^{d}\to[0,1]$ for functions $f$ and $g$ using

$\displaystyle\beta_{s,t}=\sigma\left(v_{\beta}^{\top}\operatorname{tanh}(G_{g}h_{s,t}+G_{f}W_{1}(\phi_{s,t}))\right).$
(16)

Here, $G_{g}\in\mathbb{R}^{d\times d}$ and $G_{f}\in\mathbb{R}^{\gamma\times
d}$ are trained projections of the GRU hidden state and the bottleneck layer
respectively, $v_{\beta}\in\mathbb{R}^{d}$ are learned weights and $\sigma$ is
the sigmoid function. $W_{1}$ is obtained from the prior as discussed next.

Bag-of-words (BOW) prior: Remembering history during storytelling permits to
stay on topic and advance the story in the desired direction. Although quite
intuitive, mimicking this ability is not trivial. _E.g_., most approaches for
VST generate all the sentences in parallel. Converting the parallel sentence
generation into a sequential one implies a major computational overhead during
training.

To address this, we propose a simple yet effective learnable framework that
does not require sequential training while still exploiting information found
in prior sentences. The history is represented via a bag-of-words histogram
$\phi_{s,t}$, which includes all words that have been used until timestep $t$
for the $s$-th sentence. During training, we initialize $\phi_{s,t=0}$ with
the ground truth history counts found in the previous $s-1$ sentences. We
update the statistics at each timestep with the predicted word
$y_{s^{\prime},t}$ for $s^{\prime}<s$, and produce the next state of the
counter $\phi_{s,t+1}$. At inference we generate sentences sequentially and
update $\phi_{s,t}$ with the predicted words. $\phi_{s,t}$ is fed through a
shallow bottleneck network to obtain the prior $f$, composed of two layers
$W_{1}\in\mathbb{R}^{|{\cal Y}|\times\gamma}$ and
$W_{2}\in\mathbb{R}^{\gamma\times|{\cal Y}|}$ without activation, where
$\gamma$ is the bottleneck dimension:

$\displaystyle f(\phi_{s,t})=W_{2}(W_{1}(\phi_{s,t})).$ (17)

Also note the use of $W_{1}(\phi_{s,t})$ in the gate (Eq. (16)).

Intra-repetition regularization: To regularize intra-repetitions, we decay the
probability of previously used words during sentence generation. A critical
aspect of this approach is to exclude words that appear frequently in the
language (_e.g_., was, were, am). For this we pre-process the training set to
calculate the average story frequency $\rho({w})$ of a word $w$ via
$\rho({w})=\frac{\\#\ \text{appearances of word~{}}w}{\\#\
\text{stories~{}}w\text{~{}was used}}$. The final count for word $w$ at
timestep $t$ is calculated as
$\phi_{s,t}^{\prime}(w)=\text{max}[0,(\phi_{s,t}(w)-\rho(w)+1)]$. Intuitively,
a word will not be penalized before it is used more than the prior belief
average $\rho(w)$. The final probability for word $w$ being used is given by

$\displaystyle
p^{\prime}(y_{s,t}=w|y_{s,t-1},c_{s})=\frac{p(y_{s,t}=w|y_{s,t-1},c_{s})}{\pi\cdot\phi_{s,t}^{\prime}(w)+1},$
(18)

where $\pi\geq 0$ is a constant hyper-parameter. A penalty of 2 proved to work
best on the validation set.

##  4 Results

Dataset: To train and test the model we use the VIST dataset [14]. This
dataset is composed of stories. Each story has 5 images and $N=5$
corresponding sentences. All images were collected from Flickr albums.
Sequences of images belong to the same album. Each image sequence is annotated
with 5 ground-truth reference stories. On average, around 2.5 stories are
based on the images, and the rest are rewrites. The overall numbers are 40,098
training stories, 4,988 validation stories, and 5,050 test stories.

Training Setup: We extracted the image features using a pre-trained F-RCNN
model with a ResNet152 backbone [10, 26, 1]. We set the number of extracted
objects $K=36$. Bounding box coordinates were normalized between 0 and 1.
Words that appear less than 3 times in the training set are represented by an
<UNK> token. The vocabulary size is 12,210 words. Word representations were
initialized using GloVe embeddings [25]. We set the decay parameter $\pi=2$
and the image representation dimension $d=512$. We set the dropout parameter
to 0.3. We use cross-entropy loss to maximize likelihood of ground-truth
stories. At decoding time we employ a beam search algorithm, with beam width
set to 3. We use Adam [18] optimizer with a learning-rate of 0.0004, which is
decayed by a factor of 0.8 if the validation score (METEOR) does not improve
after 4 epochs. The total amount of trainable parameters is 13,092,194.
Training converges after $\sim$20 epochs. Each epoch needs 20 minutes on an
Nvidia V100 GPU.

###  4.1 Quantitative analysis

Evaluation metrics: As suggested by the creators of VIST [14], METEOR [2]
correlates best with human judgement. Following their example, we use METEOR
as the primary metric. We also compute BLEU [23], ROUGE [20], and CIDEr [32]
and compare to prior work where available. For evaluation we use the
evaluation script of Yu _et al_. [41]111http://github.com/lichengunc/vist_eval
\- Codebase for commonly used evaluation scripts..

Comparison to state-of-the-art: In Tab. 1 we compare the method to recent
baselines. Early methods did not take into account visual-spatial information,
which harms the performance (_e.g_., 35.5% _vs_. 36.8% on METEOR) [14, 36, 9].
Wang _et al_. [35] utilize image representations similar to our approach but
do not consider relations between different images, resulting in a 1% drop on
METEOR, showing that ordered structure encoding with OIA is beneficial. SGVST
and StoryAnchor [41, 43] use different methods for mapping the image sequence
to distinct topics. Differently, our approach is trained end-to-end. Further,
our image representations are dependant on all the images in the sequence.
Notably, unlike our method, SGVST uses scene graphs. This requires an
additional model pre-trained on external scene graph data. Finally, Yang _et
al_. [39] utilize an external commonsense dataset to enrich the input. Their
CIDEr score is significantly higher, yet this improvement does not translate
to all other metrics. The approach improves upon the current state-of-the-art
by a margin (36.8% _vs_. 35.8% on METEOR). Note, the ROUGE-L metric is based
on finding the longest subsequence matched to human generated stories.
However, this score is almost identical for all prior works, indicating that
this metric doesn’t capture story generation improvements. We also report the
performance with spatial ResNet152 features [10], which outperforms the state-
of-the-art as well. This shows that the method is stable irrespective of image
features.

Model | M | B-4 | R | C | #Params
---|---|---|---|---|---
attention
w/o OIA | 36.0 | 14.1 | 30.0 | 8.4 | 11M
w/o ISA | 35.9 | 14.2 | 29.9 | 9.3 | 11M
w/o attention | 35.8 | 13.6 | 29.7 | 7.2 | 11M
no-direction | 36.1 | 14.5 | 28.9 | 8.4 | 12M
decoding
w/o rep. regularization | 36.2 | 14.5 | 29.8 | 8.7 | 13M
w/o count norm | 36.4 | 14.6 | 29.9 | 9.4 | 13M
w/o BOW prior | 36.4 | 14.5 | 30.0 | 9.7 | 13M
Transformer | 36.7 | 15.7 | 30.0 | 9.9 | 13M
Ours | 36.8 | 15.3 | 30.2 | 10.1 | 13M

Table 2: Components ablation analysis.

Local | Self | Directional | M | B-1 | B-2 | B-3 | B-4 | R | C
---|---|---|---|---|---|---|---|---|---
$\times$ | ✓ | ✓ | 36.2 | 67.4 | 42.4 | 24.2 | 14.5 | 30.0 | 9.3
✓ | $\times$ | ✓ | 36.0 | 67.8 | 42.3 | 24.2 | 14.4 | 29.8 | 9.2
✓ | ✓ | $\times$ | 35.9 | 67.6 | 42.2 | 24.0 | 14.2 | 29.9 | 8.5
✓ | ✓ | ✓ | 36.8 | 68.4 | 42.7 | 25.2 | 15.3 | 30.2 | 10.1

Table 3: Factor ablation analysis.

Ablation study: In Tab. 2 we show the importance of different components via
an ablation study. In ‘w/o OIA,’ we replace the OIA module (Sec. 3.1) with
simple averaging of the $K$ object representations of image $I_{i}$, resulting
in a 0.8% drop on METEOR. Similarly, in ‘w/o ISA,’ we replace the ISA unit
(Sec. 3.2) with averaging, leading to a 0.9% drop on METEOR. In ‘w/o
attention,’ we removed both OIA and ISA, which dropped the METEOR score to
35.8%. For the method referred to as ‘no-direction,’ we use the same factor
for preceding and proceeding interaction (_i.e_.,
$L_{\text{bwd}}=L_{\text{fwd}}$ and $R_{\text{bwd}}=R_{\text{fwd}}$). Here,
METEOR results drop by 0.7%. Hence, ordered interactions are beneficial. Next,
we assess the decoding components (Sec. 3.3). We first remove the intra-
repetition regularization (_i.e_., $\rho({w})$), which causes METEOR score to
drop by 0.6%. Removing the popular words count ($\phi^{\prime}_{s,t}$),
results in a 0.4% drop on METEOR. The METEOR score drops by 0.4% when we
remove the BOW prior. Last, we replace the GRU decoding layer with a
Transformer, which did not change results a lot.

In Tab. 3 we show an ablation analysis of the different factors used in OIA.
We found that each factor contributes to the model’s performance, and the
directional factors (_i.e_., $\Psi_{\text{fwd}}$ and $\Psi_{\text{bwd}}$) have
the biggest impact.

In Tab. 5, we show the ability to reduce repetitions. As proposed by Bertoldi
_et al_. [5], text repetitiveness is measured by the repetition rate of non-
singleton n-grams within each story. In our experiment, we use up to 4-grams.
The use of intra-repetition regularization reduces text repetition (0.14 to
0.04). Combined with the trainable bag-of-words prior module, we further
improve this measure (0.008 _vs_. 0.14). We also report sentence
repetitiveness, _i.e_., the average number of repeated sentences in a story.

(a) Human-like property comparison.

(b) Coherence property comparison.

Figure 5: Human evaluation to compare properties. Method | Focused | Coherent | Share | Human-like | Grounded | Detailed
---|---|---|---|---|---|---
AREL | 3.49 | 3.18 | 3.18 | 3.26 | 3.32 | 3.15
Ours | 3.67 | 3.52 | 3.20 | 3.56 | 3.54 | 3.32
GT | 3.72 | 3.57 | 3.34 | 3.64 | 3.56 | 3.53
Table 4: Human evaluation results for rating survey (scores are between 1-5).

###  4.2 Human Evaluation

The subjective nature of the VST task calls for a human evaluation. We use a
sample of 150 image sequences and test different story qualities by asking 3
MTurk annotators to rank or compare them to other methods. We compare our
results to the AREL baseline since none of the more recent baselines are
publicly available. Note that we also compare coherency against a model
without ordered-factors, which already improves upon the prior state-of-the-
art.

In Fig. 5(a) we provide the results when asking annotators to pick the most
human-like story. We use the majority vote to decide the best model per story.
The generated stories outperform the AREL baseline (73.87% _vs_. 22.53%).
Surprisingly, in many cases, the annotators found the generated stories to be
more human-like than the ground truth stories (41% _vs_. 48.57%). In Fig.
5(b), we assess coherency. An important aspect of our work are the directional
factors for coherency. To validate their effectiveness, we compared to a model
that does not incorporate direction into the attention representation (_i.e_.,
we use the same factor for preceding and proceeding interactions). The
comparison shows a significant coherency improvement (64.2% _vs_. 28.7%).
Also, a comparison against the AREL baseline demonstrates a more significant
improvement (70.24% _vs_. 25.32%).

To further evaluate the quality of the stories, we follow the criteria set by
the Visual Storytelling Challenge222http://visionandlanguage.net/workshop2018
and conduct a survey where judges are asked to rate six categories between
1-5: _1\. Focused_ : the story contains information that is “naturally”
relevant to the rest of the story; _2\. Coherence_ : the sentences in the
story are related and consistent; _3\. Share_ : the inclination to share the
story; _4\. Human-like_ : the story was likely written by a human; _5\.
Grounded_ : the story directly reflects concrete entities in the image; and
_6\. Detailed_ : the story provides an appropriate level of detail. To obtain
the final score, we average the annotators’ scores per sample, followed by
averaging across the entire sample set. From Tab. 4 we observe: the model
improved on all the criteria compared to the AREL model. Importantly, the
generated stories are comparable to the ground-truth stories, indicating
success in reducing the shortcomings found in prior methods. Nonetheless, the
level of detail is still lacking, supporting the observation of Holtzman  _et
al_. [11] that current decoding strategies tend to generate well-formed yet
somewhat generic text.

Figure 6: An illustration of an image sequence along with three different
stories generated by: (1) AREL baseline [36], (2) No History: a model without
intra-repetition regularization and BOW prior (see Sec. 3.3); and (3) With
History: the final model. Repeated sentences are highlighted with a yellow
colored marker. Repeated words in a sentence are emphasized in red color.

Model | Text Rep. | Sent. Rep.
---|---|---
AREL [36] | 0.16 | 0.4
BOG prior | Intra-repetition reg. |
No | No | 0.14 | 0.33
Yes | No | 0.10 | 0.18
No | Yes | 0.04 | 0.04
Yes | Yes | 0.008 | 0.0

Table 5: Story generation ablation analysis.

###  4.3 Qualitative evaluation

In Fig. 6, we show the ability of the method in reducing repetitions. We
observe the AREL baseline to repeat the same sentences, for example, “…had a
great time at…”. We also observe this repetitiveness when we remove the bag-
of-words prior and the intra-sentence regularization (_i.e_., No History
column). Nevertheless, the method remains on topic, _i.e_., family in the
pool.

In Fig. 7 we sketch the attention maps along with the generated story. The
first sentence, “We went to the mountains,” sets the theme for the story,
which requires the processing of subsequent images. Notably, the ISA module
picked the proceeding images. In contrast, for the second sentence, the
attention focuses mostly on the second image resulting in a description of the
lake observed exclusively in this image. The third sentence relates to the
scenery. Hence the attention focuses on preceding and proceeding images.

Figure 7: Illustration of OIA and ISA attention maps, the ground-truth story
and the final generated story. Each row corresponds to a story sentence and
shows objects OIA highlights. The attended images’ border specifies the
relevancy to sentence generation, from red (important) to blue (not
important).

##  5 Conclusion

We present a novel approach for VST, which encourages coherency of generated
story. We incorporate structure between images with a new attention method
that selects the important objects in an ordered image sequence. Human
evaluation and quantitative analysis demonstrate that the approach outperforms
existing methods. Further, we perform ablation and qualitative analysis to
show effectiveness.

## References

  * [1] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang.  Bottom-up and top-down attention for image captioning and visual question answering.  In CVPR, 2017.
  * [2] Satanjeev Banerjee and Alon Lavie.  Meteor: An automatic metric for mt evaluation with improved correlation with human judgments.  In ACL, 2005.
  * [3] Kobus Barnard, Pinar Duygulu Sahin, David A. Forsyth, Nando de Freitas, David M. Blei, and Michael I. Jordan.  Matching words and pictures.  JMLR, 2003.
  * [4] Hedi Ben-Younes, Rémi Cadene, Matthieu Cord, and Nicolas Thome.  Mutan: Multimodal tucker fusion for visual question answering.  In ICCV, 2017.
  * [5] Nicola Bertoldi, Mauro Cettolo, and Marcello Federico.  Cache-based online adaptation for machine translation enhanced computer assisted translation.  In MT Summit, 2013.
  * [6] Xinlei Chen and C. Lawrence Zitnick.  Mind’s eye: A recurrent visual representation for image caption generation.  In CVPR, 2015.
  * [7] Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach.  Multimodal compact bilinear pooling for visual question answering and visual grounding.  In EMNLP, 2016.
  * [8] Peng Gao, Zhengkai Jiang, Haoxuan You, Pan Lu, Steven CH Hoi, Xiaogang Wang, and Hongsheng Li.  Dynamic fusion with intra-and inter-modality attention flow for visual question answering.  In CVPR, 2019.
  * [9] Diana Gonzalez-Rico and Gibran Fuentes Pineda.  Contextualize, show and tell: A neural visual storyteller.  In Storytelling Workshop, NAACL, 2018.
  * [10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2015.
  * [11] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi.  The curious case of neural text degeneration.  In ICLR, 2020.
  * [12] Chao-Chun Hsu, Zi-Yuan Chen, Chi-Yang Hsu, Chih-Chia Li, Tzu-Yuan Lin, Ting-Hao Huang, and Lun-Wei Ku.  Knowledge-enriched visual storytelling.  In AAAI, 2020.
  * [13] Qiuyuan Huang, Zhe Gan, Asli Çelikyilmaz, Dapeng Wu, Jianfeng Wang, and Xiaodong He.  Hierarchically structured reinforcement learning for topically coherent visual story generation.  In AAAI, 2018.
  * [14] Ting-Hao Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross B. Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell.  Visual storytelling.  In NAACL, 2016.
  * [15] Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang.  Bilinear attention networks.  In NeurIPS, 2018.
  * [16] Jin-Hwa Kim, Kyoung-Woon On, Woosang Lim, Jeonghee Kim, Jung-Woo Ha, and Byoung-Tak Zhang.  Hadamard product for low-rank bilinear pooling.  In ICLR, 2017.
  * [17] Taehyeong Kim, Min-Oh Heo, Seonil Son, Kyoung-Wha Park, and Byoung-Tak Zhang.  Glac net: Glocal attention cascading networks for multi-image cued story generation.  In CoRR, 2018.
  * [18] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In CoRR, 2014.
  * [19] Jiacheng Li, Haizhou Shi, Siliang Tang, Fei Wu, and Yueting Zhuang.  Informative visual storytelling with cross-modal rules.  In MM, 2019.
  * [20] Chin-Yew Lin.  Rouge: A package for automatic evaluation of summaries.  In ACL, 2004.
  * [21] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh.  Hierarchical question-image co-attention for visual question answering.  In NeurIPS, 2016.
  * [22] Yatri Modi and Natalie Parde.  The steep road to happily ever after: an analysis of current visual storytelling models.  In Workshop on Shortcomings in Vision and Language, NAACL, 2019\.
  * [23] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.  Bleu: a method for automatic evaluation of machine translation.  In ACL, 2001.
  * [24] Cesc C. Park and Gunhee Kim.  Expressing an image stream with a sequence of natural sentences.  In NeurIPS, 2015.
  * [25] Jeffrey Pennington, Richard Socher, and Christopher D Manning.  Glove: Global vectors for word representation.  In EMNLP, pages 1532–1543, 2014.
  * [26] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In IEEE PAMI, 2015.
  * [27] Idan Schwartz, Alexander G. Schwing, and Tamir Hazan.  High-order attention models for visual question answering.  In NeurIPS, 2017.
  * [28] Idan Schwartz, Seunghak Yu, Tamir Hazan, and Alexander G Schwing.  Factor graph attention.  In CVPR, 2019.
  * [29] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le.  Sequence to sequence learning with neural networks.  In NeurIPS, 2014.
  * [30] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In CVPR, 2015.
  * [31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In NeurIPS, 2017.
  * [32] Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh.  Cider: Consensus-based image description evaluation.  In CVPR, 2014.
  * [33] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan.  Show and tell: A neural image caption generator.  In CVPR, 2014.
  * [34] Bairui Wang, Lin Ma, Wei Zhang, Wenhao Jiang, and Feng Zhang.  Hierarchical photo-scene encoder for album storytelling.  In AAAI, 2019.
  * [35] Ruize Wang, Zhongyu Wei, Piji Li, Qi Zhang, and Xuanjing Huang.  Storytelling from an image stream using scene graphs.  In AAAI, 2019.
  * [36] Xin Wang, Wenhu Chen, Yuan fang Wang, and William Yang Wang.  No metrics are perfect: Adversarial reward learning for visual storytelling.  In ACL, 2018.
  * [37] Huijuan Xu and Kate Saenko.  Ask, attend and answer: Exploring question-guided spatial attention for visual question answering.  In ECCV, 2016.
  * [38] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio.  Show, attend and tell: Neural image caption generation with visual attention.  In ICML, 2015.
  * [39] Pengcheng Yang, Fuli Luo, Peng Chen, Lei Li, Zhiyi Yin, Xiaodong He, and Xu Sun.  Knowledgeable storyteller: A commonsense-driven generative model for visual storytelling.  In IJCAI, 2019.
  * [40] Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alexander J Smola.  Stacked attention networks for image question answering, 2015.
  * [41] Licheng Yu, Mohit Bansal, and Tamara L. Berg.  Hierarchically-attentive rnn for album summarization and storytelling.  In EMNLP, 2017.
  * [42] Zhou Yu, Jun Yu, Chenchao Xiang, Jianping Fan, and Dacheng Tao.  Beyond bilinear: Generalized multimodal factorized high-order pooling for visual question answering.  In NeurIPS, 2018.
  * [43] Bowen Zhang, Hexiang Hu, and Fei Sha.  Visual storytelling via predicting anchor word embeddings in the stories.  In ICCV. Workshop on Closing the Loop Between Vision and Language, 2020.


# UniParma at SemEval-2021 Task 5: Toxic Spans Detection Using CharacterBERT
and Bag-of-Words Model

Akbar Karimi
&Leonardo Rossi

University of Parma
{akbar.karimi, leonardo.rossi<EMAIL_ADDRESS>
&Andrea Prati

###### Abstract

With the ever-increasing availability of digital information, toxic content is
also on the rise. Therefore, the detection of this type of language is of
paramount importance. We tackle this problem utilizing a combination of a
state-of-the-art pre-trained language model (CharacterBERT) and a traditional
bag-of-words technique. Since the content is full of toxic words that have not
been written according to their dictionary spelling, attendance to individual
characters is crucial. Therefore, we use CharacterBERT to extract features
based on the word characters. It consists of a CharacterCNN module that learns
character embeddings from the context. These are, then, fed into the well-
known BERT architecture. The bag-of-words method, on the other hand, further
improves upon that by making sure that some frequently used toxic words get
labeled accordingly. With a $\sim$4 percent difference from the first team,
our system ranked $36^{th}$ in the competition. The code is available for
further research and reproduction of the
results111https://github.com/IMPLabUniPr/UniParma-at-semeval-2021-task-5.

##  1 Introduction

The user generated digital content is increasing rapidly every second of the
day. This can include some toxic language whose detection can be difficult due
to the complexities of human languages. We address this problem by
participating in SemEval Workshop 2021 Task 5 (Pavlopoulos et al., 2021).

In many cases, the data, which are considered to be toxic, contain words that
have not been written in their standard forms. There might also be a lot of
misspelling or letter replacements. In addition, usually the words that are
considered to be the most offensive are bleeped which makes them difficult to
be recognized if we use a model which learns the content representation based
on the words. Apart from word related issues, the context also plays a crucial
role in the meaning that a word conveys since words in different contexts can
have various meanings.

Therefore, in order to cope with these issues, we opt for a recently pre-
trained language model which has been trained on character level.
CharacterBERT (El Boukkouri et al., 2020) is a deep neural network model that
has been pre-trained on Wikipedia and OpenWebText (Gokaslan and Cohen, )
corpora using the BERT architecture (Devlin et al., 2019) with an addition of
a character-aware Convolutional Neural Network (CNN) (Kim et al., 2016; Peters
et al., 2018). BERT-based models have now become pervasive in many different
natural language processing tasks such as reading comprehension (Xu et al.,
2019), named entity recognition (Liang et al., 2020), sentiment analysis
(Karimi et al., 2020), and language understanding (Jiao et al., 2020) as well
as similar ones to toxic language detection such as propaganda detection
(Jurkiewicz et al., 2020). While the BERT model is beneficial in extracting
the contextual information from the text on the word level, the character-
aware CNN attends to the individual letters which helps in dealing with out-
of-vocabulary, unknown and rare words.

In addition to using a deep language model for detecting toxic language, we
employ a very simple Bag-of-Words model that can achieve a close performance
to that of the deep model. By building a dictionary of toxic words from the
training data and by taking into account their frequency and ratio of
toxicity, we come up with a simple model that performs as closely as about $2$
percent difference in performance to the deep model’s result. Moreover, we
improve the results of CharacterBERT by combining it with the output of a
version of the Bag-of-Words model.

spans |  text
---|---
[8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39] |  Another violent and aggressive immigrant killing a innocent and intelligent US Citizen…. Sarcasm
[0, 1, 2, 3] |  Damn, a whole family. Sad indeed.
[7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17] |  What a knucklehead. How can anyone not know this would be offensive??
Table 1: Three examples from the training set

##  2 System Description

Our system consists of four main stages namely, pre-processing, applying
CharacterBERT, applying Bag-of-Words model, and finally combining the results
of the two models. We describe each of these stages in the following
subsections.

###  2.1 Pre-processing

The training dataset consists of rows of various lengths and an array of
character spans indicating their toxic parts. Each row can contain several
sentences. Table 1 shows three examples of the training data.

We approach the task of toxic spans detection as a sequence labelling task
where each word of the input row is classified into one of the predefined
classes. We define three classes of {B, I, O}, meaning that each word can be
the first word (B) of a set of continuous toxic words, in between (I), or not
toxic (O). Therefore, in order for both of our models to be able to process
these inputs, we first need to break the rows into words and label them as one
of the above-mentioned classes. This was carried out by simply splitting each
input row at the $space$ characters. Then, after creating a dataset that has
been labeled on the word level, we can use it as the input of our models. The
same is done for the Bag-of-Words model with a difference in treating the
bleeped words which is described in Subsection 2.3.

###  2.2 CharacterBERT

CharacterBERT model is almost identical to the well-known BERT model with a
difference in initial embedding. In the general BERT model, words are broken
into pieces and the embeddings for these word pieces are computed. In
CharacterBERT, however, words are divided into letters or characters. Then,
using CNN modules the embeddings are computed on the character level (Figure
1). This makes the network extract features on the lowest level, making it
suitable for contexts which contain many unseen vocabulary terms such as
misspelled words or technical jargon. After the initial character-aware CNN
layer, there is the BERTbase architecture which contains $12$ layers (blocks)
of Transformer (Vaswani et al., 2017) with the hidden size of $768$ and $12$
attention heads. The final layer representations are converted into logits
using a fully connected layer after which a Softmax layer is applied to
extract the token’s (word’s) class.

Figure 1: The difference between the BERT and CharacterBERT models is the way
they compute the initial embeddings. The former uses word-piece embeddings
while the latter uses character embeddings. Figure taken from El Boukkouri et
al. (2020). Figure 2: Performance of the Bag-of-Words model on validation set.
The frequencies and ratios are the minimum thresholds that specify whether or
not to consider a word toxic.

###  2.3 Bag-of-Words Model

This model is a simple script of fewer than 80 lines of code. However, its
performance on the Toxic Spans Detection task can get very close to the
CharacterBERT model which has millions of parameters. In this model, by
examining the training set, we first build a dictionary of toxic words with
their frequency. Table 2 presents the top ten words of this dictionary in
terms of frequency.

Word | Frequency
---|---
stupid | 973
idiot | 557
idiots | 353
stupidity | 223
ignorant | 190
dumb | 157
moron | 147
fool | 141
pathetic | 138
crap | 121
Table 2: Top 10 toxic words in the training set

Then, we locate the words from the toxic dictionary in each sentence of the
test set. If the word is found and its frequency as well as its toxicity ratio
in the training set are higher than certain values, it is labeled as toxic.
This ratio which we call toxicity ratio (defined below) along with the term
frequency are the only parameters of the Bag-of-Words model.

$\textbf{toxicity ratio}=\frac{\text{labeled as toxic frequency}}{\text{total
frequency}}$

The test dataset also contains words that are bleeped. Since these words can
be considered toxic with a high certainty (otherwise they would not be
bleeped), we extract them separately from the test set and label them directly
as toxic.

###  2.4 Combining the Two Models

In order to get the improved version of the toxic language labeling, the union
of the spans detected by the bag-of-words model and that of CharacterBERT is
taken. The results will improve if there are words labeled correctly with the
Bag-of-Words model that are not in the output for CharacterBERT. This can be
achieved by specifying a high toxicity ratio for a word to be labeled as
toxic. Also, the wrongly labeled tokens should not be too many since it can
have a negative effect on the F1 score. Therefore, the frequency with which a
toxic word appears should be somewhat high. Striking a balance between these
two parameters can help improve the output of CharacterBERT.

##  3 Experiments and Results

###  3.1 Performance of CharacterBERT

We ran the experiments for the general domain CharacterBERT with its default
setting on a GPU (GeForce RTX 2070) which had 8GB of memory. We specified
batch sizes of $4$ for both training and testing and fine-tuned it on the
toxic data only for one epoch which produces an F1 score of $65.13$. It is
worth noting that more training did not improve the performance.

###  3.2 Analysis of the Bag-of-Words model

In order to experiment with the Bag-of-Words model, we divide the original
training set into a resized training set with $7000$ sentences and a
validation set with $939$ sentences which were taken from the end of the
original training set. Then, we find the best parameters on the validation set
and using those parameters on the test data, we get a performance of almost
$63$ percent which is only $2$ percent smaller than our deep model. Figure 2
shows this model’s performance for its two parameters on the validation set.
One parameter represents the minimum frequency with which a toxic word appears
in the resized training set and the other one is its minimum toxicity ratio in
the resized training data.

We can see from Figure 2 that the best results are achieved when the minimum
frequency is $20$ and the minimum ratio is $0.3$ or $0.4$. Since a larger
ratio can be a sign of more toxicity, we choose $0.4$ as the ratio and a
frequency of $20$ as the thresholds with which we apply the model on the test
set. This gives an F1 score of $62.79$ percent (Table 3) which is not that
much below the result of the deep model.

We can also see from Table 3 that although combining the output of the Bag-of-
Words model with that of CharacterBERT improves the results a little bit, it
is still is not as significant as the first version. In the first version of
the Bag-of-Words model, which was found during our primary experiments, the
minimum word frequency is $40$ and the minimum toxicity ratio is $0.7$. With
these parameters, only 10 words are selected from the training set. The
frequency and toxicity ratio of these words can be seen in Table 4.

Model | F1
---|---
CharacterBERT | 65.13
BoW (v1) | 51.75
BoW with best parameters (v2) | 62.79
CharacterBERT + BoW (v2) | 65.87
CharacterBERT + BoW (v1) | 66.72
Table 3: Comparing results of the proposed models. The boldfaced one is the submitted version. BoW: Bag-of-Words model. Word | Frequency | Toxicity Ratio
---|---|---
stupid | 973 | 0.78
idiot | 557 | 0.84
idiots | 353 | 0.81
stupidity | 223 | 0.77
moron | 147 | 0.71
idiotic | 98 | 0.74
hypocrite | 75 | 0.88
shit | 56 | 0.72
scum | 52 | 0.70
hypocrites | 44 | 0.76
Table 4: Words selected as the toxic words with minimum frequency of 40 and
minimum toxicity ratio of 0.7 (BoW (v1))

Although the performance of this version is a lot lower than the second
version (v2) of the BoW model, it helps to improve the performance of
CharacterBERT. The reason for this behavior can be attributed to the fact that
models with higher thresholds both in terms of frequency and toxicity ratio
tend to output more certain results, albeit fewer words than the ones that
should be labeled as toxic. Therefore, many toxic words that are less probable
are not extracted and F1 score drops.

Figure 3: Heatmap of the results (F1 scores) with different values of term
frequency and toxicity ratio before combining with CharacterBERT Figure 4:
Heatmap of the results (F1 scores) with different values of term frequency and
toxicity ratio after combining with CharacterBERT

Looking at Figure 3, we can see that, indeed, the best parameters from the
experiments on the validation set (ratios $0.3$ and $0.4$ with frequencies
$10$ and $20$) yield some of the best results on the test set. However, when
these results are combined with the output of the CharacterBERT, we see that
the higher the toxicity ratio the better the results (Figure 4) until $0.7$
which gives the maximum improvement. The $0.8$ ratio makes the predictions
still a little better but $0.9$ does not affect them since the words that are
labeled as toxic with this certainty have most probably been found also by
CharacterBERT.

##  4 Conclusion

We described the system we utilized to detect toxic language. In our approach,
we first fine-tune CharacterBERT, a character-level pre-trained language
model, on the toxic training data. Then using a simple bag-of-words model, we
further improve the results of this system. The Bag-of-Words model labels the
words based on their frequency and the ratio of toxicity in the training data.
We showed that this model, although extremely simple, gives a close
performance to that of CharacterBERT with millions of parameters.

## References

  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * El Boukkouri et al. (2020) Hicham El Boukkouri, Olivier Ferret, Thomas Lavergne, Hiroshi Noji, Pierre Zweigenbaum, and Jun’ichi Tsujii. 2020.  CharacterBERT: Reconciling ELMo and BERT for word-level open-vocabulary representations from characters.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 6903–6915.
  * (3) Aaron Gokaslan and Vanya Cohen.  Openwebtext corpus.
  * Jiao et al. (2020) Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020.  TinyBERT: Distilling BERT for natural language understanding.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings_ , pages 4163–4174.
  * Jurkiewicz et al. (2020) Dawid Jurkiewicz, Łukasz Borchmann, Izabela Kosmala, and Filip Gralinski. 2020\.  ApplicaAI at semeval-2020 task 11: On RoBERTa-CRF, span CLS and whether self-training helps them.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 1415–1424.
  * Karimi et al. (2020) Akbar Karimi, Leonardo Rossi, and Andrea Prati. 2020.  Improving BERT performance for aspect-based sentiment analysis.  _arXiv preprint arXiv:2010.11731_.
  * Kim et al. (2016) Yoon Kim, Yacine Jernite, David Sontag, and Alexander Rush. 2016.  Character-aware neural language models.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 30.
  * Liang et al. (2020) Chen Liang, Yue Yu, Haoming Jiang, Siawpeng Er, Ruijia Wang, Tuo Zhao, and Chao Zhang. 2020.  Bond: Bert-assisted open-domain named entity recognition with distant supervision.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 1054–1064.
  * Pavlopoulos et al. (2021) John Pavlopoulos, Léo Laugier, Jeffrey Sorensen, and Ion Androutsopoulos. 2021\.  Semeval-2021 task 5: Toxic spans detection (to appear).  In _Proceedings of the 15th International Workshop on Semantic Evaluation_.
  * Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2227–2237.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , pages 6000–6010.
  * Xu et al. (2019) Hu Xu, Bing Liu, Lei Shu, and S Yu Philip. 2019.  BERT post-training for review reading comprehension and aspect-based sentiment analysis.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2324–2335.