# Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for
Trajectory Prediction

Hao Zhou, Dongchun Ren, Xu Yang, Mingyu Fan, Hai Huang Hao Zhou is with the
National Key Laboratory of Science and Technology of Underwater Vehicle,
Harbin Engineering University, Harbin, China, and also with the State Key
Laboratory of Management and Control for Complex System, Institute of
Automation, Chinese Academy of Sciences, Beijing, China.Dongchun Ren is with
the Research Center for Autonomous Vehicles, Meituan, Beijing, China.Xu Yang
is with the State Key Laboratory of Management and Control for Complex System,
Institute of Automation, Chinese Academy of Sciences, Beijing, China.Mingyu
Fan is with the College of Computer Science, Wenzhou University, Wenzhou,
China, and also with the Research Center for Autonomous Vehicles, Meituan,
Beijing, China.Hai Huang is with the with the National Key Laboratory of
Science and Technology of Underwater Vehicle, Harbin Engineering University,
Harbin, China.

###### Abstract

Pedestrian trajectory prediction is a key technology in many applications such
as video surveillance, social robot navigation, and autonomous driving, and
significant progress has been made in this research topic. However, there
remain two limitations of previous studies. First, with the continuation of
time, the prediction error at each time step increases significantly, causing
the final displacement error to be impossible to ignore. Second, the
prediction results of multiple pedestrians might be impractical in the
prediction horizon, i.e., the predicted trajectories might collide with each
other. To overcome these limitations, this work proposes a novel trajectory
prediction method called CSR, which consists of a cascaded conditional
variational autoencoder (CVAE) module and a socially-aware regression module.
The cascaded CVAE module first estimates the future trajectories in a
sequential pattern. Specifically, each CVAE concatenates the past trajectories
and the predicted points so far as the input and predicts the location at the
following time step. Then, the socially-aware regression module generates
offsets from the estimated future trajectories to produce the socially
compliant final predictions, which are more reasonable and accurate results
than the estimated trajectories. Moreover, considering the large model
parameters of the cascaded CVAE module, a slide CVAE module is further
exploited to improve the model efficiency using one shared CVAE, in a slidable
manner. Experiments results demonstrate that the proposed method exhibits
improvements over state-of-the-art method on the Stanford Drone Dataset (SDD)
and ETH/UCY of approximately 38.0% and 22.2%, respectively.

###### Index Terms:

sliding sequence prediction, time variant social-aware rethinking, trajectory
prediction.

††publicationid: pubid: 0000–0000/00$00.00 © 2021 IEEE

##  I Introduction

Figure 1: The variations of the average displacement error of each frame for
the PECNet, the proposed CSR and the proposed S-CSR on the SDD dataset.

The prediction of future pedestrian trajectories according to their past
trajectories is a key problem for many applications such as autonomous driving
[1, 2, 3], robotic systems [vivacequa2017self, 4], and surveillance system [5,
6]. For example, in autonomous driving scenario, a accurate prediction of
pedestrians trajectories is required before the vehicle can plan a safe and
effective trajectory; otherwise, a car accident may occur.

A key challenge for pedestrian trajectory prediction is the high degree of
uncertainty caused by two factors, i.e., the inherently multimodal attribute
and the complex social interactions of pedestrians. In Figure 1, the
uncertainties of one state-of-the-art method PECNet [7] are visualized using
the average displacement error (ADE) between the ground truth and predictions.
The visualization reveals that the degree of uncertainty increases over the
prediction horizon. This is expected, as the movement patterns and social
interactions change over time; thus, it is difficult to predict the long-term
trajectories using the past trajectories. To cope with the uncertainty
problem, numerous methods [7, 8, 9] have been proposed in recent years. The
core notion of these methods is to first model the target prediction, and then
to predict the future trajectory conditioned on the predicted target. Despite
their competitive performance, the prediction of the target point, which has a
tremendous influence on the final prediction, is inherently a difficult
problem that has the maximum degree of uncertainty. For example, two different
target points will lead to the generation of two completely different
trajectories.

In Figure 1, the uncertainty can also be analyzed from another perspective,
namely that the first prediction time step has the minimum degree of
uncertainty; thus, it can be easily captured by the past trajectory and has
the best prediction accuracy. This is also expected because the first
prediction time step has a movement pattern similar to that of the past
trajectories. Accordingly, in this paper, interest has been placed in the
development of a model that can sequentially predict the following time step
using the updated past trajectory as the input. Recurrent Neural Networks
(RNN)-based sequential methods [10, 11, 12, 13, 14] are similar to the idea
explored in this study. They use the hidden states of the predicted points
before the prediction of the next point as the input. However, the hidden
state feature of the decoder cannot explicitly encode the predicted points as
the past trajectories.

This work presents a novel cascaded conditional variational autoencoder (CVAE)
with socially-aware rethinking method, as illustrated in Figure 2. The
proposed method consists of two modules, namely the cascaded CVAE module and
the socially-aware regression module. In the cascaded CVAE module, unshared
CVAEs are used to sequentially predict the future points of different time
steps using the updated past trajectories as the input. The past trajectory
after each prediction is updated by concatenating it with the newly predicted
point at the current time. Via this strategy, the predicted points are treated
as pseudo-labels to decrease the degree of prediction uncertainty of the next
point, thereby improving the long-term trajectory prediction performance. The
average displacement error of the proposed method is also visualized in Figure
1, from which it is evident that the proposed method is characterized by the
decreases of the average displacement errors of long-term future points.

Moreover, the improvement is also partially attributed to the socially-aware
regression module. The future trajectories predicted by the cascaded CVAE
module may be socially unacceptable because the CVAE can not model
interactions among pedestrians. Thus, the socially-aware regression module is
proposed to refine the predicted trajectories via social interactions. This
module first extracts features of the past trajectories and the predicted
trajectories by two encoders. Then the two features are concatenated and fed
to a social attention mechanism to extract the interaction feature. Finally,
the decoder decodes the interaction feature to produce the offsets of the
future trajectories, which are then used to refine the future trajectories
predicted by the cascaded CVAE module.

In the cascaded CVAE module, $\delta$ unshared CVAEs are used to predicted
$\delta$ future points using incremental inputs. Thus, the proposed module has
large model parameters and slow inference speed. Actually, the predictions of
different future points fit the independent and identically distributed. Thus,
one CVAE is enough to predict $\delta$ future points. Based on this
observation, a more lightweighted slide CVAE module is introduced to adopt one
shared CVAE to predict future points using the updated past trajectory of
fixed length, in a slidable manner, as illustrated in Figure 5(c). Moreover,
the proposed slide CVAE module is integrated with the socially-aware
regression module resulting in an more efficiency end-to-end trajectory
predicition method, called the S-CSR. The average displacement error of the
S-CSR is also visualized in Figure 1, from which it is indicates that the
slide CVAE module can futher improve the prediction accuracy of the long-term
future points.

The contributions are summarized as follows:

  * •

A cascaded CVAE with socially-aware rethinking method is proposed for
pedestrian trajectory prediction. The proposed method consists of the cascaded
CVAE module or slide CVAE module and the socially-aware regression module.

  * •

The proposed cascaded CVAE module uses unshared CVAEs to sequentially predict
the future points using the updated past trajectories as the input.

  * •

The proposed socially-aware regression module extracts interaction features to
generate offsets to refine the future trajectories predicted by the cascaded
CVAE module.

  * •

The proposed slide CVAE module uses one shared CVAE to predict the future
points using the updated past trajectory of fixed length, in a slidable
manner.

  * •

The results of extensive experiments demonstrate that the proposed CSR and
S-CSR surpass state-of-the-art methods by a large margin on two benchmark
datasets.

Figure 2: The overview of the proposed CSR model. First, the cascaded CVAE
module estimates the future trajectory in a sequential manner. Specifically,
each CVAE concatenates the past trajectory and the predicted points so far as
the input and predicts the location at the following time step. Then, the
socially-aware regression module extracts interaction features to refine the
predicted trajectory.

##  II Related Work

###  II-A Direct Prediction Methods

The direct strategy involves the utilization of one model that is capable of
predicting the entire future trajectory in a one-shot manner. Most trajectory
prediction methods are characterized by direct prediction. For example, early
works adopted the Bayesian network method [15], the Gaussian process
regression method [16], and the kinematic method [17] to directly predict the
future sequence. However, these method did not consider the interactions
between agents, and usually failed in crowded scenarios.

Recently, with the development of deep learning, convolutional neural network
(CNNs) and multilayer perceptron (MLPs) have been used in trajectory
prediction. For example, GRIP [18] uses a temporal graph to model the
interactions and a temporal CNN to predict the future sequences. VectorNet
[19] utilizes self-attention to aggregate interactions and the MLP to predict
future sequences. Social-STGCN [20] and AST-GNN [21] capture interactions
using a graph attention (GAT) and predict future sequences using a temporal
CNN.

Albert it is simple and efficient, the direct strategy uses only the past
trajectory for prediction, but has no opportunity to extract useful
information from the predicted future points. In contrast, the proposed method
uses the predicted points to update the past trajectory, thereby improving the
prediction accuracy of the following time step.

###  II-B Recursive Prediction Methods

The recursive strategy involves the use of a one-step model multiple times, in
which the predictions for the prior time steps are used for the prediction of
the following time step. RNN models including long short-term memory (LSTM)
[22] and gated recurrent unit (GRU) [23] have been widely adopted in the
recursive prediction methods. For example, S-LSTM [10] uses LSTM to extract
the past trajectories, and then aggregates the interactions of different
pedestrians in the hidden state using a social pooling mechanism. CF-LSTM [14]
improves LSTM by employing a cascaded-feature that can simultaneously capture
location and velocity information. S-GAN [11] extends S-LSTM by using GAN to
generate multimodal predictions. SoPhie [12] and PIF [13] use a CNN to extract
scene features and LSTM to extract motion features, and then adopt LSTM to
predict scene-compliant trajectories. Trajectron++ [24] introduces a modular
graph-structured recurrent model that can integrate agent dynamics and
heterogeneous data into trajectory prediction. Although the hidden state of
the predicted poins has been adopted to assist in the prediction of the
following time step, it does not explicitly model the predicted points as the
past trajectories.

###  II-C Target-conditioned Methods

The target-conditioned strategy first predicts the target points and then
generates the future trajectories conditioned on the targets. For example,
PECNet [7] first predicts distant trajectory endpoints using a CVAE and then
generates multimodal predictions conditioned on the endpoints. TNT [8]
improves PECNet by introducing a scoring and selection stage that ranks and
selects multimodal predictions with likelihood scores. Y-net [9] further
improves PECNet by iteratively predicting intermediate waypoints and
trajectories. The performance of target-conditioned methods is easily affected
by the predicted targets, however, target prediction is inherently a difficult
problem with a high degree of uncertainty.

##  III Methodology

Pedestrian trajectory prediction is a task to predict the future trajectories
of pedestrians based on their past trajectories. First, it is assumed that
videos of pedestrians walking are preprocessed to obtain the spatial
coordinates of pedestrians. For example,
$p_{i}^{t}=\left(x_{i}^{t},y_{i}^{t}\right)$ represents the coordinate of the
$i$-th pedestrian at the $t$-th time step. Then, the coordinates of each
pedestrian in the scene are classified into the past and future trajectories
according to two default values, namely the observation horizon $\tau$ and the
prediction horizon $\delta$. The definitions of the past trajectory
$\mathbf{X}_{i}$ and the future trajectory $\mathbf{Y}_{i}$ are as follows:

$\mathbf{X}_{i}=\left\\{\left(x_{i}^{t},y_{i}^{t}\right)|\forall
t\in\left\\{1,2,...,\tau\right\\}\right\\},$ (1)
$\mathbf{Y}_{i}=\left\\{\left(x_{i}^{t},y_{i}^{t}\right)|\forall
t\in\left\\{\tau+1,\tau+2,...,\tau+\delta\right\\}\right\\}.$ (2)

Finally, given the past trajectories $\left\\{\mathbf{X}_{i}|\forall
i\in\left\\{1,2,...,N\right\\}\right\\}$ of pedestrians in scene, the goal is
to generate future predictions $\left\\{\hat{\mathbf{Y}}_{i}|\forall
i\in\left\\{1,2,...,N\right\\}\right\\}$ that minimize the average
displacement error with the ground truth future trajectories
$\left\\{\mathbf{Y}_{i}|\forall i\in\left\\{1,2,...,N\right\\}\right\\}$,
where $N$ represents the number of pedestrians in a scene. The definition of
$\hat{\mathbf{Y}}_{i}$ is as follows:

$\hat{\mathbf{Y}}_{i}=\left\\{\left(\hat{x}_{i}^{t},\hat{y}_{i}^{t}\right)|\forall
t\in\left\\{\tau+1,\tau+2,...,\tau+\delta\right\\}\right\\}.$ (3)

To generate reliable predictions, a pedestrian trajectory prediction method
that consists of a cascaded CVAE module and a socially-aware regression module
is proposed, as shown in Figure 2. Because the direct prediction of long-range
future trajectory $\mathbf{Y}_{i}$ has a high degree of uncertainty, a
cascaded CVAE module is first designed to predict the future points in a
sequential manner using the updated past trajectoies as the input. Then,
considering that the CVAE is unable to model social interactions, a socially-
aware regression module is proposed to further refine the predicted future
trajectories using social interaction knowledge.

Figure 3: The illustration of the cascaded CVAE unit. The CVAE uses the
updated past trajectory and ground truth future point to train a model for
multimodal future point prediction. The dots and circles respectively
represent the predicted future points and the ground truth future points. Red
dotted lines denote the layers utilized only during training.

###  III-A Cascaded CVAE Module

The prediction of the future trajectories is inherently ambiguous and has a
high degree of uncertainty. Thus, it is difficult to learn a deterministic
function $f$ that directly maps the past trajectories
$\left\\{\mathbf{X}_{i}|\forall i\in\left\\{1,2,...,N\right\\}\right\\}$ to
the future trajectories $\left\\{\mathbf{Y}_{i}|\forall
i\in\left\\{1,2,...,N\right\\}\right\\}$.

However, as described in the Introdction, the degrees of the uncertainty of
the future points are increase with the continuation of time, and the first
future point has the minimum degree of uncertainty. According to this
phenomenon, the use of cascaded CVAEs is proposed to decompose the prediction
of the future trajectories into the prediction of the future points using the
updated past trajectories, in a sequential manner, as illustrated in Figures 2
and 5(b).

The detailed structure of each cascaded CVAE is illustrated in Figure 3. The
CVAE infers a distribution of future points based on the updated past
trajectory. The training of the CVAE requires two inputs, namely the updated
past trajectory $\widetilde{\mathbf{X}}_{i}^{t}$ and the ground point
$p_{i}^{t}=\left(x_{i}^{t},y_{i}^{t}\right)$ of pedestrian $i$ at future time
step $t$. The updated past trajectory consists of the past trajectory and the
predicted future points before time step $t$ using the concatenation
operation, such that:

$\widetilde{\mathbf{X}}_{i}^{t}=Concat\left(\mathbf{x}_{i},\left\\{\left(\hat{x}_{i}^{s},\hat{y}_{i}^{s}\right)|\forall
s\in\left\\{\tau+1,\tau+2,...,t-1\right\\}\right\\}\right).$ (4)

The updated past trajectory $\widetilde{\mathbf{X}}_{i}^{t}$ and the ground
truth future point $p_{i}^{t}$ are first encoded using two MLP encoders
$E_{upast}$ and $E_{point}$, as follows:

$\mathbf{f}_{upast}=E_{upast}\left(\widetilde{\mathbf{X}}_{i}^{t}\right),$ (5)
$\mathbf{f}_{point}=E_{point}\left(p_{i}^{t}\right).$ (6)

Then, the two feature representations $\mathbf{f}_{upast}$ and
$\mathbf{f}_{point}$ are concatenated together and fed to the MLP encoder
$E_{latent}$ to generate parameters
$\left(\bm{\mu}_{i}^{t},\bm{\sigma}_{i}^{t}\right)$ of the latent
distribution, such that:

$\left(\bm{\mu}_{i}^{t},\bm{\sigma}_{i}^{t}\right)=E_{latent}\left(Concat\left(\mathbf{f}_{upast},\mathbf{f}_{point}\right)\right).$
(7)

Finally, the latent variable $\mathbf{z}_{i}^{t}$ is randomly sampled from
distribution $\mathcal{N}\left(\bm{\mu}_{i}^{t},\bm{\sigma}_{i}^{t}\right)$,
and is concatenated with the feature $\mathbf{f}_{upast}$ and yield prediction
$\hat{p}_{i}^{t}=\left(\hat{x}_{i}^{t},\hat{y}_{i}^{t}\right)$ using the MLP
decoder $D_{latent}$, as follows:

$\hat{p}_{i}^{t}=D_{latent}\left(Concat\left(\mathbf{z}_{i}^{t},\mathbf{f}_{point}\right)\right).$
(8)

In the inference stage, because the ground truth point $p_{i}^{t}$ is
unavailable, the latent variable $\mathbf{z}_{i}^{t}$ is randomly sampled from
a prior distribution $\mathcal{N}\left(0,\mathbf{I}\right)$, and concatenated
with the feature $\mathbf{f}_{upast}$, and the prediction $\hat{p}_{i}^{t}$ is
then generated using the trained decoder $D_{latent}$.

For each pedestrian $i$, there are $\delta$ future time steps. Thus, $\delta$
CVAEs with the same structure are sequentially operated to generate the
predictions of $\delta$ future points
$\hat{p}_{i}^{\tau+1},\hat{p}_{i}^{\tau+2},...,\hat{p}_{i}^{\tau+\delta}$. It
should be noted that model parameters for the CVAE, including
$E_{upast},E_{point},E_{latent}$ and $D_{latent}$, are not shared because the
length of the updated past trajectory of each CVAE is different, and the CVAEs
are used to predict future points at different future time steps.

Figure 4: The illustration of the social non-local pooling layer proposed by
[7].

###  III-B Socially-aware Regression Module

The CVAE cannot model interactions among different pedestrians; thus, the
future trajectories predicted by the cascaded CVAE module may rough and
socially unacceptable. To cope with this problem, a socially-aware regression
module, as illustrated in Figure 2, is proposed to estimate the offsets to
refine the predicted future trajectories.

To generate effective offsets, two MLP encoders $E_{opast}$ and $E_{pfuture}$
are first used to extract features from the original past trajectory and the
predicted future trajectory, as follows:

$\mathbf{f}_{opast}=E_{opast}\left(\mathbf{X}_{i}\right),$ (9)
$\mathbf{f}_{pfuture}=E_{pfuture}\left(\hat{\mathbf{Y}}_{i}\right).$ (10)

Then, the two feature representations $\mathbf{f}_{opast}$ and
$\mathbf{f}_{pfuture}$ are concatenated and passed into a social interaction
layer $SE$ to produce an interaction feature, such that:

$\mathbf{f}_{st}=SE\left(Concat\left(\mathbf{f}_{opast},\mathbf{f}_{pfuture}\right)\right).$
(11)

Finally, the extracted feature $\mathbf{f}_{st}$ is fed to the MLP decoder
$D_{offsets}$ to estimate the offsets of the predicted future trajectories:

$\Delta\hat{\mathbf{Y}}_{i}=D_{offsets}\left(\mathbf{f}_{st}\right),$ (12)

where
$\Delta\hat{\mathbf{Y}}_{i}=\left\\{\left(\Delta\hat{x}_{i}^{t},\Delta\hat{y}_{i}^{t}\right)|\forall
t\in\left\\{\tau+1,\tau+2,...,\tau+\delta\right\\}\right\\}$. Using the
estimated offsets, the predictions $\hat{\mathbf{Y}}_{i}$ is refined by
$\hat{\mathbf{Y}}_{i}+\Delta{\hat{\mathbf{Y}}_{i}}$.

In this paper, the social non-local pooling layer proposed by [7] is adopted
as the $SE$ layer. This method integrates the social pooling [10] into the
non-local attention mechanism [25], as illustrated in Figure 4. Specifically,
three MLPs are first used to extract the query, key, and value features. Then,
a matrix multiplication is applied between the query and key features to
produce an attention map. Note that the social mask is a binary matrix
calculated using the distance between the pedestrians. Next, a social mask is
used point-wise on the attention map to mask out the non-consecutive neighbors
in the scene. Finally, a weighted sum between the masked attention map and the
value feature is conducted to generate a social interaction feature.

(a)

(b)

(c)

Figure 5: The illustration of the baseline method, the cascaded CVAE module
and the slide CVAE module.
$\overline{\mathbf{X}}_{i}^{t}=\left\\{\begin{array}[]{cc}Concat\left(\left\\{\left(x_{i}^{s},y_{i}^{s}\right)|\forall
t\in\left\\{t-\alpha,...,\tau\right\\}\right\\},\left\\{\left(\hat{x}_{i}^{s},\hat{y}_{i}^{s}\right)|\forall
s\in\left\\{\tau+1,\tau+2,...,t-1\right\\}\right\\}\right)&{\tau+1\leq
t\leq\tau+\alpha}\\\
Concat\left(\left\\{\left(\hat{x}_{i}^{s},\hat{y}_{i}^{s}\right)|\forall
s\in\left\\{\tau+1,\tau+2,...,t-1\right\\}\right\\}\right)&{\tau+\alpha+1\leq
t\leq\tau+\delta}\\\ \end{array}\right.$ (13)

###  III-C Slide CVAE Module

Even though the updated past trajectory that contains the predicted future
points before current time step is adopted by cascade CVAE module to assist in
the predicting of the future point at the following time step. There exists
two drawbacks of the proposed module. First, $\delta$ CVAEs are need to
predict $\delta$ future points, resulting in large model parameters. Second,
with the continuation of time, the updated past trajectory is increase, and
thus decrease the inference speed of CVAE.

Actually, the predictions of different future points fit the independent and
identically distributed. Thus, one CVAE is enough to predict $\delta$ future
points. Based on this observation, different from the cascaded CVAE module, a
slide CVAE module is proposed to decompose the prediction of the future
trajectories into the prediction of the future points using the updated past
trajectories of fixed length, in a slidable manner, as illustrated in Figure
5(c). In this manner, the model parameters are decreased by using only one
CVAE, and the inference speed is increased by using the updated past
trajectory of fixed length.

The proposed slide CVAE has the same structure of each cascaded CVAE, except
for the input. The slide CVAE module uses the updated trajectories of fixed
length. Specifically, at the time step $t$, the input of slide CVAE is
consists of the original past trajectory and the predicted points before
current time step. To keep the fixed length of the updated past trajectory,
the input $\overline{\mathbf{X}}_{i}^{t}$ of the slide CVAE module is defined
as Equation 13, where $\alpha$ is the fixed length of the updated past
trajectory.

###  III-D Training the Model

To train the $\delta$ CVAEs in the cascaded CVAE module, two loss terms,
namely the average point loss $\mathcal{L}_{AP}$ and the Kullback-Leibler (KL)
divergence loss $\mathcal{L}_{KLD}$, are used. The definitions of
$\mathcal{L}_{AP}$ and $\mathcal{L}_{KLD}$ are as follows:

$\mathcal{L}_{AP}=\frac{1}{N}\sum_{i=1}^{N}\sum_{t=\tau+1}^{\tau+\delta}\left\|p_{i}^{t}-\hat{p}_{i}^{t}\right\|^{2},$
(14)
$\mathcal{L}_{KLD}=\frac{1}{N}\sum_{i=1}^{N}\sum_{t=\tau+1}^{\tau+\delta}D_{KL}\left(\mathcal{N}\left(\bm{\mu}_{i}^{t},\bm{\sigma}_{i}^{t}\right)||\mathcal{N}\left(0,\mathbf{I}\right)\right).$
(15)

The KL divergence loss is a regularization loss that measures the distance
between the sampling distribution at the test stage and the sampling
distribution of the latent variable learned at the training stage, and is used
to train the CVAEs. The average point loss measures the displacement error
between the predicted future points and the ground truth future points, and is
used to train $E_{upath}$, $E_{point}$, $E_{latent}$, and $D_{latent}$. In
addition, a regression loss $\mathcal{L}_{R}$ is used to train the socially-
aware regression module. The definition of $\mathcal{L}_{R}$ is:

$\mathcal{L}_{R}=\frac{1}{N}\sum_{i=1}^{N}\sum_{t=\tau+1}^{\tau+\delta}\left\|p_{i}^{t}-\hat{p}_{i}^{t}-\Delta\hat{p}_{i}^{t}\right\|,$
(16)

where $\Delta
p_{i}^{t}=\left(\Delta\hat{x}_{i}^{t},\Delta\hat{y}_{i}^{t}\right)$. The
regression loss measures the displacement between the predicted offsets and
the real offsets, and is used to train $E_{opast}$, $E_{pfuture}$, and
$D_{offsets}$. Finally, the total loss $\mathcal{L}_{T}$ is defined as
follows:

$\mathcal{L}_{T}=\mathcal{L}_{KLD}+\mathcal{L}_{AP}+\mathcal{L}_{R}.$ (17)

This multi-task loss is used to train the entire model in an end-to-end
manner.

##  IV Experiments

###  IV-A Datasets

Two pedestrian trajectory prediction datasets, namely the Stanford Drone
Dataset (SDD) and the ETH/UCY dataset, were used to validate the proposed
method. The trajectories in both datasets were sampled with a frame rate of
2.5Hz; the former 3.2 seconds of trajectories (eight frames) are used as the
inputs, and the latter 4.8 seconds (twelve frames) are used as the future
trajectories to be predicted.

The SDD [26] is an aerial-view pedestrian trajectory prediction dataset
captured by a drone. The dataset contains eight scenes of a college campus. In
total, there are totally 11000 pedestrians with 185000 interactions between
agents and 40000 interactions between agents and scenes [26]. The standard
method for the division of the training and test sets used in [12, 11, 27] was
adopted in the experiments.

The ETH/UCY [28, 29] is a dataset group that contains two datasets, i.e. ETH
and UCY datasets. The ETH and UCY datasets contains five scenes, namely ETH,
HOTEL, ZARA1, ZARA2, and UNIV. In these scenes, there are a total of 1536
unique pedestrians and thousands of nonlinear trajectories. To ensure fair
comparisons, the same leave-one-out strategy adopted in previous studies [10,
11, 30, 12] was used to train and evaluate the proposed model.

###  IV-B Implementation Details

The proposed CSR was implemented in PyTorch [31] installed on a desktop
computer ran the Ubuntu 16.04 operating system and contained an NVIDIA 1080TI
GPU. The Adam algorithm [32] with a learning rate of $3e^{-4}$ was used as the
optimizer to train the CSR model. In each experiment, the model was trained
for 600 epochs with a batch size 512.

Name | Network Architecture
---|---
$E_{upast}$ | $2t\times 512\times 256\times 16\left(\tau\leq t\leq\tau+\delta-1\right)$
$E_{point}$ | $2\times 8\times 16\times 16$
$E_{latent}$ | $32\times 8\times 50\times 32$
$D_{latent}$ | $32\times 1024\times 512\times 1024\times 2$
$E_{opast}$ | $16\times 512\times 256\times 16$
$E_{future}$ | $24\times 512\times 256\times 16$
$D_{offsets}$ | $32\times 1024\times 512\times 1024\times 24$
TABLE I: The detailed architectures of all sub-networks.

Methods | S-GAN | CF-VAE | P2TIRL | SimAug | PECNet | Y-net⋆ | CSR | S-CSR
---|---|---|---|---|---|---|---|---
ADE | 27.23 | 12.60 | 12.58 | 10.27 | 9.96 | 7.85 | 4.87 | 2\. 77
FDE | 41.44 | 22.30 | 22.07 | 19.71 | 15.88 | 11.85 | 6.32 | 3\. 45
(a) Best of 20 samples

Methods | DESIRE | TNT | PECNet | Y-net⋆ | CSR | S-CSR
---|---|---|---|---|---|---
ADE | 19.25 | 12.79 | 12.23 | 11.49 | 8.38 | 6.02
FDE | 34.05 | 29.58 | 21.16 | 20.23 | 13.43 | 8.74
(b) Best of 5 samples

TABLE II: The comparison of the proposed CSR and S-CSR models with several
baseline methods and previous state-of-the-art methods (labeled by ⋆) on the
SDD dataset. (a) Results are tested using best of 5 samples. (b) Results are
tested using best of 20 samples. The best results are highlighted in bold.

Scenes | S-LSTM | S-GAN | Sophie | CGNS | PECNet | Trajectron++ | Y-net⋆ | CSR | S-CSR
---|---|---|---|---|---|---|---|---|---
Metrics | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE
ETH | 0.70 | 1.40 | 0.81 | 1.52 | 0.70 | 1.43 | 0.62 | 1.40 | 0.54 | 0.87 | 0.39 | 0.83 | 0.28 | 0.33 | 0.28 | 0.53 | 0.19 | 0.35
HOTEL | 0.37 | 0.73 | 0.72 | 1.61 | 0.76 | 1.67 | 0.70 | 0.93 | 0.18 | 0.24 | 0.12 | 0.21 | 0.10 | 0.14 | 0.07 | 0.08 | 0.06 | 0.07
UNIV | 0.60 | 1.32 | 0.60 | 1.26 | 0.54 | 1.24 | 0.48 | 1.22 | 0.35 | 0.60 | 0.20 | 0.44 | 0.24 | 0.41 | 0.24 | 0.35 | 0.13 | 0.21
ZARA1 | 0.49 | 1.15 | 0.34 | 0.69 | 0.30 | 0.63 | 0.32 | 0.59 | 0.22 | 0.39 | 0.15 | 0.33 | 0.17 | 0.27 | 0.07 | 0.09 | 0.06 | 0.07
ZARA2 | 0.39 | 0.89 | 0.42 | 0.84 | 0.38 | 0.78 | 0.35 | 0.71 | 0.17 | 0.30 | 0.11 | 0.25 | 0.13 | 0.22 | 0.05 | 0.09 | 0.05 | 0.08
AVG | 0.51 | 1.10 | 0.58 | 1.18 | 0.54 | 1.15 | 0.49 | 0.97 | 0.29 | 0.48 | 0.19 | 0.41 | 0.18 | 0.27 | 0.14 | 0.23 | 0.10 | 0.16

TABLE III: The comparison of the proposed CSR and S-CSR models with several
baseline methods and previous state-of-the-art methods (labeled by ⋆) on the
ETH/UCY dataset. Results are tested using best of 20 samples. The best results
are highlighted in bold.

Network Configuration: The sub-networks of the two proposed modules are
consists of the MLP and ReLU activation functions. The detailed network
architectures of these sub-networks are presented in Table I.

Evaluation Metrics: To evalulate the proposed method, two commonly used
evaluation metrics, namely the average displacement error (ADE) and the final
displacement error (FDE) were employed. The definitions of the ADE and FDE are
as follows:

$ADE=\frac{\sum_{i=1}^{N}\sum_{t=\tau+1}^{\tau+\delta}\left\|p_{i}^{t}-\hat{p}_{i}^{t}\right\|}{N\delta},$
(18)
$FDE=\frac{\sum_{i=1}^{N}\left\|p_{i}^{\tau+\delta}-\hat{p}_{i}^{\tau+\delta}\right\|}{N}.$
(19)

The ADE measures the mean Euclidean distance between the predicted future
points and the ground truth future points, and the FDE measures the Euclidean
distance between the predicted final points and the ground truth final points.

Baselines: The proposed method was compared with the following state-of-the-
art baseline methods:

  * •

S-LSTM [10] Each pedestrian is modeled using an LSTM, and the interactions are
extracted by pooling the hidden states among neighbors.

  * •

DESIRE [33]. An inverse optimal control (IOC)-based trajectory planning method
is used to refine the predicted trajectories.

  * •

S-GAN [11]. This method improves S-LSTM by introducing a GAN to generate
multimodal prediction results.

  * •

SoPhie [12]. This method improves GAN-based methods by applying an attention
to model social relationships and physical constraints.

  * •

P2TIRL [27]. A maximum entropy inverse reinforcement learning strategy is
introduced to learn a grid-based trajectory prediction method.

  * •

CF-VAE [34]. A conditional normalizing flow-based prior is proposed to improve
the VAE for the generation of effective predictions.

  * •

CGNS [35]. Conditional variational divergence minimization and conditional
latent space learning are used to predict the future trajectory.

  * •

CF-LSTM [14]. A feature-cascaded method that can simultaneously capture
location and velocity informations is proposed to improve LSTM.

  * •

Trajectron++ [24]. This method presents a modular graph-structured recurrent
model that can intergrate agent dynamics and heterogeneous data.

  * •

PECNet [7]. This method first predicts distant trajectory endpoints and then
generates the future predictions conditioned on endpoints.

  * •

TNT [8]. This method improves PECNet using a scoring and selection stage that
ranks and selects multimodal predictions with likelihood scores.

  * •

Y-net [9]. This method improves PECNet and TNT by iteratively predicting
intermediate waypoints and trajectories.

Modules | ETH | HOTEL | UNIV | ZARA1 | ZARA2 | AVG
---|---|---|---|---|---|---
$BL$ | $CCM$ | $SCM$ | $SRM$ | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE
✓ |  |  |  | 0.48 | 0.94 | 0.19 | 0.35 | 0.30 | 0.61 | 0.22 | 0.42 | 0.16 | 0.32 | 0.27 | 0.53
| ✓ |  |  | 0.33 | 0.57 | 0.08 | 0.10 | 0.28 | 0.36 | 0.09 | 0.10 | 0.07 | 0.09 | 0.17 | 0.24
| ✓ |  | ✓ | 0.28 | 0.53 | 0.07 | 0.08 | 0.24 | 0.35 | 0.07 | 0.09 | 0.05 | 0.09 | 0.14 | 0.23
|  | ✓ | ✓ | 0.19 | 0.35 | 0.06 | 0.07 | 0.13 | 0.21 | 0.06 | 0.07 | 0.05 | 0.08 | 0.10 | 0.16

TABLE IV: Ablation study of different modules of the proposed method on the ETH/UCY dataset. $BL$ represents the introduced baseline method. $CCM$ denotes the proposed cascaded CVAE module. $SCM$ denotes the proposed slide CVAE module. $SRM$ denotes the proposed socially-aware regression module. Modules | Performance
---|---
$BL$ | $CCM$ | $SCM$ | $SRM$ | ADE | FDE
✓ |  |  |  | 11.56 | 15.71
| ✓ |  |  | 6.12 | 7.87
| ✓ |  | ✓ | 4.87 | 6.32
|  | ✓ | ✓ | 2.77 | 3.45
TABLE V: Ablation study of different modules of the proposed method on the SDD
dataset. $BL$ represents the introduced baseline method. $CCM$ denotes the
proposed cascaded CVAE module. $SCM$ denotes the proposed slide CVAE module.
$SRM$ denotes the proposed socially-aware regression module.

###  IV-C Quantitative Analysis

The proposed CSR model was compared with the aforementioned baseline methods
on the SDD and ETH/UCY datasets via the ADE and FDE metrics.

####  IV-C1 Comparison with VAE-based Methods

The VAE is widely used to generate the latent variables of trajectory
distributions, and many VAE-based prediction methods have been proposed.
Unlike these methods, the proposed method uses multiple CVAEs to predict the
future points, in a cascaded manner. The proposed CSR was compared with
several VAE-based methods, including DESIRE, CF-VAE, and CGNS. As reported in
Table II, on the SDD, CSR respectively surpassed CF-VAE and DESIRE by
61.3%/71.7% and 56.5%/60.6% in terms of the ADE/FDE metrics. Moreover, as
reported in Table III, CSR outperformed CGNS by 71.4%/76.3% on the ETH/UCY.
This comparison validates that the proposed CSR, which utilizes CVAEs in a
cascaded manner, performs better than VAE-based methods.

####  IV-C2 Comparison with RNN-based Methods

The proposed CSR predicts the future trajectory in a sequential manner, which
is similar to RNN-based methods. However, RNN based methods do not explicitly
encode the predicted points as the past trajectories. To vallidate the updated
past trajectory used in the proposed model, it was compared with several RNN-
based methods, including DESIRE, CF-VAE, CGNS and Trajectron++. As shown in
Table II, on the SDD, the proposed CSR exhibited respective improvements over
DESIRE and CF-VAE of 56.5%/60.6% and 61.3%/71.7% in terms of the ADE/FDE
metrics. Moreover, as shown in Table III, the proposed CSR exhibited
respective improvements over CGNS and Trajectron++ of 71.4%/76.3% and
26.3%/52.1% on the ETH/UCY dataset. This comparison verifies that the proposed
method, which explicitly encodes the predicted points as the past
trajectories, achieves superior performance as compared to RNN-based methods.

####  IV-C3 Comparison with Target-conditioned Methods

Target-conditioned trajectory prediction methods have recently become popular.
These methods first predict the target point and then generate future
predictions condtioned on the predicted target. The performance of this type
of methods is easily affected by the predicted target; however, target
prediction is inherently a difficult problem. The proposed CSR was compared
with three target-conditioned methods, namely PECNet, TNT, and Y-net. As
presented in Table II, CSR exhibited respective ADE/FDE improvements over TNT,
PECNet, and Y-net (best of 5 samples) of 34.5%/54.6%, 31.5%/36.5%, and
27.1%/33.6% on the SDD. Moreover, as illustrated in Table III, the proposed
CSR respectively outperformed PECNet and Y-net by 51.7%/52.1% and 22.2%/14.8%
on the ETH/UCY dataset. This comparison demonstrates that the proposed CSR
performs better than target-conditioned trajectory prediction methods.

(a)

(b)

(c)

(d)

Figure 6: Visualization of the trajectories predicted by the cascaded CVAE
module with or without the socially-aware regression module. The visualized
trajectories are best predictions sampled from 20 trials. The white dots,
yellow dots, green dots, and red dots respectively represent the past
trajectories, ground truth future trajectories, predictions without using the
socially-aware regression module, and predictions using the socially-aware
regression module.

####  IV-C4 Comparison with State-of-the-art Methods

Tables II and III reveal that Y-net achieved the best performance among all
the selected baselines on the SDD and ETH/UCY. Compared to Y-net, the proposed
CSR exhibited improvements in the ADE/FDE metrics of 38.0%/46.7% (best of 20
samples) on the SDD and 22.2%/14.8% on the ETH/UCY dataset, thereby achieving
new state-of-the-art performance.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 7: Qualitative results of CSR in SDD dataset. The visualized trajectories are best predictions sampled from 20 trials. The white dots, yellow dots, green dots, and red dots respectively represent the past trajectories, ground truth future trajectories, predictions of PECNet, and predictions of the proposed CSR. Modules | Performance
---|---
$CCM$ | $SCM$ | $SRM$ | Parameters | Inference time
✓ |  | ✓ | 15.88 M | 0.32 S/B
| ✓ | ✓ | 2.24 M | 0.30 S/B
TABLE VI: Analysis of the model efficiency of the cascaded CVAE module and the
slide CVAE module on the SDD dataset. $CCM$ denotes the proposed cascaded CVAE
module. $SCM$ denotes the proposed slide CVAE module. $SRM$ denotes the
proposed socially-aware regression module. ”S” represents million and ”S/B”
represents seconds per batch. All experiments are evaluated on an NVIDIA
1080TI GPU.

(a)

(b)

(c)

(d)

(e)

Figure 8: Qualitative results of CSR and S-CSR in ETH/UCY dataset. The
visualized trajectories are best predictions sampled from 20 trials. The red
lines, yellow lines, green dashed lines, and blue dashed lines respectively
represents the past trajectories, ground truth future trajectories,
predictions of CSR, and predictions of S-CSR.

###  IV-D Ablation Study

This experiment is designed to validate the effectiveness of the proposed
cascaded CVAE module, socially-aware regression module and slide CVAE module.
To validate those modules, a baseline method that consists of $\delta$ CVAEs
is introduced for comparison. In the baseline method, $\delta$ future points
are respectively predicted by $\delta$ CVAEs that using the original past
trajectory as the input, as illustrated in Figure 5(a).

####  IV-D1 Cascaded CVAE Module

The proposed cascaded CVAE module predict future trajectory in a cascaded
manner, in which each CVAE sequentially predict a future point using a updated
past trajectory that consists of the original past trajectory and the
predicted future points before the current time step. To prove the effect of
the cascaded CVAE module, it is compared with a baseline method which $\delta$
CVAEs that using the original past trajectory as the input are respectively
used to predict $\delta$ future points. As reported in Tables IV and V, the
cascaded CVAE module outperforms the baseline by 37.0%/54.7% on the ETH/UCY
and 47.1%/49.9% on the SDD, in terms of the ADE/FDE metrics. This demonstrates
that the cascaded CVAE module can extract useful information from the
predicted future points, and thus improves the predicition performance of the
future trajectory.

####  IV-D2 Socially-aware Regression Module

The future trajectory predicted by the cascaded CVAE module may be impractical
because the CVAE cannot model interactions among pedestrians. Thus, the
socially-aware regression module was introduced to refine the final
predictions via social interaction. To prove its effectiveness, the model are
respectively trained with and without it. As presented in Tables IV and V, the
socially-aware regression module can further boost the performance of the
cascaded CVAE module by 17.6%/4.2% on the ETH/UCY and 20.4%/19.7% on the SDD,
in terms of the ADE/FDE metrics, which indicates that this module can extract
interaction features from the predicted trajectories to refine the final
predictions. Besides, the predictions of the model trained with or without the
socially-aware regression module are visualized in Figure 6. Figures 6(a),
6(b) and 6(c) reveal that, compared to the less coordinated trajectories
predicted by the cascade CVAE module, more socially compliant trajectories are
refined by the socially-aware regression module. Figure 6(d) demonstrates that
the the collision problem in trajectory prediction can be improved by the
socially-aware regression module using interaxtions between pedestrians.

####  IV-D3 Slide CVAE Module

The cascaded CVAE module is effectiveness in using the predicted future points
to assist in trajectory prediction. But its efficiency is directly affected by
two factors, namely the multiple unshared CVAEs and the incremental inputs. To
increase the inference speed and decrease the model parameters, a slide CVAE
module is proposed to further improve the cascaded CVAE module. As shown in
the 6-th and 7-th columns of Tables IV and V, the slide CVAE module improves
the cascaded CVAE module by 28.6%/30.4% on the ETH/UCY and 43.1%/45.4% on the
SDD, in terms of the ADE/FDE metrics. Moreover, as depicted in Table VI, the
slide CVAE module decreases the model parameters by 85.9% and increases the
inference speed by 6.3%. It is evident that the slide CVAE module has a far
better performance and a higher efficiency than the cascaded CVAE module. This
is expected, because one trajectory can train one slide CVAE for $\delta$
times. While for the cascaded CVAE modules, one trajectory need to train
$\delta$ CVAEs, which has more parameters and is more difficult to fitting.

###  IV-E Qualitative Analysis

Qualitative results of CSR and PECNet on the SDD are visualized in Figure 7.
Figures 7(a) and 7(b) present predictions of linear trajectories. The two
methods can both produce trajectories that very close to the ground truth. It
is expected because the prediction of linear trajectories is easy. Figure 7(c)
and 7(d) show predictions of crooked trajectories. In this case, the
predictions of CSR are more accurate than that of the PECNet. These results
demonstrate that the proposed CSR has better performance in predicting
nonlinear trajectories.

Figures 7(e) and 7(f) respectively visualize two easy interaction samples,
i.e. turning together and straight going together. Both PECNet and CSR can
generate trajectories that are close to the ground truth because the
interactions between pedestrians are simple. Figure 7(g) and 7(h) visualize
predictions of two complex interacions such as crossing and collision
avoidance. The predictions of CSR are socially compliant and more reasonable
than PECNet. These results demonstrate that the proposed CSR can effectively
model the complex interactions between pedestrians.

Qualitative results of CSR and S-CSR on the ETH/UCY dataset are also
visualized in Figure 8. For each scene, three samples are randomly sampled and
visualized. Figure 8 shows that CSR can capture interactions, and generate
socially compliant trajectories in most cases. And compared with the
trajectories produced by CSR, the trajectories that more closer to the ground
truth future trajectories are predicted by S-CSR.

##  V Conclusion

In this work, a novel cascaded CVAE with socially-aware rethinking (CSR)
method is proposed for pedestrian trajectory prediction. It consists of a
cascaded CVAE module and a socially-aware regression module. The cascaded CVAE
module predicts the future trajectories in a sequential manner using the
updated past trajectory. And the socially-aware regression module extracts
interactions features from both the past trajectories and the predicted
trajectories to refine the final predictions. Besides, a slide CVAE module is
proposed to further improve the model efficiency and performance of the
cascade CVAE module using one shared CVAE, in a slidable manner. Extensive
experiments on the SDD and ETH/UCY datasets demonstrate the effectiveness of
the proposed methods. Further more, the results indicate that the proposed CSR
and S-CSR outperforms other state-of-the-art methods by a large margin and
achieves a new state-of-the-art performance.

## References

  * [1] J. Hong, B. Sapp, and J. Philbin, “Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions,” in _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019\.
  * [2] P. Raksincharoensak, T. Hasegawa, and M. Nagai, “Motion planning and control of autonomous driving intelligence system based on risk potential optimization framework,” _International Journal of Automotive Engineering_ , vol. 7, no. AVEC14, pp. 53–60, 2016.
  * [3] A. Vemula, K. Muelling, and J. Oh, “Social attention: Modeling attention in human crowds,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , May 2018, pp. 4601–4607.
  * [4] Y. Luo, P. Cai, A. Bera, D. Hsu, W. S. Lee, and D. Manocha, “Porca: Modeling and planning for autonomous driving among many pedestrians,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 3418–3425, 2018.
  * [5] H. Xue, D. Huynh, and M. Reynolds, “Location-velocity attention for pedestrian trajectory prediction,” in _2019 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , Jan 2019, pp. 2038–2047.
  * [6] M. Luber, J. A. Stork, G. D. Tipaldi, and K. O. Arras, “People tracking with human motion predictions from social forces,” in _2010 IEEE International Conference on Robotics and Automation_. IEEE, 2010, pp. 464–469.
  * [7] K. Mangalam, H. Girase, S. Agarwal, K.-H. Lee, E. Adeli, J. Malik, and A. Gaidon, “It is not the journey but the destination: Endpoint conditioned trajectory prediction,” in _European Conference on Computer Vision_. Springer, 2020, pp. 759–776.
  * [8] H. Zhao, J. Gao, T. Lan, C. Sun, B. Sapp, B. Varadarajan, Y. Shen, Y. Shen, Y. Chai, C. Schmid _et al._ , “Tnt: Target-driven trajectory prediction,” _arXiv preprint arXiv:2008.08294_ , 2020.
  * [9] K. Mangalam, Y. An, H. Girase, and J. Malik, “From goals, waypoints & paths to long term human trajectory forecasting,” _arXiv preprint arXiv:2012.01526_ , 2020.
  * [10] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “Social lstm: Human trajectory prediction in crowded spaces,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 961–971.
  * [11] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, “Social gan: Socially acceptable trajectories with generative adversarial networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 2255–2264.
  * [12] A. Sadeghian, V. Kosaraju, A. Sadeghian, N. Hirose, H. Rezatofighi, and S. Savarese, “Sophie: An attentive gan for predicting paths compliant to social and physical constraints,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 1349–1358.
  * [13] J. Liang, L. Jiang, J. C. Niebles, A. G. Hauptmann, and L. Fei-Fei, “Peeking into the future: Predicting future person activities and locations in videos,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 5725–5734.
  * [14] Y. Xu, J. Yang, and S. Du, “Cf-lstm: Cascaded feature-based long short-term networks for predicting pedestrian trajectory,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 07, 2020, pp. 12 541–12 548.
  * [15] S. Lefèvre, C. Laugier, and J. Ibañez-Guzmán, “Exploiting map information for driver intention estimation at road intersections,” in _2011 ieee intelligent vehicles symposium (iv)_. IEEE, 2011, pp. 583–588.
  * [16] C. Ramussen and C. Williams, “Gaussian processes for machine learning (adaptive computation and machine learning),” 2006.
  * [17] R. Toledo-Moreo and M. A. Zamora-Izquierdo, “Imm-based lane-change prediction in highways with low-cost gps/ins,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 10, no. 1, pp. 180–185, 2009.
  * [18] X. Li, X. Ying, and M. C. Chuah, “Grip: Graph-based interaction-aware trajectory prediction,” in _2019 IEEE Intelligent Transportation Systems Conference (ITSC)_. IEEE, 2019, pp. 3960–3966.
  * [19] J. Gao, C. Sun, H. Zhao, Y. Shen, D. Anguelov, C. Li, and C. Schmid, “Vectornet: Encoding hd maps and agent dynamics from vectorized representation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 525–11 533.
  * [20] A. Mohamed, K. Qian, M. Elhoseiny, and C. Claudel, “Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 14 424–14 432.
  * [21] H. Zhou, D. Ren, H. Xia, M. Fan, X. Yang, and H. Huang, “Ast-gnn: An attention-based spatio-temporal graph neural network for interaction-aware pedestrian trajectory prediction,” _Neurocomputing_ , vol. 445, pp. 298–308, 2021.
  * [22] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [23] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” in _NIPS 2014 Workshop on Deep Learning, December 2014_ , 2014.
  * [24] T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16_. Springer, 2020, pp. 683–700.
  * [25] X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7794–7803.
  * [26] A. Robicquet, A. Sadeghian, A. Alahi, and S. Savarese, “Learning social etiquette: Human trajectory understanding in crowded scenes,” in _European conference on computer vision_. Springer, 2016, pp. 549–565.
  * [27] N. Deo and M. M. Trivedi, “Trajectory forecasts in unknown environments conditioned on grid-based plans,” _arXiv preprint arXiv:2001.00735_ , 2020\.
  * [28] S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool, “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in _2009 IEEE 12th International Conference on Computer Vision_. IEEE, 2009, pp. 261–268.
  * [29] A. Lerner, Y. Chrysanthou, and D. Lischinski, “Crowds by example,” in _Computer graphics forum_ , vol. 26, no. 3. Wiley Online Library, 2007, pp. 655–664.
  * [30] V. Kosaraju, A. Sadeghian, R. Martín-Martín, I. Reid, H. Rezatofighi, and S. Savarese, “Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks,” _Advances in Neural Information Processing Systems_ , vol. 32, pp. 137–146, 2019.
  * [31] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga _et al._ , “Pytorch: An imperative style, high-performance deep learning library,” _Advances in neural information processing systems_ , vol. 32, pp. 8026–8037, 2019.
  * [32] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _International Conference on Learning Representations (ICLR)_ , 2015.
  * [33] N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. Torr, and M. Chandraker, “Desire: Distant future prediction in dynamic scenes with interacting agents,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 336–345.
  * [34] A. Bhattacharyya, M. Hanselmann, M. Fritz, B. Schiele, and C.-N. Straehle, “Conditional flow variational autoencoders for structured sequence prediction,” in _4th Workshop on Bayesian Deep Learning_. bayesiandeeplearning. org, 2019.
  * [35] J. Li, H. Ma, and M. Tomizuka, “Conditional generative neural system for probabilistic trajectory prediction,” in _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2019, pp. 6150–6156.


# A New UV Spectral Feature on Europa: Confirmation of NaCl in Leading-
hemisphere Chaos Terrain

Samantha K. Trumbo Division of Geological and Planetary Sciences, California
Institute of Technology, Pasadena, CA 91125, USA  Cornell Center for
Astrophysics and Planetary Science, Cornell University, Ithaca, NY 14853, USA
Tracy M. Becker Southwest Research Institute, 6220 Culebra Road, San Antonio,
TX 78238, USA  Michael E. Brown Division of Geological and Planetary Sciences,
California Institute of Technology, Pasadena, CA 91125, USA  William T. P.
Denman Division of Geological and Planetary Sciences, California Institute of
Technology, Pasadena, CA 91125, USA  Philippa Molyneux Southwest Research
Institute, 6220 Culebra Road, San Antonio, TX 78238, USA  Amanda Hendrix
Planetary Science Institute, Tucson, AZ 85719, USA  Kurt D. Retherford
Southwest Research Institute, 6220 Culebra Road, San Antonio, TX 78238, USA
Lorenz Roth Space and Plasma Physics, KTH Royal Institute of Technology,
Stockholm, Sweden  Juan Alday AOPP, Department of Physics, University of
Oxford, Oxford, UK

(Accepted November 16, 2021)

###### Abstract

Recent visible-wavelength observations of Europa’s surface obtained with the
Hubble Space Telescope revealed the presence of an absorption feature near 450
nm that appears spatially correlated with leading-hemisphere chaos terrain.
This feature was interpreted to reflect the presence of irradiated sodium
chloride ultimately sourced from the interior. Here, we use ultraviolet
spectra also collected with the Hubble Space Telescope to detect an additional
previously unseen absorption near 230 nm, which spatially correlates with the
450 nm feature and with the same leading-hemisphere chaos terrain. We find
that the new ultraviolet feature is also well-matched by irradiated sodium
chloride at Europa-like conditions. Such confirmation of sodium chloride
within geologically young regions has important implications for Europa’s
subsurface composition.

Galilean satellites (627), Europa (2189), Planetary surfaces (2113), Surface
composition (2115)

††software: astropy (Astropy Collaboration et al., 2013)

##  1 Introduction

Salt hydrates ultimately sourced from Europa’s putative subsurface ocean have
long been proposed as components of its geologically young surface (e.g.
McCord et al., 1998), which could allow remote-sensing studies to constrain
the ocean’s composition and chemical history. Observations of Europa’s tenuous
atmosphere revealed the presence of sodium and potassium (Brown & Hill, 1996;
Brown, 2001), and subsequent observations and modeling have suggested a
potential link to surface salts and, ultimately, the ocean below (e.g.
Johnson, 2000; Leblanc et al., 2002; Johnson et al., 2002; Leblanc et al.,
2005). However, the precise identification of Europa’s endogenous surface
salts has been an ongoing subject of debate. Numerous studies interpret
Galileo Near-infrared Mapping Spectrometer (NIMS) spectra to reflect
endogenous sulfates (e.g. McCord et al., 1998; Dalton et al., 2005; Dalton,
2007; Dalton et al., 2012), while more recent, ground-based spectroscopic
studies have either favored a chloride-dominated interpretation (Brown & Hand,
2013; Fischer et al., 2015, 2016) or, in the interpretation of Ligier et al.
(2016), a mixture of magnesium chloride, chlorate, and perchlorate.

The persistence of this debate has been facilitated, in part, by the paucity
of compositionally diagnostic absorptions in 1–2.5 $\micron$ spectra of
Europa’s surface. Discrete features of many of the proposed sulfates were
below the spectral resolution of NIMS or could be smoothed out with
sufficiently complex linear-mixture modeling (e.g. Dalton et al., 2012), while
similar modeling approaches, rather than characteristic absorptions, were used
to infer the compositions of Ligier et al. (2016). In contrast, like Europa,
the proposed sodium and potassium chloride of Brown & Hand (2013) are simply
spectrally smooth in the near infrared, which makes them hard to identify in
such data, even when present in high abundance.

Though smooth in the infrared, alkali chlorides do form spectrally distinct
“color-center” absorptions at visible-wavelengths when subjected to energetic
particle irradiation (Hand & Carlson, 2015; Poston et al., 2017; Hibbitts et
al., 2019), a process that occurs across the entire surface of Europa
(Paranicas et al., 2009). Indeed, recent spatially resolved visible-wavelength
spectra from the Hubble Space Telescope (HST) detected a previously unseen
absorption feature near 450 nm that is consistent with the F-center of
irradiated sodium chloride (NaCl) (Trumbo et al., 2019). The NaCl F-center
feature appears only on the leading hemisphere, sheltered from the
magnetospheric sulfur implantation and radiolytic alteration of the trailing
hemisphere (Carlson et al., 2002; Paranicas et al., 2009), and corresponds
particularly to Tara Regio, a large, visibly discolored region of geologically
young “chaos” terrain (Doggett et al., 2009; Leonard et al., 2018). This
distribution strongly suggests an interior origin, implying that the F-centers
are forming in relatively pristine endogenous material containing NaCl.

Here, we use mid-UV HST spectra of Europa’s surface to detect a previously
unseen absorption near 230 nm, which geographically correlates with the
recently detected F-center feature. The parallel laboratory work of Brown et
al. (2022) shows that NaCl irradiated under Europa-like conditions forms an
additional feature, known as the V3 center, at similar wavelengths. We find
that the V3 band matches our observed feature well and argue that this
detection confirms the presence of NaCl within geologically young chaos
regions on Europa. The nondetection of the 230 nm feature in previous mid-UV
datasets can be explained by their lack of high-quality, spatially resolved
coverage of Europa’s leading hemisphere at these wavelengths. The
International Ultraviolet Explorer (IUE) had poor signal-to-noise below
$\sim$240 nm (Nelson et al., 1987; Domingue & Lane, 1998) and the Galileo UVS
observations of the leading hemisphere were either disk-integrated or
challenged by the radiation environment at Europa (Hendrix et al., 1998;
Carlson et al., 1999). A complete analysis of the full mid-UV HST dataset,
which presents the first high-quality, spatially resolved coverage of Europa
in the 200–240 nm range, is given in Becker et al. (2022).

##  2 Observations and Data Reduction

We present HST observations of Europa’s surface made with the Space Telescope
Imaging Spectrograph (STIS) across four HST visits. Table 1 lists the
corresponding dates, times, and geometries of each visit. The 52′′ x 0.2′′
slit was used in two pointings per visit in order to cover the central
$\sim$40$\%$ of the disk. The G230L low-resolution, first-order grating
(R$\sim$500; spectral range: 157–318 nm) was used with exposure times of 987.2
seconds per slit position for the 2018 visits and 777.5 or 808.2 seconds for
the 2019 visit. This approach provided high-quality, spatially resolved
spectra from 200–300 nm, with shorter wavelengths ($<$200 nm) dominated by
noise.

In our analysis, we used the flux- and wavelength-calibrated data provided by
HST after standard reduction with the STIS calibration pipeline (calstis). We
extracted single spectra by taking individual rows from the two-dimensional
spectral images, corresponding to the 0.025′′ pixel-scale (equal to the
0.025′′ or $\sim$80-km diffraction-limited resolution at 230 nm). We then
divided each by a corresponding solar spectrum from the same date of
observation, as obtained by the SORCE/SOLSTICE ultraviolet spectrometer
(Mcclintock et al., 2005) and downloaded from the LASP Interactive Solar
Irradiance Datacenter (LISIRD). Prior to division, we smoothed and binned the
solar spectra to match the STIS spectral resolution and sampling. Finally, we
calculated the corresponding latitude/longitude coordinates of each extracted
pixel using the aperture geometry information in the HST FITS headers and the
phase, angular size, and pole orientation of Europa from JPL Horizons for the
time of each exposure.

Table 1: Table of Observations Date | Time | Central | Central | Angular
---|---|---|---|---
(UT) | (Start/End) | Lon. | Lat. | Diameter
2018 Apr 11 | 07:09/07:29 | 197$\degree$W | -3.89$\degree$ | 0.95′′
2018 Apr 19 | 02:42/03:02 | 270$\degree$W | -3.88$\degree$ | 0.96′′
2018 Jun 1 | 10:03/10:23 | 345$\degree$W | -3.74$\degree$ | 0.96′′
2019 Jun 21 | 07:23/07:42 | 91$\degree$W | -3.14$\degree$ | 1.00′′

##  3 Feature Identification and Mapping

The HST spectra of some locations on Europa reveal a subtle absorption near
230 nm (Figure 1), which appears discrete against the sloping continuum in the
mid-UV and at roughly the wavelength of the NaCl V3 band at 120 K (Brown et
al., 2022). To facilitate comparison with the laboratory data, we remove
linear continua from both an average HST spectrum containing the feature
(Figure 1) and a 120 K laboratory NaCl spectrum from Brown et al. (2022) using
identical bounds. We find that a linear continuum works best for both the HST
and the laboratory data and seems most reasonable based on leading-hemisphere
spectra that do not contain the 230 nm absorption. We thus fit a linear
continuum from 200 to 265 nm, excluding the 202–260 nm region, which
corresponds to the feature in both spectra, from the fit. As the laboratory
data do not extend below 200 nm and the HST spectra are of very poor quality
at those wavelengths, it appears likely that these bounds do not entirely
capture the extent of the feature on the short-wavelength end. The result of
the laboratory comparison is included in Figure 1, which demonstrates that the
band minimum, shape, and apparent width of the observed 230 nm feature are
well-matched by the V3 center of laboratory-irradiated NaCl at 120K. Indeed,
we obtain equivalent, marginally different fits using any of the spectra of
Brown et al. (2022), which show slight variations in the V3 band with
different laboratory conditions.

Figure 1: Average HST/STIS spectrum of Tara Regio showing the 230 nm feature,
which is consistent with the V3 center of irradiated NaCl. The dashed line is
a linear continuum fit, and the continuum-removed feature is included above
the spectrum. The continuum-removed NaCl V3 band from the 120 K laboratory
spectrum of Brown et al. (2022) is overlain in red and scaled to match the
depth of the observed feature in the HST data.

Though the 230 nm absorption spectrally matches NaCl quite well, its
geographic distribution in relation to the proposed NaCl F-center is also
critical to cementing this identification. In order to map the 230 nm feature,
we measure its integrated band area in each HST spectrum after fitting and
removing a linear continuum as described above. As we are now dealing with
individual pixels rather than a high signal-to-noise average, we slightly
adjust the bounds for continuum removal to 200–285 nm, excluding the 215–275
nm region across the absorption, in order to minimize the effects of increased
noise at the short-wavelength end of the spectra. We then integrate the
residual feature to obtain the band area or equivalent width (i.e. the width
of a 100% depth absorption of the same integrated area) for each pixel, and
map the resultant values across the surface of Europa using the coordinates as
obtained in Section 2.

We find that the largest 230 nm absorptions are nearly perfectly constrained
to Tara Regio (Figure 2), the same leading-hemisphere chaos region in which
the proposed NaCl F-center absorption is strongest (Trumbo et al., 2019).
Similar to what was seen for the F-center, we also find weaker absorptions
across the other large-scale leading-hemisphere chaos west of Tara Regio, as
well as immediately outside of Tara Regio, which presents a mixture of
smaller-scale chaos and lenticulated terrain amongst older plains (Doggett et
al., 2009; Leonard et al., 2018). As discussed in Trumbo et al. (2019), color-
centers indicate irradiated NaCl, so their distribution on Europa should
reflect a convolution of NaCl abundance and radiation intensity. Chlorides
emplaced onto the leading hemisphere would be irradiated by $\geq$20 MeV
electrons, which impact in an equatorial lens centered on the leading apex
(Nordheim et al., 2018), providing the energy for color-center growth. Such a
convolution may explain why Tara Regio, in particular, exhibits the strongest
features, as well as why comparable absorptions to those observed in other
leading chaos can be seen in more mixed low-latitude terrain. The concurrent
match to NaCl in both wavelength and geography provides strong evidence that
the 230 nm absorption results from V3 centers forming in irradiated NaCl,
which simultaneously strengthens the past identification of the 450 nm feature
as an NaCl F-center absorption.

Figure 2: Map of Europa’s 230 nm feature. This newly detected absorption
appears exclusively on the leading hemisphere. Black outlines indicate large-
scale leading-hemisphere chaos regions, mapped approximately from Doggett et
al. (2009). The strongest 230 nm absorptions are associated with the chaos
region Tara Regio ($\sim$85$\degree$W), which also contains the strongest NaCl
F-center absorptions seen in previous visible-wavelength HST spectra (Trumbo
et al., 2019). Background image credit: NASA/JPL/Björn Jónsson

The primary difference between the HST and laboratory spectra is the observed
ratios of the F- and V3-center bands. The laboratory ratio is nearly 1.8 times
that observed anywhere within Tara Regio in the current data (6.9 in the lab
vs. 3.9 where the strongest F-centers are observed). However, color-centers
are dynamic and dependent on the instantaneous balance between destruction by
solar photobleaching and creation via energetic particle bombardment. This
balance is not well replicated in the lab and is time-variable on Europa (e.g.
Paranicas et al., 2009; Bagenal & Dols, 2020). As color-center absorptions
depend on the radiation flux, and as the UV spectra of Tara Regio were taken
two years after their visible-wavelength counterparts during likely different
magnetospheric conditions, the HST ratio of the dynamic F- and V3-center bands
may be more misleading than meaningful. The data of Brown et al. (2022) also
hint that differing effects of photobleaching on the F- and V3\- centers may
shrink their ratio under true Europa conditions. In fact, concurrent
laboratory work by Denman et al. (2022) investigating the dynamics of NaCl
color centers in response to varying flux and photobleaching conditions shows
that similar dynamics operating on timescales relevant to Europa’s daily
rotation likely also explain the lack of an M-center band in the visible-
wavelength HST data, supporting the hypothesis posed at the time of the
F-center detection (Trumbo et al., 2019).

##  4 Discussion

The new detection of an NaCl V3-center absorption, in combination with the
parallel laboratory work explaining the HST data in the context of color-
center formation and decay at Europa-like conditions (Brown et al., 2022;
Denman et al., 2022), effectively confirms the presence of NaCl on the surface
of Europa. In addition, the clear association of both the V3\- and F-center
features with geologically disrupted chaos terrain implies that this NaCl
originated in Europa’s internal ocean, which has important implications for
the nature of Europa’s endogenous salts. NaCl within chaos terrain on the
leading hemisphere is most consistent with the chloride hypothesis of Brown &
Hand (2013), in which the endogenous salts emplaced onto the surface are
chloride-rich, preserved as chloride-rich within the geology of the leading
hemisphere, and converted radiolyticially to a more sulfate-rich composition
within the chaos of the sulfur-bombarded trailing hemisphere. In contrast, the
NaCl confirmation runs counter to the sulfate-rich interpretations of NIMS
data (e.g. McCord et al., 1998; Dalton et al., 2012) and the magnesium-rich
chlorate, perchlorate, and chloride mixtures of Ligier et al. (2016), whose
hypothesis places the highest abundances of chlorinated species on the
trailing hemisphere, where we find no NaCl absorption.

It is possible that the detectable presence of NaCl on the surface may be used
to constrain the relative abundances of chloride (Cl-) vs. sulfate
(SO${}_{4}^{2-}$) and sodium (Na+) vs. magnesium (Mg2+) in the source material
(e.g. Johnson et al., 2019), depending on the freezing history of the material
and the mechanisms by which it was emplaced onto the surface. The brine-
freezing experiments of Vu et al. (2016) suggest that, in the absence of flash
freezing, Na+, Mg2+, Cl-, and SO${}_{4}^{2-}$ containing brines extruded onto
the surface would only form abundant NaCl if the Na+/Mg2+ and
Cl-/SO${}_{4}^{2-}$ ratios were high, as mirabilite (NaSO4 $\cdot$ 10H2O) and
MgCl2 were the primary phases frozen from their solutions. Alternatively, more
rapid freezing rates (e.g. Vu et al., 2020) or the fractionation of upwelling
ocean material (e.g. Zolotov & Shock, 2001) may complicate using such surface
detections to place constraints on subsurface ionic abundances. However,
regardless of how directly the NaCl relates to its relative abundance in the
ocean, the concrete identification of endogenous chloride on Europa’s surface
warrants rethinking the long-standing idea that Europa’s internal ocean may be
sulfate-dominated (e.g. Kargel, 1991; Fanale et al., 2001).

##  5 Conclusions

Using HST/STIS spectroscopy, we detect a previously unseen absorption feature
near 230 nm, which we attribute to V3 centers of irradiated NaCl. The feature
appears strongest within leading-hemisphere chaos terrain, particularly Tara
Regio, and corresponds geographically to the proposed F-center absorption of
NaCl detected in recent visible-wavelength HST data. The excellent spectral
and spatial match between the 230 nm feature and NaCl confirms the presence of
NaCl within geologically young terrain on Europa, implying the presence of
NaCl in the interior ocean, and challenging past interpretations concerning
the composition of Europa’s endogenous salts.

Based on observations made with the NASA/ESA Hubble Space Telescope, obtained
at the Space Telescope Science Institute, which is operated by the Association
of Universities for Research in Astronomy, Inc., under NASA contract
NAS5-26555. These observations are associated with program $\\#$15095\. This
work is also associated with archival program $\\#$15789\. Support for
programs $\\#$15095 and $\\#$15789 was provided by NASA through grants from
the Space Telescope Science Institute, which is operated by the Association of
Universities for Research in Astronomy, Inc., under NASA contract NAS5-26555.
This work was also supported by grant number 668346 from the Simons
Foundation. S.K.T. is supported by the Heising-Simons Foundation through a 51
Pegasi b postdoctoral fellowship. All of the data presented in this paper were
obtained from the Mikulski Archive for Space Telescopes (MAST) at the Space
Telescope Science Institute. The specific observations analyzed can be
accessed via https://doi.org/10.17909/t9-5tsd-s772 (catalog
10.17909/t9-5tsd-s772). HST(STIS)

## References

  * Astropy Collaboration et al. (2013) Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33, doi: 10.1051/0004-6361/201322068
  * Bagenal & Dols (2020) Bagenal, F., & Dols, V. 2020, Journal of Geophysical Research: Space Physics, 125, e2019JA027485
  * Becker et al. (2022) Becker, T. M., Trumbo, S. K., Molyneux, P., et al. 2022, Planetary Science Journal, in review
  * Brown (2001) Brown, M. E. 2001, Icarus, 151, 190
  * Brown et al. (2022) Brown, M. E., Denman, W. T. P., & Trumbo, S. K. 2022, Planetary Science Journal, in press
  * Brown & Hand (2013) Brown, M. E., & Hand, K. P. 2013, The Astronomical Journal, 145, 110, doi: 10.1088/0004-6256/145/4/110
  * Brown & Hill (1996) Brown, M. E., & Hill, R. E. 1996, Nature, 380, 229
  * Carlson et al. (2002) Carlson, R. W., Anderson, M. S., Johnson, R. E., Schulman, M. B., & Yavrouian, A. H. 2002, Icarus, 157, 456, doi: 10.1006/icar.2002.6858
  * Carlson et al. (1999) Carlson, R. W., Anderson, M. S., Johnson, R. E., et al. 1999, Science, 283, 2062
  * Dalton et al. (2005) Dalton, J., Prieto-Ballesteros, O., Kargel, J., et al. 2005, Icarus, 177, 472
  * Dalton (2007) Dalton, J. B. 2007, Geophysical Research Letters, 34, L21205
  * Dalton et al. (2012) Dalton, J. B., Shirley, J. H., & Kamp, L. W. 2012, Journal of Geophysical Research: Planets, 117, doi: 10.1029/2011je003909
  * Denman et al. (2022) Denman, W. T., Trumbo, S. K., & Brown, M. E. 2022, Planetary Science Journal, in press
  * Doggett et al. (2009) Doggett, T., Greeley, R., Figueredo, P., & Tanaka, K. 2009, Europa, ed. R. T. Pappalardo, W. B. McKinnon, & K. Khurana (Tucson, AZ: University of Arizona Press), 137–160
  * Domingue & Lane (1998) Domingue, D. L., & Lane, A. L. 1998, Geophysical research letters, 25, 4421
  * Fanale et al. (2001) Fanale, F., Li, Y.-H., De Carlo, E., et al. 2001, Journal of Geophysical Research: Planets, 106, 14595
  * Fischer et al. (2015) Fischer, P. D., Brown, M. E., & Hand, K. P. 2015, The Astronomical Journal, 150, 164, doi: 10.1088/0004-6256/150/5/164
  * Fischer et al. (2016) Fischer, P. D., Brown, M. E., Trumbo, S. K., & Hand, K. P. 2016, The Astronomical Journal, 153, 13
  * Hand & Carlson (2015) Hand, K. P., & Carlson, R. W. 2015, Geophysical Research Letters, 42, 3174, doi: 10.1002/2015gl063559
  * Hendrix et al. (1998) Hendrix, A. R., Barth, C. A., Hord, C. W., & Lane, A. L. 1998, Icarus, 135, 79
  * Hibbitts et al. (2019) Hibbitts, C. A., Stockstill-Cahill, K., Wing, B., & Paranicas, C. 2019, Icarus, 326, 37
  * Johnson et al. (2019) Johnson, P. V., Hodyss, R., Vu, T. H., & Choukroun, M. 2019, Icarus, 321, 857
  * Johnson et al. (2002) Johnson, R., Leblanc, F., Yakshinskiy, B., & Madey, T. 2002, Icarus, 156, 136
  * Johnson (2000) Johnson, R. E. 2000, Icarus, 143, 429
  * Kargel (1991) Kargel, J. S. 1991, Icarus, 94, 368
  * Leblanc et al. (2002) Leblanc, F., Johnson, R., & Brown, M. 2002, Icarus, 159, 132
  * Leblanc et al. (2005) Leblanc, F., Potter, A., Killen, R., & Johnson, R. 2005, Icarus, 178, 367
  * Leonard et al. (2018) Leonard, E. J., Patthoff, D. A., Senske, D. A., & Collins, G. C. 2018, in Planetary Geologic Mappers Annual Meeting, Vol. 2066, 7008
  * Ligier et al. (2016) Ligier, N., Poulet, F., Carter, J., Brunetto, R., & Gourgeot, F. 2016, The Astronomical Journal, 151, 163
  * Mcclintock et al. (2005) Mcclintock, W. E., Rottman, G. J., & Woods, T. N. 2005, Solar Physics, 230, 225
  * McCord et al. (1998) McCord, T. B., Hansen, G. B., Fanale, F. P., et al. 1998, Science, 280, 1242
  * Nelson et al. (1987) Nelson, R. M., Lane, A. L., Matson, D. L., et al. 1987, Icarus, 72, 358
  * Nordheim et al. (2018) Nordheim, T. A., Hand, K. P., & Paranicas, C. 2018, Nature Astronomy, 2, 673
  * Paranicas et al. (2009) Paranicas, C., Cooper, J. F., Garrett, H. B., Johnson, R. E., & Sturner, S. J. 2009, Europa, ed. R. T. Pappalardo, W. B. McKinnon, & K. Khurana (Tucson, AZ: The University of Arizona Press), 529–544
  * Poston et al. (2017) Poston, M. J., Carlson, R. W., & Hand, K. P. 2017, Journal of Geophysical Research: Planets, 122, 2644
  * Trumbo et al. (2019) Trumbo, S. K., Brown, M. E., & Hand, K. P. 2019, Science Advances, 5, doi: 10.1126/sciadv.aaw7123
  * Vu et al. (2020) Vu, T. H., Choukroun, M., Hodyss, R., & Johnson, P. V. 2020, Icarus, 349, 113746
  * Vu et al. (2016) Vu, T. H., Hodyss, R., Choukroun, M., & Johnson, P. V. 2016, The Astrophysical Journal Letters, 816, L26
  * Zolotov & Shock (2001) Zolotov, M. Y., & Shock, E. L. 2001, Journal of Geophysical Research: Planets, 106, 32815, doi: 10.1029/2000je001413