acmlicensed 978-1-4503-6708-0/20/04 $15.00

# To Explain or Not to Explain: A Study on the
Necessity of Explanations for Autonomous Vehicles

Yuan Shen

Shanduojiao Jiang

Yanlin Chen

Eileen Yang

Xilun Jin

Yuliang Fan

Katie Driggs-Campbell

University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>University of Illinois at Urbana-Champaign  Urbana, USA<EMAIL_ADDRESS>
(2020)

###### doi:

https://doi.org/10.1145/3313831.XXXXXXX

###### keywords:

autonomous vehicle; explainable AI; explanation necessity

††conference: UIST’20, October 20–23, 2020, Minneapolis, MN, USA

##  1 abstract

Explainable AI, in the context of autonomous systems, like self-driving cars,
has drawn broad interests from researchers. Recent studies have found that
providing explanations for an autonomous vehicle’s actions has many benefits
(e.g., increase trust and acceptance), but put little emphasis on when an
explanation is needed and how the content of explanation changes with context.
In this work, we investigate which scenarios people need explanations and how
the critical degree of explanation shifts with situations and driver types.
Through a user experiment, we ask participants to evaluate how necessary an
explanation is and measure the impact on their trust in the self-driving cars
in different contexts. We also present a self-driving explanation dataset with
first-person explanations and associated measure of the necessity for 1103
video clips, augmenting the Berkeley Deep Drive Attention dataset.
Additionally, we propose a learning-based model that predicts how necessary an
explanation for a given situation in real-time, using camera data inputs. Our
research reveals that driver types and context dictates whether or not an
explanation is necessary and what is helpful for improved interaction and
understanding.

<ccs2012> <concept>
<concept_id>10003120.10003121.10003122.10003334</concept_id>
<concept_desc>Human-centered computing User studies</concept_desc>
<concept_significance>500</concept_significance> </concept> <concept>
<concept_id>10010147.10010178.10010224</concept_id> <concept_desc>Computing
methodologies Computer vision</concept_desc>
<concept_significance>500</concept_significance> </concept> <concept>
<concept_id>10010147.10010178.10010187.10010193</concept_id>
<concept_desc>Computing methodologies Temporal reasoning</concept_desc>
<concept_significance>500</concept_significance> </concept> <concept>
<concept_id>10003120.10011738</concept_id> <concept_desc>Human-centered
computing Accessibility</concept_desc>
<concept_significance>500</concept_significance> </concept> <concept>
<concept_id>10003120.10003123.10010860.10010859</concept_id>
<concept_desc>Human-centered computing User centered design</concept_desc>
<concept_significance>500</concept_significance> </concept> </ccs2012>

[500]Human-centered computing User studies [500]Computing methodologies
Computer vision [500]Computing methodologies Temporal reasoning [500]Human-
centered computing Accessibility [500]Human-centered computing User centered
design

##  2 introduction

Artificial intelligence (AI) is becoming prevalent in everyday life, powering
smart devices, personalizing assistants, and enabling autonomous vehicles.
However, people encounter difficulties to accept, understand, or trust this
technology [13]. This phenomenon is especially true in the self-driving car
industry, where people are hesitant to hand over control of the steering wheel
to AI [14, 21]. One of the reasons for this distrust is that people are
uncertain about how those sophisticated models make car control decisions.
With the ambiguity in the decision process, it is hard for passengers to tell
whether the model makes the right judgment given the current situation. The
consequence of this ambiguity includes financial loss, legal issues, and even
loss of human lives [5].

To improve the understanding of the self-driving car decision process better,
researchers have explored different perspectives to investigate explainable AI
(XAI) in the autonomous vehicle domain [9, 11]. For example, the state-of-the-
art research has shown that introducing explanations for self-driving control
decisions can increase trust [7]. The study found that people trust (and
prefer) explanations presented before the car takes an action, compared to
after-action, no explanation, and intervention-based explanations. Others
argued that explanations for autonomous vehicles should be modified based on
context [10].

To generate text-based descriptions, researchers collected the Berkeley
DeepDrive Explanation Dataset, where they extensively annotated every single
car decision with explanations [17]. However, the existing research over-
emphasized the benefits of introducing explanations for self-driving cars and
neglected the possibility that people might not need those explanations during
certain scenarios. In other words, it is still unclear when it is necessary to
introduce explanations about the autonomous decision, and whether we should
address individuals differently.

In this work, we examine when and how explanations should be presented to
users of autonomous vehicles. Specifically, we investigate which scenarios
people need explanations for and how the critical degree of an explanation
shifts with situations and driver types. We focus on text-based descriptions
with varying content to assess what information and narrative is preferred.
Our findings are validated in a survey-based user experiment online, in which
subjects imagine themselves as passengers of the vehicles in driving video
clips of a variety of scenarios. For each video, we record each participants’
reported explanation necessity rating, attentiveness score, and preferred
explanation content. During the post-survey, we collect participants’
responses on their driver types and general trust level on autonomous
vehicles.

Using the data collected, we aim to understand the relationship between driver
types and the necessity of an explanation, for particular contexts.
Specifically, in early tests, we observed disagreements among participants on
the explanation necessity level. For instance, we found that people tend to
agree on that explanations are necessary in near-crash events, but there was
no obvious agreement for ordinary or anomalous driving situations in
aggregate. As we will show, when examining factors like driver type (i.e.,
cautious, aggressive) and context, a relationship is uncovered linking the
necessity of explanations with the scenario and driver type.

Building on this insight from the user study, we also present a data-driven
model for estimating the explanation necessity level online. This model would
allow us to assess when and if an explanation is needed, and can be tuned to
particular user preference. To explore the explanation necessity level for
more diverse scenarios, we present a self-driving explanation dataset by
augmenting the Berkeley Deep Drive Attention dataset [33]. Each video in the
dataset is annotated with explanation content, the explanation time interval,
and associated measures of the necessity for all 1103 video clips. The
associated explanation necessity score ranges from 0 to 1, suggesting how
critical an explanation is needed for the given scenarios. Our proposed model
makes personalized, real-time prediction about whether or not an explanation
is needed. The model takes a sliding window of a camera video as the input and
outputs a binary decision on whether an explanation is necessary.

The proposed solution can adapt to the state-of-the-art explanation generation
model by suggesting explanation moments. The combined system can generate a
text-based explanation at the right moment for different individuals, where
existing methods continually generate explanations with little to no
personalization. Tuning the frequency of explanations (or generally
interventions) are key for improving acceptance and trust in autonomous
systems [27, 6].

In summary, our contributions are as follows:

  1. 1.

Our user study shows that the explanation necessity level is affected by
driver types and driving scenarios or context.

  2. 2.

We find that people tend to agree on the explanation necessity more for near-
crash/emergent driving scenarios, and less for the ordinary driving situation.

  3. 3.

We present a dataset with video clips labeled with an explanation necessity
degree, an explanation moment, and first-person explanation content.

  4. 4.

We propose a model that makes personalized predictions in real-time about
whether or not an explanation is needed.

The paper is organized as follows. In the next section, we present related
work on XAI for the autonomous vehicles and its relationship to trust and user
acceptance. Then, we introduce our user study experiment on the necessity of
explanations. Using the insight of how explanation necessity varies between
scenarios, we present an explanation necessity dataset, which augments the
original Berkeley Deep Drive Attention dataset. Finally, with our explanation
necessity data available, we propose a learning-based model that can predict
whether an explanation is needed in real-time. We conclude with a discussion
of the implications, limitations, and findings of our work.

##  3 related work

As has been found for many AI-enabled systems, autonomous cars (and driving
assistance systems) are likely better off with self-explaining capabilities,
which we hypothesize would facilitate its acceptance among mainstream
consumers.

###  3.1 Benefits of Explainable AI

There has been a considerable amount of studies on the benefits of XAI [2].
Specifically in the area of autonomous vehicles, Choi et al. concluded that
trust is a significant determinant for people’s intention of using self-
driving cars [4]. With the role of trust established, Koo et al. demonstrated
that providing the why information, which specifies the reasoning of an
autonomous vehicle’s action, created the least anxiety and the highest trust
[18]. These propositions of the relationship between humans and AI were soon
confirmed by Petersen et al., who investigated trust in vehicles with driver
assistance systems by manipulating drivers’ situational awareness. Their study
reached the conclusion that situational awareness promoted and moderated the
impact of human trust in autonomous vehicles [20].

###  3.2 Forms of Explanations

To properly provide the “why" information or the situational awareness
mentioned above, one needs to specify how an explanation will be provided.
Wiegand G., et al. conducted a user experiment on the effect of nine visual
explanations about the driving scenario [29]. In their study regarding how
these explanations should be provided, Ruijten et al. presented an intelligent
user interface that mimics human behavior [25]. Their research took place in a
fixed-base driving simulator, and the simulator would have a spoken
conversation with the participants whenever an automated action took place.
Performed in a similar experiment setting, Koo et al. ’s study, which also
took place in a simulator, provided the explanations via voice alerts [18].
Different from the above formats, our experiment takes the form of an online
survey, without specifying how explanations will be provided, as we are
interested in the explanation timing and content.

###  3.3 Contents of Explanations

Besides the various formats of providing the information, there are also
studies about the optimal content of explanations. Studies in user interface
design have shown that the amount and type information conveyed impacts user
trust and situational awareness [24, 23]. As mentioned above, Koo et al.
analyzed the different outcomes of providing different explanations. In
specific, providing only how information which describes the action itself
(e.g. “The car is braking") led to poor driving performance; providing only
why information which describes the reasoning for actions (e.g. "Pedestrians
ahead") led to better driving performance and was preferred by drivers;
providing the how and why information led to the safest driving performance
but increased negative feelings in drivers [18]. While Koo’s study categorized
possible explanations and suggested the optimal explanation content, they
focused on providing explanations in a third-person perspective. They ignored
the different reactions of participants when using the first-person narrative.

In our experiment, we investigated the favorability rankings of explanations
among: "cause and effect" information in the first-person perspective, "cause
and effect" information in third-person perspective, "effect" information in
the first-person perspective, and "effect" information in third-person
perspective. Furthermore, we also asked our participants whether they would
prefer to add a human-centered component (e.g., "Don’t worry") to their top-
ranked explanation content choice.

###  3.4 Timing of Explanations

Apart from the forms and contents of explanations, we argue that it is also
crucial to explore the timing of explanations. Timing for warnings and
potential interventions is a key concern for (semi-) autonomous vehicles [19],
motivating the fact the identification of key events is crucial for
trustworthy autonomy. Koo et al. claim that it is critical to provide
information to drivers/passengers ahead of an event [18]. Haspiel et al.
designed a user experiment that introduces the importance of timing
explanations in promoting trust in AVs [12]. Their study has discovered a
pattern that suggests that explanations provided before the AV action promote
more trust than explanations provided after.

Although existing studies have come to the agreement that explanations are
more meaningful when put before an autonomous vehicle action, they failed to
take into account that the necessities of providing explanations vary in
different driving scenarios. In our experiment, we introduced a "critical
score," which is a number ranging between 0 and 1, indicating how necessary an
explanation is needed at each timestamp.

###  3.5 Definition of Critical Score

In previous studies about the relationship between human and autonomous
vehicles, there have been various definitions of a critical score, namely, how
critical a driving situation is for the passenger. Notably, Yurtsever et al.
asked ten participants to watch driving video clips and give a score
(subjectively) for the risk of maneuver seen in the videos [34]. After
normalizing each annotators’ ratings and taking the mean score as the final
risk rating, they defined the top 5% of the riskiest videos as risky. However,
it is necessary to distinguish "critical" (subjective) driving scenarios from
"accident likely" (objective) situations. In another study, it has been proven
that the perceived risk by humans is not necessarily proportional to the
actual collision or accident probability associated with a specific driving
situation [8]. Keeping these in mind, we proposed a human-centric, XAI-
friendly definition of critical score: the necessity of explanation related to
a particular driving maneuver.

###  3.6 Self-driving Explanation Dataset

The existing explanation dataset for self-driving suffers from various issues.
For example, Berkeley DeepDrive eXplanation Dataset exhaustively labeled 6970
driving clips with explanations in specified video intervals [17]. However, a
large portion of their driving clips is uneventful samples (e.g., cruising on
the highway with constant speed), where humans require little need for the
self-driving system to explain the situations. Furthermore, portions of some
video clips are anomalous where the drivers do not follow the traffic rules
(e.g., does not stop at a stop sign), thus have a poor (or illogical)
explanation given the rules of the road. Meanwhile, as the explanations
focused on describing the car model’s decisions, the explanation content may
not be ideal to promote a smooth conversation with human.

##  4 user study

###  4.1 Overview

Figure 1:  User study overview: Given the video dataset, we identify cluster
centers to give us representative driving scenarios to be used in the user
study. These key videos were used to collect feedback on the necessity of
explanations as well as user specific data.

To understand people’s need for a self-driving explanation for different
scenarios, we conducted an online survey-based experiment (Figure 1). The
experiment took 40 minutes, where we showed participants driving video clips
and collected their responses. The goal of this experiment is to understand:
(1) how necessary is a text-based explanation about self-driving car actions
for different scenarios; (2) what the generally preferred explanation content
is and if this is related to context; and (3) the relationship between user
trust and explanations for autonomous vehicles.

###  4.2 Hypothesis

In our experiment, we target the following outcomes (dependent variables):
explanation necessity, preferred explanation content, and user trust. We
manipulate, vary, or estimate the following independent variables and
influential factors: attention (how much attention participants pay to the
videos); driver types (aggressive or cautious); driving scenarios; explanation
content (cause, effect, narrative type, Table 1); and presence of
explanations. Based on the above three sets of dependent / independent
variables, we derived three hypotheses:

  1. 1.

Explanation necessity is correlated with attention, driver types, and driving
scenarios.

  2. 2.

User’s preferred explanation content is dependent on driving scenarios.

  3. 3.

The presence of explanations will increase a user’s trust in the automated
vehicle.

###  4.3 Participants

In total, we have 18 participants for this user experiment. The majority of
our participants are college students age between 18 and 51. The participants
are qualified as long as they can see the video clips through our survey
system. Among the participants, 16 out of 18 participants have a driver’s
license, and their driving experience is evenly distributed from 0 to 6 years,
with one participant having more than six-year experience.

###  4.4 Sample Strategy

The driving video clips are sampled from our explanation dataset, described in
the Dataset section. To capture a variety of typical scenarios, we conducted a
text-based clustering using our annotated explanations for each video clip in
our dataset.

Our goal is to capture the different but representative scenarios in the
dataset. We used hierarchical clustering with average linkage [16]. In
specific, we pre-processed the input text by lowercase and converting to TF-
IDF score [22]. We used cosine similarity for the distance metric of the
clustering. In the end, we used the videos in the 38 cluster centers for the
user experiment.

###  4.5 Study Design & Procedure

We used Google Forms as our platform for this user experiment. The experiment
took 40 minutes to finish, including a break every fifteen minutes.

Table 1: Formats of explanation contents. We prepared explanation contents in four different formats manually for each of the 38 driving scenarios. During the user experiment, we asked participants to rank the proposed explanation contents. Format |  Narrative |  Example
---|---|---
action + reason |  first-person |  I’ll slow down because the traffic light is broken.
action + reason |  third-person |  The car will slow down because the traffic light is broken.
action |  first-person |  I’ll slow down.
action |  third-person |  The car is about to slow down.

During the experiment, our participants watched 38 independent short driving
video clips (as described in the previous subsection). Participants were told
to imagine themselves as passengers riding in the vehicles. While the video
was playing, the participants were free to be slightly distracted. One thing
to note here is that the video shown to participants was the raw video,
without an explanation. Moreover, we randomized the order of sampled videos to
reduce bias, and each participant watched all sampled videos at the end of the
experiment. After each video, participants answered several follow-up
questions. The users rated how necessary an explanation is for the clip,
referred to as a necessity score. Then, they described how attentive they were
while watching the clip. Finally, they ranked several explanation candidates
that we had prepared for each of the sampled videos. In particular, we
prepared four different types of explanation contents separately for all of
the 38 driving scenarios, as presented in Table 1.

During the post-study phase, we prepared several questions related to how the
text-based explanation can affect people’s trust in autonomous vehicles, and
to what the participant’s driver types were. In particular, we asked
participants about seat preferences under the ordinary car, autonomous
vehicle, and autonomous vehicle with an explanation (Figure 2).

Figure 2: Driver seats: Among the post-study questions, we asked participants
about seat preferences under ordinary car, autonomous vehicle and autonomous
vehicle with explanation.

###  4.6 Quantitative Analysis

We summarized our findings in three different aspects:

  1. 1.

A correlation analysis between different scenarios and the reported
explanation necessity level using Pearson correlation and Point-Biserial
Correlation was performed [3, 28].

  2. 2.

To test whether there is a global preferred explanation content format, we
performed a Friedman test to check the distribution of different explanation
options [30].

  3. 3.

We analyzed the relationship between the presence of explanations and trust
through the seat preference changing under different conditions.

For this section, we start by describing general statistics about the user
study result, and then we elaborate on the three aspects above.

To identify driver types, we considered a participant aggressive if she
satisfies any of the following condition: (1)The actual driving speed usually
is above 35 mph for the road with the speed limit at 30 mph (2) She describes
her driving type as aggressive explicitly (3) She reports changing lane
frequently even if unnecessary. Otherwise, we will consider the participant as
cautious.

####  4.6.1 General Statistics

From our post-study questions, we learned some general views from our
participants on autonomous vehicles. Even though 84.1 % of people are
personally excited about autonomous vehicles by giving the rating higher than
five from a 1 to 10 Likert scale, participants expressed general doubts on the
feasibility of autonomous vehicles, where 77.8 % of participants expressed a
low level of trust in autonomous vehicles. Among our participants, only 10.6 %
of people believe the self-driving techniques will be readily available to the
public in the next two years. On the other hand, participants expressed
overall trust in the reliability of explanations from autonomous vehicles -
72.2% of the participants rated scores higher than five from a 1 to 10 Likert
scale. This finding slightly suggests introducing the right explanation
contents has the potential to influence people’s trust in autonomous vehicles.

From the driving clip questions, we learned that the average of explanation
necessity level for each of the 38 driving scenarios ranges from 2.27 to 8.22,
in a 1 to 10 Likert scale. We observed that people generally disagree on how
necessary an explanation is needed for the same scenario, with an average
standard deviation at 2.97 across 38 driving scenarios. Furthermore, we
observed that the average explanation necessity score given by aggressive
drivers is 18% lower than the cautious driver.

####  4.6.2 Correlation Analysis on Explanation Necessity

We did correlation analysis on the explanation necessity level for the
following four different aspects: attentive level, driver types, the presence
of motion sickness, and driving scenarios.

For the attentive level, we used the Pearson Correlation to calculate the
correlation between explanation necessity and attention score since they are
both continuous variables [3]. The result is 0.19, which indicates that there
exists a positive relationship between being attentive to the video clips and
the need for scenario explanations. In other words, the more attention a
participant pays to the video clips, the more necessarily she needs an action
explanation from autonomous vehicles.

For driver types, we used the Point-Biserial Correlation to calculate the
correlation between explanation necessity and driver type [28]. The resulting
correlation is -0.14, which means that there exists a negative relationship
between being an aggressive driver and the need for scenario explanations. In
other words, the more aggressive a driver is, the less he or she needs an
account from the vehicle.

Similarly, for the presence of motion sickness among the participants, we used
the Point-Biserial Correlation to calculate its correlation with explanation
necessity. The result is 0.046, which indicates that there is a weak positive
relationship between motion sickness and needs for explanations.

Table 2: Correlation between Necessity and Different Scenarios: We used the Point-Biserial Correlation to calculate the correlation between explanation necessity and the scenario. This table suggests explanation necessity level is not necessarily related to speed-reducing but related to whether anything that is out of expectation occurs.  Scenarios | Correlation with Necessity
---|---
Near Crash | 0.1818
Merge/Switch Lane | 0.1218
Slow down | 0.1134
Parking | 0.0979
Pedestrians | 0.0235
Stop Sign | 0.0145

Finally, for driving scenarios, we tagged binary attributes for the 38 driving
scenarios in advance (e.g., whether the vehicle in a video slows down, or
whether the video is a near-crash scenario). Then, we used Point-Biserial
Correlation to calculate the correlation between explanation necessity and the
corresponding scenario, as shown in Table 2) [28]. For example, the first row
shows that the correlation between explanation necessity and whether the video
is a near-crash scenario is 0.18, indicating that an explanation is highly
necessary for near-crash situations.

The result of the correlations of explanation necessity with attention, driver
types, and driving scenarios proves that our hypothesis 1 - Explanation
necessity is correlated with attention, driver types, and driving scenarios -
holds.

####  4.6.3 Explanation Content Preference

To investigate if there is a generally preferred explanation (Table 1) format
across all scenarios, we performed Friedman tests separately for different
driving situations to deal with the ranking data of explanation contents [30].
Our null hypothesis, $h_{o}$ is that there is no difference for different
explanation contents. We set the $\alpha$ level to be 0.05. According to our
result, only 16 out of 38 scenarios reject the null hypothesis. However, we
did not find those 16 scenarios sharing quantitative attributes based on our
data. Therefore, we concluded that there is no globally preferred explanation
format for self-driving scenarios, and thus our hypothesis 2 - Explanation
content is correlated with driving scenarios - does not hold.

Figure 3: Seating Preference in Different Vehicles. This figure shows the
percentage of preferred seat among front driver seat, front passenger seat,
back driver side, and back passenger side in three types of vehicles: ordinary
vehicles (Ordinary), autonomous vehicles (AV), and autonomous vehicles with
explanations (AV w/ Explanations).

####  4.6.4 Explanation and Trust

In the post-study questionnaire, we asked our participants about their
preferred seats in three types of vehicles: ordinary vehicles, autonomous cars
without explanations, and autonomous vehicles with explanations. From the
results of the changes in seat preferences, as shown in Figure 4, we derived
two types of sentiment changes. One is the relaxation of participants, which
is reflected by changing from front seats to back seats because of the comfort
back seats could bring. Another one is due to the anxiety of the participants.
People either change from other positions to the driver seat so that one can
take control of the vehicles when necessary, or change from front seats to
back seats because statistically, back seats are safer than front seats.

When the condition changed from an ordinary car to an autonomous vehicle
without explanations, among people who changed seats, only 12.5% changed due
to feeling relieved and relaxed in autonomous cars. At the same time, 87.5% of
people changed their positions because they do not fully trust the vehicles
and would like to take control if anything unexpected happens quickly. Then,
we found that among those participants who changed their seat preferences when
the vehicle condition changed from an autonomous vehicle without explanations
to an autonomous vehicle with explanations, 83.3% changed their seats because
they feel relieved and relaxed in an autonomous vehicle with explanations.

Therefore, we conclude that providing explanations in general, can help people
be less stressed and worried in an autonomous vehicle, which we believe
indicates an elevation of trust. Therefore, our hypothesis 3 - Trust in the
vehicles is correlated with the presence of explanations - holds.

Figure 4: Seat transitions matrix. The vertical axis represents the seat a
participant was at, and the horizontal axis represents the seat a participant
changed to. The back seat represents both the driver-side and passenger-side
at the back, namely, position B and D in Figure 2.

###  4.7 Qualitative Analysis

Besides the quantitative analysis above, we performed a qualitative study on
the relationship between scenarios and explanation necessity. We noticed that
the standard deviation for explanation necessity scores given by the
participants varies a lot for different situations. To investigate whether
there are common properties among scenarios that people mostly agree or
disagree on, we compared the scene that has the highest standard deviation of
explanation necessity with the situation that has the lowest standard
deviation of explanation necessity, as shown in Figure 5. From this
comparison, we observed that people tend to agree on the near-crash/emergent
driving situations (e.g., cars cutting-in suddenly). On the other hand, for
the ordinary scenarios (e.g., driving smoothly on the highway), people’s
opinion on explanation necessity varies a lot.

Figure 5: Frame sequences for videos with the highest and lowest standard
deviation on the necessity for explanation. The first row corresponds to the
video with the highest std, 3.41, and the second row corresponds to the one
with the lowest std, 2.11. The average standard deviation for the 38 video
clips is 2.97. We noticed that people are easier to reach an agreement on the
explanation necessity for the near-crash/emergent driving situations. For
example, participants gave similar explanation necessity for the second row,
which describes a sudden brake between 4 seconds and 6 seconds. However, for
those ordinary or not emergent driving situations, people’s opinions for
explanations vary a lot. For instance, participants have different views on
the frame sequence in the first row, which is about a car gradually speeding
up.

###  4.8 Discussion

Through our quantitative and qualitative analysis, we found that factors
related to driving scenarios and passenger identities influence the need for
explanations, and therefore generalizing the explanation necessity level for
different driving scenarios is challenging. In other words, the explanation
necessity has to be analyzed/predicted on a case-by-case basis. We are
wondering if a learning-based model can achieve this goal with temporal
representations of a driving scene. Given the rich diversity of driving
scenarios, we need a large-scale dataset related to the explanation necessity
to start with the training process. Therefore, we annotated explanation-
related metadata on the video clips of the BDD-A dataset[33]. We introduced
our explanation necessity dataset in the next section.

##  5 dataset

###  5.1 Overview

Our user study suggests that the explanation necessity is correlated with
driving scenarios, and has to be analyzed on a case-by-case basis. Using this
insight, we aim to build a data-driven model that can learn to predict
necessity scores that can be tailored to the driving types and contexts.

In order to build this model, we present an explanation necessity dataset for
autonomous vehicles. We limit our scope of explanation format to text-based
explanation. The purpose of this dataset is to provide precise, case-by-case,
and first-person perspective explanations that resolve the following issues:
(1) when people need a reason for a driving decision; (2) how critical the
explanation should be; and (3) what the first-person perspective explanation
content it should be.

Instead of from the Berkeley DeepDrive Explanation (BDD-X) dataset [17], we
selected the driving video clips from the Berkeley DeepDrive Attention (BDD-A)
dataset [33], which initially contains 1232 braking event driving videos
captured by a front-mirror dashcam. Even though the dataset size of the BDD-X
dataset is six times greater than the BDD-A dataset, through our empirical
analysis, we find a large portion of the videos are ordinary driving scenes
(e.g., driving on the highway), which do not contain moments worth explaining
explicitly. And the average video duration of the BDD-X is around 30 seconds
to 60 seconds, which is much longer for the BDD-A dataset, whose video usually
lasts approximately 10 seconds. Besides, in terms of information, BDD-A also
provides human gaze, car speed, and GPS metadata per frame.

###  5.2 Dataset Assumption

Our high-level assumption is similar to the data collection assumption
proposed by Xia Y., et al. [33], where participants imagine themselves in the
car of the driving videos. Specifically, we made the following assumptions:

  1. 1.

The actual driver in the video follows the traffic laws.

  2. 2.

The car action in the video is safe. In other words, the car action should not
put the passengers into a high-risk car accident.

  3. 3.

The recipient of the explanation would be a passenger of a fully autonomous
vehicle. This assumption means that the perspective and sense/ability of
control is different.

  4. 4.

Every driving clips has at least one explanation moment. But the degree of
explanation necessity can vary.

If any video clip fails to obey any of the assumptions, we removed it from our
dataset.

###  5.3 Dataset Statistics

Our dataset contains 1103 driving video clips in total. From the video clips
in the BDD-A dataset, we filtered out driving clips that did not meet our
assumption criteria (e.g., drivers did not follow the traffic rules, poor
videos quality like skipping frames) [33]. Five annotators were recruited for
this dataset; the background of the annotators is college students age between
18 to 25 years old who have a valid driving license in the US.

###  5.4 Data Model

Table 3: Data Model. This table shows the attributes in the dataset for one video example. The necessity score here reflects the generalized score extracted from 5 different individual’s responses. Attribute |  Type |  Content
---|---|---
vid |  string |  video id
message |  string |  text explanation in first-person perspective
gazemap |  mp4 |  human gazemap at each timestamp
video |  mp4 |  driving video
necessity score |  float |  necessity degree for explanation
speed |  float |  car speed at each timestamp
course |  float |  car speed at each timestamp
explanation interval |  float tuple |  time segments for explanation to occur

Every video clips is annotated with one explanation moment. Each explanation
contains a time interval that the explanation should take place, a first-
person perspective explanation, and an explanation necessity score, indicating
how critical an explanation is needed at one moment (Table 3).

Instead of a binary response, the explanation necessity score is a floating
number ranging from 0 to 1. To get a generalized explanation score for each
driving clips, we collected responses from 5 different people and used
truncated mean to get a general critical score.

###  5.5 Data Collection Platform

We made an iOS application (Figure 6) to annotate the driving clips dataset,
with Firebase as our backend support. The app randomly distributed video clips
from BDD-A to annotators, collected explanation responses, and upload the
results to the cloud backend. At the end of the data collection process, each
person has annotated every single video clip in the dataset.

Figure 6: Workflow on the annotation app. For each of the video clips, the
data annotators clicked on Record button at the moment that needs an
explanation from the self-driving system. Then, they entered a score
indicating how critical an explanation is needed and gave a human-centered
explanation. Finally, when they were directed back to the video page, they
double-checked and fine-tuned the explanation moments. To note, the targeted
range of our annotated timestamp is between the moment that people can get the
first visual perception and the moment that the car takes action.

###  5.6 Data Collection Procedure

For each of the video clips, the data annotators started by playing the
driving video clips (Figure 6). Once they reached a point where they
considered the driving scene needed an explanation, the data annotators
clicked on the Record button. Then, a pop-up window was presented to ask the
participants to give a floating score, indicating how necessary people need an
explanation for this moment based on their judgment. Finally, they fine-tuned
the timestamp to reflect the moment that needs an explanation. The cases were
removed in which either the driver does not follow traffic rules, or the
driver action is too risky to be considered as the desired self-driving
behaviors. For the explanation moment, we focused on recording before-action
explanation, because previous research indicated people trust more on the
before-action interpretation in autonomous vehicle settings [12].

###  5.7 Data Post-Processing Steps

To extract a general explanation necessity score from different people’s
reactions per example, we used truncated mean, a statistical measure of
central tendency [31]. In specific, for each data piece, we calculated the
average of the explanation necessity scores after discarding the highest and
lowest score. The advantage of the truncated mean is that it can reduce the
influence of extreme scores.

As for the explanation timestamp for each video example, we sorted the records
annotated by different people for the corresponding explanation event. In
other words, the format of the explanation time is a time interval that
captures the moments where the relevant explanations should occur.

##  6 Model

###  6.1 Overview

Figure 7: Model Architecture. The goal of our model is to identify explanation
moments in real-time. The input is a sliding window of video frames, which
lasts for four seconds, and it outputs a binary decision on whether to
generate an explanation at the end of the input video. First, the model
generates a temporal representation from the video data. In particular, we use
the foveal feature map, which the model focuses on extracting visual features
at the human gaze regions for each frame, proposed by Xia Y., et al. Then, we
use a set of fully connected layers to classify the visual features so that we
can output a necessity logit at the end.

Our user study suggests that the explanation necessity has to be analyzed on a
case-by-case basis. To predict the need for explanation of a particular
scenario, we propose a classification model that can predict in real-time
about whether a person needs an explanation. The proposed solution can be
combined with the state-of-the-art explanation model to generate a text-based
explanation at the right moment. The model is trained on our explanation
dataset. The input of the model is image data at each time frame. The model
outputs a time series of binary decisions for explanation. The architecture of
the model is in Figure 7.

For this task, we considered it as a classification problem instead of a
regression problem. The reason is that, due to the labeling noise by a human,
the annotated necessity score can only reflect a relative level of explanation
necessity instead of the absolute value. Therefore, we transformed the
regression task to a classification task by introducing a pre-specified
necessity threshold, $p_{0}$. The threshold, $p_{0}$ is influenced by human
personal related properties, like driver types. For this paper, we assume
$p_{0}$ is available directly. We leave how we should infer about the
threshold value $p_{0}$ based on different personal attributes for future
work.

###  6.2 Task & Notation

The data of the model is a sliding window of video which lasts for four
seconds, and a binary label $y$ indicating explanation necessity at the end of
the video. Our task is to train a classifier $f$ to predict whether an
explanation is necessary at the end of the input video clip:

$\hat{y}_{i=N}=sgn(f(V)-\sigma),$ (1)

where $\hat{y}$ represents the predicted explanation necessity, $i$ represents
the frame index (1,2,..,N), $V$ is the input video frame sequence, $\sigma$ is
explanation necessity threshold that depends on the passengers’ attributes
(e.g., driver types), and $sgn$ represents sign function.

###  6.3 Model Detail

First, the model generates a temporal representation of the image for the last
frame in the video. To generate visual features from video frames, we use the
Foveal Visual Encoder [32], proposed by Xia Y. et al., 111Compared with
Imagenet pre-trained Alexnet features, the foveal features improved our test
AUC from 9.9% to 24.6%. The proposed image encoder predicts small regions
where human eyes will focus on each frame, then extracts the image features
only for the focused area from a high-resolution video. Then, our model passes
those frame features ($A_{i}$ in Figure 7) into Conv2dGRU module [26] and
several convolution layers to generate a temporal-spatial representation,
$v_{i}^{s}$ for each frame $i$, where i = 1,2,…, N and $s$ stands for spatial.
We extract the output of the last frame, $v^{s}_{i=N}$, for the next step,
where $v^{s}_{i=N}\in\mathbb{R}^{3}$.

The model converts the previous spatial features, $v^{s}_{i=N}$, into a linear
representation, $v_{i=N}\in\mathbb{R}$, in the spatial transform module. With
the benefits of zero information loss, the model flattens the spatial features
directly. We also experimented with the weighted sum approach across spatial
locations, suggested by Kim J., et al. in Equation (1) of their paper [17] for
their context vector construction. However, the performance decreased due to
information loss during their feature compression across spatial locations.

We concatenate the visual features, $v_{i=N}$ and acceleration, $a_{i=N}$
together. Then, we use several fully connected layers with ReLU nonlinear,
BatchNorm [15], and dropout layers followed. The last step is to use Sigmoid
function to generate predicted necessity scores, $\hat{y}$, where
$\hat{y}\in[0,1]$. We use Binary Cross-Entropy Loss as our loss function.

In summary, we build a recurrent model to learn explanation necessity. Through
our comparison study on different model architecture, we found the foveal
feature encoder has better performance in terms of test AUC over Alexnet pre-
trained network, even though the foveal feature encoder has information loss
on unattended regions at each frame. On the other hand, we learned that
flattening works better than the weighted sum proposed by [17] in the Spatial
Transform Module for our purpose.

###  6.4 Training Details

We split our dataset into a 70% training set, a 10% validation set, and a 20%
test set. We sampled the video frames at 10 Hz. Then, we extracted training
data for video with a sliding window size of 40 frames, (i.e., 4 seconds). For
those data piece that has explanation necessity score higher than the specific
necessity threshold, we extracted the video window with its end index within
the critical time interval (described in the Dataset section) and marked the
explanation label as 1. For the rest, we randomly sliced a window with a size
of 40 frames in the videos and marked as 0.

We use Adam optimizer with a learning rate of 1e-2, alpha 0.999. The dropout
rate is 0.7. To reduce the data imbalance, we did a weighted sampling based on
the explanation necessity labels. We set the batch size to be the same as the
size of our training dataset to help converge faster. The training of our
model took around 1 hour on Nvidia Tesla V100 GPU with 16GB GPU memory, for
300 epochs.

###  6.5 Evaluation

To evaluate our model classification performance, we calculated Area Under the
Receiver Operating Characteristic Curve (ROC AUC) on the test set with the
Sklearn library function. We trained our model for different explanation
necessity thresholds, $p_{0}$, based on the general explanation necessity
level for each video. We report the result in Table 4. As shown in the table,
the model performance decreases with a lower $p_{0}$.

One possible reason is that we are using general explanation scores calculated
from different people’s ratings. From our user study result (Figure 5), we
learned that for dull video scene without any near-crash case, explanation
necessity rating tends to have more considerable variance among people, which
makes it harder for the model to distinguish scenarios that people do not get
an agreement on.

Table 4: Model Performance in terms of test set AUC score. $p_{0}$ stands for explanation necessity threshold. As shown in the table, the model performs better in high explanation necessity threshold. However, for the low classification threshold, the model performance decreases.  $p_{0}$ | random guess | our model
---|---|---
0.5 | 0.5803 | 0.6295
0.6 | 0.5427 | 0.6438
0.7 | 0.5359 | 0.6794

##  7 discussion & future work

This paper investigates in-depth about when explanations are necessary for
fully automated vehicles. There are two main aspects of results that are
interesting for discussion.

We initially hypothesized that people would reach a certain amount of
agreement on the explanation necessity level for different scenarios. However,
our user study results indicate that people’s opinions on explanation
necessity might be the opposite of a certain amount of driving scenarios.
Through our qualitative study, people tend to agree on the explanation
necessity more for near-crash/emergent driving scenarios and less for the
ordinary driving situation. Through our quantitative research, We found that
both contexts and individual attributes had a significant influence on the
desire for explanation. We are wondering how the passengers will respond if we
provide too many explanations to passengers. One possibility is that the
overuse of account annoys the passengers such that they turned off the
explanation features completely. Another chance is that they might get
insensitive to reasons, which might ignore the critical one. They also might
be completely okay with additional comments and have no adverse influence. The
first two cases will discourage the effects of explanations. We will
experiment on this track for future work.

We previously considered that explanation of necessity should be highly
correlated with speed changes. In other words, passengers should be more
likely to ask for an explanation during speed decreasing moments. However, our
correlation analysis between explanation necessity level and scenario types
shows that this is not necessarily true. From Table 2, we noticed scenarios
related to stop signs have the lowest correlation with necessity, even though
the car is expected to decrease speed whenever approaching a stop sign.
However, if we look at the scenarios that have a higher correlation with
explanation necessity, they are, more or less, events that are not aligned
with the passenger’s original expectation. In other words, how different the
scenario is from the expectation of passengers might be positively related to
the explanation necessity level. We plan to test this hypothesis for future
work.

Together with this paper, we presented a model that can identify explanation
moments in real-time. To provide personalized prediction, we proposed using
the pre-specified explanation thresholds to capture personal attributes, like
driver types. However, in this paper, we did not explore how we should build
the mapping function from individual characteristics to threshold explicitly.
We leave this exploration for our future study.

Finally, we quantified the explanation necessity level for different scenarios
by collecting people’s ratings directly. However, due to difference in
individual criteria for explanation necessity, the recorded explanation
necessity for scenarios have relatively high standard deviations so that it
makes it hard to argue the general explanation necessity level for a given
situation. Moreover, a linear representation to capture explanation necessity
might be problematic, since the necessity level might be in the high-dimension
space where each dimension corresponds to a different factor to necessity,
like risk. In future work, we plan to explore using reinforcement learning to
systematically represent explanation scores for different states based on the
rewards from future events.

##  8 conclusion

In this paper, we investigated in-depth on the necessity of explanations for
autonomous vehicles. Our user experiment results showed that the need for
explanation depends on specific driving scenarios and passenger identities.
Along with this paper, we presented a self-driving explanation necessity
dataset with first-person explanations and associated measure of necessity for
1103 video clips, augmenting the Berkeley Deep Drive Attention dataset [33].
Finally, we proposed a learning-based model that can offer a personalized
prediction on how necessary an explanation for a given situation in real-time,
using camera data. Our work can highlight the importance of context and human
elements in explainable AI for autonomous vehicles.

## References

  * [1]
  * [2] Jose M. Alonso. 2019.  From Zadeh’s Computing with Words Towards eXplainable Artificial Intelligence. In Fuzzy Logic and Applications, Robert Fullér, Silvio Giove, and Francesco Masulli (Eds.). Springer International Publishing, Cham, 244–248.
  * [3] Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009.  Pearson correlation coefficient.  In Noise reduction in speech processing. Springer, 1–4.
  * [4] Jong Kyu Choi and Yong Gu Ji. 2015.  Investigating the Importance of Trust on Adopting an Autonomous Vehicle.  International Journal of Human–Computer Interaction 31, 10 (2015), 692–702.  DOI:http://dx.doi.org/10.1080/10447318.2015.1070549
  * [5] Biju Dominic. 2018.  Blameless driverless cars (and their consequences).  (2018).  https://www.livemint.com/Opinion/DB7zd5kjJWib2OWkqbbuaM/Blameless-driverless-cars-and-their-consequences.html
  * [6] Katherine Driggs-Campbell, Victor Shia, and Ruzena Bajcsy. 2015.  Improved driver modeling for human-in-the-loop vehicular control. In IEEE International Conference on Robotics and Automation (ICRA). 1654–1661.
  * [7] Na Du, Jacob Haspiel, Qiaoning Zhang, Dawn Tilbury, Anuj K. Pradhan, X. Jessie Yang, and Lionel P. Robert. 2019.  Look who’s talking now: Implications of AV’s explanations on driver’s trust, AV preference, anxiety and mental workload.  Transportation Research Part C: Emerging Technologies 104 (2019), 428 – 442.  DOI:http://dx.doi.org/https://doi.org/10.1016/j.trc.2019.05.025
  * [8] Ray Fuller. 2005.  Towards a general theory of driver behaviour.  Accident; analysis and prevention 37, 3 (May 2005), 461—472.  DOI:http://dx.doi.org/10.1016/j.aap.2004.11.003
  * [9] L. H. Gilpin, D. Bau, B. Z. Yuan, A. Bajwa, M. Specter, and L. Kagal. 2018.  Explaining Explanations: An Overview of Interpretability of Machine Learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA). 80–89.
  * [10] Jon Glomsrud, André Ødegårdstuen, Asuncion Clair, and Oyvind Smogeli. 2019.  Trustworthy versus Explainable AI in Autonomous Vessels.
  * [11] David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. 2019.  XAI—Explainable artificial intelligence.  Science Robotics 4, 37 (2019).  DOI:http://dx.doi.org/10.1126/scirobotics.aay7120
  * [12] Jacob Haspiel, Na Du, Jill Meyerson, Lionel P. Robert Jr., Dawn Tilbury, X. Jessie Yang, and Anuj K. Pradhan. 2018.  Explanations and Expectations: Trust Building in Automated Vehicles. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’18). Association for Computing Machinery, New York, NY, USA, 119–120.  DOI:http://dx.doi.org/10.1145/3173386.3177057
  * [13] Monika Hengstler, Ellen Enkel, and Selina Duelli. 2016.  Applied artificial intelligence and trust—The case of autonomous vehicles and medical assistance devices.  Technological Forecasting and Social Change 105 (2016), 105–120.
  * [14] Matthew Hutson. 2017.  People don’t trust driverless cars. Researchers are trying to change that.  (2017).
  * [15] Sergey Ioffe and Christian Szegedy. 2015.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning \- Volume 37 (ICML’15). JMLR.org, 448–456.
  * [16] Stephen C Johnson. 1967.  Hierarchical clustering schemes.  Psychometrika 32, 3 (1967), 241–254.
  * [17] Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. 2018.  Textual Explanations for Self-Driving Vehicles. In The European Conference on Computer Vision (ECCV).
  * [18] Jeamin Koo, Jungsuk Kwac, Wendy Ju, Martin Steinert, Larry Leifer, and Clifford Nass. 2015.  Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance.  International Journal on Interactive Design and Manufacturing (IJIDeM) 9, 4 (01 Nov 2015), 269–275.  DOI:http://dx.doi.org/10.1007/s12008-014-0227-2
  * [19] Brian Mok, Mishel Johns, Key Jung Lee, David Miller, David Sirkin, Page Ive, and Wendy Ju. 2015.  Emergency, automation off: Unstructured transition timing for distracted drivers of automated vehicles. In 2015 IEEE 18th international conference on intelligent transportation systems. IEEE, 2458–2464.
  * [20] Luke Petersen, Lionel Robert, X. Jessie Yang, and Dawn M. Tilbury. 2019\.  Situational Awareness, Drivers Trust in Automated Driving Systems and Secondary Task Performance.  (2019).
  * [21] Kaspar Raats, Vaike Fors, and Sarah Pink. 2019.  Understanding Trust in Automated Vehicles. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction (OZCHI’19). Association for Computing Machinery, New York, NY, USA, 352–358.  DOI:http://dx.doi.org/10.1145/3369457.3369493
  * [22] Juan Ramos and others. 2003.  Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, Vol. 242. Piscataway, NJ, 133–142.
  * [23] T. Rezvani, K. Driggs-Campbell, and R. Bajcsy. 2017.  Optimizing interaction between humans and autonomy via information constraints on interface design. In IEEE International Conference on Intelligent Transportation Systems (ITSC).
  * [24] Tara Rezvani, Katherine Driggs-Campbell, Dorsa Sadigh, S Shankar Sastry, Sanjit A Seshia, and Ruzena Bajcsy. 2016.  Towards trustworthy automation: User interfaces that convey internal and external awareness. In IEEE International Conference on Intelligent Transportation Systems (ITSC). 682–688.
  * [25] P.A.M. Ruijten, J.M.B. Terken, and S.N. Chandramouli. 2018.  Enhancing trust in autonomous vehicles through intelligent user interfaces that mimic human behavior.  Multimodal Technologies and Interaction 2, 4 (12 2018).  DOI:http://dx.doi.org/10.3390/mti2040062
  * [26] Xingjian SHI, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-kin Wong, and Wang-chun WOO. 2015.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting.  In Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc., 802–810.
  * [27] Victor A Shia, Yiqi Gao, Ramanarayan Vasudevan, Katherine Driggs Campbell, Theresa Lin, Francesco Borrelli, and Ruzena Bajcsy. 2014.  Semiautonomous vehicular control using driver modeling.  IEEE Transactions on Intelligent Transportation Systems 15, 6 (2014), 2696–2709.
  * [28] Robert F. Tate. 1954.  Correlation Between a Discrete and a Continuous Variable. Point-Biserial Correlation.  The Annals of Mathematical Statistics 25, 3 (1954), 603–607.  http://www.jstor.org/stable/2236844
  * [29] Gesa Wiegand, Matthias Schmidmaier, Thomas Weber, Yuanting Liu, and Heinrich Hussmann. 2019.  I Drive - You Trust: Explaining Driving Behavior Of Autonomous Cars. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, Article Paper LBW0163, 6 pages.  DOI:http://dx.doi.org/10.1145/3290607.3312817
  * [30] Wikipedia contributors. 2020a.  Friedman test — Wikipedia, The Free Encyclopedia.  (2020).  https://en.wikipedia.org/w/index.php?title=Friedman_test&oldid=951227618 [Online; accessed 3-May-2020 ].
  * [31] Wikipedia contributors. 2020b.  Truncated mean — Wikipedia, The Free Encyclopedia.  (2020).  https://en.wikipedia.org/w/index.php?title=Truncated_mean&oldid=936108583 [Online; accessed 3-May-2020 ].
  * [32] Ye Xia, Jinkyu Kim, John F. Canny, Karl Zipser, and David Whitney. 2019a.  Periphery-Fovea Multi-Resolution Driving Model guided by Human Attention.  CoRR abs/1903.09950 (2019).  http://arxiv.org/abs/1903.09950
  * [33] Ye Xia, Danqing Zhang, Jinkyu Kim, Ken Nakayama, Karl Zipser, and David Whitney. 2019b.  Predicting Driver Attention in Critical Situations. In Computer Vision – ACCV 2018, C.V. Jawahar, Hongdong Li, Greg Mori, and Konrad Schindler (Eds.). Springer International Publishing, Cham, 658–674.
  * [34] Ekim Yurtsever, Yongkang Liu, Jacob Lambert, Chiyomi Miyajima, Eijiro Takeuchi, Kazuya Takeda, and John H. L. Hansen. 2019.  Risky Action Recognition in Lane Change Video Clips using Deep Spatiotemporal Networks with Segmentation Mask Transfer.  2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 2019, pp. 3100-3107, (2019).  DOI:http://dx.doi.org/10.1109/ITSC.2019.8917362