# Generating camera failures as a class of physics-based adversarial examples

Manav Prabhakar∗ Jwalandhar Girnar Arpan Kusari
University of Michigan Transportation Research Institute
University of Michigan
Ann Arbor, MI - 48103
{prmanav, jwala<EMAIL_ADDRESS>
Equal contributionCorresponding author

###### Abstract

While there has been extensive work on generating physics-based adversarial
samples recently, an overlooked class of such samples come from physical
failures in the camera. Camera failures can occur as a result of an external
physical process, i.e. breakdown of a component due to stress, or an internal
component failure. In this work, we develop a simulated physical process for
generating broken lens as a class of physics-based adversarial samples. We
create a stress-based physical simulation by generating particles constrained
in a mesh and apply stress at a random point and at a random angle. We perform
stress propagation through the mesh and the end result of the mesh is a
corresponding image which simulates the broken lens pattern. We also develop a
neural emulator which learns the non-linear mapping between the mesh as a
graph and the stress propagation using constrained propagation setup. We can
then statistically compare the difference between the generated adversarial
samples with real, simulated and emulated adversarial examples using the
detection failure rate of the different classes and in between the samples
using the Frechet Inception distance. Our goal through this work is to provide
a robust physics based process for generating adversarial samples.

##  1 Introduction

Cameras are ubiquitous as remote sensors, collecting data from an unstructured
and dynamic external environment, often in harsh conditions. As such, failures
can develop in a camera over time through normal wear and tear (known as
incipient failure) or they can fail suddenly (known as abrupt failure) (Li et
al.,, 2020). A failure or fault in a sensor is a divergence from the
functional state in at least one given parameter of the system (van Schrick,,
1997). A sensor fault can occur due to internal (such as wear and tear) and
external (temperature, humidity etc) causes. In RGB cameras, internal causes
include dead pixels while external causes include broken lens and
condensation. These abrupt failures are hard to detect and would have negative
effects on the object detection algorithms - making them hard to accurately
detect objects and leading to hallucination. The failures occurring in an
automated vehicle (AV) for example, can lead to safety failures resulting in
crashes and in some cases, fatalities. Currently, to the best of authors’
knowledge, there are no rigorous methods for generating camera based sensor
failures (Ceccarelli and Secci,, 2022).

In this paper, we will be focusing on the sensor failure occurring due to
broken lens, although the process detailed in this paper can be used for any
of the camera failures listed in (Ceccarelli and Secci,, 2022). Broken or
cracked lens in a camera can be caused due to an external object hitting the
lens or a result of heat and/or pressure developing suddenly within the camera
system. In the parlance of neural networks, the image with broken lens can be
considered as an adversarial sample (see Fig. 1). Researchers have shown that
small amounts of corruptions, sometimes difficult to be seen by human eyes,
are enough to completely fool the neural networks where a subtle change of
inputs can lead to drastic change in outputs, thus known as adversarial
samples (Akhtar and Mian,, 2018, Carlini and Wagner,, 2017, Szegedy et al.,,
2013). We would like to note that Li et al., (2019) provided a physical
camera-based adversarial attack paradigm, which serves as the closest related
work in this domain. They presented a modification of the image using a
overlay of a translucent, carefully crafted sticker which led to
misclassification.

Figure 1: Object detection error using YOLOv8 on KITTI image due to broken
lens - upper - image overlaid with broken lens pattern; lower - clean image

Thus, given all of this previous literature on adversarial sample generation,
how is this paper different? The difference stems from the physical process of
the broken lens - even though the principal point of the fracture in the lens,
the force and angle of incidence may be random, the spread of the fracture and
the resulting pattern follows an inherently physical process (being either
linear or radial). Generating this kind of adversarial sample can be done
using real world procedure, where we use calibrated force on lenses to
determine the failure. However, this procedure is expensive and time-
consuming. An alternate procedure (and one that is significantly less
expensive) would be to simulate fracture on a physics based simulation
platform using the material properties of glasses similar to that used in the
lens. In this work, we build a fracture simulation based on the randomly
generated particles in a triangular mesh and perform stress propagation
through the mesh using finite element modeling (FEM) methods.

The fracture simulation produces a triangular mesh at every discrete time
state $\delta t$, but we also want to generate a rendering of the glass
fracture. We utilize the OpenCV library to convert the given mesh to a
corresponding broken lens pattern image. We can then learn the combined non-
linear mapping of the fracture simulation and the rendering as part of the
sensor failure generation. Using a neural network, we can then learn the
physical process through which the images of thin glass fractures over
discrete time states can be generated given the physical parameters.

We can then generate the adversarial images by overlaying the images with the
generated broken lens pattern as a filter for two popular open source datasets
- KITTI(Geiger et al.,, 2013) and MS-COCO (Lin et al.,, 2014). A common
process for testing the generated adversarial images is to find the number of
images that are not detected using the adversarial filters. However, in our
case, due to the adversarial effect being local, we cannot rely simply on an
image based measure. We therefore, use the adversarial images (similar to the
upper figure of Fig. 1) and extract the objects which lie in the region where
the fracture exists using the ground truth bounding boxes. We then utilize
state-of-the-art object detection neural network algorithm YOLOv8 (Jocher et
al.,, 2023) to find the percentage of objects that fail when the adversarial
filters are applied. We have a choice in terms of simulated versus emulated
samples and we test the effect of realism of the samples in terms of failure
rate. In terms of ablation studies, we aim to understand the distributional
differences between real broken glass images collected from the internet and
the generated images. We perform dimensionality reduction on the images and
test them based on Frechet Inception distance (FID). We show that while the
distributions of simulated and emulated images are not same to the real broken
lens distributions, they can create the same adversarial problems for the
model.

The specific contributions of the paper are:

  * •

we focus on camera failures as a class of adversarial samples and provide a
structured way of generating them,

  * •

we devise a physically sound process for simulation of stress propagation
through glass and output an image of the final crack, and

  * •

we present a generative emulator for combined non-linear mapping of the
physical process that aims at spanning the subspace of broken glass images.

##  2 Background

###  2.1 Physics based adversarial samples

The problem of adversarial sample can be defined as follows: for a model $M$
that classifies an input sample $X$ correctly to its designated class i.e.
$M(X)=y_{true}$, adding an error $\epsilon$ to the input sample $X$, results
in an altered sample $\hat{X}$ such that $M(\hat{X})\neq y_{true}$. Thus, the
injection of the error $\epsilon$ results in an adversarial sample that causes
the model to fail.

Although the idea of adversarial manipulation of the model has been identified
in the context of machine learning quite some time ago (Dalvi et al.,, 2004),
in the last decade, the focus has squarely been on the adversarial attacks on
neural networks (Szegedy et al.,, 2013, Goodfellow et al.,, 2014). In these
papers, the researchers showed that a small targeted injection of noise,
almost imperceptible to the human eye, changed the labels completely (Szegedy
et al.,, 2013) and conversely, images could be generated that looked
completely unrecognizable to humans but which had perfect classifications from
the DNNs (Nguyen et al.,, 2015).

While these adversarial samples probe the model for possible failures, they
lack any physical realism behind their generation and need access to the
model. To address this, some recent research has targeted building physically
relevant adversarial samples. One of the first forays into this was made by
Kurakin et al., (2018) who targeted the accuracy of the models in the physical
world by feeding noisy images from a cell-phone camera that led the model to
incorrectly classify a large fraction of the samples. Along the similar vein,
Eykholt et al., (2018) demonstrated that real traffic signs can be perturbed
with simple physical stickers placed strategically to fool state-of-the-art DL
algorithms almost perfectly even with viewpoint changes. Other researchers
have placed adversarial images (Kong et al.,, 2020) or LiDAR surfaces (Tu et
al.,, 2020) to generate samples which fool object detectors. While these prior
research use physics in terms of generating the samples, they do not come from
modeling a rigorous physical process and we aim to fill this gap in this work.

###  2.2 Neural emulation of physical simulations

Any physics based process can be succinctly expressed as a function $\Phi$
which given a state $s_{t}$ at time $t$, control input $u_{t}$ and external
force $f_{t}$ provides the output state at time $t+\delta t$ as

$s_{t+\delta t}=\Phi[s_{t},u_{t},f_{t}]$ (1)

Emulations of physical simulations have been undertaken in the statistical
community since Sacks et al., (1989) published their seminal work on utilizing
statistical models to provide efficient predictions of computationally
expensive computer models. They modeled the deterministic output from the
physical simulation as a stochastic process, which provided a measure of
uncertainty of predictions. Thus, the statistical models they devise learn an
approximation to the dynamic model by observing instances of the state
transitions. Since then, a variety of machine learning algorithms have been
utilized to perform the emulation process, chief among which is the Gaussian
process (Sung and Tuo,, 2024). Gaussian process has been shown to have
drawbacks in high-dimensional non-linear settings due to the curse of
dimensionality (Binois and Wycoff,, 2022).

On the other hand, the artificial intelligence community has been building
neural network models to emulate high-dimensional, non-linear physical
processes which can provide high accuracy but lack in uncertainty
quantification (Grzeszczuk et al.,, 1998). While they started with simple
3-layer feedforward network, in the recent years, with the explosion of deep
learning, there has been approaches using deep neural networks to discover
constraints between particles or bodies (Battaglia et al.,, 2016, Han et al.,,
2022). Particularly, deep graph neural networks have been used successfully to
provide physical constraints between particles/bodies (Mrowca et al.,, 2018,
Xu et al.,, 2019). A related field is the inferring of physics simulations
from image frames where the researchers learn the model dynamics from
following the object motion in the image frames (Watters et al.,, 2017).
Since, we have access to the physical simulation process, in our proposed
work, our goal is to emulate the physical process and then render the image
corresponding to the output of the process.

##  3 Methodology

Generating realistic lens failures require creating large-scale physics based
simulations by solving fracture dynamics on a triangulated finite element mesh
with glass properties. This triangulated mesh can then be converted into a
corresponding image. The image can then be placed as an alpha layer into a
captured "clean" image. The entire pipeline for creating a physical simulation
and developing an emulation pipeline is described below and in Fig. 2 and 3.

Figure 2: We provide the simulation pipeline for our proposed work - (a)
provides a diagram of how the stress propagates in the simulation - for a
splitting plane given in blue, the summed positive stress $q1$ and summed
negative stress $q2$ are compared and the propagation happens on the side with
greater summed stress and chosen node which is the closest to the splitting
plane (given in red); (b) shows the triangulated mesh with the fracture; (c)
shows the corresponding image

###  3.1 Broken lens simulation

The subject of how glass breaks and how it propagates is still an open
research question and one that has been contentious with multiple physical
theories being proposed (Rouxel and Brow,, 2012). While the microscopic
procedure of glass fracture is being debated on, on a macroscopic level, the
fracture dynamics is well understood. Liu et al., (2021) analyzed the process
of fracture of glass lens in the precision glass molding application using FEM
with a three-dimensional model in a physical simulation software. The physical
parameters were input into the software and the crack paths were analyzed
using the simulation results. The authors performed a temperature and stress
simulation of a high-precision three-dimensional mesh model of the molded
glass. Another work which is closely related is the simulation of glass
breaking as a thin sheet (Pfaff et al.,, 2014). Although, the examples are
striking and realistic, the accompanying code for glass did not work as
intended. However, these previous works serve as the inspiration.

Glass can be represented using a randomly sampled particles constrained in a
mesh or graph. The number and position of the particles within the mesh can be
randomly assigned but the mesh is formed using a constrained Delaunay
triangulation, which removes ill-shaped triangles and avoids uneven and
unrealistic edges as given in Fig. 2(a). Each particle $p_{i}$ has a position
$x_{i}$ and has nearest neighbors $\mathbf{k_{i}}$ within a radius $r$ which
have existing edges with $p_{i}$. Mathematically, the triangulation mesh
$\mathcal{M}$ represents a finite set of 2-simplices such that if

$\forall(K,K^{\prime})\in\mathcal{M}\times\mathcal{M},|K|\cap|K^{\prime}|=|K\cap
K^{\prime}|.$ (2)

The crack patterns in glass occur due to stress from the external force (F) at
the initial impact point $p_{I}$ by assuming a specific deformation law
(elasticity, plasticity) of the glass (G) (Kuna,, 2013). We then compute the
strength parameters in the form of effective stress $\sigma_{V}$ at the impact
point (V) as the stress state of the impact point. The critical stress values
for the strength of glass $\sigma_{C}$ is found using tests on simple samples
with elementary loading conditions (e.g. tension test). The fracture then
occurs when the effective stress is larger than the critical stress divided by
the safety fractor (S):

$\sigma_{V}(G,F)>\dfrac{\sigma_{C}}{S}.$ (3)

From the classical theory of strength of materials, we know that the failure
in most cases is controlled by the principal stresses $\sigma_{I}$ and
$\sigma_{II}$ for 2D elements. The initial crack happens either by the normal-
planar crack where the fracture faces are located perpendicular to the
direction of the highest principal stress $\sigma_{I}$ (Rankine,, 1857) or
shear-planar crack where the fracture faces coincide with the intersection
planes of the maximum shear stress $\tau_{max}=(\sigma_{I}-\sigma_{II})/2$
(Coulumb,, 1776). In the case of glass, we assume that the initial fracture
happens perpendicular to the direction of the maximum principal stress.

From the initial impact point $p_{I}$, the stress propagation through glass is
unstable since the crack grows abruptly without the need to increase external
loading. From $p_{I}$, stress propagates in the vertex neighborhood
$\mathbf{k_{i}}$ as the stress along the direction
$\overrightarrow{p_{I}p_{j}}$ where $p_{j}\in\mathbf{k_{i}}$ as

$\sigma_{p_{j}}=\sigma_{V}*\dfrac{\overrightarrow{p_{I}p_{j}}\cdot\overrightarrow{n}}{|\overrightarrow{p_{I}p_{j}}||\overrightarrow{n}|}.$
(4)

With the stress calculated for each edge, the summed positive stress can then
be given as:

$q1=\int_{\partial\Omega}\sigma_{p_{j}}\mathbbm{I}(\sigma_{p_{j}}>0)dA$ (5)

for continuous surface $\Omega$ where $\mathbbm{I}$ is the indicator function.
The summed positive stress for discrete simplices in the corresponding area A
of radius R is given as

$q1=\sum_{K\in{A_{R}}}\sigma_{K}\mathbbm{I}(\sigma_{K}>0).$ (6)

Similarly, the summed negative stress $q2$ is calculated. Then for greater
magnitude $max(|q1|,|q2|)$, we choose the corresponding edge with the highest
concentration of stress in the given segment as the optimal splitting plane
since that provides the maximum stress relief. Thus, the stress travel along
the mesh edges, dissipating the stress at each node point.

The recursive application of the stress propagation is run until convergence
of the stress in all states i.e.
$\sigma_{p}^{(t)}\simeq\sigma_{p}^{(t-1)}\forall p\in V$. The final output of
the simulation is the realization of the graphs as an image which corresponds
to broken lens pattern. The pseudo-code for the stress propagation algorithm
is given in Algorithm 1.

Even though we provide a relatively low-fidelity simulation, we intended to
make it’s combined non-linear mappings as realistic as possible. For this we
render the crack width and reflection/color based on the stress values. We
define the width $(W)$ of a crack using a non-linear transformation for a
gradual decay with distance from the impact point and node of maximum stress.

$W=1+\sqrt{\frac{\sigma}{\sigma_{max}}}\times(W_{max}-W_{min})$ (7)

where $W_{max}$ and $W_{min}$ are chosen discreetly to induce realism.

Algorithm 1 Stress Propagation

1:$pt\leftarrow\text{Impact Point}$

2:$F\leftarrow\text{Impact Force}$

3:$PE\leftarrow\text{Parent Edge}$

4:$v\leftarrow\text{Impact Vector}$

5:$R\leftarrow\text{Nearest neighbor radius}$

6:

7:procedure PropagateStress($Pt,F,V,PE$)

8: $frontiers\leftarrow KDTree-queryRadius(R)$

9: $NN\leftarrow\frac{frontiers-pt}{||frontiers-pt||}$

10: $cos(\theta)\leftarrow NN\cdot v$

11: $stress\leftarrow calculateStress(cos(\theta),F)$

12: $frontiers\leftarrow frontiers[argmax(stress)]$

13: $v\leftarrow v[argmax[stress]]$

14: $PE\leftarrow PE[argmax[stress]]$

15: PropagateStress($Pt,F,V,PE$)

16:end procedure

Figure 3: Our neural emulator. We use a ResNet34(He et al., 2016a, )
architecture to induce physical constraints in our generative branch. The
ResNet block is pretrained using the simulated images to learn $f_{\phi}$ (Eq.
9). This learnt representation is used to backpropagate the weights of the
emulator to generate realistic-physics - guided images

###  3.2 Neural emulation of physical simulations

Given a simulation, our motive is to emulate the broken glass structure
through a physics-informed pipeline. We approach this along the lines of Jiang
and Willett, (2022) where their objective is to learn an estimation of the
parameter space given the given high dimensional simulation output. Since our
parameter space is a uniform distribution, we define it as
$M({\phi_{a},\phi_{b}})$. We now sample our simulator input from this
distribution. Due to its recursive nature, the simulator allows us to create a
state-action space by parameterizing the simulation output at any time $t$
using the current state $Z_{t}$ and the corresponding physical parameters
$\phi_{t}$. Then the simulator maps a non-linear function $H$ while adding
uniform random noise $\eta$ to give us the broken glass image ($Z$). For n
samples ($i=1,2,3...,n$), and $t$ timesteps, we then have

$Z_{i,t}=H(\phi_{i,t-1})+\eta_{i}$ (8)

For the emulation, we now learn an inverse function $f_{\phi}$ that maps
$Z_{i,t}$ to $\phi_{i,t-1}$. We use ResNet34 (He et al., 2016b, ) as the
encoder followed by an MLP head and train it in a supervised fashion to
predict $\phi_{i}$. This encoder serves as a physics guide to our generative
emulator (see Fig. 3) Given a generator $G$ and a discriminator $D$, we modify
the binary cross-entropy generative loss by adding an additional term for
calculating Z’s deviation from M.

$L_{G}=-E_{x~{}p(x)}[logD(G(x))]+||f_{\phi}(D(G(x)))-(\phi_{a},\phi_{b})||$
(9)

while the discriminator loss is forced to classify any $f_{\phi}$ $\notin$
$M(\phi_{a},\phi_{b})$ as a fake image

$L_{D}=-E_{Z~{}p(Z)}[logD(Z)]-E_{x~{}p(x)}[log(1-D(G(x)))]$ (10)

where x is some random noise and Z denotes our real/simulated broken glass
image. We bind M using physical characteristics like glass toughness, initial
force, angle and point of impact.

##  4 Results

###  4.1 Benchmarking results

We benchmark three types of broken lens pattern - real, simulated and emulated
- on two popular open-source datasets - KITTI(Geiger et al.,, 2013) and MS-
COCO (Lin et al.,, 2014). The real broken lens pattern images are collected
from FreePik website111https://www.freepik.com and represent the baseline in
our case. We collected $65$ images in total which we expanded to a set of
$10,000$ via image augmentation such as random shifts and image flips. Then,
we run each image through a random adversarial image and collect the aggregate
results. We use a pretrained YOLOv8 model to generate these results. A major
shift from most of the previous adversarial examples papers is that our
generated adversarial patterns do not affect all pixels in an image
universally and therefore, the comparison needs to be done only for the image
region where the pattern exists. Therefore, we create a binary mask of each
pattern and output the results of the objects which exist in that pattern
only.

Table 1: Average precision (in percentage) of different classes in KITTI and MS-COCO under different adversarial images Dataset | IoU threshold | Class or category | Original | Real | Simulated | Emulated
---|---|---|---|---|---|---
KITTI | 0.25 | Pedestrian | 32.67 | 81.39 | 22.07 | 87.88
Truck | 12.39 | 3.85 | 3.84 | 33.33
Car | 69.6 | 61.5 | 67.2 | 54.55
0.5 | Pedestrian | 25.64 | 69.72 | 17.84 | 83.07
Truck | 12.39 | 3.59 | 3.76 | 33.33
Car | 58.99 | 50.7 | 57.73 | 44.53
0.75 | Pedestrian | 6.83 | 33.88 | 6.02 | 37.27
Truck | 11.29 | 2.67 | 2.79 | 33.33
Car | 31.25 | 23.85 | 30.15 | 18.66
MS-COCO | 0.25 | Person | 1.15 | 0.93 | 1.13 | 1.10
Vehicles | 3.45 | 3.22 | 5.16 | 2.43
Food | 35.05 | 27.98 | 34.11 | 9.56
0.5 | Person | 0.03 | 0.02 | 0.02 | 0.02
Vehicles | 1.83 | 1.23 | 1.54 | 0.00
Food | 29.11 | 24.31 | 26.05 | 4.69
0.75 | Person | 0.03 | 0.02 | 0.02 | 0.00
Vehicles | 1.45 | 0.92 | 0.93 | 0.00
Food | 21.66 | 16.8 | 20.95 | 0.00

Table 1 shows the results of the average precision (AP) under the real,
simulated and emulated adversarial images for different classes. For KITTI,
the average precision of other classes drop as expected with the decrease in
AP corresponding to the percentage of image occupied, the truck class
recording the highest drop. There is a significant drop for real adversarial
images and emulated images, with some drop in the simulated images. A very
intriguing and counter-intuitive result is that the pedestrian class has a
multiple-fold increase in average precision under real and emulated broken
lens patterns.

Figure 4: FID between real and generated image distributions

For MS-COCO, most of the objects occupy smaller image space and its difficult
to get meaningful results. Therefore, we have aggregate the AP for the super-
categories - person, vehicles and food categories. The emulated images have
the highest drop in AP for all the classes with most super-categories not
being detected at all, which implies that the emulated image is able to create
a much more challenging problem for the model.

###  4.2 Ablation studies

Given the results above, we would like to understand how close the generated
images are to the real broken lens images in the distributional sense. We
start by comparing the FID between the given image distributions. FID or
2-Wasserstein distance computes the differences between two image
distributions by inputting the distributions through Inception V3 model
trained on ImageNet and then uses the two Gaussian distributions generated by
a layer to calculate the distance as:

$d_{F}(\mathcal{N}(\mu,\Sigma),\mathcal{N}(\mu^{\prime},\Sigma^{\prime}))=||\mu-\mu^{\prime}||^{2}_{2}+tr(\Sigma+\Sigma^{\prime}-2(\Sigma\Sigma^{\prime})^{0.5})$
(11)

The difference can then be given in the form of a confusion matrix in Fig. 4
where the FID between the same distributions are 0.0 and it increases as the
distributions become dissimilar. We see from the confusion matrix, that the
simulated images are close in distribution to the real images than the
emulated images which makes sense.

##  5 Discussions

###  5.1 Implementation of simulation and emulation models

Our simulation model is developed by randomly sampling $10^{5}$ particles from
a uniform spatial distibution in the given frame in a CPU. A KD-tree from the
SciPy python package (Virtanen et al.,, 2020) using default parameters is
constructed to find the approximate nearest neighbors of each particle. A
constrained Delaunay triangulation with an angle threshold of 20°is then run
on the particles to create a constrained triangular mesh. We use an impact
force of 1000 units with a random impact point and a random impact vector. The
stress propagation happens until a threshold of 600 units is reached. The
image generation is performed using OpenCV and Python.

We finetune a pretrained ResNet34 feature extractor with an MLP head on 10,000
samples of simulated broken glass images for 30 epochs using the Adam
optimizer (Kingma and Ba,, 2017) and learning rate of $10^{-3}$. We then use
this pre-trained model for computing our GAN losses as in Eq. 9 and 10. We
train our GAN for 50 epochs, on the same simulated dataset. We use the Adam
optimizer for both the generator and discriminator with a learning rate of
$2\times 10^{-4}$.

###  5.2 Limitations

Since the camera failures are governed by physical processes, generation of
each kind of camera failure through simulation and then using emulation to
learn the state transitions is a computationally expensive process. For some
failures such as condensation, the physical process is still not explained to
the point where we can build realistic simulations. Therefore, in this paper,
we restrict ourselves to broken lens as a particular example of camera
failure.

Our physical simulation uses some simplifying assumptions -

  * •

the glass cracks using the maximum principal stress model initially;

  * •

crack propagation happens along the direction of maximum stress relief only;

  * •

multiple paths of stress propagation do not occur;

  * •

no other forces (such as manufacturing defects) are present.

In the generation of adversarial images of broken lens, we have used the
generated image without any consideration for visual changes that occur in the
fracture due to refraction or glare. Computer graphics software such as
Blender or Autodesk Maya can output the visual effects of fracture (without
any physical process constraints) which can be incorporated to provide more
realistic rendering of broken lens.

The emulated images are not high resolution due to the structure of the GAN
network. While this itself is not a hindrance towards creating adversarial
images, we would like to have high resolution images in the future.

##  6 Conclusions

We have introduced a novel class of adversarial failures resulting from the
physical process of failures in the camera. In this paper, we provide an
approach to generate the broken lens pattern from a physical simulation and
subsequently, train a neural network to emulate the physical constraints. We
show that the simulated and emulated adversarial images can lead to
significant errors in object detection even when the distributions don’t span
the same space. Our future work will be in performing realistic rendering of
the simulated and emulated image. Our aim is to provide a alternative and
important means of generating realistic adversarial examples.

## References

  * Akhtar and Mian, (2018) Akhtar, N. and Mian, A. (2018).  Threat of adversarial attacks on deep learning in computer vision: A survey.  Ieee Access, 6:14410–14430.
  * Battaglia et al., (2016) Battaglia, P., Pascanu, R., Lai, M., Jimenez Rezende, D., et al. (2016).  Interaction networks for learning about objects, relations and physics.  Advances in neural information processing systems, 29.
  * Binois and Wycoff, (2022) Binois, M. and Wycoff, N. (2022).  A survey on high-dimensional gaussian process modeling with application to bayesian optimization.  ACM Transactions on Evolutionary Learning and Optimization, 2(2):1–26.
  * Carlini and Wagner, (2017) Carlini, N. and Wagner, D. (2017).  Adversarial examples are not easily detected: Bypassing ten detection methods.  In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 3–14.
  * Ceccarelli and Secci, (2022) Ceccarelli, A. and Secci, F. (2022).  Rgb cameras failures and their effects in autonomous driving applications.  IEEE Transactions on Dependable and Secure Computing.
  * Coulumb, (1776) Coulumb, C.-A. (1776).  Essai sur une application des regles des maximis et minimis a quelquels problemesde statique relatifs, a la architecture.  Mem. Acad. Roy. Div. Sav., 7:343–387.
  * Dalvi et al., (2004) Dalvi, N., Domingos, P., Mausam, Sanghai, S., and Verma, D. (2004).  Adversarial classification.  In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 99–108.
  * Eykholt et al., (2018) Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., and Song, D. (2018).  Robust physical-world attacks on deep learning visual classification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1625–1634.
  * Geiger et al., (2013) Geiger, A., Lenz, P., Stiller, C., and Urtasun, R. (2013).  Vision meets robotics: The kitti dataset.  International Journal of Robotics Research (IJRR).
  * Goodfellow et al., (2014) Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014).  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572.
  * Grzeszczuk et al., (1998) Grzeszczuk, R., Terzopoulos, D., and Hinton, G. (1998).  Neuroanimator: Fast neural network emulation and control of physics-based model.  In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pages 9–20.
  * Han et al., (2022) Han, J., Huang, W., Ma, H., Li, J., Tenenbaum, J., and Gan, C. (2022).  Learning physical dynamics with subequivariant graph neural networks.  Advances in Neural Information Processing Systems, 35:26256–26268.
  * (13) He, K., Zhang, X., Ren, S., and Sun, J. (2016a).  Deep Residual Learning for Image Recognition.  In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’16, pages 770–778. IEEE.
  * (14) He, K., Zhang, X., Ren, S., and Sun, J. (2016b).  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778.
  * Jiang and Willett, (2022) Jiang, R. and Willett, R. (2022).  Embed and emulate: Learning to estimate parameters of dynamical systems with uncertainty quantification.  In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A., editors, Advances in Neural Information Processing Systems, volume 35, pages 11918–11933. Curran Associates, Inc.
  * Jocher et al., (2023) Jocher, G., Chaurasia, A., and Qiu, J. (2023).  Ultralytics YOLO.
  * Kingma and Ba, (2017) Kingma, D. P. and Ba, J. (2017).  Adam: A method for stochastic optimization.
  * Kong et al., (2020) Kong, Z., Guo, J., Li, A., and Liu, C. (2020).  Physgan: Generating physical-world-resilient adversarial examples for autonomous driving.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14254–14263.
  * Kuna, (2013) Kuna, M. (2013).  Finite elements in fracture mechanics.  Solid mechanics and its applications, 201:153–192.
  * Kurakin et al., (2018) Kurakin, A., Goodfellow, I. J., and Bengio, S. (2018).  Adversarial examples in the physical world.  In Artificial intelligence safety and security, pages 99–112. Chapman and Hall/CRC.
  * Li et al., (2020) Li, D., Wang, Y., Wang, J., Wang, C., and Duan, Y. (2020).  Recent advances in sensor fault diagnosis: A review.  Sensors and Actuators A: Physical, 309:111990.
  * Li et al., (2019) Li, J., Schmidt, F., and Kolter, Z. (2019).  Adversarial camera stickers: A physical camera-based attack on deep learning systems.  In International conference on machine learning, pages 3896–3904. PMLR.
  * Lin et al., (2014) Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. (2014).  Microsoft coco: Common objects in context.  In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer.
  * Liu et al., (2021) Liu, Y., Xing, Y., Li, C., Yang, C., and Xue, C. (2021).  Analysis of lens fracture in precision glass molding with the finite element method.  Applied Optics, 60(26):8022–8030.
  * Mrowca et al., (2018) Mrowca, D., Zhuang, C., Wang, E., Haber, N., Fei-Fei, L. F., Tenenbaum, J., and Yamins, D. L. (2018).  Flexible neural representation for physics prediction.  Advances in neural information processing systems, 31.
  * Nguyen et al., (2015) Nguyen, A., Yosinski, J., and Clune, J. (2015).  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427–436.
  * Pfaff et al., (2014) Pfaff, T., Narain, R., De Joya, J. M., and O’Brien, J. F. (2014).  Adaptive tearing and cracking of thin sheets.  ACM Transactions on Graphics (TOG), 33(4):1–9.
  * Rankine, (1857) Rankine, W. J. M. (1857).  Ii. on the stability of loose earth.  Philosophical transactions of the Royal Society of London, (147):9–27.
  * Rouxel and Brow, (2012) Rouxel, T. and Brow, R. K. (2012).  The flow and fracture of advanced glasses—an overview.  International Journal of Applied Glass Science, 3(1):1–2.
  * Sacks et al., (1989) Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989).  Design and analysis of computer experiments.  Statistical science, 4(4):409–435.
  * Sung and Tuo, (2024) Sung, C.-L. and Tuo, R. (2024).  A review on computer model calibration.  Wiley Interdisciplinary Reviews: Computational Statistics, 16(1):e1645.
  * Szegedy et al., (2013) Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2013).  Intriguing properties of neural networks.  arXiv preprint arXiv:1312.6199.
  * Tu et al., (2020) Tu, J., Ren, M., Manivasagam, S., Liang, M., Yang, B., Du, R., Cheng, F., and Urtasun, R. (2020).  Physically realizable adversarial examples for lidar object detection.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13716–13725.
  * van Schrick, (1997) van Schrick, D. (1997).  Remarks on terminology in the field of supervision, fault detection and diagnosis.  IFAC Proceedings Volumes, 30(18):959–964.
  * Virtanen et al., (2020) Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J., Polat, İ., Feng, Y., Moore, E. W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E. A., Harris, C. R., Archibald, A. M., Ribeiro, A. H., Pedregosa, F., van Mulbregt, P., and SciPy 1.0 Contributors (2020).  SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.  Nature Methods, 17:261–272.
  * Watters et al., (2017) Watters, N., Zoran, D., Weber, T., Battaglia, P., Pascanu, R., and Tacchetti, A. (2017).  Visual interaction networks: Learning a physics simulator from video.  Advances in neural information processing systems, 30.
  * Xu et al., (2019) Xu, Z., Wu, J., Zeng, A., Tenenbaum, J. B., and Song, S. (2019).  Densephysnet: Learning dense physical object representations via multi-step dynamic interactions.  arXiv preprint arXiv:1906.03853.


# Focused Adaptation of Dynamics Models for Deformable Object Manipulation

Peter Mitrano1, Alex LaGrassa2, Oliver Kroemer2, and Dmitry Berenson1
1Department of Robotics, University of Michigan2Robotics Institute, Carnegie
Mellon UniversityThis work was supported in part by the Office of Naval
Research Grants N00014-21-1-2118 and N00014-18-1-2775, NSF grants IIS-1956163,
CMMI-1925130, IIS-1750489 and IIS-2113401, and ARL grant W911NF-18-2-0218.

###### Abstract

In order to efficiently learn a dynamics model for a task in a new
environment, one can adapt a model learned in a similar source environment.
However, existing adaptation methods can fail when the target dataset contains
transitions where the dynamics are very different from the source environment.
For example, the source environment dynamics could be of a rope manipulated in
free space, whereas the target dynamics could involve collisions and
deformation on obstacles. Our key insight is to improve data efficiency by
focusing model adaptation on only the regions where the source and target
dynamics are similar. In the rope example, adapting the free-space dynamics
requires significantly less data than adapting the free-space dynamics while
also learning collision dynamics. We propose a new method for adaptation that
is effective in adapting to regions of similar dynamics. Additionally, we
combine this adaptation method with prior work on planning with unreliable
dynamics to make a method for data-efficient online adaptation, called FOCUS.
We first demonstrate that the proposed adaptation method achieves
statistically significantly lower prediction error in regions of similar
dynamics on simulated rope manipulation and plant watering tasks. We then show
on a bimanual rope manipulation task that FOCUS achieves data-efficient online
learning, in simulation and in the real world.

##  I INTRODUCTION

Autonomous systems often rely on a dynamics model, which predicts future
states given actions, to reach a desired goal state. However, general and
accurate dynamics models only exist for a narrow range of robotics problems.
Learning these dynamics models is an increasingly popular paradigm, in part
because learned models can be repeatedly improved using autonomously collected
real-world data. However, fine-tuning an initial dynamics model on new data
can perform poorly when the data contains complex dynamics on which the
dynamics model was not initially trained. For example, suppose we want to
manipulate a rope amongst clutter, and we have a dynamics model trained on
free-space motions in simulation. Free-space transitions in the real world are
fairly similar to free-space transitions in simulation, but transitions where
the rope deforms on objects in the scene are very different from anything seen
in simulation. We call these transitions _distracting_ , because they are hard
to learn from a few examples, and because they make it harder to adapt
accurately to the free-space dynamics. More generally, transitions from
regions of dissimilar dynamics can inhibit effective transfer to regions of
similar dynamics. This problem is similar to “cleaning” data in machine
learning [1, 2, 3]. For dynamics learning, defining what “clean” means can be
difficult, and has not been studied extensively. Instead, the dominant
paradigm is simply to train on all the collected data.

However, training on all the data can fail because real-world datasets for
learning dynamics are often too small to learn generalized models over the
entire state-action space. In our experiments, we show that simply fine-tuning
on all the data can yield a model that is not accurate enough for planning. If
the task can be completed while remaining in regions where dynamics are
similar, then it can be worth trading accuracy in dissimilar regions for
accuracy in similar regions. Our key insight is that, when we are adapting
from an initial model, we can leverage the initial model to achieve
significantly lower prediction error by focusing on transitions where the
source and target dynamics are the most similar. The idea that transfer is
easier when the source and target data are similar is well-supported in the
transfer learning literature [4, 5].

Figure 1:  (A) An illustration of how our adaptation method focuses on regions
where the source and target dynamics are similar. When focusing adaption on
free-space dynamics, the prediction errors decrease for other free-space data
(similar), and do not decrease for collision dynamics (dissimilar). (B) A
mock-up of a car engine bay. The robot must move the rope and place it under
the engine without snagging it to set up for lifting the engine. We use our
proposed adaptation method to improve the success rate during online learning
for this task.

To implement this strategy, we propose an adaptation method that minimizes
prediction error in regions where the source and target dynamics are similar.
The proposed method minimizes prediction error in these regions by fine-tuning
on an initially small set of data from these regions, and growing that subset
over the course of training. This is done with a loss function inspired by
curriculum learning that weights transitions according to their prediction
error, assigning a higher weight to low-error transitions. Under the
assumption that there are paths to the goal where the source and target
dynamics are similar, this adaptation method can be used to achieve high task
success in the target environment.

The first contribution of this paper is a method for adapting dynamics models
to datasets that contain distracting transitions. We demonstrate that the
proposed method is successful in filtering out distracting data and that the
resulting model is more accurate in the regions of state-action space where
the source and target dynamics are similar. The second contribution is a data-
efficient online-learning method that pairs our adaptation method with prior
work on planning with unreliable dynamics models [6, 7]. We call our combined
method for online learning FOCUS. FOCUS achieves higher success rates in the
low-data regime because the adapted dynamics are more accurate, which leads to
finding more reliable plans.

##  II RELATED WORK

Adapting dynamics models:  One approach to adaptation is to use system
identification to fit a global dynamics model of the target domain [8, 9, 10,
11]. Alternatively, domain randomization uses random variations of conditions
during training to make the model robust to that type of variance during test
time [12]. Some methods go further, and iteratively refine the noise
distribution by comparing simulations to rollouts executed in the real world
[13, 14]. These methods require knowledge of how parameters can vary between
the source and the target domains. By contrast, our approach needs no
knowledge of how the system parameters such as dynamical parameters, object
geometry, and kinematics may vary. Prior work has also used estimates of
similarity between the source and target system to guide adaptation, for
instance by selecting training data most similar to the source system [4] or
selecting the most useful source domain for a given target domain [15].
Similarly, our approach focuses both training and data collection on
transitions with low prediction error, which are easier to learn. Previous
work avoids negative transfer of dynamics from the source domain that is
harmful to performance in the target domain [4, 16]. In contrast, our work
focuses on avoiding distracting dynamics in the target domain that inhibit
effective transfer to dynamics where the source and target are similar.

Data Cleaning:  Data Cleaning is a term for methods that aim to remove
training examples that are harmful, incorrect, or unhelpful [17]. Methods such
as Majority Vote Filtering or the Iterative-Partitioning Filter can remove
mislabeled classification data, but they do not apply to regression problems
and assume that the type of data we want to fit is the majority type [18, 1,
2]. In contrast, learning dynamics is a regression problem, and in our
experiments, the data we want to adapt to may be a minority.

Curriculum learning:  Transfer learning methods work best between similar
source and target domains [4, 16]. To tackle larger differences, curriculum
learning methods use intermediate problems through mechanisms such as pseudo-
labels or task selection, which has been successful in classification,
reinforcement learning, and machine translation [19, 20, 21, 22]. Like
curriculum learning, our proposed method tries to train on easier data first,
and keep the difference between the old and new data small [23]. However,
unlike in standard curriculum learning, we do not necessarily converge to
training on all data.

Planning with unreliable dynamics models:  If a dynamics model has a known
probabilistic transition model, belief-space planning can be used [24, 25].
Since meaningful predictive uncertainty distributions are difficult to
estimate for novel scenarios, some methods instead directly predict model
error and use it as a constraint for planning [7, 6, 26]. Rather than avoiding
uncertainty, model-based reinforcement learning methods try to reduce
uncertainty by collecting new data and fine-tuning the dynamics [27, 11]. In
this work, we both adapt the dynamics online and avoid inaccurate predictions,
which we find results in higher task success rates with less data.

Rope Manipulation and Plant Watering:  Prior model-based rope manipulation and
plant watering methods optimize parameters of a simulator or neural network to
match real-world dynamics, as we do here. [14] also uses a mechanism to avoid
including data that is high error for a particular local model. Typically, the
goal is to learn a model that is accurate for a wide range of dynamics for a
task, without prioritizing dynamics that may require less data to learn [28,
29, 30, 31, 32, 33, 34, 35]. Since linear deformable objects are so high-
dimensional, more work explores methods to learn or adapt models for a subset
of configurations [26, 6, 36].

##  III METHODS

First, we formalize the dynamics adaptation problem studied in this paper.
Then, we describe our method for adapting the dynamics. Finally, we describe
how our adaptation method can be used in an online adaptation framework to
efficiently learn a rope manipulation task.

###  III-A Problem Statement

The problem addressed in this paper is to adapt a dynamics model trained in a
source environment to data collected in a target environment, where the source
and target environments have dynamics that are similar in some regions of the
state-action space, but different in others. Furthermore, we consider the case
where data collection is done by planning and executing paths to goals in the
target environment using the learned dynamics model.

To formalize this, first consider the standard dynamics learning problem with
a dataset $\mathcal{D}$ of transitions of states, actions, and next states
$(s,a,s^{\prime})$. We also assume a distance function $d(s_{1},s_{2})$ that
returns a scalar is given, and that the state is fully observable. The true
dynamics are $s^{\prime}=f(s,a)$, and the learned dynamics are
$\hat{s}^{\prime}=\hat{f}(s,a)$. The initial dynamics $\hat{f}_{0}$ is the
model that is pre-trained in the source environment and not yet adapted to the
target environment. We define the source and target environment dynamics as
similar for a transition if $d\big{(}f_{S}(s,a),f_{T}(s,a)\big{)}<\gamma$. The
threshold $\gamma$ should be small enough that it excludes distracting
transitions, but large enough to include as much data from the target
environment as possible. Let $\mathcal{D}_{ST}$ be the set of transitions from
the target environment where this similarity condition holds.

In order to minimize the amount of data needed for adaptation to generalize,
we aim to adapt the dynamics only to transitions from regions where this
similarity condition holds ($\mathcal{D}_{ST}$). However, we also care about
successfully completing the task, and therefore we also assume there are paths
$\\{s^{0},a^{0},\dots,a^{T-1},s^{T}\\}$ to the goal $s^{T}\in\mathcal{G}$
within the regions of similar dynamics
$(s^{t},a^{t},s^{t+1})\in\mathcal{D}_{ST}$. This also means starting states
are assumed to be in $\mathcal{D}_{ST}$. If this is not the case, a different
source dynamics model may be needed.

While the ultimate objective of adaptation should be to maximize task success
in the target environment, an important condition for task success is
minimizing prediction error on $\mathcal{D}_{ST}$. If the goal is reachable
within $\mathcal{D}_{ST}$ (as we assume), the prediction error on
$\mathcal{D}_{ST}$ is small (our objective), and our motion planner is
constrained to stay in $\mathcal{D}_{ST}$, then we can also expect high task
success. Next, we discuss how our adaptation method minimizes error on
$\mathcal{D}_{ST}$, followed by how we can achieve high task success by
additionally constraining a motion planner to $\mathcal{D}_{ST}$.

Figure 2: Block diagram showing the steps of our full online adaptation
method. A dynamics model is initialized offline in the source environment
(left), then adapted online in the target environment.

###  III-B Adapting the Dynamics

Our objective is to minimize prediction error on $\mathcal{D}_{ST}$. At a high
level, our method dynamically weights the training data $\mathcal{D}$ such
that transitions that are likely to be in $\mathcal{D}_{ST}$ are given weights
near 1, and transitions unlikely to be in $\mathcal{D}_{ST}$ are given weights
near 0. Given a transition from our training data
$(s,a,s^{\prime})\in\mathcal{D}$, we cannot directly evaluate whether that
transition is in $\mathcal{D}_{ST}$, since that would require knowing the true
dynamics $f_{T}$. However, since the initial dynamics $\hat{f}_{0}$ is trained
to have low prediction error in the source environment, transitions from the
target environment which also have low prediction error
$d\big{(}\hat{f}_{0}(s,a),s^{\prime}\big{)}<\gamma$ are therefore in
$\mathcal{D}_{ST}$. By training on transitions with initially low error, we
expect the prediction error on other transitions which belong to
$\mathcal{D}_{ST}$ to also decrease. This slowly brings more and more
transitions to have prediction errors below $\gamma$. On the other hand, the
prediction error is unlikely to decrease for transitions not in
$\mathcal{D}_{ST}$, because they are dissimilar to the transitions with low
initial error.

Thus, at each step $j$ of training, we assign each transition a weight as a
function of the prediction error $||\hat{s}^{\prime}-s^{\prime}||^{2}$ and
multiply this weight by the loss. The full loss is shown in Equation (1).

$\begin{split}\mathcal{L}_{f}&=\frac{1}{T}\sum_{t=1}^{T}\Big{(}||\hat{s}^{t}-s^{t}||^{2}w^{t}\Big{)}\\\
w^{t}&=1-\sigma\big{(}\phi(j)(||\hat{s}^{t}-s^{t}||^{2}-\gamma)\big{)}\end{split}$
(1)

When $\phi(j)$ is higher, the sigmoid $\sigma$ makes the boundary harder. For
example, we use $\phi(j)=0.5j$ and $\phi(j)=5j+3$ in our experiments. When $j$
is large, the boundary is almost hard, and transitions with error below
$\gamma$ have a weight near 1 and transitions with error above have a weight
near 0. When $j$ is small, the boundary is soft, and the weights vary less. We
found that allowing the weighting to be soft during early training steps
improves the stability in the case where few or no transitions have errors
below $\gamma$ at the beginning of training. Adding a constant term ensures
that $\phi(j)>0$, which starts training with a harder boundary, reducing
catastrophic forgetting at the beginning. The parameter $\gamma$ can be chosen
based on either the maximum error that can be corrected by a low-level
controller, or based on the distribution of error on a validation set from the
source environment (e.g. the 97th percentile).

###  III-C Online Learning

Figure 3: Source environment for bimanual rope manipulation (left) and
simulated target environment (right) where there is a robot, obstacles, and
the rope damping and stiffness are changed. Figure 4: Source environment for
plant watering (left) and target environment (right) where there is an
additional plant, and the viscosity is tripled.

In this section, we describe how the proposed adaptation method can be
combined with prior work on planning with unreliable dynamics to achieve data-
efficient online adaptation of dynamics models. A block diagram of the full
method, which we call FOCUS, is shown in Figure 2. FOCUS consists of an
offline phase and an online phase. In the offline phase, we train a dynamics
model using data from the source environment, which in our experiments is a
simple simulation. We use random actions to collect a diverse set of data and
standard techniques for training the fully-connected neural network dynamics
model [6].

In the online phase, we adapt the learned dynamics model to the target
environment (e.g. the real world). This process alternates between (1)
collecting new data in the target environment by planning and executing, (2)
fine-tuning the dynamics, and (3) fine-tuning the model deviation estimator
(MDE). We now explain the data collection and MDE fine-tuning steps.

Figure 5: (center) Histograms showing weights assigned to the data according
to Equation (1) during the first 20 epochs of training. A histogram is shown
for each epoch, where color varies with epoch, and these histograms are
staggered along the y-axis. Initially, the weights vary only slightly across
the data, but the distribution becomes strongly bimodal as training
progresses. Examples of transitions given weight 0 (left) and weight 1 (right)
at the end of training.

####  III-C1 Planning and Execution for Data Collection

We use a kinodynamic RRT planner where nodes are propagated using the learned
dynamics model. Since the learned dynamics are adapting to only
$\mathcal{D}_{ST}$ and are not accurate everywhere, we additionally constrain
the planner to stay in $\mathcal{D}_{ST}$. Since $\mathcal{D}_{ST}$ is not
known a priori, we train another neural network, called a model deviation
estimator (MDE) [7, 26, 6], to predict the error of the dynamics model (more
details in Section III-C2). In addition to producing more robust plans,
planning with MDEs has the additional benefit of focusing data collection on
$\mathcal{D}_{ST}$. By collecting more data where the source and target
dynamics are close, a larger fraction of $\mathcal{D}$ collected by the
adaptation procedure is likely to be in $\mathcal{D}_{ST}$ and more useful for
training.

We use the MDE in planning as a constraint. If the dynamics error predicted by
the MDE is below a threshold $d_{\text{max}}$, then we add it to the planning
tree. We also randomly accept transitions with high predicted dynamics error
with low probability (0.01), so that the planner will occasionally return
paths with exploratory actions (we call these random-accepts). These
exploratory actions are essential for training the MDE since they can correct
over-estimation of model error from the MDE. The threshold $d_{\text{max}}$
for allowable error is similar to $\gamma$ but may be set higher or lower to
control the exploration/exploitation tradeoff.

The robot uses the planner to attempt the task, and repeatedly plans and
executes open-loop until a timeout or the goal is reached. If no plan is found
that reaches the goal, the plan which gets closest to the goal is executed.
This repeated planning and execution is called one episode. After some fixed
number of episodes (e.g. 10) we fine-tune the dynamics and the MDE using all
data collected so far during the online phase.

####  III-C2 Fine-tune MDE

The MDE is used to constrain planning to regions where the dynamics model is
predicted to be accurate, which has three benefits. First, we do not know the
true error during planning, so the main purpose of the MDE is to give an
estimate of error. Second, it helps bias data collection to contain
transitions from $\mathcal{D}_{ST}$. Third, it makes reaching the goal more
likely since it avoids plans that do not match the true dynamics. The MDE
$\hat{d}=h(\mathcal{E},s,a,\hat{s}^{\prime})$ is a convolutional neural
network that takes as input the environment, state, action, and next predicted
state, and predicts the error of the dynamics model $\hat{d}$. We represent
the environment $\mathcal{E}$ as a voxel grid of the static scene. The ground
truth error used for training is the error between the true observed state and
the state predicted by the learned dynamics:
$d=d(\hat{s}^{\prime},s^{\prime})$. The loss function is shown in Equation
(2). Intuitively, the MDE should be easier to learn with less data than
learning the dynamics accurately everywhere, since the MDE need only predict
the magnitude of the error as opposed to the full state vector [6].

$\mathcal{L}_{h}=||\hat{d}-d||^{2}e^{-{k_{h}d}}$ (2)

$k_{h}$ is a hyperparameter that reduces the need to predict high dynamics
errors with high accuracy. We set $k_{h}=10$.

##  IV RESULTS

We begin by describing the two domains for our experiments: bimanual rope
manipulation and plant watering. We then validate the claims that (1) our
proposed adaptation method achieves lower prediction error in regions of
similar dynamics, and (2) that FOCUS achieves higher success rates more
quickly in the online adaptation setting compared to baselines that train on
all data equally.

Figure 6: Prediction error for our method versus two baselines, evaluated on a
dataset of transitions from regions where the source and target dynamics are
similar.

Bimanual Rope Manipulation:  In this task, a 16-DOF dual-arm robot is holding
two ends of a rope in a scene resembling the engine bay of a car (scene shown
in Figures 1,3). The task mimics putting on lifting straps on the engine,
which requires moving the rope through narrow passages and around protrusions.
The goal is to place the middle of the rope in a goal region defined as a
sphere of radius $0.045\text{\,}\mathrm{m}$. The planner outputs gripper
position actions, and a local controller executes the actions while
maintaining gripper orientations. The learned dynamics model predicts the
state of the rope, represented as 25 points in 3D, given the initial rope
state and gripper position actions. The distance function is the L2 norm of
all rope points.

In the rope manipulation experiments, the source simulation has no obstacles
and the robot is simplified to floating kinematic grippers. We then test
adaptation to two different target environments: (1) another simulation that
includes the robot and obstacles and has different damping and stiffness
parameters for the rope, and (2) the real world. This tests adaptation to a
different rope despite the distracting transitions where the rope deforms on
the robot or the obstacles. Gazebo with ODE physics is used for simulation
[37]. For rope manipulation, the set $\mathcal{D}_{ST}$ would be the
transitions from the target environment where the rope is in free space. We
use $\gamma=0.08$.

Rope Manipulation

Plant Watering

Figure 7: Post-learning evaluation of planning during test time in simulation:
Metrics shown over the 20 iterations of online learning. If the planner does
not find a plan that reaches the goal, the plan with the lowest distance to
the goal is executed. We report (1) overall success, (2) success given that
the plan reaches the goal, (3) and the percentage of plans that reach the
goal. The shaded interval is the 95% confidence interval of the mean, with the
boot-strapping method used by Seaborn. Figure 8: Success rate for the
AllDataNoMDE baseline (left) versus FOCUS (right) for online adaptation to
real-world bimanual rope manipulation. Error bars show the 95% confidence
interval.

Plant Watering:  The goal in this task, illustrated in Figure 4 is to pour at
least 75% of the initial volume from a controlled container into a target
container without spilling more than 2%. The source environment is a variation
from the SoftGym PourWater environment [38]. The controlled container can
rotate about the z-axis. The state space is the 4-DOF $[x,y,z,\theta]$ pose of
the controlled container, 3-DOF $[x,y,z]$ pose of the target container,
control volume, and target volume. The action space is a target pose
$[x_{des},y_{des},\theta_{des}]$ which is followed by a proportional
controller. The distance function $d$ is a weighted sum of the distance in x,
y, theta, and volume.

The target environment has triple the viscosity, a shorter container, and a
plant in the target container. Although the agent can pour from above, that
causes the water to splatter, which is more difficult to predict than the
free-space pours of the target environment. Additionally, the box can collide
with the plant, which is dissimilar to the source dynamics where there are no
obstacles. The set $\mathcal{D}_{ST}$ contains free-space motions and pours,
whereas collision with and pouring on the plant is not in $\mathcal{D}_{ST}$.
We use $\gamma=0.10$.

###  IV-A Validating the Adaptation Method

We now evaluate whether the proposed adaptation method achieves lower
prediction error on $\mathcal{D}_{ST}$. We start by creating validation sets
that contain transitions not used for training, and which are known to be in
$\mathcal{D}_{ST}$. For rope, this means transitions where the rope is in free
space. For water, this means transitions that do not collide with or pour over
the plant. We evaluate our method and two baselines, all starting from the
same pre-trained model and adapting to the same dataset from the target
environment. The baseline AllData fine-tunes on all transitions with equal
weights. The baseline LowInitialError uses our weighting function, but
computes the weights once using $\hat{f}_{0}$ and does not re-compute them
throughout training. Our method uses our proposed loss function (Equation (1))
which re-computes the weights on each batch during training. We do not compare
to domain randomization methods, which require knowledge of which physical
parameters may vary and corresponding bounds.

For bimanual rope manipulation, the dataset contains 6288 transitions and the
validation set contains 792 transitions. For plant watering, the training
dataset contains 854 transitions and the validation set contains 130
transitions. The results are visualized in Figure 6. In both experiments, the
error of our method is statistically significantly lower than both baselines
($p<0.0001$).

Figure 5 demonstrates the intuition behind our adaptation method. In the
center, we show histograms over time of the weights assigned to the
transitions in the training dataset, for water and for rope. The distribution
is initially unimodal since the weighting function when $j=0$ is soft, but as
training progresses the distribution rapidly becomes bimodal, where most
transitions are given a weight of 1, but some transitions are given a weight
of 0. We show examples of these low and high weight transitions on either
side. For rope manipulation, we found that the number of the transitions with
prediction error below $\gamma$ increases from 52% at epoch 1 to 80% at epoch
20, which shows that the subset of data we train on grows. This explains why
our method outperforms the LowInitialError baseline, since that baseline is
not making use of as much of the data as our method does. The presence of
transitions with 0 weight (e.g. 20% for the rope domain at epoch 20) shows
that our method is not converging to training on all examples, which does not
perform well based on the AllData baseline.

###  IV-B Online Learning Experiments

We show that FOCUS achieves higher task success with less data than baselines
which fine-tune the dynamics on all available data. The first baseline, called
AllDataNoMDE does not use our proposed adaptation method and does not use MDEs
when planning, making it a conventional online learning method. The second,
called AllData includes MDEs in planning but ablates our fine-tuning method.
First, we evaluate our method in the rope manipulation domain on adaptation
from one simulation to another (see Figure 3). We ran 20 iterations of online
learning, where each iteration consists of 10 episodes for rope manipulation,
and 27 episodes for plant watering ($\approx$6000 total transitions during
learning). We then repeated this 10 times for each method/baseline with
different random seeds.

After learning, we took the models saved after each learning iteration and ran
100 episodes of evaluation per method. To maximize the success rates of all
methods, we use a longer timeout and do not allow random-accepts when
planning. We also stop execution and replan if the error between the plan and
the observed state exceeds a large threshold on model error (0.25).

The results of this experiment are summarized in Figure 7. In both tasks, the
proposed method (FOCUS) shows the highest data efficiency in achieving the
goal. In the rope manipulation task, AllData, never finds paths to the goal.
This is because its dynamics are not sufficiently accurate, and so the MDE
constraint makes the planning problem infeasible. Accurately learning the
dynamics using AllData or AllDataNoMDE involves predicting the deformation of
the rope on obstacles, which is challenging given a dataset of only a few
thousand transitions. The boundary of $\mathcal{D}_{ST}$ is less extreme in
the watering task, enabling both baselines to achieve high success rates,
though using more data than FOCUS. By inspecting the behavior of the methods,
we found that our method quickly focuses on pours that do not interact with
the plant, improving data efficiency.

###  IV-C Real Robot Results

To demonstrate how FOCUS enables a robot to quickly learn a task in the real
world, we performed a similar experiment to the first rope manipulation
experiment, but on real robot hardware, where sensor and actuator noise are
substantial factors (approximately $5\text{\,}\mathrm{cm}$ of end-effector
error). We use CDCPD2 [39] to track the rope state. The geometry of the car
scene is approximated with primitive geometric shapes. We use the same source
simulation environment as for the simulation rope experiment, but now the
target environment is the real world. The robot should adapt the simulated
free-space rope dynamics from simulation to the real world, despite the
different real-world free-space dynamics and the fact that the rope can deform
on the robots’ arms or on the objects in the scene. Because perception error
and actuation error are higher in the real world than in simulation, we use
$\gamma=0.2$.

We ran the online learning procedure with a single start configuration and a
single goal region for one random seed and compare FOCUS to the AllDataNoMDE
baseline, since AllDataNoMDE performed best in simulation. After 15 iterations
of learning, we freeze the models and evaluate task success 32 times. In our
unoptimized implementation, planning and fine-tuning the models took
comparable amounts of time, each taking between 1 and 30 minutes depending on
task complexity/dataset size. A more advanced implementation would run fine-
tuning in parallel with execution so that the robot is idle only while
planning.

The success rates are shown in Figure 8. With FOCUS, the robot successfully
placed the rope under the engine 15/32 times, while AllDataNoMDE succeeded
11/32 times. Failure modes for FOCUS include the rope getting pulled out of
the robots’ hands, getting too close and catching on obstacles, and failing to
find plans that reach the goal. We find less improvement in the real world
than in simulation, which may be due to perception and significant actuator
error.

##  V CONCLUSION

This paper studies the problem of adapting learned dynamics models to datasets
that contain transitions where the dynamics are very different from the source
environment. This type of domain mismatch is common in online dynamics
learning settings, where the source dynamics are learned in simulation or on a
simpler task. Traditional adaptation methods can fail in this setting because
trying to fit data from regions of dissimilar dynamics leads to poor
predictions even in regions where the source and target dynamics are similar.

Our key insight is to instead focus adaptation on regions where the source and
target dynamics are similar. We propose an adaptation method that assigns high
weight to transitions with low prediction error, and dynamically re-assigns
weights during the course of training. The set of low-error transitions is
initially a small set, but grows as training pulls down the prediction error
for other similar transitions. We combine our adaptation method with prior
work on planning with unreliable dynamics to make FOCUS, a data-efficient
online adaptation method. We demonstrate that FOCUS can learn a bimanual rope
manipulation task in simulation and in the real world, and achieves higher
task success rates than baselines that attempt to fit all the training data.

## References

  * [1] Carla E. Brodley and Mark A. Friedl.  Identifying mislabeled training data.  Journal of Articial Intelligence Research, 1999.
  * [2] Gaoxia Jiang, Wenjian Wang, Yuhua Qian, and Jiye Liang.  A unified sample selection framework for output noise filtering: An error-bound perspective.  JMLR, 2021.
  * [3] Daniel Bogdoll, Maximilian Nitsche, and J. Marius Zöllner.  Anomaly detection in autonomous driving: A survey.  CVPR Workshop, 2022.
  * [4] Michael J. Sorocky, Siqi Zhou, and Angela P. Schoellig.  Experience selection using dynamics similarity for efficient multi-source transfer learning between robots.  In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 2739–2745, 2020.
  * [5] Botond Bócsi, Lehel Csató, and Jan Peters.  Alignment-based transfer learning for robot models.  In The 2013 International Joint Conference on Neural Networks (IJCNN), pages 1–7, 2013.
  * [6] Peter Mitrano, Dale McConachie, and Dmitry Berenson.  Learning Where to Trust Unreliable Models in an Unstructured World for Deformable Object Manipulation.  Science Robotics, 2021.
  * [7] Alex LaGrassa and Oliver Kroemer.  Learning model preconditions for planning with multiple models.  CoRL, 2022.
  * [8] Sosale Shankara Sastry and Alberto Isidori.  Adaptive control of linearizable systems.  IEEE Transactions on Automatic Control, 34(11):1123–1131, 1989\.
  * [9] Karol Arndt, Ali Ghadirzadeh, Murtaza Hazara, and Ville Kyrki.  Few-shot model-based adaptation in noisy conditions.  IEEE Robotics and Automation Letters, 6(2):4193–4200, 2021.
  * [10] Ben Evans, Abitha Thankaraj, and Lerrel Pinto.  Context is everything: Implicit identification for dynamics adaptation.  In 2022 International Conference on Robotics and Automation, ICRA 2022, Philadelphia, PA, USA, May 23-27, 2022, pages 2642–2648. IEEE, 2022.
  * [11] Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S. Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn.  Learning to adapt in dynamic, real-world environments through meta-reinforcement learning.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [12] Kendall Lowrey, Svetoslav Kolev, Jeremy Dao, Aravind Rajeswaran, and Emanuel Todorov.  Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system.  In 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), pages 35–42. IEEE, 2018.
  * [13] Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox.  Closing the sim-to-real loop: Adapting simulation randomization with real world experience.  In 2019 International Conference on Robotics and Automation (ICRA), pages 8973–8979, 2019.
  * [14] Joshua D Langsfeld, Krishnanand N Kaipa, and Satyandra K Gupta.  Selection of trajectory parameters for dynamic pouring tasks based on exploitation-driven updates of local metamodels.  Robotica, 36(1):141–166, 2018.
  * [15] Anthony Courchesne, Andrea Censi, and Liam Paull.  On assessing the usefulness of proxy domains for developing and evaluating embodied agents.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4298–4305, 2021.
  * [16] Lisa Torrey and Jude Shavlik.  Transfer learning.  In Handbook of research on machine learning applications and trends: algorithms, methods, and techniques, pages 242–264. IGI global, 2010\.
  * [17] Xu Chu, Ihab F. Ilyas, Sanjay Krishnan, and Jiannan Wang.  Data cleaning: Overview and emerging challenges.  International Conference on Management of Data, 2016.
  * [18] Taghi M. Khoshgoftaar and Pierre Rebours.  Improving software quality prediction by noise filtering techniques.  J. Comput. Sci. Technol., 2007.
  * [19] Ananya Kumar, Tengyu Ma, and Percy Liang.  Understanding self-training for gradual domain adaptation.  Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, 2020.
  * [20] Yang Zhang, Philip David, and Boqing Gong.  Curriculum domain adaptation for semantic segmentation of urban scenes.  In Proceedings of the IEEE international conference on computer vision, pages 2020–2030, 2017.
  * [21] Runzhe Zhan, Xuebo Liu, Derek F. Wong, and Lidia S. Chao.  Meta-curriculum learning for domain adaptation in neural machine translation.  In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 14310–14318. AAAI Press, 2021.
  * [22] Rémy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer.  Automatic curriculum learning for deep RL: A short survey.  IJCAI, 2020.
  * [23] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston.  Curriculum learning.  ICML, 2009.
  * [24] Leslie Pack Kaelbling and Tomás Lozano-Pérez.  Integrated task and motion planning in belief space.  The International Journal of Robotics Research, 32(9-10):1194–1227, 2013.
  * [25] Robert Platt Jr., Russ Tedrake, Leslie Pack Kaelbling, and Tomás Lozano-Pérez.  Belief space planning assuming maximum likelihood observations.  In Yoky Matsuoka, Hugh F. Durrant-Whyte, and José Neira, editors, Robotics: Science and Systems VI, Universidad de Zaragoza, Zaragoza, Spain, June 27-30, 2010. The MIT Press, 2010.
  * [26] Dale McConachie, Thomas Power, Peter Mitrano, and Dmitry Berenson.  Learning When to Trust a Dynamics Model for Planning in Reduced State Spaces.  IEEE Robotics and Automation Letters, 2020.
  * [27] Marc Peter Deisenroth, Dieter Fox, and Carl Edward Rasmussen.  Gaussian Processes for Data-Efficient Learning in Robotics and Control.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 37:408–423, 2015.
  * [28] Wilson Yan, Ashwin Vangipuram, Pieter Abbeel, and Lerrel Pinto.  Learning predictive representations for deformable objects using contrastive estimation.  In Conference on Robot Learning, pages 564–574. PMLR, 2021.
  * [29] Daniel Seita, Pete Florence, Jonathan Tompson, Erwin Coumans, Vikas Sindhwani, Ken Goldberg, and Andy Zeng.  Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks.  In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 4568–4575. IEEE, 2021.
  * [30] Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B Tenenbaum, and Antonio Torralba.  Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids.  In International Conference on Learning Representations, 2018.
  * [31] Tatiana López Guevara, Nicholas K Taylor, Michael U Gutmann, Subramanian Ramamoorthy, and Kartic Subr.  Adaptable pouring: Teaching robots not to spill using fast but approximate fluid simulation.  In Proceedings of the Conference on Robot Learning (CoRL), volume 2, 2017.
  * [32] Yoshiyuki Noda and Kazuhiko Terashima.  Modeling and feedforward flow rate control of automatic pouring system with real ladle.  Journal of Robotics and Mechatronics, 19(2):205–211, 2007.
  * [33] Takaaki Tsuji and Yoshiyuki Noda.  High-precision pouring control using online model parameters identification in automatic pouring robot with cylindrical ladle.  In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 2563–2568. IEEE, 2014.
  * [34] Zijie Li and Amir Barati Farimani.  Graph neural network-accelerated lagrangian fluid simulation.  Computers & Graphics, 103:201–211, 2022.
  * [35] Changhao Wang, Yuyou Zhang, Xiang Zhang, Zheng Wu, Xinghao Zhu, Shiyu Jin, Te Tang, and Masayoshi Tomizuka.  Offline-online learning of deformation model for cable manipulation with graph neural networks.  RA-L, 2022.
  * [36] Mingrui Yu, Kangchen Lv, Hanzhong Zhong, Shiji Song, and Xiang Li.  Global model learning for large deformation control of elastic deformable linear objects: An efficient and adaptive approach.  IEEE Transactions on Robotics, 2022.
  * [37] Nathan P. Koenig and Andrew Howard.  Design and use paradigms for Gazebo, an open-source multi-robot simulator.  IROS, 2004.
  * [38] Xingyu Lin, Yufei Wang, Jake Olkin, and David Held.  Softgym: Benchmarking deep reinforcement learning for deformable object manipulation.  In Jens Kober, Fabio Ramos, and Claire J. Tomlin, editors, 4th Conference on Robot Learning, CoRL 2020, 16-18 November 2020, Virtual Event / Cambridge, MA, USA, volume 155 of Proceedings of Machine Learning Research, pages 432–448. PMLR, 2020.
  * [39] Yixuan Wang, Dale McConachie, and Dmitry Berenson.  Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints.  ICRA, 2021.