# CosmosDSR - a methodology for automated detection and tracking of orbital
debris using the Unscented Kalman Filter11footnotemark: 1

Daniel S. Roll222First author: Email<EMAIL_ADDRESS>Zeyneb Kurt
Wai Lok Woo

###### Abstract

The Kessler syndrome refers to the escalating space debris from frequent space
activities, threatening future space exploration. Addressing this issue is
vital. Several AI models, including Convolutional Neural Networks, Kernel
Principal Component Analysis, and Model-Agnostic Meta-Learning have been
assessed with various data types. Earlier studies highlighted the combination
of the YOLO object detector and a linear Kalman filter (LKF) for object
detection and tracking. Advancing this, the current paper introduces a novel
methodology for the Comprehensive Orbital Surveillance and Monitoring Of Space
by Detecting Satellite Residuals (CosmosDSR) by combining YOLOv3 with an
Unscented Kalman Filter (UKF) for tracking satellites in sequential images.
Using the Spacecraft Recognition Leveraging Knowledge of Space Environment
(SPARK) dataset for training and testing, the YOLOv3 precisely detected and
classified all satellite categories (Mean Average Precision=97.18%, F1=0.95)
with few errors (TP=4163, FP=209, FN=237). Both CosmosDSR and an implemented
LKF used for comparison tracked satellites accurately for a mean squared error
(MSE) and root mean squared error (RME) of MSE=2.83/RMSE=1.66 for UKF and
MSE=2.84/RMSE=1.66 for LKF. The current study is limited to images generated
in a space simulation environment, but the CosmosDSR methodology shows great
potential in detecting and tracking satellites, paving the way for solutions
to the Kessler syndrome.

###### keywords:

Unscented Kalman Filter, YOLO, Orbital Debris

††journal: Advances in Space Research

[1]organization=Northumbria University, addressline=College Street,
city=Newcastle upon Tyne, postcode=NE1 8ST, country=England
[2]organization=University of Sheffield, addressline=Western Bank,
city=Sheffield, postcode=S10 2TN, country=England

##  1 Introduction

###  1.1 Background

As machine learning (ML) applications increase in prevalence, one area that
has gained much traction over the last few decades is computer vision (CV), a
field concerned with developing methods for a computer to interpret and
understand visual information from images and videos. One common CV technique
that involves identifying objects within such media is known as object
detection (OD), combining the technology of image processing with deep
learning (DL). The benefits of OD technology are numerous, including increased
efficiency and performance by automating tasks that require human
participation, reducing necessity of manual labour as well as being able to do
so in a time-sensitive manner that is scalable to large datasets [33]. An
increasingly significant domain that has potential to profit greatly from the
application of OD is that of space – specifically in terms of orbital debris
and satellite detection.

Decades of humanity’s endeavour to investigate, explore and innovate
concerning the stellar environment have led to the accumulation of unwanted
items in orbit, ranging in size and origin from millimetre-wide flecks of
paint eroded from spacecraft hulls to intact dysfunctional satellites [42].
More than 23,000 tracked ‘resident-space objects’ (RSO’s) currently circle the
Earth. Such objects are known colloquially as ‘space junk’, comprising mostly
of discarded elements of humanity’s previous space ventures, with only a small
percentage amounting to functional satellites or spacecraft [24]. Such items
pose serious risks for the future of both space investigation and travel.
Research estimating the speed of collisions involving pieces of orbital debris
posited possible speeds of up to 50,000 mph [32], not only fatal to life, but
in the event of a collision with another object, catalysing the production of
further debris fragments from the resulting impact. Thus, if thorough analysis
and cataloguing of orbital debris is not rigorously maintained, such risks
will increase exponentially in line with the degradation of the celestial
environment. Known as The Kessler Syndrome [26], this idea of cascading
collisions has the potential to not only severely delay future space travel,
but even end it entirely. The accumulation of such objects, therefore, must be
addressed. One common issue, however, is the necessity for accurate
measurements of information relevant to approximating the relative positions
of debris fragments. Accomplishing this could incorporate the design and
implementation of AI technologies and as shown in the literature, such
intelligent solutions have already been theorised and tested. Currently, this
includes the use of ResNet [45]; customised traditional CNN models [21]; KPCA
with 2-D wavelet transformations [35] and MAML [17]. Such attempts are varied,
and target vastly different sub-domains within orbital debris; yet work
together to build a picture of the scale and complexity of the problem.
Understandably, with such variation, the data employed in the studies is also
varied, including light-curve estimations [5], RGB images [2] and radar scans
[39], with image data being the most common. The possibility to use such
images in conjunction with CV technologies such as OD offers the ability to
design and test bespoke software that can subsequently be tested and deployed
using sporadic real data. One piece of previous literature that applies
contemporary OD to identify space objects is research by Fitzgerald [15] to
determine objects in low-resolution, wide field-of-view (WFOV) synthetic
images of the night sky representative of physical images taken by the PANDORA
sensor array located at the Air Force Maui Optical and Supercomputing Site in
the USA. We propose architecture, in the form of an object detector and
tracker using a blend of YOLO and a Kalman Filter (KF), to be deployed to
autonomously monitor space objects for cataloguing and anomaly detection
purposes.

###  1.2 You Only Look Once (YOLO

The YOLO algorithm used was first introduced by Redmon et al. [49] in 2016 as
an improvement to its contemporaries with regards to speed and performance.
Whilst previous two-step detector algorithms such as R-CNN require their
nominal two steps, firstly identifying regions of interest and then
classifying those regions [10], YOLO revolutionised this with the ability to
process images in a single forward pass. This is achieved by applying a single
neural network to the whole image, allowing for the simultaneous prediction of
both bounding boxes and class probabilities. YOLO therefore proposed
remarkable improvements in speed and efficiency, establishing itself as a
method for real-time object detection and offering a solution for applications
requiring instant feedback. Following the development of the initial YOLO
algorithm, many other iterations have been published, such as YOLO9000 (v2)
[50] and recently, YOLOv8 [51]. Each version aimed to address limitations of
the previous, offering improved class predictions, deeper architectures and
more detection layers. The widespread deployment of YOLO across vast domains
and topics is demonstrated, including algorithms trained to identify traffic
lights for the development of autonomous vehicles [40]; for cell counting in
fluorescence microscopy for medical research [3]; for pest detection within
agricultural settings [63]. With regards to the stellar environment, versions
of YOLO have already been successfully applied to various topics including
residual space object (RSO) detection and tracking [38], satellite component
recognition [36] and autonomous target detection from satellite imagery [56].
There are certain inherent drawbacks, however, such as difficulty with
recognising small or overlapping objects or the impact of noise and occlusion
on the detection [57].

###  1.3 The Kalman Filter

The Kalman Filter is a recursive mathematical algorithm developed by the
electrical engineer Rudolf E. Kálmán in the 1960’s as a method for simplifying
linear filtering and prediction problems [25]. A KF, in terms of a system,
builds upon previous output to compute future output by continuously updating
its estimates dependent on new information; using prior state estimates and
current data, the filter predicts the next state of the system. This allows a
KF to evaluate the changing state of a linear, dynamic system whilst
incorporating noise and incomplete measurements with the prior estimates,
mostly negating their negative impact on estimate accuracy [58]. As such, the
practical applications of KF are diverse, including time-series forecasting
and stock price prediction within finance [47, 37], autonomous robot
localisation [62] and many fault-diagnosis or sensor-control functions within
industry [6]. The KF offers benefits such as optimum state estimation ability
for such linear systems with Gaussian noise, as well as its efficiency in
real-time scenarios, yet is therefore somewhat restricted as assumptions of
linearity and Gaussian noise must be upheld [27]. This, therefore, makes KF
less suitable for non-linear systems or those with non-Gaussian noise and led
to the development of the Unscented Kalman Filter (UKF). The UKF approximates
non-linear functions using a deterministic sampling technique – the Unscented
Transformation (UT) – as opposed to attempting to linearise them [52]. This
results in high accuracy of estimation for many applications and is therefore
routinely employed in areas where such high accuracy of state estimations is
vital, such as autonomous vehicle sensor fusion and other advanced navigation
systems [28].

###  1.4 Combined Approaches

Unsurprisingly, attempts to blend the KF into deeper architecture, as a method
to track detected objects in OD applications such as YOLO algorithms, have
been attempted. Research by Barreiros et al. [8] attempted to detect and track
zebrafish within videos captured in an experimental setting reminiscent of
natural marine conditions. In order to successfully track fish during rapid
movement sequences, in which occlusion causes inconsistency of detections, a
new approach was necessary as previous literature mainly studied shallower
water or more stationary fish. The authors designed a convolutional network
for their object recognition using YOLOv2 to delimit the region of a fish’s
head and facilitate individual fish detection. Following this, a KF was used
to estimate the head position and track the trajectory of each fish throughout
subsequent frames. Results indicated superior performance of the YOLO,
indicating its ability to detect fish even when presented with varying numbers
or occlusion. The tracking with KF also performed well on frames with optimal
characteristics, such as a low number of fish, however encountered problems
and lowered accuracy when presented with a higher number, faster swimming
motions or more occlusion. Overall, however, the proposed system worked to
effectively detect and track zebrafish within the experimental setting.
Furthermore, similar architecture has been applied to the field of space as
observed in the previously mentioned study by Fitzgerald [15]. The author
applied a model consisting of a deep-learned object detector using YOLOv5
combined with KF for object tracking to synthetic images of RSO’s within the
Geosynchronous Equatorial Orbit (GEO) belt representative of physical images
taken by the WFOV sensors of the PANDORA system. Aiming to first process the
images, to detect low-light objects, before tracking those objects as a method
of passive monitoring, the model was then applied to a real, physical dataset
as to evaluate performance in a tangible application. Moreover, the
architecture was compared and contrasted to classical OD methods: Scale
Invariant Feature Transformation (SIFT), Histogram of Oriented Gradients:
Orientation Rank (HOGOR) and Form Factor (FF). The author elucidates that the
proposed YOLO/KF architecture was not only efficient and accurate when
performing object tracking and detection across all test sets of images, but
also comparatively outperformed the classic detectors. Fitzgerald notes that
whilst the research was a success, increasing GEO RSO detection accuracy from
the classical methods, design improvements could be made, and future work
could aim to also compare the results to other sensors involved in space
object monitoring.

###  1.5 Research Gap

The available literature therefore advocates for the usage of OD methods,
specifically object detection and tracking combinations, in the domain of RSO
identification. The state of the stellar landscape is exponentially declining,
and measures must be taken to ‘clean up’ the environment. The justification to
apply the above methods to this problem is therefore substantiated, allowing
for a novel design that combines leading technologies into an optimised
solution for RSO classification and tracking. Whilst the research by
Fitzgerald focused on the integration of the linear KF, based on the observed
benefits from the literature, there exists a gap in which to probe the
efficacy of the UKF for RSO tracking and integrate it in combination with YOLO
architecture to facilitate identification and classification. Due to the
complex, multifaceted and often non-linear nature of the orbital debris
problem, incorporating an algorithm inherently capable of dealing with such
non-linearity proposes many benefits. As such, this thesis outlines a model
and methodology for RSO identification and tracking from synthetic images of
satellites using a combination of a deep-learned object detector using YOLOv3
and an Unscented Kalman filter. The study aims to evaluate the performance of
the UKF by directly comparing prediction accuracies and precision to those
generated by a linear KF. Successful implementation of novel architecture
would stand to fill numerous gaps within the research area, addressing some of
the associated limitations brought forth by previous work and further
developing the understanding of RSO tracking methodologies. This, in turn,
would offer a solution to orbital debris tracking that could be further
implemented into removal technologies as a solution to address the developing
Kessler syndrome, stimulating stellar clean-up and expediating future space
exploration.

##  2 Data Selection

###  2.1 Data Evaluation

Upon review, there existed not enough easily accessible or publicly available
real RSO images to form a dataset (even with data augmentation), and it was
noted that there is a distinct lack of readily available real image data
regarding satellites available to researchers. Moving forward, collating a
repository of real images is a must, even if this only results in a dataset
for usage as a confirmation set, as to test architecture on real, empirical
data. However, this was determined to be outside the scope of the current
project and a decision was made to focus on synthetic datasets containing
simulated, computer-generated images. Literature already supports the use of
synthetic data within the field, not only the previous study by Fitzgerald
[15] but also research by Zhang et al. [64] which developed a diverse space-
target dataset including both satellites and debris. The current study,
however, focuses on the Spacecraft Recognition Leveraging Knowledge of Space
Environment (SPARK) dataset, available from the University of Luxembourg and
first published in research by Musallam et al. [41].

###  2.2 The SPARK Dataset

The SPARK is a unique multi-modal space object image dataset that was
developed to be used in Space Situational Awareness (SSA) applications. All
images were generated under a realistic space simulation environment, with
diverse sensing conditions producing diverse orbital scenarios, with the
authors positing that preliminary experimental evaluation suggests the dataset
is both valid and relevant. This would identify its utility in training OD
applications. The dataset contains images of n=11 distinct classes of RSO, the
names of which can be seen in Table 1.

The data is split into two streams, Stream-1 and Stream-2. The first contains
images in 1080x1080 resolution of all 11 classes, supplied with relevant
bounding-box data. The second is for trajectory predictions and contains
numerous folders each containing a set of consecutive images in 1440x1080
resolution of one RSO, usually in a much smaller format.

###  2.3 Data Retrieval and Processing

A formal request for access to the dataset was made to the University of
Luxembourg. For the current study, using the entire dataset was deemed to be
excessive and a subset was created for use throughout. This resulted in a
current data pool that comprised n=22000 images, equally distributed between
the 11 classes, from Stream-1 as well as n=1500 images from n=5 sets from
Stream-2. Example images from both streams are shown in Figure 1.

Following successful partitioning of the dataset, annotation files were
created for each image. This was streamlined with the use of a simple
processing script written in R using R Studio v2021.09.0. The label files were
added to the Stream-1 folder and, alongside the relevant sets from Stream-2,
it was uploaded to Google drive for use within Google Colab.

Fig. 1: Examples of images from Stream-1 (LEFT) and Stream-2 (RIGHT) of the SPARK Table 1: RSO classes within the SPARK Class ID | Name
---|---
0 | lisa_pathfinder
1 | proba_data3_csc
2 | smart_1
3 | xmm_newton
4 | soho
5 | earth_observation_sat_1
6 | debris
7 | proba_2
8 | proba_3_ocs
9 | cheops
10 | double_star

##  3 Algorithm Selection

###  3.1 YOLO

As discussed above, the current study elected to utilise a YOLO algorithm for
OD purposes, in part due to the wealth of available literature demonstrating
its effectiveness in similar scenarios, but also due to the possibility to
easily integrate a KF for object tracking. Various versions of YOLO were
analysed for their suitability for use within the current study. YOLOv3 was
developed and published in 2018 by Redmon & Farhadi [51] as a successor to the
widely popular YOLO9000. It offered a larger overall architecture that
remained a state-of-the-art real-time detector, as well as computing an
‘objectness score’ for bounding boxes using logistic regression, translating
as a score of 1 for the anchor box that best overlaps the ground truth, and 0
for other anchor boxes [57]. As well as simplifying the prediction mechanism,
this also helps to optimise performance by ensuring only high-objectness-score
bounding boxes are considered, reducing the likelihood for false positives.
Furthermore, the switch from softmax to the use of binary cross-entropy to
train independent logistic classifiers poses the classification problem as a
multilabel classification, allowing multiple labels to be assigned to the same
bounding box, permitting for overlapping labels (e.g. an object can be a Cat
and an Animal). YOLOv3 also allows for multi-scale predictions, making three
predictions at three different scales, increasing both its adaptability and
performance detecting objects of different sizes [51], something extremely
applicable to the proposed problem of satellite identification. As such, after
gauging the available options, YOLOv3 was selected.

###  3.2 YOLO Architecture

In terms of architecture, YOLOv3 boasts a sizeable feature extractor. Its
backbone is referred to as Darknet-53, comprising 53 convolutional layers each
with batch normalisation and leaky-ReLU activation. Residual connections link
input of the 1×1 convolutions across the network with the output of the 3×3
convolutions. The multi-scale detection architecture consists of three
elements: y1, y2 and y3 representing small, medium and large detection scales.
YOLOv3 starts with a 13×13 grid as its initial output at y1, before moving to
a 26×26 grid at y2, merging features from earlier layers by upsampling the
13×13 y1 grid and concatenating it with the medium-scale features. This
process of integration through upsampling then combination is then repeated in
y3 with a 52×52 grid, allowing the algorithm to detect objects of small,
medium and large scales in progression. This, therefore, allows YOLOv3 to
detect objects of different sizes effectively. For the SPARK dataset used in
the current research, with 11 classes, each scale provides an output tensor
with a shape of:

$N\times N\times\left[3\times(4+1+11)\right]$ (1)

where:

  * 1.

$N\times N$ represents the size of the feature maps y1, y2, or y3.

  * 2.

The factor of 3 indicates the number of boxes per cell.

  * 3.

The term $(4+1)$ signifies the four bounding box coordinates plus the
objectness score.

The Darknet-53 backbone and the multi-scale architecture are shown in Figure
2. and Figure 3. respectively.

Fig. 2: The YOLOv3 Darknet-53 Backbone Fig. 3: The YOLOv3 Multi-Scale
Architecture

###  3.3 UKF

The responsibility of object tracking was handled by an Unscented Kalman
Filter, as discussed above. The UKF, in essence, is a sequence of equations
that seek to estimate the most plausible future state of a system [52].
Developed and first published by Julier & Uhlmann [23], the UKF built on the
limitations of the previous linear KF and Extended Kalman Filter as well as
offering a solution to non-linear systems that required KF functionality. The
UKF relies on a process known as an Unscented Transformation, a deterministic
sampling technique for calculating statistics of a random variable undergoing
a non-linear transformation. Instead of linearising the functions, the UT
represents them with a set of sample points, known as Sigma points, which can
accurately represent both the mean and covariance (uncertainty) values. Two
fundamental components of the UKF are the process model, representing the
change in system state change over time, and the measurement model,
representing how observed measurements relate to system state. These
components are crucial to the inner workings of the UKF, which involves two
main steps: a prediction step and an update step. In the prediction step,
Sigma points are passed through the non-linear process model to estimate state
before the predicted mean and covariance are calculated from the propagated
points’ weighted average and weighted covariance. In the update step, the
predicted points are then passed through the non-linear measurement model, and
predicted mean and covariance values are again computed. The cross-covariance
between the predicted state and measurement is assessed, determining Kalman
gain which proposes the scale of necessary adjustment. The predicted state is
then adjusted using the Kalman gain and measurement residual. In design, the
scaling parameter and Sigma point weights can be adjusted to influence the
spread of Sigma points around the mean, in turn tuning performance.

###  3.4 UKF Architecture

In terms of application to an image-based problem such as the object tracking
of the current study, the UKF works to estimate the state of an object, such
as a satellite, as it moves from frame-to-frame. In this sense, the state
definition can be represented by the bounding box coordinates for the image,
such as those produced by the YOLO output. In the prediction step, Sigma
points are generated, representing states the satellite could be in, and
propagated through a motion model to predict where the object could be in the
following frame. For the current implementation surrounding the SPARK, a
constant velocity model was selected. As well as simplifying the estimation
process and being the most common design [7], objects in space that are in
stable orbits, such as satellites, are governed by fundamental principles that
support the use of a constant velocity model, such as Newton’s Law of Inertia
[14] and a related lack of external resistive forces such as atmospheric drag
or volatile gravitational interactions [59]. In the update step, the object is
then detected in the next frame, the predicted location is computed from the
predicted Sigma points and the UKF adjusts its predicted state by the residual
between estimation and measurement. The UKF repeats this process, iteratively,
throughout all frames or images, updating its state estimates. Displaying the
UKF’s predicted bounding box on the image, next to the actual prediction from
an object detection algorithm such as YOLO, can help to visualise the
performance of the model on an image-to-image basis. In terms of the current
study, the performance of the UKF would be weighed against performance of a
linear KF applied to the problem, as employed in the previous research [15]
and offering a good baseline comparison.

###  3.5 CosmosDSR

The current study aims to combine the benefits and functionality of both the
YOLOv3 and UKF, proposing a novel algorithm for Comprehensive Orbital
Surveillence and Monitoring Of Space by Detecting Satellite Residuals
(CosmosDSR). The mathematical representation of the algorithm is presented
below.

First, we can assume a typical YOLO output, relating to the predicted bounding
box coordinates, and consider this the initial state of the Unscented Kalman
Filter at time step $k$:

$\hat{x}_{k}=[x_{k},y_{k},w_{k},h_{k},\dot{x}_{k},\dot{y}_{k},\dot{w}_{k},\dot{h}_{k}]^{T}$
(2)

The relevant velocities $\dot{x}_{k},\dot{y}_{k},\dot{w}_{k},\dot{h}_{k}$ are
calculated using:

$\dot{x}_{k}=\frac{x_{k}-x_{k-1}}{\Delta t}$ (3)

$\dot{y}_{k}=\frac{y_{k}-y_{k-1}}{\Delta t}$ (4)

$\dot{w}_{k}=\frac{w_{k}-w_{k-1}}{\Delta t}$ (5)

$\dot{h}_{k}=\frac{h_{k}-h_{k-1}}{\Delta t}$ (6)

Thus, for predicting the set of sigma points for time step $k+1$, we can
establish the state transition function333$b$, a constant coefficient
representing the extent that external forces or perturbations $U_{k}$
influence the state variables, is set to $b=0$ for the current application.
$f$:

$X_{k+1}^{\prime(i)}=f\left(X_{k}^{(i)},U_{k}\right)$ (7)

$f(X_{k},U_{k})=\begin{bmatrix}x_{k}+\dot{x_{k}}\Delta t+(b^{(x)}\times
U_{k}^{(x)})\\\ y_{k}+\dot{y_{k}}\Delta t+(b^{(y)}\times U_{k}^{(y)})\\\
w_{k}+\dot{w_{k}}\Delta t+(b^{(w)}\times U_{k}^{(w)})\\\
h_{k}+\dot{h_{k}}\Delta t+(b^{(h)}\times U_{k}^{(h)})\\\ \dot{x_{k}}\\\
\dot{y_{k}}\\\ \dot{w_{k}}\\\ \dot{h_{k}}\\\ \end{bmatrix}$ (8)

We can denote the first sigma points at time step $k$, which is equal to the
above state estimate at time step $k$:

$X_{k}^{(0)}=\hat{x}_{k}$ (9)

Using the known dimensions of $n=8$, we can denote the $i$th sigma points at
time step $k$ as:

$X_{k}^{(i)}=\hat{x}_{k}+\sqrt{(n+\lambda)}\cdot\mathrm{Chol}(P_{k})^{(i)}\quad\forall
i=1,...,n$ (10)

$X_{k}^{(i)}=\hat{x}_{k}-\sqrt{(n+\lambda)}\cdot\mathrm{Chol}(P_{k})^{(i-n)}\quad\forall
i=n+1,...,2n$ (11)

Where $\lambda$ can be defined as:

$\lambda=\alpha^{2}(n+\kappa)-n$ (12)

where:

$\lambda$

is the scaling parameter.

$\alpha$

determines the spread of the sigma points (usually a small positive value).

$n$

is the dimensionality of the state.

$\kappa$

is a secondary scaling parameter (often set to 0 or 3 - n).

$Chol$

is the Cholesky decomposition.

The predicted state at the next time step $k+1$ can be calculated by taking
the weighted average of the propagated sigma points:

$\hat{x}_{k+1}=\sum_{i=0}^{2n}w_{m}^{(i)}X_{k+1}^{\prime(i)}$ (13)

where:

$W_{m}$

represents the mean weights.

The predicted covariance at this time step can be defined:

$P_{k+1|k}=\sum_{i=0}^{2n}w_{c}^{(i)}\left(X_{k+1}^{\prime(i)}-\hat{x}_{k+1}\right)\left(X_{k+1}^{\prime(i)}-\hat{x}_{k+1}\right)^{T}+Q_{1}$
(14)

where:

$W_{c}$

represents the covariance weights.

$Q_{1}$

represents the process noise covariance matrix.

The mean weights $Wm$ and covariance weights $Wc$ can be defined as:

$w_{m}^{(0)}=\frac{\lambda}{n+\lambda}$ (15)

$w_{c}^{(0)}=\frac{\lambda}{n+\lambda}+(1-\alpha^{2}+\beta)$ (16)

$w_{m}^{(i)}=w_{c}^{(i)}=\frac{1}{2(n+\lambda)}$ (17)

where:

$\beta$

provides prior knowledge about the state distribution.

We can then transform the predicted sigma points into the measurement space:

$Z_{k+1}^{(i)}=h\left(X_{k+1}^{\prime(i)}\right)=H\cdot X_{k+1}^{\prime(i)}$
(18)

Where the measurement function $H$ can be defined as:

$H=\begin{bmatrix}1&0&0&0&0&0&0&0\\\ 0&1&0&0&0&0&0&0\\\ 0&0&1&0&0&0&0&0\\\
0&0&0&1&0&0&0&0\\\ \end{bmatrix}$ (19)

Now, the predicted measurement mean can be calculated using the transformed
sigma points:

$\hat{z}_{k+1}=\sum_{i=0}^{2n}w_{m}^{(i)}Z_{k+1}^{(i)}$ (20)

As can the measurement covariance:

$P^{z}_{{k+1|k+1}}=\sum_{i=0}^{2n}w_{c}^{(i)}\left(Z_{k+1}^{(i)}-\hat{z}_{k+1}\right)\left(Z_{k+1}^{(i)}-\hat{z}_{k+1}\right)^{T}+Q_{2}$
(21)

where:

$Q_{2}$

represents the measurement noise covariance matrix.

Following this, the cross-covariance between the state and measurement can be
calculated:

$P^{xz}_{{k+1|k+1}}=\sum_{i=0}^{2n}w_{c}^{(i)}\left(X_{k+1}^{\prime(i)}-\hat{x}_{k+1}\right)\left(Z_{k+1}^{(i)}-\hat{z}_{k+1}\right)^{T}$
(22)

The Kalman gain $K$ can then be computed:

$K_{k+1}=P^{xz}_{k+1|k+1}\left(P^{z}_{k+1|k+1}\right)^{-1}$ (23)

Finally, we can update the state estimate and the covariance using the new
measurements:

$X_{k+1}=\hat{x}_{k+1}+K_{k+1}(y_{k+1}-\hat{z}_{k+1})$ (24)

$P_{k+1|k+1}=P_{k+1|k}-K_{k+1}P^{z}_{k+1|k+1}K_{k+1}^{T}$ (25)

###  3.6 Implementation

One of the major benefits in using YOLOv3 is the amount of research,
implementations and support available to augment development. Specifically,
the current research elected to use the Darknet YOLOv3 wrapper developed by
Alexey Bochkovskiy and available in his GitHub repository [4]. This is an
extremely popular fork of the original Darknet repository, offering a wrapper
that enhances Redmon & Farhadi [51]’s YOLOv3, offering improvements in speed,
stability and flexibility as well as integrated customisability. Although
primarily designed to be used in C, Python functionality is included and, as
such, the wrapper can be used within Google Colab. In order to tune relevant
hyperparameters, a GridSearch was used. This involved systematically exploring
a range of values for a set of defined hyperparameters as to identify which
combination of values result in the best performance on a small test set of
relevant images, thus the optimal configuration [31]. This was applied to a
subset of the SPARK data consisting of n=1000 images which was split into
train (n=800) and validate (n=200) sets. This data was used to run YOLOv3 for
each set of parameters, n=200 iterations per combination, before calculating
average loss to approximate performance. The best parameters were then
extracted, giving optimal values of [learning rate: 0.001, momentum: 0.9,
decay: 0.0005]. The YOLO output information, including class predictions and
bounding box information, of the eventual trained YOLO algorithm when
presented with an image would be extracted and saved to a file to be used by
the UKF for object tracking.

The UKF for the current study was developed using the filterpy library, a
comprehensive toolkit designed for filter and state estimation processes
within Python, published by Roger Labbe and available in his GitHub repository
[29, 30]. This specific implementation was chosen as it offers a robust and
optimised version of the core structure of both necessary KF methods required
in the current research. Regarding the current study’s KF code, developed
using Python within Google Colab, the use of the filterpy library allowed for
efficient customisation and flexibility to tailor the filter parameters to the
current application. The UKF was initialised with Merwe-Scaled Sigma Points,
to preserve the Gaussian characteristics of the system’s state whilst offering
high stability, accuracy and performance [60]; the measurement and process
noise covariance matrices were initialised as identity matrices scaled to an
arbitrary value of 0.1 for neutrality and simplicity [16]. The process for
designing the linear KF was mostly identical. The state transition function
$f\left(X_{k}^{(i)},U_{k}\right)$ and measurement function $H$ were designed
in line with the selected constant velocity model [46], and the time delta
$dt$ was adjusted to an optimal value of 0.01. The design of the code
interfaced with the previous YOLO algorithm, post-training, to detect the
bounding box for an image before passing that image through both a linear KF
and UKF algorithm to predict the state of the satellite in the following
image. This was then repeated with each image in sequence, before mean squared
error (MSE) and root mean squared error (RMSE) between the predicted state and
the detected bounding box were calculated for each algorithm, thus allowing
for the performance over that set of images to be compared and contrasted. The
creation and displaying of images was handled with the cv2 and matplotlib
packages [9, 20] and the generated images, containing both bounding boxes,
were also saved to a data-specific folder for visual inspection.

##  4 Method

###  4.1 YOLO Pre-Training

The first task of the current study was to train the YOLOv3 algorithm to
correctly detect and classify the eleven categories of satellite experienced
within the SPARK dataset. Stream-1 images were split into training and
validation sets at a ratio of 4:1, resulting in n=17600 images with associated
annotations in the train set and n=4400 images with annotations in the
validation set. After building Darknet within Colab, the train command was
invoked, setting the parameters to match the prepared .data and .cfg files,
setting the initial weights to the downloaded darknet53.conv.74 weights file.
The YOLOv3 algorithm was trained for one epoch of n=22000 iterations within
Colab using an NVIDIA Tesla V100-SXM2-16GB GPU. Following successful
completion of training, the map command was invoked using the generated
weights file. Finally, a set of varied test images of the 11 classes, taken
from other SPARK Stream-1 data not used in the training, was uploaded and the
test command was invoked on each to visually gauge performance of the YOLO.

###  4.2 CosmosDSR Implementation

With the successful training of the YOLOv3 algorithm, the generated weights
file would be used for all Darknet initialisations going forward. The next aim
of the research was to implement a UKF to track the position of RSOs between
consecutive frames of the Stream-2 data. The UKF code was executed on one of
the previously elected Stream-2 sets of consecutive images. For each image,
Darknet was used to generate a bounding box prediction using the trained
weights file, before storing this information to a file. The UKF code then
read and parsed this information to give definitive bounding box coordinates,
which would be displayed in green on the image. The state estimation from the
UKF was displayed as a red box on the image, and this was saved to a nominal
folder in sequence. Following the application of the UKF to all images in the
set, this was repeated with the linear KF, generating another folder of output
images. After both algorithms had been applied, the MSE and RMSE for both was
calculated. The MSE can be thought to represent the squared difference between
the KF estimated and YOLO predicted values, an important metric for algorithm
cross-evaluation due to its inherent method of penalising larger errors more
than smaller errors [19]. However, this is hard to intuitively interpret and,
as such, computing the RMSE, representing the average prediction error in
terms of pixels [1], allows the results to be both understandable and
relatable. This process was then repeated with further Stream-2 image sets.

##  5 Results

###  5.1 YOLOv3 Classification

Following training, the YOLOv3 algorithm returned a Mean Average Precision
(mAP) score, a current benchmark metric used within CV research to establish
the performance of a model with regards to making accurate predictions [44],
of 97.18% at a standard Intersection over Union (IoU) threshold of 0.5 across
the validation set. The average IoU value was 83.03%, indicating accurate
object localisation. Furthermore, an F1-score, the harmonic mean of precision
and recall scores [18], of 0.95 was achieved. Overall, there were n=4163 true
positives, n=209 false positives and n=237 false negatives across all classes.
Class-wise prediction statistics are shown below in Table 2.

Table 2: YOLOv3 class-wise prediction statistics after one epoch Class ID | Class Name | Average Precision (AP%) | True Positives | False Positives
---|---|---|---|---
0 | lisa_pathfinder | 99.94 | 361 | 2
1 | proba_data3_csc | 98.30 | 370 | 3
2 | smart_1 | 96.66 | 365 | 22
3 | xmm_newton | 94.55 | 375 | 58
4 | soho | 96.49 | 400 | 8
5 | earth_observation_sat_1 | 90.02 | 369 | 28
6 | debris | 99.26 | 421 | 4
7 | proba_2 | 98.81 | 364 | 9
8 | proba_3_ocs | 98.06 | 403 | 66
9 | cheops | 98.56 | 374 | 5
10 | double_star | 98.31 | 361 | 4

The observed results indicate a high accuracy across all classes, suggesting
excellent performance of the YOLOv3 algorithm on the contemporary SPARK data.
The models adeptness at classifying RSO’s, as well as localising them
correctly within an image, suggests robust detection capabilities and
signifies that the first aim of the current research was a success. An example
of visual output, where YOLO correctly predicts distinct RSO classes with high
certainty, is shown in Figure 4.

Fig. 4: YOLOv3 test output of Stream-1 images

###  5.2 CosmosDSR Performance

On the initial Stream-2 validation test, the YOLO performed well, once again
predicting the correct class with 99% certainty, as can be seen in Figure 5.

Fig. 5: YOLOv3 test output of a Stream-2 image

This suggests the algorithm was able to generalise relatively well and further
stands to uphold the success of its implementation. The UKF code was then
executed on each of the Stream-2 sets of images, results of which are shown
below in Table 3.

Table 3: Evaluation of the UKF/KF on Stream-2 image sets (* at a value lower than 2dp) Data | UKF MSE | UKF RMSE | LKF MSE | LKF RMSE | Best Performance
---|---|---|---|---|---
GT086 | 1.88 | 1.37 | 1.88 | 1.37 | UKF*
GT066 | 1.86 | 1.37 | 1.86 | 1.37 | UKF*
GT011 | 4.15 | 2.04 | 4.17 | 2.04 | UKF
GT053 | 3.55 | 1.88 | 3.55 | 1.88 | LKF*
GT047 | 2.73 | 1.65 | 2.74 | 1.65 | UKF
Mean $(\mu)$ | 2.83 | 1.66 | 2.84 | 1.66 | UKF

The results imply that the UKF performed well when applied to track the SPARK
RSOs across consecutive Stream-2 images. Overall, the results indicate that
the UKF did outperform the linear KF for four image sets, however the linear
KF also performed well, and the disparity was minimal. Computed RMSE results
for all experiments, as shown in Table 3., were low $(\mu=1.66)$ with
differences generally being minute. Considering the 440x1080 resolution of the
Stream-2 images, the results were sufficient to conclude that in the current
research, the UKF algorithm was able to accurately track a satellite; and when
combined with the SPARK-trained YOLOv3 into a novel algorithm, function as a
relatively strong object detection and tracking model. Although the sample
size of image sets was small, the results across each of the tests are
sufficiently congruent to theorise that further application to similar data
would result in similar results. A panel, showing the first sixteen generated
images of the UKF execution on the GT086 dataset is shown below in Figure 6.
This is followed by a similar panel of images from the linear KF execution on
the same data in Figure 7. The predicted bounding box from the YOLO is shown
in green and the object state estimation from the UKF/LKF is shown in red.

Fig. 6: The first 16 generated images from the UKF application on the GT086
image set. Green bounding box represents YOLO prediction, red bounding box
represents UKF estimation. Fig. 7: The first 16 generated images from the LKF
application on the GT086 image set. Green bounding box represents YOLO
prediction, red bounding box represents UKF estimation.

Upon reviewing the output images from both the UKF and LKF, apart from some
small disparity, it is hard to visually distinguish between the two. Both
algorithms exhibited good performance in tracking the satellites and this
mirrors the achieved MSE/RMSE results. Visually, the KF state estimate
bounding boxes mostly overlap the predicted OD bounding boxes, suggesting the
filters’ predictions strongly align with the YOLO detections and signifying
both the consistency and reliability of the combined approaches. Overall,
despite such similarities, and being cognisant of the limited number of
experiments completed, the UKF did outperform the linear KF with regards to
SPARK RSO tracking in consecutive images.

##  6 Discussion

The end performance of the algorithm is indicative that hyperparameter values
were appropriately selected. It may be of some use for future studies to
include extra variables into the tests such as dropout, batch size,
subdivisions or the number of epochs. As the code is already prepared,
including these would need minimal further work and should be considered in
any resulting future research, as it would also further increase the
efficiency of the final algorithm.

The performance of the YOLOv3 itself was also exemplary, with results
suggesting the algorithm could accurately classify all eleven categories of
satellite/RSO correctly with minimal issues such as false positives or
negatives. A paper by Razavian et al. [48] supports the idea that pre-trained
networks can learn universally useful features, highlighting powerful ‘off-
the-shelf’ features from networks trained on ImageNet. In the current study,
the pre-trained weights file seemed to be a good starting point for the SPARK-
focused training, with the favourable results suggesting that the model
converged quickly; this in turn may suggest that transfer learning may have
occurred. Furthermore, it can be inferred that the SPARK data itself helped to
facilitate the training performance as it is a well-established and well-
developed dataset that has already been successfully applied to ML tasks [41].
The clarity of the data may have led to the certainty of the YOLO when
predicting images with values of >0.99%, which is rarely observed. This can
possibly be attributed to the clear categorisation of the data, as well as the
fact it has been synthetically generated and as such, has minimal levels of
noise and occlusion. Such drawbacks of YOLO, as outlined in the previous
literature [57, 8], were therefore not sufficiently tested. This is something
that must be explored in future research; according to Zheng et al. [66],
astronomical images have higher proportions of noise, large dynamic ranges of
intensities and often unclear boundaries when compared to ordinary ‘non-Space’
images. Undoubtedly, the organic conditions of Space are also complex and in
many cases hostile, making clear imaging of satellites in any way that closely
resembles the SPARK impractical, which may somewhat explain the lack of easily
retrievable real images observed when selecting the data for the current
project. Therefore, although the YOLO’s performance on the test set of the
SPARK images was good, applying this to real scenarios may not be
straightforward. Seemakurthy et al. [53] explain that although algorithms may
perform well on similar data, problems with generalisation, the application of
a specific model to a wider problem domain, remain a concern. With regards to
RSO classification, an exceedingly complex and ever-changing problem that
contains far more classes and designations of objects than can be expressed by
even a comprehensive synthetic dataset such as the SPARK, a lack of
generalisation quickly negates the validity of the approach. However, the
ability of the algorithm to accurately predict the satellite in the Stream-2
images, despite the resolution of the image increasing and the object size
being mostly vastly different from the Stream-1 images used for training,
suggests the YOLO was at least able to generalise to different conditions even
if the subject remained the same. Future research could apply the weights
generated by the current study to real images of space, or another synthetic
dataset, as to assess the model’s ability to generalise to the broader domain
of RSO classification. Comparing the performance to that of the method
outlined by Fitzgerald [15], the current YOLOv3 resulted in an F1-score of
0.95, which is larger than the F1-score of 0.81 observed by the author’s
detection algorithm on their main dataset. Additionally, the current study’s
algorithm performed as good or better when compared to all tested classical
algorithms – SIFT, HOGOR and FF – across all data subsets, suggesting at least
comparable if not better performance of the current YOLOv3 model. Again, it is
important to note that the current research focused on a non-identical,
simpler problem with fewer experiments and as such, any comparison can only be
thought of in rough terms despite regarding a similar domain. Nevertheless,
the obtained results suggest the current YOLOv3 architecture is an appropriate
algorithm for RSO identification and classification, offering a new angle to
approach the problem and establishing a base for further development.

The second stage of the project, integrating the trained YOLO into a novel
solution for RSO tracking through the use of UKF, was also completed with
relative success. Building from the future research by Fitzgerald [15] and
Barreiros et al. [8], in which linear Kalman filters were combined with YOLO
algorithms for such purposes, the current research elected to use UKF. This
mediated the evaluation of another algorithm, whilst allowing for both
internal and external comparisons through equating performance to both the
previous literature and a concurrently developed LKF in the current study.
Following the design of the code, the performance of both the UKF and LKF on
the five sets of Stream-2 data was gathered, with results indicating good
performance of both algorithms on each dataset. Across the five sets, the
average RMSE for the UKF was 1.66 and the average RMSE for the LKF was also
1.66. The differences between the results for both algorithms, both
numerically and visually, were extremely small and indicated respectable
object tracking of the RSO across the consecutive images. Furthermore, when
the model was unable to make a prediction with YOLO due to uncertainty or loss
of object, the KF still accurately estimated the position across images,
correctly indicating that a satellite is in the area. The accuracy of such
predictions is reflected in the RMSE – by trait a measure susceptible to
skewed values due to small amounts of high-value outliers [11] – thus even a
single bad prediction could drastically increase the error. In this vein, RMSE
can be thought of as a robust measure of object tracker performance
comparisons that is used in many studies within the literature [65, 61, 55],
thus a low value for both UKF and LKF signifies not only good but,
comparatively, analogous performance. With regards to the main, internal
comparison between the two algorithms, for the current application, either
would be a suitable choice. However, the UKF overall did slightly outperform
the LKF on the test sets, and although such disparity is minute, in a complex
domain such as orbital surveillance even small increases in performance or
efficiency may be important. This comparison should be expanded on by future
research into the area as to test the algorithms’ performances on larger
samples of similar yet more problematic tasks, to ascertain how the disparity
changes when some non-linearity is introduced. The current study focused on a
relatively simple, linear problem and as such, and as shown by the results,
the LKF would have been sufficient. However, as explained by Kaineg [24] and
substantiated by Chen [13], the problem of orbital debris is multifaceted,
everchanging and extremely challenging, thus encountering non-linearity is
almost certain whilst attempting to mitigate it. In such cases, the advantages
of using the UKF would be much more apparent, and good performance on the
linear problem within the current research implies that the algorithm would be
well suited to the task. The results stand to uphold the sentiments made by
the authors of the previous literature; that adopting deep-learned detection
and tracking algorithms are extremely beneficial for improving our
understanding and proficiency with object identification. The YOLOv5/LKF
combination applied by Fitzgerald [15] generated results that significantly
improved on previous attempts, with the author describing the research as a
success and Barreiros et al. [8]’s application of a YOLOv2/LKF combination
proposed a similarly effective method for a distinctly unique problem. As the
YOLOv3/UKF architecture utilised in the current study was designed to build
upon such research, testing a new combination of algorithms, the favourable
results suggest it too was a success. Obviously, as with the previous
literature, the research was not without its limitations.

One such limitation may arise from the lack of ability to validly evaluate.
Whilst a comparison can be made to the previous studies, in the sense that all
surrounded a novel application of an object detection and tracking algorithm
to a problem with a high rate of success, making a direct quantitative
assessment is not possible as all propose a different evaluation metric. As
previously stated, the current study used MSE/RMSE to evaluate performance due
to benefits of robustness, outlier detection and simplicity of internal
comparisons. Fitzgerald [15], however, evaluated tracking performance using
solely F1-score. This has the benefit of balancing recall and precision
scores, which is crucial for balancing tracked objects and minimising missed
predictions within object tracking tasks [54]. However, F1-score treats each
image or frame independently and does not give a good representation of the
temporal continuity of the tracking, thus it does not measure the fluidity of
the algorithm’s application across data and may disregard crucial information
about its performance. As such, researchers often provide multiple metrics to
bypass such issues, such as offering F1-score alongside RMSE [12]. Future
research should endeavour to calculate and present sufficient metrics to
evaluate performance from all angles. [8] did not provide either score; due to
the uniqueness of the problem, the authors offered bespoke metrics regarding
the number of frames tracked correctly for a single target and the probability
for correct target re-identification following occlusion. As such, no
quantitative comparison is possible. Another limitation may arise from the
Stream-2 data used for the object tracking. Whilst the SPARK dataset as a
whole has proved a good asset for the research, helping to correctly train the
YOLO on eleven distinct classes contained within the Stream-1 data, the sets
of images used for the KF all contained the same satellite, proba2, indicating
class imbalance. Problems with imbalance within object detection applications
are already well documented [43, 22], such as reduced versatility and limited
generalisation which negatively impacts the applicability of the model. Future
research must build on this by applying the novel model not only to real data
as to ensure it translates correctly [34], but also to Stream-2 sets of other
classes as to assess KF performance on different satellites. However, as
detection accuracies were high for all classes, there is no reason to assume
that the KF algorithms could not accurately track any of them with similar
success to its performance detecting proba2.

##  7 Conclusion

In conclusion, the current study aimed to design a novel object detection and
tracking model, comprising a YOLOv3 object detector trained on SPARK RSO image
data combined with a UKF algorithm to facilitate object tracking across
consecutive images, as to offer a solution to automatically track orbital
debris. Results indicated that the YOLOv3 was able to accurately detect and
classify satellites into eleven distinct categories found within the SPARK,
with minimal errors such as false positives or negatives. Furthermore, the UKF
was able to successfully track a satellite throughout sets of consecutive
images with good performance. This was directly compared to an LKF, which also
performed well, with results being extremely close and UKF displaying only a
slight improvement. Interpreting the results suggests the study, based on its
aims, was a success and the novel YOLOv3/UKF algorithm developed can be
considered an appropriate solution for RSO detection and tracking. However,
future research should attempt to gather non-synthetic images and apply the
model to a larger sample of tasks, as well as diversifying the subject of
images used within the KF predictions. Additionally, all future research
should aim to give a more comprehensive array of performance evaluation
metrics as to be able to more easily externally validify the results with
previous literature. Thought must be given to incorporating the model with
debris mitigation strategies. Finally, the issues of occlusion and noise need
to be investigated as to further assess their significance on performance in a
domain where they are regularly encountered. Accomplishing the above would
supplement the findings of the current study, increase the knowledge of the
research area and ultimately move towards finding a solution to the Kessler
syndrome, cleaning up the stellar environment and facilitating future space
exploration.

## Acknowledgments

Thanks to Dr. Djamila Aouada and the SPARK Organisation Team at the University
of Luxembourg, specifically Dr. Arunkumar Rathinam for facilitating access to
the SPARK dataset and mediating further queries relating to its usage.

Further thanks to Dr. Vijayan Asari of the University of Ohio for providing
information surrounding the previous research.

The data that support the findings of this study are available from the
corresponding author upon reasonable request.

## References

  * Ajala et al. [2022] Ajala, S., Muraleedharan Jalajamony, H., Nair, M. et al. (2022).  Comparing machine learning and deep learning regression frameworks for accurate prediction of dielectrophoretic force.  Scientific Reports, 12. doi:10.1038/s41598-022-16114-5.
  * AlDahoul et al. [2022] AlDahoul, N., Karim, H. A., & Momo, M. A. (2022).  Rgb-d based multi-modal deep learning for spacecraft and debris recognition.  Scientific Reports, 12. doi:10.1038/s41598-022-07846-5.
  * Aldughayfiq et al. [2023] Aldughayfiq, B., Ashfaq, F., Jhanjhi, N. Z. et al. (2023).  Yolov5-fpn: A robust framework for multi-sized cell counting in fluorescence images.  Diagnostics, 13, 2280. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10341068/. doi:10.3390/diagnostics13132280.
  * Alexey [2020] Alexey (2020).  Alexeyab/darknet.  URL: https://github.com/AlexeyAB/darknet.
  * Allworth et al. [2021] Allworth, J., Windrim, L., Bennett, J. et al. (2021).  A transfer learning approach to space debris classification using observational light curve data.  Acta Astronautica, 181, 301–315. doi:10.1016/j.actaastro.2021.01.048.
  * Auger et al. [2013] Auger, F., Hilairet, M., Guerrero, J. M. et al. (2013).  Industrial applications of the kalman filter: A review.  IEEE Transactions on Industrial Electronics, 60, 5458–5471. doi:10.1109/tie.2012.2236994.
  * Baisa [2020] Baisa, N. (2020).  Derivation of a constant velocity motion model for visual tracking.  URL: https://arxiv.org/pdf/2005.00844.pdf.
  * Barreiros et al. [2021] Barreiros, M. d. O., Dantas, D. d. O., Silva, L. C. d. O. et al. (2021).  Zebrafish tracking using yolov2 and kalman filter.  Scientific Reports, 11, 3219. URL: https://www.nature.com/articles/s41598-021-81997-9. doi:10.1038/s41598-021-81997-9.
  * Bradski [2020] Bradski, G. (2020).  The opencv library. dr. dobb’s journal of software tools, 120; 122-125. - references - scientific research publishing.  URL: https://www.scirp.org/(S(351jmbntvnsjt1aadkposzje))/reference/ReferencesPapers.aspx?ReferenceID=1692176.
  * Carranza-García et al. [2020] Carranza-García, M., Torres-Mateo, J., Lara-Benítez, P. et al. (2020).  On the performance of one-stage and two-stage object detectors in autonomous vehicles using camera data.  Remote Sensing, 13, 89. doi:10.3390/rs13010089.
  * Chai & Draxler [2014] Chai, T., & Draxler, R. R. (2014).  Root mean square error (rmse) or mean absolute error (mae)? – arguments against avoiding rmse in the literature.  Geoscientific Model Development, 7, 1247–1250. doi:10.5194/gmd-7-1247-2014.
  * Chen et al. [2021] Chen, J., Ding, G., Yang, Y. et al. (2021).  Dual-modality vehicle anomaly detection via bilateral trajectory tracing.  URL: https://arxiv.org/abs/2106.05003. doi:10.48550/arXiv.2106.05003.
  * Chen [2011] Chen, S. (2011).  The space debris problem.  Asian Perspective, 35, 537–558. URL: https://www.jstor.org/stable/42704771.
  * Earman & Friedman [1973] Earman, J., & Friedman, M. (1973).  The meaning and status of newton’s law of inertia and the nature of gravitational forces.  Philosophy of Science, 40, 329–359. doi:10.1086/288536.
  * Fitzgerald [2022] Fitzgerald, G. (2022).  Space object detection and monitoring using persistent wide field of view camera arrays.  URL: https://etd.ohiolink.edu/acprod/odb_etd/ws/send_file/send?accession=dayton1651754893298694&disposition=inline.
  * Formentin & Bittanti [2014] Formentin, S., & Bittanti, S. (2014).  An insight into noise covariance estimation for kalman filter design.  IFAC Proceedings Volumes, 47, 2358–2363. doi:10.3182/20140824-6-za-1003.01611.
  * [17] Furfaro, R., Linares, R., & Reddy, V. ().  Space debris identification and characterization via deep meta-learning.  URL: https://www.hou.usra.edu/meetings/orbitaldebris2019/orbital2019paper/pdf/6123.pdf.
  * Hicks et al. [2022] Hicks, S. A., Strümke, I., Thambawita, V. et al. (2022).  On evaluation metrics for medical applications of artificial intelligence.  Scientific Reports, 12, 5979. URL: https://www.nature.com/articles/s41598-022-09954-8. doi:10.1038/s41598-022-09954-8.
  * Hodson et al. [2021] Hodson, T. O., Over, T. M., & Foks, S. S. (2021).  Mean squared error, deconstructed.  Journal of Advances in Modeling Earth Systems, 13. doi:10.1029/2021ms002681.
  * Hunter [2007] Hunter, J. D. (2007).  Matplotlib: A 2d graphics environment.  Computing in Science & Engineering, 9, 90–95. URL: https://ieeexplore.ieee.org/document/4160265. doi:10.1109/mcse.2007.55.
  * Jahirabadkar et al. [2020] Jahirabadkar, S., Narsay, P., Pharande, S. et al. (2020).  Space objects classification techniques: A survey.  URL: https://ieeexplore.ieee.org/abstract/document/9199996. doi:10.1109/ComPE49325.2020.9199996.
  * Johnson & Khoshgoftaar [2019] Johnson, J. M., & Khoshgoftaar, T. M. (2019).  Survey on deep learning with class imbalance.  Journal of Big Data, 6. doi:10.1186/s40537-019-0192-5.
  * Julier & Uhlmann [2004] Julier, S., & Uhlmann, J. (2004).  Unscented filtering and nonlinear estimation.  Proceedings of the IEEE, 92, 401–422. doi:10.1109/jproc.2003.823141.
  * Kaineg [2020] Kaineg, S. (2020).  The growing problem of space debris the growing problem of space debris.  Hastings Environmental Law Journal Hastings Environmental Law Journal, 26. URL: https://repository.uchastings.edu/cgi/viewcontent.cgi?article=1588&context=hastings_environmental_law_journal.
  * Kalman [1960] Kalman, R. E. (1960).  A new approach to linear filtering and prediction problems.  Journal of Basic Engineering, 82, 35. URL: https://pdfs.semanticscholar.org/bb55/c1c619c30f939fc792b049172926a4a0c0f7.pdf?_ga=2.33129974.916463580.1563730984-1622409074.1563730984. doi:10.1115/1.3662552.
  * Kessler & Cour-Palais [1978] Kessler, D. J., & Cour-Palais, B. G. (1978).  Collision frequency of artificial satellites: The creation of a debris belt.  Journal of Geophysical Research, 83, 2637. doi:10.1029/ja083ia06p02637.
  * Kim & Bang [2018] Kim, Y., & Bang, H. (2018).  Introduction to Kalman Filter and Its Applications.  IntechOpen.  URL: https://www.intechopen.com/chapters/63164.
  * Krauss & Stilwell [2022] Krauss, S. T., & Stilwell, D. J. (2022).  Unscented kalman filtering on manifolds for auv navigation – experimental results.  URL: https://arxiv.org/abs/2210.06510. doi:10.48550/arXiv.2210.06510.
  * Labbe [2023a] Labbe, R. (2023a).  Filterpy — filterpy 1.4.4 documentation.  URL: https://filterpy.readthedocs.io/en/latest/.
  * Labbe [2023b] Labbe, R. (2023b).  rlabbe/filterpy.  URL: https://github.com/rlabbe/filterpy.
  * Liashchynskyi & Liashchynskyi [2019] Liashchynskyi, P., & Liashchynskyi, P. (2019).  Grid search, random search, genetic algorithm: A big comparison for nas.  arXiv:1912.06059 [cs, stat], . URL: https://arxiv.org/abs/1912.06059.
  * Liou & Johnson [2006] Liou, J.-C., & Johnson, N. L. (2006).  Risks in space from orbiting debris.  Science, 311, 340–341. doi:10.1126/science.1121337.
  * Liu et al. [2019] Liu, L., Ouyang, W., Wang, X. et al. (2019).  Deep learning for generic object detection: A survey.  International Journal of Computer Vision, 128, 261–318. doi:10.1007/s11263-019-01247-4.
  * Ljungqvist et al. [2023] Ljungqvist, M. G., Nordander, O., Skans, M. et al. (2023).  Object detector differences when using synthetic and real training data.  SN computer science, 4. doi:10.1007/s42979-023-01704-5.
  * Ma et al. [2011] Ma, S., Gong, Q., & Zhang, J. (2011).  Space target recognition based on 2-d wavelet transformation and kpca.  URL: https://ieeexplore.ieee.org/document/6014322. doi:10.1109/ICCSN.2011.6014322.
  * Mahendrakar et al. [2021] Mahendrakar, T., White, R., Wilde, M. et al. (2021).  Real-time satellite component recognition with yolo-v5.  Small Satellite Conference, . URL: https://digitalcommons.usu.edu/smallsat/2021/all2021/51/.
  * Martinelli & Rhoads [2016] Martinelli, R., & Rhoads, N. (2016).  Predicting market data using the kalman filter.  Stocks & Commodities V, 28, 1–44. URL: http://in.ruc.edu.cn/wp-content/uploads/2016/09/Predicting-Market-Data-Using-The-Kalman-Filter.pdf.
  * Mastrofini et al. [2023] Mastrofini, M., G, G., Agostinelli, I. et al. (2023).  Yolo v4 based algorithm for resident space object detection and tracking.  Studies in computational intelligence, (pp. 19–33). doi:10.1007/978-3-031-25755-1_2.
  * Mehrholz et al. [2002] Mehrholz, D., Leushacke, L., Flury, W. et al. (2002).  Detecting, tracking and imaging space debris.  URL: https://www.esa.int/esapub/bulletin/bullet109/chapter16_bul109.pdf.
  * Mostafa & Ghantous [2022] Mostafa, M., & Ghantous, M. (2022).  A yolo based approach for traffic light recognition for adas systems.  URL: https://ieeexplore.ieee.org/document/9781682. doi:10.1109/MIUCC55081.2022.9781682.
  * Musallam et al. [2021] Musallam, M., Al Ismaeil, K., Oyedotun, O. et al. (2021).  Spark: Spacecraft recognition leveraging knowledge of space environment.  URL: https://arxiv.org/pdf/2104.05978.pdf.
  * NASA [2019] NASA (2019).  Ares — orbital debris program office.  URL: https://orbitaldebris.jsc.nasa.gov.
  * Oksuz et al. [2020] Oksuz, K., Cam, B. C., Kalkan, S. et al. (2020).  Imbalance problems in object detection: A review.  IEEE Transactions on Pattern Analysis and Machine Intelligence, (pp. 1–1). doi:10.1109/tpami.2020.2981890.
  * Padilla et al. [2021] Padilla, R., Passos, W. L., Dias, T. L. B. et al. (2021).  A comparative analysis of object detection metrics with a companion open-source toolkit.  Electronics, 10, 279. doi:10.3390/electronics10030279.
  * Perez et al. [2021] Perez, M., Musallam, M., Henaff, P. et al. (2021).  Detection & identification of on-orbit objects using machine learning.  URL: https://conference.sdo.esoc.esa.int/proceedings/sdc8/paper/258/SDC8-paper258.pdf.
  * Popoli & Blackman [1999] Popoli, R., & Blackman, S. (1999).  Design and analysis of modern tracking systems. artech house, norwood, ma. - references - scientific research publishing.  URL: https://www.scirp.org/(S(351jmbntvnsjt1aadkozje))/reference/referencespapers.aspx?referenceid=2390079.
  * Rankin [2023] Rankin, J. (2023).  Kalman filtering approach to market price forecasting.  URL: https://core.ac.uk/download/pdf/38899158.pdf.
  * Razavian et al. [2014] Razavian, A. S., Azizpour, H., Sullivan, J. et al. (2014).  Cnn features off-the-shelf: an astounding baseline for recognition.  arXiv:1403.6382 [cs], . URL: https://arxiv.org/abs/1403.6382.
  * Redmon et al. [2016] Redmon, J., Divvala, S., Girshick, R. et al. (2016).  You only look once: Unified, real-time object detection.  URL: https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Redmon_You_Only_Look_CVPR_2016_paper.pdf.
  * Redmon & Farhadi [2016] Redmon, J., & Farhadi, A. (2016).  Yolo9000: Better, faster, stronger.  URL: https://arxiv.org/abs/1612.08242.
  * Redmon & Farhadi [2018] Redmon, J., & Farhadi, A. (2018).  Yolov3: An incremental improvement.  URL: https://arxiv.org/abs/1804.02767.
  * Ronghui-Zhan. & Wan [2006] Ronghui-Zhan., & Wan, J. (2006).  Neural network-aided adaptive unscented kalman filter for nonlinear state estimation.  IEEE Signal Processing Letters, 13, 445–448. doi:10.1109/lsp.2006.871854.
  * Seemakurthy et al. [2022] Seemakurthy, K., Fox, C., Aptoula, E. et al. (2022).  Domain generalisation for object detection.  arXiv:2203.05294 [cs], . URL: https://arxiv.org/abs/2203.05294.
  * Song et al. [2022] Song, Y., Hu, Z., Li, T. et al. (2022).  Performance evaluation metrics and approaches for target tracking: A survey.  Sensors, 22, 793. doi:10.3390/s22030793.
  * Sunindyo & Satria [2020] Sunindyo, W. D., & Satria, A. S. M. (2020).  Traffic congestion prediction using multi-layer perceptrons and long short-term memory.  URL: https://ieeexplore.ieee.org/document/9263483. doi:10.1109/EECCIS49483.2020.9263483.
  * Tahir et al. [2022] Tahir, A., Munawar, H. S., Akram, J. et al. (2022).  Automatic target detection from satellite imagery using machine learning.  Sensors, 22, 1147. doi:10.3390/s22031147.
  * Terven & Cordova-Esparaza [2023] Terven, J., & Cordova-Esparaza, D. (2023).  A comprehensive review of yolo: From yolov1 to yolov8 and beyond under review in acm computing surveys.  URL: https://arxiv.org/pdf/2304.00501.pdf.
  * Urrea & Agramonte [2021] Urrea, C., & Agramonte, R. (2021).  Kalman filter: Historical overview and review of its use in robotics 60 years after its creation.  URL: https://www.hindawi.com/journals/js/2021/9674015/.
  * Vallado & Finkleman [2014] Vallado, D. A., & Finkleman, D. (2014).  A critical assessment of satellite drag and atmospheric density modeling.  Acta Astronautica, 95, 141–165. doi:10.1016/j.actaastro.2013.10.005.
  * Van Der Merwe & Wan [2003] Van Der Merwe, R., & Wan, E. (2003).  Sigma-point kalman filters for probabilistic inference in dynamic state-space models.  URL: https://www.gatsby.ucl.ac.uk/~byron/nlds/merwe2003a.pdf.
  * Wahn et al. [2023] Wahn, B., König, P., & Kingstone, A. (2023).  Predicting group benefits in joint multiple object tracking.  Attention, perception & psychophysics, . doi:10.3758/s13414-023-02693-6.
  * Zhafri et al. [2017] Zhafri, N., Nasir, M., Aizzat Zakaria, M. et al. (2017).  Autonomous mobile robot localization using kalman filter.  URL: https://www.matec-conferences.org/articles/matecconf/pdf/2017/04/matecconf_aigev2017_01069.pdf.
  * Zhang et al. [2022a] Zhang, W., Huang, H., Sun, Y. et al. (2022a).  Agripest-yolo: A rapid light-trap agricultural pest detection method based on deep learning.  Frontiers in Plant Science, 13. URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9800973/. doi:10.3389/fpls.2022.1079384.
  * Zhang et al. [2022b] Zhang, Z., Deng, C., & Deng, Z. (2022b).  A diverse space target dataset with multidebris and realistic on-orbit environment.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 9102–9114. URL: https://ieeexplore.ieee.org/iel7/4609443/4609444/09878111.pdf. doi:10.1109/JSTARS.2022.3203042.
  * [65] Zhao, C., Mei, P., Xu, S. et al. ().  Performance evaluation of visual object detection and tracking algorithms used in remote photoplethysmography.  URL: https://openaccess.thecvf.com/content_ICCVW_2019/papers/CVPM/Zhao_Performance_Evaluation_of_Visual_Object_Detection_and_Tracking_Algorithms_Used_ICCVW_2019_paper.pdf.
  * Zheng et al. [2015] Zheng, C., Pulido, J., Thorman, P. et al. (2015).  An improved method for object detection in astronomical images.  Monthly Notices of the Royal Astronomical Society, 451, 4445–4459. doi:10.1093/mnras/stv1237.