# Weighted Twisted Inhomogeneous Diophantine Approximation

Mumtaz Hussain  Mumtaz Hussain, Department of Mathematical and Physical
Sciences, La Trobe University, Bendigo 3552, Australia.
<EMAIL_ADDRESS>and  Benjamin Ward  Ben Ward, Department of
Mathematical and Physical Sciences, La Trobe University, Bendigo 3552,
Australia<EMAIL_ADDRESS>
###### Abstract.

We prove a multidimensional weighted analogue of the well-known theorem of
Kurzweil (1955) in the metric theory of inhomogeneous Diophantine
approximation. Let $\sum_{i=1}^{m}\alpha_{i}=~{}m$ and
$|\cdot|_{\alpha}=\max_{1\leq i\leq m}|\cdot|^{1/\alpha_{i}}$. Given an
$n$-tuple of monotonically decreasing univariable functions
$\Psi=~{}(\psi_{1},\dots,\psi_{n})$ with
$\psi_{i}:{\mathbb{R}}_{+}\to{\mathbb{R}}_{+}$ such that each $\psi_{i}(r)\to
0$ as $r\to\infty$ and fixed $A\in{\mathbb{R}}^{n\times m}$ define

$W_{A}(\Psi):=\left\\{\mathbf{b}\in[0,1]^{n}:\begin{array}[]{l}|A_{i}\cdot\textbf{q}-b_{i}-p_{i}|<\psi_{i}(|\textbf{q}|_{\alpha})\quad(1\leq
i\leq n),\\\\[4.30554pt] \text{for infinitely many
}(\mathbf{p},\textbf{q})\in{\mathbb{Z}}^{n}\times({\mathbb{Z}}^{m}\backslash\\{0\\})\end{array}\right\\}.$

We prove that the set $W_{A}(\Psi)$ has zero-full Lebesgue measure under
convergent-divergent sum conditions with some mild assumptions on $A$ and the
approximating functions $\Psi$. We also prove the Hausdorff dimension results
for this set. Along with some geometric arguments, the main ingredients are
the weighted ubiquity and weighted mass transference principle introduced
recently by Kleinbock & Wang [Adv. Math. 428 (2023), Paper No. 109154], and
Wang & Wu [Math. Ann. 381 (2021), no. 1-2, 243–317] respectively.

##  1\. Introduction

For a fixed $\xi\in{\mathbb{R}}$ consider the sequence $(\\{\xi
q\\})_{q\in{\mathbb{N}}}$, where $\\{x\\}$ denotes the fractional part of
$x\in{\mathbb{R}}$. When $\xi\in{\mathbb{Q}}$ the sequence is periodic, but
for $\xi\in{\mathbb{R}}\backslash{\mathbb{Q}}$ the sequence is dense on the
unit interval. In 1901 Minkowski proved that for any irrational
$\xi\in{\mathbb{R}}$ the inequality

$|\xi q-b-p|<\frac{1}{4q}$

has infinitely many solutions $(q,p)\in{\mathbb{N}}\times{\mathbb{Z}}$ for any
$b\not\in{\mathbb{Z}}+\xi{\mathbb{Z}}$ [19]. The constant $\frac{1}{4}$ was
later made optimal by Khintchine [12]. Note that rather than the classical
setting of Diophantine approximation where one approximates the space $[0,1]$
by rational points $\tfrac{p}{q}\in{\mathbb{Q}}$, here we are approximating
the space $[0,1]$ by the points $(\\{\xi q\\})_{q\in{\mathbb{N}}}$, hence the
name “twisted” inhomogeneous Diophantine approximation. Naturally, this
requires knowledge of the distribution of the points $(\\{\xi
q\\})_{q\in{\mathbb{N}}}$, and so the choice of $\xi\in{\mathbb{R}}$ heavily
affects that rate of approximation.

A natural question to ask is how many $b\in{\mathbb{R}}$ satisfy the above
inequality when the right hand side is replaced by some general decreasing
function $\psi:{\mathbb{N}}\to{\mathbb{R}}_{+}$. That is, how large is the set

$W_{\xi}(\psi):=\left\\{b\in[0,1]:|\xi q-b-p|<\psi(q)\,\,\text{ for infinitely
many}\,\,(q,p)\in{\mathbb{N}}\times{\mathbb{Z}}\right\\}.$

In 1955, Kurzweil made the following contribution to answering the above
question. Recall that $\xi$ is badly approximable if there exists a constant
$c(\xi)>0$ such that

$|\xi q-p|\geq\frac{c(\xi)}{q}\quad\text{ for all
}(q,p)\in{\mathbb{N}}\times{\mathbb{Z}}.$

Throughout, let $\lambda_{d}(A)$ denotes the $d$-dimensional Lebesgue measure
of a set $A\subseteq{\mathbb{R}}^{d}$.

######  Theorem ([17]).

Let $\psi:{\mathbb{N}}\to{\mathbb{R}}_{+}$ be a non-increasing function and
$\xi\in{\mathbb{R}}\backslash{\mathbb{Q}}$. Then,
$\lambda_{1}(W_{\xi}(\psi))\in\\{0,1\\}$. Furthermore, for $\xi$ badly
approximable

$\lambda_{1}(W_{\xi}(\psi))=\begin{cases}0\quad\text{\rm if
}\quad\sum\limits_{r\in{\mathbb{N}}}\psi(r)<\infty\,,\\\\[8.61108pt]
1\quad\text{\rm if
}\quad\sum\limits_{r\in{\mathbb{N}}}\psi(r)=\infty\,.\end{cases}$

Heuristically, it makes sense to consider $\xi$ badly approximable. Note that,
by the Three Distance Theorem (see for example [25]), the sequence $(\\{\xi
q\\})_{q\in{\mathbb{N}}}$ does not cluster in any particular region of
$[0,1]$, and so are reasonably good at approximating the unit interval.
Theorem Theorem has since been refined to apply to more general
$\xi\in{\mathbb{R}}$. In particular, Fuchs and Kim improved on the statement
by considering the principle convergents of $\xi$, see [9, Theorem 1.2] and
[22] for further details and contributions to this theory.

When the Lebesgue measure of the set is null, for example, all functions of
the form $\psi_{\tau}(q)=q^{-\tau}$ with $\tau>1$ have
$\lambda_{1}(W_{\xi}(\psi_{\tau}))=0$, one can ask for a refined statement to
distinguish between the null sets. In this regard, Bugeaud [6, Theorem 1], and
Schmeling and Troubetzkoy [23, Theorem 3.2] independently proved the following
result.

######  Theorem ([6, 23]).

For any $\xi\in{\mathbb{R}}\backslash{\mathbb{Q}}$ and $\tau>1$

$\operatorname{\dim_{H}}W_{\xi}(\psi_{\tau})=\frac{1}{\tau},$

where $\operatorname{\dim_{H}}$ denotes the Hausdorff dimension.

We should remark that one of the first results in this direction was proven by
Bernik and Dodson [5, p.105] who proved the above result for
$\lambda_{1}$-almost all $\xi\in[0,1]$. Furthermore, the above theorem has
since been generalised to a range of more general approximation functions
$\psi$ [8, Theorem 3] and a restricted set of $\xi\in{\mathbb{R}}$, where
$\tau$ in the dimension is replaced by the lower order at infinity of the
function $\psi$. Perhaps more importantly, it was also shown that there exists
$\xi\in{\mathbb{R}}$ where the expected dimension result is false [8,
Theorem2].

###  1.1. Higher dimensional twisted inhomogeneous approximation

The above setup can be readily extended to higher dimensions. Throughout
suppose $v=(v_{1},\dots,v_{n})\in~{}{\mathbb{R}}^{n}_{+}$ and
$\alpha=(\alpha_{1},\dots,\alpha_{m})\in{\mathbb{R}}^{m}_{+}$ are vectors
satisfying

$\sum_{i=1}^{n}v_{i}=n\,,\quad\sum_{i=1}^{m}\alpha_{i}=m,$

and let

$|\cdot|_{v}=\max_{1\leq i\leq
n}|\cdot|^{1/v_{i}}\,,\quad|\cdot|_{\alpha}=\max_{1\leq i\leq
m}|\cdot|^{1/\alpha_{i}}\,.$

Let ${\mathbb{R}}^{n\times m}$ denote the set of $n\times m$ matrices with
real number entries and fix some $A\in{\mathbb{R}}^{n\times m}$. Given an
$n$-tuple of monotonic decreasing functions $\Psi=(\psi_{1},\dots,\psi_{n})$
with $\psi_{i}:{\mathbb{R}}_{+}\to{\mathbb{R}}_{+}$, we say
$\mathbf{b}=(b_{1},\dots,b_{n})\in[0,1]^{n}$ is $\Psi$-approximable for $A$ if
there exists infinitely many
$(\textbf{q},\mathbf{p})=(q_{1},\dots,q_{m},p_{1},\dots,p_{n})\in({\mathbb{Z}}^{m}\backslash\\{0\\})\times{\mathbb{Z}}^{n}$
solving

(1.1)
$|A_{i}\cdot\textbf{q}-b_{i}-p_{i}|<\psi_{i}(|\textbf{q}|_{\alpha})\quad(1\leq
i\leq n),$

where $A_{i}$ denotes the $i$th row of $A$. Denote by $W_{A}(\Psi)$ the set of
such $\Psi$-approximable vectors for $A$, that is

$W_{A}(\Psi):=\left\\{\mathbf{b}\in[0,1]^{n}:\text{ \eqref{deftwisted} is
solved for infinitely many
}(\textbf{q},\mathbf{p})\in({\mathbb{Z}}^{m}\backslash\\{0\\})\times{\mathbb{Z}}^{n}\right\\}.$

One should note that the one dimensional Lebesgue measure results presented
above, that is the theorems of Kurzweil, Fuchs and Kim, rely in parts on the
theory of continued fractions, and so higher dimensional analogues do not
readily follow. Similarly, the dimension theory result of Bugeaud, Schmeling,
and Troubetzkoy, uses the Three Distance Theorem, again a result strongest in
the one dimensional setting.

In generalising the technique presented in [15] to the weighted ubiquitous
setting we are able to prove higher dimensional weighted analogues of the
classical results. In order to state our results we need the following
definitions.

We say, $A\in{\mathbb{R}}^{n\times m}$ is $(v,\alpha)$-singular if for any
$\varepsilon>0$ and for all sufficiently large $N\geq 1$, there exists
$(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}$ solving the inequalities

(1.2) $\begin{cases}|A_{i}\cdot\textbf{q}-p_{i}|<\varepsilon
N^{-v_{i}\frac{m}{n}}\quad(1\leq i\leq n),\\\
0<|\textbf{q}|_{\alpha}<N\,.\end{cases}$

Let $Sing_{\alpha}(v)$ denote the set of all $(v,\alpha)$-singular matrices.
That is,

$Sing_{\alpha}(v)=\left\\{A\in{\mathbb{R}}^{n\times
m}:\lim_{N\to\infty}\left(N\min_{0<|\textbf{q}|_{\alpha}<N}\max_{1\leq i\leq
n}|A_{i}\cdot\textbf{q}-p_{i}|^{\frac{n}{mv_{i}}}\right)=0\right\\}.$

We also define the set of $(v,\alpha)$-badly approximable points as

$\mathbf{Bad}_{\alpha}(v):=\left\\{A\in{\mathbb{R}}^{n\times
m}:\liminf_{|\textbf{q}|_{\alpha}\to\infty}\left(\max_{1\leq i\leq
n}|\textbf{q}|_{\alpha}|A_{i}\cdot\textbf{q}-p_{i}|^{\frac{n}{mv_{i}}}\right)>0\right\\}.$

Lastly, define the set

$L_{\alpha}(v,A,\varepsilon):=\left\\{\ell\in{\mathbb{N}}:\begin{cases}|A_{i}\cdot\textbf{q}-p_{i}|<\varepsilon
2^{-\ell v_{i}\frac{m}{n}}\quad(1\leq i\leq n),\\\
0<|\textbf{q}|_{\alpha}<2^{\ell}\end{cases}\text{ has no solution
}(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}\right\\}.$

Given the above definitions, we are able to state our results.

######  Theorem 1.

Let $A\in Sing_{\alpha}(v)^{c}$ and let $\Psi=(\psi_{1},\dots,\psi_{n})$ be an
$n$-tuple of monotonic approximation functions with each

$\psi_{i}(r)\ll r^{-v_{i}\frac{m}{n}}\quad(1\leq i\leq n)$

for all $r\in{\mathbb{R}}_{+}$ with the implied constants independent of $r$.
Then

$\lambda_{n}\left(W_{A}(\Psi)\right)=1\quad\text{\rm if }\sum_{\ell\in
L_{\alpha}(v,A,\varepsilon)}2^{m\ell}\prod_{i=1}^{n}\psi_{i}\left(2^{\ell}\right)=\infty.$

######  Corollary 1.

Let $A\in Bad_{\alpha}(v)$ and let $\Psi=(\psi_{1},\dots,\psi_{n})$ be an
$n$-tuple of monotonic approximation functions with each

$\psi_{i}(r)\ll r^{-v_{i}\frac{m}{n}}\quad(1\leq i\leq n)$

for all $r\in{\mathbb{R}}_{+}$ with the implied constants independent of $r$.
Then

$\lambda_{n}\left(W_{A}(\Psi)\right)=\begin{cases}0\quad\text{\rm if
}\sum\limits_{r\in{\mathbb{N}}}r^{m-1}\prod_{i=1}^{n}\psi_{i}\left(r\right)<\infty\,,\\\\[8.61108pt]
1\quad\text{\rm if
}\sum\limits_{r\in{\mathbb{N}}}r^{m-1}\prod_{i=1}^{n}\psi_{i}\left(r\right)=\infty\,.\end{cases}$

###### Proof.

This easily follows from Theorem 1 on the observation that
$\mathbf{Bad}_{\alpha}(v)\subset Sing_{\alpha}(v)^{c}$ and that
$L_{\alpha}(v,A,\varepsilon)\supset{\mathbb{N}}_{\geq k}$ for some
$\varepsilon>0$ and $k\in{\mathbb{N}}$ when $A\in\mathbf{Bad}_{\alpha}(v)$
(see Lemma 2 in §2). The convergence case will be proven at the end of §3.1. ∎

In [15] Kim proves the analogue of Theorem 1 in the case where
$\alpha=(1,\dots,1)$, $v=(1,\dots,1)$ and $\Psi=(\psi,\dots,\psi)$. It is from
this paper that we draw our inspirations to prove our weighted analogues, see
[15, Theorem 1.3, Corollary 1.6] for more details. It should be noted, in
particular, that Kim’s version of our result is in terms of the Hausdorff
$s$-measure for $0\leq s\leq n$, and so, in particular, they prove that, for
$A\in\mathbf{Bad}_{(1,\dots,1)}(1,\dots,1)$ and
$\Psi(q)=(q^{-\tau},\dots,q^{-\tau})$ with $\tau>\frac{m}{n}$, that

$\operatorname{\dim_{H}}W_{A}(\Psi)=\frac{m}{\tau}.$

Given our setup, we are able to prove the weighted analogue of their result as
follows.

######  Theorem 2.

Let $v=(1,\dots,1)$. Let $A\in Bad_{\alpha}(v)$ and let
$\Psi=(\psi_{1},\dots,\psi_{n})$ be an $n$-tuple of functions with each

$\psi_{i}(r)=r^{-\tau_{i}}\quad\text{ and
}\quad\tau_{i}>\frac{m}{n}\quad(1\leq i\leq n).$

Then,

$\operatorname{\dim_{H}}W_{A}(\Psi)=\min_{1\leq i\leq
n}\left\\{\frac{m+\sum_{i:\tau_{j}>\tau_{i}}(\tau_{j}-\tau_{i})}{\tau_{j}}\right\\}=s.$

Furthermore, for any ball $B\subset[0,1]^{n}$ we have that

${\mathcal{H}}^{s}\left(B\cap W_{A}(\Psi)\right)=\infty\,.$

######  Theorem 3.

Fix $n=2$ and let $v=(v_{1},v_{2})$. Let $A\in Bad_{\alpha}(v)$ and let
$\Psi=(\psi_{1},\psi_{2})$ be a pair of functions with each

$\psi_{i}(r)=r^{-\tau_{i}}\,,\quad\text{ with }\quad\tau_{i}\geq
v_{i}\frac{m}{2}\quad(i=1,2).$

Furthermore, suppose that

(1.3)
$\min\left\\{\frac{\min\\{v_{1},v_{2}\\}m}{\min\\{\tau_{1},\tau_{2}\\}n},\frac{\min\\{v_{1},v_{2}\\}}{\max\\{v_{1},v_{2}\\}}\right\\}\geq\frac{m-\min\\{\tau_{1},\tau_{2}\\}}{\max\\{\tau_{1},\tau_{2}\\}}\,.$

Then,

$\operatorname{\dim_{H}}W_{A}(\Psi)=\frac{m+\max\\{\tau_{1},\tau_{2}\\}-\min\\{\tau_{1},\tau_{2}\\}}{\max\\{\tau_{1},\tau_{2}\\}}.$

######  Remark 1.

The difference between Theorem 2 and Theorem 3 is subtle. When each of the
approximation functions $\psi_{i}$ decreases at a faster rate than the
Dirichlet exponent of approximation (in this setting the exponent is
$\frac{m}{n}$) then Theorem 2 is optimal. However, when this is not the case,
Theorem 3 is required.

Condition (1.3) seems unnecessary but it is required for our method to work in
the lower bound. In particular, observe that if
$\min\\{\tau_{1},\tau_{2}\\}\geq m$, that is the approximation functions are
decreasing fast enough, then (1.3) is automatically satisfied.

It should be noted that a set of particular interest within the setting of
twisted inhomogeneous approximation is the set of $\xi$-badly approximable
points, which are generally the set of points $b\in[0,1]$ such that (1)
becomes false when the right hand side is multiplied by some arbitrarily small
constant. In this article, we do not consider such a set (although we can
deduce a metric result on weighted higher dimensional analogue of $\xi$-badly
approximable points from Theorem 1). For more details on the metric properties
of these sets, we refer the reader to [1, 10, 11, 13, 14, 21, 24, 18] and
references within. This is a particularly active area of research, as far as
we are aware there are at least two forthcoming papers in this area of
research. In [3] the metric results of higher dimensional analogues of
$\xi$-badly approximable points are studied in detail and generalised to the
$S$-arithmetic setting, and in [20] the measure of $\xi$-badly approximable
points when shifted by some constant is proven to be null for any algebraic
measure on the $m$-dimensional torus (see [18, Theorem 1.6]). In both these
settings $\xi$ satisfies certain properties.

The rest of the paper is laid out as follows. In the next section, we recall
and prove a few basic properties on the set of $(v,\alpha)$-non-singular
matrices. We also recall the setup for weighted ubiquitous systems as
introduced in [16, 26]. Lastly, in §3, we give the proofs of our main results.

Acknowledgments: The research of both authors is supported by the Australian
Research Council discovery project 200100994. We would also like to thank
Victor Beresnevich for many useful comments on an earlier draft.

##  2\. Preliminaries and Auxiliary Results

###  2.1. Properties of singular and badly approximable matrices

The following observation was made in [15] in the case $v=(1,\dots,1)$ and
$\alpha=(1,\dots,1)$. As we will require such an observation in our results we
state and prove the following easy lemma.

######  Lemma 1.

$A\in Sing_{\alpha}(v)$ if and only if for any $\varepsilon>0$ the set

$L_{\alpha}(v,A,\varepsilon):=\left\\{\ell\in{\mathbb{N}}:\begin{cases}|A_{i}\cdot\textbf{q}-p_{i}|<\varepsilon
2^{-\ell v_{i}\frac{m}{n}}\quad(1\leq i\leq n),\\\
0<|\textbf{q}|_{\alpha}<2^{\ell}\end{cases}\text{\rm has no solution
}(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}\right\\}$

is finite.

######  Remark 2.

Observe that this result naturally implies that if $A$ is $(v,\alpha)$\- non-
singular, that is, $A\in Sing_{\alpha}(v)^{c}={\mathbb{R}}^{n\times
m}\backslash Sing_{\alpha}(v)$, then $L_{\alpha}(v,A,\varepsilon)$ is
unbounded. In particular this means the summation appearing in Theorem 1 is
infinite.

###### Proof.

The forward implication that $A\in Sing_{\alpha}(v)$ implies
$L_{\alpha}(v,A,\varepsilon)$ is immediate by the definition and setting
$N=2^{\ell}$. To see the reverse implication note that if
$L_{\alpha}\left(v,A,\tfrac{\varepsilon}{2^{\max v_{i}}}\right)$ is finite,
say $L_{\alpha}\left(v,A,\tfrac{\varepsilon}{2^{\max
v_{i}}}\right)\subset\\{1,\dots,k\\}$, then for all $\ell>k$ and any
$2^{\ell}\leq N<2^{\ell+1}$ we have that

$\begin{cases}|A_{i}\cdot\textbf{q}-p_{i}|<\frac{\varepsilon}{2^{\frac{m}{n}\max
v_{i}}}2^{-\ell v_{i}\frac{m}{n}}\leq\varepsilon
2^{-(\ell+1)v_{i}\frac{m}{n}}<\varepsilon N^{-v_{i}\frac{m}{n}}\quad(1\leq
i\leq n),\\\ 0<|\textbf{q}|_{\alpha}<2^{\ell}\leq N\end{cases}$

has solution $(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}$. Since this is
true for any choice of $\varepsilon>0$ we have that $A\in Sing_{\alpha}(v)$. ∎

As stated in [15] in the case of $v=(1,\dots,1)$ and $\alpha=(1,\dots,1)$ we
have the following result.

######  Lemma 2.

$A\in\mathbf{Bad}_{\alpha}(v)$ if and only if for some $\varepsilon>0$ there
exists $k\in{\mathbb{N}}$ such that
$L_{\alpha}(v,A,\varepsilon)\supset{\mathbb{N}}_{\geq k}$.

###### Proof.

If $A\in\mathbf{Bad}_{\alpha}(v)$ then there exists some $c(A)>0$ such that

$\max_{1\leq i\leq
n}|A_{i}\cdot\textbf{q}-p_{i}|^{1/v_{i}}>c(A)|\textbf{q}|_{\alpha}^{-\frac{m}{n}}$

for all sufficiently large $\textbf{q}\in{\mathbb{Z}}^{m}\backslash\\{0\\}$.
Without loss of generality assume this is true for all
$\textbf{q}\in{\mathbb{Z}}^{m}$ such that $|\textbf{q}|_{\alpha}\geq 2^{t}$.
Hence for any $\ell>t$ and any $|\textbf{q}|_{\alpha}<2^{\ell}$ we have that

$\max_{1\leq i\leq
n}|A_{i}\cdot\textbf{q}-p_{i}|^{1/v_{i}}>c(A)|\textbf{q}|_{\alpha}^{-\frac{m}{n}}>c(A)2^{-\ell\frac{m}{n}}.$

Thus $L_{\alpha}\left(v,A,c(A)^{\min v_{i}}\right)\supset{\mathbb{N}}_{>t}$.

For the reverse implication, we have that for all $\ell>k\in{\mathbb{N}}$

$\begin{cases}\max_{1\leq i\leq
n}|A_{i}\cdot\textbf{q}-p_{i}|^{1/v_{i}}>\varepsilon
2^{-\ell\frac{m}{n}}\,,\\\ 0<|\textbf{q}|_{\alpha}<2^{\ell}\,.\end{cases}$

Hence for all sufficiently large $\textbf{q}\in{\mathbb{Z}}^{m}$ (all
$\textbf{q}\in{\mathbb{Z}}^{m}$ such that $|\textbf{q}|_{\alpha}>2^{k}$) there
exists $\ell>k$ such that $2^{\ell}<|\textbf{q}|_{\alpha}\leq 2^{\ell+1}$ so
that

$\begin{cases}\max_{1\leq i\leq
n}|A_{i}\cdot\textbf{q}-p_{i}|^{1/v_{i}}>\varepsilon
2^{-(\ell+1)\frac{m}{n}}=\frac{\varepsilon}{2^{\frac{m}{n}}}2^{-\ell\frac{m}{n}}>\frac{\varepsilon}{2^{\frac{m}{n}}}|\textbf{q}|_{\alpha}^{-\frac{m}{n}},\\\
\text{ for all }0<|\textbf{q}|_{\alpha}<2^{\ell+1}\,.\end{cases}$

This is true for all $\ell>k$, hence $A\in\mathbf{Bad}_{\alpha}(v)$. ∎

###  2.2. A Dirichlet-type theorem in the case of non-singular matrices

We need the following weighted analogue of [7, Chapter V, Theorem VI], which
can readily be proven from [7, Chapter V, Theorem V]. For completeness, we
prove the result here.

######  Lemma 3.

Let $A\in{\mathbb{R}}^{nm}$ and suppose there are no integer solutions
$(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}\backslash\\{0\\}$ to

$\left\\{\begin{array}[]{c}|A\textbf{q}-\mathbf{p}|_{v}<C,\\\
|\textbf{q}|_{\alpha}<N,\end{array}\right.\,$

for norms $|\cdot|_{v}=\max_{1\leq i\leq n}|\cdot|^{1/v_{i}}$,
$|\cdot|_{\alpha}=\max_{1\leq i\leq m}|\cdot|^{1/\alpha_{i}}$ and vectors
$v=(v_{1},\dots,v_{n})\in{\mathbb{R}}^{n}_{+}$ and
$\alpha=(\alpha_{1},\dots,\alpha_{m})\in{\mathbb{R}}^{m}_{+}$ satisfying

$\sum_{i=1}^{n}v_{i}=n\quad\text{ and }\quad\sum_{i=1}^{m}\alpha_{i}=m\,.$

Then for any $\textbf{b}\in{\mathbb{R}}^{n}$ there exists
$(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}$ solving

$\left\\{\begin{array}[]{c}|A\textbf{q}-\textbf{b}-\mathbf{p}|_{v}\leq
c_{1}C,\\\ |\textbf{q}|_{\alpha}<c_{1}N,\end{array}\right.$

for constant

$c_{1}=\underset{1\leq j\leq n}{\max_{1\leq i\leq
m}}\left\\{\left(\frac{1}{2}\left(C^{-n}N^{-m}+1\right)\right)^{1/v_{i}},\left(\frac{1}{2}\left(C^{-n}N^{-m}+1\right)\right)^{1/\alpha_{j}}\right\\}.$

###### Proof.

Consider the system of inequalities on $n+m$ variables
$(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}$

$\left\\{\begin{array}[]{c}|C^{-v_{i}}(A_{i}\cdot\textbf{q}-p_{i})|<1,\quad(1\leq
i\leq n)\\\ |N^{-\alpha_{j}}q_{j}|<1,\quad(1\leq j\leq m).\end{array}\right.$

For ease of notation let
$f_{i}(\textbf{q},\mathbf{p})=C^{-v_{i}}(A_{i}\cdot\textbf{q}-p_{i})$ for
$1\leq i\leq n$ and $f_{i+m}(\textbf{q},\mathbf{p})=N^{-\alpha_{j}}q_{j}$ for
$1\leq i\leq m$. Then (3) is equivalent to

(2.1) $\max_{1\leq i\leq n+m}|f_{i}(\textbf{q},\mathbf{p})|<1\,.$

By assumption, (2.1) has no integer solutions
$(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}\backslash\\{0\\}$. Furthermore,
observe that the $(n+m)\times(n+m)$ matrix associated to the system of linear
forms (2.1) has determinant

$C^{-\sum_{i=1}^{n}v_{i}}N^{-\sum_{i=1}^{m}\alpha_{i}}=C^{-n}N^{-m},$

and so by [7, Theorem V] for any real number
$\textbf{b}^{*}\in{\mathbb{R}}^{n+m}$ there are integer solutions to

(2.2) $\max_{1\leq i\leq
n+m}|f_{i}(\textbf{q},\mathbf{p})-b^{*}_{i}|<\frac{1}{2}(C^{-n}N^{-m}+1)\,.$

Setting
$\textbf{b}^{*}=(C^{-v_{1}}b_{1},\dots,C^{-v_{n}}b_{n},0,\dots,0)\in{\mathbb{R}}^{n+m}$
in (2.2) gives us that that the system of inequalities

$\left\\{\begin{array}[]{c}|A_{i}\cdot\textbf{q}-b_{i}-p_{i}|\leq\frac{1}{2}(C^{-n}N^{-m}+1)C^{v_{i}},\quad(1\leq
i\leq n)\\\ |q_{i}|<\frac{1}{2}(C^{-n}N^{-m}+1)N^{\alpha_{i}},\quad(1\leq
i\leq m)\end{array}\right.$

has integer solutions
$(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{n+m}\backslash\\{0\\}$ for any
$\textbf{b}\in{\mathbb{R}}^{n}$. Rearranging in terms of the norms
$|\cdot|_{v}$ and $|\cdot|_{\alpha}$ completes the proof. ∎

For Lemma 3 we can easily deduce the following corollary

######  Corollary 2.

Let $A\in Sing_{\alpha}(v)^{c}=[0,1]^{nm}\backslash Sing_{\alpha}(v)$. Then
for any $\mathbf{b}\in{\mathbb{R}}^{n}$ there exists some $\varepsilon>0$ such
that for all $\ell\in L_{\alpha}(v,A,\varepsilon)$ the system of inequalities

$\begin{cases}|A_{i}\cdot\textbf{q}-p_{i}-b_{i}|<\varepsilon c_{2}2^{-\ell
v_{i}\frac{m}{n}}\quad(1\leq i\leq n),\\\
|\textbf{q}|_{\alpha}<c_{2}2^{\ell}\end{cases}$

has integer solution $(\textbf{q},\mathbf{p})\in{\mathbb{Z}}^{m+n}$ for
$c_{2}=\left(\frac{1}{2}(\varepsilon^{-n}+1)\right)^{\frac{1}{\min_{i,j}\\{v_{i},\alpha_{j}\\}}}$.

###### Proof.

Note that since $A\in Sing_{\alpha}(v)^{c}$, there exists $\varepsilon>0$ such
that the set $L_{\alpha}(v,A,\varepsilon)$ has infinite cardinality by Lemma
1. Now take $C=\varepsilon 2^{-\ell\frac{m}{n}}$ and $N=2^{\ell}$ as in Lemma
3. ∎

Observe that Corollary 2 implies that $A\textbf{q}({\rm mod}1)$ is dense in
$[0,1]^{m}$, and so, by Kronecker’s Theorem (see for example [7, Chapter III
Theorem IV]), the subgroup
$G(^{t}A):=^{t}A{\mathbb{Z}}^{m}+{\mathbb{Z}}^{n}\subset{\mathbb{R}}^{n}$ has
maximal rank $n+m$ over ${\mathbb{Z}}$. This allows us to use the following
result.

######  Lemma 4 ([15, Proposition 3.5]).

Suppose $A\in Sing_{\alpha}(v)^{c}$. Then for any ball $B\subset[0,1]^{n}$

$\frac{\\#\\{A\textbf{q}\in B:|\textbf{q}|_{\alpha}\leq
N\\}}{\\#\\{\textbf{q}\in{\mathbb{Z}}^{n}:|\textbf{q}|_{\alpha}\leq
N\\}}\to\lambda_{n}(B)\quad\text{ as }N\to\infty\,.$

The proof of this Lemma follows in exactly the same way as [15, Proposition
3.5], the only difference being in the latter stages of the proof, where the
summation on $q_{1}$ ranges over $-N^{\alpha_{1}}$ to $N^{\alpha_{1}}$ and is
averaged over $N^{\alpha_{1}}$.

###  2.3. Weighted ubiquitous systems

In this section we give the definition of local ubiquity for rectangles as
given in [16]. This definition is a generalisation of ubiquity for rectangles
as found in [26], which is in turn a generalisation of local ubiquity for
balls introduced in [4]. For brevity, we will state the results of [16, 26] in
the special setting of $n$-dimensional real space. We will also assume that
each resonant set, see definition below, is a finite collection of points and
so we can omit the notion of $\kappa$-scaling. See [16, 26] for the full
statements.

Consider the product space $({\mathbb{R}}^{n},|\cdot|_{\infty},\lambda_{n})$,
where $|\cdot|_{\infty}=\max_{1\leq i\leq n}|\cdot|$. For any
$x\in{\mathbb{R}}^{n}$ and $r\in{\mathbb{R}}_{+}$ define the open ball

$B(x,r)=\left\\{y\in{\mathbb{R}}^{n}:|x-y|_{\infty}<r\right\\}=\prod_{i=1}^{n}B_{1}(x_{i},r),$

where $B_{1}$ are the usual open intervals with centre $x_{i}$ and diameter
$2r$ in ${\mathbb{R}}$. Let $J$ be a countably infinite index set, and
$\beta:J\to{\mathbb{R}}_{+}$, $\alpha\mapsto\beta_{\alpha}$ a positive
function satisfying the condition that for any $N\in{\mathbb{N}}$

$\\#\left\\{\alpha\in J:\beta_{\alpha}<N\right\\}<\infty.$

Let $l_{k},u_{k}$ be two sequences in ${\mathbb{R}}_{+}$ such that $u_{k}\geq
l_{k}$ with $l_{k}\to\infty$ as $k\to\infty$. Define

$J_{k}=\\{\alpha\in J:l_{k}\leq\beta_{\alpha}\leq u_{k}\\}.$

Let $\rho=(\rho_{1},\dots,\rho_{n})$ be an $n$-tuple of non-increasing
functions $\rho_{i}:{\mathbb{R}}_{+}\to{\mathbb{R}}_{+}$ such that each
$\rho_{i}(x)\to 0$ as $x\to\infty$. For each $1\leq i\leq n$, let
$(R_{\alpha,i})_{\alpha\in J}$ be a sequence of finite collections of points
in ${\mathbb{R}}$. The family of sets $(R_{\alpha})_{\alpha\in J}$ where

$R_{\alpha}=\prod_{i=1}^{n}R_{\alpha,i},$

for each $\alpha\in J$, are called resonant sets.

Define

$\Delta(R_{\alpha},\rho(r))=\prod_{i=1}^{n}\Delta_{i}(R_{\alpha,i},\rho_{i}(r)),$

where for some set $Y\subset{\mathbb{R}}$ and $b\in{\mathbb{R}}_{+}$

$\Delta_{i}(Y,b)=\bigcup_{a\in Y}B_{1}(a,b)$

is the union of balls in ${\mathbb{R}}$ of radius $b$ centred at all possible
points in $Y$.

The following notion of ubiquity for rectangles can be found in [16].

######  Definition 1 (Local ubiquitous system of rectangles).

Call the pair $\big{(}(R_{\alpha})_{\alpha\in J},\beta\big{)}$ a local
ubiquitous system of rectangles with respect to $\rho$ if there exists a
constant $c>0$ such that for any ball $B\subset X$ and all sufficiently large
$k\in{\mathbb{N}}$

$\lambda_{n}\left(B\cap\bigcup_{\alpha\in
J_{k}}\Delta(R_{\alpha},\rho(u_{k}))\right)\geq cm(B).$

For $n$-tuple of approximation functions $\Psi=(\psi_{1},\dots,\psi_{n})$ with
each $\psi_{i}:{\mathbb{R}}_{+}\to{\mathbb{R}}_{+}$ define

$W(\Psi)=\left\\{x\in{\mathbb{R}}^{n}:x\in\Delta\left(R_{\alpha},\Psi(\beta_{\alpha})\right)\,\text{
for infinitely many }\alpha\in J\right\\}.$

The following theorem, due to Kleinbock and Wang [16], provides the Lebesgue
measure theory on $W(\Psi)$.

######  Theorem 4.

Let $0<c<1$. A function $f$ is said to be $c$-regular with respect to a
sequence $\\{r_{i}\\}_{i\in{\mathbb{N}}}$ if $f(r_{i+1})\leq cf(r_{i})$ for
all sufficiently large $i$. Let $W(\Psi)$ be defined as above and assume that
$(\\{R_{\alpha}\\}_{\alpha\in J},\beta)$ is a local ubiquitous systems of
rectangles with respect to $\rho$. Suppose that

  1. (I)

each $\psi_{i}$ is decreasing,

  2. (II)

for each $1\leq i\leq n$, $\psi_{i}(r)\leq\rho_{i}(r)$ for all
$r\in{\mathbb{R}}_{+}$ and $\rho_{i}(r)\to 0$ as $r\to\infty$,

  3. (III)

either $\rho_{i}$ is $c$-regular on $\\{u_{k}\\}_{k\in{\mathbb{N}}}$ for all
$1\leq i\leq n$ or $\psi_{i}$ is $c$-regular on
$\\{u_{k}\\}_{k\in{\mathbb{N}}}$ for all $1\leq i\leq n$ for some $0<c<1$.

Then

$\lambda_{n}(W(\Psi))=\textit{full}\quad\text{\rm if
}\quad\sum_{k=1}^{\infty}\prod_{i=1}^{n}\left(\frac{\psi_{i}(u_{k})}{\rho_{i}(u_{k})}\right)\,=\infty.$

Here, by full we mean that the complement is a Lebesgue nullset.

For the Hausdorff theory we have the following theorem due to Wang and Wu
[26].

######  Theorem 5.

Let $W(\Psi)$ be defined as above and assume that
$(\\{R_{\alpha}\\}_{\alpha\in J},\beta)$ is a local ubiquitous systems of
rectangles with respect to $\rho=(\rho^{a_{1}},\dots,\rho^{a_{n}})$ for some
function $\rho:{\mathbb{R}}_{+}\to{\mathbb{R}}_{+}$ and
$(a_{1},\dots,a_{n})\in{\mathbb{R}}^{n}_{+}$ with $\rho(N)\to 0$ as
$N\to\infty$. Then, for $\Psi=(\rho^{a_{1}+t_{1}},\dots,\rho^{a_{n}+t_{n}})$
for some $\textbf{t}=(t_{1},\dots,t_{n})\in{\mathbb{R}}^{n}_{+}$

$\operatorname{\dim_{H}}W(\Psi)\geq\min_{A_{i}\in
A}\left\\{\sum_{j\in\mathcal{K}_{1}}1+\sum_{j\in\mathcal{K}_{2}}1+\frac{\sum_{j\in\mathcal{K}_{3}}a_{j}-\sum_{j\in\mathcal{K}_{2}}t_{j}}{A_{i}}\right\\}=s,$

where $A=\\{a_{i},a_{i}+t_{i},1\leq i\leq n\\}$ and
$\mathcal{K}_{1},\mathcal{K}_{2},\mathcal{K}_{3}$ are a partition of
$\\{1,\dots,n\\}$ defined as

$\mathcal{K}_{1}=\\{j:a_{j}\geq
A_{i}\\},\quad\mathcal{K}_{2}=\\{j:a_{j}+t_{j}\leq
A_{i}\\}\backslash\mathcal{K}_{1},\quad\mathcal{K}_{3}=\\{1,\dots
n\\}\backslash(\mathcal{K}_{1}\cup\mathcal{K}_{2}).$

Furthermore, for any ball $B\subset{\mathbb{R}}^{n}$

${\mathcal{H}}^{s}(B\cap W(\Psi))={\mathcal{H}}^{s}(B).$

##  3\. Proof of main results

The main results will be proven by showing that we have a weighted ubiquitous
system for a certain setup. Given this statement, we can use the theorems of
Kleinbock & Wang (Theorem 4), and Wang & Wu (Theorem 5) to obtain the
divergence case of Theorem 1 and the lower bound of Theorem 2. The
corresponding convergence and upper bound statements use standard covering
arguments, but for completeness, we include them in their relevant sections.

###  3.1. Proof of the weighted ubiquity statement

Choose any $A\in Sing_{\alpha}(v)^{c}$ and fix such a choice for the remainder
of this section. Let

$\displaystyle
J=\left\\{\textbf{q}\in{\mathbb{Z}}^{m}\right\\}\quad\beta:J\to{\mathbb{R}}_{+},\textbf{q}\mapsto\beta_{\textbf{q}}=|\textbf{q}|_{\alpha},$
$\displaystyle R_{\textbf{q},i}=\\{A_{i}\cdot\textbf{q}\\}\quad
R_{\textbf{q}}=\\{A\textbf{q}\\}\,,$

where $\\{X\\}$ denotes the vector composing of the fractional part in each
coordinate. Define the sequences

$l_{i}=c_{3}u_{i}\text{ for constant }0<c_{3}<1\text{ dependent on
$\varepsilon$}\,,\quad u_{i}=c_{2}2^{\ell_{i}}\\\ $

where $\varepsilon>0$ is the real number such that the set
$L_{\alpha}(v,A,\varepsilon)$ is infinite, and
$\\{\ell_{i}\\}_{i\in{\mathbb{N}}}$ corresponds to the ordered sequence of
such integers ($c_{2}$ is the constant given in Corollary 2).

######  Proposition 1.

Let each

$\rho_{i}(r)=\varepsilon
c_{2}^{1+v_{i}\frac{m}{n}}r^{-v_{i}\frac{m}{n}}\quad(1\leq i\leq n)\,.$

Then for any ball $B\subset[0,1]^{n}$

$\lambda_{n}\left(B\cap\bigcup_{\textbf{q}\in
J_{k}}\Delta\left(R_{\textbf{q}},\rho(u_{k})\right)\right)\geq\frac{1}{2}\lambda_{n}(B),$

for all sufficiently large $k\in{\mathbb{N}}$.

###### Proof.

Fix $B\subset[0,1]^{n}$. By Corollary 2 we have that for any $\ell_{j}\in
L_{\alpha}(v,A,\varepsilon)$

$\lambda_{n}\left(B\cap\bigcup_{\textbf{q}\in{\mathbb{Z}}^{m}:|\textbf{q}|_{\alpha}\leq
c_{2}2^{\ell_{k}}}\Delta\left(R_{\textbf{q}},\left(\varepsilon
c_{2}2^{-\ell_{k}v_{1}\frac{m}{n}},\dots,\varepsilon
c_{2}2^{-\ell_{k}v_{n}\frac{m}{n}}\right)\right)\right)=\lambda_{n}(B).$

Note that each

$\varepsilon c_{2}2^{-\ell_{k}v_{i}\frac{m}{n}}=\varepsilon
c_{2}^{1+v_{i}\frac{m}{n}}\left(c_{2}2^{\ell_{j}}\right)^{-v_{i}\frac{m}{n}}=\rho_{i}(u_{k})\quad(1\leq
i\leq n).$

This allows us to deduce that for any $k\in{\mathbb{N}}$

$\displaystyle\lambda_{n}(B)$
$\displaystyle\leq\lambda_{n}\left(B\cap\bigcup_{\textbf{q}\in{\mathbb{Z}}^{m}:|\textbf{q}|_{\alpha}\leq
l_{k}}\Delta\left(R_{\textbf{q}},\rho(u_{k})\right)\right)+\lambda_{n}\left(B\cap\bigcup_{\textbf{q}\in{\mathbb{Z}}^{m}:\textbf{q}\in
J_{k}}\Delta\left(R_{\textbf{q}},\rho(u_{k})\right)\right)\,,$

and so the proof is complete in showing that

(3.1)
$\lambda_{n}\left(B\cap\bigcup_{\textbf{q}\in{\mathbb{Z}}^{m}:|\textbf{q}|_{\alpha}\leq
l_{k}}\Delta\left(R_{\textbf{q}},\rho(u_{k})\right)\right)\leq\frac{1}{2}\lambda_{n}(B).$

To show (3.1) we note, by Lemma 4, that there exists sufficiently large
$k_{B}\in{\mathbb{N}}$ such that for all $k>k_{B}$

$\\#\\{A\textbf{q}\in 2B:|\textbf{q}|_{\alpha}\leq l_{k}\\}\leq
2l_{k}^{m}\lambda_{n}(B).$

Hence, taking $k\in{\mathbb{N}}$ sufficiently large such that $\max_{1\leq
i\leq n}\rho_{i}(u_{k})<r(B)$, we have that

$\displaystyle\lambda_{n}\left(B\cap\bigcup_{\textbf{q}\in{\mathbb{Z}}^{m}:|\textbf{q}|_{\alpha}\leq
l_{k}}\Delta\left(R_{\textbf{q}},\rho(u_{k})\right)\right)$ $\displaystyle\leq
2l_{k}^{m}\lambda_{n}(B)2^{n}\prod_{i=1}^{n}\rho_{i}(u_{k})$
$\displaystyle=2^{n+1}\varepsilon^{n}c_{3}^{m}c_{2}^{n+m}\lambda_{n}(B)$
$\displaystyle\leq\frac{1}{2}\lambda_{n}(B),$

where the last inequality follows since we can choose the constant $c_{3}$ so
that

$c_{3}<\left(2^{-(n+2)}\varepsilon^{-n}c_{2}^{-(n+m)}\right)^{1/m}$

independent of $B$ and $k$. ∎

###  3.2. Proof of Theorem 1

To prove Theorem 1 we apply Theorem 4. The bulk of the proof (the weighted
ubiquity statement) has been proven in the previous section. It remains to
verify that the choice of $\rho$ function satisfies the conditions of Theorem
4 and that the summations given in Theorem 4 and Theorem 1 are equivalent.

By the conditions of Theorem 1, each $\psi_{i}$ is monotonically decreasing.
Furthermore each $\rho_{i}(r)\geq\psi_{i}(r)$. Note that each $\psi_{i}$ may
need to be multiplied by some constant to make this strictly true, but as
shown in for example [2, Lemma 5.7], this would not change the Lebesgue
measure of $W_{A}(\Psi)$. Observe that each

$\rho_{i}(u_{k+1})=\varepsilon
c_{2}^{1+v_{i}\frac{m}{n}}u_{k+1}^{-v_{i}\frac{m}{n}}=\varepsilon
c_{2}2^{-\ell_{k+1}v_{i}\frac{m}{n}}\leq\varepsilon
c_{2}2^{-(\ell_{k}+1)v_{i}\frac{m}{n}}=2^{-v_{i}\frac{m}{n}}\rho_{i}(u_{k})\leq
2^{-\frac{m}{n}\min_{i}v_{i}}\rho_{i}(u_{k})$

and so $\rho$ is $2^{-\frac{m}{n}\min_{i}v_{i}}$-regular. Thus, conditions
$(I)-(III)$ of Theorem 4 are satisfied, and so

$\lambda_{n}(W_{A}(\Psi))=1\quad\text{\rm if
}\sum_{k=1}^{\infty}\prod_{i=1}^{n}\left(\frac{\psi_{i}(u_{k})}{\rho_{i}(u_{k})}\right)=\infty.$

Lastly, observe that

$\displaystyle\sum_{k=1}^{\infty}2^{m\ell_{k}}\prod_{i=1}^{n}\psi_{i}(2^{\ell_{k}})$
$\displaystyle=\varepsilon^{n}c_{2}^{n}\sum_{k=1}^{\infty}\varepsilon^{-n}c_{2}^{-(n+m)}(c_{2}2^{\ell_{k}})^{m}\prod_{i=1}^{n}\psi_{i}(2^{\ell_{k}})$
$\displaystyle=\varepsilon^{n}c_{2}^{n}\sum_{k=1}^{\infty}\prod_{i=1}^{n}\frac{\psi_{i}(2^{\ell_{k}})}{\rho_{i}(u_{k})}$
$\displaystyle\geq\varepsilon^{n}c_{2}^{n}\sum_{k=1}^{\infty}\prod_{i=1}^{n}\frac{\psi_{i}(u_{k})}{\rho_{i}(u_{k})}$

where the last line follows by the monotonicity of $\psi_{i}$ and that
$c_{2}\geq 1$. Hence

$\sum_{k=1}^{\infty}2^{m\ell_{k}}\prod_{i=1}^{n}\psi_{i}(2^{\ell_{k}})=\infty$

is sufficient to prove full measure of $W_{A}(\Psi)$.

For the convergence case of Corollary 1, apply the Borel-Cantelli Lemma to see
that

$\lambda_{n}(W_{A}(\Psi))=0\quad\text{\rm if
}\quad\sum_{r\in{\mathbb{N}}}\sum_{2^{r}<|\textbf{q}|_{\alpha}\leq
2^{r+1}}\lambda_{n}\left(\Delta(A\textbf{q},\Psi(|\textbf{q}|_{\alpha}))\right)<\infty.$

Observe that

$\displaystyle\sum_{r\in{\mathbb{N}}}\sum_{2^{r}<|\textbf{q}|_{\alpha}\leq
2^{r+1}}\lambda_{n}\left(\Delta(\\{A\textbf{q}\\},\Psi(|\textbf{q}|_{\alpha}))\right)$
$\displaystyle\leq\sum_{r\in{\mathbb{N}}}c2^{rm}2^{n}\prod_{i=1}^{n}\psi_{i}(2^{r})\,,$
$\displaystyle\leq
c2^{n+1}\sum_{r=1}^{\infty}r^{m-1}\prod_{i=1}^{n}\psi_{i}(r),$

for constant $c=m2^{m+1}3^{m-1}\left(1-2^{-\min_{i}\alpha_{i}}\right)$
independent of $r$, and so the convergence summation condition of Corollary 1
implies zero measure.

###  3.3. Proof of Theorem 2-3

For the upper bound consider the standard cover

$\bigcup_{\textbf{q}\in{\mathbb{Z}}^{m}\backslash\\{0\\}:|\textbf{q}|_{\alpha}\geq
N}\Delta\left(\\{A\textbf{q}\\},\Psi(\textbf{q})\right)$

of $W_{A}(\Psi)$. Observe that each rectangle
$\Delta\left(\\{A\textbf{q}\\},\Psi(\textbf{q})\right)$ can be covered by

$\prod_{i=1}^{n}\max\left\\{1,\frac{|\textbf{q}|_{\alpha}^{-\tau_{i}}}{|\textbf{q}|_{\alpha}^{-\tau_{j}}}\right\\}$

balls of radius $|\textbf{q}|_{\alpha}^{-\tau_{j}}$. Thus

$\displaystyle{\mathcal{H}}^{s}(W_{A}(\Psi))$
$\displaystyle\leq\sum_{\textbf{q}\in{\mathbb{Z}}^{m}:|\textbf{q}|_{\alpha}\geq
N}\prod_{i=1}^{n}\max\left\\{1,\frac{|\textbf{q}|_{\alpha}^{-\tau_{i}}}{|\textbf{q}|_{\alpha}^{-\tau_{j}}}\right\\}|\textbf{q}|_{\alpha}^{-s\tau_{j}}\,,$
$\displaystyle=\sum_{r\in{\mathbb{N}}_{\geq
N}}\,\sum_{\textbf{q}\in{\mathbb{Z}}^{m}:|\textbf{q}|_{\alpha}=r}r^{\sum_{i:\tau_{j}>\tau_{i}}(\tau_{j}-\tau_{i})-s\tau_{j}}\,,$
$\displaystyle\leq 2^{m}\sum_{r\in{\mathbb{N}}_{\geq
N}}\,r^{m-1+\sum_{i:\tau_{j}>\tau_{i}}(\tau_{j}-\tau_{i})-s\tau_{j}}\to 0$

as $N\to\infty$ for any

$s>\frac{m+\sum_{i:\tau_{j}>\tau_{i}}(\tau_{j}-\tau_{i})}{\tau_{j}}.$

This argument is true for any choice of $1\leq j\leq n$, hence

$\operatorname{\dim_{H}}W_{A}(\Psi)\leq\min_{1\leq i\leq
n}\left\\{\frac{m+\sum_{i:\tau_{j}>\tau_{i}}(\tau_{j}-\tau_{i})}{\tau_{j}}\right\\},$

completing the upper bound result.

For the lower bound dimension result we appeal to Theorem 5. Given the setup
provided in Theorem 5 we have that

$a_{i}=\frac{m}{n}\,,\quad t_{i}=\tau_{i}-\frac{m}{n}\quad(1\leq i\leq
n)\,,\quad\rho(r)=\left(\varepsilon
c_{2}^{1+\frac{m}{n}}\right)^{\frac{n}{m}}r^{-1}\,.$

For ease of notation, let

$d(A)=\sum_{j\in\mathcal{K}_{1}}1+\sum_{j\in\mathcal{K}_{2}}1+\frac{\sum_{j\in\mathcal{K}_{3}}a_{j}-\sum_{j\in\mathcal{K}_{2}}t_{j}}{A}$

with each set $\mathcal{K}_{1},\mathcal{K}_{2}$ and $\mathcal{K}_{3}$ defined
by $A$. Consider the two cases:

  * •

$A=\frac{m}{n}$: Then the sets appearing in Theorem 5 are

$\mathcal{K}_{1}=\\{1,\dots,n\\}\,,\quad\mathcal{K}_{2}=\emptyset\,,\quad\mathcal{K}_{3}=\emptyset\,$

and so $d(A)=n$, giving us a trivial lower bound.

  * •

$A=\tau_{j}$ for some $1\leq j\leq n$: Then

$\mathcal{K}_{1}=\emptyset\,,\quad\mathcal{K}_{2}=\\{i:\tau_{i}\leq\tau_{j}\\}\,,\quad\mathcal{K}_{3}=\\{i:\tau_{i}>\tau_{j}\\}=\\{1,\dots,n\\}\backslash\mathcal{K}_{2}\,$

and so

$\displaystyle d(A)$
$\displaystyle=\\#\mathcal{K}_{2}+\frac{\frac{m}{n}(n-\\#\mathcal{K}_{2})-\sum_{i:\tau_{i}\leq\tau_{j}}\left(\tau_{i}-\frac{m}{n}\right)}{A}$
$\displaystyle=\frac{m+(A-\frac{m}{n})\\#\mathcal{K}_{2}-\sum_{i\in\mathcal{K}_{2}}\left(\tau_{i}-\frac{m}{n}\right)}{A}$
$\displaystyle=\frac{m+\sum_{i\in\mathcal{K}_{2}}\left(A-\tau_{i}\right)}{A}\,.$

Hence, replacing $A$ by some $\tau_{j}$ and inputting the definition of
$\mathcal{K}_{2}$ we have that

$d(\tau_{j})=\frac{m+\sum_{i:\tau_{i}\leq\tau_{j}}\left(\tau_{j}-\tau_{i}\right)}{\tau_{j}}$

Taking the minimum over all possible choices of $1\leq j\leq n$ we have the
desired lower bound. The measure result is simply due to the second part of
Theorem 5 and the observation that $s<n$.

For the lower bound of Theorem 3 let

$a_{i}=v_{i}\frac{m}{n}\,,\quad
t_{i}=\tau_{i}-v_{i}\frac{m}{n}\quad(i=1,2)\,\quad\rho(r)=\left(\varepsilon
c_{2}^{1+\frac{m}{n}}\right)^{\frac{n}{m}}r^{-1}\,.$

Without loss of generality we assume $v_{1}<v_{2}$.

  * •

$v_{1}\frac{m}{n}<v_{2}\frac{m}{n}<\tau_{1}<\tau_{2}$: Consider the sets
$\mathcal{K}_{1},\mathcal{K}_{2}$ and $\mathcal{K}_{3}$, and the corresponding
dimension bound for each of the following

    * –

$A=v_{1}\frac{m}{n}$:

$\mathcal{K}_{1}=\\{1,2\\}\,,\quad\mathcal{K}_{2}=\emptyset\,,\quad\mathcal{K}_{3}=\emptyset\,.$

So

$d(A)=2.$

    * –

$A=v_{2}\frac{m}{n}$:

$\mathcal{K}_{1}=\\{2\\}\,,\quad\mathcal{K}_{2}=\emptyset\,,\quad\mathcal{K}_{3}=\\{1\\}\,.$

So

$d(A)=1+\frac{v_{1}\frac{m}{n}}{v_{2}\frac{m}{n}}=1+\frac{v_{1}}{v_{2}}.$

    * –

$A=\tau_{1}$:

$\mathcal{K}_{1}=\emptyset\,,\quad\mathcal{K}_{2}=\\{1\\}\,,\quad\mathcal{K}_{3}=\\{2\\}\,.$

So

$d(A)=1+\frac{v_{1}\frac{m}{n}+v_{2}\frac{m}{n}-\tau_{1}}{\tau_{1}}=1+\frac{m-\tau_{1}}{\tau_{1}}.$

    * –

$A=\tau_{2}$:

$\mathcal{K}_{1}=\emptyset\,,\quad\mathcal{K}_{2}=\\{1,2\\}\,,\quad\mathcal{K}_{3}=\emptyset\,.$

So

$d(A)=2-\frac{\left(\tau_{1}-v_{1}\frac{m}{n}\right)+\left(\tau_{2}-v_{2}\frac{m}{n}\right)}{\tau_{2}}=1+\frac{m-\tau_{1}}{\tau_{2}}.$

Since $\tau_{2}>\tau_{1}$ we have, in this case, that

$\min
d(A)=1+\min\left\\{\frac{m-\tau_{1}}{\tau_{2}},\frac{v_{1}}{v_{2}}\right\\}.$

  * •

$v_{1}\frac{m}{n}<v_{2}\frac{m}{n}<\tau_{2}<\tau_{1}$: This is similar to the
previous case with $\tau_{1}$ and $\tau_{2}$ switching roles. In particular we
get

$\min
d(A)=1+\min\left\\{\frac{v_{1}}{v_{2}},\frac{m-\tau_{2}}{\tau_{1}}\right\\}.$

Combining these cases together, and using the condition that

$\frac{v_{1}}{v_{2}}\geq\frac{m-\min\\{\tau_{1},\tau_{2}\\}}{\max\\{\tau_{1},\tau_{2}\\}}$

we obtain our lower bound. Lastly, consider the following case:

  * •

$v_{1}\frac{m}{n}<\tau_{1}<v_{2}\frac{m}{n}<\tau_{2}$: Consider the sets
$\mathcal{K}_{1},\mathcal{K}_{2}$ and $\mathcal{K}_{3}$, and the corresponding
dimension bound for each of the following

    * –

$A=v_{1}\frac{m}{n}$:

$\mathcal{K}_{1}=\\{1,2\\}\,,\quad\mathcal{K}_{2}=\emptyset\,,\quad\mathcal{K}_{3}=\emptyset\,.$

So

$d(A)=2.$

    * –

$A=v_{2}\frac{m}{n}$:

$\mathcal{K}_{1}=\\{2\\}\,,\quad\mathcal{K}_{2}=\\{1\\}\,,\quad\mathcal{K}_{3}=\emptyset\,.$

So

$d(A)=2-\frac{\tau_{1}-v_{1}\frac{m}{n}}{v_{2}\frac{m}{n}}=1+\frac{v_{1}\frac{m}{n}+v_{2}\frac{m}{n}-\tau_{1}}{v_{2}\frac{m}{n}}=1+\frac{m-\tau_{1}}{v_{2}\frac{m}{n}}.$

    * –

$A=\tau_{1}$:

$\mathcal{K}_{1}=\\{2\\}\,,\quad\mathcal{K}_{2}=\\{1\\}\,,\quad\mathcal{K}_{3}=\emptyset\,.$

So,

$d(A)=2-\frac{\tau_{1}-v_{1}\frac{m}{n}}{\tau_{1}}=1+\frac{v_{1}\frac{m}{n}}{\tau_{1}}.$

    * –

$A=\tau_{2}$:

$\mathcal{K}_{1}=\emptyset\,,\quad\mathcal{K}_{2}=\\{1,2\\}\,,\quad\mathcal{K}_{3}=\emptyset\,.$

So

$d(A)=2-\frac{\left(\tau_{1}-v_{1}\frac{m}{n}\right)+\left(\tau_{2}-v_{2}\frac{m}{n}\right)}{\tau_{2}}=1+\frac{m-\tau_{1}}{\tau_{2}}.$

In this case, noting $\tau_{2}>v_{2}\frac{m}{n}$, we have that

$\min
d(A)=1+\min\left\\{\frac{v_{1}\frac{m}{n}}{\tau_{1}},\frac{m-\tau_{1}}{\tau_{2}}\right\\}.$

Using (1.3), this minimum again becomes our desired lower bound.

## References

  * [1] P Bengoechea and N Moshchevitin.  Badly approximable points in twisted Diophantine approximation and Hausdorff dimension.  Acta Arith., 177(4):301–314, 2017.
  * [2] V. Beresnevich, S. Datta, A. Ghosh, and B. Ward.  Bad is null.  Preprint: arXiv:2307.10109, 2023.
  * [3] V. Beresnevich, S. Datta, A. Ghosh, and B. Ward.  Rectangular shrinking targets for $\mathbb{Z}^{m}$ actions on tori: well and badly approximable systems.  Preprint: arXiv:2307.10122, 2023.
  * [4] Victor Beresnevich, Detta Dickinson, and Sanju Velani.  Measure theoretic laws for lim sup sets.  Mem. Amer. Math. Soc., 179:no. 846, x+91 pp., 2006.
  * [5] V. I. Bernik and M. M. Dodson.  Metric Diophantine approximation on manifolds, volume 137 of Cambridge Tracts in Mathematics.  Cambridge University Press, Cambridge, 1999.
  * [6] Yann Bugeaud.  A note on inhomogeneous Diophantine approximation.  Glasg. Math. J., 45(1):105–110, 2003.
  * [7] John W. S. Cassels.  An introduction to Diophantine approximation.  Cambridge Tracts in Mathematics and Mathematical Physics, No. 45. Cambridge University Press, New York, 1957.
  * [8] Ai-Hua Fan and Jun Wu.  A note on inhomogeneous Diophantine approximation with a general error function.  Glasg. Math. J., 48(2):187–191, 2006.
  * [9] Michael Fuchs and Dong Han Kim.  On Kurzweil’s 0-1 law in inhomogeneous Diophantine approximation.  Acta Arith., 173(1):41–57, 2016.
  * [10] Stephen Harrap.  Twisted inhomogeneous Diophantine approximation and badly approximable sets.  Acta Arith., 151(1):55–82, 2012.
  * [11] Stephen Harrap and Nikolay Moshchevitin.  A note on weighted badly approximable linear forms.  Glasg. Math. J., 59(2):349–357, 2017.
  * [12] A. Khintchine.  Sur le problème de Tchebycheff.  Bull. Acad. Sci. URSS. Sér. Math. [Izvestia Akad. Nauk SSSR], pages 281–294, 1946.
  * [13] Dong Han Kim.  The shrinking target property of irrational rotations.  Nonlinearity, 20(7):1637–1643, 2007.
  * [14] Dong Han Kim.  Refined shrinking target property of rotations.  Nonlinearity, 27(9):1985–1997, 2014.
  * [15] Taehyeong Kim.  On a kurzweil type theorem via ubiquity.  Preprint: arXiv:2306.00847, 2023.
  * [16] Dmitry Kleinbock and Baowei Wang.  Measure theoretic laws for limsup sets defined by rectangles.  Adv. Math., 428:Paper No. 109154, 2023.
  * [17] Jaroslav Kurzweil.  On the metric theory of inhomogeneous diophantine approximations.  Studia Math., 15:84–112, 1955.
  * [18] A. Shapira U. Yifrach Y. Lachman, G. Rao.  K-divergent lattices.  Preprint: arXiv:2307.09054, 2023.
  * [19] Hermann Minkowski.  Ueber die Annäherung an eine reelle Grösse durch rationale Zahlen.  Math. Ann., 54(1-2):91–124, 1900.
  * [20] A. Shapira U. Moshchevitin, N. Rao.  Grids with dense values 2.  In preparation.
  * [21] Felipe A. Ramírez.  A characterization of bad approximability.  Nonlinearity, 31(9):4246–4262, 2018.
  * [22] David Simmons.  An analogue of a theorem of Kurzweil.  Nonlinearity, 28(5):1401–1408, 2015.
  * [23] S. Troubetskoĭ and Ĭ. Schmeling.  Inhomogeneous Diophantine approximations and angular recurrence for billiards in polygons.  Mat. Sb., 194(2):129–144, 2003.
  * [24] Jimmy Tseng.  On circle rotations and the shrinking target properties.  Discrete Contin. Dyn. Syst., 20(4):1111–1122, 2008.
  * [25] Tony van Ravenstein.  The three gap theorem (Steinhaus conjecture).  J. Austral. Math. Soc. Ser. A, 45(3):360–370, 1988.
  * [26] Baowei Wang and Jun Wu.  Mass transference principle from rectangles to rectangles in Diophantine approximation.  Math. Ann., 381(1-2):243–317, 2021.


# A template for the _arxiv_ style

Madhu Kiran Laboratoire d’imagerie, de vision et d’intelligence artificielle
(LIVIA), Ecole de technologie superieure, Montreal, CanadaCorresponding
author<EMAIL_ADDRESS>Le Thanh Nguyen-Meidine11footnotemark: 1 Rajat
Sahay Vellore Institute of Technology, Vellore Rafael Menelau Oliveira E
Cruz11footnotemark: 1 Louis-Antoine Blais-Morin Genetec Inc. Eric
Granger11footnotemark: 1

# Dynamic Template Selection Through Change Detection for Adaptive Siamese
Tracking

Madhu Kiran Laboratoire d’imagerie, de vision et d’intelligence artificielle
(LIVIA), Ecole de technologie superieure, Montreal, CanadaCorresponding
author<EMAIL_ADDRESS>Le Thanh Nguyen-Meidine11footnotemark: 1 Rajat
Sahay Vellore Institute of Technology, Vellore Rafael Menelau Oliveira E
Cruz11footnotemark: 1 Louis-Antoine Blais-Morin Genetec Inc. Eric
Granger11footnotemark: 1

###### Abstract

Deep Siamese trackers have recently gained much attention in recent years
since they can track visual objects at high speeds. Additionally, adaptive
tracking methods, where target samples collected by the tracker are employed
for online learning, have achieved state-of-the-art accuracy. However, single
object tracking (SOT) remains a challenging task in real-world application due
to changes and deformations in a target object’s appearance. Learning on all
the collected samples may lead to catastrophic forgetting, and thereby corrupt
the tracking model. In this paper, SOT is formulated as an online incremental
learning problem. A new method is proposed for dynamic sample selection and
memory replay, preventing template corruption. In particular, we propose a
change detection mechanism to detect gradual changes in object appearance, and
select the corresponding samples for online adaption. In addition, an entropy-
based sample selection strategy is introduced to maintain a diversified
auxiliary buffer for memory replay. Our proposed method can be integrated into
any object tracking algorithm that leverages online learning for model
adaptation. Extensive experiments conducted on the OTB-100, LaSOT, UAV123, and
TrackingNet datasets highlight the cost-effectiveness of our method, along
with the contribution of its key components. Results indicate that integrating
our proposed method into state-of-art adaptive Siamese trackers can increase
the potential benefits of a template update strategy, and significantly
improve performance.

Figure 1: Motivation of our proposed method: A t-SNE plot of deep CNN features
vectors extracted from an object bounding boxes during tracking. A vector was
sampled every time a gradual change was flagged by a change detector. Similar
points indicate similar appearance, while similar colors indicate vectors
sampled close in time, until a more abrupt change is detected (see the
legend). Although vectors sampled between more abrupt changes tend to be
clustered, they may also overlap with those extracted at a different time of
change.

.

##  1 Introduction

Along with the success of deep learning (DL) models [1, 2, 3, 4, 5], several
object trackers have adopted online learning [6, 7, 8, 9, 10, 11] to improve
performance. These trackers can be categorized as adaptive tracking methods
(or tracking-by-detection methods), and template matching methods [1, 4, 5,
2]. Adaptive Siamese tracking methods learn the target model online either
through stochastic gradient descent or other methods of optimization, such as
steepest gradient descent [9, 10, 11, 12]. They learn an object classifier
online to distinguish an object from the background. Template matching methods
compare an initial object appearance template with a search region. They
handle change in object appearance by interpolating the initial template with
the recently generated object template, as in [13, 14].

Some early DL models for trackers, such as MDNet and TCNN [6, 15], rely on
computationally expensive online model adaptation methods by gradient descent.
Despite the high level of tracking accuracy, they were not applicable for
real-time scenarios. Recent trackers, such as [7, 8, 9, 10, 11], have
introduced efficient optimization methods for online learning of classifiers
given a memory of previously seen templates localized by the tracker. Given
the ability to efficiently update these classifiers in response to changing
object appearance, adaptive Siamese trackers generally provide higher accuracy
than template matching.

In this paper, we focus on the problem of online model adaptation in adaptive
Siamese trackers, or tracking-by-detection methods. Some of the main
challenges with object tracking, and in particular with adaptive Siamese
trackers are: (1) object appearance changes over time, can be recurrent or a
permanent change; (2) model adaptation with template generated by the tracker
can be noisy, and during online training of the classifier, these are
considered as samples with pseudo labels; and (3) rapid model adaptation is
required to address real-time applications. Most adaptive Siamese trackers
update the classifier after some basic sanity check. Frequent classifier
training leads to integration of noise from tracker localized samples, and
thereby the overall model adaptation is not efficiently utilized.

Figure 1 shows the t-SNE plot of bounding boxed sampled from a LaSOT [16]
video, where each sample is represented using features extracted by the DiMP
[9] tracker. Assume that a change detector [17] is applied on the similarity
matching scores to assess concept drift111We define concept drift as a change
in the distribution of data over time in online learning [17]. Gradual,
recurrent, and abrupt changes are more relevant in this work (see definitions
in Section 2.C). on template samples localized by the tracker. Similar colors
indicate vectors sampled close in time, while similar points correspond to
templates of the object that have a similar appearance, between two detected
gradual changes. Given the more abrupt changes in object appearance over time,
multiple modes emerge in the object feature distribution. Between two
consecutive abrupt changes, object template features generally appear to
cluster together. However, some points of different colors overlap, indicating
recurring concepts in the object appearance space.

In this paper, a SOT framework is proposed to perform online-incremental
learning for model adaptation. Based on Figure 1, we hypothesize that object
appearances of consecutive samples do not change much until a more abrupt
concept drift occurs. Additionally, the changes can be recurring, indicating
that the object’s appearance can return to one of the earlier appearances. A
tracker should only adapt online when a concept drift is detected, to prevent
the tracker from adapting frequently, and integrating noisy samples into the
system, even when the adaptation is unnecessary. Adapting the model only in
the presence of drift reduces the overall complexity by adapting the
classifier fewer times.

Most recent adaptive Siamese trackers [9, 10, 11] rely on a budgeted memory to
train the classifier online with a limited number of samples, resulting in low
overall complexity. However, drifting object appearances, as discussed above,
may lead to a corrupted tracking model due to catastrophic forgetting of the
past, as the memory limits the storage of previously seen templates.
Catastrophic forgetting can efficiently be addressed in online learning by
using a replay buffer, as described in [18, 19, 20, 21]. These methods have
been initially proposed for online multi-class continual learning problems.
Inspired by the entropy maximization-based sampling [20] we extend the
algorithm to our problem, i.e., single-class incremental learning with pseudo
labels. In particular, we propose a classifier score discretization-based
algorithm to maintain an auxiliary memory, and alleviate the catastrophic
forgetting that may occur during online tracking.

In this paper, the overall performance of online learned SOT is improved in
two respects. First, we hypothesize and set up the online learning problem in
tracking as an incremental learning problem with catastrophic forgetting with
change detection. Secondly, we propose a classifier score discretization-based
algorithm to sustain an auxiliary memory of diverse samples from the previous
time frames, and limit knowledge corruption. We integrate our proposed method
into several state-of-the-art adaptive Siamese tracker, including DiMP [9],
PrDimp [10], and TrDiMP [12], and evaluated on the OTB video dataset [22, 16,
23, 24] to demonstrate the effectiveness of our proposed method. Performance
improves a margin of 2% on these datasets, without resorting to additional
parameters or change in CNN architectures and with lower overall complexity.

Figure 2: Overall framework of our proposed method applied to an adaptive
Siamese tracker. It relies on a change detection mechanism to trigger
adaptation of the model, and an additional auxiliary memory to store diverse
prior template information, and alleviate catastrophic forgetting in tracking.
The regions indicated in dotted lines point to our contributions in this
paper.

.

##  2 Related Work

###  2.1 Siamese Tracking:

Pioneered by SINT [25] and SiamFC [1], the Siamese family of trackers evolved
from Siamese networks, trained offline through metric learning for similarity
matching. These networks were trained on a large dataset to learn generic
feature representations for object tracking. SiamRPN [2] further improves on
the pioneering work by employing region proposals to produce a target-specific
anchor-based detector. Then, the following Siamese trackers mainly involved
designing powerful backbones [5, 26], or more powerful proposal networks, like
in [27]. ATOM [8] and DIMP [9] are robust online trackers that differ from the
general offline Siamese trackers. Other paradigms of Siamese trackers are
distractor-aware training and domain-specific tracking [4, 28].

###  2.2 Online Learning for Tracking:

In [29], an LSTM is incorporated to learn long-term relationships during
tracking and turns the VOT problem into a consecutive decision-making process
of selecting the best model for tracking via reinforcement learning [30]. In
[13] and [4], models are updated online by a moving average based learning.
These methods integrate the target region extracted from tracker output into
the initial target. In [31], a generative model is learned via adversarial
learning to generate random masks that produce shifted versions of target
templates from the original template. Then, an adversarial function is used to
decide whether or not the generated template is from the same distribution and
if they will be used as templates for tracking. In [32], an LSTM is employed
to estimate the current template by storing previous templates in a memory
bank. In [33], authors propose to compute transformation matrix with reference
to the initial template, with a regularised linear regression in the Fourier
domain. Finally, in [34], authors propose to learn the updating co-efficient
of a correlation filter-based tracker using SGD online. All these methods use
the tracker output as the reference template while updating on top of the
initial template. [9, 10] propose a model where an adaptive discriminative
model is generated online by the steepest gradient descent method. They differ
from another online learned method like [6] due to their real-time
performance. Similarly [35] introduce online model prediction but employ a
fast conjugate gradient algorithm for model prediction. Foreground score maps
are estimated online, and the classification branch is combined by weighted
addition. All these trackers update the template every frame after some sanity
check of the templates. [36] is somewhat close to our work by the fact that
they consider when a how to update the tracker. However, it involves a
computationally expensive model – runs at a slow speed of 13fps –which is
trained to decide whether the tracker is ready for an update.

###  2.3 Online Incremental Learning:

Learning new classes or distributions of existing classes online with
sequential data is called incremental learning [20, 37]. In such scenarios,
models suffer from poor performance on previous tasks caused by knowledge
corruption of catastrophic forgetting. Regularization methods have employ
constraints to learning of new tasks via a suitable regularization term [38,
39]. Structural methods freezing weight and expand the architecture along with
new tasks [40, 41]. Rehearsal methods use some sort of memory to alleviate
catastrophic forgetting by storing previously seen examples. Gradient episodic
memory (GEM) uses examples from previously seen task and during training of a
new task forces the gradient descent steps such that they don’t increase the
loss on them. Generative replay [42] learns a Generative adversarial network
(GAN) during training of previous tasks to implicitly store samples of older
tasks and mix them in appropriate proportions while training of new tasks.
Gradient based Sample Selection (GSS) [21] maintains a buffer of diverse
samples suing gradient information. [20] use a two step method to maintain a
buffer of diverse samples for a multi-class incremental learning problem using
entropy. Their first step maintains a set of samples with diverse class
labels. Inspired from their method, be adapt it our single class incremental
learning problem.

###  2.4 Change Detection in Streaming Data:

Concept drift can be categorised as gradual change, abrupt change, incremental
change and recurrent change. In gradual change the duration of change in
concept is relatively large compared to abrupt drift. In Recurrent drift
concepts re-appear later after a drift and in incremental drift the concepts
continuously change over time and they are also gradual changes. Drift
detection methods can be categorised as 1) S equential, 2) Adaptive Windowing
3) Fixed Cumulative windowing 4) Statistical. Sequential analysis based
methods predict the drift sequentially such as CUSUM [43] and Page-Hinckeley
[44]. Window based methods use a fixed or adaptive window to summarize old
information and newer ones to asses the difference between them ex: ADWIN
[45], HDDM [46]. Statistical methods use statistical parameters to to predict
the drift DDM [47], RDDM [48]. Some of the Neural network based model includes
[49, 50] both designed for drift detection in text data streams. These
included a neural network model for drift detection. A few drift detection
algorithms were presented for unlabelled data stream drift detection. This
includes [51] which use margin density for drift detection and [52], using
k-means clustering and Page-Hinkeley test.

##  3 Proposed Method

Figure 2 shows the framework of our proposed method integrated with an
adaptive Siamese tracker. The tracker accepts localised object template, and
then uses the feature extractor $\Theta$ to produce the features $\varphi{t}$.
These features are then stored in the main memory – a buffer to hold $B$
instances of object template features from previous frames. $B$ is the budget
of the memory buffer. Once the memory budget is met, older samples are removed
on a first in, first out basis. An online classifier is trained by the
optimizer using all the instances from the memory buffer to produce a model
$f$ which is convolved (Conv in Fig 2) with test image features
$\varphi_{S_{t}}$ to obtain a classifications score map $S$. This score map
distinguishes the object foreground from the background for localization.

The framework discussed above is a common among some of the recent trackers
such as [9, 10, 11, 12]. In this work, we propose using a change detector and
training the model only when a concept drift is detected, instead of the
conventional method of training the model every frame. Additionally, to
alleviate the problem of concept drift, we propose to maintain an auxiliary
memory to store older samples. This memory is sustained with a budget B, and
once the memory is full, older samples are replaced with newer ones based on
an entropy maximization algorithm [20]. This increases the overall entropy of
the samples in the memory.

###  3.1 Change Detection:

Given a time period $[0,t]$, a set of samples denoted
$S_{0,t}=\left\\{d_{0},\ldots,d_{t}\right\\}$ where
$d_{i}=\left(X_{i},y_{i}\right)$ is one instance of data with features $X_{i}$
and label $y_{i}$, and $S_{0,t}$ follows a certain distribution which is
$P_{t}(X,y)$.Concept drift can occur due to change in class conditional
probability $P_{t}(X|y)$, change in class distributions $P(y)$ or appearances
of new classes, or change in posterior probability $P(c|X)$. In general
concept drift occurs when $P_{t}(X,y)\neq P_{t+1}(X,y)$. At the beginning, the
tracker is initialized by a ground truth template representing the object.
However, since the object’s appearance may change over time, the tracker must
be updated with object samples collected by the tracker (which is prone to
noise) while tracking. The trackers in the literature updates the model
periodically with a fixed period. However, we hypothesize that a tracker needs
to be updated only when a concept drift is detected to avoid integrating noise
into the model. Hence we propose using the same model initially trained at the
start of the tracking until a concept drift is detected. The classification
score map is a good indicator of change in concept as the distribution of
incoming target samples changes the old model will perform poorly on these
samples. Since the classifier is a binary classifier, the classification score
directly gives an estimation of probability of the predicted label $y$ for a
given object localised by the tracker. Hence concept drift which can be
detected by the joint probability of $P(\varphi{t},y)$, the classifier change
is a good estimate of concept change. We employ an off-the-shelf available
sequential change detector (see Section 2) that relies on the maximum value of
the classification score map. The change detector statistically analyses the
classification scores as a time series data and triggers a flag to indicate
the change in classification score. This event triggers the classifier’s
training online to obtain a new model. This tracker will be used for tracking
until the next change is observed.

###  3.2 Entropy Maximization Sampling for Auxiliary Memory:

We propose an auxiliary memory of buffer size $B$ to hold older samples
discarded by the main memory. The auxiliary memory will contain some of the
older representations of the object in order to alleviate the problem of
catastrophic forgetting due to the limited capacity of the main memory.
Training the classifier with a large number of samples can affect the
tracker’s overall run time, making it inapplicable for real-time scenarios.
Hence, it is important to maintain a limited set of samples with maximum
variance to contain object representations from different time frames. We
consider the problem of adaptive Siamese tracking as a variation of
incremental learning from pseudo labels (labels generated by the tracker for
the incoming data). As discussed in Section 2, most of the previous work in
this field was related to multi-class incremental learning. Hence we propose a
method to adapt the problem to binary classification. Initially, the auxiliary
memory is filled by samples retrieved from the main memory by First in First
Out (FIFO). Once the buffer of size $B$ is full, we apply the entropy
maximization algorithm to maintain samples of high entropy. Each of the
samples in the main memory are stored as a pair of sample and the
corresponding classifier score. The classifier score $S$ is maintained as a
discretised label. Only samples with a classifier score greater than a
threshold $\tau$ are accepted. Then the score is discretized into range
$0\;to\;Y$ by a linear mapping function $D$. Hence each sample is represented
by its corresponding label $y$ Let the samples in the auxiliary memory be
represented by $\mathcal{T}=\left\\{\varphi_{j},y_{j}\mid 1\leq j\leq
B\right\\}$. Where $j$ is the index of the sample in the auxiliary memory. $y$
ranges from $[0\;to\;Y]$.

To train a model online, buffer
$\mathcal{B}=\left\\{\varphi_{k},y_{k}\right\\}_{k=1}^{B}$. This buffer is
used on the rehearsal of previously seen samples. When an individual input
output pair ${\varphi_{k},y_{k}}$ arrives, a suitable algorithm decides which
old sample will be replaced from the buffer. During run-time, samples from the
main and auxiliary buffers are sampled to train the model. For the model to
avoid catastrophic forgetting, the auxiliary memory samples must be as diverse
as possible. Inspired by [20], we consider the problem of selecting a sample
for rehearsal as a joint distribution of
$P(\mathbf{\varphi},Y):\mathbb{R}^{S}\times\mathbb{Z}\rightarrow[0,1]$ for
random variables $\mathbf{\varphi}$ and $Y$. The Shannon entropy of $\varphi$
and $Y$ given by, $H(\mathbf{\varphi},Y)=-\mathbb{E}[\log
P(\mathbf{\varphi},Y)]$. Since $Y$ is a discretized classifier score for the
sample, it can be considered a representation of different appearances of the
sample. If $P(\varphi,y)$ becomes more predictable, then the entropy will tend
to zero. We aim to make $\varphi$ as diverse as possible, which is similar to
maximum entropy sampling. Shannon entropy can be re-written as:

$\displaystyle H(\mathbf{\varphi},Y)$ $\displaystyle=-\mathbb{E}[\log
P(\mathbf{\varphi}\mid Y)]-\mathbb{E}[\log P(Y)]$ (1)
$\displaystyle=H(\mathbf{\varphi}\mid Y)+H(Y).$

Algorithm 1 Entropy based sample selection with discrete classifier scores

$\varphi,S$, Auxiliary Buffer $\mathcal{A}$, $B$

Updated $\mathcal{A}$, $BA$ training samples

if $\mathcal{A}<B$ then

$y\leftarrow D(S)$ $\triangleright$ Discretize classifier score to get a label

$\mathcal{A}\leftarrow\varphi,y$ $\triangleright$ Add to buffer until full

else

$\mathcal{C}\leftarrow$ samples of majority label in $\mathcal{A}$

for $\varphi\in\mathcal{C}$ do

$d_{i}\leftarrow\min_{\mathbf{x}_{j}}\in\mathcal{C}\left\|\mathbf{x}_{i}-\mathbf{x}_{j}\right\|_{2}$

end for

$i\sim P(i)\leftarrow\left(1-d_{i}\right)/\sum_{j}\left(1-d_{j}\right)$

$\mathbf{x}_{i},y_{i}\leftarrow\mathbf{x},y$

$BA\leftarrow B+n~{}samples~{}from~{}\mathcal{A}$

end if

It can be observed from Eqn 1 that the overall entropy depends on $P(Y)$ and
$P(\varphi\mid Y)$. $P(Y)$ can be minimised by keeping the most common label
as a criterion to replace the sample. While among the samples that fall into
the same labels, there could be many repetitions of similar examples given a
label. $P(\varphi\mid Y)$ is related to examples within the same labels, and
it can be minimised by estimating the distribution of $P(\varphi\mid Y)$.
Similar to [20] we use Kernel Density Estimation (KDE) to estimate
$P(\varphi\mid Y)$, defined by:

$\displaystyle P(\mathbf{x}\mid
y)\approx\frac{1}{M_{y}}\sum_{k=1}^{M_{y}},K\left(\mathbf{x}-\mathbf{x}_{y}[k]\right)$
(2)

where $M_{y}$, is the number of examples with label $y$, and
$K:\mathbb{R}^{S}\rightarrow[0,1]$ is a kernel function. Hence, given these
approximations an algorithm to maximize the joint entropy
$H(\mathbf{\varphi}\mid Y)$ is needed. First, we maximize $H(Y)$ by
maintaining an equal number of samples with labels $Y$ across all classes.
This is achieved by identifying the majority label $Y$ and then replacing the
sample in the majority class with a probability
$\left(1-d_{i}\right)/\sum_{j}\left(1-d_{j}\right)$ where $d_{i}$ is the
minimum distance of $x_{i}$ to all the examples of the same class in the
buffer. The complete process is detailed in Algorithm 1.

We summarize our proposed method as follows. When the auxiliary buffer is not
full, we keep adding all new samples that exit from the main memory until the
auxiliary buffer is full. Once the buffer is full, we use the relative
frequency of the class score discretised label to select the majority label.
We choose an example with this label for replacement where the probability of
a particular example to be replaced is high if the minimum distance to all
other examples of the same label in the buffer is small.

During tracking, all the samples from main memory and a certain fixed set of
$n$ samples from auxiliary memory are randomly sampled to create a $BA$ batch
of samples for training the classifier. The main memory ensures current
representation is learnt while the auxiliary samples ensure catastrophic
forgetting is avoided.

##  4 Results and Discussion

###  4.1 Datasets:

We evaluate trackers on four well known object tracking datasets. Two of them,
UAV123 [23] and OTB-100 [22], are composed of short tracking videos, while
LaSOT [16] and TrackingNet [24] are large scale datasets. The datasets are
described below.

  * •

OTB-100: OTB dataset is a classical benchmark in Visual Object tracking
containing a total of 100 videos with an average of 590 frames per video. We
report the results on OTB-2015 and it is known to have tended to saturation
over the years [53].

  * •

UAV123: This dataset consists of 123 low altitude aerial videos captured from
UAV. It is a challenging dataset due to small objects, fast motion as well as
the presence of distractor objects.

  * •

LaSOT: LaSOT is a high-quality benchmark for Large-scale Single Object
Tracking. LaSOT consists of a total of 280 test videos. The videos are longer
compared to other datasets with an average of 2500 frames per video in the
dataset. Thus online model adaptation and without catastrophic forgetting are
crucial for this dataset.

  * •

TrackingNet: TrackingNet is a large-scale tracking dataset consisting of
videos in the wild. It has a total of 30,643 videos split into 30,132 training
videos and 511 testing videos, with an average of 470,9 frames.

###  4.2 Experimental Setup:

Our proposed method is integrated in state-of-art DiMP [9], PrDiMP [10], and
TrDiMP [12] trackers, all online learning based trackers. These trackers have
been selected for evaluation because DiMP is one of the first successful real-
time online learning tracker while PrDiMP and TrDiMP are improvisations on
DiMP. TrDiMP is a the state-of-the-art models for real time tracking. Both
PrDiMP and TrDiMP trackers are accurate on recent visual object tracking
challenge datasets, and hence they have been selected for our study. We use
the pre-trained models provided by the authors, and do not train the tracker
further, since our proposed method is an online sample selection algorithm
during that is applied during run time. We use Page-Hinckeley [17] test for
change detection, to decide when to update the classifier. Although we could
effectively use other statistical change detection algorithms we use Page-
Hinckeley as it is a sequential analysis methods for change detection (see
Section 2). Additionally, it can work with continues values such as classifier
scores. We set a memory buffer size of 50 similar to [9, 10] and auxiliary
memory buffer as 50. The threshold $\lambda$ for change detection is set to
0.15 empirically.

In this paper, trackers are evaluated with Area Under Curve metric (AUC) of
the success plot [22], which measures the overall performance of the tracker,
under different tracked to ground truth overlap acceptance threshold.

###  4.3 Ablation Studies:

In this subsection, we characterize our proposed method on LaSOT dataset [16].
In this part of our experiments, we analyze the effectiveness of different
components of our proposed method and study the change detection threshold
$\lambda$. Table 1 shows the ablation study performed for the proposed method
on LaSOT [16] dataset. The study was performed to study the effectiveness of
different components of the proposed model. The first part of the table
studies the effect of change detection for model update. The samples for the
main memory are collected all the frames in a circular buffer keeping a
maximum of 50 samples from past. The classifier is updated only when a change
in the concept is detected by statistically analysing the classifier scores
for previously seen samples as a time-series data. This is performed by the
change detector. This indicates that the object appearance is changing and
this would be a good time to train the classifier online using the samples
from the main memory.

3

Table 1: Ablation study: AUC accuracy of different components of our proposed
method integrated in DiMP on the LaSOT dataset.

Tracker |  | AUC
---
$\cdot$ Change Detector
DiMP [9] | 57.1
DiMP Random | 55.4
DiMP Periodic 5 | 56.8
DiMP Periodic 10 | 56.5
DiMP Periodic 15 | 55.6
DiMP with CD | 57.8
$\cdot$ Auxiliary Memory
DiMP with CD + Random Replacement | 57.9
DiMP with CD + Density Replacement | 58.2
DiMP with CD + Class. Score Discretised | 58.6

For the ablation study, different experiments were performed where the
classifier was trained with samples in memory online randomly as indicated by
"DiMP+Random" in the table. Then further experiments were performed by
training the classifier online periodically as indicated by "Dimp+Periodic"
every 5, 10 and 15 frames of the incoming image stream. Then finally, change
detection was applied on the classifier’s output scores to train the
classifier online when changes were detected. It can be observed that change
detection triggered classifier training outperforms all the other strategies.
This is because we rely solely on the classifier score to detect concept drift
and hence we train the model only on need basis. All other methods use a
strategy that does not really takes classifier response into account. Hence,
change detection for model update can be a efficient method for knowing when
to update the tracker model given noisy tracker localisation.

Table 2: AUC accuracy of our proposed method integrated in DiMP50 and PrDiMP50
trackers on the UAV123, OTB-100, LaSOT and TrackingNet datasets.

Tracker | UAV123 | OTB100 | TrackingNet | LaSOT
---|---|---|---|---
ECO [7] | 53.2 | 69.1 | - | -
ATOM [8] | 63.2 | 66.4 | 70.3 | 51.5
MDNet [6] | - | 67.8 | - | 39.7
SiamRPN++ [54] | - | 69.6 | 73.3 | 49.6
DiMP50 [9] | 65.3 | 68.4 | 74 | 56.9
ADiMP50(Ours) | 67.6 | 69.1 | 75.5 | 58.6
PrDiMP50 [10] | 66.7 | 69.6 | 75.8 | 59.8
APrDiMP50(Ours) | 67.8 | 70.9 | 77.1 | 61.5
TrDiMP [12] | 67.3 | 70.6 | - | 63.0
ATrDiMP(Ours) | 68.1 | 71.5 | - | 69.0

The next part of the table contains results of experiments performed to
evaluate the effectiveness of our proposed classifier score discretization for
maintaining with high variance in memory. As discussed in the algorithm, our
proposed method replaces old samples from memory when new samples arrive. The
samples to be replaced are determined by different criteria, i.e., a discrete
label based on classifier scores and density of the samples within the same
bin of label. First, we randomly replace samples from memory when a new sample
arrives, indicated by "DiMP+CD+Random replacement". Then we use density to
replace samples considering all samples under one label, "DiMP+CD+Density
replacement". Finally, we segregate samples in individual bins determined by
discretizing classifier scores and assigning discrete values to them. Then a
density-based replacement is applied for samples within bins indicated by
"DiMP+CD+Class. Score discretized" in the Table. Our proposed classifier score
based discretization improves the tracker’s overall AUC, indicating that the
proposed method stores samples such that the samples in the memory have a high
variance. This ensures to hold most information within a budgeted memory.

(a) Success plot for OTB dataset.

(b) Precision plot for OTB dataset.

(c) Success plot for UAV dataset.

(d) Precision plot for UAV dataset.

Figure 3: Success and Precision plots to compare our proposed methods with the
state-of-the-art methods

###  4.4 Integration in State-of-art Trackers:

Our proposed approach is applied on DiMP and PrDiMP trackers, indicated by
ADiMP and APrDiMP, respectively. We compare our method with other template
matching based trackers as well as other adaptive siamese tracking based
trackers (Table 2). We can observe that our method significantly improves the
AUC accuracy of both state-of-the-art DiMP and PrDiMP trackers. In particular,
the results with the UAV dataset appear better than that with the OTB dataset
because UAV dataset videos are relatively longer than those of OTB-100. This
is because longer videos have enough samples to improve the quality of samples
with the auxiliary memory.

We also evaluate of our proposed method on these datasets similar to the
previous section.The Area Under Curve was chosen as the comparison metric. Our
proposed ADiMP and PrDiMP trackers outperform the corresponding baselines.
ADiMP achieves an improvement of 1.6% improvement over the baseline and PrDiMP
achieves an improvement of 1.7% on the LaSOT dataset. Similarly on the
TrackingNet dataset they achieve 1.5% and 1.2% respectively.

###  4.5 Success and Precision plots:

Fig. 3 shows the comparison of success plots and precision plots of our
proposed method along with state-of-the-art methods. We can observe from the
plots that we achieve better AUC score in comparison with methods without our
proposed change detection and sample selection. PrDiMP and TrDiMP are relative
improvements over DiMP tracker with significant architectural changes or
changes with additional loss functions. But Our proposed method achives a
similar improvement with a simple computationally efficient technique as
change detection based online learning and entropy based sample selection.

###  4.6 Tracking Frame Rate

We compare the frame rate of our tracker with other online learning methods in
Table 3. The table shows that our proposed ADiMP and APrDiMP methods run
faster than the baselines by 6 and 5 FPS, respectively. This performance
improvement occurs because, different than the baseline methods, we do not
train during every frame of the tracked object. Our adaptive framework only
updates the tracker in the presence of concept drift, significantly reducing
the overall number of times the tracker was trained in a given video. Thus,
reducing the computational complexity of the system. In applications such as
video surveillance, the object tracker needs to perform at real-time to avoid
missing important events. Additionally longer tracking is an other important
aspect of video surveillance applications specially in vehicle or person
tracking to maintain unique identities during tracking. The trackers were
evaluated on a Linux server with GTX1080 graphics card.

Table 3: Comparison of tracking speed of online learning trackers with our proposed method evaluated on LaSOT dataset. Trackers | Frames/Second
---|---
ECO [7] | 60
SiamRPN++ [54] | 35
MDNet [6] | 3
ATOM [8] | 30
DiMP [9] | 38
ADiMP(ours) | 44
PrDiMP [10] | 28
APrDiMP(ours) | 33
TrDiMP [12] | 25
ATrDiMP(ours) | 29

##  5 Conclusion

We propose an online learning framework with concept drift triggered online
classifier training for adaptive Siamese tracking methods. We hypothesize that
in online learning of object tracking model, learning is necessary only when a
concept drift is detected. Additionally we identify catastrophic forgetting in
online tracking and propose entropy based sample selection for online learning
in trackers.Our proposed method could be adapted into several state-of-the-art
adaptive tracking methods. We evaluate our proposed method on DiMP, PrDiMP and
TrDimp trackers on OTB, UAV, LaSOT and TrackingNet datasets. Our approach
improves the performance of state-of-the-art adaptive tracking methods by an
average of 2% in AUC measure in addition to improving the overall tracking
speed.

## References

  * [1] L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr, “Fully-convolutional siamese networks for object tracking,” _arXiv preprint arXiv:1606.09549_ , 2016.
  * [2] B. Li, J. Yan, W. Wu, Z. Zhu, and X. Hu, “High performance visual tracking with siamese region proposal network,” in _CVPR_ , 2018.
  * [3] X. Dong and J. Shen, “Triplet loss in siamese network for object tracking,” in _ECCV_ , 2018.
  * [4] Z. Zhu, Q. Wang, B. Li, W. Wu, J. Yan, and W. Hu, “Distractor-aware siamese networks for visual object tracking,” in _ECCV_ , 2018.
  * [5] Z. Zhang and H. Peng, “Deeper and wider siamese networks for real-time visual tracking,” in _CVPR_ , 2019.
  * [6] H. Nam and B. Han, “Learning multi-domain convolutional neural networks for visual tracking,” in _CVPR_ , 2016.
  * [7] M. Danelljan, G. Bhat, F. Shahbaz Khan, and M. Felsberg, “Eco: Efficient convolution operators for tracking,” in _CVPR_ , 2017.
  * [8] M. Danelljan, G. Bhat, F. S. Khan, and M. Felsberg, “Atom: Accurate tracking by overlap maximization,” in _CVPR_ , 2019.
  * [9] G. Bhat, M. Danelljan, L. V. Gool, and R. Timofte, “Learning discriminative model prediction for tracking,” in _CVPR_ , 2019.
  * [10] M. Danelljan, L. V. Gool, and R. Timofte, “Probabilistic regression for visual tracking,” in _CVPR_ , 2020.
  * [11] G. Bhat, M. Danelljan, L. Van Gool, and R. Timofte, “Know your surroundings: Exploiting scene information for object tracking,” in _European Conference on Computer Vision_. Springer, 2020, pp. 205–221.
  * [12] N. Wang, W. Zhou, J. Wang, and H. Li, “Transformer meets tracker: Exploiting temporal context for robust visual tracking,” in _CVPR_ , 2021, pp. 1571–1580.
  * [13] J. Valmadre, L. Bertinetto, J. Henriques, A. Vedaldi, and P. H. Torr, “End-to-end representation learning for correlation filter based tracking,” in _CVPR_ , 2017, pp. 2805–2813.
  * [14] L. Zhang, A. Gonzalez-Garcia, J. v. d. Weijer, M. Danelljan, and F. S. Khan, “Learning the model update for siamese trackers,” in _ICCV_ , 2019.
  * [15] H. Nam, M. Baek, and B. Han, “Modeling and propagating cnns in a tree structure for visual tracking,” _arXiv preprint arXiv:1608.07242_ , 2016\.
  * [16] H. Fan, L. Lin, F. Yang, P. Chu, G. Deng, S. Yu, H. Bai, Y. Xu, C. Liao, and H. Ling, “Lasot: A high-quality benchmark for large-scale single object tracking,” in _CVPR_ , 2019.
  * [17] S. Agrahari and A. K. Singh, “Concept drift detection in data stream mining: A literature review,” _Journal of King Saud University-Computer and Information Sciences_ , 2021.
  * [18] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” in _CVPR_ , 2017.
  * [19] D. Lopez-Paz and M. Ranzato, “Gradient episodic memory for continual learning,” _Advances in neural information processing systems_ , vol. 30, pp. 6467–6476, 2017.
  * [20] F. Wiewel and B. Yang, “Entropy-based sample selection for online continual learning,” in _2020 28th European Signal Processing Conference (EUSIPCO)_. IEEE, 2021, pp. 1477–1481.
  * [21] R. Aljundi, M. Lin, B. Goujaud, and Y. Bengio, “Gradient based sample selection for online continual learning,” _NIPS_ , 2019.
  * [22] Y. Wu, J. Lim, and M.-H. Yang, “Online object tracking: A benchmark,” in _CVPR_ , 2013.
  * [23] M. Mueller, N. Smith, and B. Ghanem, “A benchmark and simulator for uav tracking,” in _European conference on computer vision_. Springer, 2016, pp. 445–461.
  * [24] M. Muller, A. Bibi, S. Giancola, S. Alsubaihi, and B. Ghanem, “Trackingnet: A large-scale dataset and benchmark for object tracking in the wild,” in _ECCV_ , 2018.
  * [25] R. Tao, E. Gavves, and A. W. Smeulders, “Siamese instance search for tracking,” in _CVPR_ , 2016.
  * [26] Y. Li and X. Zhang, “Siamvgg: Visual tracking using deeper siamese networks,” 2019\.
  * [27] H. Fan and H. Ling, “Siamese cascaded region proposal networks for real-time visual tracking,” in _CVPR_ , 2019.
  * [28] A. He, C. Luo, X. Tian, and W. Zeng, “A twofold siamese network for real-time object tracking,” in _CVPR_ , 2018.
  * [29] B. Zhong, B. Bai, J. Li, Y. Zhang, and Y. Fu, “Hierarchical tracking by reinforcement learning-based searching and coarse-to-fine verifying,” _IEEE Transactions on Image Processing_ , vol. 28, no. 5, pp. 2331–2341, 2018\.
  * [30] E. Duman and O. A. Erdem, “Anomaly detection in videos using optical flow and convolutional autoencoder,” _IEEE Access_ , vol. 7, pp. 183 914–183 923, 2019.
  * [31] Y. Song, C. Ma, X. Wu, L. Gong, L. Bao, W. Zuo, C. Shen, R. W. Lau, and M.-H. Yang, “Vital: Visual tracking via adversarial learning,” in _CVPR_ , 2018\.
  * [32] T. Yang and A. B. Chan, “Learning dynamic memory networks for object tracking,” in _ECCV_ , 2018, pp. 152–167.
  * [33] Q. Guo, W. Feng, C. Zhou, R. Huang, L. Wan, and S. Wang, “Learning dynamic siamese network for visual object tracking,” in _ICCV_ , 2017, pp. 1763–1771.
  * [34] Y. Yao, X. Wu, L. Zhang, S. Shan, and W. Zuo, “Joint representation and truncated inference learning for correlation filter based tracking,” in _ECCV_ , 2018.
  * [35] Z. Zhang, H. Peng, J. Fu, B. Li, and W. Hu, “Ocean: Object-aware anchor-free tracking,” _arXiv preprint arXiv:2006.10721_ , 2020.
  * [36] K. Dai, Y. Zhang, D. Wang, J. Li, H. Lu, and X. Yang, “High-performance long-term tracking with meta-updater,” in _CVPR_ , 2020.
  * [37] J. He, R. Mao, Z. Shao, and F. Zhu, “Incremental learning in online scenario,” in _CVPR_ , 2020.
  * [38] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska _et al._ , “Overcoming catastrophic forgetting in neural networks,” _Proceedings of the national academy of sciences_ , vol. 114, no. 13, pp. 3521–3526, 2017.
  * [39] F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” in _ICML_ , 2017.
  * [40] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” _arXiv preprint arXiv:1606.04671_ , 2016.
  * [41] J. Yoon, E. Yang, J. Lee, and S. J. Hwang, “Lifelong learning with dynamically expandable networks,” in _ICLR_ , 2018.
  * [42] H. Shin, J. K. Lee, J. Kim, and J. Kim, “Continual learning with deep generative replay,” _arXiv preprint arXiv:1705.08690_ , 2017.
  * [43] F. Gustafsson, _Adaptive Filtering and Change Detection_. Chichester, United Kingdom: John Wiley and Sons Ltd, 2000\.
  * [44] E. S. Page, “Continuous inspection schemes,” _Biometrika_ , vol. 41, no. 1/2, pp. 100–115, 1954.
  * [45] A. Bifet and R. Gavalda, “Learning from time-changing data with adaptive windowing,” in _SIAM_ , 2007.
  * [46] I. Frias-Blanco, J. del Campo-Ávila, G. Ramos-Jimenez, R. Morales-Bueno, A. Ortiz-Diaz, and Y. Caballero-Mota, “Online and non-parametric drift detection methods based on hoeffding’s bounds,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 27, no. 3, pp. 810–823, 2014.
  * [47] J. Gama, P. Medas, G. Castillo, and P. Rodrigues, “Learning with drift detection,” in _Brazilian symposium on artificial intelligence_. Springer, 2004, pp. 286–295.
  * [48] R. S. Barros, D. R. Cabral, P. M. Gonçalves Jr, and S. G. Santos, “Rddm: Reactive drift detection method,” _Expert Systems with Applications_ , vol. 90, pp. 344–355, 2017.
  * [49] J. L. Lobo, I. Laña, J. Del Ser, M. N. Bilbao, and N. Kasabov, “Evolving spiking neural networks for online learning over drifting data streams,” _Neural Networks_ , vol. 108, pp. 1–19, 2018.
  * [50] Y. Zhang, G. Chu, P. Li, X. Hu, and X. Wu, “Three-layer concept drifting detection in text data streams,” _Neurocomputing_ , vol. 260, pp. 393–403, 2017.
  * [51] T. S. Sethi and M. Kantardzic, “On the reliable detection of concept drift from streaming unlabeled data,” _Expert Systems with Applications_ , vol. 82, pp. 77–99, 2017.
  * [52] J. de Andrade Silva, E. R. Hruschka, and J. Gama, “An evolutionary algorithm for clustering data streams with a variable number of clusters,” _Expert Systems with Applications_ , vol. 67, pp. 228–238, 2017.
  * [53] Z. Fu, Q. Liu, Z. Fu, and Y. Wang, “Stmtrack: Template-free visual tracking with space-time memory networks,” in _CVPR_ , 2021.
  * [54] B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, and J. Yan, “Siamrpn++: Evolution of siamese visual tracking with very deep networks,” in _CVPR_ , 2019.