where the last inequality results from $\|\phi\|_{2,\infty}\leq\sqrt{d}$,
$\|\psi\|_{2,\infty}\leq\sqrt{d}$ and $\|V_{f}\|_{\infty}\leq\frac{1}{2}{\rm
sp}(V^{*})$. Combine (F.13) and
$\mathds{1}\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\geq
1\right)\leq\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge 1$, it holds that

$\sum_{t=1}^{T}\mathds{1}\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\geq
1\right)\leq\sum_{t=1}^{T}\big{\|}X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge 1\leq
d(\epsilon).$ (F.15)

Step 1: Bound over dominance coefficient.
Note the sum of Bellman errors follows that

$\displaystyle\sum_{t=1}^{T}\mathcal{E}(f_{t})(s_{t},a_{t})$
$\displaystyle=\sum_{t=1}^{T}\left(\big{(}r_{f_{t}}+\mathbb{P}_{f_{t}}V_{f_{t}}\big{)}(s_{t},a_{t})-\big{(}r_{f^{*}}+\mathbb{P}_{f^{*}}V_{f_{t}}\big{)}(s_{t},a_{t})\right)$
$\displaystyle=\sum_{t=1}^{T}(\theta_{t}-\theta^{*})^{\top}\Big{(}\psi(s_{t},a_{t})+\int_{\mathcal{S}}\phi(s_{t},a_{t},s^{\prime})V_{f_{t}}(s^{\prime}){\rm
d}s^{\prime}\Big{)}$
$\displaystyle=\sum_{t=1}^{T}W_{t}^{\top}X_{t}\cdot\left(\mathds{1}\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\leq
1\right)+\mathds{1}\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}>1\right)\right)$
$\displaystyle\leq\sum_{t=1}^{T}W_{t}^{\top}X_{t}\cdot\mathds{1}\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\leq
1\right)+({\rm sp}(V^{*})+2)\cdot\min\\{d(\epsilon),T\\}$
$\displaystyle\leq\sum_{t=1}^{T}\|W_{t}\|_{\Sigma_{t}}\cdot\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge
1\right)+({\rm sp}(V^{*})+2)\cdot\min\\{d(\epsilon),T\\},$ (F.16)

where the first inequality results from (F.15) and the last inequality arises
from the Cauchy-Swartz inequality. Combine (F.13) and (F.14), we have

$\displaystyle\sum_{t=1}^{T}\|W_{t}\|_{\Sigma_{t}}\cdot\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge
1\right)\leq\sum_{t=1}^{T}\left(2\sqrt{\epsilon}+\left[\sum_{i=1}^{t-1}\|l_{f_{i}}(f_{t},f_{t},\zeta_{i})\|_{2}^{2}\right]^{1/2}\right)\cdot\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge
1\right)$
$\displaystyle\quad\leq\left[\sum_{t=1}^{T}4\epsilon\right]^{1/2}\left[\sum_{t=1}^{T}\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge
1\right]^{1/2}+\left[\sum_{t=1}^{T}\sum_{i=1}^{t-1}\|l_{f_{i}}(f_{t},f_{t},\zeta_{i})\|_{2}^{2}\right]^{1/2}\left[\sum_{t=1}^{T}\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge
1\right]^{1/2}$ $\displaystyle\quad\leq
2\sqrt{T\epsilon\cdot\min\\{d(\epsilon),T\\}}+\left[d(\epsilon)\sum_{t=1}^{T}\sum_{i=1}^{t-1}\|l_{f_{i}}(f_{t},f_{t},\zeta_{i})\|_{2}^{2}\right]^{1/2},$
(F.17)

where the second inequality results from Cauchy-Swartz inequality and the last
inequality follows (F.14). Plugging the result back into the (F.16), we
conclude that

$\displaystyle\sum_{t=1}^{T}\mathcal{E}(f_{t})(s_{t},a_{t})$
$\displaystyle\leq\left[d(\epsilon)\sum_{t=1}^{T}\sum_{i=1}^{t-1}\|\mathbb{E}_{\zeta_{i}}[l_{f_{i}}(f_{t},f_{t},\zeta_{i})]\|_{2}^{2}\right]^{1/2}$
$\displaystyle\qquad+2\sqrt{T\epsilon\cdot\min\\{d(\epsilon),T\\}}+({\rm
sp}(V^{*})+2)\min\\{d(\epsilon),T\\}$
$\displaystyle\leq\left[d(\epsilon)\sum_{t=1}^{T}\sum_{i=1}^{t-1}\|\mathbb{E}_{\zeta_{i}}[l_{f_{i}}(f_{t},f_{t},\zeta_{i})]\|_{2}^{2}\right]^{1/2}+({\rm
sp}(V^{*})+3)\min\\{d(\epsilon),T\\}+T\epsilon,$ (F.18)

where the last inequality follows AM-GM inequality. Thus,
$d_{\text{G}}\leq\mathcal{O}(d\log({\rm sp}(V^{*})T/\sqrt{d}\epsilon))$.
Step 2: Bound over transferability coefficient.
Given condition that
$\sum_{i=1}^{t-1}\|\mathbb{E}[l_{f_{i}}(f_{t},f_{t},\zeta_{i})]\|_{2}^{2}\leq\beta$
for all $t\in[T]$, we have

$\displaystyle\sum_{t=1}^{T}\|\mathbb{E}[l_{f_{t}}(f_{t},f_{t},\zeta_{t})]\|_{2}^{2}$
$\displaystyle=\sum_{t=1}^{T}\left[(\theta_{i}-\theta^{*})^{\top}\Big{(}\psi(s_{t},a_{t})+\int_{\mathcal{S}}\phi(s_{t},a_{t},s^{\prime})V_{f_{i}}(s^{\prime}){\rm
d}s^{\prime}\Big{)}\right]^{2}$
$\displaystyle\leq\sum_{t=1}^{T}(W_{t}^{\top}X_{t})^{2}\cdot\mathds{1}\left(\|X_{t}\big{\|}^{2}_{\Sigma_{t}^{-1}}\leq
1\right)+({\rm sp}(V^{*})+2)^{2}\min\\{d(\epsilon),T\\}$
$\displaystyle\leq\sum_{i=1}^{T}(\beta+4\epsilon)\cdot\left(\|X_{t}\big{\|}_{\Sigma_{t}^{-1}}^{2}\wedge
1\right)+({\rm sp}(V^{*})+2)^{2}\min\\{d(\epsilon),T\\}$ $\displaystyle\leq
d(\epsilon)\beta\log T+({\rm sp}(V^{*})^{2}+4{\rm
sp}(V^{*})+6)\min\\{d(\epsilon),T\\}+2T\epsilon^{2},$ (F.19)

where we use a variant of (F.14) and (F.15), following that

$\sum_{t=1}^{T}\mathds{1}\left(\|X_{t}\big{\|}^{2}_{\Sigma_{t}^{-1}}\geq
1\right)\leq\sum_{t=1}^{T}\big{\|}X_{t}\big{\|}^{2}_{\Sigma_{t}^{-1}}\wedge
1\leq\sum_{t=1}^{T}\big{\|}X_{t}\big{\|}_{\Sigma_{t}^{-1}}\wedge 1\leq
d(\epsilon),$

and the last inequality results from a similar proof as (F.17) and (F.18)
using Cauchy-Swartz and AM-GM inequality. Thus, we have
$\kappa_{\text{G}}\leq\mathcal{O}(d\log({\rm sp}(V^{*})T/\sqrt{d}\epsilon))$.
$\Square$

###  F.5 Discussion about Performance on Concrete Examples

In this subsection, we show performance of Loop for specific problems. Loop
achieves an $\tilde{\mathcal{O}}(\sqrt{T})$ regret, which is nearly minimax
optimal in $T$, in linear AMDP and linear mixture AMDP.

##### Linear AMDP

Recall that linear function class is defined as
$\mathcal{H}=\big{\\{}\left(Q,J\right):Q(\cdot,\cdot)=\mathbf{\omega}^{\top}\phi(\cdot,\cdot)\big{|}\
\|\omega\|_{2}\leq\frac{1}{2}{\rm
sp}(V^{*})\sqrt{d},J\in\mathcal{J}_{\mathcal{H}}\big{\\}}$. Consider the
$\rho$-covering number, note that

$\lvert
Q(s,a)-Q^{\prime}(s,a)\rvert\leq\lvert(\omega-\omega^{\prime})^{\top}\phi(s,a)\rvert\leq\sqrt{2}\cdot\|\omega-\omega^{\prime}\|_{1}.$

Based on the Lemma G.9, combine the fact that $|\mathcal{J}_{\mathcal{H}}|\leq
2$ and its $\rho$-covering number
$\mathcal{N}_{\rho}({\mathcal{J}_{\mathcal{H}}})$ is at most $2\rho^{-1}$, we
can get the log covering number of the hypotheses class $\mathcal{H}$ is upper
bounded by

$\log\mathcal{N}_{\mathcal{H}}(\rho)\leq d\log\Big{(}{{\rm
sp}(V^{*})2^{\frac{3}{2}}d^{\frac{3}{2}}}{\rho}^{-2}\Big{)},$ (F.20)

by taking $\alpha=w,\ P=d,\ B={\rm sp}(V^{*})\sqrt{d}/2$. Recall that
Proposition C.1 indicates that

$d_{\text{G}}\leq\mathcal{O}\big{(}d\log\big{(}{\rm
sp}(V^{*})\sqrt{d}\rho^{-1}\big{)}\log
T\big{)},\quad\kappa_{\text{G}}\leq\mathcal{O}\big{(}d\log\big{(}{\rm
sp}(V^{*})\sqrt{d}\rho^{-1}\big{)}\big{)}.$ (F.21)

Combine (F.20), (F.21) and the regret guarantee in Theorem 4.1, we get

$\displaystyle{\rm Reg}(T)$ $\displaystyle\leq{\mathcal{O}}\Big{(}{\rm
sp}(V^{*})\max\\{d_{\text{G}},\kappa_{\text{G}}\\}\sqrt{T\log\big{(}T\mathcal{N}^{2}_{\mathcal{H}\cup\mathcal{G}}(1/T)/\delta\big{)}{\rm
sp}(V^{*})}\Big{)}\leq\tilde{\mathcal{O}}\Big{(}{\rm
sp}(V^{*})^{\frac{3}{2}}d^{\frac{3}{2}}\sqrt{T}\Big{)}.$

For linear AMDPs, our method achieves $\tilde{\mathcal{O}}({\rm
sp}(V^{*})^{\frac{3}{2}}d^{\frac{3}{2}}\sqrt{T})$ regret for both linear and
generalized linear AMDPs. In comparison, the Fopo algorithm (Wei et al.,,
2021) achieves the best-known $\tilde{\mathcal{O}}({\rm
sp}(V^{*})d^{\frac{3}{2}}\sqrt{T})$ regret. Our method incurs an additional
constant ${\rm sp}(V^{*})^{\frac{1}{2}}$ in the regret bound.

##### Linear mixture

Recall that the Proposition C.4 posits that AGEC of the linear mixture probelm
satisfies that
$\max\\{\sqrt{d_{\text{G}}},\kappa_{\text{G}}\\}\leq\mathcal{O}(d\sqrt{\log
T})$. Note that the hypotheses class is defined as

$\mathcal{H}=\\{(\mathbb{P},r):\mathbb{P}(\cdot|s,a)=\theta^{\top}\phi(s,a,\cdot),\
r(s,a)=\theta^{\top}\psi(s,a)|\ \|\theta\|_{2}\leq 1\\}.$

Consider the covering number note that both transition function and reward can
be written as

$\displaystyle\mathrm{(i).}\
|(\mathbb{P}-\mathbb{P}^{\prime})(s^{\prime}|s,a)|=|(\theta-\theta^{\prime})^{\top}\phi(s,a,s^{\prime})|\leq\sqrt{d}\cdot\|\theta-\theta^{\prime}\|_{1},$
$\displaystyle\mathrm{(ii).}\
|(r-r^{\prime})(s,a)|=|(\theta-\theta^{\prime})^{\top}\psi(s,a)|\leq\sqrt{d}\cdot\|\theta-\theta^{\prime}\|_{1}.$

Based on the Lemma G.9, the log covering number of $\mathcal{H}_{\text{LM}}$
is upper bounded by

$\log\mathcal{N}_{\mathcal{H}}(\rho)\leq
2d\log\Big{(}{{d}^{\frac{3}{2}}}{\rho^{-1}}\Big{)},$

by taking $\alpha=\theta,\ P=d,\ B=1$. Combine results above and Theorem 4.1,
we get

$\displaystyle{\rm Reg}(T)$ $\displaystyle\leq{\mathcal{O}}\Big{(}{\rm
sp}(V^{*})\max\\{d_{\text{G}},\kappa_{\text{G}}\\}\sqrt{T\log\big{(}T\mathcal{N}^{2}_{\mathcal{H}\cup\mathcal{G}}(1/T)/\delta\big{)}{\rm
sp}(V^{*})}\Big{)}\leq\tilde{\mathcal{O}}\Big{(}{\rm
sp}(V^{*})^{\frac{3}{2}}d^{\frac{3}{2}}\sqrt{T}\Big{)}.$

At our best knowledge, the UCRL2-VTR (Wu et al.,, 2022) achieves the best
$\tilde{\mathcal{O}}(Dd\sqrt{T})$ regret for linear mixture AMDP, where $D$ is
the diameter under communicating AMDP assumption and it is provable that ${\rm
sp}(V^{*})\leq D$ (Wang et al.,, 2022). We remark that the two algorithms are
incomparable under different assumptions and both achieve a near minimax
optimal regret at $\tilde{\mathcal{O}}(\sqrt{T})$.

##  Appendix G Technical Lemmas

In this section, we provide useful technical lemmas used in later theoretical
analysis. Most are directly borrowed from existing works and proof of modified
lemmas is provided in Section G.1.

######  Lemma G.1.

Given function class $\Phi$ defined on $\mathcal{X}$, and a family of
probability measures $\Gamma$ over $\mathcal{X}$. Suppose sequence
$\\{\phi_{k}\\}_{k=1}^{K}\subset\Phi$ and
$\\{\mu_{k}\\}_{k=1}^{K}\subset\Gamma$ satisfy that for all $k\in[K]$,
$\sum_{t=1}^{k-1}(\mathbb{E}_{\mu_{t}}[\phi_{k}])^{2}\leq\beta$. Then, for all
$k\in[K]$, we have

$\sum_{t=1}^{k}\mathds{1}\Big{(}\big{|}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{|}>\epsilon\Big{)}\leq\Big{(}\frac{\beta}{\epsilon^{2}}+1\Big{)}{\rm
dim_{DE}(\Phi,\Pi,\epsilon)}.$

_Proof._ See Lemma 43 of Jin et al., (2021) for detailed proof.

######  Lemma G.2 (Pigeon-hole principle).

Given function class $\Phi$ defined on $\mathcal{X}$ with $|\phi(x)|\leq C$
for all $\phi\in\Phi$ and $x\in\mathcal{X}$, and a family of probability
measure over $\mathcal{X}$. Suppose sequence
$\\{\phi_{k}\\}_{k=1}^{K}\subset\Phi$ and
$\\{\mu_{k}\\}_{k=1}^{K}\subset\Gamma$ satisfy that for all $k\in[K]$, it
holds $\sum_{t=1}^{k-1}(\mathbb{E}_{\mu_{t}}[\phi_{k}])^{2}\leq\beta$. Let
$d_{\rm DE}={\rm dim_{DE}(\Phi,\Gamma,\epsilon)}$ be the DE dimension, then
for all $k\in[K]$ and $\epsilon>0$, we have

$\sum_{t=1}^{k}\Big{|}\mathbb{E}_{\mu_{t}}[\phi_{t}]\Big{|}\leq 2\sqrt{d_{\rm
DE}\beta k}+\min\\{k,d\\}C+k\epsilon,$

and

$\sum_{t=1}^{k}\Big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\Big{]}^{2}\leq d_{\rm
DE}\beta\log k+\min\\{k,d\\}C^{2}+k\epsilon^{2}.$

_Proof._ See Section G.1.1.

######  Lemma G.3.

Given function class $\Phi$ defined on $\mathcal{X}$ with $|\phi(x)|\leq C$
for all $\phi\in\Phi$ and $x\in\mathcal{X}$, and a family of probability
measure over $\mathcal{X}$. Let $d_{\rm DE}={\rm
dim_{DE}(\Phi,\Gamma,\epsilon)}$ be the DE dimension, then for all $k\in[K]$
and $\epsilon>0$, we have

$\sum_{t=1}^{k}\Big{|}\mathbb{E}_{\mu_{k}}[\phi_{k}]\Big{|}\leq\left[{d_{\rm
DE}\left(1+\log
K\right)}\sum_{k=1}^{K}\sum_{t=1}^{k-1}(\mathbb{E}_{\mu_{t}}[\phi_{k}])^{2}\right]^{1/2}+\min\\{d_{\rm
DE},k\\}C+k\epsilon.$

_Proof._ See Section G.1.2.

######  Lemma G.4 ($d$-upper bound).

Let $\Phi$ and $\Psi$ be sets of $d$-dimensional vectors and $\|\phi\|_{2}\leq
B_{\phi}$, $\|\psi\|_{2}\leq B_{\psi}$ for any $\phi\in\Phi$ and
$\psi\in\Psi$. If there exists set $(\phi_{1},\dots,\phi_{m})$ and
$(\psi_{1},\dots,\psi_{m})$such that for all $t\in[m]$,
$\sqrt{\sum_{k=1}^{t-1}\langle\phi_{t},\psi_{k}\rangle^{2}}\leq
c_{1}\varepsilon$ and
$\lvert\langle\phi_{t},\psi_{t}\rangle\rvert>c_{2}\varepsilon$, where
$c_{1}\geq c_{2}>0$ is a constant and $\varepsilon>0$, then the number of
elements in set is bounded by
$m\leq\mathcal{O}\big{(}d\log(B_{\phi}B_{\psi}/\varepsilon)\big{)}$.

_Proof._ See Section G.1.3.

######  Lemma G.5.

For any sequence of positive reals $x_{1},\dots,x_{m}$, it holds that
$\frac{\sum_{i=1}^{m}x_{i}}{\sqrt{\sum_{i=1}^{m}ix_{i}^{2}}}\leq\sqrt{1+\log
n}.$

_Proof._ See Lemma 6 in Dann et al., (2021) for detailed proof.

######  Lemma G.6.

Let $\\{x_{i}\\}_{i\in[t]}$ be a sequence of vectors defined over Hilbert
space $\mathcal{X}$. Let $\Lambda_{0}$ be a positive definite matrix and
$\Lambda_{t}=\Lambda_{0}+\sum_{i=1}^{t-1}x_{t}x_{t}^{\top}$. It holds that

$\sum_{i=1}^{t}\|x_{t}\|_{\Lambda_{t}^{-1}}^{2}\wedge 1\leq
2\log\left(\frac{\det\Lambda_{t+1}}{\det\Lambda_{0}}\right).$

_Proof._ See Elliptical Potential Lemma (EPL) in Dani et al., (2008) for a
detailed proof.

######  Lemma G.7 (Freedman’s inequality).

Let $X_{1},\dots,X_{T}$ be a real-valued martingale difference sequence
adapted to filtration $\\{\mathscr{F}_{t}\\}_{t=1}^{T}$. Assume for all
$t\in[T]$ $X_{t}\leq R$, then for any $\eta\in(0,1/R)$, with probability
greater than $1-\delta$

$\sum_{t=1}^{T}X_{t}\leq\mathcal{O}\Big{(}\eta\sum_{t=1}^{T}\mathbb{E}\big{[}X_{t}^{2}|\mathscr{F}_{t}\big{]}+\frac{\log(1/\delta)}{\eta}\Big{)},$

_Proof._ See Lemma 7 in Agarwal et al., (2014) for detailed proof.

######  Lemma G.8 (Scaling lemma).

Let $\phi:\mathcal{S}\times\mathcal{A}\mapsto\mathbb{R}^{d}$ be a
$d$-dimensional feature mapping, there exists an invertible linear
transformation $A\in\mathbb{R}^{d\times d}$ such that for any bounded function
$f:\mathcal{S}\times\mathcal{A}\mapsto\mathbb{R}$ and
$\mathbf{z}\in\mathbb{R}^{d}$ defined by

$f(s,a)=\phi(s,a)^{\top}\mathbf{z},$

we have $\|A\phi(s,a)\|\leq 1$ and $\|A^{-1}z\|\leq\sup_{s,a}\lvert
f\rvert\sqrt{d}$ for all $(s,a)\in\mathcal{S}\times\mathcal{A}$.

_Proof._ See Lemma 8 in Wei et al., (2021) for detailed proof.
In Theorem 4.1, the proved regret contains the logarithmic term of the
$1/T$-covering number of the function classes
$\mathcal{N}_{\mathcal{H}}(1/T)$, which can be regarded as a surrogate
cardinality of the function class $\mathcal{H}$. Here, we provide a formal
definition of $\rho$-covering and the upper bound of $\rho$-covering number.

######  Definition 17 ($\rho$-covering).

The $\rho$-covering number of a function class $\mathcal{F}$ is the minimum
integer $t$ satisfying that there exists subset
$\mathcal{F^{\prime}}\subseteq\mathcal{F}$ with $|\mathcal{F^{\prime}}|=t$
such that for any $f\in\mathcal{F}$ we can find a correspondence
$f^{\prime}\in\mathcal{F^{\prime}}$ that it holds
$\|f-f^{\prime}\|_{\infty}\leq\rho$.

######  Lemma G.9 ($\rho$-covering number).

Let $\mathcal{F}$ be a function defined over $\mathcal{X}$ that can be
parametrized by $\bm{\alpha}=(\alpha_{1},\dots,\alpha_{P})\in\mathbb{R}^{P}$
with $\lvert\alpha_{i}\rvert\leq B$ for all $i\in[P]$. Suppose that for any
$f,f^{\prime}\in\mathcal{F}$ it holds that ${\sup}_{x\in\mathcal{X}}\lvert
f(x)-f^{\prime}(x)\rvert\leq L\|\bm{\alpha}-\bm{\alpha}^{\prime}\|_{1}$ and
let $\mathcal{N}_{\mathcal{F}}(\rho)$ be the $\rho$-covering number of
$\mathcal{F}$, then

$\log\mathcal{N}_{\mathcal{F}}(\rho)\leq
P\log\Big{(}\frac{2BLP}{\rho}\Big{)}.$

_Proof._ See Lemma 12 in Wei et al., (2021) for detailed proof.

###  G.1 Proof of Technical Lemmas

In this subsection, we present the proofs of technical auxiliary lemmas with
modifications.

####  G.1.1 Proof of Lemma G.2

_Proof of Lemma G.2._ The first statement is directly from Lemma 41 in Jin et
al., (2021), and the second statement follows a similar procedure as below.
Note that Lemma G.1 suggests that

$\sum_{t=1}^{k}\mathds{1}\Big{(}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}>\epsilon^{2}\Big{)}\leq\Big{(}\frac{\beta}{\epsilon^{2}}+1\Big{)}{\rm
dim_{DE}(\Phi,\Gamma,\epsilon)},$

and note that the sum of squared expectation can be decomposed as

$\displaystyle\sum_{t=1}^{k}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}$
$\displaystyle=\sum_{t=1}^{k}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\mathds{1}\Big{(}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}>\epsilon^{2}\Big{)}+\sum_{t=1}^{k}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\mathds{1}\Big{(}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\leq\epsilon^{2}\Big{)}$
$\displaystyle\leq\sum_{t=1}^{k}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\mathds{1}\Big{(}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}>\epsilon^{2}\Big{)}+k\epsilon^{2}.$
(G.1)

Assume sequence
$\big{[}\mathbb{E}_{\mu_{1}}[\phi_{1}]\big{]}^{2},\dots,\big{[}\mathbb{E}_{\mu_{k}}[\phi_{k}]\big{]}^{2}$
are sorted in the decreasing order and consider $t\in[k]$ such that
$[\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}>\epsilon^{2}$, there exists a
constant $\alpha\in(\epsilon^{2},[\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2})$
satisfying

$t\leq\sum_{i=1}^{k}\mathds{1}\Big{(}\big{[}\mathbb{E}_{\mu_{i}}[\phi_{i}]\big{]}^{2}>\alpha\Big{)}\leq\Big{(}\frac{\beta}{\alpha}+1\Big{)}{\rm
dim_{DE}(\Phi,\Gamma,\sqrt{\alpha})}\leq\Big{(}\frac{\beta}{\alpha}+1\Big{)}{\rm
dim_{DE}(\Phi,\Gamma,\epsilon)},$

where the last inequality is based on the fact that the DE dimension is
monotonically decreasing in terms of $\epsilon$ as proposed in Jin et al.,
(2021). Denote $d_{\rm DE}={\rm dim_{DE}(\Phi,\Gamma,\epsilon)}$ and the
inequality above implies that $\alpha\leq{d_{\rm DE}\beta}/{t-d}$. Thus, we
have $\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\leq{d_{\rm
DE}\beta}/{t-d}$. Beside, based on the definition we also have
$\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\leq C^{2}$ and thus
$\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\leq\min\\{{d_{\rm
DE}\beta}/{t-d},C^{2}\\}$, then

$\displaystyle\sum_{t=1}^{k}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}\mathds{1}\Big{(}\big{[}\mathbb{E}_{\mu_{t}}[\phi_{t}]\big{]}^{2}>\epsilon^{2}\Big{)}$
$\displaystyle\leq\min\\{d_{\rm
DE},k\\}C^{2}+\sum_{t=d+1}^{k}\biggl{(}\frac{d_{\rm DE}\beta}{t-d_{\rm
DE}}\biggl{)}$ $\displaystyle\leq\min\\{d_{\rm DE},k\\}C^{2}+d_{\rm
DE}\cdot\beta\int_{0}^{k}\frac{1}{t}{\rm d}t$ $\displaystyle\leq\min\\{d_{\rm
DE},k\\}C^{2}+d_{\rm DE}\cdot\beta\log k.$ (G.2)

Combine (G.1) and (G.2), then finishes the proof. $\Square$

####  G.1.2 Proof of Lemma G.3

We remark that the proof provided in this subsection follows the almost same
procedure as Lemma 3.16 in Zhong et al., (2022) with adjustment, and we
preserve it for comprehension.
_Proof of Lemma G.3._ Denote $d_{\rm DE}={\rm
dim_{DE}}(\Phi,\Gamma,\epsilon)$,
$\widehat{\epsilon}_{t,k}=|\mathbb{E}_{\mu_{t}}[\phi_{k}]|$ and
${\epsilon}_{t,k}=\widehat{\epsilon}_{t,k}\mathds{1}(\widehat{\epsilon}_{t,k}>\epsilon)$
for $t,k\in[K]$, $\mu_{t}\in\Gamma$ and $\phi_{k}\in\Phi$. The proof follows
the procedure below. Consider $K$ empty buckets $B_{0},\dots,B_{K-1}$ as
initialization, and we examine $\epsilon_{k,k}$ one by one for all $k\in[K]$
as below:

  1. Case 1

If $\epsilon_{k,k}=0$, i.e., $\widehat{\epsilon}_{k,k}\leq\epsilon$, then
discard it.

  2. Case 2

If $\epsilon_{k,k}>0$, i.e., $\widehat{\epsilon}_{k,k}>\epsilon$, at bucket
$j$ we add $k$ into $B_{j}$ if $\sum_{t\leq k-1,t\in
B_{j}}(\epsilon_{t,k})^{2}\leq(\epsilon_{k,k})^{2}$, otherwise we continue
with the next bucket $B_{j+1}$.

Denote by $b_{k}$ the index of bucket that at step $k$ the non-zero
$\epsilon_{k,k}$ falls in, i.e. $k\in B_{b_{k}}$. Based on the rule above, it
holds that

$\sum_{k=1}^{K}\sum_{t=1}^{k-1}(\epsilon_{t,k})^{2}\geq\sum_{k=1}^{K}\sum_{0\leq
j\leq b_{k}-1,b_{k}\geq 1}\sum_{t\leq k-1,t\in
B_{j}}(\epsilon_{t,k})^{2}\geq\sum_{k=1}^{K}b_{k}\cdot(\epsilon_{k,k})^{2},$

where the first inequality arises from $\\{t\in B_{j}:t\leq k-1,\ 0\leq j\leq
b_{k}-1,b_{k}\geq 1\\}\subseteq[k-1]$ due to the discarding of the $b_{k}$th
bucket, and the second equality directly follows the allocation rule such that
$\sum_{t\leq k-1,t\in B_{j}}(\epsilon_{t,k})^{2}\geq(\epsilon_{k,k})^{2}$ for
any $j\leq b_{k}-1$. Recall that based on the definition of distributional
eluder (DE) dimension, it is suggested the size $|B_{j}|$ is no larger than
$d_{\rm DE}$. Then,

$\displaystyle\sum_{k=1}^{K}b_{k}(\epsilon_{k,k})^{2}=\sum_{j=1}^{K-1}j\sum_{t\in
B_{j}}(\epsilon_{t,t})^{2}$ (re-summation)
$\displaystyle\geq\sum_{j=1}^{K-1}\frac{j}{|B_{j}|}\left(\sum_{t\in
B_{j}}\epsilon_{t,t}\right)^{2}\geq\sum_{j=1}^{K-1}\frac{j}{d_{\rm
DE}}\left(\sum_{t\in B_{j}}\epsilon_{t,t}\right)^{2}$ ($|B_{j}|\leq d_{\rm
DE}$) $\displaystyle\geq\left({d_{\rm DE}\left(1+\log
K\right)}\right)^{-1}\left(\sum_{j=1}^{K-1}\sum_{t\in
B_{j}}\epsilon_{t,t}\right)^{2}=\left({d_{\rm DE}\left(1+\log
K\right)}\right)^{-1}\left(\sum_{t\in[K]\backslash
B_{0}}\epsilon_{t,t}\right)^{2},$ (G.3)

where the second inequality follows Lemma G.5. Combine the (G.1.2) and (G.3)
above, we have

$\displaystyle\sum_{k=1}^{K}\widehat{\epsilon}_{k,k}$
$\displaystyle\leq\sum_{k=1}^{K}{\epsilon}_{k,k}+K\epsilon\leq\sum_{t\in[K]\backslash
B_{0}}{\epsilon}_{t,t}+\min\\{d_{\rm DE},K\\}\|\phi\|_{\infty}+K\epsilon$
$\displaystyle\leq\left[{d_{\rm DE}\left(1+\log
K\right)}\sum_{k=1}^{K}\sum_{t=1}^{k-1}(\epsilon_{t,k})^{2}\right]^{1/2}+\min\\{d_{\rm
DE},K\\}C+K\epsilon$ $\displaystyle\leq\left[{d_{\rm DE}\left(1+\log
K\right)}\sum_{k=1}^{K}\sum_{t=1}^{k-1}(\widehat{\epsilon}_{t,k})^{2}\right]^{1/2}+\min\\{d_{\rm
DE},K\\}C+K\epsilon.$

Substitute the definition
$\widehat{\epsilon}_{t,k}=|\mathbb{E}_{\mu_{t}}[\phi_{k}]|$ back into the
inequality, then finishes the proof. $\Square$

####  G.1.3 Proof of Lemma G.4

_Proof of Lemma G.4._ For notation simplicity, denote
$\Lambda_{t}=\sum_{k=1}^{t-1}\psi_{t}\psi_{t}^{\top}+\frac{\varepsilon^{2}}{B_{\phi}^{2}}\cdot\mathbf{I}$,
then for all $t\in[m]$ we have
$\|\phi_{t}\|_{\Lambda_{t}}\leq\sqrt{\sum_{k=1}^{t-1}(\phi_{t}^{\top}\psi_{k})^{2}+\frac{\varepsilon^{2}}{B_{\phi}^{2}}\|\phi_{t}\|_{2}^{2}}=\sqrt{c_{1}^{2}+1}\
\varepsilon$ based on the given condition. Using the Cauchy-Swartz inequality
and results above, then it holds
$\|\psi_{t}\|_{\Lambda_{t}^{-1}}\geq{\lvert\langle\phi_{t},\psi_{t}\rangle\rvert}/{\|\phi_{t}\|_{\Lambda_{t}}}={c_{2}}/{\sqrt{c_{1}^{2}+1}}.$
On one hand, the matrix determinant lemma ensures that

$\det\Lambda_{m}=\det\Lambda_{0}\cdot\prod_{t=1}^{m-1}\big{(}1+\|\psi_{t}\|_{\Lambda_{t}^{-1}}^{2}\big{)}\geq\left(1+\frac{c_{2}^{2}}{1+c_{1}^{2}}\right)^{m-1}\left(\frac{\varepsilon^{2}}{B_{\phi}^{2}}\right)^{d}.$
(G.4)

On the other hand, according to the definition of $\Lambda_{t}$, we have

$\det\Lambda_{m}\leq\left(\frac{{\rm Tr}(\Lambda_{m})}{d}\right)^{d}\
\leq\left(\sum_{k=1}^{t-1}\frac{\|\psi_{k}\|_{2}^{2}}{d}+\frac{\varepsilon^{2}}{B_{\phi}^{2}}\right)^{d}\leq\left(\frac{B_{\psi}^{2}(m-1)}{d}+\frac{\varepsilon^{2}}{B_{\phi}^{2}}\right)^{d}.$
(G.5)

Combine (G.4) and (G.5), if we take logarithms at both sides, then we have

$m\leq
1+{d\log\left(\frac{B_{\phi}^{2}B_{\psi}^{2}(m-1)}{d\varepsilon^{2}}+1\right)}\biggl{/}{\log\left(1+\frac{c_{2}^{2}}{1+c_{1}^{2}}\right)}.$

After simple calculations, we can obtain that $m$ is upper bounded by
$\mathcal{O}\big{(}d\log(B_{\phi}B_{\psi}/\varepsilon)\big{)}$. $\Square$

##  Appendix H Supplementary Discussions

###  H.1 Proof Sketch of MLE-based Results

In this subsection, we provide the proof sketch of Theorem B.1. We first
introduce several useful lemmas, which is the variant of ones in Appendix D
for MLE-based problems, and most have been fully researched in Liu et al.,
(2022); Liu et al., 2023a ; Xiong et al., (2023). As there’s no significant
technical gap between episodic and average-reward for model-based problems, we
only provide a proof sketch.

######  Lemma H.1 (Akin to Lemma D.1).

Under Assumptions 1-2, Mle-Loop is an optimistic algorithm such that it
ensures $J_{t}\geq J^{*}$ for all $t\in[T]$ with probability greater than
$1-\delta$.

_Proof Sketch of Lemma H.1._ See Proposition 13 in Liu et al., (2022) with
slight modifications. $\Square$

######  Lemma H.2 (Akin to Lemma D.2).

For fixed $\rho>0$ and a pre-determined optimistic parameter
$\beta=c(\log\big{(}T\mathcal{B}_{\mathcal{H}}(\rho)/\delta)+T\rho\big{)}$
where constant $c>0$, it holds that

$\sum_{i=1}^{t-1}\|\mathbb{E}_{\zeta_{i}}[l_{f_{i}}(f_{t},f_{t},\zeta_{i})]\|_{1}=\sum_{i=1}^{t-1}\text{TV}\big{(}\mathbb{P}_{f_{t}}(\cdot|s_{i},a_{i}),\mathbb{P}_{f^{*}}(\cdot|s_{i},a_{i})\big{)}\leq\mathcal{O}(\sqrt{\beta
t}),$ (H.1)

for all $t\in[T]$ with probability greater than $1-\delta$.

_Proof Sketch of Lemma H.1._ See Proposition 14 in Liu et al., (2022) with
slight modifications. $\Square$

######  Lemma H.3 (Akin to Lemma D.3).

Let $\mathcal{N}(T)$ be the switching cost with time horizon $T$, given fixed
covering coefficient $\rho>0$ and pre-determined optimistic parameter
$\beta=c\big{(}\log\big{(}T\mathcal{B}_{\mathcal{H}}(\rho)/\delta\big{)}+T\rho\big{)}$
where $c$ is a large enough constant, with probability greater than
$1-2\delta$ we have

$\mathcal{N}(T)\leq\mathcal{O}\big{(}\kappa_{\text{G}}\cdot{\rm poly}(\log
T)+\beta^{-1}T\epsilon^{2}\big{)},$

where $\kappa_{\text{G}}$ is the transferability coefficient with respect to
$\text{\rm MLE-AGEC}(\mathcal{H},\\{l_{f^{\prime}}\\},\epsilon)$.

_Proof Sketch of Lemma H.3._ The proof is almost the same as Lemma D.3.

##### Step 1:

Bound the difference of discrepancy between the minimizer and $f^{*}$.
As proposed in Proposition 14, Liu et al., (2022),
$0\leq\sum_{i=1}^{t}\text{TV}\big{(}\mathbb{P}_{f^{*}}(\cdot|s_{i},a_{i}),\mathbb{P}_{g_{i}}(\cdot|s_{i},a_{i})\big{)}^{2}\leq\beta$
holds with high probability if the update happens at $t$-th step. Based on the
AM-GM inequlaity, we have

$0\leq\sum_{i=1}^{t}\text{TV}\big{(}\mathbb{P}_{f^{*}}(\cdot|s_{i},a_{i}),\mathbb{P}_{g_{i}}(\cdot|s_{i},a_{i})\big{)}\leq\sqrt{\beta
t}.$ (H.2)

##### Step 2:

Bound the expected discrepancy between updates.
Note that for all $t+1\in[T]$, the update happens only if

$\sum_{i=1}^{t}\text{TV}\big{(}\mathbb{P}_{f_{t}}(\cdot|s_{i},a_{i}),\mathbb{P}_{g_{i}}(\cdot|s_{i},a_{i})\big{)}>3\sqrt{\beta
t}.$ (H.3)

Combine the (H.2) and (H.3) above, and apply the triangle inequality, we have

$\displaystyle\quad\sum_{i=1}^{t}\text{TV}\big{(}\mathbb{P}_{f_{t}}(\cdot|s_{i},a_{i}),\mathbb{P}_{f^{*}}(\cdot|s_{i},a_{i})\big{)}$
$\displaystyle\geq\sum_{i=1}^{t}\text{TV}\big{(}\mathbb{P}_{f_{t}}(\cdot|s_{i},a_{i}),\mathbb{P}_{g_{t}}(\cdot|s_{i},a_{i})\big{)}-\text{TV}\big{(}\mathbb{P}_{f^{*}}(\cdot|s_{i},a_{i}),\mathbb{P}_{g_{t}}(\cdot|s_{i},a_{i})\big{)}\geq
2\sqrt{\beta t}.$

and the construction of confidence set ensures that
$\sum_{i=1}^{\tau_{t}}\text{TV}\big{(}\mathbb{P}_{f_{t}}(\cdot|s_{i},a_{i}),\mathbb{P}_{f^{*}}(\cdot|s_{i},a_{i})\big{)}\leq\sqrt{\beta
t}$ with high probability (Liu et al.,, 2022, Proposition 14). Recall the
definition of the MLE-transferability coefficient, then the switching cost can
be bounded following the same argument in Lemma D.3. $\Square$
_Proof Sketch of Theorem B.1._ Recall that

$\displaystyle{\rm Reg}(T)$
$\displaystyle\leq\underbrace{\sum_{i=1}^{T}\mathcal{E}(f_{t})(s_{t},a_{t})}_{\displaystyle\text{Bellman
error}}+\underbrace{\sum_{t=1}^{T}\Big{(}\mathbb{E}_{s_{t+1}\sim\mathbb{P}(\cdot|s_{t},a_{t})}[V_{t}(s_{t+1})]-V_{t}(s_{t})\Big{)}}_{\displaystyle\text{
Realization error}},$ (H.4)

where the inequality follows the optimism in Lemma H.1. Combine Lemma H.1,
Lemma H.3 and the definition of MLE-AGEC (see Definition 9), then we can
finish the proof. $\Square$

###  H.2 Extended Value Iteration (EVI) for Model-Based Hypotheses

In model-based problems, the discrepancy function sometimes relies on the
optimal state bias function $V_{f}$ and optimal average-reward $J_{f}$ (see
linear mixture model in Section C). In this section, we provide an algorithm,
extended value iteration (EVI) proposed in Auer et al., (2008), to output the
optimal function and average-reward under given a model-based hypothesis
$f=(\mathbb{P}_{f},r_{f})$. See Algorithm 3 for complete pseudocode. The
convergence of EVI is guaranteed by the theorem below.

######  Theorem H.4.

UnderAssumption 1, there exists a unique centralized solution pair
$(Q^{*},J^{*})$ to the Bellman optimality equation for any AMDP
$\mathcal{M}_{f}$ characterized by hypothesis $f\in\mathcal{H}$. Then, if the
extended value iteration (EVI) is stopped under the condition that

$\max_{s\in\mathcal{S}}\\{V^{(i+1)}(s)-V^{(i)}(s)\\}-\min_{s\in\mathcal{S}}\\{V^{(i+1)}(s)-V^{(i)}(s)\\}\leq\epsilon,$

then the achieved greedy policy $\pi^{(i)}$ is $\epsilon$-optimal such that
$J^{\pi^{(i)}}_{\mathcal{M}_{f}}\geq J^{*}_{\mathcal{M}_{f}}+\epsilon$.

_Proof Sketch:_ See Theorem 12 in Auer et al., (2008).

Algorithm 3 Extended Value Iteration (EVI)

1:hypothesis $f=(\mathbb{P}_{f},r_{f})$, desired accuracy level $\epsilon.$

2:$V^{(0)}(s)=0$ for all $s\in\mathcal{S}$, $J^{(0)}=0$ and counter $i=0$.

3:repeat

4: for $s\in\mathcal{S}$ and $a\in\mathcal{A}$ do

5: Set $Q^{(i)}(s,a)\leftarrow
r_{f}(s,a)+\mathbb{E}_{s^{\prime}\sim\mathbb{P}_{f}(s,a)}[V^{(i)}(s^{\prime})]-J^{(i)}$

6: Update $V^{(i+1)}(s)\leftarrow\max_{a\in\mathcal{A}}Q^{(i)}(s,a)$

7: Update counter $i\leftarrow i+1$

8:until
$\max_{s\in\mathcal{S}}\\{V^{(i+1)}(s)-V^{(i)}(s)\\}-\min_{s\in\mathcal{S}}\\{V^{(i+1)}(s)-V^{(i)}(s)\\}\leq\epsilon$


∎

11institutetext: H. Çakmak22institutetext: Istanbul University Science
Faculty, Astronomy and Space Science Department
34119 Beyazıt / Istanbul - Turkey
22email<EMAIL_ADDRESS>
# Computer-aided measurement
of the heliographic coordinates of sunspot groups

H. Çakmak

(Received: date / Accepted: date)

###### Abstract

Heliographic coordinates are used to identify the positions of solar features,
especially the sunspot groups on the Sun’s surface. Tracking the positions of
sunspot groups provides information about solar rotation and the movement
behavior of sunspot groups over Solar Cycles. Two heliographic coordinates are
defined: Carrington and Stonyhurst. The calculations used here depend on three
solar parameters, position angle P, latitude angle $\it{B_{0}}$ and the
starting longitude $\it{L_{0}}$. These values are calculated by using
Astronomical Almanac for the observation time. In this study, a computer
program called Computer Aided Measurement for Sunspots (CAMS) is presented.
The main aim of the program is to determine the heliographic latitude and
longitude of the sunspot groups besides other features like the latitudinal
and longitudinal length, rectangular area and the tilt angle. This is
accomplished by generating the corresponding Stonyhurst disk for the P and
$\it{B_{0}}$ angles of the time of observation and superimposing it onto the
scanned drawing. Since 2009, CAMS is being used to process daily solar
drawings at the Istanbul University Observatory.

###### Keywords:

computer-aided measurement, heliographic coordinates, sunspots

##  1 Introduction

Sunspots are the first noticeable feature on the visible surface of the Sun.
The number of spots on the Sun’s surface increase and decrease during a
specific time range which is 11 years on average. This period is known as
Solar Cycle or Sunspot Cycle. During the minimum phase of a cycle, no sunspots
are seen, but, at the maximum phase, many sunspots can be observed on the
Sun’s surface (Fig. 1). Sometimes big-sized spots are visible to the naked-eye
under proper conditions. These sunspots have attracted attention through the
centuries. Therefore, since the 17th century, sunspot observations are being
continued without interruption.

Figure 1: Two typical views of a Sunspot Cycle. $\it Left$: Minimum, $\it
Right$: maximum phase. Images were taken from the SOHO (Solar and Heliospheric
Observatory) archive.

One of the most known Sun observers, Galileo showed that movement of the
sunspots across the apparent disk of the Sun is similar to rotating across a
spherical surface, rather than a flat disk. This concept made it possible to
develop a spherical coordinate system of solar latitude and longitude, similar
to the latitude and longitude system on Earth. Today, two heliographic
coordinates are defined: Carrington and Stonyhurst. They differ on whether the
lines of longitude are fixed for the Sun’s rotation (in Carrington) or fixed
for the observer (in Stonyhurst) Carrington1863 ; Teague1996 ; Thompson2006 ;
SanchezBajo2013 . More explanations about both heliographic coordinates can be
found in the articles of ThompsonThompson2006  and Sanchez-Bajo & Vaquero
SanchezBajo2013 .

More properties about the positions of the sunspot groups were discovered with
increasing solar observations in the past. German astronomer Christoph
Scheiner showed that the solar rotation rate varied with latitude. This meant
that the Sun does not rotate as a solid body. Carrington published his
observations of sunspot positions in 1863. And, he estimated the correlation
between solar rotation rate and latitudeSanchezBajo2013 . In 1904, Edward
Maunder showed the latitude of sunspot emergence changes from the beginning to
the end of the sunspot cycle in a diagram called butterfly. All these movement
behaviors of sunspot groups provide information about the Sun: e.g. the
rotation period of the Sun, the nature of the solar cycle and the existence of
differential rotation.

Most of observatories spread around the world are continuing to draw the
sunspots manually. But, on the other hand, technological developments are
making it also necessary to process these sunspot drawings in computer
environment. Some of the observatories transferred their old drawings into
digital environment via scanners to process them later. Some studies are done
on determining the positions of sunspot groups from sunspot drawings. The most
known applications are the Helio programs developed by Peter
MeadowsMeadows2002 . These are sufficient in some extents, but not adequate
for our own needs. Moreover, similar programs are developed by different
people: the program HSUNSPOTS is developed by Cristoet al.Cristo2011  to
analyze ancient solar drawings and the program DigiSun is used in the SIDC of
the Royal Observatory of BelgiumClette2011 .

The CAMS program which will be introduced here has been developed with a
different perspective from the others. It aims to obtain as many features of
the sunspot groups as possible. Moreover, not only the scanned solar drawings
but also CCD and solar digital images can be used with the program by
adjusting their size and resolution properly. In this program, a Stonyhurst
disk overlaying technique is used. This technique gives the opportunity for
calculating most of the properties of the sunspot groups easily. All
operations are carried out in two simple steps, superimposing Stonyhurst disk
onto image and, marking and classifying the sunspot groups separately. Most of
the other processes are performed automatically such as calculating the solar
parameters at the time of observation, the heliographic coordinates and other
measurable parameters of the sunspot groups and, altering and sorting the
groups by their latitude and longitude.

In the sunspot observations, the appearance of the Stonyhurst disk depends
mainly on the P and $\it{B_{0}}$ angles. Therefore, the calculations of these
solar parameters have a special importance and are introduced in detail in
Section 2. The algorithm for drawing the Stonyhurst disks is explained in
Section 3. And, the basics of CAMS’s workings are given in Section 4. The
advantages, importance and possible future developments of the CAMS are
presented in the Discussion Section.

##  2 Calculations of the Solar Parameters

Any coordinate system used in astronomy must be set up using a plane and a
starting point. For heliographic coordinates the plane in question is the
equator of the Sun, but the starting point is a little difficult to describe,
because there are no permanent features on the surface of the Sun, which can
be taken as a fixed point. Carrington defined this virtual point as a point
occupied by the ascending node of the solar equator on the ecliptic at noon on
January 1st 1854. By this assumption, heliographic coordinates of a point on
the surface of the Sun are defined by the spherical definitions as shown in
Fig. 2aDuffett1988 .

Figure 2: (a) Definition of the heliographic coordinates of a point X on the
surface of the Sun according to the spherical definitions. (b) Position of the
rotation axis of the Sun with respect to the rotation axis of the Earth and
north $\&$ south line of the solar projection disk.

The line of sight is shown by the line towards the Earth in Fig. 2a and it
shows the direction to the center of the apparent solar disk, which is labeled
as E in Fig. 2b. Also, the rotation axis of the Sun is shown by E$P_{N}$ line
and the angle between this line and the rotation axis of the Earth is P, the
position angle of the Sun. The angle $\widehat{FCE}$ is the latitude angle
$B_{o}$ and represents the angular distance of the Sun’s equator from the
center of the apparent solar disk. In accordance with the definitions as shown
in Fig. 2a, the starting longitude $L_{o}$ (point O) and $B_{o}$ are
calculated by SmartGreen1977

$\displaystyle\begin{aligned}
L_{o}&=\arctan\left(\tan(\lambda_{\odot}-\Omega)\cos I\right)+M,\\\\[4.0pt]
B_{o}&=\arcsin\left(\sin(\lambda_{\odot}-\Omega)\sin I\right),\end{aligned}$

where $\Omega$ is the angle between the ascending node of the Sun’s equator on
the ecliptic and the point of the vernal equinox ($\Upsilon$N angle),
$\lambda_{\odot}$ is the ecliptical longitude of the Sun, I is the inclination
of the Sun’s equator with respect to the ecliptic plane and its value is
7.25∘. M is the angular distance of the ascending node of the Sun’s equator on
the ecliptic from the starting longitude. Here, M is given by Duffett1988

$\displaystyle\begin{aligned}
M=360-M^{\prime}\end{aligned}\quad\quad\text{with}\quad\quad\begin{aligned}
M^{\prime}&=\frac{360}{25.38}(JD-2398220.0)\end{aligned}$

where JD is the Julian date for the day in question, 2 398 220 is the Julian
date at noon on 1 January 1854. M ′ must be reduced to the range 0 - 360
degrees.

P is the angle between both rotation axis directions of the Sun and the Earth.
But, this cannot be calculated directly. Therefore, both axis inclinations
with respect to ecliptic are calculated separately and P is the sum of both.
So, let $\theta_{1}$ be the inclination of the Earth’s axis and let
$\theta_{2}$ be the inclination of the Sun’s axis. And, P is given by
SmartGreen1977

$\displaystyle\begin{aligned} \quad
P=\theta_{1}+\theta_{2}\end{aligned}\quad\qquad\text{with}\quad\qquad\begin{aligned}
\theta_{1}&=\arctan(-\cos\lambda_{\odot}\tan\varepsilon),\\\\[4.0pt]
\theta_{2}&=\arctan\\{-\cos(\lambda_{\odot}-\Omega)\tan I\\},\end{aligned}$

where $\varepsilon$ is the angular distance of the Earth’s rotation axis from
the ecliptic plane. More detailed explanations are given in the Textbook on
Spherical Astronomy of Smart & Green SmartGreen1977 .

In general, $\Omega$, $\lambda_{\odot}$ and $\varepsilon$ are changing with
time. Therefore, these parameters should also be calculated at the time under
consideration and they are given by Duffett1988

$\displaystyle\begin{aligned}
\Omega&=75^{\circ}46^{\prime}+84^{\prime}T,\\\\[4.0pt]
\varepsilon&=23^{\circ}26^{\prime}21.45^{\prime\prime}-46.815^{\prime\prime}T-0.0006^{\prime\prime}T^{2}+0.00181^{\prime\prime}T^{3},\\\\[4.0pt]
\lambda_{\odot}&=\frac{360}{365.242191}T+\frac{360}{\pi}e\sin\left\\{\frac{360}{365.242191}T+\varepsilon_{g}-\varpi_{g}\right\\}+\varepsilon_{g},\end{aligned}$

with

$\displaystyle\begin{aligned} \varepsilon_{g}&=279.6966778+36000.76892\hskip
1.42271ptT+0.0003025\hskip 1.42271ptT^{2},\\\\[4.0pt]
\varpi_{g}&=281.2208444+1.719175\hskip 1.42271ptT+0.000452778\hskip
1.42271ptT^{2},\\\\[4.0pt] e&=0.01675104-0.0000418\hskip
1.42271ptT-0.000000126\hskip 1.42271ptT^{2},\end{aligned}$

where T is the number of Julian centuries since the epoch 2000 January 1.5,
$\varepsilon_{g}$ is the mean ecliptic longitude of the Sun at the epoch,
$\varpi_{g}$ is the longitude of the Sun at perigee and e is the eccentricity
of the Earth’s orbit around the Sun.

##  3 Preparation of the Stonyhurst disk

In order to form a Stonyhurst disk on an image, it is necessary to form a two-
dimensional projection of a three-dimensional sphere. But only half of the
sphere or the visible side of the sphere must be taken into consideration. Let
this sphere be indicated in Figure 3a. If N is any point on this sphere and
its spherical coordinates in three-dimensional space are the radius of the
sphere (r), latitude angle ($\theta$) and longitude angle ($\varphi$), its
Cartesian coordinates (x, y, z) are given by GovilPai2004

$\displaystyle\begin{aligned} x&=r\cos\theta\sin\varphi,\\\ y&=r\sin\theta,\\\
z&=r\cos\theta\cos\varphi,\end{aligned}$

where the angle $\theta$ is selected between -90∘ and +90∘ and, the angle
$\varphi$ is selected between 0∘ and 180∘ SmartGreen1977 . These three-
dimensional (3D) coordinates have to be converted to two-dimensional (2D)
coordinates to draw the Stonyhurst disk depending on P and $B_{o}$ parameters
of the Sun. These are done with the well known transformation and projection
equations WrightSweet2000 ; GovilPai2004 ; Stephens2000 ; Salomon2006 . Let us
assume that an angle is rotated in the anti-clockwise direction of its
positive and in clockwise direction of its negative values. And also, let us
assume that P and $B_{o}$ angles are positive. First, the x axis is rotated
$B_{o}$ degrees and $x_{1}$, $y_{1}$, $z_{1}$ values are obtained, and then,
the z axis is rotated P degrees and $x_{2}$, $y_{2}$, $z_{2}$ values are
obtained (Fig. 3b). These equations are specified as follows Salomon2006

$\displaystyle\begin{aligned} x_{1}&=x,&\hskip 28.45274ptx_{2}&=y_{1}\sin
P+x_{1}\cos P,\\\ y_{1}&=z\sin B_{o}+y\cos B_{o},&y_{2}&=y_{1}\cos P-x_{1}\sin
P,\\\ z_{1}&=z\cos B_{o}-y\sin B_{o},&z_{2}&=z_{1},\end{aligned}$

Here, $x_{2}$ and $y_{2}$ are the 2D projection coordinates of the point N
with respect to the sphere center. When these are rearranged, we get

(a) (b)

Figure 3: (a) Spherical ($\it r,\theta,\varphi$) and Cartesian ($\it{x,y,z}$)
coordinates of the point N in three-dimensional space. (b) Projection
coordinates of the point N are expressed as $\it{x_{2},y_{2}}$ and $\it z_{2}$
in the $\it{x^{\prime}y^{\prime}z^{\prime}}$ projection system. Here, z axis
(blue) is rotated P degree and x axis (blue) is rotated $\it B_{0}$ degree.
Also, blue colored circles represent the rotated planes of the point N.

$\displaystyle x_{2}$ $\displaystyle=$ $\displaystyle(z\sin B_{o}+y\cos
B_{o})\sin P+x\cos P,$ $\displaystyle y_{2}$ $\displaystyle=$
$\displaystyle(z\sin B_{o}+y\cos B_{o})\cos P-x\sin P,$

Since the zero point of the screen coordinates in the computer environments is
the top-left corner, the x-axis’s value is increasing from the left to right
of the screen and, the y-axis’s value is increasing from the top to bottom of
the screen, Therefore, the center of the any solar disk image on the screen
does not coincide with the zero point of the screen coordinates. Hence, in
order to draw the sphere onto solar image, the center of the sphere must be
moved to the center of the solar disk image. This is done by adding x and
y-axis values of the screen coordinates of the center of the solar disk image
to the x and y-axis values of the all points of the sphere separately. Let
$x_{p}$ and $y_{p}$ be the screen coordinates of the point N. The following
equations are obtained with the explanations above:

$\displaystyle\begin{aligned} x_{p}&=x_{0}+x_{2},&\hskip
28.45274ptx_{p}&=&x_{o}+(z\sin B_{o}+y\cos B_{o})\sin P+x\cos P,\\\
y_{p}&=y_{0}-y_{2},&y_{p}&=&y_{o}-(z\sin B_{o}+y\cos B_{o})\cos P+x\sin
P,\end{aligned}$

where $x_{0}$ and $y_{0}$ are the coordinates of the disk image center in
screen coordinates. The subtraction is carried out in y-values because of the
increase in the reverse direction contrary to the Cartesian system. With these
projection equations, Stonyhurst disk can be prepared with any values of P and
$B_{o}$. As an example, the Stonyhurst disk for $B_{o}$ = 5∘ and P = 0∘ values
is shown in Figure 4a. This is compared with the one used in the observatory
(Fig. 4b) and their can be seen agreement in Fig. 4c. Also, other Stonyhurst
disks (for $B_{o}$ = 0,1,2,3,4,5,6,7∘) were checked as well and exact fits
were obtained.

Figure 4: (a) Stonyhurst disk prepared for $B_{o}$ = 5∘ in computer. (b)
Plastic Stonyhurst disk with the same $B_{o}$. (c) Two disks superposed.

##  4 Computer-aided process of the solar drawings

CAMS is mainly designed to process the daily solar drawings, and the interface
is presented in Fig. 5. All processes are performed in this window, such as a
new process, checking the previous processes, calculation of the solar
parameters and transferring the results of the process to the web. Processing
the solar drawing has two stages. First stage is the transferring the drawing
into the program and superposing the Stonyhurst disk onto it.The second stage
is the determining the heliographic coordinates of each sunspot group besides
other features like the group length and tilt angle.

Figure 5: CAMS main window with an example of the transferred sunspot drawing.
The solar parameter calculation and image quality selection windows are seen
at the bottom, respectively.

Figure 6: Defining the center and radius of the circle represent the size of
the Stonyhurst disk over the image of the drawing.

###  4.1 Transferring the solar drawing into program and superposing
Stonyhurst disk on it

After all sunspots on the solar surface are drawn manually on the observation
paper, the processing stage starts with the calculation of the solar
parameters by entering date and time of the observation in the parameter
calculation window. Then, the drawing is transferred into the program by using
the scanner interface. After this is done, the center and a point on the outer
border of the drawing disk must be marked, respectively. A circle representing
the size of the Stonyhurst disk is imposed spontaneously over the image (Fig.
6). Conformity between circle and disk is checked visually by changing the
radius and by shifting the center of the circle. Also, the location of the
North point of the drawing is adjusted properly by the arrows of the
positioning window. After approval of conformity between circle and disk, the
cleaning of the outer region and the clipping process of the image is carried
out and, the image is transfered into the measurement window.

Figure 7: (a) Marking the sunspot group using an enclosed rectangular frame.
The beginning, middle and last sequence of the stage. (b) The adjustment can
be made easily by moving the handles on the border of the frame.

###  4.2 Measuring the heliographic coordinate of the sunspot groups

Stonyhurst disk is drawn automatically over the clipped image of the drawing
for the time of observation entered before as seen in Fig. 5. This disk can be
redrawn again, if needed, by readjusting the date or the time of the
observation on the date & time section of the program. The measurement process
is started by passing big picture mode of the measurement window and the big-
sized image of the drawing is loaded into the window. When the mouse moves
over the Stonyhurst disk, a spherical cursor will appear and display its
position in apparent heliographic coordinates. The sunspot group to be
processed is marked as an enclosed rectangular area with this spherical
cursor. Some stages of this process are shown in Fig. 7a.

The rectangular frame’s edges should be as narrow as possible while containing
the whole group. This proximity can be adjusted more precisely by moving the
small handles placed on four edges of the frame, after the marking process is
completed (Fig. 7b). Then, a group information window is displayed on top-
middle of the measurement window showing all information about the sunspot
group. The coordinates at the center of this rectangular frame are the
heliographic coordinates of the sunspot group. Its latitudinal and
longitudinal width is the height and length of the group, respectively. The
type of the group and the number of the sunspots are entered by selecting
proper values from the dropdown lists (Fig. 8a). If an A-type group which has
a length smaller than 1∘ is encountered, point source measurement is fulfilled
by touching to the center of the group. No length, tilt angle and rectangular
area for the group is calculated in this case (Fig. 8b).

Figure 8: (a) Selecting the type of the group and the number of the sunspot
with the drop-down lists. (b) Measuring the point like sunspot and information
about it. Fine adjustments are done using the arrow buttons. (c) Positioning
the tilt line according to the tilt direction of the group.

Figure 9: A finished observation process example that all the sunspot groups
are measured. Left: Calculated values for the sunspot groups. Right:
Rectangular frames of the groups with the corresponding sequence number.

The tilt angle of the group is started to adjust by pressing the E symbol on
the top-left corner of the group’s frame. A line with two small handles at its
ends appears in the middle of the frame. Then, the inclination of this line is
adjusted by moving these handles in proper position to indicate the tilt
direction of the group (Fig. 8c). Adjusting the tilt is concluded by pressing
again the E symbol again. After all the sunspot group are marked and processed
individually (Fig. 9), all data about the observation is saved into the
database of the program and sent to web page server of the observatory.
Detailed information about the CAMS and steps of the processing the
observations with CAMS are given in the Çakmak’s symposium
proceedingCakmak2010 .

As an example of the precision of the CAMS, the comparative measurement values
of a sample observation day is listed in the Table 1. Image of the solar
drawing of this day is shown in Fig. 10. Old-dated solar observation is
specially selected to show the capability of the built-in solar ephemeris of
the program. In this table, the measurements of the both plastic Stonyhurst
disk and CAMS are given for every sunspot group separately. The last two
columns of the right panel are the values calculated with CAMS, latitudinal
length and tilt angle of the sunspot group, respectively.

Table 1: The comparative measurement values of a sample observation day. Left
panel results are calculated manually by using the plastic Stonyhurst disk
(for $\it B_{0}$ = 2), whereas right panel results are obtained with CAMS.

Date | 19.12.1989 | P | +8.06 | +8.08 |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---
Time | 09:45 UT | $\it B_{0}$ | -1.51 | -1.52 |  |  |  |  |  |
$\it L_{0}$ | 273.50 | 273.47 |  |  |  |  |  |
Type | S | B |  $\Delta$L | L | U | B |  $\Delta$L | L | U | UB | Tilt
DAO | 4 | +09 | 72 | W | 345 | 7.5 | +08.97 | 71.94 | W | 344.94 | 7.9 | 3.5 | 18.94
AX | 1 | +34 | 70 | W | 343 | - | +32.84 | 69.48 | W | 342.48 | - | - | -
BXO | 2 | +30 | 63 | W | 336 | 4.0 | +29.75 | 62.84 | W | 335.84 | 4.9 | 2.2 | 25.10
FKC | 43 | +21 | 32 | W | 305 | 15.0 | +20.85 | 32.86 | W | 305.86 | 14.7 | 9.3 | 24.25
DKC | 29 | +27 | 03 | W | 276 | 10.0 | +26.69 | 02.48 | W | 275.48 | 9.7 | 6.5 | 26.35
CRO | 6 | +17 | 09 | E | 264 | 4.5 | +16.88 | 09.54 | E | 263.46 | 4.7 | 1.5 | 10.92
FKI | 43 | +26 | 11 | E | 262 | 18.0 | +25.44 | 11.40 | E | 261.96 | 18.6 | 9.0 | 18.30
EHI | 9 | +18 | 55 | E | 218 | 14.0 | +17.27 | 54.47 | E | 218.53 | 13.3 | 3.2 | 1.45
? | 9 | +22 | 74 | E | 199 | 11.0 | +21.61 | 75.22 | E | 197.78 | 12.6 | 2.7 | 9.80
CRO | 6 | -13 | 63 | W | 336 | 6.5 | -13.21 | 63.83 | W | 336.83 | 6.8 | 1.6 | 13.59
FKC | 22 | -33 | 51 | W | 324 | 17.0 | -32.68 | 50.85 | W | 323.85 | 16.8 | 4.9 | 12.65
DAC | 16 | -21 | 08 | E | 265 | 5.5 | -21.35 | 08.20 | E | 264.80 | 5.8 | 2.2 | 12.15
BXO | 9 | -23 | 15 | E | 258 | 4.0 | -23.30 | 14.47 | E | 258.53 | 4.0 | 1.9 | 32.29

$\it S$: Spot number, $\it B$: Latitude, $\Delta\it L$: Apparent Longitude,
$\it L$: Longitude, $\it U$: Length of the group, $\it UB$: Latitudinal
Length, $\it Tilt$: Inclination of the group with respect to the solar
equator.

Figure 10: Solar drawing of the observatory taken in 1989.12.19. Stonyhurst
disk is superimposed for the time of the observation, 09:45 UT.

##  5 Discussion

Nowadays, i.e. in the computer age, observation techniques, calculation steps
and archiving the obtained knowledge are important and essential. From this
perspective, the main purpose of CAMS is the shortening the time spent on
processing the observation and to eliminate possible observer errors from the
calculations. Second purpose is calculating the heliographic coordinates,
latitudinal and longitudinal length, tilt angle and the rectangular area of
the sunspot groups more precisely than manual measurements. Therefore, the
measured values are saved with two decimals. When working on the positions of
the sunspot groups, increasing the numerical precision in the coordinates
gives a more accurate approach in the calculations. For example, in the
butterfly diagram, the distributions of the sunspot groups with latitudes will
appear to be more accurate instead of being concentrated on the latitude
lines.

Figure 11: Examples of the using digital solar image with CAMS. Left: A HMIIF
(Helioseismic and Magnetic Imager Intensitygram - Flattened) image of the
Solar Dynamics Observatory (SDO). Right: A chromospheric H-Alpha image taken
with CCD at the Istanbul University Observatory.

Another point of the CAMS is that the Stonyhurst disks are drawn at the true
angle of the latitude angle instead of the nearest integer value. That is to
say, it is drawn at the angle, for example, 7.12∘ or 7.25∘ or 7.57∘ or 7.92∘,
so not just for 7.0∘. The same is valid for the position angle, as well.
Although CAMS is designed to process daily solar drawings, any digital solar
image can be used by the program. Two examples of this are shown in Fig. 11.
One is a HMIIF (Helioseismic and Magnetic Imager Intensitygram - Flattened)
image of the Solar Dynamics Obsevatory, and the other is the chromospheric
H-Alpha image taken with CCD at the university observatory. The heliographic
coordinates of the sunspot groups can be measured more precisely with respect
to the drawings. This can be done to obtain more accurate results or to
eliminate the drawing errors of the observers in their studies.

The development of CAMS is still continuing and the integration with the
database is under development. After this is done, it will be possible to
obtain any group’s evolution separately and all statistical information about
the positions and types of the sunspot groups. Also the daily, monthly changes
and general trend of the sunspot relative number can be easily generated
graphically. Since the relative numbers are archived by making a distinction
between the Northern and Southern, the variations of the relative number in
both will be also examined separately. After sufficient data is gathered, the
whole solar sunspot cycle will be analyzed in detail.

###### Acknowledgements.

Thanks to Dr. Tuncay Özışık from the Turkish National Observatory to develop
the CAMS and Assoc. Prof. Dr. Nurol Al Erdoğan from the Istanbul University
Science Faculty, Astronomy and Space Sciences Department for the idea to
prepare and publish this article. Also, thanks to Asst. Başar Coşkunoğlu for
his contributions towards improving the language of the manuscript. Also,
thanks to anonymous reviewer for his/her valuable suggestions and comments
improving manuscript This work supported by the Istanbul University Scientific
Research Projects Commission with the project number 24242.

## References

  * (1) Carrington, R.C., Observations of the Spots on the Sun from November 9, 1853, to March 24, 1861, Made at Redhill, London (Edinburgh: Williams and Norgate) (1863)
  * (2) Teague, E.T.H.: J. Br. Astron. Assoc., 106, 2 (1996)
  * (3) Thompson, W.T.: A&A, 449, 791 (2006)
  * (4) Sanchez-Bajo, F. and Vaquero, J.M.: Eur. J. Phys., 34, 527 (2013)
  * (5) Meadows, P.: J. Br. Astron. Assoc., 112, 6 (2002)
  * (6) Cristo, A., Vaquero, J.M. and Sánchez-Bajo, F.: J. Atmos. Solar-Terr. Phys., 73, 187 (2011)
  * (7) Clette, F.: J. Atmos. Solar-Terr. Phys., 73, 182 (2011)
  * (8) Smart, W.M. and Green, R.M.: Textbook on Spherical Astronomy, Cambridge University Press (1977)
  * (9) Duffett-Smith, P.: Practical Astronomy with Your Calculator, 3rd ed., Cambridge University Press (1988)
  * (10) Salomon, D.: Transformations and Projections in Computer Graphics, Springer-Verlag London Limited (2006)
  * (11) Wright, R.S., and Sweet, M.: OpenGL Super Bible, 2nd ed., Waite Group Press (2000)
  * (12) Govil-Pai, S.: Principles of Computer Graphics, Springer (2004)
  * (13) Stephens, R.: Visual Basic Graphics Programming 2nd ed., Wiley Computer Publishing (2000)
  * (14) Çakmak, H.: Solar and Solar-Like Stars Symposium, 158-167, in Özkan, M.T., Al Erdoğan, N., and Başal, M. (Ed.), Istanbul (2010)