¬$\neg$ ²^2 ³^3 ·$\cdot$ ¹^1 ×$\times$ ÷$\div$ ʲ^j ʳ^r ˡ^l –̷／ Γ$\Gamma$
Δ$\Delta$ ΗH Θ$\Theta$ Λ$\Lambda$ Ξ$\Xi$ Π$\Pi$ Σ$\Sigma$ Φ$\Phi$ Ψ$\Psi$
Ω$\Omega$ α$\mathnormal{\alpha}$ β$\beta$ γ$\mathnormal{\gamma}$
δ$\mathnormal{\delta}$ ε$\mathnormal{\varepsilon}$ ζ$\mathnormal{\zeta}$
η$\mathnormal{\eta}$ θ$\mathnormal{\theta}$ ι$\mathnormal{\iota}$
κ$\mathnormal{\kappa}$ λ$\mathnormal{\lambda}$ μ$\mathnormal{\mu}$
ν$\mathnormal{\mu}$ ξ$\mathnormal{\xi}$ π$\mathnormal{\pi}$
π$\mathnormal{\pi}$ ρ$\mathnormal{\rho}$ σ$\mathnormal{\sigma}$
τ$\mathnormal{\tau}$ φ$\mathnormal{\phi}$ χ$\mathnormal{\chi}$
ψ$\mathnormal{\psi}$ ω$\mathnormal{\omega}$ ϕ$\mathnormal{\phi}$
ϵ$\mathnormal{\epsilon}$ ᵀT ᵏ^k ᵐ^m ᵢi ᶿ^θ †† ′′ ″${}^{\second}$
‴${}^{\third}$ ⁱ^i ⁵5 ⁺+ ⁻- ⁿ^n ₀0 ₁1 ₂2 ₃3 ₊+ ₋- ₘ_m ₙ_n –⃰∗ ℂ$\mathbb{C}$
ℒ$\mathscr{L}$ ℕN ℚ$\mathbb{Q}$ ℝ$\mathbb{R}$ ℤ$\mathbb{Z}$ ℳM ⅋$\parr$
←$\leftarrow$ ↑$\uparrow$ →$\rightarrow$ ↔$\leftrightarrow$ ↖↖ ↗↗ ↝$\leadsto$
↦$\mapsto$ ⇆$\leftrightarrows$ ⇐$\Leftarrow$ ⇒$\Rightarrow$ ⇔$\Leftrightarrow$
∀$\forall$ ∂$\partial$ ∃$\exists$ ∅$\varnothing$ ∈$\in$ ∉$\not\in$ ∋$\ni$
∎$\qed$∏∏ ∑∑ ∗$\ast$ ∘$\circ$ ∙$\bullet$ ∝∝ ∞$\infty$ ∣$\mid$ ∧∧∨∨∩$\cap$
∪$\cup$ ∫∫ ∷:: ∼$\sim$ ≃$\simeq$ ≅$\cong$ ≈$\approx$
≜$\stackrel{{\scriptstyle\scriptscriptstyle{\triangle}}}{{=}}$
≝$\stackrel{{\scriptstyle\scriptscriptstyle{\text{def}}}}{{=}}$
≟$\stackrel{{{}_{\text}{\textbf{?}}}}{{\text{\textbf{=}}\negthickspace\negthickspace\text{\textbf{=}}}}$
≠$\neq$≡$\equiv$≤$\leq$ ≥$\geq$ ⊂$\subset$ ⊃$\supset$ ⊆$\subseteq$
⊇$\supseteq$ ⊎$\uplus$ ⊑$\sqsubseteq$ ⊒$\sqsupseteq$ ⊓$\sqcap$ ⊔$\sqcup$
⊕$\oplus$ ⊗$\otimes$ ⊙$\odot$ ⊛$\circledast$ ⊢$\vdash$ ⊤$\top$ ⊥$\bot$ ⊧⊧ ⊨⊧ ⊩
⊬／⊢ ⊸$\multimap$ ⋁$\bigvee$ ⋃$\bigcup$ ⋄$\diamond$ ⋅$\cdot$ ⋆$\star$ ⋮$\vdots$
⋯$\cdots$ ─— ■$\blacksquare$ □$\square$ ▴$\blacktriangledown$ ▵$\triangle$
▹$\triangleright$ ▻$\rhd$ ▾$\blacktriangle$ ▿$\triangledown$ ◃$\triangleleft$
◅$\lhd$ ◇$\diamond$ ◽$\square$ ★$\star$ ♭$\flat$ ♯$\sharp$ ✓$\checkmark$ ⟂⊥
⟦$\llbracket$ ⟧$\rrbracket$ ⟨$\langle$ ⟩$\rangle$ ⟶⟶ ⟷ $\leftrightarrow$
⟹$\Longrightarrow$ ⦇ ⦈ ⪆$\gtrapprox$ ⪅$\lessapprox$ ⱼ_j 𝒟$\mathcal{D}$
𝒢$\mathcal{G}$ 𝒦$\mathcal{K}$ 𝒫$\mathcal{P}$ 𝔸$\mathds{A}$ 𝔹$\mathds{B}$ 𝔼E
𝟙$\mathds{1}$

# Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax

Jean-Philippe Bernardy Shalom Lappin  Centre for Linguistic Theory and Studies
in Probability
Department of Philosophy, Linguistics and Theory of Science
University of Gothenburg<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We show that both an LSTM and a unitary-evolution recurrent neural network
(URN) can achieve encouraging accuracy on two types of syntactic patterns:
context-free long distance agreement, and mildly context-sensitive cross
serial dependencies. This work extends recent experiments on deeply nested
context-free long distance dependencies, with similar results. URNs differ
from LSTMs in that they avoid non-linear activation functions, and they apply
matrix multiplication to word embeddings encoded as unitary matrices. This
permits them to retain all information in the processing of an input string
over arbitrary distances. It also causes them to satisfy strict
compositionality. URNs constitute a significant advance in the search for
explainable models in deep learning applied to NLP.

##  1 Introduction

[8, 11] proposed end-to-end-trainable vector space semantic models based on
the syntactic types of a pregroup grammar [20]. More recently, [23] construct
a vector space semantics with a modality, within a Lambek calculus-based type
logical grammar.111[30] use the syntactic type representations of a
Combinatory Categorial Grammar [28] to train a deep neural network to learn
word representations.

A significant inspiration for much of this work is the category theory within
which quantum mechanics is formulated. The types of the Lambek calculus
grammar generate a syntactic structure that is interpreted through the
category of matrices. The set of such matrices constitutes a semantic
representation for lexical items. They are anchored in distributional word
vectors that correspond to the word embeddings of contemporary deep neural
networks. The values of these matrices can be set to optimise a variety of NLP
objectives, such as the evaluation of semantic distance and relatedness among
sentences. Following this template, one obtains what the authors call a
_compositional vector space semantics_. The compositionality of these semantic
representations comes from the type system of the grammar, which is assumed as
given.

In this paper, we consider a model of compositional vector _syntax_ , based on
premises related to those assumed by the work just described. The main
similarity with this work is that the word representations are unitary
matrices, which can be trained end-to-end. The main difference is that our
model does not rely on the types of an existing syntactic representation.
Instead it learns syntactic structure directly from the data.

The only algebraic structure that our model invokes is that of sequences—
mathematically, a free monoid over input symbols. The relevant compositional
principle specifies that the representation $⟦w⟧$ of an input sequence $w$ is
a monoid homomorphism, defined using an associative combination operator
$(\textperiodcentered)$. This principle requires that the following equation
holds for any two sub-sequences $w_{1}$ and $w_{2}$:

$⟦w_{1}w_{2}⟧=⟦w_{1}⟧\textperiodcentered⟦w_{2}⟧$ (1)

Hence the representation of a concatenation is the combination of
representations with the operator $(\textperiodcentered)$. Additionally, the
representation of the empty string must be the unit of
$(\textperiodcentered)$.

In this paper, we consider the unitary-evolution recurrent neural network
(URN), first suggested by [3]. This is a kind of RNN where the function
applied to the hidden state vector is a unitary transformation. We analyse the
capability of the URN to model two syntactic patterns: a context free
structure and a mildly context sensitive one. We do this by looking at
synthetic languages, which allows us to abstract from the noise that pervades
natural language corpus data. We compare the syntactic capabilities of the URN
with the LSTM, the dominant architecture for current RNNs.

We recap the definitions of the models and study the theoretical properties of
the URN in Section 2. Our experiments are described in Sections 3 and 4. We
discuss the results in Section 5, and related work in Section 6, before
summarising our conclusions in Section 7.

##  2 Models

$\sigma$$\sigma$Tanh$\sigma$$\times$+$\times$$\times$Tanh$c_{t-1}$$h_{t-1}$$x_{t}$$c_{t}$$h_{t}$$h_{t}$

(a) LSTM

$\times$expskew$h_{t}$$x_{t}$$h_{t+1}$$h_{t+1}$$S_{t}$$Q_{t}$

(b) URN

Figure 1: Schematic representation of models

We consider two generative models: an LSTM [14] and an URN [3]. We portray
them schematically in Fig. 1. All models are trained end-to-end as generative
language models: we use cross-entropy loss for each symbol in the training
set, which we sum for each position, up to the stop symbol. There is no task-
specific supervised training. The training method is by stochastic gradient
descent. More precisely, we use an Adam optimiser with a learning rate of
$0.001$, and a batch size of 512. The number of parameters for each model is
listed in Tables 1 and 2.

###  2.1 LSTM

We provide the definition of an LSTM here to specify which version we are
using, and to highlight the contrasts with URNs.

$\displaystyle v_{t}$ $\displaystyle=h_{t-1}◇x_{t}$ $\displaystyle f_{t}$
$\displaystyle=σ(W_{f}v_{t}+b_{f})$ $\displaystyle i_{t}$
$\displaystyle=σ(W_{i}v_{t}+b_{i})$ $\displaystyle o_{t}$
$\displaystyle=σ(W_{o}v_{t}+b_{i})$ $\displaystyle\tilde{c}_{t}$
$\displaystyle=σ(W_{c}v_{t}+b_{c})$ $\displaystyle c_{t}$
$\displaystyle=(f_{t}⊙c_{t-1})+(i_{t}⊙\tilde{c}_{t})$ $\displaystyle h_{t}$
$\displaystyle=(o_{t}⊙\mathsf{tanh}(c_{t}))$

Here $σ$ refers to the sigmoid function and $(◇)$ is vector concatenation. We
apply dropout to the vectors $h_{t}$ and $x_{t}$ for every timestep $t$.
Predictions are obtained by applying a projection layer to $h_{t}$, with
softmax activation. The input $x_{t}$ is obtained by an embedding layer.

###  2.2 URN

The URN, in the variant that we employ, has a much simpler definition.

$\displaystyle h_{t}$ $\displaystyle=Q_{t}h_{t-1}$ $\displaystyle S_{t}$
$\displaystyle=skew(x_{t})$ $\displaystyle Q_{t}$ $\displaystyle=e^{S_{t}}$

$skew(x)$ is a function that takes a vector and produces a skew-symmetric
matrix by arranging the elements of $x$ in a triangular pattern. For example,
with an input vector of size 3, we have:

$skew(x)=\left(\begin{smallmatrix}0&x_{0}&x_{1}\\\ -x_{0}&0&x_{2}\\\
-x_{1}&-x_{2}&0\end{smallmatrix}\right)$

The upper triangle of $S_{t}$ is provided by the previous layer (typically the
word embedding layer), and its lower triangle is its negated symmetric. This
setup ensures that $S_{t}$ is anti-symmetric. This feature, together with the
properties of matrix exponential, insures that $Q_{t}$ is unitary. We call
$Q_{t}$ the _unitary embedding_ of the input symbol at position $t$.

#### Compositional property

We can now show that the URN exhibits compositionality, in the sense of Eq. 1.
In general, if we let $⟦w[t]⟧=Q_{t}$, then the output of the URN model for any
input sequence is:

$\displaystyle h_{n+1}$
$\displaystyle=Q_{n}\texttimes(Q_{n-1}\texttimes(⋯(Q_{0}\texttimes h_{0})))$
$\displaystyle=⟦w[n]⟧\texttimes(⟦w[{n-1}]⟧\texttimes(⋯\texttimes(⟦w[0]⟧\texttimes
h_{0})))$
$\displaystyle=(⟦w[n]⟧\texttimes⟦w[{n-1}]⟧\texttimes⋯\texttimes⟦w[0]⟧)\texttimes
h_{0}$ by associativity of matrix-vector product
$\displaystyle=⟦w[0]…w[n]⟧\texttimes h_{0}$

With:

$\displaystyle⟦w[t]⟧=e^{skew(x_{t})}$ (atomic symbol)
$\displaystyle⟦w_{1}w_{2}⟧=⟦w_{2}⟧\texttimes⟦w_{1}⟧$ (concatenation)

The above definition of $⟦\\_⟧$ satisfies Eq. 1, with $P\textperiodcentered
Q=Q\texttimes P$.222We leave it to the reader to check that this definition of
the $(\textperiodcentered)$ operator is associative. Because unitary matrices
form a monoid under product, $⟦w⟧$ can be represented by a unitary matrix for
any input string $w$.

#### Long-distance properties of URN models

There are two key properties of unitary interpretations that motivate their
use. First, the absence of non-linear activation functions on the recurrent
path entails that they are not subject to exploding or vanishing gradients, as
[3] observe. Second, because the transformations involved in their processing
of input are always unitary, any difference in the start state is preserved in
the output state. For every unitary matrix $Q$, we have

$\textlangle Qh,Qs\textrangle=\textlangle h,s\textrangle$

Another manifestation of this property is that unitary matrices can always be
inverted, by transposition:333Additionally, one should take the complex
conjugate when dealing with complex-valued matrices.

$Q^{*}Q=I$

As a consequence, no information is lost through time steps. This formal
analysis suggests that the URN will be good at tasks which require long-term
memory. We devote the remainder of this paper to confirming experimentally
that this property is observed for context-free and mildly context-sensitive
inputs, when training the word embeddings by stochastic gradient descent
(using the Adam optimiser).

#### Training regime

We apply dropout to the matrix $S_{t}$ for every timestep $t$. The input
matrix $S_{t}$ has dimensions $n\texttimes n$, with $n$ being referred as the
number of units below. Predictions are obtained in the same way as for the
LSTM.

##  3 Cross-Serial dependencies

[26] demonstrated the non-context-free nature of interleaved verb-object
relations in Dutch and Swiss German. One of Shieber’s examples of an embedded
verb-object crossing dependency in Swiss German is given in example (A) below.

  1. (A)

Jan sät das mer d’chind em _Hans_ es huus lönd _hälfe_ aastriiche
Jan said that we the children-ACC Hans-DAT the house-ACC let help paint
_Jan said that we let the children help Hans paint the house._

Similar patterns have been observed in other languages. They can be expressed
by indexed grammars [2, 24], as well as a variety of other Mildly Context-
Sensitive grammar formalisms [17, 27].

[26] observed that cross-serial dependency patterns of case marked nouns and
their corresponding verbs can be iterated in this construction. The above
pattern can be abstracted as a set of $a^{m}b^{n}c^{m}d^{n}$ structures, which
together form a Mildly Context-Sensitive language.

Formally, we consider the family of languages
$\mathcal{C}_{k}=\\{a^{m}b^{n}c^{m}d^{n}\mid m+n<k\\}$. Note that if $k<l$,
then $\mathcal{C}_{k}\subset\mathcal{C}_{l}$. The training set consists of
51,200 strings picked uniformly from ${\mathcal{C}_{8}}$. The test set
contains 5,120 strings picked uniformly from ${\mathcal{C}_{10}}$.

We recall that the RNNs are trained as generative language models. That is,
assuming a sample string $w∈\mathcal{C}_{10}$, RNNs are trained to predict the
symbol $w_{i+1}$ given $w_{0}$ to $w_{i}$. Special start and a stop symbols
are added to the input strings, as is standard.

This is to be contrasted with the testing procedure. At test time, given the
prefix $w_{0}…w_{i}$, a prediction of symbol $w_{i+1}$ is deemed correct if it
is a possible continuation for $w_{0}…w_{i}$; that is, if $w_{0}…w_{i+1}$ is a
prefix of some string in ${\mathcal{C}_{10}}$. A set of predictions for a full
string $x_{0}…x_{k}$ is classified as correct, if all predictions are correct
up to and including the stop symbol. We report error rates for full strings
only. This is because when models make a mistake, it is typically for a single
symbol near the end of a string.

Number of units | LSTM | URN
---|---|---
32 | 7314 | 5290
16 | 2738 | 1370
8 | 1218 | 370
Table 1: Number of parameters for cross-dependency pattern models

###  3.1 Results

Both RNNs struggle to generalise these patterns. They can model the training
data well, but produce incorrect patterns in some cases on strings of any
greater length than the samples in the training corpus.

We report four sets of results. The first set (Fig. 2(a)) is the cross-entropy
loss for the _test set_ obtained by each model across training epochs. Low
losses indicate that, on a per-character basis, the models reproduce the
_exact_ strings in the test set, rather than making correct predictions, as
defined above. We see that LSTM models generally make better guesses than
URNs, across the board.

The second set (Fig. 2(b)) is the error rate over number of epochs, for each
tested model. The best models can achieve less then ten percent error rate on
average on the test set. However, the LSTM models with a larger number of
units exhibit overfitting.

In the third set we show the error rate for a given training loss in Fig.
2(c). The corresponding ratio is a measure of a model’s capacity for correct
predictions of a given quality of approximation of the training set. It can be
taken as an indication of the model’s bias for this task, relative to
generative language modelling. The URN models tend to achieve lower error
rates for this task, even though they do less well from a generative language
modelling perspective. For instance, the 32-unit URN is able to obtain an
error rate below 0.4 with a training loss as high as 1\. In general, the URNs
provide a smoother decrease in error rate as they learn the language. In
contrast, the LSTM models exhibit a sharp drop in error rate around 0.7
training loss.

Finally, we show the error rate broken down by length of pattern (reported as
$n+m$). We see here that the LSTMs tend to do better overall than the URNs for
lengths unseen in the training data. However, the LSTMs do worse when the
number of units increases, while the URNs do better as that number increases,
thanks to a lack of overfitting.

$0$$20$$40$$60$$80$$100$$0.5$$1$$1.5$$2$$2.5$ (a) Test loss across epochs

$0$$20$$40$$60$$80$$100$$0$$0.2$$0.4$$0.6$$0.8$$1$ (b) Error rate across
epochs

$0.5$$1$$1.5$$2$$2.5$$0$$0.2$$0.4$$0.6$$0.8$$1$32-unit LSTM16-unit LSTM8-unit
LSTM32-unit URN16-unit URN8-unit URN (c) Error rate against training loss.

$2$$4$$6$$8$$10$$0$$0.2$$0.4$$0.6$$0.8$$1$ (d) Error rate against $n+m$, at
epoch 100

Figure 2: Cross-Serial dependencies results for various models

##  4 Generalised Dyck Languages

In the next experiment, we evaluate the long-distance modelling capabilities
of an RNN for a context-free language. As before, we do it in a way that
abstracts away from the noise of natural language, by constructing synthetic
data. Following [4], we use a (generalised) Dyck language. This language is
composed solely of matching parenthesis pairs. So the strings “`{([])}<>`” and
“`{()[<>]}`” are part of the language, while “`<)`” is not.

Formally, we use the language $\mathcal{D}$ defined as the set of strings
generated by the following context-free rules: $E::=ε;E::=EE;E::=oEc$, where
$(o,c)$ stands for a pair of matching parenthesis pairs. In all of our tests,
we use $5$ types of pairs (corresponding, for example, to the pairs `()` ,
`[]`, `{}`, `<>` and ‘’.)

The aim of the task is to predict the correct type of _closing_ parenthesis at
every point in a string. It should be noted that this experiment is an
idealised version of the agreement task proposed by [22]. The opening
parenthesis plays the role of a word (say a noun) which governs a feature of a
subsequent word (say the number of a verb), represented by the closing
parenthesis. Matching of parentheses corresponds to agreement. [22] point out
that sustaining accuracy over long distances requires that the model have
knowledge of hierarchical syntactic structure. If an RNN captures the long-
distance dependencies involved in agreement relations, it cannot rely solely
on the nearby governing symbols. In particular, the accuracy must be sustained
as the number _attractors_ increases. For our experiment, an attractor is
defined as an opening parenthesis occurring within a matching pair, but of the
wrong kind. For instance, in “`{()}`”, the parenthesis “`(`” is an attractor.

To generate a string with $N$ matching pairs, we perform a random walk between
opposite corners of a square grid of width and height $N$, such that one is
not allowed to cross the diagonal. When not restricted by the boundary, a step
can be taken either along the $x$ or $y$ axis with equal probability. A step
along the $x$ axis corresponds to opening a parenthesis, and one along the $y$
axis involves closing one. The type of parenthesis pair is chosen randomly and
uniformly.

In this task, we use strings with a length of exactly 20 characters. We train
on 102400 and test on 5120 random strings. In the previous experiment we
varied the string length between training and testing. In this experiment, we
vary the _nesting depth_ , from 3 to 9. For this purpose, we define the depth
of the string is the maximum nesting level reached within it. For instance
“`[{}]`” has depth 2, while “`{([()]<>)}`” has depth 4.

As in the first experiment, for the training phase the RNNs are treated as
generative language models, applying a cross-entropy loss function at each
position in the string. At test time, we evaluate the model’s ability to
predict the right kind of closing parenthesis at each point. We ignore
predictions regarding opening parentheses, because they are always acceptable
for the language.

Training is performed with a learning rate of 0.01, and a dropout rate of
$ρ=0.05$, for 100 epochs.

###  4.1 Results

We report four sets of results. The first set (Fig. 3(a)) is the cross-entropy
loss for the _test set_ achieved by each model across training epochs. Low
losses indicate that, on a per-character basis, the models reproduce the
_exact_ strings in the test set. These losses cannot drop to zero because it
is always valid to predict an opening parenthesis. As in the cross-serial
task, we observe that LSTM models make better guesses than URN, at least for a
similar number of units. However, we see that the training of the URN is
uniformly monotonous, while the LSTM can sometimes become worse for a few
epochs before converging. In fact, for 8 units, the LSTM exhibits overfitting.
The test loss increases slowly after epoch 30.

To analyse the performance of each model on the task, we break down the error
rate by number of attractors (Fig. 3(b)). The URN models are weakest for a low
number of attractors, and they achieve near excellent accuracy for a large
number of attractors. According to [22], this suggests that the models are
highly successful in learning hierarchical structure. A high numbers of
attractors corresponds to outer pairs, while a low number of attractors
corresponds to inner pairs. In sum, in inner positions the URN suffers from
some confusion. This confusion decreases as the number of units increases.

The LSTM is able to predict outer pairs rather well (but nowhere near as
successfully as the URN models). It reaches almost perfect accuracy for
adjacent pairs, with zero attractors, such as []. It does worst on pairs with
3 to 4 attractors. The LSTM is good at making a prediction which depends only
on the previous symbol. LSTM models with a larger number of units are also
good at making a prediction for a pair which encloses the whole string. This
indicates that it is fairly limited in its ability to capture hierarchical
structures over long distances, even though it does much better than the
majority class baseline, which stands at an 80% error rate.

In what follows, we will consider only the the _maximum_ error rate for any
given number of attractors, for varying epochs. That is, we report the peak
value from the previous graph as training progresses. Using this metric, a URN
performs consistently better than an LSTM with the same number of units (and a
similar number of parameters). It does so consistently across the training
period (Fig. 3(c)). However, we note that every model is capable of
generalising to deeper nesting levels to some extent, with an accuracy well
above a majority class baseline (with an error rate of 80%). The URN models
beat the majority class baseline within the first epoch, while the LSTM needs
a couple of epochs to do so.

Finally we report the error rate against training loss (Fig. 3(d)), as we did
for the cross-serial dependency task. Again, we do not report the average
error rate, but rather the _maximum_ error rate for any given number of
attractors. Here too, the relationship between error-rate and training loss
corresponds to the bias of the model for the task at hand, compared to a
generative language model task. We observe that the URN models outperform the
LSTM models across the board.

$0$$20$$40$$60$$80$$100$$1.8$$2$$2.2$ (a) Testing loss across epochs

$0$$2$$4$$6$$8$$10$$0$$0.1$$0.2$ (b) Error rate per number of attractors (best
epoch)

$0$$20$$40$$60$$80$$100$$0.2$$0.4$$0.6$32-unit LSTM16-unit LSTM8-unit
LSTM32-unit URN16-unit URN8-unit URN (c) Error rate across epochs

$1.2$$1.4$$1.6$$1.8$$2$$2.2$$2.4$$0.2$$0.4$$0.6$$0.8$ (d) Error rate against
training loss

Figure 3: Dyck language experiment results. When reporting error rates as training progresses, we use the maximum error rate across number of attractors. Number of units | LSTM | URN
---|---|---
32 | 6300 | 6348
16 | 2204 | 1644
8 | 924 | 444
Table 2: Number of parameters Dyck language models

##  5 Discussion

In summary, both the URN and the LSTM perform reasonably well on our
experiments. On the cross-serial dependency task, both architectures can model
the training data, but they do not generalise perfectly to extended sequences.
Still on both tasks, we observe that the URN exhibits a bias for the task,
rather than for pure generative language model predictions. Additionally, the
URN appears to be less prone to overfitting, on both tasks.

These experiments are significant for at least two reasons. We believe that
the URN is the first RNN capable of recognising hierarchical syntactic
structures of the sort that characterise natural language syntax. While the
LSTM can capture the patterns appearing in the training set, the URN displays
better generalisation capabilities than the LSTM, in addition to being
mathematically tractable. A particularly attractive result, is that URN models
achieve high accuracy with less than a couple of thousand parameters.

Second, our experiments illustrate the effectiveness of URNs as devices for
tracking and predicting complex dependency relations, over long strings, in a
fully compositional way. Unlike LSTMs, they do not suffer from opaqueness due
to non-linear activation functions. They do not make use of such functions,
and so their behaviour is, in principle, amenable to analysis using standard
tools from linear algebra. They are not blackbox processing devices that
require indirect methods of analysis and assessment, as is the case with most
other deep neural networks.

##  6 Related Work

###  6.1 Long-distance agreement

The capacity of Recurrent Neural Networks (RNNs), particularly LSTMs, to
identify context-free long distance dependencies has been widely discussed in
the NLP and cognitive science literature [9, 22, 6, 4, 25, 12, 21]. These
discussions have considered dependency patterns in both artificial systems,
particularly Dyck languages, and in natural languages, with subject-verb
agreement providing a paradigm case.

[10] already observed that it is useful to experiment with artificial systems
to filter out the noise of real world natural language data.

[4] tested the ability of the LSTM to predict closing parenthesis types in a
Dyck language. The results are qualitatively similar. In both cases the LSTM
makes the worse predictions for a moderate number of attractors. However he
reports worse results than us, despite using an LSTM with more units. We
attribute this difference to a better implementation of the LSTM. A less
perspicuous application of dropouts is a likely factor in the poorer
performance of [4]’s LSTM ([4]). [25] performed a set of experiments with the
same goal, and reported results compatible with those of [4].

While LSTMs (and GRUs) exhibit a certain capacity to generalise to deeper
nesting, their accuracy declines in relation to nesting depth. This is also
the case with their handling of natural language agreement. Other experimental
work has illustrated this effect [13, 25]. Similar conclusions are observed
for generative self-attention architectures [32]. Significantly, recent work
has indicated that non-generative self-attention architectures, in the style
of BERT, simply fail at this task [5]. This suggests that sequential
processing is required to solve it.

By contrast URNs achieve excellent performance on this task, without any
decline in relation to either nesting depth, or number of attractors. In
recent work [7] provide an explanation as to why this is the case, for a
version of the URN restricted to unitary matrices acting on 3 hyperplanes.
They show that the learned unitary embeddings for matching parentheses are
nearly the inverses of each other: $⟦o\,c⟧=⟦c⟧\texttimes⟦o⟧≈I$ for every pair
of matching parentheses $(o,c)$. Such an analysis illustrates the role of
compositionality in the performance of URNs on an NLP task.

###  6.2 Cross-serial patterns

[19] study both nested and cross serial dependencies with a Simple Recurrent
Network (SRN). As far as we are aware, our experiment is the first application
of both LSTMs and URNs to cross serial dependency relations. While both
achieve good accuracy on the cross serial patterns, the URN offers significant
advantages in simplicity and transparency of architecture. It also displays
enhanced stability in learning, and power of structural generalisation,
relative to loss in training data.

###  6.3 Unitary-Evolution Recurrent Networks

[3] propose Unitary-Evolution recurrent networks to solve the problem of
exploding and vanishing gradients, caused by the presence of non-linear
activation functions. Despite this, [3] suggest adding ReLU activation between
time-steps, unlike URNs. We are primarily concerned with the structure of the
underlying unitary embeddings. The connection between the two lines of work is
that if an RNN suffers exploding/vanishing gradients, it cannot track long
distance dependencies.

Additionally, [3] use a complicated function to transform word representations
into unitary matrices. We use a simpler method (exponential of anti-symmetric
matrix), previously applied by [15] for deep learning models. This method has
performed well for the tasks discussed here. Because we use a fully general
matrix exponential implementation, our model is computationally more expensive
than others [3, 16]. When testing the unitary matrix encodings of [16] and
[3], we obtained much worse results for our experiments. This may be because
we do not include ReLU activation, while they do. Another option for enforcing
the unitary character of matrices is to let back-propagation update the
unitary matrices arbitrarily $n\times n$, and project them onto the unitary
space periodically [31, 18].

#### Tensor Recurrent Neural Networks

[29] describe what they call a “tensor recurrent neural network” in which the
transition matrix is determined by each input symbol. This design appears to
be similar to URNs. However, unlike URNs, they use non-linear activation
functions, and so they inherit the complications that these functions produce.

##  7 Conclusions

The fact that URNs achieve good precision in predicting cross serial
dependencies and generalise appropriately for nested patterns suggest that
they are suitable for recognition of complex syntactic structures of the sort
that are challenging for other neural networks.

Our experiments show that URNs are biased towards predicting the patterns
found in context-free and mildly context-sensitive languages, even when
trained as generative language models. The fact that they satisfy strict
compositionality offers an important advance in the search for explainable AI
systems in deep learning models.

The move to powerful bidirectional transformers, like BERT, has produced
enhanced performance in a variety of NLP and other AI tasks. This has been
achieved at the expense of formal grounding and computational transparency. It
is even less obvious why such models perform as well as they do on some tasks,
and poorly on others, than is the case for LSTMs. By contrast, URNs offer
simple, light weight deep neural networks whose operation is fully open to
inspection and understanding at each point in the processing regime. They can
model complex syntactic structures, in the sense that they can reproduce an
extensive training set of (artificial) data containing such patterns. For
cross-serial dependency patterns, they do not generalise very well, but for
hierarchical patterns they display impressive generalisation capabilities.

URNs provide a principled solution to the problem of syntactic
compositionality. They resolve the question of how to generate the composite
values of input arguments in a principled and straightforward way. This is not
the case for LSTMs, because the combination of two cells cannot be expressed
as a single cell. By contrast, every URN cell applies matrix multiplication to
its constituents, and so the composition of the effects of two cells is simply
a matrix product.

URNs are worth exploring further as models of learning and representation that
bear some correspondence to human processing. In future work we will be
studying the application of URNs to other cognitively interesting NLP tasks.
We are particularly interested in examining possible parallels between the
ways in which URNs handle linguistic information, and human understanding of
natural language meaning and structure.

Finally, we observe that the unitary matrices through which URNs compute
output values from input arguments are identical to the gates of quantum
logic. This suggests the intriguing possibility of implementing these models
as quantum circuits. At some point in the future, this may facilitate training
these models on large amounts of data, and efficiently generating results for
tasks that are currently beyond the resources of conventional computational
systems.

In sum, we see the research that we report here as extending and modifying
some of the leading ideas in the foundational work of [8, 11]. They provided a
system for handling computational semantics compositionally with structures
that map onto the matrices of quantum circuits. We offer a model for learning
syntactic structure, and for processing in general, that is strictly
compositional. It uses some of the same core methods as the earlier work. In
future research we will attempt to apply our model to NLP tasks involving
semantic interpretation.

## Acknowledgements

The research reported in this paper was supported by grant 2014-39 from the
Swedish Research Council, which funds the Centre for Linguistic Theory and
Studies in Probability (CLASP) in the Department of Philosophy, Linguistics,
and Theory of Science at the University of Gothenburg. We presented some of
the main ideas of this paper to the CLASP Seminar, in December 2021, and to
the Cognitive Science Seminar of the School of Electronic Engineering and
Computer Science, Queen Mary University of London, in February 2022. We are
grateful to the audiences of these two events for useful discussion and
feedback.

## References

  * [1]
  * [2] Alfred V. Aho (1968): _Indexed Grammars–An Extension of Context-Free Grammars_.  Journal of the ACM 15(4), p. 647–671, 10.1145/321479.321488.
  * [3] Martin Arjovsky, Amar Shah & Yoshua Bengio (2016): _Unitary Evolution Recurrent Neural Networks_.  In: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, JMLR.org, pp. 1120–1128, 10.48550/arXiv.1511.06464.
  * [4] Jean-Philippe Bernardy (2018): _Can RNNs Learn Nested Recursion?_ Linguistic Issues in Language Technology 16, 10.33011/lilt.v16i.1417.
  * [5] Jean-Philippe Bernardy, Adam Ek & Vladislav Maraev (2021): _Can the Transformer Learn Nested Recursion with Symbol Masking?_ In: Findings of the ACL 2021, 10.18653/v1/2021.findings-acl.67.
  * [6] Jean-Philippe Bernardy & Shalom Lappin (2017): _Using Deep Neural Networks to Learn Syntactic Agreement_.  Linguistic Issues In Language Technology 15(2), p. 15, 10.33011/lilt.v15i.141.
  * [7] Jean-Philippe Bernardy & Shalom Lappin (2022): _A Neural Model for Compositional Word Embeddings and Sentence Processing_.  In: Proceedings of The Workshop on Cognitive Modeling and Computational Linguistics, Association for Computational Linguistics, 10.18653/v1/2022.cmcl-1.2.  Available at https://aclanthology.org/2022.cmcl-1.2/.
  * [8] Bob Coecke, Mehrnoosh Sadrzadeh & Stephen Clark (2010): _Mathematical Foundations for a Compositional Distributional Model of Meaning_.  Lambek Festschrift, Linguistic Analysis 36, 10.48550/arXiv.1003.4394.
  * [9] Jeffrey L. Elman (1990): _Finding structure in time_.  Cognitive Science 14(2), pp. 179–211, 10.1016/0364-0213(90)90002-E.
  * [10] Jeffrey L. Elman (1991): _Distributed representations, simple recurrent networks, and grammatical structure_.  Machine learning 7(2-3), pp. 195–225, 10.1007/BF00114844.
  * [11] Edward Grefenstette, Mehrnoosh Sadrzadeh, Stephen Clark, Bob Coecke & Stephen Pulman (2011): _Concrete Sentence Spaces for Compositional Distributional Models of Meaning_.  In: Proceedings of the Ninth International Conference on Computational Semantics (IWCS 2011).  Available at https://aclanthology.org/W11-0114.
  * [12] Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen & Marco Baroni (2018): _Colorless Green Recurrent Networks Dream Hierarchically_.  In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Association for Computational Linguistics, New Orleans, Louisiana, pp. 1195–1205, 10.18653/v1/N18-1108.
  * [13] John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang & Christopher D Manning (2020): _RNNs can generate bounded hierarchical languages with optimal memory_.  arXiv preprint arXiv:2010.07515, 10.18653/v1/2020.emnlp-main.156.
  * [14] Sepp Hochreiter & Jürgen Schmidhuber (1997): _Long short-term memory_.  Neural Computation 9(8), pp. 1735–1780, 10.1162/neco.1997.9.8.1735.
  * [15] Stephanie L Hyland & Gunnar Rätsch (2017): _Learning unitary operators with help from u(n)_.  In: Thirty-First AAAI Conference on Artificial Intelligence, 10.1609/aaai.v31i1.10928.
  * [16] Li Jing, Yichen Shen, Tena Dubček, John Peurifoi, Scott Skirlo, Yann LeCun, Max Tegmark & Marin Soljačić (2017): _Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNN_.  In: Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, JMLR.org, p. 1733–1741.
  * [17] Aravind K. Joshi, K. Vijay Shanker & David Weir (1990): _The Convergence of Mildly Context-Sensitive Grammar Formalisms_.  Technical Report, Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA.
  * [18] Bobak Kiani, Randall Balestriero, Yann Lecun & Seth Lloyd (2022): _projUNN: efficient method for training deep networks with unitary matrices_ , 10.48550/arXiv.2203.05483.  Available at https://arxiv.org/pdf/2203.05483.pdf.
  * [19] Christo Kirov & Robert Frank (2012): _Processing of nested and cross-serial dependencies: an automaton perspective on SRN behaviour_.  Connection Science 24(1), pp. 1–24, 10.1080/09540091.2011.641939.
  * [20] Joachim Lambek (2008): _Pregroup Grammars and Chomsky’s Earliest Examples_.  Journal of Logic, Language and Information 17, pp. 141–160, 10.1007/s10849-007-9053-2.
  * [21] Shalom Lappin (2021): _Deep Learning and Linguistic Representation_.  CRC Press, Taylor & Francis, Boca Raton, London, New York, 10.1201/9781003127086.
  * [22] Tal Linzen, Emmanuel Dupoux & Yoav Golberg (2016): _Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies_.  Transactions of the Association of Computational Linguistics 4, pp. 521–535, 10.1162/tacl_a_00115.
  * [23] Lachlan McPheat, Mehrnoosh Sadrzadeh, Hadi Wazni & Gijs Wijnholds (2021): _Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality (Extended Abstract)_.  Electronic Proceedings in Theoretical Computer Science 333, pp. 168–182, 10.4204/EPTCS.333.12.
  * [24] Stephen Pulman & G. D. Ritchie (1985): _Indexed Grammars and Intersecting Dependencies_.  Technical Report 23, University of East Anglia.
  * [25] Luzi Sennhauser & Robert Berwick (2018): _Evaluating the Ability of LSTMs to Learn Context-Free Grammars_.  In: Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Association for Computational Linguistics, Brussels, Belgium, pp. 115–124, 10.18653/v1/W18-5414.
  * [26] Stuart M. Shieber (1985): _Evidence against the context-freeness of natural language_.  Linguistics and Philosophy 8(3), pp. 333–343, 10.1007/BF00630917.
  * [27] Edward P. Stabler (2004): _Varieties of crossing dependencies: Structure dependence and mild context sensitivity_.  Cognitive Science 93(5), pp. 699–720, 10.1207/s15516709cog2805_4.
  * [28] Mark Steedman (2000): _The Syntactic Process_.  MIT Press, Cambridge, MA.
  * [29] Ilya Sutskever, James Martens & Geoffrey E. Hinton (2011): _Generating Text with Recurrent Neural Networks_.  In Lise Getoor & Tobias Scheffer, editors: Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, Omnipress, pp. 1017–1024.  Available at https://icml.cc/2011/papers/524_icmlpaper.pdf.
  * [30] Gijs Wijnholds, Mehrnoosh Sadrzadeh & Stephen Clark (2020): _Representation Learning for Type-Driven Composition_.  In: Proceedings of the 24th Conference on Computational Natural Language Learning, Association for Computational Linguistics, pp. 313–324, 10.18653/v1/2020.conll-1.24.
  * [31] Scott Wisdom, Thomas Powers, John Hershey, Jonathan Le Roux & Les Atlas (2016): _Full-capacity unitary recurrent neural networks_.  Advances in neural information processing systems 29, pp. 4880–4888, 10.48550/arXiv.1611.00035.
  * [32] Xiang Yu, Ngoc Thang Vu & Jonas Kuhn (2019): _Learning the Dyck language with attention-based Seq2Seq models_.  In: Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 138–146, 10.18653/v1/W19-4815.


# Variations of the Galactic Cosmic Rays in the Recent Solar Cycles

Shuai Fu State Key Laboratory of Lunar and Planetary Sciences, Macau
University of Science and Technology, Taipa 999078, Macau, PR China  CNSA
Macau Center for Space Exploration and Science, Taipa 999078, Macau, PR China
Xiaoping Zhang State Key Laboratory of Lunar and Planetary Sciences, Macau
University of Science and Technology, Taipa 999078, Macau, PR China  CNSA
Macau Center for Space Exploration and Science, Taipa 999078, Macau, PR China
Lingling Zhao Center for Space Plasma and Aeronomic Research (CSPAR),
University of Alabama in Huntsville, Huntsville, AL 35805, USA  Yong Li State
Key Laboratory of Lunar and Planetary Sciences, Macau University of Science
and Technology, Taipa 999078, Macau, PR China  CNSA Macau Center for Space
Exploration and Science, Taipa 999078, Macau, PR China

###### Abstract

In this paper, we study the galactic cosmic ray (GCR) variations over the
solar cycles 23 and 24, with measurements from the NASA’s _ACE_ /CRIS
instrument and the ground-based neutron monitors (NMs). The results show that
the maximum GCR intensities of heavy nuclei ($5\leqslant Z\leqslant 28$,
50$\sim$500 MeV/nuc) at 1 AU during the solar minimum in 2019–2020 break their
previous records, exceeding those recorded in 1997 and 2009 by $\sim$25% and
$\sim$6%, respectively, and are at the highest levels since the space age.
However, the peak NM count rates are lower than those in late 2009. The
difference between GCR intensities and NM count rates still remains to be
explained. Furthermore, we find that the GCR modulation environment during the
solar minimum $P_{24/25}$ are significantly different from previous solar
minima in several aspects, including remarkably low sunspot numbers, extremely
low inclination of the heliospheric current sheet, rare coronal mass
ejections, weak interplanetary magnetic field and turbulence. These changes
are conducive to reduce the level of solar modulation, providing a plausible
explanation for the record-breaking GCR intensities in interplanetary space.

††journal: ApJS

##  1 Introduction

Due to the supersonic expansion of the solar wind (SW), the heliosphere is
like a giant asymmetric bubble filled with various energetic particles. These
particles are the dominant source of space radiation environment, posing a
significant threat to the regular operation of spacecraft and high-altitude
aircraft, and the health of astronauts and flight personnel (e.g., Mertens &
Slaba 2019). Solar energetic particles (SEPs), galactic cosmic rays (GCRs) and
anomalous cosmic rays (ACRs) are the three commonly seen energetic particles
in the interplanetary space.

SEPs, also known as solar cosmic rays, are the particles originating from the
Sun with energies ranging from a few tens of keV to several GeV. They are
frequently associated with coronal mass ejections (CMEs) and solar flares. It
is widely accepted that CME-driven shock is an essential site for accelerating
solar energetic particles (e.g., Hu et al. 2017, 2018; Fu et al. 2019; Ding et
al. 2020). GCRs are often regarded as a stable background, and are believed to
originate from the supernova remnants (SNRs) and accelerated by supernova
blast shocks driven by expanding SNRs. This is confirmed by observations and
numerical simulations (Aharonian et al., 2007, 2011; Ptuskin et al., 2010).
During the quiet periods, the fluxes of some cosmic ray species with high
first ionization potentials, including H, He, C, N, O, Ne and Ar, are
significantly higher than the background GCRs, i.e., anomalous cosmic rays
(Hovestadt et al., 1973; McDonald et al., 1974; Hasebe et al., 1997). A
classical explanation of ACRs is that they originate from the interstellar
medium, in which the neutral atoms drift into the heliosphere and become
singly ionized by the solar wind or the ultraviolet radiation which are
subsequently accelerated to energies above 10 MeV/nuc, mostly occurring at or
near the heliospheric termination shock (Mewaldt et al., 1993; Cummings &
Stone, 2007; Gloeckler et al., 2009). New mechanisms for ACR acceleration have
been put forward (e.g., Giacalone et al., 2012, and references therein), such
as, Zhao et al., 2019 suggested that ACR protons are accelerated in the
heliosheath via reconnection processes. Studying the nature of energetic
particles provides insight into the acceleration mechanisms and the transport
processes between the sources and the observers.

When GCR particles from the outer space (the so-called primary cosmic ray)
encounter the Earth and penetrate the Earth’s atmosphere, they interact with
atmospheric atoms and generate cascades of secondary particles (e.g., Mishev
et al., 2014). The secondary particles (predominantly neutrons and muons at
the ground level) can be measured by the ground-based neutron monitors (NMs)
and muon detectors once arriving at the surface of the Earth. The history of
neutron monitors can be traced back to the 1950s (Bieber, 2013). The
combination of GCR intensities from in-situ spacecrafts and GCR count rates
from NM stations is helpful to sketch a relatively complete picture of cosmic
rays (CRs). In addition, the cosmogenic isotope recorded in tree rings and ice
cores can extend our understanding of GCRs over a longer time scale (e.g.,
Owens & Forsyth, 2013).

The transport of cosmic rays throughout the heliosphere is significantly
influenced by the large-scale solar wind flow and the turbulent interplanetary
magnetic field embedded in it, collectively referred to as “solar modulation”.
Four major modulation processes have been well described by the Parker
transport equation (Parker, 1965), including gradient and curvature drifts,
diffusion through the irregular interplanetary magnetic field (IMF),
convection in the radial expanding solar wind, and adiabatic deceleration (or
adiabatic energy loss) (e.g., Sabbah, 2000; Zhao & Zhang, 2015; Ihongo & Wang,
2016; Zhao et al., 2018). Secular measurements provide direct evidence for
evaluating their relative role in affecting cosmic ray propagation over the
last few decades (Kóta, 2013), but the numerical simulation of cosmic ray
transport is still a highly challenging task. The solar magnetic polarity
(conventionally described as [$qA$], where [$q$] denotes a positively-charged
particle) is an indispensable factor in modulating cosmic ray intensities.
Commonly, the positively charged particles drift inward along the heliospheric
current sheet (HCS) when the heliospheric magnetic field at the north pole
points inward (negative polarity, $qA<0$) and the charged particles drift
outward along the HCS when the heliospheric magnetic field points outward at
the north pole (positive polarity, $qA>0$) (e.g., Jokipii & Thomas, 1981;
Belov, 2000; Thomas et al., 2014). The 22-year heliomagnetic cycle (also
called Hale cycle) results in the alternating sharp (negative polarity,
$qA<0$) and flat-topped (positive polarity, $qA>0$) shape of cosmic ray
intensity (McDonald et al., 2010).

In late 2009, both the GCR intensities measured from near-Earth spacecrafts
and the GCR count rates measured from ground-based NM stations reached their
then all-time maximum levels (e.g., McDonald et al. 2010; Leske et al. 2013).
At that time, the heliospheric environment was unusual in several aspects,
including weakened interplanetary magnetic field and turbulence level, reduced
solar wind dynamic pressure, and prolonged solar minimum (Mewaldt et al.,
2010). The small magnitude of the IMF causes higher particle drift velocities,
and the weakened IMF turbulence leads to larger particle mean free paths
(MFPs) in the solar minimum 2009 than those in its previous solar minima.
These anomalous interplanetary behaviours would reduce the level of GCR
modulation and contribute to the unexpected high GCR intensities in 2009. As
reported, the solar cycle 24 was the weakest in magnitude in the space age
(Hajra, 2021), and it also exhibited some abnormal signs, such as, decreased
sunspots (especially large spots) (Chapman et al., 2014), reduced coronal mass
ejection events (Wang & Colaninno, 2014), and very flattened HCS. The solar
activity seems to be going through a period of “grand minimum” (Jiang & Cao,
2018; Upton & Hathaway, 2018; Gonçalves et al., 2020). Former studies have
predicted that the peak of the GCR intensities in the solar minimum 24/25
(hereafter solar minimum $P_{24/25}$) would exceed or close to the level
recorded in 2009–2010 (e.g., Strauss & Potgieter, 2014; Kuznetsov et al.,
2017; Fu et al., 2020).

The main motivation of this paper is to provide a close re-examination of the
GCR variations during the successive solar cycles 23 and 24, with particular
interest to the maximum GCR intensities observed at 1 AU during the solar
minimum $P_{24/25}$ epoch. The paper is constructed as follows. In Section 2,
we describe the sources of the dataset utilized in this work. In Section 3, we
present the observational results of GCRs and investigate the possible reasons
for the record-breaking GCR intensities in 2019–2020 observed at 1 AU. We
summarize this work in Section 4.

##  2 Data Description

Solar energetic particles and anomalous cosmic rays do not contribute to the
intensities measured by _ACE_ /CRIS, and hence they do not participate in the
following analysis.

GCR intensities at 1 AU are observed by the in-situ Cosmic Ray Isotope
Spectrometer (CRIS) instrument on board the NASA’s _Advanced Composition
Explorer_ (_ACE_) spacecraft, and GCR count rates at the ground level are
monitored by the five neutron monitor stations listed in Table 1.

The _ACE_ spacecraft was launched on 25 August 1997 and has been continuously
monitoring solar wind plasma, interplanetary magnetic fields and energetic
particles (including both solar energetic particles and cosmic rays) at the
Sun-Earth $L$1 Lagrange point (Stone et al., 1998) for nearly 24 years,
spanning the solar cycles 23 and 24. The CRIS instrument was designed to
measure GCR intensities of 24 heavy species (from boron to nickel, within the
energy range 50–500 MeV/nuc). Despite the low abundance ($\sim$1%), the high
atomic number and energy nuclei carry abundant information of the origin of
the cosmic rays, and are of special significance to space radiation (e.g.,
Zhao & Qin, 2013; Fu et al., 2020). With the large geometric acceptance and
high charge and mass resolution, the CRIS instrument records the most detailed
and statistically significant GCR data to date (Stone et al., 1998).
Historical _ACE_ /CRIS observations are well documented and publicly
accessible from the ACE Science Center (ASC) via
http://www.srl.caltech.edu/ACE/ASC/index.html. Here we apply the re-evaluated
level-2 CRIS products, and these data are organized into 27-day Bartels
rotation averages for the period from 25 August 1997 through 31 October 2020.

Pressure-corrected and 27-day averaged GCR count rates are derived from five
neutron monitor stations (HRMS, JUNG, NEWK, OULU, THUL) over the 1968–2020
time period. The detailed information of the five NM stations is given in
Table 1, where $P_{c}$ is the local geomagnetic cutoff rigidity of each NM
station, the characteristic energy (i.e., the median energy) $E_{M}$ is
defined so that cosmic rays with energy higher (or lower) than $E_{M}$
contribute to half of the detector’s counting rate (Usoskin et al., 2008; Zhao
& Zhang, 2016). All data are acquired from the Neutron Monitor Data Base
(NMDB, http://www01.nmdb.eu/).

The monthly mean sunspot number (SSN) is obtained from the Solar Influences
Data Analysis Center (WDC-SILSO, http://www.sidc.be/). The SW speed
($V_{sw}$), the SW dynamic pressure ($P_{d}$), the IMF magnitude ($B$) and the
root mean square variation in the vector IMF ($\delta B$) are acquired from
the OMNIWeb databasde (https://omniweb.gsfc.nasa.gov/ow.html). The HCS tilt
angle and the mean solar polar magnetic field strength are obtained from the
Wilcox Solar Observatory database (http://wso.stanford.edu/). The mean polar
field is defined as $(N-S)/2$, where $N$ is the polar field strength at the
northern pole, $S$ is the polar field strength at the southern pole. The halo
CME lists are collected from the $SOHO$/LASCO HALO CME catalog
(https://cdaw.gsfc.nasa.gov/CME_list/halo/halo.html), and the annual CME rates
are from CACTus CME catalog (http://www.sidc.be/cactus/). Besides, the GCR
radiation dose rates near the lunar surface are measured by the Cosmic Ray
Telescope for the Effects of Radiation (CRaTER) instrument on board the _Lunar
Reconnaissance Orbiter_ (_LRO_) and are available at https://crater-
web.sr.unh.edu/. Note that unless explicitly stated otherwise, all of the
above-mentioned data are processed as 27-day average.

Table 1: List of the neutron monitor stations used in this work

NM Station | Abbrev. | Longitude (deg) | Latitude (deg) | Altitude (m) |  $P_{c}$ (GV) |  $E_{M}$ (GeV)
---|---|---|---|---|---|---
Hermanus | HRMS | 19.22 | -34.42 | 26 | 4.58 | 12.67
Jungfraujoch | JUNG | 7.98 | 46.55 | 3570 | 4.49 | 12.58
Newark | NEWK | -75.75 | 39.68 | 50 | 2.40 | 10.99
Oulu | OULU | 25.47 | 65.05 | 15 | 0.81 | 10.30
Thule | THUL | -68.7 | 76.5 | 26 | 0.30 | 10.17

Note. — $P_{c}$ is the NM’s local geomagnetic cutoff rigidity, $E_{M}$ is the
median energy of each NM station and is defined as: $E_{M}=0.0877\cdot
P_{c}^{2}+0.154\cdot P_{c}+10.12$ (Jämsén et al., 2007; Usoskin et al., 2008).

##  3 Results and discussions

The SSN observations reveal that the solar activity has been persistently
declining since the early 1980s. As shown in Figures 1 and 2(a), the maximum
smoothed SSNs of solar cycles 21 through 24 are 232.9, 212.5, 180.3, and
116.4, respectively, showing a prominent reduction in the solar cycle
amplitude. GCR transport in the heliosphere is sensitive to varying solar
activity. An intense solar activity (such as CME and solar flare) can trigger
a series of space weather effects and effectively prevent GCR particles from
entering the solar system (strong solar modulation), while a quiet Sun is
conductive to enhance GCR fluxes as a consequence of the weakened modulation
level. Next, we review the variations of the solar wind and interplanetary
parameters from solar cycles 20 to 24 in Subsection 3.1; GCR observational
results, including GCR intensities in interplanetary space and GCR counts at
the ground level, are displayed in Subsection 3.2; in Subsection 3.3, we
investigate how the inner heliospheric environment influences the GCR
intensity, with an emphasis on the record-breaking fluxes during the solar
minimum $P_{24/25}$; in Subsection 3.4, we point out a sudden dip in the GCR
intensity during the descending phase of the solar cycle 24 and give a brief
analysis of the possible causes; the measured galactic cosmic radiation doses
on the lunar surface are exhibited in Subsection 3.5.

Figure 1: Yearly averaged sunspot numbers (shaded) and annual spotless days
(histogram) (until October 31, 2020).

###  3.1 A glance of Solar Cycles 20–24

Sunspots were first observed by a telescope in the early 1600s, and the
continuous daily observations began at the Zurich Observatory in the year
1849. The number of sunspots varies periodically with time and exhibits a
remarkable quasi-11 year cycle, which is now widely recognized as a
representative of solar activity. In Figure 1, a striking feature is that the
maximum amplitude of solar cycle has been decreasing since the solar cycle 21,
and the recently complete solar cycle 24 is recorded to be the weakest one in
the era of human space exploration (Hajra, 2021). Besides, there are totally
274 days without sunspots in the year 2019, which is the most in the past 107
years (since the year 1914). The extraordinarily quiet solar minimum offers us
an opportunity to go deep into the conditions prevailing in the past grand
minima periods, such as the Maunder Minimum (1645–1715) and the Dalton Minimum
(1790–1830).

Figure 2: 27-day average solar wind/interplanetary parameters from January 1,
1968 to October 31, 2020. (a) Sunspot number. (b) Mean solar polar field
strength. (c) HCS Tilt angle. (d) IMF magnitude $B$ and its root mean square
$\delta B$. (e) Solar wind speed $V_{sw}$. (f) Solar wind dynamic pressure
$P_{d}$. (g) Pressure-corrected neutron monitor count rates (normalized to the
solar minimum $P_{23/24}$ and multiplied by arbitrary factors for
distinguishing different stations). The shown magnetic field polarity in panel
(b) corresponds to that at the northern pole. The vertical dashed line denotes
the epoch of solar minimum.

In Figure 2, we present the variations of the solar wind/interplanetary
parameters in panels (b)–(f) and the measured NM count rates in panel (g).
Some well-known features of the heliosphere and GCRs can be seen from these
panels. (1) Both the HCS tilt angle, the IMF strength $B$ and its turbulence
$\delta B$ (i.e., root mean square of vector IMF) are positively correlated
with SSN; (2) the GCR count rates at the ground level are inversely related to
SSN; (3) the solar magnetic field reverses its polarity every 11 years,
namely, 22-year heliomagnetic cycle (or Hale cycle); (4) the alternative
peaked (negative polarity, $qA<0$) and plateau (positive polarity, $qA>0$)
shapes of NM count rates during successive solar minima periods is in essence
a manifestation of the Sun’s polar magnetic field reversal, and the maximum NM
count rate during $qA>0$ cycles (1977, 1997, 2019) is approximately 3% lower
than that during $qA<0$ cycles (1987, 2009).

Figure 3: Secular variations of GCR intensities in interplanetary space and NM
count rates. (a) GCR iron intensities from _ACE_ /CRIS at 129.1–428.7 MeV/nuc
(_red curve_) and from _IMP_ -8 at 97.1–432 MeV/nuc (_black dots_ , adapted
from Tylka et al. 1997), compared with Newark GCR count rates (_cyan curve,
right axis_). (b) GCR carbon intensities from _ACE_ /CRIS at 184.8–200.4
MeV/nuc (_red curve_), compared with GCR helium intensities from _IMP8_ /MED
at $\sim$265 MeV/nuc (_black curve_ , adapted from Kuznetsov et al. 2017). (c)
69.4–237.9 MeV/nuc integral oxygen fluxes from _ACE_ /CRIS (_red curve_),
compared with the BON2020 modelled fluxes (_black curve_ , adapted from Slaba
& Whitman 2020). The horizontal dashed line marks the peak GCR intensity in
the solar minimum $P_{24/25}$.

###  3.2 Galactic Cosmic Rays during the Solar Cycle 24

Weak solar activity is expected to reduce GCR modulation level and allows more
GCR particles to penetrate the inner heliosphere. Secular GCR observations
will be helpful to highlight the unusual heliospheric conditions during the
extremely quiet solar minimum $P_{24/25}$. Different from the ground-based NM
count rates, the observation of GCRs in space is short and often
discontinuous, but those early cosmic-ray records (prior to the launch of
_ACE_ spacecraft) in interplanetary space can be rebuilt from some other
satellites or the modern state-of-the-art numerical models (such as the CRÈME
model and the Badhwar-O’Neill model). Figure 3 shows the extended GCR
intensity profiles from 1970 to 2020, in which we compare the iron intensities
from _ACE_ /CRIS with those from _IMP_ -8 in panel (a), the carbon intensities
from _ACE_ /CRIS with the helium intensities from _IMP_ -8 in panel (b), and
the oxygen intensities from _ACE_ /CRIS with the BON2020 simulations in panel
(c). In Figure 3(a), the GCR count rates from NEWK station are also plotted as
a baseline. The peak value of GCR intensities at 1 AU in late 2009 was
previously reported to be the highest in the space age (McDonald et al., 2010;
Mewaldt et al., 2010; Lave et al., 2013; Leske et al., 2013), but it is
obvious that the maximum GCR intensities reach new heights during the solar
minimum $P_{24/25}$, as depicted by the horizontal dashed line in Figure 3.
The recent anomalous high GCR intensities at 1 AU are essentially a response
to the unusual changes in the heliosphere, which will be further discussed in
Subsection 3.3.

Figure 4: (a) _ACE_ /CRIS GCR intensities of element oxygen at seven energy
intervals (69.4–89.0 MeV/nuc, 91.0–122.5 MeV/nuc, 124.0–150.3 MeV/nuc,
151.6–174.9 MeV/nuc, 176.0–197.3 MeV/nuc, 198.3–218.0 MeV/nuc, 219.1–237.9
MeV/nuc). (b) _ACE_ /CRIS GCR intensities of twelve selected GCR species (C,
O, Ne, Mg, Si, S, Ar, Ca, Ti, Cr, Fe, Ni). (c) Average intensities of the
twelve selected GCR species. (d) Neutron monitor count rates. (e) Historical
halo CME events. Note that the intensities in panels (a)–(d) are normalized to
their solar minimum $P_{23/24}$ levels, and the horizontal dashed line
represents the normalized value equal to 1. The pink-arrow dip in the GCR
intensity was caused by CMEs (see Subsection 3.4 for details).

Figures 4(a)–(c) plot the 27-day averaged _ACE_ /CRIS GCR intensities over the
solar cycles 23 and 24. Panel (a) shows the GCR intensities of element oxygen
at seven energy bins, panel (b) the intensities of twelve selected species (C,
O, Ne, Mg, Si, S, Ar, Ca, Ti, Cr, Fe, Ni), and panel (c) the averaged
intensities of these twelve species. For comparison purpose, all profiles are
normalized to the solar minimum $P_{23/24}$. It is clearly noted that the GCR
intensities at 1 AU in late 2009 are much higher than those in 1997–1998 but
slightly lower than those during the solar minimum $P_{24/25}$. The peak value
of GCR intensities in the solar minimum $P_{24/25}$ is $\sim$25% higher than
that in the solar minimum $P_{22/23}$ and $\sim$6% higher than that in the
solar minimum $P_{23/24}$.

Figure 4(d) plots the 27-day averaged NM count rates from 1997 to 2020.
Similar to the GCR intensity profiles, NM count rates also reach the highest
level in late 2009 (Moraal & Stoker, 2010), revealing that both low ($<$1 GeV)
and high energy (several GeV) GCR particles have more chances to reach the
Earth during the solar minimum $P_{23/24}$ (Mewaldt et al., 2010). However,
the peak value of NM count rates in 2019–2020 does not exceed its 2009 level,
which is not in accord with the observed GCR intensities in interplanetary
space. The ground-based NM station only observes the high-energy GCRs (several
GeV) as a consequence of the shielding effects from the Earth’s magnetosphere
and atmosphere, yet the _ACE_ /CRIS instrument measures relatively low-energy
particles ($<$1 GeV). The median energy ($E_{M}$) of the five selected NMs
ranges from 10.17 to 12.67 GeV, corresponding to the geomagnetic cutoff
rigidity of 0.30 $\sim$ 4.58 GV. The different responses of GCRs in space and
at the ground level to varying solar activity may arise from the distinct
modulation processes between high and low energy particles as they propagate
in the heliosphere. We infer that in the solar minimum $P_{24/25}$, the low-
energy GCR particles are more likely to be influenced by the weakening solar
modulation compared to the high-energy particles. In addition, the NM count
rates are sensitive to the complex changes in the Earth’s environmental
fluctuations, such as the atmospheric pressure, atmospheric temperature,
atmospheric water vapor, instrumental temperature, dynamic magnetospheric
condition, and snow effect (see Moraal & Stoker, 2010, and references
therein). The accurate reason for the difference between GCR intensities in
interplanetary space and at the ground level is still not known completely.

Figure 4(e) plots the historical halo CMEs identified from the Large Angle and
Spectrometric Coronagraph (LASCO) on the _Solar and Heliospheric Observatory_
(_SOHO_) mission since 1996. We put the discussion of this figure in
Subsection 3.4.

Figures 5(a) and 5(b) display the _ACE_ /CRIS observed energy spectra of GCR
elements C, N, O, Fe, and Figures 5(c) and 5(d) show the spectral ratios of
2019 to 2009, and 2019 to 1997, respectively. During the solar minimum
$P_{24/25}$, the significant enhancement in GCR intensity is observed at all
lower and higher energies. Although GCR nuclei differ in energy bands, their
intensity ratios are relatively consistent. The ratio of 2019 to 2009 is in
the range of 1.20 $\sim$ 1.30, and the ratio of 2019 to 1997 is in the range
of 1.05 $\sim$ 1.10. This is basically coincident with the previous analysis,
in which the peak GCR intensities are greater by $\sim$25% and $\sim$6% during
the solar minimum $P_{24/25}$ than those in the late 1997 and 2009,
respectively. Furthermore, it should be noted that the ratios at lower
energies are slightly larger than those at higher energies, indicating that
the low-energy GCRs are more sensitive to the variation of solar modulation
than the high-energy ones.

Figure 5: (a) and (b) show the comparison of the _ACE_ /CRIS GCR differential
energy spectra between 2019 and 2009, and between 2019 and 1997, respectively.
Energy spectra are multiplied by arbitrary scale factor. Solid and dashed
lines are quadratic fits to the experimental data. (c) and (d) show the
spectral ratios of the intensity between 2019 and 2009, and between 2019 and
1997, respectively.  Figure 6: Comparison of 27-day averaged GCR intensities
at 1 AU between the solar cycles 23 (1998–2010) and 24 (2010–2020), for oxygen
(a) and iron (b).

In Figure 6, we compare the _ACE_ /CRIS 27-day averaged GCR intensities
between the solar cycles 23 and 24, with panel (a) for 151.6–174.9 MeV/nuc
oxygen and panel (b) for 170.8–232.9 MeV/nuc iron. Interestingly, the GCR
intensities are significant higher in the solar cycle 24 than in the solar
cycle 23, for both ascending and descending phases, which is the desired
result of the less active Sun and weakened solar modulation in the solar cycle
24. Furthermore, the bottom of the GCR profile in the solar cycle 23 is deep
and wide, but it is shallow and short-lasting in the solar cycle 24. The
former reflects a strong and continuous solar modulation, and the latter
corresponds to a weakening and short duration solar modulation.

###  3.3 Inner Heliospheric Environment during the Solar Cycle 24

The record-breaking GCR intensities at 1 AU in the solar minimum $P_{24/25}$
naturally raise the question of the contributing factors. A general solution
to this question is to investigate the inner heliospheric conditions,
including the rate of coronal mass ejection (CME), the Sun’s polar magnetic
field strength, the tilt angle of HCS, the strength and turbulence of the
interplanetary magnetic field, as well as the speed and dynamic pressure of
the solar wind.

####  3.3.1 Coronal Mass Ejection Rate

CMEs commonly carry stronger magnetic field than the surrounding solar wind,
and cause the violent interplanetary disturbances and magnetic shielding of
charged particles when the CME-driven shocks and/or magnetic clouds pass
through. Consequently, large CMEs effectively prevent cosmic ray particles
from diffusing in the inner heliosphere (Wibberenz et al., 1998; Cane, 2000;
Kilpua et al., 2017). Moreover, CMEs are frequently accompanied with other
forms of solar activity, such as solar flare (Vršnak, 2016; Syed Ibrahim et
al., 2018), eruptive prominence and X-ray sigmoids (Gibson et al., 2002;
Pevtsov, 2002), which further enhances the shielding effectiveness of GCR
particles.

In Figure 7(a), we present the annual rate of CME acquired from the automated
CACTus catalog between 1997 and 2019. It shows that the rate of CME in 2019 is
0.29, which is $\sim$56% and $\sim$52% lower than that in 1997 and 2009,
respectively. The lower number of CMEs leads to a fewer magnetic
irregularities to inhibit GCR diffusion, which in turn contributes to an
increase in GCR flux. In addition to the insufficient number, Wang & Colaninno
(2014) suggested that the speed and the mass of the CMEs in the solar cycle 24
are much smaller than those in the solar cycle 23. These rare, slow speed and
less massive CMEs to some extent weaken the solar modulation of cosmic rays
(Paouris, 2013).

Figure 7: 27-day averaged solar wind/interplanetary parameters from 1996 to
2020. (a) Sunspot number and annual CME rate. (b) Mean solar polar field
strength. (c) HCS Tilt angle. (d) IMF magnitude $B$ and its root mean square
$\delta B$. (e) Estimated parallel MFP $\lambda_{\parallel}$. (f) Solar wind
speed (_black curve_) and dynamic pressure (_cyan curve_). (g) 69.4–237.9
MeV/nuc GCR oxygen intensity from _ACE_ /CRIS (normalized to the solar minimum
$P_{23/24}$), and the horizontal dashed line represents the normalized GCR
intensity equal to 1. The pink-arrow dip in the GCR intensity was caused by
CMEs (see Subsection 3.4 for details).

####  3.3.2 Solar Polar Field

The polarity of the Sun’s polar fields reverses or flips approximately every
11 years during the period of solar maximum (Babcock & Livingston, 1958;
Babcock, 1959; Owens & Forsyth, 2013). The last two reversals happen around
2001 and 2014, respectively, and it is currently in the positive polarity
($qA>0$). The Sun’s polar fields also display anomalous properties in the
recent two cycles, for example, the noticeably weakened polar magnetic fields
in the solar minimum $P_{23/24}$ (Wang et al., 2009), the prolonged unusual
hemispheric asymmetry of the polar field reversal pattern in the solar cycle
24 (Sun et al., 2015; Mordvinov et al., 2016; Janardhan et al., 2018).

The Sun’s polar fields influence the penetration of GCRs into the heliosphere
by altering the stream structure of the solar wind and the magnitude of the
IMF (Lee et al., 2009). It is reported that the Sun’s polar field strength in
the solar cycle 23 is $\sim$50% lower than that in the cycles 21–22, and the
flux of the open magnetic field line at the Earth orbit reaches the lowest
since 1963 (Ahluwalia & Ygbuhay, 2010). During the solar cycle 24, the Sun’s
polar field strength begins to recover after 2014 and enters into a plateau
period after the year 2017 with a stable strength $\sim$0.6 G, which is very
close to or slightly higher than that in the solar cycle 23, as shown in
Figures 2(b) and 7(b). It is possible that more GCR particles enter into the
heliosphere as a consequence of the weakening solar polar field.

####  3.3.3 Tilt Angle of the Helisopheric Current Sheet

Previous studies have studied the influence of a wavy interplanetary current
sheet on the GCR intensity, and found an inverse correlation between the HCS
tilt angle and the GCR intensity (Jokipii & Thomas, 1981; Potgieter et al.,
2001). It is because that the GCR particles mainly drift outward ($qA>0$
polarity) or inward ($qA<0$ polarity) along the HCS near the solar equator,
and a large tilt angle means that the cosmic ray particles have to drift a
longer path length toward the Earth, leading to an increase in the time lag
between the source and the observer (Ferreira & Potgieter, 2004; Mewaldt et
al., 2010; Zhao et al., 2014; Ross & Chaplin, 2019). The variation of the HCS
tilt angle may have important, even the dominant influences on the GCR
modulation process during the solar minimum epoch. In addition, the drift of
GCRs along the HCS plays a more significant role than the diffusion when
$qA>0$.

Figure 7(c) shows the temporal variation of the HCS tilt angle. The tilt angle
begins to decrease after 2014 and reaches the minimum value of $\sim$2.1∘ in
April 2020, which is $\sim$22% lower than that in the solar minimum
$P_{22/23}$ and $\sim$53% lower than that in the solar minimum $P_{23/24}$.
This extremely low tilt angle reflects a very close to the solar equatorial
HCS, which results in an enhanced outward drift velocity (for $qA>0$) and the
greatly increased GCR intensity. As shown in Figure 7(g), the HCS tilt angle
begin to increase since May 2020 and reaches a value of 14.8∘ by the end of
November 2020, accompanied by the gradual decrease of the GCR flux.

####  3.3.4 IMF Strength and Turbulence Level

The interplanetary magnetic field has important influences on both drifts and
the diffusion of the cosmic rays, and the cosmic-ray intensities are found to
be negatively correlated with the IMF strength (denoted by $B$). The drift
velocity of the GCRs increases with the decrease of $B$ (e.g., Jokipii & Levy,
1977; Jokipii & Kota, 1989). However, the turbulence in the IMF regulates the
pitch-angle scattering of the cosmic rays, where the diffusion coefficient of
the cosmic rays is proportional to $1/B$ or some power of $1/B$ (e.g., Jokipii
& Davila, 1981; Ferreira & Potgieter, 2004). The cosmic-ray diffusion
associated with the turbulent IMF includes both parallel and perpendicular
components, and the relation between diffusion coefficient ($\kappa$) and
particle’s mean free path ($\lambda$) is given by:

$\lambda_{\perp,\parallel}\equiv 3\kappa_{\perp,\parallel}/v,$ (1)

where $v$ is the particle velocity, $\lambda_{\parallel}$ and
$\lambda_{\perp}$ are the parallel and perpendicular MFPs,
$\kappa_{\parallel}$ and $\kappa_{\perp}$ are the parallel and perpendicular
diffusion coefficients, respectively (Zank et al., 1998; Pei et al., 2010).
The radial MFP $\lambda_{rr}$ governed by the parallel diffusion
$\lambda_{\parallel}$ is almost constant within the inner heliosphere (Zhao et
al., 2017). The perpendicular MFP $\lambda_{\perp}$ is nearly three orders of
magnitude smaller than the parallel MFP $\lambda_{\parallel}$ (Zhao et al.,
2018). The perpendicular diffusion $\kappa_{\perp}$ is often assumed to scale
as the parallel diffusion $\kappa_{\parallel}$ (Ferreira & Potgieter, 2004).

Considering the standard quasi-linear theory (QLT) and assuming magnetostatic
turbulence, the parallel MFP $\lambda_{\parallel}$ is proportional to
$B^{5/3}/(\delta B)^{2}$, where $B$ is the mean IMF magnitude and $\delta B$
is the root mean square variation in the vector IMF (Zank et al., 1998;
Mewaldt et al., 2010). With the simplified relation $\lambda_{\parallel}\sim
B^{5/3}/(\delta B)^{2}$, the parallel MFP component $\lambda_{\parallel}$ at 1
AU is estimated over the cycles 23 and 24 (Figure 7(e)). The IMF magnitude $B$
reaches a minimum value of 3.8 nT in the solar minimum $P_{24/25}$, which is
$\sim$24% smaller than that in the solar minimum of cycles 20 to 22 ($\sim$5
nT) but $\sim$5.6% larger than that in late 2009 ($\sim$3.6 nT). The IMF
turbulence $\delta B$ is $\sim$2.1 nT in the solar minimum $P_{24/25}$, which
is $\sim$5% higher than that in the solar minimum $P_{23/24}$. The weak IMF
strength and turbulence lead to a noticeable increase in the estimated cosmic-
ray MFP ($\lambda_{\parallel}$), and the maximum value of
$\lambda_{\parallel}$ in the solar minimum $P_{24/25}$ exceed those in 1997 by
$\sim$31% and those in 2009 by $\sim$10%. It is seen that the prominent peak
on the $\lambda_{\parallel}$ profile (around March 2020) is followed by the
record-breaking GCR intensity.

####  3.3.5 Solar Wind

The solar wind condition (including speed, density, and temperature) varies
over time and influences the GCR transport throughout the heliosphere. The SW
speed affects both the outward convective rate of GCRs and the adiabatic
energy loss rate, and generally speaking, a low solar wind speed reduces the
outward convection rate and the adiabatic cooling energy loss rate. The GCR
intensities are found to be anticorrelated with the solar wind speed, which
reveals that the lower the solar wind, the higher the GCR intensities and
vice-versa (e.g., Ihongo & Wang, 2016; Zhao et al., 2014).

Figure 7(f) shows the temporal variation of the SW speed ($V_{sw}$, _black
curve_) and the SW dynamic pressure ($P_{d}$, _blue curve_). The average
speeds during the last three solar minima periods are $371\pm 19$ km/s,
$363\pm 26$ km/s, and $374\pm 24$ km/s, respectively, and the corresponding
dynamic pressures are $2.4\pm 0.2$ nPa, $1.5\pm 0.1$ nPa, and $1.8\pm 0.1$
nPa, respectively. It is clear that in the solar minimum $P_{23/24}$, the SW
speed and dynamic pressure reach their lowest values over the past three solar
minima, which is thought to be an important reason for the unusual high GCR
intensities in late 2009 (Leske et al., 2013; Zhao et al., 2014). In contrast,
the enhanced SW speed and dynamic pressure in the solar minimum $P_{24/25}$
are not conducive to the increase of GCR intensity to a certain extent.

###  3.4 A Sudden Dip in the GCR Intensity during the Descending Phase of
Cycle 24

During the descending phase of the solar cycle 24, the GCR intensity observed
at 1 AU occurs a short-duration but significant dip in the second half of 2017
(around September), after which the GCR intensity continues to rise. This dip
is also recorded by the ground-based NM stations, as marked by the pink arrows
in Figures 4 and 7.

The solar wind and interplanetary disturbances are naturally considered to be
the causes of this GCR anomaly. The successive CMEs between September 4 to 10,
2017 (including the fastest halo CME on 10 September 2017, see Figure 4(e))
lead to a highly disturbed interplanetary and geospace environment (Guo et
al., 2018; Lee et al., 2018; Ding et al., 2020). Furthermore, the increased
sunspot number (Figure 7(a)), the elevated HCS tilt angle (Figure 7(c)) and
the increased SW speed (Figure 7(f)) may also result in a drop in the GCR
intensity as a consequence of the enhanced solar modulation.

###  3.5 GCR Radiation Dose Rate on the Lunar Surface

The deep-space radiation environment, consisting of solar and galactic cosmic
rays, poses a great threat to the manned space missions. Therefore, the long-
term and the short-term radiation effects must be considered before designing
a deep space mission. As a stable background of the energetic particles, GCRs
are the most difficult to shield against and can cause severe hazards to
astronauts and precision payloads on spacecrafts. Schwadron et al., 2010
previously reported that the GCR dose rates near the lunar surface during the
prolonged solar minimum $P_{23/24}$ are the largest since the year 1987.
Considering that the peak value of the GCR intensities in the solar minimum
$P_{24/25}$ is much higher than that in the 2009 solar minimum, we naturally
want to know what the space radiation environment is like now.

Figure 8: GCR radiation dose rates on the lunar surface measured by the _LRO_
/CRaTER. The horizontal dash-dotted line marks the peak value of does rate in
the solar minimum $P_{24/25}$.

The Cosmic Ray Telescope for the Effects of Radiation (CRaTER) instrument on
board the _Lunar Reconnaissance Orbiter_ (_LRO_) was intended to investigate
the lunar radiation environment, and is currently orbiting the Moon at an
altitude of 50 km (Spence et al., 2010). Since its launch on June 18, 2009,
_LRO_ has been measuring the lunar radiation environment for nearly 12 years.
Figure 8 shows the hourly and daily average dose rates observed by the _LRO_
/CRaTER, and the horizontal dash-dotted line is the peak value of dose rates
in the solar minimum $P_{24/25}$. It can be seen that the dose rates are
tightly correlated with the solar activity as well as the GCR intensities. In
the recent solar minimum, the GCR radiation dose rate and the GCR intensity
increase substantially because of the weakening solar modulation. We find that
the peak value of dose rates in the first half of 2020 is $\sim$5% higher than
that in 2009–2010, which basically corresponds to the $\sim$6% increase in the
GCR intensities measured at 1 AU. We infer that this may be the highest dose
rates on the lunar surface since the 1980s, which puts forward higher
requirements for radiation shielding and protection in the outer space. Our
result can be used to simulate the GCR space radiation risk to human beings in
the near-Earth space, and provide a certain reference value in designing the
spacecraft materials in future deep space exploration missions.

##  4 Conclusions

The Sun was extraordinarily quiet during the recently complete solar cycle 24,
including very low sunspot numbers, extremely flat heliospheric current sheet,
reduced CME eruption rate, weak interplanetary magnetic field and turbulence
level. The observed galactic cosmic ray intensities (both in interplanetary
space and at the ground level) are sensitive to the heliospheric conditions,
the so-called solar modulation. In this work, we focus on the variations of
the GCR intensities for the period from 1997 through 2020 with measurements
from the _ACE_ /CRIS instrument, and investigate the influence of inner
heliospheric environments on the GCR intensities. Besides, the long-term
variations of the ground-based NM count rates are also studied. The
conclusions are summarized as follows.

(1) The GCR intensities observed at 1 AU reach the highest levels in the solar
minimum $P_{24/25}$ since the launch of _ACE_ spacecraft, which is $\sim$25%
higher than that in the solar minimum $P_{22/23}$ and $\sim$6% higher than
that in the solar minimum $P_{23/24}$. With the extension of _IMP_ -8
measurements and BON2020 numerical simulation results, we find that the GCR
intensities during the recent solar minimum are at the highest levels since
the space age, having the particle energy from tens to several hundred
MeV/nuc. The record-breaking GCR intensities at 1 AU increase the radiation
dose rates near the lunar surface by $\sim$5% in comparison to the solar
minimum $P_{23/24}$, from the measurements of _LRO_ /CRaTER instrument.

(2) The peak value of NM count rates in the solar minimum $P_{24/25}$ is lower
than that in late 2009. The difference between GCR intensities in
interplanetary space and NM count rates seems to be relevant to the different
modulation processes of the high and low energy particles. We think that the
high-energy GCR particles are not heavily modulated as the low-energy ones,
which means the low-energy GCR particles are more likely to be influenced by
varying solar modulation. An alternate or additional possibility is that the
NM count rates are sensitive to the conditions of the Earth’s magnetosphere
and atmosphere. It remains to be further studied in a future work.

(3) We find that during the solar minimum $P_{24/25}$, the mean solar polar
field strength remains weak and is close to the value in the 2009 solar
minimum; the HCS tilt angle reaches a minimum value of $\sim$2.1∘ in April
2020, which is $\sim$22% lower than that in the solar minimum $P_{22/23}$ and
$\sim$53% lower than that in the solar minimum $P_{23/24}$; the CME eruption
rate is very small, which is less than half of the CME rate during the solar
minima $P_{22/23}$ and $P_{23/24}$; the strength and turbulence of IMF keep at
relatively low levels, which causes an $\sim$10% increase in the estimated
cosmic ray MFP ($\lambda_{\parallel}$) compared to that in late 2009. All
these factors collectively contribute to less solar modulation and unusual
increase in the GCR intensity.

This study is a pure statistical analysis of the GCR variations in the
relatively low energy band (50–500 MeV/nuc). In the meanwhile, we discuss the
unusual heliospheric conditions in the inner heliosphere in order to explain
our findings of the record-breaking GCR intensities at 1 AU. How much of the
heliospheric changes influence the GCR intensities needs to be studied.
Further work includes the quantitative diagnosis of the effects of drifts,
diffusion, and convection on the cosmic-ray modulation from the perspective of
numerical simulation (e.g., Strauss & Potgieter, 2014; Zhao et al., 2014; Shen
& Qin, 2018; Shen et al., 2019).

This work is supported by the Science and Technology Development Fund, Macau
SAR (File Nos. 008/2017/AFJ, 0042/2018/A2, and 0002/2019/APD) and the National
Natural Science Foundation of China (Grant No. 11761161001). The authors
appreciate beneficial discussions with Dr. Laxman Adhikari at UAH.

## References

  * Aharonian et al. (2007) Aharonian, F., Akhperjanian, A. G., Bazer-Bachi, A. R., et al. 2007, A&A, 464, 235
  * Aharonian et al. (2011) Aharonian, F., Akhperjanian, A. G., Bazer-Bachi, A. R., et al. 2011, A&A, 531, C1
  * Ahluwalia & Ygbuhay (2010) Ahluwalia, H. S. & Ygbuhay, R. C. 2010, in 12th International Solar Wind Conference, 1216, 699
  * Babcock & Livingston (1958) Babcock, H. D. & Livingston, W. C. 1958, Science, 127, 1058
  * Babcock (1959) Babcock, H. D. 1959, ApJ, 130, 364
  * Belov (2000) Belov, A. 2000, Space Sci. Rev., 93, 79
  * Bieber (2013) Bieber, J. W. 2013, Centenary Symposium 2012: Discovery of Cosmic Rays, 1516, 234
  * Cane (2000) Cane, H. V. 2000, Space Sci. Rev., 93, 55
  * Chapman et al. (2014) Chapman, G. A., de Toma, G., & Cookson, A. M. 2014, Sol. Phys., 289, 3961
  * Cummings & Stone (2007) Cummings, A. C. & Stone, E. C. 2007, The Composition of Matter, 389
  * Ding et al. (2020) Ding, Z. Y., Li, G., Hu, J. X., et al. 2020, Research in Astronomy and Astrophysics, 20, 145
  * Ferreira & Potgieter (2004) Ferreira, S. E. S. & Potgieter, M. S. 2004, ApJ, 603, 744
  * Fu et al. (2019) Fu, S., Jiang, Y., Airapetian, V., et al. 2019, ApJ, 878, L36
  * Fu et al. (2020) Fu, S., Zhao, L., Zank, G. P., et al. 2020, Science China Physics, Mechanics, and Astronomy, 63, 219511
  * Giacalone et al. (2012) Giacalone, J., Drake, J. F., & Jokipii, J. R. 2012, Space Sci. Rev., 173, 283
  * Gibson et al. (2002) Gibson, S. E., Fletcher, L., Del Zanna, G., et al. 2002, ApJ, 574, 1021
  * Gloeckler et al. (2009) Gloeckler, G., Fisk, L. A., Geiss, J., et al. 2009, From the Outer Heliosphere to the Local Bubble, 163
  * Gonçalves et al. (2020) Gonçalves, Í. G., Echer, E., & Frigo, E. 2020, Advances in Space Research, 65, 677
  * Guo et al. (2018) Guo, J., Dumbović, M., Wimmer-Schweingruber, R. F., et al. 2018, Space Weather, 16, 1156
  * Hajra (2021) Hajra, R. 2021, Sol. Phys., 296, 33
  * Hasebe et al. (1997) Hasebe, N., Kondoh, K., Mishima, Y., et al. 1997, Advances in Space Research, 19, 813
  * Hovestadt et al. (1973) Hovestadt, D., Vollmer, O., Gloeckler, G., et al. 1973, Phys. Rev. Lett., 31, 650
  * Hu et al. (2017) Hu, J., Li, G., Ao, X., et al. 2017, Journal of Geophysical Research (Space Physics), 122, 10, 938
  * Hu et al. (2018) Hu, J., Li, G., Fu, S., et al. 2018, ApJ, 854, L19
  * Ihongo & Wang (2016) Ihongo, G. D. & Wang, C. H. T. 2016, Ap&SS, 361, 44
  * Jämsén et al. (2007) Jämsén, T., Usoskin, I. G., Räihä, T., et al. 2007, Advances in Space Research, 40, 342
  * Janardhan et al. (2018) Janardhan, P., Fujiki, K., Ingale, M., et al. 2018, A&A, 618, A148
  * Jiang & Cao (2018) Jiang, J. & Cao, J. 2018, Journal of Atmospheric and Solar-Terrestrial Physics, 176, 34
  * Jokipii & Levy (1977) Jokipii, J. R. & Levy, E. H. 1977, ApJ, 213, L85
  * Jokipii & Kota (1989) Jokipii, J. R. & Kota, J. 1989, Geophys. Res. Lett., 16, 1
  * Jokipii & Davila (1981) Jokipii, J. R. & Davila, J. M. 1981, ApJ, 248, 1156
  * Jokipii & Thomas (1981) Jokipii, J. R. & Thomas, B. 1981, ApJ, 243, 1115
  * Kilpua et al. (2017) Kilpua, E., Koskinen, H. E., & Pulkkinen, T. I. 2017, Living Reviews in Solar Physics, 14(1), 5
  * Kóta (2013) Kóta, J. 2013, Space Science Reviews, 176, 391
  * Kuznetsov et al. (2017) Kuznetsov, N. V., Popova, H., & Panasyuk, M. I. 2017, Journal of Geophysical Research (Space Physics), 122, 1463
  * Lave et al. (2013) Lave, K. A., Wiedenbeck, M. E., Binns, W. R., et al. 2013, ApJ, 770, 117
  * Lee et al. (2009) Lee, C. O., Luhmann, J. G., Zhao, X. P., et al. 2009, Sol. Phys., 256, 345
  * Lee et al. (2018) Lee, C. O., Jakosky, B. M., Luhmann, J. G., et al. 2018, Geophys. Res. Lett., 45, 8871
  * Leske et al. (2013) Leske, R. A., Cummings, A. C., Mewaldt, R. A., et al. 2013, Space Sci. Rev., 176, 253
  * McDonald et al. (1974) McDonald, F. B., Teegarden, B. J., Trainor, J. H., et al. 1974, ApJ, 187, L105
  * McDonald et al. (2010) McDonald, F. B., Webber, W. R., & Reames, D. V. 2010, Geophys. Res. Lett., 37, L18101
  * Mertens & Slaba (2019) Mertens, C. J. & Slaba, T. C. 2019, Space Weather, 17, 1650
  * Mewaldt et al. (1993) Mewaldt, R. A., Cummings, A. C., Cummings, J. R., et al. 1993, Geophys. Res. Lett., 20, 2263
  * Mewaldt et al. (2010) Mewaldt, R. A., Davis, A. J., Lave, K. A., et al. 2010, ApJ, 723, L1
  * Mishev et al. (2014) Mishev, A. L., Kocharov, L. G., & Usoskin, I. G. 2014, Journal of Geophysical Research (Space Physics), 119, 670
  * Moraal & Stoker (2010) Moraal, H. & Stoker, P. H. 2010, Journal of Geophysical Research (Space Physics), 115, A12109
  * Mordvinov et al. (2016) Mordvinov, A., Pevtsov, A., Bertello, L., et al. 2016, Solar-Terrestrial Physics, 2, 3
  * Owens & Forsyth (2013) Owens, M. J. & Forsyth, R. J. 2013, Living Reviews in Solar Physics, 10, 5
  * Paouris (2013) Paouris, E. 2013, Sol. Phys., 284, 589
  * Parker (1965) Parker, E. N. 1965, Planet. Space Sci., 13, 9
  * Pei et al. (2010) Pei, C., Bieber, J. W., Breech, B., et al. 2010, Journal of Geophysical Research (Space Physics), 115, A03103
  * Pevtsov (2002) Pevtsov, A. A. 2002, Sol. Phys., 207, 111
  * Ptuskin et al. (2010) Ptuskin, V., Zirakashvili, V., & Seo, E.-S. 2010, ApJ, 718, 31
  * Potgieter et al. (2001) Potgieter, M. S., Burger, R. A., & Ferreira, S. E. S. 2001, Space Sci. Rev., 97, 295
  * Ross & Chaplin (2019) Ross, E. & Chaplin, W. J. 2019, Sol. Phys., 294, 8
  * Sabbah (2000) Sabbah, I. 2000, Geophys. Res. Lett., 27, 1823
  * Schwadron et al. (2010) Schwadron, N. A., Boyd, A. J., Kozarev, K., et al. 2010, Space Weather, 8, S00E04
  * Shen & Qin (2018) Shen, Z. N. & Qin, G. 2018, ApJ, 854, 137
  * Shen et al. (2019) Shen, Z. N., Qin, G., Zuo, P., et al. 2019, ApJ, 887, 132
  * Slaba & Whitman (2020) Slaba, T. C., & Whitman, K. 2020, Space Weather, 18, 6
  * Spence et al. (2010) Spence, H. E., Case, A. W., Golightly, M. J., et al. 2010, Space Sci. Rev., 150, 243
  * Stone et al. (1998) Stone, E. C., Cohen, C. M. S., Cook, W. R., et al. 1998, Space Sci. Rev., 86, 285
  * Strauss & Potgieter (2014) Strauss, R. D., & Potgieter, M. S. 2014, Solar Physics, 289, 3197
  * Sun et al. (2015) Sun, X., Hoeksema, J. T., Liu, Y., et al. 2015, ApJ, 798, 114
  * Syed Ibrahim et al. (2018) Syed Ibrahim, M., Shanmugaraju, A., Moon, Y. J., et al. 2018, Advances in Space Research, 61, 540
  * Thomas et al. (2014) Thomas, S. R., Owens, M. J., & Lockwood, M. 2014, Sol. Phys., 289, 407
  * Tylka et al. (1997) Tylka A. J., Adams J. H.,Boberg P. R., et al., 1997, IEEE Trans. Nucl. Sci. 44, 2150
  * Upton & Hathaway (2018) Upton, L. A. & Hathaway, D. H. 2018, Geophys. Res. Lett., 45, 8091
  * Usoskin et al. (2008) Usoskin, I. G., Braun, I., Gladysheva, O. G., et al. 2008, Journal of Geophysical Research (Space Physics), 113, A07102
  * Vršnak (2016) Vršnak, B. 2016, Astronomische Nachrichten, 337, 1002
  * Wang et al. (2009) Wang, Y. M., Robbrecht, E., & Sheeley, N. R. 2009, ApJ, 707, 1372
  * Wang & Colaninno (2014) Wang, Y. M. & Colaninno, R. 2014, ApJ, 784, L27
  * Wibberenz et al. (1998) Wibberenz, G., Le Roux, J. A., Potgieter, M. S., et al. 1998, Space Sci. Rev., 83, 309
  * Zank et al. (1998) Zank, G. P., Matthaeus, W. H., Bieber, J. W., Moraal, H. 1998, Journal of Geophysical Research: Space Physics, 103, 2085
  * Zhao & Qin (2013) Zhao, L. L. & Qin, G. 2013, Journal of Geophysical Research (Space Physics), 118, 1837
  * Zhao et al. (2014) Zhao, L. L., Qin, G., Zhang, M., et al. 2014, Journal of Geophysical Research (Space Physics), 119, 1493
  * Zhao & Zhang (2015) Zhao, L. L. & Zhang, H. 2015, ApJ, 805, 6
  * Zhao & Zhang (2016) Zhao, L. L. & Zhang, H. 2016, ApJ, 827, 13
  * Zhao et al. (2017) Zhao, L. L., Adhikari, L., Zank, G. P., et al. 2017, ApJ, 849, 88
  * Zhao et al. (2018) Zhao, L. L., Adhikari, L., Zank, G. P., et al. 2018, ApJ, 856, 94
  * Zhao et al. (2019) Zhao, L. L., Zank, G. P., Hu, Q., et al. 2019, ApJ, 886, 144