# Xavier-Enabled Extreme Reservoir Machine for Millimeter-Wave Beamspace
Channel Tracking

Hosein Zarini†, Mohammad Robat Mili§, Mehdi Rasti†,⋆, Pedro H. J.
Nardelli⋆,⋆⋆, and Mehdi Bennis⋆⋆ †Department of Computer Engineering,
Amirkabir University of Technology, Tehran, Iran
§Electronics Research Institute, Sharif University of Technology, Tehran, Iran
⋆Lappeenranta-Lahti University of Technology, Lappeenranta, Finland
⋆⋆University of Oulu, Oulu, Finland

###### Abstract

In this paper, we propose an accurate two-phase millimeter-Wave (mmWave)
beamspace channel tracking mechanism. Particularly in the first phase, we
train an extreme reservoir machine (ERM) for tracking the historical features
of the mmWave beamspace channel and predicting them in upcoming time steps.
Towards a more accurate prediction, we further fine-tune the ERM by means of
Xavier initializer technique, whereby the input weights in ERM are initially
derived from a zero mean and finite variance Gaussian distribution, leading to
49% degradation in prediction variance of the conventional ERM. The proposed
method numerically improves the achievable spectral efficiency (SE) of the
existing counterparts, by 13%, when signal-to-noise-ratio (SNR) is 15dB. We
further investigate an ensemble learning technique in the second phase by
sequentially incorporating multiple ERMs to form an ensembled model, namely
adaptive boosting (AdaBoost), which further reduces the prediction variance in
conventional ERM by 56%, and concludes in 21% enhancement of achievable SE
upon the existing schemes at SNR = 15dB.

###### Index Terms:

Millimeter-Wave (mmWave) beamspace, extreme reservoir machine (ERM), Xavier
initializer, ensemble learning, adaptive boosting (AdaBoost).

##  I Introduction

In recent years, beamspace multiple-input-multiple-output MIMO technology[1]
has been attended into a great extent, so as to compensate for the inefficient
functionality of the conventional massive MIMO architecture in high-frequency
bands, such as millimeter-Wave (mmWave) communications. In conventional mmWave
massive MIMO systems, each antenna element requires a dedicated radio-
frequency (RF) chain111RF chains are known as dominant modules in energy
consumption, hardware cost and complexity order of conventional massive MIMO
systems., which practically is prohibited. In contrast, the beamspace
technology employs a large-scale antenna array mounted on the focal surface of
an energy-focusing electromagnetic lens, whereby limited RF chains are
concentrated on few dominant mmWave directions (beams), leading to a
remarkable reduction in system dimensionality, hardware cost and energy
consumption.

In a lens-aided architecture yet, the beamspace channel is very hard to be
precisely estimated. In this regard, due to the severe computational burdens,
the earlier channel estimation schemes e.g.,[2], based on complicated
optimization techniques are ill-suited in current real-time applications.
Thanks to their computational efficiency, deep learning techniques have been
widely adopted for wireless network research over the recent years. For
instance, the authors of [3] and [4], applied deep learning techniques to
overcome the computational burdens and improve the accuracy of the
conventional approximate message passing (AMP)[5] scheme for beamspace channel
estimation. In vehicular communications as well, the authors of[6]
investigated the long-short-term-memory (LSTM) as a deep learning method for
real-time channel estimation. In [7], the authors exploited environmental
samples to scrutinize the performance of the existing deep learning schemes,
proposed for channel estimation. In specific, they investigated deep learning
schemes in literature for predicting the beam angles on transceivers. The
results nevertheless, revealed non-negligible errors, principally stemmed from
training challenges such as overfitting in deep learning techniques.

Towards alleviating this drawback, we propose a precise two-phase mmWave
beamspace channel estimation scheme in this paper from a time-series
prediction perspective (i.e., a beamspace channel tracking scheme) as follows.

  * •

We utilize an extreme reservoir machine (ERM)[8] in the first phase for
tracking the historical features of the mmWave beamspace channel as time-
series sequences, and predict them in upcoming time steps, to construct the
beamspace channel. Despite [3], [4], and [6], where a fully-connected
structure is employed, ERM is advantageous for sparse connectivity, which
thereby concludes lower training epochs to converge.

  * •

The trained ERM is thereafter fine-tuned by means of Xavier initialization
technique[9]. Through this way, the initial input weights in ERM are drawn
from a finite variance and zero mean Gaussian distribution, which diminishes
the prediction variance in conventional ERM by 49%. In terms of achievable
spectral efficiency (SE), the proposed fine-tuned ERM is being shown superior
to the optimization-driven policy [2] and the deep learning policy [6], by up
to 27% and 13%, respectively, when the signal-to-noise-ratio (SNR) is 15dB.

  * •

In the second phase, we additionally improve the performance of the proposed
Xavier-enabled ERM scheme by sequentially incorporating multitude of ERMs
(each one is known as a weak learner) into an ensembled model (known as a
strong learner)[10]. The proposed strong learner not only further advances the
prediction accuracy of the weak learners each, but also alleviates their
prediction variance by up to 56%, while exhibiting the same complexity order
as a weak learner. Numerically, the proposed ensembled model outperforms the
achievable SE of the prior counterparts [2] and [6], by up to 36% and 21%,
respectively, when SNR = 15dB.

The remainder of this paper is organized as follows. Section II and III
describe the system setup and the problem statement, as well as the proposed
solution. Finally, simulations and conclusions are presented in Sections IV
and V, respectively.

##  II System Setup

Figure 1: MmWave channel is transformed from the spatial domain into the
beamspace domain using DFT operation[11] in lens antenna array.

###  II-A Hybrid Analog-Digital Architecture

We consider a downlink mmWave communication, wherein a transmitter employs
$N_{\textrm{t}}$ ($N_{\textrm{t}}^{RF}$) transmit antennas (transmit RF
chains) to serve a receiver with $N_{\textrm{r}}$ ($N_{\textrm{r}}^{RF}$)
receive antennas (receive RF chains) and the number of simultaneous data
streams (i.e., the system multiplexing gain) is $N_{\textrm{s}}$ =
min$(N_{\textrm{t}}^{RF},N_{\textrm{r}}^{RF})$. As a flexible, economical and
energy efficient structure, we assume a hybrid analog-digital beamspace
architecture[1] for the transceivers. According to Fig. 1, the transmitter
exploits a baseband digital matrix
$\mathbf{F}_{\textrm{BB}}\in\mathbb{C}^{N_{\textrm{t}}^{RF}\times
N_{\textrm{s}}}$ for precoding the normalized-power transmit symbols
$\mathbf{s}\in\mathbb{C}^{N_{\textrm{s}}\times 1}$, where
$\mathbb{E}\left[\mathbf{s}\mathbf{s}^{H}\right]=\mathbf{I}_{N_{\textrm{s}}}$.
Subsequently, the analog beam selection network at the transmitter
$\mathbf{S}_{\textrm{t}}\in{\mathbb{R}^{N_{\textrm{t}}\times{N_{\textrm{t}}^{RF}}}}$
maps $N_{\textrm{t}}^{RF}$ RF chains into a subset of $N_{\textrm{t}}$
transmit antennas, using analog switches. An energy-focusing electromagnetic
lens is afterwards embedded with a deployed antenna array on its focal
surface. At the receiver side reversely, the lens antenna array receives the
signals, where the analog beam selection network $\mathbf{S}_{\textrm{r}}$
$\in\mathbb{R}^{N_{\textrm{r}}\times N_{\textrm{r}}^{RF}}$ maps a subset of
the receive antennas into the RF chains. As the transmitter, a digital
baseband combining matrix $\mathbf{W}_{\textrm{BB}}$
$\in\mathbb{C}^{N_{\textrm{r}}^{RF}\times N_{\textrm{s}}}$ is responsible at
the receiver to obtain the symbols. Hence, the received discrete-time complex
baseband signal $y$ can be expressed as
$y=\mathbf{W}_{\textrm{BB}}^{H}\mathbf{S}_{\textrm{r}}^{H}\mathbf{H}_{b}\mathbf{x}+\mathbf{W}_{\textrm{BB}}^{H}\mathbf{S}_{\textrm{r}}^{H}\mathbf{n}$,
where $\mathbf{H}_{b}$ is the mmWave beamspace channel generated by the
discrete Fourier transformation (DFT) in lens antenna array[11] and
$\mathbf{n}$ $\sim N(0,\sigma^{2}\mathbf{I}_{N_{\textrm{r}}})$ denotes the
additive white Gaussian noise (AWGN) with $\sigma^{2}$, denoting the noise
power.

###  II-B Communicating MmWave Channel

Consider a clustered ray-based mmWave channel based on the well-known Saleh-
Valenzuela geometric model [12], with $N_{{cl}}$ cluster of scatterers, where
a typical cluster $l$ has a limited angle-of-departure/arrival (AoD/AoA)
spread, denoted by $\psi_{\textrm{t}}^{l}$ and $\psi_{\textrm{r}}^{l}$
respectively. It is assumed that the $l$th cluster is contributed with
$N_{{ray}}$ propagation rays, where a typical ray $u$, has a physical AoD/AoA
denoted by $\theta_{\textrm{t}}^{{l,u}}\in\psi_{\textrm{t}}^{{l}}$ and
$\theta_{\textrm{r}}^{{l,u}}\in\psi_{\textrm{r}}^{{l}}$, respectively, as well
as a complex-valued gain, denoted by $\alpha^{{l,u}}$. Let us denote, the
spatial AoD/AoA, by
$\phi_{\textrm{t}}^{{l,u}}=(d/\lambda)\sin\theta_{\textrm{t}}^{{l,u}}$ and
$\phi_{\textrm{r}}^{{l,u}}=(d/\lambda)\sin\theta_{\textrm{r}}^{{l,u}},$
respectively, in which $d$ denotes the antenna element spacing and $\lambda$
is the wavelength. Hence, the discrete-time narrowband mmWave channel
$\mathbf{H}\in\mathbb{C}^{N_{\textrm{r}}\times N_{\textrm{t}}}$ is given by
$\mathbf{H}=\gamma\sum_{{l}=1}^{N_{{cl}}}\sum_{{u}=1}^{N_{{ray}}}\alpha^{{l,u}}\mathbf{a}_{\textrm{r}}\left(\phi_{\textrm{r}}^{{l,u}}\right)\mathbf{a}_{\textrm{t}}^{\textrm{H}}\left(\phi_{\textrm{t}}^{{l,u}}\right),$
where $\gamma$ is the normalization factor given by
$\gamma=\sqrt{N_{\textrm{r}}N_{\textrm{t}}/{N_{{cl}}N_{{ray}}}}.$ Also,
$\mathbf{a}_{\textrm{t}}\left(\phi_{\textrm{t}}^{{l,u}}\right)\in\mathbb{C}^{N_{\textrm{t}}\times
1}$ and
$\mathbf{a}_{\textrm{r}}\left(\phi_{\textrm{r}}^{{l,u}}\right)\in\mathbb{C}^{N_{\textrm{r}}\times
1}$, denote the antenna array responses at the transmitter and receiver,
respectively, based on uniform linear array (ULA) and given by
$\mathbf{a}_{\textrm{t}}\left(\phi_{\textrm{t}}^{{l,u}}\right)=\frac{1}{\sqrt{N_{\textrm{t}}}}\left[1,e^{j2\pi\phi_{\textrm{t}}^{{l,u}}},...,e^{j2\pi\left(N_{\textrm{t}}-1\right)\phi_{\textrm{t}}^{{l,u}}}\right]^{H}$
and
$\mathbf{a}_{\textrm{r}}\left(\phi_{\textrm{r}}^{{l,u}}\right)=\frac{1}{\sqrt{N_{\textrm{r}}}}\left[1,e^{j2\pi\phi_{\textrm{r}}^{{l,u}}},...,e^{j2\pi\left(N_{\textrm{r}}-1\right)\phi_{\textrm{r}}^{{l,u}}}\right]^{H}$.
In this regard relying on DFT operations[11] in lens antenna array, the mmWave
channel $\mathbf{H}$ in spatial domain can be efficiently transformed into the
equivalent channel in beamspace domain i.e., $\mathbf{H}_{b}$ (see [11]).

###  II-C Problem Statement

For given the transceiver design parameters
$\mathbf{F}_{\textrm{BB}},\mathbf{W}_{\textrm{BB}}$, ${\mathbf{S}_{{t}}}$ and
${\mathbf{S}_{{r}}}$, the beamspace channel estimation as a signal recovery
optimization problem[3] is defined as

$\displaystyle\min_{\mathbf{H}_{b}}~{}||\mathbf{H}_{b}||_{0}^{2}$ (1)
$\displaystyle
s.t.~{}||y-\sqrt{\psi/N_{\textrm{s}}}\mathbf{W}_{\textrm{BB}}^{\textrm{H}}\mathbf{S}_{\textrm{r}}^{\textrm{H}}\mathbf{H}_{{b}}\mathbf{S}_{\textrm{r}}\mathbf{F}_{\textrm{BB}}\mathbf{s}||\leq{\varepsilon},$

where $||.||_{0}$ indicates the number of non-zero elements and $\varepsilon$
is the error tolerance. From a time-series prediction perspective, we are
seeking for minimizing
$||\mathbf{H}_{b}(t)-\widehat{\mathbf{H}_{b}}(t)||_{2}$, where
$\widehat{\mathbf{H}_{b}}(t)$ is the predicted beamspace channel at time step
$t$. This is achieved via a two-phase mmWave beamspace channel tracking
mechanism, proposed in this paper as follows.

##  III First phase: MmWave Beamspace Channel Tracking via ERM

In the first phase, ERM is briefly introduced and fine-tuned to learn the
historical mmWave beamspace channel features, and forecast them in upcoming
timesteps.

###  III-A Training Sample Set Acquisition

We define a motion feature state vector for a typical transceiver as
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}\mathrel{\ensurestackMath{\stackon[1pt]{=}{\scriptstyle\Delta}}}[\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{{\theta}_{\textrm{r}}},\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\theta_{\textrm{t}}}]$,
with
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\theta}_{\textrm{r}}=\\{\theta_{\textrm{r}}^{1},\theta_{\textrm{r}}^{2},...,\theta_{\textrm{r}}^{|N^{eff}_{\textrm{r}}|}\\}$
and
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\theta}_{\textrm{t}}=\\{\theta_{\textrm{t}}^{1},\theta_{\textrm{t}}^{2},...,\theta_{\textrm{t}}^{|N^{eff}_{\textrm{t}}|}\\}$,
where $|N^{eff}_{\textrm{t}}|=|N_{\textrm{t}}|\times|N_{cl}|\times|N_{ray}|$,
$|N^{eff}_{\textrm{r}}|=|N_{\textrm{r}}|\times|N_{cl}|\times|N_{ray}|$, and
$|N^{eff}|=|N^{eff}_{\textrm{t}}|+|N^{eff}_{\textrm{r}}|$. As in [6], we
generally assume that the feature states such as AoDs and AoAs are time-
varying and follow their historical time steps i.e.,
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(1),\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(2),...,\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t-1)$,
indicating a time-series form to predict
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t)$.
Relying on the predicted values in
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t)$
afterwards, one can derive the non-zero elements in beamspace channel
$\textbf{H}_{b}$ through the pilot transmission based on the classical least
square method[3]. In what follows, it is well elaborated how to predict
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t)$,
relying on its historical time-series
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(1),\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(2),...,\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t-1)$.

###  III-B Time-Series Forecasting

Artificial neural networks (ANN) with temporal dynamic behavior and time-
varying nature, are renowned for their time-series data (i.e., sequences of
data over the time steps) processing capability. To this goal, time-series ANN
are trained by exploiting the historical knowledge and to predict them in next
time steps. The feature states of the transceivers generally exhibit dynamic
characteristics over the time[7] and thus, time-series ANNs are leveraged in
this paper for predicting their upcoming time steps.

As a time-series ANN, ERM is a reservoir computing[8] trained by an extreme
learning policy[13], whereby the sparse neurons in its reservoir layer(s) are
randomly associated with each other (rather than the fully-connected
structure) and remain untuned during the training process. Thanks to the
random and sparse connectivity of neurons in reservoir layer(s), ERM is
superior to the conventional fully-connected counterparts in terms of
prediction time. As depicted in Fig. 2, the input layer of ERM is fed by the
past motion state features
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(1),\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(2),...,\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t-1)$,
and the reservoir layer state in time step $t$ is given by[8]

$\displaystyle
Res(t)=f^{act}[\widehat{W}_{res}^{res}.Res(t-1)+\widehat{W}_{in}^{res}.{\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}}(t-1)],$
(2)

with $\widehat{W}_{res}^{res}$ and $\widehat{W}_{in}^{res}$, indicating the
weight connections of reservoir layer(s) and input layer, respectively. The
hidden weights on reservoir layer $\widehat{W}_{res}^{res}$ as already
mentioned, are randomly assigned and never updated (i.e. they follow a random
projection with non-linear transforms), or can be inherited from their
ancestors (if multitude of reservoir layers coexist) without being changed.
The activation function $f^{act}$ in (2), is a sigmoid or logsig function,
which injects non-linearity to the training process. The readout layer
according to Fig. 2, predicts the motion state features at time step $t$ as
following

$\displaystyle\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t)=\widehat{W}_{res}^{out}.Res(t-1),$
(3)

with $\widehat{W}_{res}^{out}$, denoting the weights of readout layer in ERM
and mostly learned in a single step, which essentially amounts to learning a
linear model[8].

In (2), initializing the input weights $\widehat{W}_{in}^{res}$ by proper
values is crucial for ERM to converge within a reasonable amount of
iterations/epochs. The behaviour of activation functions clarifies this
requirement more. One can observe from Fig. 3 that both functions expose an
approximately linear behaviour for the input values adjacent to zero. If the
input weights in (2) are initialized too small, the variance of the input
signals
$\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(1),\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(2),...,\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\chi}(t-1)$
starts diminishing layer by layer, ending up with trivial values which
basically removes the non-linearity feature of $f^{act}$ and thereby, the
advantage of deep layers is lost. On the opposite side, both of the functions
in Fig. 3, tend to become flat and saturate for larger values of input
signals. By adopting the initial weights too large, the variance of the input
signals tends to escalate in a layer-wise fashion. In this case, the gradient
starts approaching zero, thereby updating the weights will not be properly
accomplished. In the following section, we provide a brief overview of the
Xavier initializer to regulate the input signal initial weights.

###  III-C Xavier Initializer

Towards a proper weight initialization, we answer the following questions on
the basis of Xavier Initialization technique. 1) Which distribution is
adequately proper for $\widehat{W}_{in}^{res}$ to be drawn from. 2) What is
the effective variance of $\widehat{W}_{in}^{res}$ to this end.

Figure 2: The architecture of ERM.

Suppose that the well-known Gaussian distribution with a zero mean and a
finite variance is the candidate. For the sake of exposition, consider (2) for
a typical user in a time step, which is a linear combination of the input
element $\chi_{j}$ and its corresponding weight
$\widehat{w}_{in}^{res}(j)\in{\widehat{W}_{in}^{res}}$ as
$y^{lin}=\widehat{w}_{in}^{res}(1).\chi_{1}+\widehat{w}_{in}^{res}(2).\chi_{2}+...+\widehat{w}_{in}^{res}({|N^{eff}|}).\chi_{|N^{eff}|}$.
The aim here is to keep the variance of $\chi_{j}$ and $y^{lin}$ identical.
The variance of $y^{lin}$ can be given by

$\displaystyle\textrm{var}(y^{lin})\\!=$
$\displaystyle\textrm{var}\big{(}\widehat{w}_{in}^{res}(1).\chi_{1}+$ (4)
$\displaystyle\widehat{w}_{in}^{res}(2).\chi_{2}+...+\widehat{w}_{in}^{res}({|N^{eff}|}).\chi_{|N^{eff}|}\big{)}.$

By definition of variance, it can be concluded that

$\displaystyle\textrm{var}(\widehat{w}_{in}^{res}(j).\chi_{j})\\!=$
$\displaystyle\mathbb{E}(\widehat{w}_{in}^{res}(j))^{2}.\textrm{var}(\chi_{j})+$
(5)
$\displaystyle\mathbb{E}(\chi_{j})^{2}.\textrm{var}(\widehat{w}_{in}^{res}(j))\\!\\!+\\!\\!\textrm{var}(\widehat{w}_{in}^{res}(j)).\textrm{var}(\\!\chi_{j}\\!),$

where $\mathbb{E}(.)$ stands for the expectation or mean value. For the
Gaussian distribution with zero mean, (5) will be given by
$\textrm{var}(\widehat{w}_{in}^{res}(j).\chi_{j})=\textrm{var}(\widehat{w}_{in}^{res}(j)).\textrm{var}(\chi_{j})$.
Therefore, (4) can be rewritten as follows

$\displaystyle\textrm{var}(y^{lin})=$
$\displaystyle\textrm{var}(\widehat{w}_{in}^{res}(1)).\textrm{var}(\chi_{1})+\textrm{var}(\widehat{w}_{in}^{res}(2)).\textrm{var}(\chi_{2})+$
(6)
$\displaystyle...+\textrm{var}(\widehat{w}_{in}^{res}(|N^{eff}|)).\textrm{var}(\chi_{|N^{eff}|}).$

For the identical distribution of weights and input elements, (6) can be
stated as
$\textrm{var}(y^{lin})=|N^{eff}|.\textrm{var}(\widehat{w}_{in}^{res}(j)).\textrm{var}(\chi_{j})$.
So, if we want the variance of $\chi_{j}$ and $y^{lin}$ being identical, we
can conclude that $|N^{eff}|.\textrm{var}(\widehat{w}_{in}^{res}(j))=1$ and
$\textrm{var}(\widehat{w}_{in}^{res}(j))=1/|N^{eff}|$, which verifies that the
weights should be drawn from a Gaussian distribution with zero mean and a
finite variance of $1/|N^{eff}|$.

Figure 3: The behaviour of sigmoid and logsig non-linear functions. Figure 4:
Adaptive boosting mechanism schematic.

###  III-D Second Phase: Enhancing Accuracy via Ensemble Learning

In the second phase, adaptive boosting (AdaBoost) is briefly reviewed and
investigated as an effective strategy of ensemble learning, so as to improve
the prediction capability of ERM in mmWave beamspace channel tracking. The
idea in ensemble learning is to train a strong ensembled learner, which merges
the predictions made by various weak learners (e.g., the Xavier-enabled ERM
modules in this paper) to make a more accurate prediction [10]. In doing so,
we adopt an AdaBoost mechanism according to Fig. 4, whereby the weak learners
are sequentially trained in an adaptive fashion. In fact, it is aimed at
boosting the performance of the current weak learner through focusing on the
improperly trained observations by the prior weak learners. Since the ERMs
predict on continuous domain, the beamspace channel tracking is a regression
problem and the ERMs are regressors, as well. First, $M_{1}$ subsets
$\chi_{m}(m\in M_{1})$ are allocated to $M_{1}$ ERM regressors as the weak
learners, which are sequentially trained for predicting the next time step,
i.e., $\chi(t+1)$. The strong ensembled learner is defined based on a weighted
sum of $M_{1}$ weak learners as
$\Phi_{M_{1}}^{ens}=\sum_{m=1}^{M_{1}}c_{m}\chi_{m}$, with $c_{m}$ denoting
the weight of the $m$th ERM as the performances of this weak model, i.e., the
better a weak learner performs, the more it contributes to the strong
ensembled model. The strong ensembled learner thus, is generally less biased
than the weak learners. The main issue is to achieve the optimal order for the
ERMs to be deployed along the ensembling chain. Note that the optimal
$\Phi_{M_{1}}^{ens}$ is complicated to obtain when the ensembling chain is
long. Instead, rather than globally seeking among the weak learners all, the
best possible pairs of ($c_{m},\chi_{m}$) are locally built and added to the
strong ensembled model one by one, in an iterative sub-optimal manner.
Recurrently, the strong ensembled model can be indicated by
$\Phi_{m}^{ens}=\Phi_{m-1}^{ens}+c_{m}\chi_{m}$, and the best
($c_{m},\chi_{m}$) pair is acquired as
$(c_{m},\chi_{m})=\operatorname*{arg\,min}_{c,\chi}E(\Phi_{m-1}^{ens}+c\chi)$,
where $E(.)$ indicates the fitting error of the strong ensembled model. The
prediction made by the strong ensembled model by contributing the weak
learners and their impact at time step $t$ finally, is given by
$\hat{\chi}(t)=\frac{1}{M_{1}}\sum_{m=1}^{M_{1}}c_{m}\chi_{m}(t)$.

###  III-E Complexity Analysis

The computational complexity for training ERM mainly incorporates the training
time for its readout weights ${W}_{res}^{out}$, evaluated as
$\mathcal{O}(\textrm{ERM})=(T_{\text{max}}-T_{0}+1)^{3}$, with
$T_{\text{max}}$ and $T_{0}$ indicating the maximum training time and the
initial washout time, respectively[14]. In an AdaBoost ensembled model
nonetheless, since multiple ERM modules are sequentially incorporated, the
complexity can be expressed as
$\mathcal{O}(\textrm{ENS})=\mathcal{O}(\textrm{ERM}_{1})+\mathcal{O}(\textrm{ERM}_{2})+...+\mathcal{O}(\textrm{ERM}_{M_{1}})={max}\\{\mathcal{O}(\textrm{ERM}_{j})\\},~{}j\in{\\{1,2,...,M_{1}\\}}$.

TABLE I:
ERM configurations ERM Parameter |  Value
---|---
TrainingSize |  75%
ValidationSize |  25%
Number of batches |  250
No. input delay $n_{\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\theta}_{\textrm{t}}}$ |  6
No. output delay $n_{\textpdfrender{TextRenderingMode=FillStroke,LineWidth=.35pt,}{\theta}_{\textrm{r}}}$ |  6
No. observations |  20
Observation interval |  1(s)

##  IV Simulation Results

We consider a clustered mmWave channel model incorporating $N_{{cl}}=8$
clusters, where each cluster contributes $N_{{ray}}=6$ propagation rays. The
signal wavelength is supposed to be $\lambda$ = 1.36, while AoAs/AoDs are
uniformly distributed within $[-\frac{1}{2},\frac{1}{2}]$ and the complex
path-gain follows $\mathcal{CN}$(0, 1). Simulations are performed in a lens-
aided MIMO system equipped with $N_{\textrm{t}}=64,N_{\textrm{r}}=16$ and
$N_{\textrm{s}}=N_{\textrm{t}}^{\textrm{RF}}=N_{\textrm{r}}^{\textrm{RF}}=4$.
As observed from Table I, 10000 samples are investigated in 250 batches,
contributing 75% for the training and the rest for the validation and test,
where we observe 20 time slots of tracking with 1sec interval and the number
of delay steps for the input and output layers are $n_{\theta_{\textrm{t}}}$ =
$n_{\theta_{\textrm{r}}}$ = 6.

###  IV-A Performance Measures

To evaluate the performance of our proposed method, we analyze the error
metrics, RMSE and NMSE over $Q$ samples and respectively given by
$\textrm{RMSE}=\sqrt{\sum_{i=1}^{{Q}}||\mathbf{H}^{i}_{b}(t)-\widehat{\mathbf{H}^{i}_{b}}(t)||^{2}_{2}}$,
and
$\textrm{NMSE}=\mathbb{E}\Bigg{\\{}\sum_{i=1}^{{Q}}||\mathbf{H}^{i}_{b}(t)-\widehat{\mathbf{H}^{i}_{b}}(t)||^{2}_{2}\times\big{(}\sum_{i=1}^{{Q}}||\widehat{\mathbf{H}^{i}_{b}}(t)||^{2}_{2}\big{)}^{-1}\Bigg{\\}}$,
where ${\mathbf{H}^{i}_{b}}(t)$ and $\widehat{\mathbf{H}^{i}_{b}}(t)$ are the
ground truth and the predicted values of the mmWave beamspace channel for the
$i$th sample on time step $t$, respectively. We also define the standard
deviation and the mean absolute deviation metrics for evaluating the beamspace
channel tracking variance, respectively given by
$\tau_{1}=\sqrt{\sum_{i=1}^{{M}}||\widehat{\mathbf{H}^{i}_{b}}(t)-\mu||^{2}_{2}\times
Q^{-1}}$, and
$\tau_{2}=\sum_{i=1}^{{M}}||\widehat{\mathbf{H}^{i}_{b}}(t)-\mu||_{2}\times
Q^{-1}$, with $M$ and $\mu$, denoting the number of predictions and the
prediction average, respectively. Finally, the achievable SE is expressed as
$\\!SE\\!=\textrm{log}_{2}\big{|}\textbf{I}_{N_{\textrm{s}}}\\!+\\!\frac{\rho}{\sigma^{2}N_{\textrm{s}}}R_{n}^{-1}(\textbf{W}_{\textrm{BB}})^{H}(\textbf{S}_{\textrm{r}})^{H}\textbf{H}_{{b}}\textbf{S}_{\textrm{t}}\textbf{F}_{\textrm{BB}}(\textbf{F}_{\textrm{BB}})^{H}(\textbf{S}_{\textrm{t}})^{H}(\textbf{H}_{{b}})^{H}\textbf{S}_{\textrm{r}}\textbf{W}_{\textrm{BB}}\big{|},$
where
$R_{n}=\\!(\textbf{W}_{\textrm{BB}})^{H}(\textbf{S}_{\textrm{r}})^{H}\textbf{S}_{\textrm{r}}\textbf{W}_{\textrm{BB}}$
is the noise covariance matrix after combining.

Figure 5: NMSE versus the SNR. Our proposed ERM+Xavier scheme outperforms LSTM
and other baseline schemes especially in low SNR values. Figure 6: RMSE versus
the number of convergence epochs.

###  IV-B Performance Evaluation

We use Gaussian mixture learning-aided AMP (GM-LAMP)[3], learning-aided AMP
(LAMP)[4], LSTM[6] and orthogonal matching pursuit (OMP)[2], as the baseline
schemes to evaluate the efficiency of the proposed ERM+Xavier and ensembeld
schemes. We have trained the LAMP and GM-LAMP schemes under the simulation
parameters given in [3], while the configurations for LSTM and ERM are
identical.

Figure 7: Analyzing the prediction variance by mean absolute deviation and
standard deviation. Figure 8: Number of samples vs. loss range.

In Fig. 5 for varying SNR in 0dB$\sim$30dB, we analyze the NMSE of the
baseline schemes. By increasing SNR, all the schemes evidently perform more
accurately, leading to a degradation trend for the NMSE. As observed, our
proposed ERM+Xavier scheme as a fine-tuned deep learning technique outperforms
others, due to the higher time-series prediction accuracy. Furthermore, the
ensembled model exhibits lower NMSE compared to all the schemes, thanks to a
lower prediction variance, as well as an enhanced prediction capability. Fig.
6 indicates ERM convergence behaviour during training, validation and test
procedure with negligible RMSE at the convergence point.

Figure 9: Average spectral efficiency against SNR.

The convergence is achieved in limited epochs thanks to the sparse
connectivity in ERM architecture (specifically in reservoir layer(s)), as well
as the utilization of Xavier initialization. One can compare the required
epochs to converge in Fig. 6, to a typical ANN such as spiking neural network
(SNN) with far more epochs taking to converge (see Fig. 7 in [15]). Fig. 7
verifies that fine-tuning ERM with Xavier initializer technique makes the
prediction variance lower, compared to the conventional ERM with random
initial input weights, by up to 26% and 49%, for $\tau_{1}$ and $\tau_{1}$,
respectively. In addition, compared to a single conventional ERM, the strong
ensembled model leads to 31% and 56% reduction in prediction variance for
$\tau_{1}$ and $\tau_{1}$, respectively.

Fig. 8 shows the loss range upon training, validation and test phases for
varying number of samples in a batch. Obviously, less errors are observed,
when more number of samples are incorporated within a batch. What's more, the
errors on validation and test phases are mostly dispersed in adjacent to the
zero error line, which is negligible and verifies that ERM enjoys an accurate
prediction. Fig. 9 compares the achievable SE for the baseline schemes, where
the ideal CSI is the upper bound to this end. According to Fig. 5, the
proposed ERM+Xavier scheme enjoys a more accurate prediction among the
counterparts, which accordingly results in 27% and 13% improvement in
achievable SE of OMP[2] and the LSTM[6] schemes at SNR = 15dB, respectively.
By improving ERM+Xavier scheme one step further, the ensembled model is
superior among all the schemes as well, with 36% and 21% more SE upon OMP[2]
and LSTM[6] schemes at SNR = 15dB, respectively. Whats more, as the SNR
increases, all the schemes get closer to the perfect CSI, due to the better
pilot transmission.

##  V Conclusions

By tracking the historical features, we proposed a two-phase mmWave beamspace
channel estimation scheme from a time-series prediction perspective (i.e., a
mmWave beamspace channel tracking scheme). We fine-tuned the conventional ERM
by means of Xavier initializer technique in the first phase, for lower
prediction variance. The fine-tuned ERM shown to be capable of accurately
learning from the past features and predicting the upcoming variations of the
mmWave beamspace channel. In the second phase, the AdaBoost as an ensemble
learning technique, applied on the fine-tuned Xavier-initialized ERM for even
further reduced prediction variance, as well as an improved predicting
precision. Simulations revealed a two-step improvement upon the counterparts
in literature.

## Acknowledgement

This work is supported by the Academy of Finland: (a) ee-IoT n.319009, (b)
EnergyNet n.321265/n.328869, and (c) FIREMAN n.326270/CHISTERA-17-BDSI-003;
and by JAES Foundation via STREAM project.

## References

  * [1] J. Brady, N. Behdad, and A. M. Sayeed,“Beamspace MIMO for millimeter-wave communications: system architecture, modeling, analysis and measurements,” IEEE Trans. Antennas Propag., vol. 61, no. 7, pp. 3814-3827, Jul. 2013.
  * [2] A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter-wave cellular systems,” IEEE J. Sel. Top. Signal Process., vol. 8, no. 5, pp. 831-846, Oct. 2014.
  * [3] X. Wei, C. Hu, L. Dai, “Knowledge-Aided Deep Learning for Beamspace Channel Estimation in Millimeter-Wave Massive MIMO Systems”, arXiv preprint, arXiv:1910.12455, January 2020.
  * [4] H. He, C. K. Wen, S. Jin, and G. Y. Li, “Deep learning-based channel estimation for beamspace mmWave massive MIMO systems,” IEEE Commun. Lett., vol. 7, no. 5, pp. 852–855, Oct. 2018.
  * [5] M. Borgerding, P. Schniter, and S. Rangan, “AMP-inspired deep networks for sparse linear inverse problems,” IEEE Trans. Signal Process., vol. 65, no. 16, pp. 4293-4308, Aug. 2017.
  * [6] T. Peng, R. Zhang, X. Cheng and L. Yang, “LSTM-Based Channel Prediction for Secure Massive MIMO Communications Under Imperfect CSI,” IEEE Int. Conf. Commun. (ICC), Dublin, Ireland, 2020, pp. 1-6.
  * [7] A. Klautau, P. Batista, N. Gonzalez-Prelcic, Y. Wang and R. W. Heath, “5G MIMO data for machine learning: application to beam selection using deep learning,” Proc. ITA, pp. 1-9, 2018.
  * [8] B. Schrauwen, D. Verstraeten, and J. V. Campenhout, “An overview of reservoir computing: theory, applications, and implementations,” Pro. Eurp. Symp. Artf. Neu. Netw. (ESANN), pp. 471-482, Jan. 2007.
  * [9] X. Glorot, and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” Aistats, vol. 9, pp. 249–256, 2010.
  * [10] J. Ehrlinger and H. Ishwaran, “Characterizing L2Boosting,” Aistats, arXiv:1207.5367, 2012.
  * [11] W. Shen, X. Bu, X. Gao, C. Xing and L. Hanzo, “Beamspace Precoding and Beam Selection for Wideband Millimeter-Wave MIMO Relying on Lens Antenna Arrays,” IEEE Trans. Signal Process., vol. 67, no. 24, pp. 6301-6313, Dec. 2019.
  * [12] A. A. M. Saleh and R. Valenzuela, “A statistical model for indoor multipath propagation,” IEEE J. S. Areas Commun., vol. 5, no. 2, pp. 128-137, Feb. 1987.
  * [13] G. Huang, Q.Zhu, and C.Siew, “Extreme learning machine: theory and applications,” Neurocomputing, pp. 489–501, Dec. 2006.
  * [14] S. Mosleh, L. Liu, C. Sahin, Y. R. Zheng, and Y. Yi, “Brain-inspired wireless communications: Where reservoir computing meets MIMO-OFDM,” IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 10, pp. 4694-4708, Oct. 2018.
  * [15] K. Hamedani, L. Liu, S. Hu, J. Ashdown, J. Wu and Y. Yi, “Detecting Dynamic Attacks in Smart Grids Using Reservoir Computing: A Spiking Delayed Feedback Reservoir Based Approach,” IEEE Trans. Emrg. Top. Comput. Intell., vol. 4, no. 3, pp. 253-264, June 2020.