2022

[1]Toma Oregel-Chaumont

[1]Electromagnetic Compatibility Laboratory, EPFL, , Lausanne, 1015, VD,
Switzerland

2]Department of Engineering Sciences, Uppsala University, , Uppsala, 751, ,
Sweden

3]IICT, HEIG-VD, , Yverdon-les-Bains, 1401, VD, Switzerland

# Direct Observations of X-Rays Produced by Upward Positive Lightning

<EMAIL_ADDRESS>Antonio Šunjerga<EMAIL_ADDRESS>Pasan
Hettiarachchi<EMAIL_ADDRESS>Vernon Cooray
<EMAIL_ADDRESS>Marcos Rubinstein<EMAIL_ADDRESS>Farhad Rachidi<EMAIL_ADDRESS>*  [  [

###### Abstract

X-rays have been observed in natural downward cloud-to-ground lightning for
over twenty years and in rocket-triggered lightning for slightly less. In both
cases, this energetic radiation has been detected during the stepped and dart
leader phases of downward negative flashes. More recently, X-rays have also
been reported during the dart leader phase of upward negative flashes. In this
study, we present the observations of four upward positive lightning flashes
from the Säntis Tower (2.5 km ASL) in Switzerland. These consist of the
simultaneous records of electric current passing through the tower, and
electric field strength and X-ray flux 20 meters from the tower base. One of
the flashes was captured by a high-speed camera operating at 24,000 frames per
second, stills from which are also presented. We detected X-rays during the
initial phase of upward negative leader propagation, which can be associated
with the leader-stepping process from electric field and current waveforms. To
the best of our knowledge, this is the first time that such measurements are
reported in the literature. The obtained time-synchronised data confirm that
the X-ray emissions detected are associated with the initial steps of the
upward negative leader. The frequency and energy of X-ray pulses appear to
decrease as a function of time, with pulses disappearing altogether within the
first millisecond of the leader initiation. The X-ray pulse energy appears to
increase with the maximum current-derivative and the electric field change of
its associated leader step. These observations contribute to improving the
currently lackluster understanding of upward lightning, which is a primary
source of damage to tall structures such as telecommunications towers and wind
turbines, as well as airplanes during take-off and landing.

###### keywords:

lightning, X-rays, leaders, observations

##  1 Introduction

The first unambiguous observation of X-ray generation from lightning flashes
was made by Moore et al. 2001 moore_energetic_2001 , who recorded X-ray bursts
with energies in excess of 1 MeV during the stepped-leader phase of three
natural downward negative lightning flashes. This confirmed the relativistic
runaway electron avalanche (RREA) model proposed by Gurevich et al. 1992
(gurevich_runaway_1992, ), which predicted the production of X-rays via the
bremsstrahlung interaction of electrons with air. Since then, X-ray emissions
have been measured in both natural and artificially-triggered cloud-to-ground
(CG) lightning via a series of experiments conducted at Camp Blanding, Florida
(dwyer_energetic_2003, ; dwyer_x-ray_2005, ; saleh_properties_2009, ;
mallick_study_2012, ). Bursts of energetic radiation were detected during both
the stepped-leader phase and dart leader–return stroke transition, with
energies ranging from 100s of keV to 10s of MeV.

Measurements of X-ray emissions from natural upward lightning, however, were
scanty until recently. Yoshida et al. 2008 yoshida_high_2008  observed
increased counts associated with seven lightning flashes on their plastic and
NaI scintillators designed for detecting high-energy electron and photon
bursts, though the 1-ms sampling interval of their detectors did not permit
precise identification of the emitting phase. Out of the seven, they reported
results on two, an upward negative flash and an upward positive flash.
Montanyà et al. 2014 montanya_registration_2014  made measurements of X-ray
emissions from several upward lightning flashes from the mountaintop Eagle
Nest tower located at 2537 m above sea-level (ASL) in the Pyrenées. They
observed a 17 X-ray pulse burst (with an 806 keV maximum) during the stepped
leader phase of a natural downward negative flash, but did not detect X-ray
emissions during the 13 upward-initiated flashes reported. Hettiarachchi et
al. 2018 hettiarachchi_x-ray_2018  were the first to directly measure X-ray
emissions from upward-initiated lightning flashes: though either rare or very
weak, they detected X-rays with energies up to 700 keV occurring both in
bursts and as single events during the dart/dart-stepped leader phase of 3
natural upward negative flashes at Gaisberg Tower in Austria.

Herein we report, to the best of our knowledge, the first association of
X-rays with the stepping of the upward negative leader in upward positive
lightning flashes, as measured by the comprehensive Säntis lightning
measurement system. The data consist of simultaneous records of lightning
current and its derivative, near electric field (20 m), and high-speed camera
(HSC) images. A summary of the data types analysed in this study and the
aforementioned studies is presented in Table 1.

Table 1: Lightning X-ray measurement studies – A comparison Study | E-Field | Current |  High-speed camera |  Interfer -ometer | Scintillators
---|---|---|---|---|---
Moore et al. 2001 | YES | YES | NO | NO | 1
Dwyer et al. 2003-5 | YES | YES | YES | NO | 12
Yoshida et al. 2008 | YES | YES | NO | YES | 2
Saleh et al. 2009 | YES | YES | NO | NO | 45
Mallick et al. 2012 | YES | NO | NO | NO | 1
Montanyà et al. 2014 | YES | NO | YES | NO | 1
Hettiarachchi et al. 2018 | YES | YES | NO | NO | 2
This study | YES | YES | YES | NO111Interferometric (IFM) data is available but not analysed in this study. | 2
|  |  |  |  |

##  2 Methods

Figure 1: Photo of the Säntis peak, with arrows indicating the Radome, which
houses the electric field probe and scintillators, and the Tower, where the
current and current-derivative sensors are located.

The Mt. Säntis Lightning Research Facility, shown in Figure 1, is situated at
2502 m ASL in the Appenzell Alps of northeastern Switzerland, and experiences
$>$100 direct lightning strikes per year to its 124 meter-tall tower, which is
equipped with a Rogowski coil – $\dot{B}$ sensor pair at two different heights
(24 and 82 meters above ground level), for measurement of the current and
current derivative, respectively.222All four sensors operate with a sampling
rate of 50 MHz. The nearby Radome houses a near E-field sensor and two X-ray
detectors (described below), which have a common sampling rate of 20 MHz. Our
Mélopée fast E-field probe has a frequency range of 1 kHz to 150 MHz and is
described in more detail in Šunjerga et al. 2021 sunjerga_santis_2021 . Five
kilometers away, atop Mt. Krönberg (1663 ASL), is a high-speed camera (HSC)
operating at 24,000 fps, with an exposure time of 41 $\mu$s. Electric field
measurements are also taken 15 km away by a flat-plate antenna with line-of-
site in Herisau, Switzerland, though this data is not presented herein.
Additionally, during the Summer of 2021, when the flashes discussed below
occurred, a University of New Mexico interferometer (IFM) was installed in
Schwägalp, at the base of Mt. Säntis. These interferometric results will be
the subject of a separate paper.

One of our two NaI X-ray detectors belong to Uppsala University and the other
to the University of California – Santa Cruz (UCSC); the former records
waveforms and is triggered by the tower current, whereas the latter records
the peak energies of events and is working continuously. The Uppsala
scintillator, whose data are presented in this report, has a measuring range
of $\sim$10 keV to 2 MeV, a temporal resolution of $\sim$1/4 $\mu$s, and is of
the same design as that used by Hettiarachchi et al.; refer to their 2018
paper hettiarachchi_x-ray_2018  for a detailed description. This X-ray
detector is connected to the same digitiser as our radome E-field probe and
therefore by default synchronised; synchronisation of these two with the tower
current and current derivative signals (which are themselves synchronised in
the same manner) is done by aligning the time of the first E-field “step” with
the time of the $\dot{B}$ extremum associated with the first current pulse, as
these tend to be the sharpest. The HSC and “far” E-field data are synchronised
with the rest by GPS time-stamp if the antennae are functional at the time of
the flash. If not, manual synchronisation can be carried out via waveform
matching. More detailed information on the Säntis measurement system can be
found in rachidi_santis_2022 .

All computational data analysis and presentation were carried out using the
Python programming language, with the NumPy, SciPy and Matplotlib libraries in
particular.

##  3 Results

Table 2: Positive Lightning Flashes Analysed – Data Summary Flash |  Date
UTC |  Prior Activity |  Current (& derivative) |  High-speed camera |  Interfer -ometer |  E-field333Only near E-field waveforms are presented and analysed herein. (20-m & 15-km)
---|---|---|---|---|---|---
UP0 |  2021-06-28
23:26:53 | YES | YES | NO | NO | YES
UP1 |  2021-07-24
16:06:07 | NO | YES | NO | YES | YES
UP2 |  2021-07-24
16:24:03 | NO | YES | YES | NO | YES
UP3 |  2021-07-30
18:00:10 | YES | YES | NO | YES | YES
|  |  |  |  |  |

We analysed 4 upward positive and 8 upward negative flashes with associated
X-ray emissions that occurred during the Summer 2021 thunderstorm season.444A
comprehensive analysis of all 12 flashes will be the subject of a separate
paper. The data available for the 4 upward positive flashes (UPFs) presented
here are summarised in Table 2. In addition to tower current and electric
field measurements, two (UP1 and UP3) were recorded by the interferometer
(subject to a separate analysis), and one (UP2) was captured by the high-speed
camera. UP0 and UP3 also saw preceding lightning activity in the vicinity
(i.e. intra-cloud flashes); studies have shown that this may impact the
formation of leaders from the strike object sunjerga_initiation_2021 . It
should be noted that flashes UP1, UP2, and UP3 occurred during the Laser
Lightning Rod project presented in Houard et al. 2023 houard_laser-guided_2023
(therein called L1, L2, and L3, respectively), while the laser was on, whereas
flash UP0 did not.

Here, we define the initial continuous current (ICC) at the start of the
leader, i.e., the first significant deviation from zero of the electric field,
current and current-derivative. For the latter two, we have chosen the
convention of a negative current corresponding to a positive charge transfer
from cloud to ground. Each positive flash had between two and seven X-ray
events associated with this “stepping” of the upward negative leader, also
indicated by pulses in the current waveform.

Table 3: Positive flash pulses with associated X-rays Flash |  $t_{SL}$ [$\mu$s] |  $I_{p}$ [kA] |  $t_{mr}$ [$\mu$s] | |$\frac{dI}{dt}$| [$\frac{\mathrm{kA}}{\mu\mathrm{s}}$] |  $\Delta E$ [$\frac{\mathrm{V}}{\mathrm{m}}$] |  $t_{Er}$ [$\mu$s] | XRE [keV]
---|---|---|---|---|---|---|---
UP0 | 150.9 | 0.81$\pm$0.05 | 0.13${}^{+0.02}_{-0.01}$ | 6.2$\pm$0.4 | 545$\pm$55 | 0.15 | 63.4$\pm$0.9
UP1 | 0.0 | 1.66$\pm$0.05 | 0.12${}^{+0.01}_{-0.01}$ | 14.2$\pm$0.3 | 1025$\pm$40 | 0.20 | 259.3$\pm$0.7
89.5 | 2.01 | 0.24${}^{+0.02}_{-0.02}$ | 8.2 | 885 | 0.30 | 85.1
334.6 | 1.91 | 0.88${}^{+0.18}_{-0.14}$ | 2.2 | 635 | 9.50 | 87.0
345.7 | 2.89 | 0.79${}^{+0.09}_{-0.08}$ | 3.7 | 635 | 9.50 | 123.8
465.6 | 2.20 | 1.53${}^{+0.50}_{-0.32}$ | 1.4 | 330 | 10.55 | 31.6
554.1 | 2.80 | 1.36${}^{+0.29}_{-0.21}$ | 2.1 | 590 | 17.10 | 79.8
774.0 | 2.66 | 1.78${}^{+0.55}_{-0.35}$ | 1.5 | 350 | 19.50 | 31.1
UP2 | 46.3 | 1.14$\pm$0.05 | 0.13${}^{+0.01}_{-0.01}$ | 8.7$\pm$0.3 | 1450$\pm$40 | 0.20 | 54.6$\pm$0.7
267.5 | 0.41 | 0.31${}^{+0.14}_{-0.09}$ | 1.3 | 280 | 11.05 | 44.1
UP3 | 118.0 | 2.44$\pm$0.05 | 0.47${}^{+0.04}_{-0.04}$ | 5.2$\pm$0.3 | 720$\pm$40 | 0.25 | 26.5$\pm$0.8
355.0 | 2.71 | 1.28${}^{+0.26}_{-0.19}$ | 2.1 | 475 | 2.35 | 50.4
483.0 | 1.82 | 1.08${}^{+0.30}_{-0.20}$ | 1.7 | 175 | 6.70 | 53.0
785.0 | 5.20 | 2.52${}^{+0.51}_{-0.37}$ | 2.1 | 565 | 21.10 | 42.8
$\mu_{a}\pm\sigma_{a}$ | – | 2.19$\pm$1.11 | 0.90$\pm$0.71 | 4.3$\pm$3.7 | 620$\pm$320 | 7.75$\pm$7.30 | 73.8$\pm$57.6
$\mu_{g}\sigma_{g}^{\pm 1}$ | – | $1.88^{+1.55}_{-0.85}$ | $0.59^{+1.05}_{-0.38}$ | $3.2^{+3.6}_{-1.7}$ | $540^{+375}_{-220}$ | $2.60^{+14.90}_{-2.20}$ | $60.4^{+48.0}_{-26.7}$
|  |  |  |  |  |  |

Table 3 presents the measured data for each pulse with correlated X-rays. The
time $t_{SL}$ (“stepped leader”) is measured from the onset of the ICC.
$I_{p}$ represents the absolute value peak current of a given pulse, and
|$dI/dt$|${}_{\mathrm{max}}\,$ its maximum current derivative (slope).
Together they make the minimum current rise-time, defined by Giri et al. 2009
giri_relationship_2009  as:

$t_{mr}=\frac{I_{p}}{\|\frac{dI}{dt}\|_{\mathrm{max}}}$ (1)

which has a temporal accuracy of 20 ns. The change in the electric field is
given by $\Delta E$ and its 80% rise-time by $t_{Er}$ (with a temporal
resolution of 50 ns). Finally, XRE is the associated X-ray energy. The first
row of each flash provides the error associated with sensor noise at that
time, with the exception of the calculated $t_{mr}$, whose errors vary with
measurement. The last two rows in the table provide the arithmetic and
geometric means and standard deviations of each data set above. One can
already see that all parameters except $I_{p}$ exhibit at least some degree of
temporal variation, as will be confirmed later in Section 4.

(a) The entire duration of the flash. A 100 kHz low-pass filter has been
applied to the current and $dI/dt$ waveforms to remove intermittent noise.

(b) Zoom on the X-ray events during the upward stepping negative leader phase.
The brown vertical dotted lines indicate the event times. See Table 3 for
pulse data.

Figure 2: Waveforms of UP1, a Type 2 upward positive flash that occurred on
July 24, 2021 at 16:06:07 UTC. “PEMb” and “Bdtt” specify the bottom Rogowski
coil and top $\dot{B}$ sensor, respectively. $E_{z}$ is the measured vertical
component of the electric field at 20 m. The time is from the beginning of the
recording ($\sim$1 second before the current peak).

The current, $dI/dt$, near E-field, and X-ray waveforms of UP1 are shown in
Figure 2. It can be seen from Figure 2(a) that this was a Type 2 upward
positive flash, as defined by Romero et al. 2013 romero_positive_2013 ; i.e.,
it lacks a return stroke–like main pulse following the stepped-leader phase.
Figure 2(b) presents an expanded view of the initiation of the upward leader
and its stepping. It clearly shows how steps in the electric field are
associated with ICC pulses; 1/3 of which were associated with X-ray emissions.
The 7 X-ray pulses have a median temporal separation on the order of 100
$\mu$s, and median energy on the order of 80 keV. Note, however, the decrease
in pulse peak energy as time goes on.

Figure 3: Waveforms and integrated HSC frames of UP2, a Type 1 upward positive
flash that occurred on July 24, 2021 at 16:24:03 UTC. A 100 kHz low-pass
filter has been applied to the current and $dI/dt$ waveforms to remove
intermittent noise. See Figure 4 for a zoom-in view on the X-ray events.

The whole flash waveforms and integrated HSC frames of UP2 are shown in Figure
3. This was clearly a Type 1 upward positive flash romero_positive_2013 , with
a very obvious return stroke–like pulse after the upward-stepping leader. In
the right-hand panel of the figure, one can make out the Säntis tower, from
which the flash initiated, at the base of the rather tortuous plasma
channel.555The faint black streak running diagonally across the integrated
stills are raindrops streaming down the camera’s protective window pane. The
plots at the bottom of Figure 4 provide a zoom on the beginning of the ICC,
when the two X-ray pulses occurred and the top pictures are HSC stills
containing these pulses, which occurred 221 $\mu$s apart with an average
energy of 49 keV. Once again, these are clearly associated with the leader-
stepping process, though unlike the Type 2 UPFs, only 1/10 of the leader steps
had accompanying X-rays detected.

(a) HSC Frame 655

(b) HSC Frame 661

Figure 4: X-ray events during the upward stepping negative leader phase of
UP2. HSC frames containing the two X-ray pulses observed are shown above, and
their approximate temporal width ($\sim$42 $\mu$s) is highlighted by the red-
shaded regions in the waveforms below. E-field steps without associated X-rays
are also indicated by the violet vertical dotted lines. See Table 3 for pulse
data and Figure 3 for a zoomed-out view of the waveforms.

UP3 was a Type 2 flash like UP1; its 4 X-ray pulses (about 1/6 of all ICC
pulses) had a mean temporal separation of $\sim$220 $\mu$s and an average
energy of 43 keV. UP0 was likely a Type 1 flash like UP2; its singular X-ray
pulse ($<$5% of all ICC pulses) had an energy of 63 keV. These flashes’
waveforms are not depicted here for the sake of conciseness, though all their
pulse data have been included in the following analysis. See Appendix Figures
7 & 8 for plots.

##  4 Discussion

Figure 5 shows how both X-ray count and energy decrease as a functions of time
from the onset of the stepped leader, $t_{SL}$. Note how, in comparison with
the X-ray pulse to non-X-ray pulse ratios presented in Section 3, ICC pulses
with measured accompanying X-rays compose a steadily decreasing percentage of
all measured pulses, starting at $\sim$29% during the first 200 $\mu$s, and
dropping to 0% after 800 $\mu$s. Although one could argue that this count
decrease observed in Figure 5(a) is simply due to the decrease in photon flux
at the sensor location as the leader tip (where the X-rays are presumed to be
emitted rakov_lightning_2003 ; moss_monte_2006 ) moves away, the same argument
cannot be made for observed energy decrease in Figure 5(b), as the waveforms
are indicative of single events, rather than photon-burst energy pile-ups
(compare with Figure 4 of Saleh et al. 2009 saleh_properties_2009 ). The best-
fit lines in this plot imply the existence of a finite time ($>$1 ms) after
which X-rays would no longer be generated and/or detected.

(a) Histogram of ICC pulse counts, both with and without accompanying X-rays,
as a function of time.

(b) Scatter plot of X-ray energy as a function of time. The affine (green) and
exponential (blue) fit lines have r2 values of 0.19 and 0.24, respectively.

Figure 5: Plots depicting the temporal dependence of the X-ray counts and
pulse energy for flashes UP0, UP1, UP2 and UP3. Time $t_{SL}=0$ is set to the
start of the stepped leader / ICC. Data taken from Table 3.

Figure 6(a) shows the scatter plot of X-ray energy versus maximum current
derivative, |$\frac{dI}{dt}$|max. It is clear from the color map that the
latter also decreases as a function of time $t_{SL}$ (alternatively, the pulse
rise-time $t_{mr}$ increases). The best fit lines show that the X-ray energy
increases with |$\frac{dI}{dt}$|max, either linearly or exponentially.

(a)

(b)

Figure 6: Color-mapped scatter plots depicting the parametric dependence of
the X-ray energy for flashes UP1, UP2 and UP3, similar to Figure 15 of Mallick
et al. 2012 mallick_study_2012 . Time $t_{SL}=0$ is set to the start of the
stepped leader / ICC. Data taken from Table 3.

Figure 6(b) shows the scatter plot of X-ray energy versus electric field
change, $\Delta E$. It is clear from the color map that $\Delta E$ decreases
as a function of time, as does the X-ray energy, albeit to a lesser extent
(see Figure 5(b)). The former can be at least partly explained by the fact
that our sensor measures only the vertical component of the electric field,
and the leader tip is moving away from the tower. The best fit lines show how
the X-ray energy increases with $\Delta E$, possibly logarithmically or as a
power-law. Note that, should a logarithmic relation be confirmed by further
investigation, the x-intercept at $\Delta E\approx 140$ V/m implies the
existence of a minimum E-field change needed for X-rays to be produced.

As such, it has been suggested (e.g., dwyer_implications_2004 ;
cooray_electric_2009 ) that the so-called “cold runaway electron mechanism”,
as opposed to the RREA model, is active in X-ray emissions associated with
lightning leaders. For the cold runaway mechanism to be active, it is
necessary for the background electric field at atmospheric pressure to exceed
about 20 MV/m moss_monte_2006 ; cooray_electric_2009 . If the electric field
increases slowly in atmospheric air, as its value reaches around 3 MV/m the
normal electrical breakdown process takes over and the resulting increase in
conductivity of the discharge channel limits further increase of the electric
field strength. The electric field will therefore be clamped to a value equal
to or below this breakdown threshold.666If the pressure is below atmospheric,
these mechanisms remain the same except that the threshold fields are scaled
down linearly with pressure. Since a certain amount of time is needed for the
completion of standard breakdown, in order to achieve the cold runaway
mechanism the electric field has to increase very rapidly in a given region of
space so that there isn’t sufficient time for the standard breakdown mechanism
to take over and clamp the electric field at $\sim$3 MV/m. Thus, only very
fast discharge processes (sub-microsecond scale) can generate the strong
electric fields needed to push electrons into the cold runaway regime quickly
enough cooray_electric_2009 . This is in agreement with the observation that
X-ray emissions occur during discharge processes with rapidly changing
currents, such as those seen in this study.

##  5 Conclusion

Herein we reported, to the best of our knowledge, the first measurements of
X-rays produced by positive lightning flashes, specifically during the
stepping of the upward negative leader. We presented the waveforms of the
current, current derivative, electric field, and X-ray energy for the four
flashes in question (two Type 2 upward positive flashes), as well as high-
speed camera stills for one of them (a Type 1 upward positive flash). These
time-synchronised data served to confirm that the X-ray emissions detected are
associated with the initial steps of the upward negative leader. Further
analysis of the parameters at play revealed three additional points of
interest:

  * •

The frequency and energy of X-ray pulses appear to decrease as a functions of
time, with pulses disappearing altogether within the first millisecond of
leader initiation;

  * •

the Type 1 upward positive flashes exhibited the lowest percentages of pulses
with accompanying X-rays, which also ended sooner;

  * •

X-ray pulse energy appears to increase with the maximum current-derivative and
the electric field change of its associated leader step. This supports the
cold runaway electron model as the active mechanism for lightning leader X-ray
production.

These observations contribute to improving our understanding of upward
lightning, and will soon be followed by a more comprehensive review, including
X-ray–emitting upward negative flashes observed at the Säntis tower, and
simultaneous interferometric data gathered during the summer 2021 experimental
campaign.

Supplementary information

None.

Acknowledgments

This work was supported in part by the Swiss National Science Foundation
(Project no. 200020_204235) and the European Union’s Horizon 2020 research and
innovation program (grant agreement no. 737033-LLR). The authors would like to
thank Florent Aviolat for developing a data-visualisation software that
expedited the identification of events.

## Declarations

### Competing interests

None.

### Data availability

All processed data analysed during this study are included in this published
article (and its supplementary information files). Raw data sets generated
during the current study are available from the corresponding author on
reasonable request.

### Code availability

Upon request from corresponding author.

### Authors’ contributions

T.O.C. wrote the manuscript and completed the data analysis and processing
begun by A.S., who also gathered the data alongside M.R. and F.R., heads of
the Säntis Lightning Research Facility, during the summer 2021 experimental
campaign. P.H. and V.C. designed and built the X-ray detector providing the
data.

##  Appendix A Flashes UP0 & UP3

(a) The entire duration of the flash. A 100 kHz low-pass filter has been
applied to the current and $dI/dt$ waveforms to remove intermittent noise.

(b) Zoom on the X-ray event during the upward stepping negative leader phase.
The brown vertical dotted lines indicate the event times. See Table 3 for
pulse data.

Figure 7: Data associated with the Type 1 upward positive flash UP0, that
occurred on June 28, 2021 at 23:26:29 UTC. “PEMb” and “Bdtt” specify the
bottom Rogowski coil and top $\dot{B}$ sensor, respectively. $E_{z}$ is the
measured vertical component of the electric field. The time is from the
beginning of the recording ($\sim$1 second before the current peak).

(a) The entire duration of the flash. A 100 kHz low-pass filter has been
applied to the current and $dI/dt$ waveforms to remove intermittent noise.

(b) Zoom on the X-ray events during the upward stepping negative leader phase.
The brown vertical dotted lines indicate the event times. See Table 3 for
pulse data.

Figure 8: Data associated with the Type 2 upward positive flash UP3, that
occurred on July 30, 2021 at 18:00:10 UTC. “PEMb” and “Bdtt” specify the
bottom Rogowski coil and top $\dot{B}$ sensor, respectively. $E_{z}$ is the
measured vertical component of the electric field. The time is from the
beginning of the recording ($\sim$1 second before the current peak).

## References

  *   * (1) Moore, C.B., Eack, K.B., Aulich, G.D., Rison, W.: Energetic radiation associated with lightning stepped-leaders. Geophys. Res. Lett. 28(11), 2141–2144 (2001). https://doi.org/10.1029/2001GL013140. Accessed 2022-09-30
  * (2) Gurevich, A.V., Milikh, G.M., Roussel-Dupre, R.: Runaway electron mechanism of air breakdown and preconditioning during a thunderstorm. Physics Letters A 165(5-6), 463–468 (1992). https://doi.org/%****␣sn-article.bbl␣Line␣75␣****10.1016/0375-9601(92)90348-P. Accessed 2023-03-22
  * (3) Dwyer, J.R., Uman, M.A., Rassoul, H.K., Al-Dayeh, M., Caraway, L., Jerauld, J., Rakov, V.A., Jordan, D.M., Rambo, K.J., Corbin, V., Wright, B.: Energetic Radiation Produced During Rocket-Triggered Lightning. Science 299(5607), 694–697 (2003). https://doi.org/%****␣sn-article.bbl␣Line␣100␣****10.1126/science.1078940. Accessed 2022-10-06
  * (4) Dwyer, J.R., Rassoul, H.K., Al-Dayeh, M., Caraway, L., Chrest, A., Wright, B., Kozak, E., Jerauld, J., Uman, M.A., Rakov, V.A., Jordan, D.M., Rambo, K.J.: X-ray bursts associated with leader steps in cloud-to-ground lightning. Geophys. Res. Lett. 32(1), 01803 (2005). https://doi.org/10.1029/2004GL021782. Accessed 2022-10-06
  * (5) Saleh, Z., Dwyer, J., Howard, J., Uman, M., Bakhtiari, M., Concha, D., Stapleton, M., Hill, D., Biagi, C., Rassoul, H.: Properties of the X-ray emission from rocket-triggered lightning as measured by the Thunderstorm Energetic Radiation Array (TERA). J. Geophys. Res. 114(D17), 17210 (2009). https://doi.org/%****␣sn-article.bbl␣Line␣150␣****10.1029/2008JD011618. Accessed 2023-05-22
  * (6) Mallick, S., Rakov, V.A., Dwyer, J.R.: A study of X-ray emissions from thunderstorms with emphasis on subsequent strokes in natural lightning: X-RAY EMISSIONS FROM NATURAL LIGHTNING. J. Geophys. Res. 117(D16), (2012). https://doi.org/10.1029/2012JD017555. Accessed 2023-05-22
  * (7) Yoshida, S., Morimoto, T., Ushio, T., Kawasaki, Z.-I., Torii, T., Wang, D., Takagi, N., Watanabe, T.: High energy photon and electron bursts associated with upward lightning strokes: HIGH ENERGY PHOTON AND ELECTRON BURSTS. Geophys. Res. Lett. 35(10) (2008). https://doi.org/10.1029/2007GL032438. Accessed 2023-05-22
  * (8) Montanyà, J., Fabró, F., van der Velde, O., Romero, D., Solà, G., Hermoso, J.R., Soula, S., Williams, E.R., Pineda, N.: Registration of X-rays at 2500 m altitude in association with lightning flashes and thunderstorms. J. Geophys. Res. Atmos. 119(3), 1492–1503 (2014). https://doi.org/10.1002/2013JD021011. Accessed 2022-09-30
  * (9) Hettiarachchi, P., Cooray, V., Diendorfer, G., Pichler, H., Dwyer, J., Rahman, M.: X-ray Observations at Gaisberg Tower. Atmosphere 9(1), 20 (2018). https://doi.org/10.3390/atmos9010020. Accessed 2022-05-25
  * (10) Sunjerga, A., Mostajabi, A., Paolone, M., Rachidi, F., Romero, C., Hettiarachchi, P., Cooray, V., Azadifar, M., Rubinstein, A., Rubinstein, M., Pavanello, D., Smith, D.: Säntis Lightning Research Facility Instrumentation. ICLP, 6 (2021)
  * (11) Rachidi, F., Rubinstein, M.: Säntis lightning research facility: a summary of the first ten years and future outlook. Elektrotech. Inftech. 139(3), 379–394 (2022). https://doi.org/10.1007/s00502-022-01031-2. Accessed 2023-07-02
  * (12) Sunjerga, A., Rubinstein, M., Rachidi, F., Cooray, V.: On the Initiation of Upward Negative Lightning by Nearby Lightning Activity: An Analytical Approach. Geophys Res Atmos 126(5) (2021). https://doi.org/10.1029/2020JD034043. Accessed 2022-05-16
  * (13) Houard, A., Walch, P., Produit, T., Moreno, V., Mahieu, B., Sunjerga, A., Herkommer, C., Mostajabi, A., Andral, U., André, Y.-B., Lozano, M., Bizet, L., Schroeder, M.C., Schimmel, G., Moret, M., Stanley, M., Rison, W.A., Maurice, O., Esmiller, B., Michel, K., Haas, W., Metzger, T., Rubinstein, M., Rachidi, F., Cooray, V., Mysyrowicz, A., Kasparian, J., Wolf, J.-P.: Laser-guided lightning. Nat. Photon. 17(3), 231–235 (2023). https://doi.org/10.1038/s41566-022-01139-z. Accessed 2023-08-16
  * (14) Giri, D.V., Prather, W.D., Baum, C.E.: The Relationship Between NEMP Standards and Simulator Performance Specifications. Sensor and Simulation Notes Note 538 (2009)
  * (15) Romero, C., Rachidi, F., Rubinstein, M., Paolone, M., Rakov, V.A., Pavanello, D.: Positive lightning flashes recorded on the Säntis tower from May 2010 to January 2012: POSITIVE LIGHTNING SÄNTIS TOWER. J. Geophys. Res. Atmos. 118(23), 12879–12892 (2013). https://doi.org/10.1002/2013JD020242. Accessed 2023-05-01
  * (16) Rakov, V.A., Uman, M.A.: Lightning: Physics and Effects. Cambridge University Press, Cambridge, U.K. ; New York (2003)
  * (17) Moss, G.D., Pasko, V.P., Liu, N., Veronis, G.: Monte Carlo model for analysis of thermal runaway electrons in streamer tips in transient luminous events and streamer zones of lightning leaders. J. Geophys. Res. 111(A2), 02307 (2006). https://doi.org/10.1029/2005JA011350. Accessed 2023-05-22
  * (18) Dwyer, J.R.: Implications of x-ray emission from lightning: IMPLICATIONS OF X-RAYS FROM LIGHTNING. Geophys. Res. Lett. 31(12), (2004). https://doi.org/10.1029/2004GL019795. Accessed 2023-04-27
  * (19) Cooray, V., Becerra, M., Rakov, V.: On the electric field at the tip of dart leaders in lightning flashes. Journal of Atmospheric and Solar-Terrestrial Physics 71(12), 1397–1404 (2009). https://doi.org/10.1016/j.jastp.2009.06.002. Accessed 2023-09-26


# Mixed Attention Network for Hyperspectral Image Denoising

Zeqiang Lai Ying Fu
Beijing Institute of Technology
{laizeqiang<EMAIL_ADDRESS>
###### Abstract

Hyperspectral image denoising is unique for the highly similar and correlated
spectral information that should be properly considered. However, existing
methods show limitations in exploring the spectral correlations across
different bands and feature interactions within each band. Besides, the low-
and high-level features usually exhibit different importance for different
spatial-spectral regions, which is not fully explored for current algorithms
as well. In this paper, we present a Mixed Attention Network (MAN) that
simultaneously considers the inter- and intra-spectral correlations as well as
the interactions between low- and high-level spatial-spectral meaningful
features. Specifically, we introduce a multi-head recurrent spectral attention
that efficiently integrates the inter-spectral features across all the
spectral bands. These features are further enhanced with a progressive
spectral channel attention by exploring the intra-spectral relationships.
Moreover, we propose an attentive skip-connection that adaptively controls the
proportion of the low- and high-level spatial-spectral features from the
encoder and decoder to better enhance the aggregated features. Extensive
experiments show that our MAN outperforms existing state-of-the-art methods on
simulated and real noise settings while maintaining a low cost of parameters
and running time. Code is available at https://github.com/Zeqiang-Lai/MAN.

##  1 Introduction

Hyperspectral image (HSI) is made up of numerous bands across a wide range of
the spectrum. Different from common color images, HSIs divide the spectrum
into much more bands than three and their wavelengths can be extended beyond
the visible. Such properties make HSI especially attractive and useful for the
applications of remote sensing [3, 29, 41], face recognition [45, 43],
classification [2, 46, 6], _etc_. However, due to the limitation of existing
hyperspectral imaging techniques, the captured HSIs often suffer from severe
corruption, which makes the development of robust denoising algorithms an
urgent need.

Figure 1: Speed and performance comparison. Our method outperforms state-of-
the-art methods with low-cost parameters and runtime. Comparisons are
performed on the ICVL dataset with blind Gaussian noise.

Traditionally, optimization algorithms are often adopted to solve HSI
denoising with different hand-crafted priors exploring the domain knowledge of
the HSI, _i.e_., global correlation along the spectrum and spatial-spectral
correlation. Typical priors that are extensively studied includes, total
variation [36, 31], wavelet [23], low-rank [44, 28, 33], and etc. By
considering the spectral and spatial redundancy, non-local patch-similarity
[22] is also widely used in conjugation with variable splitting algorithms
[15] and tensor-based dictionary learning [26]. These methods generally have
no requirement for a large amount of training data or even be training-free,
but most of them are time-consuming (see Figure 1) and their performance is
strongly correlated with the matching degree of handcrafted priors with the
underlying characteristics of HSIs, which weakens their performance for real-
world denoising with complex noise.

Recent works on HSI denoising shift their attention to the learning-based
approaches to model complex intrinsic characteristics of HSI, _e.g_., inter-
spectral correlations with sliding-window convolution [37], and local spatial-
spectral correlations with 3D convolution [34]. Augmented with various
commonly used techniques, _e.g_., residual connection [37], skip connection
[13], and recurrent network [34], these methods have obtained much better
results than optimization-based ones. Nevertheless, their performance and
efficiency are still limited and the thorough consideration of inter- and
intra-spectral correlations is still lacking. Besides, though most HSI
restoration models [13, 34] employ skip connections to prevent the loss of
low-level information, the development of such modules mostly stays at the
original additive or concat versions. However, these vanilla ones either
neglect the different importance of the features from encoder and decoder, or
mix the tasks of features weighting and processing, which makes them less
effective for handling diverse information from different feature channels and
spectral bands. Therefore, it is foreseeable that whether a model can
comprehensively take information from different aspects into account would be
the key to further boosting the HSI denoising performance.

In this paper, we propose a Mixed Attention Network (MAN) for hyperspectral
image denoising, which simultaneously considers the inter- and intra-spectral
correlations as well as the interactions between low- and high-level spatial-
spectral meaningful features. Specifically, we introduce a Multi-Head
Recurrent Spectral Attention (MHRSA) block in place of vanilla spectral fusion
methods [34, 37]. It applies recurrent attention across the spectral dimension
to dynamically mix the inter-spectral features from all spectral bands. MHRSA
adopts two simple MLPs rather than dense convolutions to directly transform
the input features into candidate values and attention maps, which makes it
not only computationally efficient but also lightweight in terms of model
parameters. Another important characteristic of our MHRSA is the efficient bi-
directional context aggregation through a multi-head feature partition. This
is achieved by splitting rather than repeating the features into two heads and
performing parallel recurrent attention in reverse directions. Apart from
inter-spectral correlations, we propose a Progressive Spectral Channel
Attention (PSCA) that sequentially applies the dynamic channel mixing and the
static channel mixing to progressively mix the intra-spectral features for
each band. The MHRSA and PSCA cooperate with each other and enable the
informative features to propagate and interact along both inter-spectral and
intra-spectral dimensions, which results in exceedingly discriminative
features for subsequent reconstruction.

Furthermore, we reexamine the skip connections of the U-shaped models [13],
which are used to ensure the preservation of fine-grained details. By
analyzing the underlying operations of the most commonly used additive and
concat versions, we propose a novel Attentive Skip Connection (ASC) that
adaptively combines the low- and high-level features from encoder and decoder.
The ASC assigns element-wise fusion weight for each spatial-spectral location,
which allows subsequent layers to selectively focus on more informative
regions, thus leading to better denoising results. We conduct comprehensive
experiments and demonstrate the state-of-the-art performance of our MAN on
various HSI denoising datasets under simulated and real-world noise.

Our contributions are summarized as follows:

  * •

We propose a mixed attention network for hyperspectral image denoising, which
simultaneously considers the inter- and intra-spectral as well as low- and
high-level spatial-spectral feature correlations.

  * •

We introduce a multi-head spectral recurrent attention block to dynamically
aggregate the inter-spectral features, and a progressive spectral channel
attention block for integrating the intra-spectral features.

  * •

We present an attentive skip connection to adaptively strengthen the important
spatial-spectral meaningful features that flow forward from low- to high-
level.

##  2 Related Works

The methods for HSI denoising could generally be divided into optimization-
based methods, deep-learning methods, as well as hybrid methods. In this
section, we provide an overview of their recent major approaches.

###  2.1 Optimization-based Methods

Traditional methods solve the HSI denoising by treating it as an optimization
problem, where they attempt to find unknown clean HSI by minimizing an
optimization objective that incorporates the properties of spectrum and
images. Such incorporation is generally achieved by designing different hand-
crafted priors, e.g., total variation priors [36, 31], wavelet priors [23],
and low-rank priors [44, 28, 31, 33]. By considering the non-local self-
similarity in the spectral and spatial dimensions, many works such as block-
matching and 4-D filtering (BM4D) [22] and the tensor dictionary learning [26]
are also proposed.

These optimization-based HSI denoising methods are flexible to remove
different types of noise [16, 10] and can be even extended to tasks beyond
denoising [9, 14, 15, 25]. However, their performance is significantly
restricted by the matching degree of handcrafted prior and the underlying
properties of HSI, which is difficult to ensure for complex real scenes. To
address this problem, plug-and-play methods [7, 12, 47] integrate the
optimization-based method with a learning-based prior, _i.e_., a plug-and-play
Gaussian denoiser [11, 40], to tackle complex noises that cannot be easily
modeled by handcrafted prior. Specifically, Liu _et al_ [20] propose a fibered
rank constrained tensor restoration framework and adopt BM3D [11] as an extra
plug-and-play regularization. In [21], FFDNet [40] is used for local
regularity, alongside the Kronecker-basis-representation-based tensor low-
rankness for global structures regularity.

###  2.2 Deep-Learning-based Methods

Deep-learning-based methods have gained superior popularity in recent years.
Inspired by 2D image denoising network DnCNN [39], Chang _et al_. [8] propose
HSI-DeNet that learns multi-channel 2-D filters to model spectral correlation.
Yuan _et al_. [37] introduce residual network structure with a sliding window
strategy for remote sensed HSI. To further exploit the spatial-spectral
correlation, Dong _et al_. [13] designed a 3D U-net architecture. These
methods have been successfully applied to different HSIs, but most of them are
limited at exploring the inter-spectral correlations, which is significantly
important for HSI denoising. To address such issue, Wei _et al_. [34] proposed
a 3D convolutional quasi-recurrent neural network (QRNN3D), which combines the
3D convolution to extract spatial-spectral correlated features, and a quasi-
recurrent network to integrate information from different bands in a global
perspective. Based on it, Lai _et al_. [19] propose GRUNet that further
improves the performance with residual blocks. Recent works also explore some
hybrid approaches by borrowing key techniques from different types of methods.
For example, T3SC [4] is introduced as a hybrid method that combines sparse
coding principles and deep neural networks. TRQ3DNet [24] augments the QRNN3D
with a Uformer [32] block to enhance the capabilities for capturing long-range
spatial dependency. Compared with these methods, our method considers not only
the inter-spectral correlation with a more powerful multi-head recurrent
spectral attention but also the intra-spectral interaction with a progressive
spectral channel attention block, which results in more discriminative
features for the denoising. Moreover, we reexamine the low- and high-level
feature fusion and propose an attentive skip connection for better feature
aggregation, which is ignored in most existing works.

##  3 Mixed Attention Network

In this section, we first describe the overall architecture of our Mixed
Attention Network (MAN). Then we provide detailed illustrations for each
attention block, including multi-head recurrent spectral attention,
progressive spectral channel attention, and attentive skip connection.

#### Overall Architecture.

The overall architecture of the proposed MAN is shown in Figure 2. The input
noisy image $\mathrm{X}\in\mathbb{R}^{H\times W\times S}$ is sequentially
processed by a multi-level encoder to obtain multi-scale image features, which
is then decoded by a symmetrically designed decoder to obtain the
reconstructed clean image. We build the entire network by stacking Mixed
Attention Blocks (MAB) that consist of a 3D convolution, a multi-head
recurrent spectral attention, and a progressive spectral channel attention to
explore the inter- and intra-spectral correlations. Each MAB takes input
features $\mathrm{F}\in\mathbb{R}^{H\times W\times S\times C}$ and generates
output features $\mathrm{\hat{F}}\in\mathbb{R}^{\hat{H}\times\hat{W}\times
S\times\hat{C}}$ with the same spectral resolution $S$. Different from other
encoder-decoder-based networks, MAN replaces the vanilla skip connection with
an attentive one that adaptively controls which part of low-level features
should flow forward from the encoder to the decoder.

Figure 2: Overall architectures of the proposed mixed attention network and
mixed attention block.

###  3.1 Multi-Head Recurrent Spectral Attention

Figure 3: Illustration of the multi-head recurrent spectral attention. MLP
stands for two-layer linear projections with Tanh activation. $C$, $S$, $HW$
denote the feature, spectral, and spatial dimensions, respectively.

Different from RGB images, HSIs have a wide range of the spectrum, and the
images across different bands share generally identical spatial content.
Moreover, the pixel values with the same spatial location across different
bands are correlated with specific spectral patterns, which are determined by
the material of the object they belong to. This illustrates one important
characteristic of HSI, _i.e_., inter-spectral correlation/similarity. Similar
to non-local spatial correlation/similarity, it is straightforward to utilize
inter-spectral correlations to perform denoising with methods like non-local
means (NLM) [5]. However, unlike spatial similar pixels, pixels along the
spectrum have a different range of values, which makes naive NLM unsuitable as
its averaging operation could break the spectral relationship of each pixel
across the spectrum.

To address the above issue, we propose a Multi-Head Recurrent Spectral
Attention (MHRSA) block that dynamically computes the weights for averaging
the pixels along the spectrum for each band. Each band is assigned a different
weight for averaging information from other bands, which avoids the
destruction of spectral dependency. An illustration of the MHRSA block is
given in Figure 3. Given a feature map $\mathrm{F}\in\mathbb{R}^{H\times
W\times S\times C}$ extracted from the previous 3D convolution, it first uses
two MLPs followed by two different activation functions to transform the input
features into the candidate features $\mathrm{Z}$ and merging weights
$\mathrm{W}$, as

$\displaystyle\mathrm{Z}$
$\displaystyle=\operatorname{tanh}(\operatorname{MLP_{1}}(\mathrm{F}))$ (1)
$\displaystyle\mathrm{W}$
$\displaystyle=\operatorname{sigmoid}(\operatorname{MLP_{2}}(\mathrm{F}))$
$\displaystyle\operatorname{MLP}(\mathrm{X})$
$\displaystyle=\mathrm{W_{1}}\cdot(\operatorname{tanh}(\mathrm{W_{2}}\cdot\mathrm{X}))$

where $\mathrm{W_{1}},\mathrm{W_{2}}\in\mathbb{R}^{C\times C}$. These
processes can be equivalently considered as the query, key, and value
projections in self-attention [30]. The one difference is that we directly
compute the attention weight instead of obtaining the attention map through
the covariance of key and query. The other is that we perform the attention
operation through a recurrent merging step that only requires linear memory
and time complexity other than the quadratic ones, which makes our method more
suitable for high dimensional HSI data.

Specifically, the recurrent merging step for spectral mixing is performed by
accumulating the candidate features $\mathrm{Z}$ for each band based on
merging weight $\mathrm{W}$ as

$\mathrm{O_{i}}=(1-\mathrm{W_{i}})\odot\mathrm{Z_{i}}+\mathrm{W_{i}}\odot\mathrm{O_{i-1}}$
(2)

where $\mathrm{O_{i}}$, $\mathrm{Z_{i}}$, $\mathrm{W_{i}}$ are the output
features, candidate features, and merging weights of $i^{th}$ band,
respectively. It can be observed that such a merging step fuse the features
from all the previous bands $\mathrm{Z_{i}},i<j$ for $j^{th}$ band. Thus, it
correlates the inter-spectral features and can potentially utilize the
information from cleaner bands for denoising noisier bands.

Furthermore, though the above merging step progressively aggregates the inter-
spectral features, each band can only see the features in one direction. It
can lead to incomplete spectral context. To address the issue, we propose a
multi-head attention in which we equally split the input features into two
parts and perform parallel merging for each part with different directions.
With such a strategy, we could achieve feature aggregation with global
spectral context without any extra computation and parameters.

###  3.2 Progressive Spectral Channel Attention

The MHRSA provides stronger capabilities for exploring the inter-spectral
correlations, which makes our model powerful at borrowing the features from
more informative bands for less informative ones. However, these advantages
would be weakened if the features of each band itself are not discriminative
enough. Therefore, we propose to further strengthen the features of each band
by exploring the intra-spectral correlations with a Progressive Spectral
Channel Attention (PSCA) module, as shown in Figure 4.

Our PSCA generalizes the band-wise Squeeze-Excitation (SE) [17] like channel
attention to a pixel-wise operation with a progressive attention pipeline,
which contains a Dynamic Channel Mixing (DCM) and a Static Channel Mixing
(SCM) processes, as

$\mathrm{Y}=\operatorname{SCM}(\operatorname{GELU}(\operatorname{DCM}(X)))$
(3)

where $\mathrm{X},\mathrm{Y}$ are the input and output feature maps.

#### Static Channel Mixing.

Different from SE-like channel attention that performs channel-wise gating to
control which channel should pass more information to subsequent layers, our
SCM mix the information of all channels through a static attention map
$\mathrm{W}\in\mathbb{R}^{C_{in}\times C_{out}}$ as,

$\operatorname{SCM}(\mathrm{X})=\mathrm{X}\cdot\mathrm{W}.$ (4)

The attention map is jointly learned with the network training and it encodes
the dataset level prior for selecting more diverse and important information
across different features.

#### Dynamic Channel Mixing.

To further strengthen the capability at identifying the essential features for
each input HSI. We propose DCM that adopts a pixel-wise dynamic weight for
channel mixing. Specifically, DCM first computes a scaling weight $\mathrm{S}$
for each input through a linear projection,
$\mathrm{W_{2}}\in\mathbb{R}^{C\times C}$. Then, we rescale the static
attention map $\mathrm{W_{1}}$ with the scaling weight through element-wise
multiplication for each pixel location. Finally, we perform the channel mixing
with the new attention map as

$\displaystyle\operatorname{DCM}(\mathrm{X})$
$\displaystyle=\mathrm{W_{1}}\cdot\mathrm{S}\odot\mathrm{X},$ (5)
$\displaystyle\mathrm{S}$ $\displaystyle=\mathrm{X}\cdot\mathrm{W_{2}}.$ (6)

With these two channel mixing steps, we could obtain substantially better
representations for each spectral band, thus could lead to better performance.

###  3.3 Attentive Skip Connection

Skip connection is the key part that distinguishes the U-Net [13] from other
network architectures. It provides a direct but effective way to recover the
low-level information that is lost during the downsampling operations of the
commonly used encoder-decoder convolutional neural network.

The most commonly used additive skip connection is implemented as an addition
between encoder and decoder features at the same depth. Supposing features of
encoder and decoder at the $i^{th}$ depth level are $\mathrm{F_{e}^{i}}$ and
$\mathrm{F_{d}^{i}}$, respectively, the addition skip connection can be
precisely described as

$\mathrm{F_{d}^{i+1}}=\mathrm{F_{d}^{i}}+\mathrm{F_{e}^{i}}.$ (7)

The major problem of it lies in the same weight on the shallow features from
the encoder and highly processed features from the decoder. This could be
problematic due to the imbalanced information density of different spectral
bands and feature channels. For example, it would be more helpful to preserve
more low-level features for cleaner spectral bands to retain the details while
we might want more high-level features for noisy bands to properly remove all
the noise.

To address the issue, we propose an Attentive Skip Connection (ASC) that
explicitly weights the features from different sources using the attention
weight computed from two convolution layers. By denoting the weight of
$i^{th}$ depth level as $\mathrm{M_{i}}$, the detailed formulation is

$\displaystyle\mathrm{F}$
$\displaystyle=\operatorname{LeakyReLU}(\operatorname{Conv1x1}([\mathrm{F_{d}^{i}},\mathrm{F_{e}^{i}}]))$
(8) $\displaystyle\mathrm{M_{i}}$
$\displaystyle=\sigma(\operatorname{Conv3x3}(\mathrm{F})),$
$\displaystyle\mathrm{F_{d}^{i+1}}$
$\displaystyle=(1-\mathrm{M_{i}})\odot\mathrm{F_{d}^{i}}+\mathrm{M_{i}}\odot\mathrm{F_{e}^{i}},$

where $\sigma$ denotes the sigmoid function. A visualization of our attentive
skip connection is shown in Figure 5. Our ASC adaptively strengthens the
features of more important spatial locations and spectral bands through
element-wise gating, thus leading to better reconstruction quality.

Figure 4: Illustration of progressive spectral channel attention. $\odot$,
$\otimes$ denote element-wise- and original matrix multiplication.

#### Analysis and in-depth comparison.

We provide an in-depth analysis of the difference between different types of
skip connections. Typically, the output of the skip connection module would be
processed by the subsequent convolution layer. For additive skip connection,
the computation can be formulated as,

$\mathrm{F_{o}}=\mathrm{W}\times(\mathrm{F_{e}}+\mathrm{F_{d}})=\mathrm{W}\times\mathrm{F_{e}}+\mathrm{W}\times\mathrm{F_{d}}.$
(9)

where $\mathrm{F_{o}}$ is the output features from the subsequent convolution
layer, $\mathrm{W}$ denotes the convolution kernel, $\times$ denotes
convolution, and $\mathrm{F_{e}}$ and $\mathrm{F_{d}}$ are the features from
the encoder and the decoder. By using distributive law, we could know that we
actually treat the features from the encoder and decoder as features in the
same manifold, which are processed by the same convolution kernel.

For concat skip connection, the formulation would be,

$\mathrm{F_{o}}=\mathrm{W}\times[\mathrm{F_{e}},\mathrm{F_{d}}]=\mathrm{W_{1}}\times\mathrm{F_{e}}+\mathrm{W_{2}}\times\mathrm{F_{d}}.$
(10)

where $\mathrm{W}$ is a convolution that transforms features with 2C channels
to C channels. According to the properties of convolution, we could divide
$\mathrm{W}$ into two parts, $\mathrm{W_{1}}$ and $\mathrm{W_{2}}$, which
indicates that concat skip connection assigns different kernels for features
from the encoder and decoder.

From the above analysis, we know that additive skip connection averages the
features from the encoder and decoder extracted by the same convolution
kernel. This might weaken the discriminative features due to the feature
average. On the other hand, the concat skip connection extracts feature using
two different kernels and then performs feature average. Thus, it is possible
for it to generate features that avoid the problem of feature weakening in
additive skip connection. However, its success requires two convolution
kernels to simultaneously consider the feature extraction and reweighting,
which is difficult to achieve. This not only includes the training
difficulties but also the limitation of convolution itself for pixel-wise
reweighting.

To address the aforementioned issues, our ASC separates the feature
reweighting and extraction into two steps as,

$\displaystyle\mathrm{F_{o}}$
$\displaystyle=\mathrm{W}\times(\underbrace{\mathrm{M}\odot\mathrm{F_{e}}+(1-\mathrm{M})\odot\mathrm{F_{d}}}_{\text{feature
reweighting}})$ (11)

where $\mathrm{M}$ is the pixel-wise weight for reweighting. Such
decomposition has two advantages. First, it avoids the need to learn two very
discriminative convolutions as the concat skip connection, which makes the
network concentrate more on finding more useful features. Second, it
explicitly reweights the features from encoder and decoder, which
distinguishes the more informative features to pass through the subsequent
layers.

Figure 5: Illustration of the attentive skip connection block.

##  4 Experiments

In this section, we provide the experiments for simulated and real-world HSI
denoising. An ablation study and discussion of each component are also
included.

Sigma | Metric |  Noisy |  ITSReg [35] |  WLRTR [9] |  KBR [27] |  NGmeet [15] |  HSID [37] |  QRNN3D [34] |  T3SC [4] |  GRUNet [19] |  TRQ3D [24] |  MAN-S (ours) |  MAN-M (ours) |  MAN-L (ours)
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
30 | PSNR | 18.59 | 41.48 | 42.62 | 41.48 | 42.99 | 41.72 | 42.22 | 42.36 | 42.84 | 43.25 | 43.83 | 44.07 | 44.17
SSIM | 0.110 | 0.961 | 0.988 | 0.984 | 0.989 | 0.987 | 0.988 | 0.986 | 0.989 | 0.990 | 0.991 | 0.991 | 0.991
SAM | 0.807 | 0.088 | 0.056 | 0.088 | 0.050 | 0.067 | 0.062 | 0.079 | 0.052 | 0.046 | 0.043 | 0.042 | 0.041
50 | PSNR | 14.15 | 38.88 | 39.72 | 39.16 | 40.26 | 39.39 | 40.15 | 40.47 | 40.75 | 41.30 | 41.60 | 41.84 | 41.94
SSIM | 0.046 | 0.941 | 0.978 | 0.974 | 0.980 | 0.980 | 0.982 | 0.980 | 0.983 | 0.985 | 0.985 | 0.986 | 0.986
SAM | 0.991 | 0.098 | 0.073 | 0.100 | 0.059 | 0.083 | 0.074 | 0.087 | 0.062 | 0.053 | 0.052 | 0.050 | 0.049
70 | PSNR | 11.23 | 36.71 | 37.52 | 36.71 | 38.66 | 37.77 | 38.30 | 39.05 | 39.02 | 39.86 | 40.05 | 40.32 | 40.40
SSIM | 0.025 | 0.923 | 0.967 | 0.961 | 0.974 | 0.972 | 0.974 | 0.974 | 0.977 | 0.980 | 0.980 | 0.981 | 0.981
SAM | 1.105 | 0.112 | 0.095 | 0.113 | 0.067 | 0.096 | 0.094 | 0.096 | 0.080 | 0.061 | 0.060 | 0.058 | 0.057
Blind | PSNR | 17.34 | 40.62 | 41.66 | 40.68 | 42.23 | 40.95 | 41.37 | 41.52 | 42.03 | 42.47 | 43.08 | 43.35 | 43.44
SSIM | 0.114 | 0.953 | 0.983 | 0.979 | 0.985 | 0.984 | 0.985 | 0.983 | 0.987 | 0.988 | 0.988 | 0.989 | 0.989
SAM | 0.859 | 0.087 | 0.064 | 0.080 | 0.053 | 0.072 | 0.068 | 0.085 | 0.057 | 0.054 | 0.046 | 0.045 | 0.044

((a)) GT

((b)) WLRTR

((c)) KBR

((d)) NGmeet

((e)) HSID

((f)) QRNN3D

((g)) T3SC

((h)) GRUNet

((i)) TRQ3D

((j)) MAN-M

Table 1: Simulated Gaussian noise removal results under several noise levels
on ICVL. _Blind_ suggests each image is corrupted by Gaussian noise with
random sigma (ranged from 30 to 70). Visual results show the $20^{th}$ band
under noise level 50.

###  4.1 Datasets

We conducted experiments of simulated HSI denoising with ICVL [1] and real-
worlding HSI denoising with Urban and HSIDwRD [42]. ICVL contains 201 clean
HSI in the resolution of $1392\times 1300$ that divides the spectrum into 31
spectral bands. Urban and HSIDwRD contain 1 HSI and 59 HSIs with 210 and 34
bands. Following [34], we split the ICVL into three parts, _i.e_., 100 images
for training, 51 for validation, and 50 for testing. For training, we process
the original HSIs into multiple overlapped smaller data cubes via even-stride
cropping. The spatial resolution of each cube is $64\times 64$ and the
spectral resolution remains unchanged. Random rotation and scaling are also
employed to further augment the dataset. For testing, the main region in the
size of $512\times 512\times 31$ is used for ICVL, and 15 images in HSIDwRD
are randomly selected. We use pretrained models of ICVL for Urban and HSIDwRD.

Type | Metric |  Noisy |  LRMR [38] |  LRTV [16] |  NMoG [10] |  TDTV [31] |  HSID [37] |  QRNN3D [34] |  T3SC [4] |  GRUNet [19] |  TRQ3D [24] |  MAN-S (ours) |  MAN-M (ours)
---|---|---|---|---|---|---|---|---|---|---|---|---|---
G+Stripe | PSNR | 17.80 | 32.62 | 33.49 | 33.87 | 37.67 | 37.77 | 42.35 | 40.85 | 42.39 | 43.05 | 44.24 | 44.60
SSIM | 0.159 | 0.717 | 0.905 | 0.799 | 0.940 | 0.942 | 0.976 | 0.986 | 0.991 | 0.992 | 0.993 | 0.994
SAM | 0.910 | 0.187 | 0.078 | 0.265 | 0.081 | 0.104 | 0.055 | 0.072 | 0.050 | 0.043 | 0.039 | 0.038
G+Deadline | PSNR | 17.61 | 31.83 | 32.37 | 32.87 | 36.15 | 37.65 | 42.23 | 39.54 | 42.11 | 42.95 | 44.13 | 44.51
SSIM | 0.155 | 0.709 | 0.895 | 0.797 | 0.930 | 0.940 | 0.976 | 0.983 | 0.991 | 0.992 | 0.993 | 0.994
SAM | 0.917 | 0.227 | 0.115 | 0.276 | 0.099 | 0.102 | 0.056 | 0.096 | 0.050 | 0.044 | 0.040 | 0.038
G+Impulse | PSNR | 14.80 | 29.70 | 31.56 | 28.60 | 36.67 | 35.00 | 39.23 | 36.06 | 40.70 | 41.27 | 41.88 | 41.97
SSIM | 0.114 | 0.623 | 0.871 | 0.652 | 0.935 | 0.899 | 0.945 | 0.952 | 0.985 | 0.983 | 0.982 | 0.979
SAM | 0.926 | 0.311 | 0.242 | 0.486 | 0.094 | 0.174 | 0.109 | 0.203 | 0.067 | 0.075 | 0.090 | 0.095
G+Mixture | PSNR | 14.08 | 28.68 | 30.47 | 27.31 | 34.77 | 34.05 | 38.25 | 34.48 | 38.51 | 40.27 | 41.03 | 41.18
SSIM | 0.099 | 0.608 | 0.858 | 0.632 | 0.919 | 0.888 | 0.938 | 0.946 | 0.981 | 0.983 | 0.981 | 0.979
SAM | 0.944 | 0.353 | 0.287 | 0.513 | 0.113 | 0.181 | 0.107 | 0.228 | 0.081 | 0.075 | 0.092 | 0.097

((a)) GT

((b)) LRTV

((c)) NMoG

((d)) TDTV

((e)) HSID

((f)) QRNN3D

((g)) T3SC

((h)) GRUNet

((i)) TRQ3D

((j)) MAN-M

Table 2: Simulated complex noise removal results under several noise types on
ICVL. Visual results show the $20^{th}$ band under stripe noise.

###  4.2 Implementation Details

Adam [18] optimizer is adopted to minimize the mean square error. We follow
the training strategy as [34], with slight modifications on the setup of the
learning rate. The strategy is briefly described here and we refer interested
readers to [34] for more details. In short, the network is first trained on
Gaussian noise with a fixed level of 50 for 20 epochs and then finetuned with
a random noise level from a given set for 40 epochs to produce the first-stage
Gaussian denoising model. In the second stage, the complex denoising model is
obtained via another 30 epochs of fine-tuning using the trained Gaussian
denoising model. The learning rate is set to $1\times 10^{-3}$ at first, but
decayed by $0.1$ every 5 or 10 epochs, and finally reduced to $1\times
10^{-5}$ for each stage. The batch size is set to 16.

###  4.3 Results on Gaussian Noise

For the experiments with Gaussian noise, we evaluate our model with different
noise strengths, including 30, 50, 70, and random strengths ranging from 30 to
70. The simulated noisy input is generated by adding zero-mean additive white
Gaussian noise with a given variance. We use a single model to tackle
different noise levels.

For systematical evaluation, we compare our method with four traditional HSI
denoising methods, _i.e_., ITSReg [35], KBR [27], WLRTR [9], and NGmeet [15],
and five deep-learning ones, including HSID [37], QRNN3D [34], T3SC [4],
GRUNet [19], and TRQ3D [24]. In the spirit of fairness, the hyperparameters in
traditional methods are carefully tuned, and the deep-learning-based models
are retrained if needed.

The quantitative and visual results are shown in Table 1. The comparisons of
runtime and model size are provided in Table 5. It can be easily observed that
our method achieves the best performance in all different settings with a
large margin over the competing methods. Specifically, our MAN-L achieves over
0.6 dB PSNR improvement on average on four different settings while
maintaining comparable parameters and runtime. our MAN-S achieves over 0.3 dB
improvement against the best competing method, _i.e_., TRQ3D, with even fewer
parameters. Moreover, our model is better at preserving spectral fidelity than
previous methods, which is reflected by the improvement in SAM.

Metrics |  NMoG [10] |  TDTV [31] |  QRNN3D [34] |  HSID [37] |  HSIDwRD [42] |  MAN-M (ours)
---|---|---|---|---|---|---
PSNR | 30.90 | 31.14 | 31.13 | 31.05 | 31.23 | 31.36
SSIM | 0.907 | 0.881 | 0.940 | 0.934 | 0.939 | 0.940
SAM | 1.761 | 1.853 | 0.094 | 0.096 | 0.092 | 0.092
Table 3: Quantitative results on HSIDwRD [42].

###  4.4 Results on Complex Noise

Unlike the simulated noisy HSI with Gaussian noise, the real HSI is usually
corrupted by different types of complex noise, e.g., strip noise, impulse
noise, and deadline noise. To evaluate the robustness of our method in real
scenes, we also conduct experiments with different combinations of complex
noise, following the settings in [34].

We compare our method with eight recently developed methods including five
deep-learning-based methods, _i.e_., HSID [37], QRNN3D [34], GRUNet [19], T3SC
[4], and TRQ3D [24], as well as four optimization-based ones, _i.e_., LRMR
[38], LRTV [16], NMoG [10] and TDTV [31]. We choose a different set of
optimization-based methods from the ones used for Gaussian noise because these
methods only perform well for the noise settings they can solve.

Table 2 gives the quantitative and visual comparison between our method
against the competing ones. It can be seen that our method obtain substantial
improvement on various metrics, especially on PSNR, which is over 1.5 dB on
average, and ranged from 1.03 $\sim$ 1.7 dB.

MHRSA | ASC | PSCA | Params | Runtime | PSNR | SAM
---|---|---|---|---|---|---
- | - | - | 0.43 | 0.25 | 39.98 | 0.072
✓ | - | - | 0.50 | 0.46 | 41.13 | 0.058
✓ | ✓ | - | 0.81 | 0.55 | 41.48 | 0.053
✓ | ✓ | ✓ | 0.89 | 0.65 | 41.84 | 0.050
Table 4: Ablation study of the proposed network modules.

###  4.5 Results on Real-world Noise

#### Urban.

Figure 6 provide the visual result of our model on real-world noisy remotely
sensed HSI, _i.e_., Urban. It can be seen that our model could properly remove
the heavy noise while retaining the details. QRNN3D removes the most noise but
loses the details such as stripes on the roof. HSID tends to produce blurry
results than Ours.

#### HSIDwRD.

We conduct experiments on the real natural HSI dataset [42] with the first two
leading optimization-based and CNN-based methods from complex denoising and
the method proposed in [42]. The quantitative results are shown in Table 3. It
can be observed that our results outperform the other ones in terms of all
metrics, which demonstrates the effectiveness of our methods.

((a)) Noisy

((b)) NMoG [10]

((c)) TDTV [31]

((d)) HSID [37]

((e)) QRNN3D [34]

((f)) GRUNet [19]

((g)) MAN(Ours)

Figure 6: Real world noise removal results at $207^{th}$ band of Urban Dataset. Method |  ITSReg [35] |  WLRTR [9] |  KBR [27] |  NGmeet [15] |  HSID [37] |  QRNN3D [34] |  T3SC [4] |  GRUNet [19] |  TRQ3D [24] |  MAN-S (ours) |  MAN-M (ours) |  MAN-L (ours)
---|---|---|---|---|---|---|---|---|---|---|---|---
PSNR | 38.88 | 39.16 | 38.70 | 40.26 | 39.39 | 40.15 | 40.47 | 40.75 | 41.30 | 41.60 | 41.84 | 41.94
Params (M) | - | - | - | - | 0.40 | 0.86 | 0.83 | 14.2 | 0.68 | 0.50 | 0.89 | 1.39
Runtime (s) | 907 | 1600 | 1755 | 166 | 0.48 | 0.44 | 0.95 | 0.87 | 0.47 | 0.49 | 0.65 | 0.97
Table 5: Performance-Params-Runtime comparisons with the SOTA methods.
Evaluated with Gaussian denoising (sigma=50) on ICVL. Figure 7: Visualization
of the attention map of our attentive skip connection across different bands
and channels. Figure 8: Visualization of the spectral attention map from our
MHRSA. (a) The left part is the average attention map between different bands.
(b) The right part is the attention distribution of $15^{th}$ band across
different pixels.

###  4.6 Discussions

#### Ablation Study.

To verify the effectiveness of each proposed component. We evaluate the PSNR
improvement each component brings by adding them one by one, starting from a
baseline model, _i.e_., UNet with 3D convolution and activation only. All the
models are trained for the Gaussian denoising task and evaluated on 50 noise
strength. The experimental results are shown in Table 4. As we can observe,
the introduction of MHRSA brings the most significant improvement. This
indicates the importance of exploring inter-spectral correlations. The
attentive skip connection contributes 0.35 dB improvement by taking into
account low- and high-level features fusion. Our PSCA strengthens the features
from MHRSA by considering intra-spectral relationships, which further provides
0.36 dB improvement.

#### Visualization of Attentive Skip Connection.

We visualize the attention map of the first attentive skip connection block
for one sample. As is shown in Figure 7, the attention map varies across
different bands and channels, which indicates that the equal attention of
vanilla additive one might be less effective. In particular, it can be seen
that the network learns to pay attention to different regions across different
bands and channels, _i.e_., band12 and channel 21 for edges, and different
sources, _e.g_., channel 18 for encoder, channel 38 for decoder.

#### Visualization of Spectral Attention.

Figure 8 shows the spectral attention map of one particular example HSI at the
first MHRSA layer. It can be seen that our MHRSA pays attention to all bands
with different weights, which is helpful for adaptively aggregating the useful
features from different bands to assist the denoising. In general, the overall
pattern is that each band pays more attention to the bands around it, but the
attention distributions may vary with respect to pixels at different
locations.

##  5 Conclusion

In this paper, we propose a mixed attention network for hyperspectral image
denoising. Our method introduces several key components to properly explore
the inter- and intra-spectral correlations as well as the low- and high-level
spatial-spectral feature interactions. Specifically, these are achieved with a
multi-head recurrent spectral attention that recurrently merges the features
across different bands, a progressive channel attention that progressively
mixes the different features within each band, and an attentive skip
connection that aggregates the features from encoder and decoder with
different importance weight. We perform extensive experiments on simulated and
real-world noise, and it shows that our method outperforms the existing state-
of-the-art methods with significant improvement while maintaining a smaller
model size and running time.

#### Ethical considerations and future work.

Our work has no ethical issues. In this work, we explore the proposed MHRSA
with multi-head in two directions, but it is also possible to extend it to
multi-axis in which we perform attention in different dimensions, _e.g_.,
channel and spectral.

## References

  * [1] Boaz Arad and Ohad Ben-Shahar.  Sparse recovery of hyperspectral signal from natural rgb images.  In Proceedings of the European Conference on Computer Vision, pages 19–34. Springer, 2016.
  * [2] Saeideh Ghanbari Azar, Saeed Meshgini, Tohid Yousefi Rezaii, and Soosan Beheshti.  Hyperspectral image classification based on sparse modeling of spectral blocks.  Neurocomputing, 407:12–23, 2020.
  * [3] George Alan Blackburn.  Hyperspectral remote sensing of plant pigments.  Journal of Experimental Botany, 58(4):855–867, 2007.
  * [4] Théo Bodrito, Alexandre Zouaoui, Jocelyn Chanussot, and Julien Mairal.  A trainable spectral-spatial sparse coding model for hyperspectral image restoration.  34:5430–5442, 2021.
  * [5] Antoni Buades, Bartomeu Coll, and Jean-Michel Morel.  Non-local means denoising.  Image Processing On Line, 1:208–212, 2011.
  * [6] Xianghai Cao, Yiming Ge, Renjie Li, Jing Zhao, and Licheng Jiao.  Hyperspectral imagery classification with deep metric learning.  Neurocomputing, 356:217–227, 2019.
  * [7] Stanley H Chan, Xiran Wang, and Omar A Elgendy.  Plug-and-play admm for image restoration: Fixed-point convergence and applications.  IEEE Transactions on Computational Imaging, 3(1):84–98, 2016.
  * [8] Yi Chang, Luxin Yan, Houzhang Fang, Sheng Zhong, and Wenshan Liao.  Hsi-denet: Hyperspectral image restoration via convolutional neural network.  IEEE Transactions on Geoscience and Remote Sensing, 57(2):667–682, 2018.
  * [9] Yi Chang, Luxin Yan, Xi-Le Zhao, Houzhang Fang, Zhijun Zhang, and Sheng Zhong.  Weighted low-rank tensor recovery for hyperspectral image restoration.  IEEE Transactions on Cybernetics, 50(11):4558–4572, 2020.
  * [10] Yang Chen, Xiangyong Cao, Qian Zhao, Deyu Meng, and Zongben Xu.  Denoising hyperspectral image with non-iid noise structure.  IEEE Transactions on Cybernetics, 48(3):1054–1066, 2017.
  * [11] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian.  Image denoising by sparse 3-d transform-domain collaborative filtering.  IEEE Transactions on Image Processing, 16(8):2080–2095, 2007.
  * [12] Aram Danielyan, Vladimir Katkovnik, and Karen Egiazarian.  Image deblurring by augmented lagrangian with bm3d frame prior.  In Proceedings of the Workshop on Information Theoretic Methods in Science and Engineering, volume 1, 2010.
  * [13] Weisheng Dong, Huan Wang, Fangfang Wu, Guangming Shi, and Xin Li.  Deep spatial–spectral representation learning for hyperspectral image denoising.  IEEE Transactions on Computational Imaging, 5(4):635–648, 2019\.
  * [14] Ying Fu, Antony Lam, Imari Sato, and Yoichi Sato.  Adaptive spatial-spectral dictionary learning for hyperspectral image restoration.  International Journal of Computer Vision, 122(2):228–245, 2017\.
  * [15] Wei He, Quanming Yao, Chao Li, Naoto Yokoya, and Qibin Zhao.  Non-local meets global: An integrated paradigm for hyperspectral denoising.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6868–6877, 2019.
  * [16] Wei He, Hongyan Zhang, Liangpei Zhang, and Huanfeng Shen.  Total-variation-regularized low-rank matrix factorization for hyperspectral image restoration.  IEEE Transactions on Geoscience and Remote Sensing, 54(1):178–188, 2015.
  * [17] Jie Hu, Li Shen, and Gang Sun.  Squeeze-and-excitation networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  * [18] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [19] Zeqiang Lai, Kaixuan Wei, and Ying Fu.  Deep plug-and-play prior for hyperspectral image restoration.  Neurocomputing, 481:281–293, 2022.
  * [20] YunYang Liu, XiLe Zhao, YuBang Zheng, TianHui Ma, and Hongyan Zhang.  Hyperspectral image restoration by tensor fibered rank constrained optimization and plug-and-play regularization.  IEEE Transactions on Geoscience and Remote Sensing, pages 1–17, 2021.
  * [21] TianHui Ma, Zongben Xu, Deyu Meng, and XiLe Zhao.  Hyperspectral image restoration combining intrinsic image characterization with robust noise modeling.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:1628–1644, 2020.
  * [22] Matteo Maggioni, Vladimir Katkovnik, Karen Egiazarian, and Alessandro Foi.  Nonlocal transform-domain filter for volumetric data denoising and reconstruction.  IEEE Transactions on Image Processing, 22(1):119–133, 2012.
  * [23] Hisham Othman and Shen-En Qian.  Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage.  IEEE Transactions on Geoscience and Remote Sensing, 44(2):397–408, 2006.
  * [24] Li Pang, Weizhen Gu, and Xiangyong Cao.  Trq3dnet: A 3d quasi-recurrent and transformer based network for hyperspectral image denoising.  14(18):4598, 2022.
  * [25] J. Peng, Q. Xie, Q. Zhao, Y. Wang, L. Yee, and D. Meng.  Enhanced 3dtv regularization and its applications on hsi denoising and compressed sensing.  IEEE Transactions on Image Processing, 29:7889–7903, 2020.
  * [26] Yi Peng, Deyu Meng, Zongben Xu, Chenqiang Gao, Yi Yang, and Biao Zhang.  Decomposable nonlocal tensor dictionary learning for multispectral image denoising.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2949–2956, 2014.
  * [27] Xie Qi, Zhao Qian, Deyu Meng, and Zongben Xu.  Kronecker-basis-representation based tensor sparsity and its applications to tensor recovery.  IEEE Transactions on Pattern Analysis and Machine Intelligence, PP(99):1–1, 2017.
  * [28] Le Sun, Byeungwoo Jeon, Yuhui Zheng, and Zebin Wu.  Hyperspectral image restoration using low-rank representation on spectral difference image.  IEEE Geoscience and Remote Sensing Letters, 14(7):1151–1155, 2017\.
  * [29] Prasad S Thenkabail and John G Lyon.  Hyperspectral remote sensing of vegetation.  CRC Press, 2016.
  * [30] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [31] Yao Wang, Jiangjun Peng, Qian Zhao, Yee Leung, Xi-Le Zhao, and Deyu Meng.  Hyperspectral image restoration via total variation regularized low-rank tensor decomposition.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(4):1227–1243, 2017.
  * [32] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li.  Uformer: A general u-shaped transformer for image restoration.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17683–17693, 2022.
  * [33] Kaixuan Wei and Ying Fu.  Low-rank bayesian tensor factorization for hyperspectral image denoising.  Neurocomputing, 331:412–423, 2019.
  * [34] Kaixuan Wei, Ying Fu, and Hua Huang.  3-d quasi-recurrent neural network for hyperspectral image denoising.  IEEE Transactions on Neural Networks and Learning Systems, 32(1):363–375, 2021.
  * [35] Qi Xie, Qian Zhao, Deyu Meng, Zongben Xu, Shuhang Gu, Wangmeng Zuo, and Lei Zhang.  Multispectral images denoising by intrinsic tensor sparsity regularization.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1692–1700, 2016.
  * [36] Qiangqiang Yuan, Liangpei Zhang, and Huanfeng Shen.  Hyperspectral image denoising employing a spectral–spatial adaptive total variation model.  IEEE Transactions on Geoscience and Remote Sensing, 50(10):3660–3677, 2012.
  * [37] Qiangqiang Yuan, Qiang Zhang, Jie Li, Huanfeng Shen, and Liangpei Zhang.  Hyperspectral image denoising employing a spatial–spectral deep residual convolutional neural network.  IEEE Transactions on Geoscience and Remote Sensing, 57(2):1205–1218, 2018.
  * [38] Hongyan Zhang, Wei He, Liangpei Zhang, Huanfeng Shen, and Qiangqiang Yuan.  Hyperspectral image restoration using low-rank matrix recovery.  IEEE Transactions on Geoscience and Remote Sensing, 52(8):4729–4743, 2013.
  * [39] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang.  Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising.  IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.
  * [40] Kai Zhang, Wangmeng Zuo, and Lei Zhang.  Ffdnet: Toward a fast and flexible solution for cnn-based image denoising.  IEEE Transactions on Image Processing, 27(9):4608–4622, 2018.
  * [41] Liangpei Zhang and Xin Huang.  Object-oriented subspace analysis for airborne hyperspectral remote sensing imagery.  Neurocomputing, 73(4-6):927–936, 2010.
  * [42] Tao Zhang, Ying Fu, and Cheng Li.  Hyperspectral image denoising with realistic data.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2248–2257, 2021.
  * [43] Haitao Zhao and Shaoyuan Sun.  Sparse tensor embedding based multispectral face recognition.  Neurocomputing, 133:427–436, 2014.
  * [44] XiLe Zhao, Hao Zhang, TaiXiang Jiang, Michael K Ng, and Xiong-Jun Zhang.  Fast algorithm with theoretical guarantees for constrained low-tubal-rank tensor recovery in hyperspectral images denoising.  Neurocomputing, 413:397–409, 2020.
  * [45] Zhihong Pan, G. Healey, M. Prasad, and B. Tromberg.  Face recognition in hyperspectral images.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12):1552–1560, 2003.
  * [46] Feng Zhou, Renlong Hang, Qingshan Liu, and Xiaotong Yuan.  Hyperspectral image classification using spectral-spatial lstms.  Neurocomputing, 328:39–47, 2019.
  * [47] Daniel Zoran and Yair Weiss.  From learning models of natural image patches to whole image restoration.  In Proceedings of the IEEE International Conference on Computer Vision, pages 479–486. IEEE, 2011.