# Multi-scale Sampling and Aggregation Network For High Dynamic Range Imaging

Jun Xiao, Qian Ye, Tianshan Liu, Cong Zhang, Kin-Man Lam Jun Xiao, Tianshan
Liu, Cong Zhang and Kin-Man Lam are with the Department of Electronic and
Information Engineering, the Hong Kong Polytechnic University. E-mail:
<EMAIL_ADDRESS>Ye is with Graduate School of Information
Sciences, Tohoku University. E-mail<EMAIL_ADDRESS>received July 30, 2022;

###### Abstract

High dynamic range (HDR) imaging is a fundamental problem in image processing,
which aims to generate well-exposed images, even in the presence of varying
illumination in the scenes. In recent years, multi-exposure fusion methods
have achieved remarkable results, which merge multiple low dynamic range (LDR)
images, captured with different exposures, to generate corresponding HDR
images. However, synthesizing HDR images in dynamic scenes is still
challenging and in high demand. There are two challenges in producing HDR
images: 1). Object motion between LDR images can easily cause undesirable
ghosting artifacts in the generated results. 2). Under and overexposed regions
often contain distorted image content, because of insufficient compensation
for these regions in the merging stage. In this paper, we propose a multi-
scale sampling and aggregation network for HDR imaging in dynamic scenes. To
effectively alleviate the problems caused by small and large motions, our
method implicitly aligns LDR images by sampling and aggregating high-
correspondence features in a coarse-to-fine manner. Furthermore, we propose a
densely connected network based on discrete wavelet transform for performance
improvement, which decomposes the input into several non-overlapping frequency
subbands and adaptively performs compensation in the wavelet domain.
Experiments show that our proposed method can achieve state-of-the-art
performances under diverse scenes, compared to other promising HDR imaging
methods. In addition, the HDR images generated by our method contain cleaner
and more detailed content, with fewer distortions, leading to better visual
quality.

###### Index Terms:

High Dynamic Range Imaging, Image Enhancement, Image Restoration

##  I Introduction

High dynamic range (HDR) imaging is a fundamental problem in image processing
and computational photography, which aims to generate high-quality and well-
exposed images, even in varying lighting conditions. Most modern imaging
systems, such as digital cameras, often fail to capture the full range of
natural lighting, because of inherent hardware limitations. Therefore, HDR
imaging is a dispensable technique in modern imaging systems and has high
industrial values.

At the early stage, researchers in [1, 2] attempted to design specific
hardware devices for generating high-quality HDR images, but these devices
were too expensive to be widely used in commercial products. Alternatively,
many researchers focus on the multi-exposure fusion approach, which takes a
sequence of low dynamic range (LDR) images with different exposures and merges
them to generate the corresponding HDR images based on a reference LDR image.
However, most multi-exposure fusion methods fail to produce satisfactory
results from LDR images captured in dynamic scenes [3]. Unlike static scenes,
dynamic scenes introduce two challenging issues for HDR imaging. The first one
is that intensive motion between LDR images is prone to producing ghosting
artifacts and significantly reduce the quality of the generated HDR images.
Besides that, the corrupted content in under and overexposed regions will lead
to undesirable distortions, if they are not adequately compensated in the
merging stage.

In recent years, motion-removal-based methods [4, 5, 6, 7, 8, 9, 10, 11, 12,
13] and alignment-based methods [14, 15, 16, 6, 17] have shown their
effectiveness for HDR imaging in dynamic scenes. For motion-removal-based
methods, motion regions are first detected, and then removed in the merging
stage. Therefore, object motions are not considered in the reconstruction, and
ghosting artifacts are reduced in the generated results. To accurately detect
motion regions, various methods have been proposed, including threshold-based
methods [9, 5, 6, 8], gradient-based methods [18, 19], and low-rank and
sparsity-based methods [12, 10, 11]. However, these motion-removal-based
methods suffer from two intrinsic limitations. Firstly, ghosting artifacts are
easily introduced, because of inaccurate motion detection, which seriously
degrades the quality of generated HDR images. In addition, when the input LDR
images contain large-scale motions, a large amount of pixels are removed in
the merging processing, which inevitably causes information loss and harms the
performance. A recent work [13] proposed a deep learning-based model with the
spatial attention mechanism to avoid information loss in moving regions. The
unnecessary pixels are suppressed in a soft manner before the merging stage.
However, ghosting artifacts still appear in the generated images, as shown in
[3].

Unlike motion-removal-based methods, alignment-based methods first align the
input LDR images with a reference from the input, and then merge them to
generate the corresponding HDR images. For example, Kalantari et al. [15] used
optical flow [20] to align the input LDR images and then, adopted a deep
convolutional neural network (CNN) to merge the aligned LDR images for
reconstruction. However, the illumination in LDR images may vary
significantly, so the brightness constancy condition required by optical flow
is hard to satisfy [21, 22, 23] and the resulting optical flow is inaccurate,
which further harms the quality of the generated HDR images. Instead of using
optical flow, Wu et al. [17] adopted a homography transformation to globally
register LDR images in the pre-processing stage. Then, a deep CNN model with
the U-Net shape structure is used to extract multi-scale features for
reconstruction. However, this method does not consider local information and
thus cannot effectively compensate for corrupted image content in saturated
regions. Inspired by the deformable convolution [24], Pu et al. [25] adopted a
pyramid, cascading and deformable (PCD) alignment module [26] to
hierarchically align LDR images for reconstruction. Furthermore, Liu et al.
[27] proposed a dual-branch network for HDR imaging in dynamic scenes, which
adopts the spatial attention mechanism and the PCD module in different
branches, respectively. However, deformable convolution suffers from unstable
training [28]. In addition, the receptive field of deformable convolution is
restricted by its kernel size, making it difficult to handle large-scale
motions.

In this paper, we propose a multi-scale sampling and aggregation network for
HDR imaging in dynamic scenes. Unlike previous methods, our method performs
intra-scale aggregation and inter-scale aggregation to implicitly align LDR
images in a coarse-to-fine manner. Specifically, in each scale space, intra-
scale aggregation samples neighboring features around misaligned pixels and
aggregates sampled features according to their correspondences with the
reference features. These features are of higher quality because they are more
similar to the corresponding reference features. Thus, the high-quality
correspondence features can contribute more to aggregation for alignment.
Furthermore, inter-scale aggregation progressively fuses the aggregated
features across different scale spaces. Therefore, ghosting artifacts, caused
by small and large motions in LDR images, are effectively reduced. In
addition, a densely connected network based on discrete wavelet transform is
proposed for performance improvement. The proposed dense wavelet network
decomposes the input features into several non-overlapping frequency subbands
and compensates for the image content in the wavelet domain. As shown in Fig.
1, different frequency subbands can provide different information for
reconstruction. For example, the low-frequency subband mainly contains coarse
image content, while the high-frequency subbands have rich structural
information, which is beneficial for HDR imaging. The main contributions of
this paper are summarized as follows:

  1. 1.

We propose a multi-scale sampling and aggregation network for HDR imaging in
dynamic scenes. In this method, intra-scale and inter-scale aggregation
schemes are proposed to implicitly align LDR images from coarse-to-finer
scales.

  2. 2.

We further propose a densely connected network, based on discrete wavelet
transform for performance improvement, which can effectively compensate for
corrupted content in saturated regions.

  3. 3.

Experiments show that our proposed method significantly outperforms other
promising HDR imaging methods and generates remarkable results. In addition,
the HDR images generated by our method have more structural information and
fewer distortions, leading to better and higher visual quality.

Figure 1: Illustration of an HDR image (the first row) and the corresponding
LDR image (the second row) in the wavelet domain. LL, LH, HL, and HH represent
the low-low band, low-high band, high-low band, and high-low band,
respectively. The region marked by a red box in an image is enlarged and
placed at the bottom-left corner.

##  II Related Works

###  II-A Motion Removal-based Methods

Motion detection is a key point in motion removal-based methods. Bogoni [4]
applied a non-parametric model to detect under and overexposed regions before
merging LDR images. In [9], misaligned regions are rejected according to the
error map of aligned pixels. Khan et al. [5] proposed an method to detect
moving regions implicitly and iteratively fuses the LDR images, so undesirable
artifacts were reduced. Jacobs et al. [6] assumed that moving objects cause
large variations of pixel intensity in overexposed regions. Based on this
assumption, they proposed a motion-detection method based on measuring
variance [7]. Gallo et al. [8] assumed that irradiance values of the
background are linear to the exposure time, and proposed a method for
detecting ghosting regions based on the irradiance deviation. All these
methods utilize local information in regular patch grids. Instead, Raman and
Chaudhui [29] proposed to group pixels in images into superpixels and a
bottom-up framework to detect the ghosting regions. This method can
effectively handle irregular objects in saturated regions. Heo et al. [30]
proposed a coarse-to-fine pipeline to detect motion regions, which first
applies a joint probability model to different images to roughly detect motion
regions and then refines these regions with graph cut. Rather than detecting
motion regions in the spatial domain, Zhang and Cham [18] proposed a motion-
detection method based on the assumption that gradient values change
significantly in motion regions. However, this method struggles to obtain
gradient information for saturated regions, which inevitably harms the
performance. Lee et al. [12] formulated the motion-detection problem as a
rank-minimization problem, which considers misalignment errors, moving
objects, noise, and nonlinear artifacts as sparse outliers. Oh et al. [10]
further extended the low-rank model in [12] by introducing user control for
moving objects under different exposure settings, which obtained better
results. Yan et al. [11] considered that moving objects in LDR images have the
sparsity property, and thus a regularized sparsity-based model was proposed
for motion detection. Yan et al. [13] proposed a deep CNN-based model, where
the spatial attention mechanism is used to suppress misaligned regions at the
feature level. However, motion removal-based methods have several inherent
limitations. 1). When there are large-scale motions between the LDR images, a
large amount of pixels are excluded in the merging process, resulting in
distorted content. 2). The quality of the generated HDR images is very
sensitive to detection accuracy and is not robust to varying illumination.

###  II-B Alignment-based Methods

Alignment-based methods can be divided into three categories: flow-based
methods, deformable-based methods, and patch-based methods. Flow-based methods
mainly adopt optical flow and its variants to align LDR images and then merge
the aligned LDR images to generate the corresponding HDR images. Tomaszewska
and Mantiuk [31] used a homography transform to globally register LDR images
before merging. The registered LDR images are fused for HDR imaging by the
radiometric calibration method [32]. As illumination varies significantly in
LDR images, the brightness constancy condition is easily violated in the
spatial domain. Zimmer et al. [33] proposed to estimate optical flow in the
gradient domain, which guarantees the consistency of gradients. However, this
method treats flow estimation and HDR image generation separately, which may
lead to sub-optimal performance. Hafner et al. [16] proposed a method for
jointly estimating optical flow and reconstructing HDR images, so the
performance is further improved. By considering corrupted contents in
saturated regions, Hu et al. [34] proposed a method to merge LDR images in the
luminance domain, effectively reducing the intervention in saturated regions.
Inspired by the unprecedented success achieved by deep learning models in many
vision tasks, Kalantari and Ramamoorthi [15] proposed a deep learning-based
network for HDR imaging, called DeepHDR. Their method first aligns the LDR
images using the optical flow [20] and then forwards the aligned LDR images to
a deep CNN network for fusion. Unlike the method [35] using dense motion
fields, Wu et al. [17] propose a deep translation-based method that globally
registers LDR images with a homography function and then extracts multi-scale
features for reconstruction. However, flow-based methods rely on the accuracy
of optical-flow estimation and are very sensitive to varying illumination, so
these methods are prone to generating undesirable ghosting artifacts in the
generated HDR images. Inspired by deformable convolution [24], many
researchers have paid attention to deep deformable-based methods for HDR
imaging. For example, Pu et al. [25] proposed to progressively align LDR
images and reconstruct HDR images with the PCD module [26] and achieved
promising results. Liu et al. [27] further improved the method [25] by
introducing a dual-branch structure which is composed of the PCD module and
the spatial attention mechanism. The output of each branch contains different
features of the LDR images, so the performance can be improved. However,
deformable-based methods cannot effectively handle large-scale motions,
because the receptive field of the deformable convolutions is restricted by
its kernel size. In addition, deformable convolution suffers from unstable
training [28]. Instead of explicit alignment using optical flow or deformable
convolution, Sen et al. [36] proposed a patch-based method to reconstruct HDR
images, which uses a patch-match algorithm to find similar patches for
reconstruction. Yan et al. [37] used the non-local attention mechanism in a
deep CNN model, which captures pixel-wise correspondence for reconstruction.
Chen et al. [38] proposed a hybrid method for HDR imaging, which first reduces
the effect caused by motions with the spatial attention mechanism and then
aggregates the most similar patch in a coarse-to-finer manner. Patch-based
methods implicitly align LDR images, but these methods are too computationally
expensive. Motivated by RAFT [23], Ye et al. [3] proposed to progressively
select features from intermediate layers to reconstruct HDR images, which
implicitly aligns LDR images in the feature space. To compensate for the
missing content in under and overexposed regions, Niu et al. [39] proposed a
generative adversarial network (GAN)-based model, but GAN models suffer from
model collapse and unstable training [40, 41, 42], so this method is prone to
artifacts and harms the performance.

Compared with flow-based methods and deformable-based methods, our proposed
method has several advantages: 1). Flow-based methods use optical flow to
globally sample one position for alignment, but our proposed method samples
multiple neighboring positions around the misaligned pixels for alignment in a
coarse-to-fine manner. 2). Our proposed method adaptively aggregates the
sampled features which are more similar to the reference features, so ghosting
artifacts can be effectively reduced. 3). The number of sampled features in
our proposed method is not limited by the kernel size, which provides greater
flexibility than deformable-based methods.

##  III The Proposed Methods

In this paper, we propose a multi-scale sampling and aggregation network for
HDR imaging in dynamic scenes. As shown in Fig. 2, the proposed model consists
of two important sub-networks, i.e., the multi-scale correspondence alignment
sub-network and the dense wavelet sub-network, for alignment and
reconstruction, respectively. In this section, we will first describe the pre-
processing stage and the multi-scale correspondence alignment sub-network that
includes the encoder and the multi-scale sampling and aggregation module
(Multi-scale SAM). Then, we will introduce the proposed dense wavelet sub-
network. Finally, the loss function used to train our model will be described
in detail.

Figure 2: The overall structure of the proposed multi-scale sampling and
aggregation network, which consists of a multi-scale correspondence alignment
sub-network and a dense wavelet sub-network. $F_{j}^{(i)}$ denotes the
features extracted from $X_{i}$ in the $j$-th scale space, where $i,j=1,2,3$.
Figure 3: The overall structure of the encoder, multi-scale SAM, and Group
WaveNet, respectively. In Encoder, $X_{L}$ denotes the input 6-channel tensor
and $F_{i}$ represents the extracted features of different scales, for
$i=1,2,3$. In (b), $\hat{F}_{i}$ and $F_{i}$ denote the reference and non-
reference features, respectively, where $i=1,2,3$.

###  III-A The Pre-processing Stage

Given a sequence of LDR images
$\\{I_{L}^{(1)},I_{L}^{(2)},\cdots,I_{L}^{{(N)}}\\}$ with $N$ different
exposure times, the target of HDR imaging is to reconstruct a clean and ghost-
free HDR image $I_{H}^{(r)}$, which is aligned with the reference image
$I_{L}^{(r)}$, where $r\in\\{1,2,\cdots,N\\}$. Without loss of generality, we
consider three LDR images with low, medium, and high exposure conditions in
our method, denoted as $\\{I_{L}^{(1)},I_{L}^{(2)},I_{L}^{(3)}\\}$, and take
the LDR image $I_{L}^{(2)}$, which is captured with medium exposure time, as
the reference. Since HDR imaging involves manipulation of illumination values,
we first linearize the input LDR images with gamma correction. The linearized
LDR images are computed as follows:

$\hat{I}^{(i)}_{L}=\frac{\left(I_{L}^{(i)}\right)^{\gamma}}{t_{i}},\quad
i=1,2,3,$ (1)

where $\hat{I}^{(i)}_{L}$ is the linearized image of the $i$-th input LDR
image $I_{L}^{(i)}$, $\gamma>1$ is a hyper-parameter for gamma correction, and
$t_{i}$ denotes the exposure time of the LDR image $I_{L}^{(i)}$. In our
method, we follow the suggestions in [15] and empirically set $\gamma$ to
$2.2$. Then, the LDR images $I_{L}^{(i)}$ and their corresponding linearized
images $\hat{I}^{(i)}_{L}$ are concatenated to form a 6-channel tensor
$X_{i}=[I_{L}^{(i)}$, $\hat{I}^{(i)}_{L}]$, for $i=1,2,3$, which is the input
of our proposed model.

Given a number of HDR-LDR image pairs, our model attempts to learn a highly
nonlinear illumination mapping from the LDR domain to the HDR domain. The
estimated HDR image $\hat{I}_{H}$ is generated as follows:

$\hat{I}_{H}=f_{\theta}(X_{1},X_{2},X_{3}),$ (2)

where $f_{\theta}(\cdot)$ denotes the proposed model with parameters $\theta$.

###  III-B The Multi-scale Correspondence Alignment Sub-network

The multi-scale correspondence alignment sub-network aims to implicitly align
the input LDR images in a coarse-to-finer manner, which consists of two key
components, namely the encoder and multi-scale sampling aggregation module
(Multi-scale SAM). The overall structure of the multi-scale correspondence
alignment sub-network is shown in Fig. 2.

####  III-B1 Encoder

The encoder takes $\\{X_{1},X_{2},X_{3}\\}$ as input and extracts
corresponding multi-scale features. As shown in Fig. 3, the encoder in our
method contains three scale spaces. In each scale space, two convolutional
layers with a kernel size of $3\times 3$ are used to generate features. The
downsampling operator is a convolutional layer with a stride of $2$. In our
method, the kernel weights of the encoders are shared among the three inputs,
so the required computational complexity and model complexity are reduced. The
generated multi-scale features are denoted as
$\bm{F}_{i}=\\{F_{i}^{(1)},\cdots,F_{i}^{(S)}\\}$, for $i=1,2,3$, where $S$ is
the number of scales. Then, these multi-scale features are forwarded to multi-
scale SAM modules for progressive alignment in a coarse-to-fine manner.

####  III-B2 Multi-Scale Sampling Aggregation Module (Multi-scale SAM)

As illustrated in Fig. 3(b), the Multi-scale SAM module contains two important
parts, namely the sampling aggregation module (SAM) and the inter-scale
aggregation module (IAM).

Figure 4: The overall structure of the sampling and aggregation module (SAM).
$F_{s}$ and $\hat{F}_{s}$ denote the non-reference and reference features of
the $s$-th scale space. $P_{s}$ and $P_{s}$ are the sets containing sampling
positions. $\bm{p}_{s}^{(i)}$ denotes the $i$-th sampling position in the
$s$-th scale space, where $i=1,\cdots N$ and $N$ is the number of samples.
$\bar{F}^{(i)}_{s}$ denotes the $i$-th sample feature in the $s$-th scale
space. $\\{w_{i},\cdots,w_{N}\\}$ are $N$ aggregation weights.
$F^{(\text{out}})_{s}$ is the output feature associated with $F_{s}$.

Sampling and Aggregation Module (SAM). For each scale space, SAM aligns pixels
in the non-reference features by sampling its neighboring features and then
adaptively aggregating sampled features according to their correspondence with
the reference features. As illustrated in Fig. 4, the proposed SAM module
consists of three parts: sample generator, correspondence computation, and
sample aggregation. Given two input features at the $s$-th scale space,
denoted as $F_{s}$ and $\hat{F}_{s}$, which are the features extracted from
the non-reference image and the reference image, respectively, the sample
generator takes these two features, as well as the sampling map $P_{s-1}$ from
the previous scale space as input and generates the sampling map of the $s$-th
scale space $P_{s}$. The sampling map $P_{s-1}$ contains the coarse sampling
information of the $(s-1)$-th scale space, so it can be used as the prior
information to guide the generation of the sampling map of the $s$-th scale
space. In our method, the sample generator is a small convolutional network
with four convolutional layers, and the sampling map $P_{s}$ is computed as
follows:

$P_{s}=g(F_{s},\hat{F}_{s},\hat{P}_{s-1}),$ (3)

where $g(\cdot)$ denotes the sampling generator and
$P_{s}=[\bm{p}_{s}^{(1)},\cdots,\bm{p}_{s}^{(N)}]$ represents the $N$ sampling
matrices for the $s$-th scale space, where the sampling matrix
$\bm{p}_{s}^{(i)}$ is $H\times W\times 2$ dimensional, for $i=1,\cdots,N$. $H$
and $W$ represent the height and width of the input features, respectively.
These sampling matrices provide the locations of corresponding sampled
features for each pixel position. Specifically, for the $s$-th scale space,
the element of the $i$-th sampling matrix at the position $(m,n)$ is a two-
dimensional vector, denoted as
$\bm{p}^{(i)}_{s,m,n}=[p_{s,m,n}^{(i)}(x),p_{s,m,n}^{(i)}(y)]$. This vector
contains the displacement information in the horizontal and vertical
directions between the sampled feature and the reference feature. Based on the
sampling matrices, the sampled features of the pixel position $(m,n)$ are
computed as follows:

$\bar{F}^{(i)}_{s}(m,n)=F_{s}(m+p_{s,m,n}^{(i)}(x),n+p_{s,m,n}^{(i)}(y)),$ (4)

where $\bar{F}^{(i)}_{s}(m,n)$ denotes the $i$-th sampled feature
corresponding to the misaligned pixel at the position $(m,n)$ of the $s$-th
scale space, for $i=1,\cdots,N$. For fractional positions (i.e.,
$m+p_{s,m,n}^{(i)}(x)\notin\mathbb{Z}$ or
$n+p_{s,m,n}^{(i)}(y)\notin\mathbb{Z}$), bilinear interpolation is used to
compute the values. In the $s$-th scale space, SAM samples neighboring
features for all misaligned pixels according to the location information
provided by the sampling map $P_{s}$. However, not all the sampled features
are beneficial for reconstructing HDR images, especially in saturated regions.
Therefore, we propose a method to adaptively aggregate the sampled features.
In particular, we first calculate the correspondence between the sampled
features and the reference features at each pixel position, which are based on
the dot product between the sampled features and the reference features. The
sampled features having higher correspondence means that they are more similar
to the reference features, while the sampled features with lower
correspondences are less similar to the reference features. Then, we compute
the correspondence weights for aggregation. For the position $(m,n)$, the
weight is computed as follows:

$w^{(i)}_{m,n}=\frac{\exp(\bar{F}^{(i)}_{s}(m,n)\cdot\hat{F}_{s}(m,n))}{\sum_{j=1}^{N}\exp(\bar{F}^{(j)}_{s}(m,n)\cdot\hat{F}_{s}(m,n))},$
(5)

where $\cdot$ denotes the dot product and $0\leq w^{(i)}_{m,n}\leq 1$, for
$i=1,\cdots,N$. Next, the non-reference features are aligned by aggregating
the sampled features according to the computed weights, and the corresponding
aligned feature $\hat{F}_{s}(m,n)$ is computed as follows:

$\hat{F}_{s}(m,n)=w^{(1)}_{m,n}\bar{F}^{(1)}_{s}(m,n)+\cdots+w^{(N)}_{m,n}\bar{F}^{(N)}_{s}(m,n).$
(6)

As observed, aligned features are generated by a linear combination of the
sampled features. For those sampled features having high correspondence with
the reference feature, the corresponding weights will be larger, so these
features will have more contribution to the aggregation. For those sampled
features with lower correspondence, the associated weights will be smaller.
This can effectively alleviate undesirable effects caused by dissimilar
features in aggregation and avoid ghosting artifacts.

Inter-scale Aggregation Module (IAM). The input LDR images are captured in
dynamic scenes, so they unavoidably contain small and large-scale motions. To
address this issue, we propose the inter-scale aggregation module to
progressively aggregate the output features of different scales. The overall
structure of IAM is shown in Fig. 5. Given two output features $\hat{F}_{s}$
and $\hat{F}^{\prime}_{s-1}$ from the $s$-th and $(s-1)$-st scales, the
average pooling and the max pooling are applied to these two features to
extract local features. Then, the local features are concatenated along the
channel dimension. A convolutional layer is applied to the concatenated
features to generate a fusion mask $M$. Each element of the fusion mask $M$ is
in $[0,1]$. The two input features from adjacent scales are adaptively
aggregated based on this fusion mask to produce the corresponding output
features $F_{s}^{(\text{out})}$, which are computed as follows:

$F_{s}^{(\text{out})}=(1-M)\odot F_{s}+M\odot F^{\prime}_{s-1},$ (7)

where $\odot$ denotes the Hadamard product. The two input features are
spatially fused by linear combination based on the computed fusion mask $M$.
The coarse-scale features contain more contextual information, while the
finer-scale features have more detailed information. The fusion mask performs
feature selection between the two input features from adjacent scales in a
soft manner. Benefiting from this multi-scale structure, small and large-scale
motions in LDR images are aligned in a coarse-to-finer manner.

Figure 5: The overall structure of the inter-scale aggregation module. $F_{s}$
and $F^{\prime}_{s-1}$ denote the input from the $s$-th and $(s-1)$-th scale
spaces, respectively. “Avg” and “ Max” stand for average pooling and max
pooling, respectively. $F_{s}^{\text{out}}$ is the output feature of the
$s$-th scale space.

###  III-C Dense Wavelet Sub-network

####  III-C1 Network Structure

As illustrated in Fig. 2, the dense wavelet sub-network is responsible for
merging aligned features and generating corresponding HDR images.
Specifically, the dense wavelet sub-network first concatenates the aligned
features (i.e., $F^{1\rightarrow 2}_{1}$ and $F^{3\rightarrow 2}_{1}$) and the
reference feature $F^{2}_{1}$ along the channel dimension, where
$F^{i\rightarrow 2}_{1}$ denotes the features generated by aligning
$F^{i}_{1}$ with the reference feature $F^{2}_{1}$, where $i=1$ or $3$. Then,
a convolutional layer with a kernel size of $3\times 3$ is applied to compress
and fuse the concatenated features. The resulting fused feature is forwarded
to three group-wavelet modules (Group WaveNets) for feature extraction in the
wavelet domain. Each Group WaveNet is composed of two small, cascaded Dense
wavelet networks (D-WaveNets), as shown in Fig. 3(c), which play a significant
role in feature extraction and HDR image reconstruction. In our method, the
output of the second D-WaveNet is concatenated with the input, followed by a
convolutional layer for feature fusion. Furthermore, the outputs of the three
Group WaveNets are concatenated, followed by two convolutional layers for
fusion. Therefore, the proposed dense wavelet sub-network has a structure with
the local-global dense connections. This can significantly increase the
information interaction and fully utilize the features from different layers.
At the output of the network, the estimated HDR $\hat{I}_{\text{H}}$ images
are computed as follows:

$\hat{I}_{\text{H}}=F^{\prime}+\text{Conv}(F^{2}_{1}),$ (8)

where $F^{\prime}$ denotes the fused feature generated by the two
convolutional layers and $\text{Conv}(\cdot)$ represents the convolutional
layer with a kernel size of one.

####  III-C2 Dense Wavelet Network (D-WaveNet)

The D-WaveNet, based on discrete wavelet transform (DWT), decomposes input
features into several non-overlapping frequency subbands for feature
extraction. The overall structure of D-WaveNet is illustrated in Fig. 6. Given
an input feature map denoted as $F$, DWT is applied to $F$ along the channel
dimension, and then four frequency subbands are obtained, i.e., one low-
frequency subband and three high-frequency subbands. These four frequency
subbands are computed as follows:

$W_{LL}=\mathcal{L}F\mathcal{L}^{T},W_{LH}=\mathcal{H}F\mathcal{L}^{T}$ (9)
$W_{HL}=\mathcal{L}F\mathcal{H}^{T},W_{HH}=\mathcal{H}F\mathcal{H}^{T}$ (10)

where $W_{LL}$ denotes the low-frequency subband and
$\\{W_{LH},W_{HL},W_{HH}\\}$ represent the high-frequency sub-bands in the
horizontal, vertical, and diagonal directions, respectively. $\mathcal{L}$
denotes the matrix containing all the low-frequency filters
$\\{\ell_{i}\\}_{i\in\mathbb{Z}}$ and $\mathcal{H}$ is the matrix containing
all the high-frequency filters $\\{h_{i}\\}_{i\in\mathcal{Z}}$ for
decomposition, where

$\mathcal{L}=\begin{pmatrix}\cdots&\cdots&\cdots&&&&\\\
\cdots&\ell_{-1}&\ell_{0}&\ell_{1}&\cdots&\cdots&\\\
&&\cdots&\ell_{-1}&\ell_{0}&\ell_{1}&\cdots&\\\
&&&&&&\cdots&\cdots\end{pmatrix},$ (11)

and

$\mathcal{H}=\begin{pmatrix}\cdots&\cdots&\cdots&&&&\\\
\cdots&h_{-1}&h_{0}&h_{1}&\cdots&\cdots&\\\
&&\cdots&h_{-1}&h_{0}&h_{1}&\cdots&\\\ &&&&&&\cdots&\cdots\end{pmatrix}.$ (12)

In our method, Haar wavelets are adopted for feature decomposition, because
Haar wavelets have orthogonal and biorthogonal properties. These two
properties guarantee that the input features can be perfectly reconstructed
without concerns about introducing distortions. After decomposition, all
frequency subbands are concatenated along the channel dimension and forwarded
to three dilated convolutional layers with dense connections for feature
extraction over the wavelet domain. The dilated rate of these three
convolutional layers are set to 1, 2, and 3, respectively, in our method.

At the output of D-WaveNet, the extracted features are transferred from the
wavelet domain to the spatial domain with the inverse DWT (IDWT), which is
computed as follows:

$\hat{F}=\mathcal{L}^{T}\hat{W}_{LL}\mathcal{L}+\mathcal{H}^{T}\hat{W}_{LH}\mathcal{L}+\mathcal{L}^{T}\hat{W}_{HL}\mathcal{H}+\mathcal{H}^{T}\hat{W}_{HH}\mathcal{H},$
(13)

where $\\{\hat{W}_{LL},\hat{W}_{LH},\hat{W}_{HL},\hat{W}_{HH}\\}$ denote four
output wavelet subbands, and $\hat{F}$ is the corresponding output feature in
the spatial domain. Then, we use a residual connection and the channel
attention mechanism [43] to generate the output feature $F_{\text{out}}$ as
follows:

$F_{\text{out}}=\text{CA}(\hat{F})+F,$ (14)

where $\text{CA}(\cdot)$ is the channel attention mechanism, which scales the
features along the channel dimension.

Figure 6: The overall structure of D-WaveNet. The dilated rates of the three
dilated convolutional layers are set to 1, 2, and 3, respectively.

###  III-D Loss Function

Since HDR images are usually displayed on the screen after tone mapping, we
calculate the loss in tone-mapped images. Although many effective tone-mapping
methods were proposed in the past decades, they are not differentiable and
cannot be used as a loss function for training deep neural networks. In our
method, we apply the $\mu$-law function to compress the output of our network,
because the $\mu$-law function is a widely used range compressor in audio
processing and is differentiable. Specifically, given an image $I$ in the
linear domain, the corresponding tone-mapped image is computed as follows:

$T(I)=\frac{\log(1+\mu I)}{\log(1+\mu)},$ (15)

where $\mu$ denotes a hyper-parameter defining the extent of compression and
is set to $5000$ in our method. We use the $L_{1}$ norm to compute the
distance between the output $\hat{I}_{H}$ and the ground-truth image $I_{H}$
after tone mapping. Formally, the loss function is defined as follows:

$L(\hat{I}_{H},I_{H})=\Arrowvert T(\hat{I}_{H})-T(I_{H})\Arrowvert_{1}.$ (16)

In our method, the range of the output is restricted to $[0,1]$ by using the
sigmoid function.

##  IV Experiments and Analysis

###  IV-A Experiment Settings

####  IV-A1 Datasets

In the experiments, a public dataset [15], called the Kalantari dataset, was
used to train our model, which has a total of 90 samples. Each sample contains
three LDR images and a ground-truth HDR image, with exposure basis of
$\\{-2,0+2\\}$ or $\\{-3,0,+3\\}$. All images in this dataset are captured in
static and dynamic scenes. Following previous works [15, 17, 11, 38], we used
75 samples for training and 15 samples for testing. Besides that, we
additionally adopted two challenging datasets, namely the Sen dataset [36] and
the Tursun dataset [44], to evaluate the generalization of the proposed model.
However, these two datasets do not provide ground-truth images for
quantitative comparison.

####  IV-A2 Evaluation metrics

HDR images are displayed on screens through tone mapping, so we evaluate the
peak signal-to-noise ratio (PSNR) and the structural similarity index measure
(SSIM) on the tone-mapped images, which are denoted as PSNR-$\mu$ and
SSIM-$\mu$, respectively. For the sake of completeness, we also include the
PSNR and SSIM results of the generated images in the linear domain, denoted as
PSNR-L and SSIM-L, respectively. Furthermore, we adopted the HDR-VDP-2 metric
in the evaluation. This metric is specifically designed to assess the quality
of HDR images [45]. If the generated HDR images have higher HDR-VDP-2 score,
the quality of the generated HDR images is better.

####  IV-A3 Implementation details

Due to the large size of the training images, we divided the input images into
image patches of the size $200\times 200$ pixels for training, for training.
All training samples are randomly flipped vertically or horizontally, and
rotated by $90^{\circ}$, $180^{\circ}$ or $270^{\circ}$ for data augmentation.
We adopted the AdamW algorithm, with $\beta_{1}=0.9$ and $\beta_{2}=0.999$, to
adaptively update the parameters of the proposed model. In the training stage,
the batch size is set to $16$ and the initial learning rate is set to
$2.0\times 10^{-4}$. The cosine annealing strategy is utilized to adaptively
reduce the learning rate to $1.0\times 10^{-6}$ during training. We used the
Pytorch framework to implement our proposed model with two Navidia 3090 GPUs,
and the total number of epochs used in training is 300. It takes about two
days to complete the training.

TABLE I: The average PSNR-$\mu$, SSIM-$\mu$, PSNR-L, SSIM-L, and HDR-VDP-2 of different methods on the Kalantari dataset. The best results are highlighted in bold. Methods | Sen [36] | DeepHDR [15] | Wu [17] | AHDRNet [13] | PANet [25] | PSFNet [3] | MSANet (Ours)
---|---|---|---|---|---|---|---
PSNR-$\mu$ | 40.9689 | 42.7177 | 41.9977 | 43.7013 | 43.8487 | 44.0613 | 44.3847
PSNR-L | 38.3425 | 41.2200 | 41.6593 | 41.1782 | 41.6452 | 41.5736 | 42.1704
SSIM-$\mu$ | 0.9859 | 0.9889 | 0.9878 | 0.9905 | 0.9906 | 0.9907 | 0.9912
SSIM-L | 0.9764 | 0.9829 | 0.9860 | 0.9857 | 0.9870 | 0.9867 | 0.9887
HDR-VDP-2 | 60.3463 | 61.3139 | 61.7981 | 62.0521 | 62.5495 | 63.1550 | 64.5038

(a)

(b)

Figure 7: Visual comparison of different HDR imaging methods. “-2”, “0”, and
“+2” denote three different exposure settings. In the experiment, “0”
represents the reference image.

###  IV-B Experiments on the Kalantari Dataset

In this experiment, our proposed model is denoted as MSANet. We compared
MSANet with other state-of-the-art HDR methods, including Sen [36], DeepHDR
[15], Wu [17], AHDRNet [13], PANet [25], and PSFNet [3]. We used the open-
source code provided by the authors for testing, except for PANet. The code of
PANet is not publicly available, so we directly refer to the results provided
in the original paper. The average PSNR-$\mu$, PSNR-L, SSIM-$\mu$, SSIM-L, and
HDR-VDP-2 of different HDR methods on the Kalantari dataset are tabulated in
Table I. The best results are highlighted in bold. As observed, our proposed
MSANet significantly outperforms all compared HDR methods in terms of all
evaluation metrics. Compared with the second-best method (i.e., PSFNet),
MASNet achieved 0.3dB and 1.3488 higher, in terms of the PSNR-$\mu$ and HDR-
VDP-2, respectively. These results reveal that our method can generate high-
quality HDR images in the tone-mapped and linear domains than other HDR
methods. In addition, we selected two images from the Kalantari Dataset for
visual comparison, and the results generated by different HDR methods are
illustrated in Fig. 7. Two local regions marked by red and blue rectangles,
which contain moving objects, are enlarged for better comparison. We can find
that the results generated by DeepHDR, Wu, AHDRNet, and PSFNet contain
undesirable ghosting artifacts and distorted contents in motion and
overexposed regions. Although Sen can restore the corrupted content in
saturated regions, it fails to handle motion regions and produces severe
ghosting artifacts. These results show that the compared methods struggle to
overcome object motions and compensate for corrupted content in saturated
regions simultaneously. Compared with them, our method can effectively
alleviate the ghosting artifacts caused by object motions and adequately
compensate for the corrupted regions in under and overexposed regions. As a
result, the HDR images generated by our method contain less distorted image
content, leading to the best visual quality.

(a)

(b)

Figure 8: Visual comparison of different HDR imaging methods. The image
illustrated in (a) is selected from the Sen dataset, and the image shown in
(b) is from the Tursun dataset.

###  IV-C Experiments on the Sen and Tursun Datasets

We further compare our proposed model with other HDR imaging methods in two
challenging datasets, i.e., the Sen and Tursun datasets. Compared to the
images in the Kalantari dataset, the LDR images in these two datasets were
captured at different exposure settings, and the reference images are
relatively dark. This makes the reconstruction of HDR images more challenging.
Therefore, we used these two datasets to evaluate the generalization ability
of different HDR models. Since these two datasets do not provide ground-truth
images for quantitative comparison, we compare the performance of different
HDR methods through visual results. For each dataset, we selected one image
for visual comparison. The images generated by different HDR imaging methods
are illustrated in Fig. 8. For better comparison, two local regions marked by
red and blue rectangles are cropped and then enlarged. As observed, the HDR
images generated by Sen and DeepHDR are noisy and contain unpleasant artifacts
caused by object motion. Although AHDRNet and PSFNet can produce satisfactory
results in Fig. 8(a), severe ghosting artifacts can be easily observed in the
underexposed regions which contain object motion, as shown in Fig. 8(b). In
addition, PSFNet cannot preserve the shape of objects, so the generated images
lose details. This means that AHDRNet and PSFNet are sensitive to exposure
conditions and suffer from poor generalization. Compared with these HDR
methods, our method can effectively reduce ghosting artifacts caused by object
motion and has a better ability to preserve structural information in the
generated HDR images. Therefore, HDR images generated by our proposed model
are much cleaner with rich, detailed information and fewer distortions, and
have the best perceptual quality.

###  IV-D Model Analysis

####  IV-D1 The effect of the number of sampled features

Our method implicitly aligns the input LDR images in the feature space by
sampling neighboring features around the misaligned pixels. Therefore, we
conducted an experiment to investigate the effect of the number of sampled
features on the performance. In this experiment, we evaluate the performance
of our model with different numbers of sampled features in the Kalanrari
dataset. The number of sampling features $N$ is set from $2$ to $10$. Fig. 9
shows the PSNR-$\mu$ of the models with different numbers of sampling
features. As observed, the proposed model can achieve the best performance
when the number of sampled features $N$ is set to $3$. When the number of
sampled features is increased from $3$ to $10$, PSNR-$\mu$ decreases by
0.15dB, because some inaccurate sampled features are involved in the
aggregation, which harms the performance.

Figure 9: The PSNR-$\mu$ of the model with different numbers of sampled
features.

####  IV-D2 Modules in the Multi-scale Correspondence Alignment Sub-network

The multi-scale correspondence alignment sub-network is a key component in our
proposed model, which implicitly aligns LDR images in a coarse-to-fine manner.
Therefore, we conducted an experiment to evaluate the performance of the
alignment sub-network with different settings. Specifically, we will evaluate
the performance of our proposed model with and without the use of the multi-
scale structure. In addition, we will compare the performance of our model
with different fusion methods across different scales, i.e., addition,
concatenation, and the proposed inter-scale aggregation scheme. Other
configurations of the models are the same. The average PSNR-$\mu$, SSIM-$\mu$,
and HDR-VDP-2 of our model with different settings on the Kalanrari dataset
are shown in Table II. The best performance is highlighted in bold. As
observed, the multi-scale structure with our proposed inter-scale aggregation
scheme can significantly improve the performance by 0.55dB in terms of
PSNR-$\mu$. Compared with other fusion methods (i.e., addition and
concatenation), the proposed inter-scale aggregation scheme achieves the best
performance. This is because the proposed fusion scheme adaptively suppresses
unnecessary pixels in the aggregation across scales. In addition, we show the
images generated by our model with different settings in Fig. 10 for visual
comparison. The models with the multi-scale structure have a better ability to
generate images with rich, detailed information (e.g., tree branches) than
that without the multi-scale structure. These results reveal that the multi-
scale structure can effectively capture local details for better
reconstruction. For the models using addition and concatenation fusions, the
generated images suffer from color deviations. In contrast, our proposed
inter-scale aggregation scheme can significantly reduce these distortions and
produce images of better quality.

TABLE II: The average PSNR-$\mu$, SSIM-$\mu$, and HDR-VDP-2(VDP-2) of our proposed model (MSANet) with different settings on the Kalanrari dataset. “MS” represents the multi-scale structure. “A”, “C”, and “M” denotes the addition, concatenation, and our proposed masking method in inter-scale aggregation. “✔” represents the corresponding method used in MSANet. “✗” denotes the model without using the method. Models | MS | Fusion | PSNR-$\mu$ | SSIM-$\mu$ | VDP-2
---|---|---|---|---|---
A | C | M
MSANet | ✗ | ✗ | ✗ | ✗ | 44.0933 | 0.9910 | 63.6499
MSANet | ✔ | ✔ | ✗ | ✗ | 44.2538 | 0.9912 | 64.3989
MSANet | ✔ | ✗ | ✔ | ✗ | 44.1815 | 0.9911 | 64.1025
MSANet | ✔ | ✗ | ✗ | ✔ | 44.3847 | 0.9912 | 64.5038
Figure 10: The generated HDR images of our method (i.e., MASNet) with
different settings. “w/o MS” represents the model without using the multi-
scale structure. “MS+C”,“MS+A”, and “MS+M” represent that the model adopts
concatenation, addition, and the proposed progressive masking method for
fusion, respectively.

####  IV-D3 Study on the Dense Wavelet Sub-network

Our proposed dense wavelet sub-network is based on the DWT, which decomposes
the input features into several non-overlapping frequency subbands for feature
extraction and reconstruction. In addition, we utilize the group strategy and
densely concatenate the outputs of the three Group WaveNets, so that our
method can fully utilize the information from intermediate layers. Therefore,
we conduct an experiment to compare the performance of our method with and
without the use of DWT. For the model without using DWT, it simply performs
feature extraction over the spatial domain. Other configurations are the same
as the model with DWT. Besides that, we also study the performance of our
model using different numbers of groups. In our method, the total number of
groups is six, so we set the number of groups $G$ to $1,2$, and $3$,
respectively, in this experiment. The average PSNR-$\mu$, SSIM-$\mu$, and HDR-
VDP-2 of our model with different settings on the Kalanrari dataset are
tabulated in Table III. As observed, our model with DWT can achieve 0.13dB
higher than the model without DWT in terms of PSNR-$\mu$. This result shows
that extracting features in the wavelet domain is more effective, because
different frequency subbands provide different information for HDR image
reconstruction. In addition, when the number of groups is increased, the
performance of our model improves significantly, because having more groups
can increase information interaction of different layers.

TABLE III: The average PSNR-$\mu$, SSIM-$\mu$, and HDR-VDP-2 of the models with different settings on the Kalanrari dataset. The best results of different settings are highlighted in bold. | PSNR-$\mu$ | SSIM-$\mu$ | HDR-VDP-2
---|---|---|---
Domain | w DWT | 44.3847 | 0.9912 | 64.5038
w/o DWT | 44.1566 | 0.9910 | 64.0267
Group | $G=1$ | 44.1203 | 0.9911 | 63.8058
$G=2$ | 44.2136 | 0.9910 | 64.3134
$G=3$ | 44.3847 | 0.9912 | 64.5038

####  IV-D4 Study on Different Types of Wavelets

Our proposed HDR imaging method is based on DWT for HDR imaging, so we also
investigate the performance of our model with different types of wavelets.
Specifically, we compare the performance of our model with Haar wavelets, two
symmetry wavelets, and two biorthogonal wavelets. These three wavelets have
different properties. For example, Haar wavelets are asymmetric, but they are
orthogonal and biorthogonal. In the experiment, the other configurations are
the same. The average PSNR-$\mu$ and SSIM-$\mu$ of our model with different
types of wavelets on the Kalanrari dataset are tabulated in Table IV. As
observed, different wavelets used in our model will significantly affect the
performance. The performances of the model with symmetry wavelets is lower
than that with Haar and biorthogonal wavelets. Compared with biorthogonal
wavelets, the model with Haar wavelets can achieve slightly better performance
because Haar wavelets are orthogonal. These results show that orthogonal and
biorthogonal wavelets are more beneficial for reconstruction.

TABLE IV: The average PSNR-$\mu$ and SSIM-$\mu$ of our model with different wavelets on the Kalanrari dataset. The best results are highlighted in bold. | Haar | Symmetry | Biorthogonal
---|---|---|---
sym2 | sym5 | bio1.1 | bio3.5
PSNR-$\mu$ | 44.38 | 44.29 | 44.08 | 44.35 | 44.18
SSIM-$\mu$ | 0.9912 | 0.9913 | 0.9911 | 0.9912 | 0.9912

##  V Conclusion

High dynamic range (HDR) imaging in dynamic scenes is still a challenging
problem. Most existing HDR methods produce results with undesirable ghosting
artifacts caused by misalignment between low dynamic range (LDR) images. In
addition, adequate compensation should also be applied to the corrupted
contents in under and overexposed regions. Otherwise, distortions will appear
in the generated HDR images. In this paper, we propose a multi-scale sampling
and aggregation network for HDR imaging in dynamic scenes. The proposed method
implicitly aligns LDR images by sampling and aggregating high-correspondence
features around the misaligned pixels at different scale spaces. Then, the
aggregated features are progressively fused in a coarse-to-fine manner, so our
proposed method can effectively reduce the ghosting artifacts caused by object
motion. In addition, we propose a dense wavelet sub-network to merge the
aligned features and generate corresponding HDR images, which decomposes the
input into several non-overlap frequency subbands for feature extraction.
Different frequency subbands contain different information for reconstruction,
so our method can effectively compensate for corrupted content in saturated
regions. Experiments show that our proposed model significantly outperforms
other HDR methods and achieves the best results. In addition, the HDR images
generated by our model contain fewer distortions and more detailed content,
leading to the best visual quality.

## References

  * [1] J. Froehlich, S. Grandinetti, B. Eberhardt, S. Walter, A. Schilling, and H. Brendel, “Creating cinematic wide gamut hdr-video for the evaluation of tone mapping operators and hdr-displays,” in _Digital photography X_ , vol. 9023. International Society for Optics and Photonics, 2014, p. 90230X.
  * [2] M. D. Tocci, C. Kiser, N. Tocci, and P. Sen, “A versatile hdr video production system,” _ACM Transactions on Graphics (TOG)_ , vol. 30, no. 4, pp. 1–10, 2011.
  * [3] Q. Ye, J. Xiao, K.-M. Lam, and T. Okatani, “Progressive and selective fusion network for high dynamic range imaging,” in _Proceedings of the 29th ACM International Conference on Multimedia_ , 2021, pp. 5290–5297.
  * [4] L. Bogoni, “Extending dynamic range of monochrome and color images through fusion,” in _Proceedings of the International Conference on Pattern Recognition._ , vol. 3. IEEE, 2000, pp. 7–12.
  * [5] E. A. Khan, A. O. Akyuz, and E. Reinhard, “Ghost removal in high dynamic range images,” in _Proceedings of the International Conference on Image Processing_. IEEE, 2006, pp. 2005–2008.
  * [6] K. Jacobs, C. Loscos, and G. Ward, “Automatic high-dynamic range image generation for dynamic scenes,” _IEEE Computer Graphics and Applications_ , vol. 28, no. 2, pp. 84–93, 2008.
  * [7] E. Reinhard, W. Heidrich, P. Debevec, S. Pattanaik, G. Ward, and K. Myszkowski, _High dynamic range imaging: acquisition, display, and image-based lighting_. Morgan Kaufmann, 2010.
  * [8] O. Gallo, N. Gelfandz, W.-C. Chen, M. Tico, and K. Pulli, “Artifact-free high dynamic range imaging,” in _2009 IEEE International conference on computational photography (ICCP)_. IEEE, 2009, pp. 1–7.
  * [9] T. Grosch, “Fast and robust high dynamic range image generation with camera and object movement,” _Vision, Modeling and Visualization, RWTH Aachen_ , pp. 277–284, 2006.
  * [10] T.-H. Oh, J.-Y. Lee, Y.-W. Tai, and I. S. Kweon, “Robust high dynamic range imaging by rank minimization,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 37, no. 6, pp. 1219–1232, 2014.
  * [11] Q. Yan, J. Sun, H. Li, Y. Zhu, and Y. Zhang, “High dynamic range imaging by sparse representation,” _Neurocomputing_ , vol. 269, pp. 160–169, 2017.
  * [12] C. Lee, Y. Li, and V. Monga, “Ghost-free high dynamic range imaging via rank minimization,” _IEEE Signal Processing Letters_ , vol. 21, no. 9, pp. 1045–1049, 2014.
  * [13] Q. Yan, D. Gong, Q. Shi, A. v. d. Hengel, C. Shen, I. Reid, and Y. Zhang, “Attention-guided network for ghost-free high dynamic range imaging,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 1751–1760.
  * [14] Q. Yan, Y. Zhu, and Y. Zhang, “Robust artifact-free high dynamic range imaging of dynamic scenes,” _Multimedia Tools and Applications_ , vol. 78, no. 9, pp. 11 487–11 505, 2019.
  * [15] N. K. Kalantari, R. Ramamoorthi _et al._ , “Deep high dynamic range imaging of dynamic scenes.” _ACM Transactions on Graphics (TOG)_ , vol. 36, no. 4, pp. 144–1, 2017.
  * [16] D. Hafner, O. Demetz, and J. Weickert, “Simultaneous hdr and optic flow computation,” in _Proceedings of the International Conference on Pattern Recognition_. IEEE, 2014, pp. 2065–2070.
  * [17] S. Wu, J. Xu, Y.-W. Tai, and C.-K. Tang, “Deep high dynamic range imaging with large foreground motions,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 117–132.
  * [18] W. Zhang and W.-K. Cham, “Gradient-directed multiexposure composition,” _IEEE Transactions on Image Processing_ , vol. 21, no. 4, pp. 2318–2323, 2011\.
  * [19] S.-h. Lee, J. S. Park, and N. I. Cho, “A multi-exposure image fusion based on the adaptive weights reflecting the relative pixel intensity and global gradient,” in _Proceedings of the IEEE International Conference on Image Processing (ICIP)_. IEEE, 2018, pp. 1737–1741.
  * [20] C. Liu _et al._ , “Beyond pixels: exploring new representations and applications for motion analysis,” Ph.D. dissertation, Massachusetts Institute of Technology, 2009.
  * [21] D. Sun, X. Yang, M.-Y. Liu, and J. Kautz, “Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 8934–8943.
  * [22] J. J. Yu, A. W. Harley, and K. G. Derpanis, “Back to basics: Unsupervised learning of optical flow via brightness constancy and motion smoothness,” in _Proceedings of the European Conference on Computer Vision (ECCV)_. Springer, 2016, pp. 3–10.
  * [23] Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” in _Proceedings of the European Conference on Computer Vision_. Springer, 2020, pp. 402–419.
  * [24] X. Zhu, H. Hu, S. Lin, and J. Dai, “Deformable convnets v2: More deformable, better results,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 9308–9316.
  * [25] Z. Pu, P. Guo, M. S. Asif, and Z. Ma, “Robust high dynamic range (hdr) imaging with complex motion and parallax,” in _Proceedings of the Asian Conference on Computer Vision_ , 2020.
  * [26] X. Wang, K. C. Chan, K. Yu, C. Dong, and C. Change Loy, “Edvr: Video restoration with enhanced deformable convolutional networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2019, pp. 0–0.
  * [27] Z. Liu, W. Lin, X. Li, Q. Rao, T. Jiang, M. Han, H. Fan, J. Sun, and S. Liu, “Adnet: Attention-guided deformable convolutional network for high dynamic range imaging,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop_ , 2021, pp. 463–470.
  * [28] K. C. Chan, X. Wang, K. Yu, C. Dong, and C. C. Loy, “Understanding deformable alignment in video super-resolution,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 35, no. 2, 2021, pp. 973–981.
  * [29] S. Raman and S. Chaudhuri, “Reconstruction of high contrast images for dynamic scenes,” _The Visual Computer_ , vol. 27, no. 12, pp. 1099–1114, 2011.
  * [30] Y. S. Heo, K. M. Lee, S. U. Lee, Y. Moon, and J. Cha, “Ghost-free high dynamic range imaging,” in _Proceedings of the Asian Conference on Computer Vision_. Springer, 2010, pp. 486–500.
  * [31] A. Tomaszewska and R. Mantiuk, “Image registration for multi-exposure high dynamic range image acquisition,” 2007.
  * [32] T. Mitsunaga and S. K. Nayar, “Radiometric self calibration,” in _Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149)_ , vol. 1. IEEE, 1999, pp. 374–380.
  * [33] H. Zimmer, A. Bruhn, and J. Weickert, “Freehand hdr imaging of moving scenes with simultaneous resolution enhancement,” in _Computer Graphics Forum_ , vol. 30, no. 2. Wiley Online Library, 2011, pp. 405–414.
  * [34] J. Hu, O. Gallo, K. Pulli, and X. Sun, “Hdr deghosting: How to deal with saturation?” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2013, pp. 1163–1170.
  * [35] S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High dynamic range video,” _ACM Transactions on Graphics (TOG)_ , vol. 22, no. 3, pp. 319–325, 2003.
  * [36] P. Sen, N. K. Kalantari, M. Yaesoubi, S. Darabi, D. B. Goldman, and E. Shechtman, “Robust patch-based hdr reconstruction of dynamic scenes.” _ACM Transactions on Graphics (TOG)_ , vol. 31, no. 6, pp. 203–1, 2012.
  * [37] Q. Yan, L. Zhang, Y. Liu, Y. Zhu, J. Sun, Q. Shi, and Y. Zhang, “Deep hdr imaging via a non-local network,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 4308–4322, 2020.
  * [38] J. Chen, Z. Yang, T. N. Chan, H. Li, J. Hou, and L.-P. Chau, “Attention-guided progressive neural texture fusion for high dynamic range image restoration,” _IEEE Transactions on Image Processing_ , vol. 31, pp. 2661–2672, 2022.
  * [39] Y. Niu, J. Wu, W. Liu, W. Guo, and R. W. Lau, “Hdr-gan: Hdr image reconstruction from multi-exposed ldr images with large motions,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 3885–3896, 2021.
  * [40] X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 2794–2802.
  * [41] J. Adler and S. Lunz, “Banach wasserstein gan,” _Proceedings of the Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [42] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, “Improved training of wasserstein gans,” _Proceedings of the Advances in neural information processing systems_ , vol. 30, 2017.
  * [43] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 7132–7141.
  * [44] O. T. Tursun, A. O. Akyüz, A. Erdem, and E. Erdem, “An objective deghosting quality metric for hdr images,” in _Computer Graphics Forum_ , vol. 35, no. 2. Wiley Online Library, 2016, pp. 139–152.
  * [45] R. Mantiuk, K. J. Kim, A. G. Rempel, and W. Heidrich, “Hdr-vdp-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions,” _ACM Transactions on graphics (TOG)_ , vol. 30, no. 4, pp. 1–14, 2011.


Topological structures, dark matter
and gravitational waves in $E_{6}$

Rinku Maji,1 Qaisar Shafi,2 Amit Tiwari2

1 Cosmology, Gravity and Astroparticle Physics Group,

Center for Theoretical Physics of the Universe,

Institute for Basic Science, Daejeon 34126, Republic of Korea

2 Bartol Research Institute, Department of Physics and Astronomy,

University of Delaware, Newark, DE 19716, USA

We discuss the appearance of topological structures from the spontaneous
breaking of $E_{6}$ to the Standard Model via its maximal subgroup
$SO(10)\times U(1)_{\psi}$. They include dumbbells, metastable strings, as
well as domain walls bounded by necklaces. We provide a novel scenario for
producing metastable strings based on the symmetry breaking
$U(1)_{\psi}\longrightarrow\mathbb{Z}_{8}\longrightarrow\mathbb{Z}_{4}$. The
metastable string arises from the merger of $\mathbb{Z}_{8}$ strings that
bound a domain wall. An unbroken gauge $\mathbb{Z}_{2}$ symmetry from $SO(10)$
breaking yields viable stable dark matter candidates as well as topologically
stable strings. We discuss the gravitational wave emission from two varieties
of cosmic strings, namely the superheavy metastable ones and the intermediate
scale topologically stable cosmic strings.

##  1 Introduction

In $SO(10)$, more precisely $Spin(10)$ grand unification [1, 2], there are two
important discrete gauge symmetries. The first one, a $\mathbb{Z}_{2}$ gauge
symmetry, is a subgroup of the center $\mathbb{Z}_{4}$ of $SO(10)$ which, if
left unbroken, yields topologically stable cosmic strings [3]. In a
supersymmetric framework this $\mathbb{Z}_{2}$ is precisely “matter” parity,
which implies the stability of the lightest supersymmetric particle and thus
its appearance as a compelling dark matter candidate. Even in the absence of
supersymmetry this $\mathbb{Z}_{2}$ symmetry can guarantee the existence of
dark matter particle depending on the particle content of the model. For
instance, by supplementing $SO(10)$ with a $U(1)$ axion symmetry and requiring
absence of the axion domain wall problem, one is led to introduce additional
matter fields in the model that yield a WIMP-like dark matter in addition to
axions [4, 5, 6]. For other Refs. on dark matter in SO(10) model see, [7, 8,
9, 10, 11, 12, 13, 14, 15, 16].

The second discrete gauge symmetry in $SO(10)$ which plays an important role
in cosmology and particle physics is $C$-parity [17, 18], also referred to as
$D$-parity [19]. In contrast to the above discrete $\mathbb{Z}_{2}$ symmetry,
the $C$-parity is necessarily spontaneously broken, and depending on the
$SO(10)$ symmetry breaking pattern, one predicts the appearance of composite
topological structures known as “walls bounded by strings” (WBS) [17].
Remarkably, WBS have recently been discovered in superfluid 3He-B phase
transitions [20]. These composite structures, now found to be present in a
wide variety of realistic unified models [21, 22, 23, 24, 25], may have left
traces of their presence in the early universe through the emission of
gravitational waves.

In a recent paper [26] we made a systematic study of composite topological
structures that can appear in realistic $SO(10)$ models. In addition to
structures such as dumbbells and WBS, we identified some more exotic objects
such as “walls bounded by necklaces” and “starfish”. Here, a necklace refers
to a closed string with monopoles as beads on it [27, 28, 29]. The “starfish”
configuration consists of a higher charge monopole with flux tubes ending on
five distinct lower charge antimonopoles [26].

Our aim in this paper is to extend the $SO(10)$ discussion started in [26] to
$E_{6}$ grand unification [30, 31, 32]. For some earlier work on topological
structures in $E_{6}$, see Ref. [33]. We focus here on the breaking of $E_{6}$
via its subgroup $SO(10)\times U(1)_{\psi}$, and the various topological
structures we identify here are closely linked with the abelian component
$U(1)_{\psi}$. In addition to a practically stable $U(1)_{\psi}$ string, first
identified by Witten [34], we highlight dumbbells, superheavy metastable
strings [35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45] which can explain the
recent PTA observations [46, 47, 48, 49], and domain wall bounded by a
necklace. We present a novel well motivated scenario for producing metastable
cosmic strings based on the symmetry breaking
$E_{6}\longrightarrow[SO(10)\times
U(1)_{\psi}]/\mathbb{Z}_{4}\longrightarrow[SO(10)\times\mathbb{Z}_{8}]/\mathbb{Z}_{4}\longrightarrow
SO(10)$. In the presence of primordial $U(1)_{\psi}$ monopoles this breaking
pattern leads to the appearance of a domain wall bounded by a necklace.
However, if the primordial monopoles are inflated away, we can obtain
metastable strings associated with flux tubes $\psi/4$ from the merger of two
$\mathbb{Z}_{8}$ strings carrying $\psi/8$ flux each, that are initially
separate but attached to the same domain wall. The quantum mechanical
tunneling of $U(1)_{\psi}$ monopole-antimonopole pairs carrying magnetic
charges $\pm\psi/4$ makes the string metastable. Of course, if the
$U(1)_{\psi}$ monopoles are much heavier than the breaking scale of
$U(1)_{\psi}$, the quantum mechanical tunneling is exponentially suppressed
and these strings are practically stable.

By retaining an unbroken $\mathbb{Z}_{2}$ symmetry from the $SO(10)$ subgroup
of $E_{6}$, we also produce topologically stable strings and WIMP-like dark
matter candidates. We also identify a scenario with keV scale sterile DM
neutrino. We discuss the gravitational wave spectrum generated by metastable
cosmic strings with $G\mu\sim 10^{-7}$ accompanied by topologically stable
cosmic strings with $G\mu\sim 10^{-12}-10^{-10}$, where $G\mu$ denotes the
dimensionless string tension parameter. This will be tested by the ongoing
pulsar timing array experiments as well as other ongoing and proposed
experiments.

##  2 $E_{6}$ Model and $U(1)_{\psi}$ monopoles

We are primarily focused on topological structures that arise from the
symmetry breaking $E_{6}\to SO(10)\times U(1)_{\psi}/\mathbb{Z}_{4}\to
SO(10)$. There will be some overlap with the discussion carried out in Ref.
[33] which also considers the breaking of $E_{6}$ via the trinification
subgroup $SU(3)_{c}\times SU(3)_{L}\times SU(3)_{R}$.

Matter fields | Representations under $G_{\rm SM}$ | $2\sqrt{6}\,Q_{\psi}$
---|---|---
$q_{i}$ | $({\bf 3,2},1/6)$ | $1$
$u_{i}^{c}$ | $({\bf\bar{3},1},-2/3)$ | $1$
$d_{i}^{c}$ | $({\bf\bar{3},1},1/3)$ | $1$
$l_{i}$ | $({\bf 1,2},-1/2)$ | $1$
$\nu_{i}^{c}$ | $({\bf 1,1},0)$ | $1$
$e_{i}^{c}$ | $({\bf 1,1},1)$ | $1$
$\tilde{H}_{ui}$ | $({\bf 1,2},1/2)$ | $-2$
$\tilde{H}_{di}$ | $({\bf 1,2},-1/2)$ | $-2$
$D_{i}$ | $({\bf 3,1},-1/3)$ | $-2$
$D_{i}^{c}$ | $({\bf\bar{3},1},1/3)$ | $-2$
$N_{i}$ | $({\bf 1,1},0)$ | $4$

Table 1: Transformation properties of the $E_{6}$ matter fields under the SM
gauge group $G_{\text{SM}}$ and their charges under the local symmetry
$U(1)_{\psi}$, where $Q_{\psi}$ is the normalized GUT generator of
$U(1)_{\psi}$. Family indices are denoted by the subscript $i\;(=1,2,3)$.

The matter content of the model consisting of three chiral 27- plets is shown
in Table 1 which also lists their $U(1)_{\psi}$ quantum numbers. The generator
is normalized such that a minimal charge of unity is compatible with a period
of $2\pi$.

Fig. 1: Embedded inside $E_{6}$, $U(1)_{\psi}$ intersects with the $SO(10)$
group in the center $\mathbb{Z}_{4}$. Minimally charged $U(1)_{\psi}$ monopole
corresponds to a $2\pi/4$ rotation along this circle.

The breaking of $E_{6}$ to $SO(10)\times U(1)_{\psi}/\mathbb{Z}_{4}$ is
implemented with a Higgs 78-plet. The group $U(1)_{\psi}$ intersects $SO(10)$
in the center $\mathbb{Z}_{4}$ [33], and so the minimally charged monopole
produced in this breaking corresponds to $2\pi/4$ rotation along
$U(1)_{\psi}$, followed by an appropriate $SO(10)$ rotation to return to the
identity element, as shown in Fig. 1. This monopole carries $1/4$ of
$U(1)_{\psi}$ magnetic flux as well as some $SO(10)$ flux.

Next let us consider the consequence of a full $2\pi$ rotation along
$U(1)_{\psi}$. Clearly, this yields a monopole that carries one full unit of
$U(1)_{\psi}$ flux. Anticipating the Higgs phase associated with $U(1)_{\psi}$
breaking, a novel composite topological structure with finite energy can be
realized by combining a unit $\psi$ flux monopole with four minimally charged
antimonopoles.

##  3 Dumbbells and fourfold monopole

Recall that the breaking of $E_{6}$ to $SO(10)\times
U(1)_{\psi}/\mathbb{Z}_{4}$ with a 78-plet vev along the singlet direction
yields a minimally charged magnetic monopole that carries $\psi/4$ flux. With
a non-zero vev of $N$ that carries a charge of 4 (Table 1), $U(1)_{\psi}$ is
then broken to $\mathbb{Z}_{4}$ which is identified with the center of
$SO(10)$. This breaking produces a tube with flux $\psi/4$ that confines the
magnetic monopole produced in the first step. A dumbbell consisting of a
$U(1)_{\psi}$ monopole-antimonopole pair connected by the flux tube is shown
in Fig. 2.

Fig. 2: The second step of symmetry breaking involves giving a VEV to the
$N$-type component of the Higgs 27-plet, thus breaking $U(1)_{\psi}$ to
$\mathbb{Z}_{4}$ ($\subset SO(10)$). The flux of the minimal $U(1)_{\psi}$
monopole (in red) from the first step is confined in a
$U(1)_{\psi}\;\mathbb{Z}_{4}$ tube, connected to the minimal $U(1)_{\psi}$
anti-monopole (in green).

To realize the metastable string scenario that explains the PTA observations,
we can assume that the two symmetry breaking scales corresponding to vevs of
the 78-plet and $N$-plet are adequately close together, and the dimensionless
string tension parameter $G\mu$ is of order $10^{-6}$ or so. The primordial
$U(1)_{\psi}$ monopoles are inflated away, but they can break the
$U(1)_{\psi}$ string through quantum mechanical tunneling, thus making it
metastable.

Note that the fermions in the 10-plets of $SO(10)$ acquire masses through
their renormalizable couplings to $N$, while the $N_{i}$ matter fields do so
through the dimension five couplings $N_{i}N_{j}\left<N\right>\left<N\right>$,
suppressed by the reduced Planck mass $m_{\rm Pl}=2.4\times 10^{18}$ GeV.

Fig. 3: The flux of a monopole (in blue) corresponding to a $2\pi$ rotation
along $U(1)_{\psi}$ is squeezed into four tubes. These tubes end on the
minimal $U(1)_{\psi}$ antimonopoles, forming a “fourfold” configuration.

We can realize an exotic finite energy configuration by combining a “fourfold”
monopole that carries one unit of $\psi$ magnetic flux and four elementary
antimonopoles with each carrying $-1/4\;\psi$ flux, as shown in Fig. 3.

##  4 Domain wall bounded by necklace and strings from domain walls

The previous discussion provides a nice segway to a somewhat novel scenario
for producing the metastable strings in which the $N_{i}$ matter fields
acquire non-zero masses at renormalizable level from a non-zero vev of
$351^{\prime}$, which carries a $U(1)_{\psi}$ charge of $-8$. This breaks
$U(1)_{\psi}$ to $\mathbb{Z}_{8}$, which produces a tube with flux $\psi/8$.
With the $U(1)_{\psi}$ monopole carrying a flux $\psi/4$, the confinement of
these monopoles and antimonopoles now occurs via a necklace, as shown in Fig.
4.

Fig. 4: Necklace from the symmetry breaking $E_{6}\to SO(10)\times
U(1)_{\psi}/\mathbb{Z}_{4}\to SO(10)\times\mathbb{Z}_{8}/\mathbb{Z}_{4}$. The
symmetry breaking $U(1)_{\psi}\to\mathbb{Z}_{8}$ yields a flux tube carrying
half the flux of the minimal $U(1)_{\psi}$ monopole.

However, this is not the end of the story. A non-zero vev for $N$ is still
required to provide masses to the fermions in the $SO(10)$ 10-plets, which
breaks the $\mathbb{Z}_{8}$ symmetry to $\mathbb{Z}_{4}$, the center of
$SO(10)$. Breaking of the $\mathbb{Z}_{8}$ symmetry to $\mathbb{Z}_{4}$
produces a domain wall, and so the end result appears to be the presence of a
domain wall bounded by a necklace, as shown in Fig. 5. Actually this is a
reasonable conclusion since we have broken $E_{6}$ to $SO(10)$ as follows:

$\displaystyle E_{6}\to SO(10)\times U(1)_{\psi}\to
SO(10)\times\mathbb{Z}_{8}\to SO(10)$ (1)

Since $E_{6}$ is effectively broken to $SO(10)$, both the first and second
homotopy groups of $E_{6}/SO(10)$ are trivial and no topologically stable
structures are expected to be present. And so the appearance of domain walls
bounded by a necklace is compatible with this conclusion.

Fig. 5: The breaking of $\mathbb{Z}_{8}\to\mathbb{Z}_{4}$ by the vev of $N$
results in a domain wall (in orange) bounded by a necklace configuration.

How then do we realize the metastable string scenario in this framework? If we
assume that the primordial monopoles are inflated away, we are left with the
following symmetry breaking:

$\displaystyle U(1)_{\psi}\to\mathbb{Z}_{8}\to\mathbb{Z}_{4}$ (2)

The first breaking of $U(1)_{\psi}$ in Eq. (2) should produce $\mathbb{Z}_{8}$
strings, but the subsequent breaking of $\mathbb{Z}_{8}$ to $\mathbb{Z}_{4}$
creates domain walls. So we end up producing domain walls bounded by strings,
and since $U(1)_{\psi}$ is effectively broken to $\mathbb{Z}_{4}$, there also
should appear “stable” $\mathbb{Z}_{4}$ strings. This happens from the
collapse of the domain wall between the two $\mathbb{Z}_{8}$ strings. The
ensuing $\mathbb{Z}_{4}$ string ends up being metastable due to the quantum
mechanical pair production of $U(1)_{\psi}$ monopole-antimonopole pairs. Note
that the collapse time $t_{c}$ for the $Z_{8}$ string-domain wall system that
produces the $Z_{4}$ metastable string is of order $\mu/\sigma$ [50, 51, 52,
21], where $\mu$ and $\sigma$ respectively denote the string mass per unit
length and wall mass per unit area. Assuming a string scale of order $10^{16}$
GeV and the $N$ VEV lying somewhere between this value and $10$ TeV, the
collapse time $t_{c}\lesssim 10^{-4}$ sec. The gravitational wave spectrum
produced by the metastable strings is discussed in Section 6.

Finally, it is perhaps worth repeating that in the absence both of quantum
mechanical tunneling and primordial $U(1)_{\psi}$ monopoles, the symmetry
breaking $U(1)_{\psi}\longrightarrow Z_{8}\longrightarrow Z_{4}$ (center of
$SO(10)$) is expected to yield practically stable strings. We expect that the
$Z_{4}$ strings will contribute in the usual way to the gravitational wave
spectrum.

This scenario may be contrasted with the symmetry breaking
$SU(2)\longrightarrow U(1)\longrightarrow Z_{2}\longrightarrow 1$, discussed
in Ref. [26]. A domain wall bounded by a necklace is also present in this
case. However, since $SU(2)$ is a simply connected topological space, no
strings remain at the end.

##  5 Dark matter in $E_{6}$

Dark matter (DM) candidates in $E_{6}$ models has been a subject of some
discussion in recent years [53, 54, 55, 56]. There is some overlap with the
WIMP-like DM that arises in $SO(10)$ models supplemented by a $U(1)$ axion
symmetry [5, 6, 16, 13]. We briefly summarize here the DM candidate associated
with the breaking of $U(1)_{\psi}$ and the ensuing topological structure.

  * •

$U(1)_{\psi}$ breaking to $\mathbb{Z}_{4}$ through the vev of $N$. For
$\langle N\rangle\sim M_{GUT}$ we find superheavy metastable strings that can
explain the recent PTA measurements. A plausible DM candidate in this case
would be an intermediate mass stable neutral particle present in the
vectorlike $SU(2)$ doublets [54, 6]. On the other hand, the slepton-like
scalars ($\tilde{l}$ and $\tilde{\nu}$) from the scalar $SO(10)$ 16-plet are
$\mathbb{Z}_{2}$-odd and therefore could be potential dark matter candidates.
We can realize a two component dark matter scenario with one scalar and one
fermion, if the masses of $N_{i}$ and the slepton-like scalars are tuned
around the electroweak or TeV scale, whereas the masses of the vectorlike
doublet fermions remain at the GUT scale [55]. The lightest neutral scalar or
fermion is absolutely stable. The heavier component can decay into the lighter
one only through a mixing with the charge neutral fermion fields from the
vectorlike doublets, and this mixing is GUT scale suppressed. Thus, the
heavier component is metastable and the lighter one is stable.

  * •

For $\langle N\rangle\ll M_{GUT}$, the $U(1)_{\psi}$ string is essentially
stable [34], and depending on the vev of $N$, we could either have a doublet-
singlet DM candidate [16], or the lightest $N_{i}$ as DM with mass in the keV
range [53].

  * •

In the subsequent breaking of $SO(10)$ to the SM, we can obtain a metastable
pseudo-Goldstone boson as dark matter particle when the singlet from
$\overline{126}(-2)\in 351^{\prime}$ acquires a vev ($\gtrsim 10^{11}$ GeV) to
break the intermediate symmetry with rank five to the SM, and the neutrino-
like scalar component of $16(1)\in 27$ has a vev around the electroweak scale
($10^{2}-10^{5}$ GeV). One of the would be Goldstone bosons is absorbed by the
gauge boson associated with the broken diagonal generator. The other CP-odd
scalar becomes a pseudo-Goldstone boson dark matter particle [57, 58, 22].

If $U(1)_{\psi}$ is broken in two steps to $\mathbb{Z}_{4}$, namely via
$\mathbb{Z}_{8}$, we produce both walls bounded by necklaces as well as
$\mathbb{Z}_{4}$ strings from the collapse of domain walls connecting two
$\mathbb{Z}_{8}$ strings. Since the $351’$ vev is larger than the vev of $N$,
we expect the fermions $N_{i}$ to be heavier than the $10$-plet fermions in
$SO(10)$. In this case the neutral fermions in the latter multiplets provide
dark matter in the $10^{12}$ GeV mass range.

##  6 Gravitational waves from metastable and stable strings

The dumbells, formed during the decay of the metastable strings carrying
$U(1)_{\psi}$ flux, carry no unconfined flux. The subsequent breaking of the
diagonal generator orthogonal to the SM hypercharge can be done with the vev
of a tensorial 126-plet or a spinorial 16-plet scalar. The former produces
topologically stable $\mathbb{Z}_{2}$ strings, whereas the latter can produce
metastable strings depending on the symmetry breaking pattern. The
gravitational waves from metastable strings have been extensively studied in
light of the pulsar timing array experiments [35, 36, 37, 38, 39, 40, 41, 42,
43, 44, 45]. In this section we estimate the gravitational wave spectra if
both the stable and metastable strings are present.

The stochastic gravitational wave background is given by [59, 60]

$\displaystyle\Omega_{\rm GW}(f)=\frac{4\pi^{2}}{3H_{0}^{2}}f^{3}\int_{z_{\rm
min}}^{z_{F}}dz\int dl\,h^{2}(f,l,z)\frac{d^{2}R}{dz\,dl}\ ,$ (3)

where $h(f,l,z)$ is the waveform of the burst, and the burst rate per unit
space-time volume is expressed as

$\displaystyle\frac{d^{2}R}{dz\,dl}=N_{b}H_{0}^{-3}\phi_{V}(z)\frac{n(l,t)}{T(1+z)}\Delta(f,l,z).$
(4)

Here, $H_{0}$ is the present day Hubble parameter, $N_{b}$ is the average
number of burst events in a oscillation period $T$, $n(l,t)$ the number
distribution, $\Delta(f,l,z)$ is the observable fraction of the bursts [61,
59, 60], and

$\displaystyle\phi_{V}(z)=\frac{4\pi H_{0}^{3}r^{2}(z)}{(1+z)^{3}{H}(z)},$ (5)

with $H(z)$ being the Hubble parameter and $r(z)$ the proper distance at the
redshift $z$. In Eq. (3), $z_{F}$ denotes the redshift at the network
formation, and the lower limit $z_{min}$ is chosen to remove the recent
infrequent bursts from the background [61], or it corresponds the redshift
when the metastable network disappears.

Fig. 6: Gravitational wave background from the metastable cosmic strings with
$G\mu=10^{-7}$, $\sqrt{\kappa}=8.1$, and separately from intermediate scale
stable cosmic strings with $G\mu=10^{-12},10^{-11},10^{-10}$ (dot-dashed
curves). The former has contributions from the loops (solid magenta curve) as
well as segments from the decay of the long strings (dashed magenta) and loops
(dotted magenta). Clearly, the contribution from the stable strings is
subdominant. However, their contribution can be important for the lowest one
or two frequency bins for $10^{-10}\gtrsim G\mu\gtrsim 10^{-11}$. The violin
plots depict the posteriors of Hellings-Downs correlated [62] free spectra for
NANOGrav data [35]. The constraints from Big Bang Nucleosynthesis (BBN) [63]
and LIGO-VIRGO data [64] are also shown. Several proposed experiments, namely,
HLVK [65], CE [66], ET [67], DECIGO [68], BBO [69, 70], LISA [71, 72] and SKA
[73, 74] can probe the gravitational wave spectra as dictated by their
respective power-law integrated sensitivity curves [75, 76].

We assume that these two types of strings enter the scaling regime [77, 78,
79, 80, 81, 82, 83, 84] independently with loop distributions given in the
Refs. [82, 83, 64, 85]. We estimate the gravitational wave background as a
consequence of the unresolvable bursts, dominantly from the cusps [61, 60, 86,
87] for the loops. For more details see Refs. [13, 38] and references therein.
The monopole-antimonopole connected by the string segments (Fig. 2) do not
carry any unconfined flux. The subhorizon segments from the deacay of long
strings as well as the loops oscillate and radiate gravitational waves. We
estimate the gravitational waves assuming the straight string approximation
and follow the prescription of Ref. [59]. We use the number distribution of
segments from the loops from Ref. [85]. The gravitational power radiated from
a segment is $P_{\rm seg}=\tilde{\Gamma}G\mu^{2}$ and we assume
$\tilde{\Gamma}=50$ as in [35].

Fig. 6 shows the gravitational wave backgrounds from the metastable cosmic
strings with $G\mu=10^{-7}$ and the stable intermediate scale cosmic strings
with $G\mu=10^{-12}-10^{-10}$. The metastable strings with a metastability
factor $\sqrt{\kappa}\approx 8$ ($\kappa=m_{M}^{2}/\mu$ with $m_{M}$ being the
monopole mass) can explain the NANOGrav 15 year data around the nanoHertz
frequencies. It is worth mentioning that for $G\mu>10^{-7}$ [88, 38], if the
metastable strings experience some $e$-foldings of primordial inflation and
therefore enter the scaling regime at a late time $t_{F}$ after their horizon
reentry, the gravitational wave spectrum decreases as a power-law $f^{-1/3}$
at high frequencies and satisfies the constraint set by the LIGO-VIRGO data
[64]. The gravitational waves from the stable strings remain subdominant, of
course. However, for $10^{-10}\gtrsim G\mu\gtrsim 10^{-11}$ their contribution
can be important for the lowest one or two frequency bins of PTA data. Several
proposed experiments, namely, HLVK [65], CE [66], ET [67], DECIGO [68], BBO
[69, 70], LISA [71, 72] and SKA [73, 74] can probe the gravitational wave
spectra as dictated by their respective power-law integrated sensitivity
curves [75, 76].

Before closing, let us note that the metastable strings discussed here can
carry both electrically charged and electrically neutral fermion zero modes.
In addition, the stable intermediate scale string from $SO(10)$ breaking
carries right handed neutrino zero modes whose impact on the gravitational
wave spectrum has recently been discussed in Ref. [42].

##  7 Conclusion

We have explored a variety of composite topological structures that appear in
the spontaneous breaking of $E_{6}$ grand unification symmetry to the Standard
Model. We focused exclusively on structures arising from the symmetry breaking
of $E_{6}$ via its maximal subgroup $SO(10)\times U(1)_{\psi}$, and the
composite topological structures we identify provide a test of $E_{6}$ grand
unification if they appeared in the early universe. We identify a novel
scenario for realizing metastable strings in the $E_{6}$ model. The breaking
of $U(1)_{\psi}$ symmetry to $\mathbb{Z}_{4}$ (center of $SO(10)$) in this
case proceeds in two steps and yields the metastable strings from the collapse
of domain walls.

A remnant $\mathbb{Z}_{2}$ gauge symmetry from the $SO(10)$ sector in $E_{6}$
is essential in ensuring the stability of a viable dark matter particle. We
also identify a dark matter scenario consisting of keV mass sterile neutrino.
The gravitational spectrum generated by superheavy metastable cosmic strings
and intermediate scale topologically stable strings is also described.

Finally, a few remarks about topological structures and primordial inflation
may be in order here. For instance, in order to implement the metastable
string scenario the primordial monopole is inflated away but certainly not the
string, even though their symmetry breaking scales are close to each other.
This can be realized in models of hybrid inflation by suitably adjusting the
parameters associated with the appropriate ‘waterfall’ fields [38, 89]. The
$E_{6}$ model discussed here offers a variety of symmetry breaking scenarios
and associated topological structures that should be testable in the ongoing
and future experiments.

## Acknowledgments

R.M. is supported by Institute for Basic Science under the project code:
IBS-R018-D3. A.T. is partially supported by the Bartol Research Institute,
University of Delaware.

## References

  * [1] H. Georgi, _The State of the Art—Gauge Theories_ , _AIP Conf. Proc._ 23 (1975) 575.
  * [2] H. Fritzsch and P. Minkowski, _Unified Interactions of Leptons and Hadrons_ , _Annals Phys._ 93 (1975) 193.
  * [3] T.W.B. Kibble, G. Lazarides and Q. Shafi, _Strings in SO(10)_ , _Phys. Lett. B_ 113 (1982) 237.
  * [4] G. Lazarides and Q. Shafi, _Axion Models with No Domain Wall Problem_ , _Phys. Lett. B_ 115 (1982) 21.
  * [5] R. Holman, G. Lazarides and Q. Shafi, _Axions and the Dark Matter of the Universe_ , _Phys. Rev. D_ 27 (1983) 995.
  * [6] G. Lazarides and Q. Shafi, _Axion Model with Intermediate Scale Fermionic Dark Matter_ , _Phys. Lett. B_ 807 (2020) 135603 [2004.11560].
  * [7] M. Kadastik, K. Kannike and M. Raidal, _Matter parity as the origin of scalar Dark Matter_ , _Phys. Rev. D_ 81 (2010) 015002 [0903.2475].
  * [8] Y. Mambrini, N. Nagata, K.A. Olive, J. Quevillon and J. Zheng, _Dark matter and gauge coupling unification in nonsupersymmetric SO(10) grand unified models_ , _Phys. Rev. D_ 91 (2015) 095010 [1502.06929].
  * [9] S.M. Boucenna, M.B. Krauss and E. Nardi, _Dark matter from the vector of SO (10)_ , _Phys. Lett. B_ 755 (2016) 168 [1511.02524].
  * [10] S. Ferrari, T. Hambye, J. Heeck and M.H.G. Tytgat, _SO(10) paths to dark matter_ , _Phys. Rev. D_ 99 (2019) 055032 [1811.07910].
  * [11] S. Bhattacharyya and A. Datta, _Phenomenology of an E6 inspired extension of the Standard Model: Higgs sector_ , _Phys. Rev. D_ 105 (2022) 075021 [2109.08524].
  * [12] G. Lazarides, R. Maji, R. Roshan and Q. Shafi, _Heavier W boson, dark matter, and gravitational waves from strings in an SO(10) axion model_ , _Phys. Rev. D_ 106 (2022) 055009 [2205.04824].
  * [13] G. Lazarides, R. Maji, R. Roshan and Q. Shafi, _A predictive SO(10) model_ , _JCAP_ 12 (2022) 009 [2210.03710].
  * [14] P. Sahu, A. Bhatta, R. Mohanta, S. Singirala and S. Patra, _Flavour anomalies and dark matter assisted unification in SO(10) GUT_ , _JHEP_ 11 (2022) 029 [2204.06392].
  * [15] S. Bhattacharyya and A. Datta, _Dark matter perspective of left-right symmetric gauge model_ , _Nucl. Phys. B_ 991 (2023) 116197 [2206.13105].
  * [16] N. Okada, D. Raut and Q. Shafi, _Axions, WIMPs, proton decay and observable r in SO(10)_ , _Eur. Phys. J. C_ 83 (2023) 273 [2207.10538].
  * [17] T.W.B. Kibble, G. Lazarides and Q. Shafi, _Walls Bounded by Strings_ , _Phys. Rev. D_ 26 (1982) 435.
  * [18] G. Lazarides and Q. Shafi, _Superconducting Membranes_ , _Phys. Lett. B_ 159 (1985) 261.
  * [19] D. Chang, R.N. Mohapatra and M.K. Parida, _Decoupling Parity and $SU(2)_{R}$ Breaking Scales: A New Approach to Left-Right Symmetric Models_, _Phys. Rev. Lett._ 52 (1984) 1072.
  * [20] J.T. Mäkinen, V.V. Dmitriev, J. Nissinen, J. Rysti, G.E. Volovik, A.N. Yudin et al., _Half-quantum vortices and walls bounded by strings in the polar-distorted phases of topological superfluid 3He_, _Nature Commun._ 10 (2019) 237 [1807.04328].
  * [21] D.I. Dunsky, A. Ghoshal, H. Murayama, Y. Sakakihara and G. White, _GUTs, hybrid topological defects, and gravitational waves_ , _Phys. Rev. D_ 106 (2022) 075030 [2111.08750].
  * [22] R. Maji, W.-I. Park and Q. Shafi, _Gravitational waves from walls bounded by strings in SO(10) model of pseudo-Goldstone dark matter_ , _Phys. Lett. B_ 845 (2023) 138127 [2305.11775].
  * [23] M. Eto, T. Hiramatsu, I. Saito and Y. Sakakihara, _String-wall composites winding around a torus knot vacuum in an axionlike model_ , _Phys. Rev. D_ 108 (2023) 116004 [2309.04248].
  * [24] R. Roshan and G. White, _Using gravitational waves to see the first second of the Universe_ , 2401.04388.
  * [25] B. Fu, A. Ghoshal, S.F. King and M.H. Rahat, _Type-I two-Higgs-doublet model and gravitational waves from domain walls bounded by strings_ , 2404.16931.
  * [26] G. Lazarides, Q. Shafi and A. Tiwari, _Composite topological structures in SO(10)_ , _JHEP_ 05 (2023) 119 [2303.15159].
  * [27] M. Hindmarsh and T.W.B. Kibble, _Beads on Strings_ , _Phys. Rev. Lett._ 55 (1985) 2398.
  * [28] M. Aryal and A.E. Everett, _Properties of $Z_{2}$ Strings_, _Phys. Rev. D_ 35 (1987) 3105.
  * [29] T.W.B. Kibble and T. Vachaspati, _Monopoles on strings_ , _J. Phys. G_ 42 (2015) 094002 [1506.02022].
  * [30] F. Gursey, P. Ramond and P. Sikivie, _A Universal Gauge Theory Model Based on E6_ , _Phys. Lett. B_ 60 (1976) 177.
  * [31] Y. Achiman and B. Stech, _Quark Lepton Symmetry and Mass Scales in an E6 Unified Gauge Model_ , _Phys. Lett. B_ 77 (1978) 389.
  * [32] Q. Shafi, _E(6) as a Unifying Gauge Symmetry_ , _Phys. Lett. B_ 79 (1978) 301.
  * [33] G. Lazarides and Q. Shafi, _Monopoles, Strings, and Necklaces in $SO(10)$ and $E_{6}$_, _JHEP_ 10 (2019) 193 [1904.06880].
  * [34] E. Witten, _Superconducting Strings_ , _Nucl. Phys. B_ 249 (1985) 557.
  * [35] NANOGrav collaboration, _The NANOGrav 15 yr Data Set: Search for Signals from New Physics_ , _Astrophys. J. Lett._ 951 (2023) L11 [2306.16219].
  * [36] W. Buchmuller, V. Domcke and K. Schmitz, _Metastable cosmic strings_ , _JCAP_ 11 (2023) 020 [2307.04691].
  * [37] S. Antusch, K. Hinze, S. Saad and J. Steiner, _Singling out SO(10) GUT models using recent PTA results_ , _Phys. Rev. D_ 108 (2023) 095053 [2307.04595].
  * [38] G. Lazarides, R. Maji, A. Moursy and Q. Shafi, _Inflation, superheavy metastable strings and gravitational waves in non-supersymmetric flipped SU(5)_ , _JCAP_ 03 (2024) 006 [2308.07094].
  * [39] R. Maji and W.-I. Park, _Supersymmetric U(1)B-L flat direction and NANOGrav 15 year data_ , _JCAP_ 01 (2024) 015 [2308.11439].
  * [40] W. Ahmed, M.U. Rehman and U. Zubair, _Probing stochastic gravitational wave background from SU(5) × U(1) χ strings in light of NANOGrav 15-year data_, _JCAP_ 01 (2024) 049 [2308.09125].
  * [41] A. Afzal, M. Mehmood, M.U. Rehman and Q. Shafi, _Supersymmetric hybrid inflation and metastable cosmic strings in $SU(4)_{c}\times SU(2)_{L}\times U(1)_{R}$_, 2308.11410.
  * [42] A. Afzal, Q. Shafi and A. Tiwari, _Gravitational wave emission from metastable current-carrying strings in E6_ , _Phys. Lett. B_ 850 (2024) 138516 [2311.05564].
  * [43] W. Ahmed, T.A. Chowdhury, S. Nasri and S. Saad, _Gravitational waves from metastable cosmic strings in the Pati-Salam model in light of new pulsar timing array data_ , _Phys. Rev. D_ 109 (2024) 015008 [2308.13248].
  * [44] G. Lazarides and C. Pallis, _Probing the supersymmetry-mass scale with F-term hybrid inflation_ , _Phys. Rev. D_ 108 (2023) 095055 [2309.04848].
  * [45] C. Pallis, _PeV-Scale SUSY and Cosmic Strings from F-term Hybrid Inflation_ , _Universe_ 10 (2024) 5 [2403.09385].
  * [46] NANOGrav collaboration, _The NANOGrav 15 yr Data Set: Evidence for a Gravitational-wave Background_ , _Astrophys. J. Lett._ 951 (2023) L8 [2306.16213].
  * [47] EPTA, InPTA: collaboration, _The second data release from the European Pulsar Timing Array - III. Search for gravitational wave signals_ , _Astron. Astrophys._ 678 (2023) A50 [2306.16214].
  * [48] D.J. Reardon et al., _Search for an Isotropic Gravitational-wave Background with the Parkes Pulsar Timing Array_ , _Astrophys. J. Lett._ 951 (2023) L6 [2306.16215].
  * [49] H. Xu et al., _Searching for the Nano-Hertz Stochastic Gravitational Wave Background with the Chinese Pulsar Timing Array Data Release I_ , _Res. Astron. Astrophys._ 23 (2023) 075024 [2306.16216].
  * [50] A. Vilenkin and A.E. Everett, _Cosmic Strings and Domain Walls in Models with Goldstone and PseudoGoldstone Bosons_ , _Phys. Rev. Lett._ 48 (1982) 1867.
  * [51] A. Vilenkin, _Cosmic Strings and Domain Walls_ , _Phys. Rept._ 121 (1985) 263.
  * [52] X. Martin and A. Vilenkin, _Gravitational wave background from hybrid topological defects_ , _Phys. Rev. Lett._ 77 (1996) 2879 [astro-ph/9606022].
  * [53] A. Hebbar, G. Lazarides and Q. Shafi, _Light sterile neutrinos, dark matter, and new resonances in a U(1) extension of the MSSM_ , _Phys. Rev. D_ 96 (2017) 055026 [1706.09630].
  * [54] J. Schwichtenberg, _Dark matter in E 6 Grand unification_, _JHEP_ 02 (2018) 016 [1704.04219].
  * [55] T. Bandyopadhyay and R. Maji, _The E6 route to multicomponent dark matter_ , 1911.13298.
  * [56] K.S. Babu, B. Bajc and V. Susič, _A realistic theory of $\mathrm{E_{6}}$ unification through novel intermediate symmetries_, 2403.20278.
  * [57] Y. Abe, T. Toma, K. Tsumura and N. Yamatsu, _Pseudo-Nambu-Goldstone dark matter model inspired by grand unification_ , _Phys. Rev. D_ 104 (2021) 035011 [2104.13523].
  * [58] N. Okada, D. Raut, Q. Shafi and A. Thapa, _Pseudo-Goldstone dark matter in SO(10)_ , _Phys. Rev. D_ 104 (2021) 095002 [2105.03419].
  * [59] L. Leblond, B. Shlaer and X. Siemens, _Gravitational Waves from Broken Cosmic Strings: The Bursts and the Beads_ , _Phys. Rev. D_ 79 (2009) 123519 [0903.4686].
  * [60] S. Olmez, V. Mandic and X. Siemens, _Gravitational-Wave Stochastic Background from Kinks and Cusps on Cosmic Strings_ , _Phys. Rev. D_ 81 (2010) 104028 [1004.0890].
  * [61] T. Damour and A. Vilenkin, _Gravitational wave bursts from cusps and kinks on cosmic strings_ , _Phys. Rev. D_ 64 (2001) 064008 [gr-qc/0104026].
  * [62] Hellings, R. W. and Downs, G. S., _Upper limits on the isotropic gravitational radiation background from pulsar timing analysis_ , _Astrophys. J. Lett._ 265 (1983) L39.
  * [63] G. Mangano and P.D. Serpico, _A robust upper limit on $N_{\rm eff}$ from BBN, circa 2011_, _Phys. Lett. B_ 701 (2011) 296 [1103.1261].
  * [64] LIGO Scientific, Virgo, KAGRA collaboration, _Constraints on Cosmic Strings Using Data from the Third Advanced LIGO–Virgo Observing Run_ , _Phys. Rev. Lett._ 126 (2021) 241102 [2101.12248].
  * [65] KAGRA, LIGO Scientific, Virgo, VIRGO collaboration, _Prospects for observing and localizing gravitational-wave transients with Advanced LIGO, Advanced Virgo and KAGRA_ , _Living Rev. Rel._ 21 (2018) 3 [1304.0670].
  * [66] T. Regimbau, M. Evans, N. Christensen, E. Katsavounidis, B. Sathyaprakash and S. Vitale, _Digging deeper: Observing primordial gravitational waves below the binary black hole produced stochastic background_ , _Phys. Rev. Lett._ 118 (2017) 151105 [1611.08943].
  * [67] G. Mentasti and M. Peloso, _ET sensitivity to the anisotropic Stochastic Gravitational Wave Background_ , _JCAP_ 03 (2021) 080 [2010.00486].
  * [68] S. Sato et al., _The status of DECIGO_ , _Journal of Physics: Conference Series_ 840 (2017) 012010.
  * [69] J. Crowder and N.J. Cornish, _Beyond LISA: Exploring future gravitational wave missions_ , _Phys. Rev. D_ 72 (2005) 083005 [gr-qc/0506015].
  * [70] V. Corbin and N.J. Cornish, _Detecting the cosmic gravitational wave background with the big bang observer_ , _Class. Quant. Grav._ 23 (2006) 2435 [gr-qc/0512039].
  * [71] N. Bartolo et al., _Science with the space-based interferometer LISA. IV: Probing inflation with gravitational waves_ , _JCAP_ 12 (2016) 026 [1610.06481].
  * [72] P. Amaro-Seoane et al., _Laser interferometer space antenna_ , 1702.00786.
  * [73] P.E. Dewdney, P.J. Hall, R.T. Schilizzi and T.J.L.W. Lazio, _The square kilometre array_ , _Proceedings of the IEEE_ 97 (2009) 1482.
  * [74] G. Janssen et al., _Gravitational wave astronomy with the SKA_ , _PoS_ AASKA14 (2015) 037 [1501.00127].
  * [75] E. Thrane and J.D. Romano, _Sensitivity curves for searches for gravitational-wave backgrounds_ , _Phys. Rev. D_ 88 (2013) 124032 [1310.5300].
  * [76] K. Schmitz, _New Sensitivity Curves for Gravitational-Wave Signals from Cosmological Phase Transitions_ , _JHEP_ 01 (2021) 097 [2002.04615].
  * [77] T. Vachaspati and A. Vilenkin, _Gravitational Radiation from Cosmic Strings_ , _Phys. Rev. D_ 31 (1985) 3052.
  * [78] A. Vilenkin and E.P.S. Shellard, _Cosmic Strings and Other Topological Defects_ , Cambridge University Press (7, 2000).
  * [79] V. Vanchurin, K.D. Olum and A. Vilenkin, _Scaling of cosmic string loops_ , _Phys. Rev. D_ 74 (2006) 063527 [gr-qc/0511159].
  * [80] C. Ringeval, M. Sakellariadou and F. Bouchet, _Cosmological evolution of cosmic string loops_ , _JCAP_ 02 (2007) 023 [astro-ph/0511646].
  * [81] K.D. Olum and V. Vanchurin, _Cosmic string loops in the expanding Universe_ , _Phys. Rev. D_ 75 (2007) 063521 [astro-ph/0610419].
  * [82] J.J. Blanco-Pillado, K.D. Olum and B. Shlaer, _The number of cosmic string loops_ , _Phys. Rev. D_ 89 (2014) 023512 [1309.6637].
  * [83] J.J. Blanco-Pillado and K.D. Olum, _Stochastic gravitational wave background from smoothed cosmic string loops_ , _Phys. Rev. D_ 96 (2017) 104046 [1709.02693].
  * [84] Y. Cui, M. Lewicki, D.E. Morrissey and J.D. Wells, _Probing the pre-BBN universe with gravitational waves from cosmic strings_ , _JHEP_ 01 (2019) 081 [1808.08968].
  * [85] W. Buchmuller, V. Domcke and K. Schmitz, _Stochastic gravitational-wave background from metastable cosmic strings_ , _JCAP_ 12 (2021) 006 [2107.04578].
  * [86] P. Auclair et al., _Probing the gravitational wave background from cosmic strings with LISA_ , _JCAP_ 04 (2020) 034 [1909.00819].
  * [87] Y. Cui, M. Lewicki and D.E. Morrissey, _Gravitational Wave Bursts as Harbingers of Cosmic Strings Diluted by Inflation_ , _Phys. Rev. Lett._ 125 (2020) 211302 [1912.08832].
  * [88] G. Lazarides, R. Maji and Q. Shafi, _Superheavy quasistable strings and walls bounded by strings in the light of NANOGrav 15 year data_ , _Phys. Rev. D_ 108 (2023) 095041 [2306.17788].
  * [89] A. Moursy and Q. Shafi, _Primordial monopoles, black holes and gravitational waves_ , 2405.04397.