11institutetext: Department of Electrical and Computer Engineering,
Sungkyunkwan University
11email<EMAIL_ADDRESS>
# Dynamic Retraining-Updating Mean Teacher for Source-Free Object Detection

Trinh Le Ba Khanh  0000-0003-1688-9003 Huy-Hung Nguyen  0000-0001-5394-9381
Long Hoang Pham  0000-0002-3240-657X
Duong Nguyen-Ngoc Tran  0000-0001-7537-6377 Jae Wook Jeon  Corresponding
author 0000-0003-0037-112X

###### Abstract

In object detection, unsupervised domain adaptation (UDA) aims to transfer
knowledge from a labeled source domain to an unlabeled target domain. However,
UDA’s reliance on labeled source data restricts its adaptability in privacy-
related scenarios. This study focuses on source-free object detection (SFOD),
which adapts a source-trained detector to an unlabeled target domain without
using labeled source data. Recent advancements in self-training, particularly
with the Mean Teacher (MT) framework, show promise for SFOD deployment.
However, the absence of source supervision significantly compromises the
stability of these approaches. We identify two primary issues, _(1)_
uncontrollable degradation of the teacher model due to inopportune updates
from the student model, and _(2)_ the student model’s tendency to replicate
errors from incorrect pseudo labels, leading to it being trapped in a local
optimum. Both factors contribute to a detrimental circular dependency,
resulting in rapid performance degradation in recent self-training frameworks.
To tackle these challenges, we propose the Dynamic Retraining-Updating (DRU)
mechanism, which actively manages the student training and teacher updating
processes to achieve co-evolutionary training. Additionally, we introduce
Historical Student Loss to mitigate the influence of incorrect pseudo labels.
Our method achieves state-of-the-art performance in the SFOD setting on
multiple domain adaptation benchmarks, comparable to or even surpassing
advanced UDA methods. The code will be released at
https://github.com/lbktrinh/DRU.

###### Keywords:

Domain Adaptive Object Detection Selective Retraining Mean Teacher Transformer

##  1 Introduction

Unsupervised Domain Adaptation (UDA) has garnered significant attention in
object detection [33, 29, 3, 24, 7], focusing on transferring knowledge from
source domains to target domains. UDA methods typically require access to both
labeled source and unlabeled target data simultaneously to achieve feature
alignment across domains. However, practical scenarios often involve privacy
concerns, limiting access to source domain data and thereby introducing new
challenges for UDA. In response, Source-Free Object Detection (SFOD) [15, 18,
14, 28, 4] has recently emerged as a promising research direction. SFOD aims
to adapt source-trained models to the unlabeled target domain without
requiring access to labeled source data.

Figure 1: The training curves of different Mean Teacher training strategies on
the validation set of Cityscapes $\to$ Foggy Cityscapes under the SFOD
setting. These varied strategies consistently show a degradation phenomenon:
the teacher model gradually degrades due to inappropriate updates from the
student model, while the student model experiences performance deterioration
due to inaccurate pseudo labels.

In SFOD, the absence of source supervision makes self-training using pseudo
labels conventional in this setting. Recent developments in self-training
utilizing the Mean Teacher (MT) [26] have shown great potential [4, 28, 14,
10]. These MT-based methods employ a teacher model to guide the student model.
The iterative process involves updating the teacher model via the Exponential
Moving Average (EMA) weight of the student model, followed by training the
student model using pseudo labels generated by the updating teacher model.
Despite some progress, iterative self-training approaches still encounter
decline issues due to the lack of labeled source data and noisy pseudo labels
arising from domain shifts.

We investigate two factors contributing to the deterioration of self-training
within the MT framework in SFOD: _(1)_ Inappropriate updates of the teacher
model can result in susceptibility to accumulating errors, as it continuously
aggregates improper knowledge from the unevolved student model; _(2)_ When the
teacher model makes errors and provides incorrect information, the student
model reproduces these mistakes without any correction, resulting in being
stuck at a local optimum. Both factors establish a harmful cyclical
relationship, where the teacher model offers poor guidance to the student
model, and the student model provides biased knowledge to the teacher model.
This cycle ultimately leads to the destruction of the self-training paradigm
due to the absence of purposeful evolution. As illustrated in Fig. 1,
adjusting the EMA weights or updating interval [26, 4, 28] does not entirely
solve the uncontrollable degradation in the self-training MT framework.
Moreover, searching for an optimal EMA hyper-parameter for model updating is
time-consuming.

Figure 2: The comparison of the conventional Mean Teacher (MT) framework
(_left_) and our Dynamic Retraining-Updating (DRU) method (_right_). Left: In
MT, the teacher model is continuously updated by a fixed interval $m$ ($m=1$
[26] or $m=s,(s>1)$ [4, 28]). Right: In DRU, the student model is dynamically
retrained and the teacher model is dynamically updated based on prediction
feedback. Additionally, the current student model is further supervised by the
historical student model.

In this study, we propose the Dynamic Retraining-Updating (DRU) method to
address the aforementioned instability. The overview of DRU is depicted in
Fig. 2. DRU actively manages student training and teacher updating processes
to overcome the harmful interdependency inherent in the current self-training
MT paradigm. Specifically, guided by the feedback on the student’s
evolutionary state, the student is dynamically retrained if trapped in a local
optimum, while the teacher model is dynamically updated based on the evolved,
retrained student. With this mechanism, the student can get out of sub-optimal
states biased to inaccurate pseudo labels, and the teacher can accumulate
valuable insights from the evolved student, promoting co-evolution in
training. Furthermore, we introduce an additional Historical Student Loss to
mitigate rapid performance degradation caused by incorrect pseudo labels. This
loss leverages knowledge from the historical student model to provide further
supervision to the current student. With additional guidance from this loss,
the student model can reduce the influence of incorrect pseudo labels
generated by the deteriorating teacher model, resulting in more stable
training.

Through efforts, DRU achieves state-of-the-art (SOTA) results across diverse
SFOD scenarios, comparable to or even surpassing advanced UDA methods.
Extensive experiments demonstrate that DRU effectively addresses the
degradation problem inherent in self-training within the MT framework, leading
to significant stability and adaptability during training. Our contributions
can be summarized:

  * •

We explore the deterioration issue in the self-training MT-based framework,
particularly when employed with transformer-based detectors, which are
susceptible to domain shifts. To our knowledge, this is the first study to
investigate the effectiveness of transformer-based detectors for SFOD
problems.

  * •

We introduce the Dynamic Retraining-Updating (DRU) method to address the
degradation observed in the MT framework. DRU actively controls student
training and teacher updating to promote co-evolutionary training.

  * •

We propose the Historical Student Loss to prevent severe performance decline
caused by noisy pseudo labels.

  * •

On multiple benchmarks, our DRU achieves SOTA performance compared to recent
SFOD methods and surpasses several advanced UDA methods.

##  2 Related Works

###  2.1 Object Detection

Object detection is a fundamental task in computer vision. Over the years,
CNN-based detectors have attracted extensive attention [23, 17, 22, 27].
However, their reliance on hand-designed components, _e.g_. non-maximum
suppression, limits their ability to be trained end-to-end. Transformer-based
detectors [1, 39, 34] have recently emerged as an alternative for end-to-end
object detection. While these detectors demonstrate robustness when applied to
data from their training distribution, they experience degradation when faced
with domain shifts. Our objective is to enhance the model’s adaptability to
new domains without requiring annotations. In this study, we adopt Deformable
DETR [39] as our baseline detector due to its streamlined simplicity and the
adaptable transfer-learning capability of the attention mechanism.

###  2.2 Source-Free Object Detection

Source-Free Object Detection (SFOD) is introduced to address adaptation
scenarios involving privacy concerns. Without source domain supervision,
applying existing UDA methods [3, 29, 33] to SFOD may lead to undesirable
results, prone to significant instability during training. SED [15] represents
a pioneering attempt to introduce pseudo label self-training through self-
entropy descent. A2SFOD [4] integrates an adversarial component into an MT
framework to align feature spaces of source-similar and source-dissimilar
images. IRG [28] introduces a new contrastive loss guided by the Instance
Relation Graph network to enhance target domain representations. To ensure
stable self-training, HCL [10] constrains current models to prioritize
knowledge aligned with historical models. PETS [18] proposes a multi-teacher
framework consisting of a static teacher, a dynamic teacher, and a student
model to address self-training instability. While most SFOD methods leverage
the MT framework [26], they overlook the essential co-evolution between
student and teacher models, which is critical for stabilizing self-training
paradigms. Based on observations, we introduce the Dynamic Retraining-Updating
method to promote the improvement of both student and teacher models through
training progress.

###  2.3 Selective Retraining

Due to the large number of parameters, optimizing a transformer-based model is
prone to over-fitting on a small training set and getting stuck in a local
optimum. In SFOD, this problem becomes worse due to incorrect pseudo labels.
Recent approaches suggest that retraining provides an efficient solution to
address this issue [21, 38]. MRT [38] adopts periodic retraining within the MT
framework to alleviate the impact of incorrect pseudo labels in UDA tasks.
However, they rely on labeled source data to generate optimal initialization
weights. We propose a new approach that dynamically retrains the student model
to converge to a better local minimum when source data is unavailable.

Figure 3: Overview of our Dynamic Retraining-Updating (DRU) method, which is
built upon the Mean Teacher framework. DRU employs dynamic retraining of the
student model and dynamic updating of the teacher model based on the student’s
evolution assessment. The student model is further supervised by Historical
Student Loss $\mathcal{L}_{his}$. The details of the training process will be
presented in Algorithm 1

##  3 Preliminary

###  3.1 Problem Definition

In this section, we define the problem formulation of SFOD. Let
$D_{s}=\\{(x_{s},y_{s})\\}$ denote the labeled source domain data, and
$D_{t}=\\{x_{t}\\}$ represent the unlabeled target domain data. Here, $x$
refers to an image, while $y=(b,c)$ denotes an annotation containing the
bounding box $b$ and category $c$. UDA methods require labeled source data
$D_{s}$ and unlabeled target data $D_{t}$ for adaptation. In contrast, SFOD
settings utilize only a source-trained model $G$ (trained on labeled source
data $D_{s}$) and the unlabeled target data $D_{t}$ for adaptation.

###  3.2 Mean Teacher Self-training

The Mean Teacher (MT) framework [26] consists of a teacher model
$G_{\bar{\theta}}^{tea}$ and a student model $G_{\theta}^{stu}$ with identical
architectures and initialization weights. The student model is optimized with
pseudo labels provided by the teacher model, while the teacher model is
progressively updated through EMA from the student model. In addition, the
student and teacher models receive strong and weak augmentations. Enhanced
consistency in their predictions improves the detection of target data [6,
19].

Specifically, the student model is optimized using detection loss
$\mathcal{L}_{det}$ computed from the student predictions ($\hat{b}$,
$\hat{c}$) with corresponding pseudo labels ($b_{t}$, $c_{t}$) obtained from
the teacher model after Hungarian matching, as detailed in [39]:

$\mathcal{L}_{det}=\mathcal{L}_{cls}(\hat{c},c_{t})+\mathcal{L}_{L1}(\hat{b},b_{t})+\mathcal{L}_{giou}(\hat{b},b_{t})$
(1)

where $\mathcal{L}_{cls}$ denotes the classification loss, $\mathcal{L}_{L1}$
and $\mathcal{L}_{giou}$ represent the regression losses.

The teacher model is progressively updated via EMA from the student model to
enhance detection capability:

$\bar{\theta}_{t}\leftarrow\alpha\bar{\theta}_{t-1}+(1-\alpha)\theta_{t}$ (2)

where $\bar{\theta}_{t}$ and $\theta_{t}$ represent the parameters of the
teacher and student models at the $t^{th}$ iteration, respectively, and
$\alpha$ denotes a hyper-parameter update weight.

###  3.3 Masked Image Modeling

Because no ground truth is available in the target domain, utilizing noisy
pseudo labels may result in model memorization. Encouraging the model to learn
the context relations within the target domain images may mitigate the risk of
memorization. Therefore, we adapted the Masked Image Consistency (MIC) [9]
module to facilitate context relations learning. MIC employs masked images to
improve context learning for the target domain. The masked target image
$x^{M}$ is acquired through element-wise multiplication of the mask and the
image:

$x^{M}=\mathcal{M}\odot x^{S}$ (3)

where $\mathcal{M}$ represents a patch mask generated from a uniform
distribution, and $x^{S}$ denotes the strongly augmented target image from the
student branch.

During the adaptation process, the masked image $x^{M}$ operates in the same
way as the strongly augmented image $x^{S}$ within the student branch, as
illustrated in Fig. 3. Let $\mathcal{L}_{mask}$ represent the loss from
$x^{M}$ and $\mathcal{L}_{strong}$ denote the loss from $x^{S}$. The total
loss is defined as:

$\mathcal{L}_{teach}=\mathcal{L}_{strong}+\mathcal{L}_{mask}$ (4)

##  4 Proposed Method

###  4.1 Overview

Our proposed Dynamic Retraining-Updating (DRU) method aligns with the
structure of the MT framework as described in Sec. 3.2. The overview of DRU is
presented in Fig. 3. DRU introduces a Dynamic Retraining-Updating mechanism to
actively manage student training and teacher updating based on the student’s
evolution. Specifically, we evaluate the student’s progress by estimating its
prediction uncertainty over historical data $D_{his}$. Depending on the
received feedback, if the student has evolved, the teacher experiences Teacher
Dynamic Updating. In contrast, if the student has been trapped, it undergoes
Student Dynamic Retraining. Additionally, the student model receives further
supervision from Historical Student Loss, leveraging knowledge from the
historical student model. The details of the training process can be seen in
Algorithm 1.

Algorithm 1 Dynamic Retraining-Updating training process

Teacher model $G_{\bar{\theta}}^{tea}$, student model $G_{\theta}^{stu}$,
unlabeled target data $D_{t}$, uncertainty feedback $U$, and meta-iteration
$M$.

Optimized teacher model $G_{\bar{\theta}}^{tea}$

Empty buffer $D_{his}$; Copy $G_{\theta}^{stu}$ as $G_{\theta_{init}}^{stu}$;
Index $i=0$

for image batch $x_{b}$ in $D_{t}$ do

Update $G_{\theta}^{stu}$ by Eq. 8 with $x_{b}$, $G_{\bar{\theta}}^{tea}$ and
$G_{\theta_{init}}^{stu}$ $\triangleright$ with Historical Student Loss

Append $x_{b}$ to $D_{his}$; $i++$

if $i<M$ then

if $U[G_{\theta}^{stu}(D_{his})]<U[G_{\theta_{init}}^{stu}(D_{his})]$ then
$\triangleright$ Student evolved

Update $G_{\bar{\theta}}^{tea}$ by Eq. 5 with $G_{\bar{\theta}}^{tea}$ and
$G_{\theta}^{stu}$ $\triangleright$ Teacher Dynamic Updating

Reset $D_{his}$; Copy $G_{\theta}^{stu}$ as $G_{\theta_{init}}^{stu}$; $i=0$

else$\triangleright$ Student trapped in a local optimum

Reinitialize $G_{\theta}^{stu}$ with $G_{\theta}^{stu}$ and
$G_{\theta_{init}}^{stu}$ $\triangleright$ Student Dynamic Retraining

Reset $D_{his}$; Copy $G_{\theta}^{stu}$ as $G_{\theta_{init}}^{stu}$; $i=0$

return $G_{\bar{\theta}}^{tea}$

###  4.2 Dynamic Retraining-Updating Mechanism

Intuitively, the co-evolution of student and teacher models is crucial for
overcoming degradation, where the teacher appropriately integrates knowledge
from the evolved student. Therefore, the evolution of the student plays a
critical role. Inspired by research on semantic segmentation [36], which
suggests that the performance of the student model on historical samples can
reflect its evolution, we then aim to assess the student’s progress by
evaluating its performance over historical data $D_{his}$. In the scenario
where target labels are unavailable for evaluation, we can assess the
student’s performance based on its prediction uncertainty [13, 36]. Lower
uncertainty in the student’s output signifies higher adaptability to the
target domain. To estimate the student’s prediction uncertainty, we leverage
the inherent structure of the decoder, where the variation of outputs across
all decoder layers could give us a close estimate [20]. Specifically, let
consider $O_{D}\in\mathbb{R}^{L\times B\times Q\times C}$ as the outputs of
all decoder layers, where $L$, $B$, $Q$, and $C$ denote the number of decoder
layers, batch size, number of queries, and number of classes, respectively. By
computing the variance along the $L$ dimension in $O_{D}$, we yield
$u\in\mathbb{R}^{B\times Q\times C}$, which represents the student’s
prediction uncertainty for an image batch $x_{b}\in D_{his}$. Based on the
feedback $U=\\{u\\}$, we actively update the teacher and retrain the student,
as described in Algorithm 1.

####  4.2.1 Student Dynamic Retraining

Due to the lack of ground truths, the student model is only updated with noisy
pseudo labels, which always contain bias caused by domain shifts. Without
corrective measures, the student replicates this bias during training, making
it vulnerable to being stuck in a sub-optimal state. Consequently, the teacher
model could also be affected as it is continuously updated from the student
model, generating even worse pseudo labels. To overcome this, we introduce
dynamically selective retraining of the student model.

Specifically, during a training period $M$, if the student model cannot escape
a downward trend, we selectively retrain parts of the student model. Despite
being trapped in a local optimum due to noisy pseudo labels, the current
student model has acquired valuable feature knowledge from the target domain
during learning, thereby benefiting the detector. Therefore, while
continuously updating parameters of the backbone and encoder modules
throughout training progress, we re-initialize the decoder of the current
student model $G_{\theta_{t}}^{stu}$ with the weights of the decoder from the
historical student model $G_{\theta_{t-M}}^{stu}$. In this way, the retrained
decoder can leave the local optimum, guided by better-trained backbone and
encoder components. We verify this assumption in ablation studies.

####  4.2.2 Teacher Dynamic Updating

The teacher model can be viewed as an ensemble of multiple temporal student
models. Uncontrolled updates without regard to student conditions can lead to
accumulating errors in the teacher model, making it susceptible to detection
deterioration. As the student model acquires new knowledge and becomes more
reliable to the target domain, updating the teacher model facilitates the
integration of valuable insights from the student model, thereby enhancing its
detection capability. Let $\tilde{m}$ denote the dynamic update interval. The
EMA described in Eq. 2 is reformulated as follows:

$\bar{\theta}_{t}\leftarrow\alpha\bar{\theta}_{t-\tilde{m}}+(1-\alpha)\theta_{t}$
(5)

###  4.3 Historical Student Loss

In DRU, the historical student acts as an additional teacher, setting a
performance lower bound for updating the current student model. Suppose the
student model faces a severe performance decrease due to a deteriorating
teacher model. In that case, the historical student ensures that the student
model maintains a forward-reaching trajectory, thereby preventing rapid
performance deterioration.

Due to inaccuracies in predicted locations and to prevent inconsistencies with
pseudo labels bounding box from the teacher model, the Historical Student Loss
$\mathcal{L}_{his}$ includes only the classification task:

$\mathcal{L}_{his}=\mathcal{L}_{cls}(\hat{c},{c}_{his})$ (6)

To mimic the historical performance and reduce the influence of incorrect
pseudo labels, $\mathcal{L}_{cls}$ is adjusted as follows, inspired by Quality
Focal Loss [16]:

$\mathcal{L}_{cls}(\hat{c},{c}_{his})=\sum_{i=1}^{N_{pos}}\left|{s}_{i}-\hat{c_{i}}\right|^{\gamma}BCE\left(\hat{c_{i}},{s}_{i}\right)+\sum_{j=1}^{N_{neg}}\hat{c}_{j}^{\gamma}BCE\left(\hat{c_{j}},0\right)$
(7)

where $\hat{c_{i}}$, ${s}_{i}$ represent the predicted probability of current
student $G_{\theta_{t}}^{stu}$ and the corresponding confidence score of class
${c}_{his}$ from historical student $G_{\theta_{his}}^{stu}$ of the $i^{th}$
example, respectively (More details are included in supplementary material).

The $\mathcal{L}_{his}$ operates in the same way as $\mathcal{L}_{teach}$ in
Eq. 4. For simplicity, the overall objective loss for the student model is:

$\mathcal{L}_{total}=\mathcal{L}_{teach}+\mathcal{L}_{his}$ (8)

##  5 Experiments

###  5.1 Experimental Setup

####  5.1.1 Datasets

Following the setting of existing SFOD [15, 4, 18] and UDA [29, 33, 7]
methods, we evaluate the performance of DRU in three popular domain adaptation
benchmarks from four public datasets: Cityscapes [5], Foggy Cityscapes [25],
Sim10k [12], and BDD100k [32].

  * •

Normal to Foggy Adaptation Cityscapes dataset is gathered from urban
environments. It includes 3,475 annotated images, with 2,975 allocated for
training and the remaining 500 reserved for evaluation. Foggy Cityscapes is
generated from Cityscapes using a fog synthesis algorithm. We employ
Cityscapes as the source domain and Foggy Cityscapes with the highest fog
density (0.02) as the target domain.

  * •

Cross Scene Adaptation BDD100k is a large-scale driving dataset. We employ the
BDD100k daytime subset as the target domain, consisting of 36,728 training
images and 5,258 validation images with ground truth. The Cityscapes remains
as the source domain.

  * •

Synthetic to Real Adaptation Sim10k comprises 10,000 synthetic images
generated from the GTA game engine. We utilize Sim10k as the source data and
“car” instances from Cityscapes as the target data.

####  5.1.2 Implementation Details

Our method is built upon Deformable DETR [39]. We set the smoothing parameter
$\alpha=0.999$ for the EMA ($\alpha=0.9996$ for Cityscapes $\to$ BDD100k). The
threshold is defined as $0.3$ for filtering pseudo labels. A meta-iteration
for Dynamic Retraining-Updating is set to $M=5$. The network is optimized with
the Adam optimizer with an initial learning rate of $2\times 10^{-4}$. The
batch size is fixed at $8$ for all domain adaptation scenarios. The data
augmentation techniques comprise random horizontal flipping for weak
augmentation and random color jittering, grayscaling, and Gaussian blurring
for strong augmentation [38]. For the MIC module, we use a patch size $b=64$
and a mask ratio $r=0.5$ as the objects of interest. We utilize Mean Average
Precision (mAP) with a threshold of $0.5$ as the evaluation metric. All
experiments are conducted on an NVIDIA Quadro RTX 8000 GPU (48GB).

Table 1: Results of different UDA and SFOD methods for Normal to Foggy Adaptation (Cityscapes $\to$ Foggy Cityscapes). “Source Only” refers to the source-trained model. FRCNN denotes Faster R-CNN, and DefDETR represents Deformable DETR. | Method | Detector | person | rider | car | truck | bus | train | motor | bicycle | mAP
---|---|---|---|---|---|---|---|---|---|---|---
| Source Only | DefDETR | 40.0 | 41.2 | 47.0 | 13.0 | 29.1 | 6.5 | 21.5 | 38.0 | 29.5
UDA | SW-Faster [24] | FRCNN | 32.3 | 42.2 | 47.3 | 23.7 | 41.3 | 27.8 | 28.3 | 35.4 | 34.8
CR-DA-DET [31] | FRCNN | 32.9 | 43.8 | 49.2 | 27.2 | 45.1 | 36.4 | 30.3 | 34.6 | 37.4
TIA [37] | FRCNN | 34.8 | 46.3 | 49.7 | 31.1 | 52.1 | 48.6 | 37.7 | 38.1 | 42.3
PT [2] | FRCNN | 40.2 | 48.8 | 59.7 | 30.7 | 51.8 | 30.6 | 35.4 | 44.5 | 42.7
TDD [8] | FRCNN | 39.6 | 47.5 | 55.7 | 33.8 | 47.6 | 42.1 | 37.0 | 41.4 | 43.1
SFA [29] | DefDETR | 46.5 | 48.6 | 62.6 | 25.1 | 46.2 | 29.4 | 28.3 | 44.0 | 41.3
MTTrans [33] | DefDETR | 47.7 | 49.9 | 65.2 | 25.8 | 45.9 | 33.8 | 32.6 | 46.5 | 43.4
DA-DETR [35] | DefDETR | 49.9 | 50.0 | 63.1 | 24.0 | 45.8 | 37.5 | 31.6 | 46.3 | 43.5
SFOD | SED(Mosaic) [15] | FRCNN | 33.2 | 40.7 | 44.5 | 25.5 | 39.0 | 22.2 | 28.4 | 34.1 | 33.5
A2SFOD [4] | FRCNN | 32.3 | 44.1 | 44.6 | 28.1 | 34.3 | 29.0 | 31.8 | 38.9 | 35.4
LODS [14] | FRCNN | 34.0 | 45.7 | 48.8 | 27.3 | 39.7 | 19.6 | 33.2 | 37.8 | 35.8
PETS [18] | FRCNN | 42.0 | 48.7 | 56.3 | 19.3 | 39.3 | 5.5 | 34.2 | 41.6 | 35.9
IRG [28] | FRCNN | 37.4 | 45.2 | 51.9 | 24.4 | 39.6 | 25.2 | 31.5 | 41.6 | 37.1
Ours | DefDETR | 48.3 | 51.5 | 62.5 | 26.2 | 43.2 | 34.1 | 34.2 | 48.6 | 43.6

###  5.2 Comparison with SOTA Methods

This section evaluates the proposed DRU model across three challenging domain
adaptation scenarios and compares the results with other recent SOTA SFOD and
UDA methods.

####  5.2.1 Normal to Foggy Adaptation

Weather conditions frequently fluctuate, presenting significant challenges.
Object detectors must maintain reliability under all circumstances. Therefore,
we evaluate the detectors’ robustness to weather changes by transitioning from
Cityscapes to Foggy Cityscapes. As indicated in Tab. 1, DRU surpasses advanced
SOTA SFOD methods by a considerable margin (43.6% compared to the closest SOTA
37.1%).

Table 2: Results of Cross Scene Adaptation (Cityscapes $\to$ BDD100k) | Method | Detector | truck | car | rider | person | motor | bicycle | bus | mAP
---|---|---|---|---|---|---|---|---|---|---
| Source Only | DefDETR | 18.9 | 58.2 | 28.3 | 42.0 | 15.7 | 18.8 | 21.7 | 29.1
UDA | DA-Faster [3] | FRCNN | 14.3 | 44.6 | 26.5 | 29.4 | 15.8 | 20.6 | 16.8 | 24.0
SW-Faster [24] | FRCNN | 15.2 | 45.7 | 29.5 | 30.2 | 17.1 | 21.2 | 18.4 | 25.3
CR-DA-DET [31] | FRCNN | 19.5 | 46.3 | 31.3 | 31.4 | 17.3 | 23.8 | 18.9 | 26.9
AQT [11] | DefDETR | 17.3 | 58.4 | 33.0 | 38.2 | 16.9 | 23.5 | 18.4 | 29.4
O2net [7] | DefDETR | 20.4 | 58.6 | 31.2 | 40.4 | 14.9 | 22.7 | 25.0 | 30.5
MTTrans [33] | DefDETR | 25.1 | 61.5 | 30.1 | 44.1 | 17.7 | 23.0 | 26.9 | 32.6
MRT [38] | DefDETR | 24.7 | 63.7 | 30.9 | 48.4 | 20.2 | 22.6 | 25.5 | 33.7
SFOD | SED(Mosaic) [15] | FRCNN | 20.6 | 50.4 | 32.6 | 32.4 | 18.9 | 25.0 | 23.4 | 29.0
PETS [18] | FRCNN | 19.3 | 62.4 | 34.5 | 42.6 | 17.0 | 26.3 | 16.9 | 31.3
A2SFOD [4] | FRCNN | 26.6 | 50.2 | 36.3 | 33.2 | 22.5 | 28.2 | 24.4 | 31.6
Ours | DefDETR | 27.1 | 62.7 | 36.9 | 45.8 | 22.7 | 32.5 | 28.1 | 36.6

####  5.2.2 Cross Scene Adaptation

Scene configurations dynamically change in real-world applications, especially
in automated driving contexts. Hence, the model’s performance in cross-scene
adaptation is crucial. As depicted in Tab. 2, DRU achieves SOTA results
compared to recent SFOD and UDA methods with substantial improvements,
surpassing the previous SOTA method by 5.0%. Furthermore, it enhances the
results of five out of seven classes in the target domain.

Table 3: Results of Synthetic to Real Adaptation (Sim10k $\to$ Cityscapes) | Method | Detector | AP of car
---|---|---|---
| Source Only | DefDETR | 48.9
UDA | TDD [8] | FRCNN | 53.4
PT [2] | FRCNN | 55.1
SFA [29] | DefDETR | 52.6
O2net [7] | DefDETR | 54.1
DA-DETR [35] | DefDETR | 54.7
MTTrans [33] | DefDETR | 57.9
MTM [30] | DefDETR | 58.1
SFOD | SED(Mosaic) [15] | FRCNN | 43.1
IRG [28] | FRCNN | 43.2
A2SFOD [4] | FRCNN | 44.0
PETS [18] | FRCNN | 57.8
Ours | DefDETR | 58.7

Table 4: Ablation studies of adding modules to MT framework on Cityscapes $\to$ Foggy Cityscapes. “Src” denotes the Source Only trained model. “MT” represents the Mean Teacher baseline. “MIC”, “$\mathcal{L}_{his}$”, and “DRU” denote the Masked Image Modeling, Historical Student Loss, and Dynamic Retraining-Updating, respectively. Src | MT | MIC | $\mathcal{L}_{his}$ | DRU | mAP | gain
---|---|---|---|---|---|---
$\checkmark$ |  |  |  |  | 29.5 |
$\checkmark$ | $\checkmark$ |  |  |  | 37.4 |  $+$7.9
$\checkmark$ | $\checkmark$ | $\checkmark$ |  |  | 39.8 |  $+$10.3
$\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ |  | 41.3 |  $+$11.8
$\checkmark$ | $\checkmark$ | $\checkmark$ |  | $\checkmark$ | 40.9 |  $+$11.4
$\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | 43.6 |  $+$14.1

####  5.2.3 Synthetic to Real Adaptation

Synthetic images and labels can be automatically generated, providing an
alternative to address the challenges of data collection and labeling.
Therefore, enabling the model to transfer from synthetic to real images is
advantageous. As shown in Tab. 4, we evaluate DRU in the synthetic to real
adaptation scenario, achieving a SOTA performance of 58.7%. DRU surpasses the
latest SOTA SFOD method by 0.9% and is even comparable with recent SOTA UDA
methods.

###  5.3 Ablation Studies

####  5.3.1 Modules Analysis

To evaluate the effectiveness of each component in our method, we conduct
ablation studies by progressively incorporating individual components into
DRU. Table 4 presents some observations: _(1)_ Integrating the baseline MT
framework directly into a source-trained model leads to remarkable progress in
the target domain (+7.9%). This emphasizes the significance of the MT
framework in self-training adaptation, yet there is still room for further
enhancement. _(2)_ The incorporation of MIC into MT results in a performance
increase of 10.3%, attributed to its effective learning of context relations
in the target domain. _(3)_ Using additional Historical Student Loss reduces
the influence of incorrect pseudo labels, resulting in an 11.8% gain. _(4)_
Finally, incorporating a Dynamic Retraining-Updating mechanism effectively
manages teacher updating and student training, resulting in a 14.1%
improvement.

####  5.3.2 Dynamic Retraining-Updating Analysis

We analyze the usefulness of selective retraining. When the student model
becomes trapped in a local optimum, we selectively re-initialize specific
components while preserving others as enhanced parts. Table 5(a) shows that
maintaining updates to the backbone and encoder while re-initializing the
decoder produces the optimal result. This result suggests that the model has
acquired meaningful feature representations in the backbone and encoder,
whereas the decoder’s performance deteriorates due to incorrect pseudo labels.
We also evaluate the effectiveness of fixed and dynamic update intervals for
the teacher model in Tab. 5(b). With appropriate dynamic updating, the teacher
model can effectively aggregate the valuable knowledge from the evolved
student model, achieving the best result. Additionally, it addresses the
inconvenience and complication of manually searching for optimal hyper-
parameters. We examine various values of meta-iteration $M$, which is shown in
Tab. 5(c), and find that the model yields the best performance with $M=5$.
When $M$ is too small or too large, the model fails to update efficiently.

Table 5: Ablation studies of Dynamic Retraining-Updating on Cityscapes $\to$
Foggy Cityscapes. (a) Selective retraining, “Bac+Enc” denotes retrained
Backbone and Encoder, “Dec” means retrained Decoder, and “No” indicates
keeping training all modules. (b) Dynamic updating, “$\tilde{m}$” means
dynamic update interval. (c) Meta-iteration $M$.

(a) Selective retraining Retrained | No | Bac+Enc | Dec
---|---|---|---
mAP | 41.2 | 39.9 | 43.6

(b) Dynamic updating $i$ | 1 | 5 | 10 | $\tilde{m}$
---|---|---|---|---
mAP | 41.8 | 43.4 | 41.0 | 43.6

(c) Meta-iteration $M$ $M$ | 2 | 5 | 10 | 20
---|---|---|---|---
mAP | 41.9 | 43.6 | 42.6 | 41.5

###  5.4 Result Analysis

####  5.4.1 Training Stability

To validate the effectiveness of our DRU in training stability, we present the
mAP curves of the student and teacher models on the validation dataset
throughout training. As shown in Fig. 4 (a), the mAP curves of both student
and teacher models in conventional MT initially steadily increase but rapidly
degrade afterward, indicating the poor stability of the foundational self-
training MT approach. In Fig. 4 (b), the mAP curves, supplemented with
Historical Student Loss, exhibit a steady initial rise followed by a slight
decrease, highlighting the effectiveness of Historical Student Loss in
mitigating severe performance degradation caused by incorrect pseudo labels.
Finally, in Fig. 4 (c), incorporating the Dynamic Retraining-Updating
mechanism results in accelerated improvement and enhanced stability,
demonstrating its capability to promote the co-evolution of student and
teacher models through the training process. Furthermore, as illustrated in
Fig. 4 (c), (d), similar training curves across different domain adaptation
scenarios confirm the effectiveness of our method in stabilizing self-
training.

We further analyze the stability of dynamically retraining the student model
and updating the teacher model without using Historical Student Loss. As
depicted in Fig. 5 (a), appropriate dynamic updates of the teacher model
contribute to training stability and later enhance performance. However, the
student cannot escape the downward trend, resulting in a subsequent decrease
in performance. In Fig. 5 (b), with additional dynamic parameter re-
initialization, the student model can converge to a better local optimum,
leading to gradual performance improvements, particularly in the later stages
of the training process.

Figure 4: (a), (b), (c) The training curves for adding modules to MT on
Cityscapes $\to$ Foggy Cityscapes. (d) The training curves of our method on
Sim10k $\to$ Cityscapes.  Figure 5: The training curves of student and teacher
models with Dynamic Retraining-Updating mechanism on Cityscapes $\to$ Foggy
Cityscapes

####  5.4.2 Visualization

In Fig. 6, the predictions of the student and teacher models are visually
depicted at various training epochs in the Cityscapes $\to$ Foggy Cityscapes
task. Initially, there were missing predictions for objects. However, as
training progressed, more objects were detected, and the predictions gradually
became more accurate. This observation confirms the effectiveness of our
method in preventing the deterioration phenomenon in self-training and
achieving co-evolution in both student and teacher models through training
progress. We further illustrate the predictions of our method on more
challenging adaptation tasks, as depicted in Fig. 7. The positive results
suggest that our approach performs properly in many self-training adaptation
scenarios, even with limited source-trained models.

Figure 6: Detection results of our student and teacher models at different
epochs on Cityscapes $\to$ Foggy Cityscapes. Our DRU shows the gradual
improvement of both the student and teacher models through the adaptation
process.  Figure 7: Detection results of our DRU on different domain
adaptation scenarios

##  6 Conclusion

This paper focuses on addressing the challenges of domain adaptive object
detection in scenarios where the source domain is unavailable. We investigate
the causes of the deterioration issue in the self-training Mean Teacher
framework and propose relevant enhancement approaches. Specifically, we
introduce the Dynamic Retraining-Update mechanism to promote the co-evolution
in student and teacher models. Also, we propose the Historical Student Loss to
mitigate the impact of noisy pseudo labels. Across various SFOD benchmarks,
our method significantly enhances the stability and adaptability of the self-
training paradigm, achieving state-of-the-art performances that are even
comparable to advanced UDA methods. We expect our research can provide new
insights and improve the effectiveness of self-training methods in more
intricate scenarios.

## Acknowledgements

This work was supported by the Institute of Information & Communications
Technology Planning & Evaluation(IITP) grant funded by the Korea
government(MSIT) (No. 2021-0-01364, An intelligent system for 24/7 real-time
traffic surveillance on edge devices)

## References

  * [1] Carion, N., Massa, F., Gabriel Synnaeve, N.U., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: ECCV. pp. 213–229 (2020)
  * [2] Chen, M., Chen, W., Yang, S., Song, J., Wang, X., Zhang, L., Yan, Y., Qi, D., Zhuang, Y., Xie, D., Pu, S.: Learning domain adaptive object detection with probabilistic teacher. In: ICML. pp. 3040–3055 (2022)
  * [3] Chen, Y., Li, W., Sakaridis, C., Dai, D., Gool, L.V.: Domain adaptive faster r-cnn for object detection in the wild. In: CVPR. pp. 3339–3348 (2018)
  * [4] Chu, Q., Li, S., Chen, G., Li, K., Li, X.: Adversarial alignment for source free object detection. In: AAAI. vol. 37, pp. 452–460 (2023)
  * [5] Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: CVPR. pp. 3213–3223 (2016)
  * [6] Deng, J., Li, W., Chen, Y., Duan, L.: Unbiased mean teacher for cross-domain object detection. In: CVPR. pp. 4091–4101 (2021)
  * [7] Gong, K., Li, S., Li, S., Zhang, R., Liu, C.H., Chen, Q.: Improving transferability for domain adaptive detection transformers. In: ACM MM. pp. 1543–1551 (2022)
  * [8] He, M., Wang, Y., Wu, J., Wang, Y., Li, H., Li, B., Gan, W., Wu, W., Qiao, Y.: Cross domain object detection by target-perceived dual branch distillation. In: CVPR. pp. 9570–9580 (2022)
  * [9] Hoyer, L., Dai, D., Wang, H., Gool, L.V.: Mic: Masked image consistency for context-enhanced domain adaptation. In: CVPR. pp. 11721–11732 (2023)
  * [10] Huang, J., Guan, D., Xiao, A., Lu, S.: Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. In: NeurIPS. vol. 34, pp. 3635–3649 (2021)
  * [11] Huang, W.J., Lu, Y.L., Lin, S.Y., Xie, Y., Lin, Y.Y.: Aqt: Adversarial query transformers for domain adaptive object detection. In: IJCAI. pp. 972–979 (2022)
  * [12] Johnson-Roberson, M., Barto, C., Mehta, R., Sridhar, S.N., Rosaen, K., Vasudevan, R.: Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks? In: ICRA. pp. 746–753 (2017)
  * [13] Karim, N., Mithun, N.C., Rajvanshi, A., pang Chiu, H., Samarasekera, S., Rahnavard, N.: C-sfda: A curriculum learning aided self-training framework for efficient source free domain adaptation. In: CVPR. pp. 24120–24131 (2023)
  * [14] Li, S., Ye, M., Zhu, X., Zhou, L., Xiong, L.: Source-free object detection by learning to overlook domain style. In: CVPR. pp. 8014–8023 (2022)
  * [15] Li, X., Chen, W., Xie, D., Yang, S., Yuan, P., Pu, S., Zhuang, Y.: A free lunch for unsupervised domain adaptive object detection without source data. In: AAAI. vol. 35, pp. 8474–8481 (2021)
  * [16] Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., Yang, J.: Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. In: NeurIPS. vol. 33, pp. 21002–21012 (2020)
  * [17] Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: CVPR. pp. 2117–2125 (2017)
  * [18] Liu, Q., Lin, L., Shen, Z., Yang, Z.: Periodically exchange teacher-student for source-free object detection. In: ICCV. pp. 6414–6424 (2023)
  * [19] Liu, Y.C., Ma, C.Y., He, Z., Kuo, C.W., Chen, K., Zhang, P., Wu, B., Kira, Z., Vajda, P.: Unbiased teacher for semi-supervised object detection. In: ICLR (2021)
  * [20] Munir, M.A., Khan, S., Khan, M.H., Ali, M., Khan, F.S.: Cal-detr: Calibrated detection transformer. In: NeurIPS. vol. 36 (2023)
  * [21] Qu, M., Wu, Y., Liu, W., Gong, Q., Liang, X., Russakovsky, O., Zhao, Y., Wei, Y.: Siri: A simple selective retraining mechanism for transformer-based visual grounding. In: ECCV. pp. 546–562 (2022)
  * [22] Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: CVPR. pp. 779–788 (2016)
  * [23] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: NeurIPS. vol. 28, pp. 4516–4526 (2015)
  * [24] Saito, K., Ushiku, Y., Harada, T., Saenko, K.: Strong-weak distribution alignment for adaptive object detection. In: CVPR. pp. 6956–6965 (2019)
  * [25] Sakaridis, C., Dai, D., Gool, L.V.: Semantic foggy scene understanding with synthetic data. IJCV 126(9), 973–992 (2018)
  * [26] Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: NeurIPS. vol. 30 (2017)
  * [27] Tian, Z., Shen, C., Chen, H., He, T.: Fcos: Fully convolutional one-stage object detection. In: ICCV. pp. 9627–9636 (2019)
  * [28] VS, V., Oza, P., Patel, V.M.: Instance relation graph guided source-free domain adaptive object detection. In: CVPR. pp. 3520–3530 (2023)
  * [29] Wang, W., Cao, Y., Zhang, J., He, F., Zha, Z., Wen, Y., Tao, D.: Exploring sequence feature alignment for domain adaptive detection transformers. In: ACM MM. pp. 1730–1738 (2021)
  * [30] Weng, W., Yuan, C.: Mean teacher detr with masked feature alignment: A robust domain adaptive detection transformer framework. In: AAAI. vol. 38, pp. 5912–5920 (2024)
  * [31] Xu, C.D., Zhao, X.R., Jin, X., Wei, X.S.: Exploring categorical regularization for domain adaptive object detection. In: CVPR. pp. 11724–11733 (2020)
  * [32] Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., Darrell1, T.: Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In: CVPR. pp. 2636–2645 (2020)
  * [33] Yu, J., Liu, J., Wei, X., Zhou, H., Nakata, Y., Gudovskiy, D., Okuno, T., Li, J., Keutzer, K., Zhang, S.: Mttrans: Cross-domain object detection with mean teacher transformer. In: ECCV. pp. 629–645 (2022)
  * [34] Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L., Shum, H.Y.: Dino: Detr with improved denoising anchor boxes for end-to-end object detection. In: ICLR (2023)
  * [35] Zhang, J., Huang, J., Luo, Z., Zhang, G., Zhang, X., Lu, S.: Da-detr: Domain adaptive detection transformer with information fusion. In: CVPR. pp. 23787–23798 (2023)
  * [36] Zhao, D., Wang, S., Zang, Q., Quan, D., Ye, X., Jiao, L.: Towards better stability and adaptability: Improve online self-training for model adaptation in semantic segmentation. In: CVPR. pp. 11733–11743 (2023)
  * [37] Zhao, L., Wang, L.: Task-specific inconsistency alignment for domain adaptive object detection. In: CVPR. pp. 14217–14226 (2022)
  * [38] Zhao, Z., Wei, S., Chen, Q., Li, D., Yang, Y., Peng, Y., Liu, Y.: Masked retraining teacher-student framework for domain adaptive object detection. In: ICCV. pp. 19039–19049 (2023)
  * [39] Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. In: ICLR (2021)


#  Soliton metacrystals: topology and chirality

Z. Fan  Department of Physics, University of Bath, Bath BA2 7AY, UK  Centre
for Photonics and Photonic Materials, University of Bath, Bath, BA2 7AY, UK
D.N. Puzyrev  Department of Physics, University of Bath, Bath BA2 7AY, UK
Centre for Photonics and Photonic Materials, University of Bath, Bath, BA2
7AY, UK  D.V. Skryabin<EMAIL_ADDRESS>Department of Physics,
University of Bath, Bath BA2 7AY, UK  Centre for Photonics and Photonic
Materials, University of Bath, Bath, BA2 7AY, UK

###### Abstract

Designing metamaterials with the required band structure, topology and
chirality using nano-fabrication technology revolutionises modern science and
impacts daily life. The approach of this work is, however, different. We take
a periodic sequence, i.e., metacrystal, of the dissipative optical solitons
rotating in a single ring microresonator and demonstrate its properties as of
the electromagnetic metamaterial acting in the radio to terahertz frequency
range. The metacrystal unit cell consists of the bound pair of solitons, and
the distance between them is used as a control parameter. We are reporting the
soliton metacrystal band structure and its topological properties. The latter
is confirmed by the existence of the $\pi$ steps experienced by the crystal
phonons’ geometrical (Zak) phase. Furthermore, we found the phononic edge
states in the metacrystals with defects made by removing several solitons.
Optical frequency combs corresponding to the soliton metacrystals reveal the
spectral butterfly pattern serving as a signature of the spatio-temporal
chirality and bearing a resemblance to the butterfly wings illustrating
natural occurrences of chirality.

††preprint: APS/123-QED

Two prolific themes spanning across today’s solid-state, cold-atom and optical
physics are the topology of waves in periodic potentials [1, 2, 3, 4, 5, 6, 7]
and localised coherent states, such as, e.g., solitons and vortices. The
recent and ongoing work on the vortex and skyrmion matter and light [8, 9, 10,
11], soliton crystals [12, 13, 14, 15, 16, 17, 18, 19] and soliton gas [20,
21] points at the opportunity of using the localised structures of the
coherent light and matter waves for designing new electromagnetic materials.
Here, we present the proof-of-concept results revealing the topological
properties of the periodic soliton sequences, i.e., soliton metacrystals,
generated in optical microresonators.

For this work, we should distinguish the dissipative and conservative optical
solitons and single out the former for their robustness and longevity [22].
Crystals of dissipative optical solitons are known to exist in Kerr ring
microresonators [14, 15, 17, 18, 16], fibre lasers [23, 24] and in the
exciton-polariton resonators [25]. Methods of the on-demand positioning of the
individual dissipative solitons, circling with the typical repetition rates
between ten GHz and one THz, inside the ring resonators have also been
successfully demonstrated. These methods included - pump modulation [16],
using bi-chromatic pump [19], single pump frequency tuning [26], and the
excitation of the desired soliton ordering by applying a sequence of pulses
with the repetition rate higher than the resonator free spectral range [27].
Though the soliton crystals in optical resonators are well established, their
band structure and its topology remain an uncharted territory.

The focus of many prior studies of topological properties of nonlinear optical
systems has been on the cases when taking the no-nonlinearity limit leaves a
linear system prepossessing a specially designed linear periodic structure,
see, e.g., [28, 29, 30, 31, 32] for topological solitons and [33, 34, 35, 36,
37, 38, 39, 40, 41, 42] for other topology vs nonlinearity interplay in the
arrays of optical resonators and waveguides, and photonic crystals. The
soliton crystals studied below require only a single optical resonator and are
sustained by the pump-loss and dispersion-nonlinearity balances.

In this work, we demonstrate the soliton metacrystals consisting of the bound
soliton pairs so that the soliton arrangement looks like the famous Su-
Schrieffer-Heeger (SSH) lattice [4]. The relative positioning of the solitons
within a unit cell is used as the control parameter helping to reveal
topological and chiral properties. To prove the non-trivial topology of the
metacrystal band structure, we develop a theory of their geometrical phase
[43, 44], and also demonstrate the topological edge states [4] in the
metacrystals with defects. The edge states typically come in chiral pairs as,
e.g., the edge currents on the opposite interfaces of the suitably cut two-
dimensional topological crystal [3, 4]. A feature of the multimode resonators
is that the trains of the modelocked solitons they generate are periodic in
space and time. We found that the spatio-temporal chirality of the soliton
metacrystals leads to the distinct butterfly-like structure hidden in their
optical spectra. Various practical and conceptual realizations of time
crystals in condensed matter and optics [45, 46, 47, 48, 49, 50, 51] and the
cross-disciplinary applications of chirality [52, 53, 54, 55] are currently
attracting the increasing attention.

Figure 1: Soliton metacrystals and dimerization choices. The middle panel
shows how two solitons are positioned in the unit cell for varying separation
distance, $S_{\alpha}$. Two side panels show all $2K=48$ solitons along the
ring circumference. The right plot corresponds to $S_{A}=S_{B}$, i.e., when
the molecular crystal degenerates to the atomic one. The left plot shows the
molecular metacrystal and illustrates the two dimerization choices applied in
this work, $S_{A}+S_{B}=2\pi/K$. The scaled dispersion and detuning parameters
are $d_{2}/\kappa=5\times 10^{-4}$, $d_{3}/\kappa=5\times 10^{-7}$, and
$\delta/\kappa=30$. The crystal repetition rates vary with $S_{\alpha}$, but
$d_{1}/\kappa$ remains close to $-0.01059$. The dimensionless field amplitude,
$\psi$, is expressed as $\gamma|\phi|^{2}/\kappa=|\psi|^{2}$. The
dimensionless pump parameter is $\gamma h^{2}/\kappa=25$. Crystals are
computed with $512$ points across the unit cell, $\theta\in[0,2\pi/K)$.

## RESULTS AND DISCUSSION

### Band structure of soliton metacrystals

We consider an optical ring microresonator [14, 15, 16, 17] which frequency
spectrum is approximated by
$\omega_{\mu}=\omega_{0}+D_{1}\mu+D_{2}\mu^{2}/2!+D_{3}\mu^{3}/3!$, where
$\mu=0,\pm 1,\pm 2,\dots$ is the mode number, $D_{1}$ is the linear repetition
rate (inverse of the round-trip time), and $D_{2,3}$ are the second and third
order dispersion coefficients. $\omega_{\mu}$ typically belong to the
infrared-to-visible range. The pump laser frequency $\omega_{p}$ is tuned
around $\omega_{0}$, so that $\delta=\omega_{0}-\omega_{p}$ is the detuning
parameter. The envelope, $\phi$, of the multimode optical field inside the
resonator is

$\phi(\theta,t)=\sum_{\mu}\phi_{\mu}(t)e^{i\mu\theta}=\phi(\theta+2\pi,t),~{}\theta=\vartheta-\tilde{D}_{1}\tau,$
(1)

where, $\phi_{\mu}$ are the mode amplitudes, $\vartheta\in[0,2\pi)$ is the
polar angle, and $\tilde{D}_{1}$ is the nonlinear repetition rate.

A soliton crystal, with the spatial period $2\pi/K$ ($K=1,2,\dots$) and the
temporal one $2\pi/K\tilde{D}_{1}$, is defined as a particular realisation of
Eq. (1),

$\phi(\theta,t)=\Phi(\theta)=\sum_{l}\phi_{lK}e^{ilK\theta}=\Phi\left(\theta+\frac{2\pi}{K}\right),$
(2)

where the non-zero modes are separated by the step $K$, $\mu=lK$ ($l=0,\pm
1,\pm 2,\dots$). The corresponding optical spectrum of the soliton crystal is
equidistant and is expressed as
$\tilde{\omega}_{lK}=\omega_{p}+lK\tilde{D}_{1}$. The envelope $\phi$ obeys
the microresonator implementation of the Lugiato-Lefever model [56],

$i\partial_{t}\phi=\delta\phi+\sum_{j=1}^{3}\frac{d_{j}}{j!}(-i\partial_{\theta})^{j}\phi-\gamma|\phi|^{2}\phi-\frac{i\kappa}{2}(\phi-h),$
(3)

where $d_{1}=D_{1}-\tilde{D}_{1}$, $d_{2,3}=D_{2,3}$, $\kappa$ is the
linewidth, $h^{2}$ is the pump power, and $\gamma$ is the nonlinear parameter.

If a single soliton and its small excitations correspond to an atom, then a
bound state of two solitons is a soliton-molecule [57, 58, 59, 60]. A periodic
arrangement of the soliton-molecules makes up what we call a metacrystal. The
numerically computed family of such crystals with two solitons per unit cell,
$\theta\in[0,2\pi/K)$, and $K=24$ is shown in Fig. 1. The metacrystal
structure is SSH-like [4], but none of the terms in Eq. (3) is modulated,
i.e., our lattice is self-sustained by the dispersion-nonlinearity and pump-
loss balances. When the inter-soliton distance equals half of the unit cell
length, $\pi/K$, then the metacrystal (molecular crystal) with period $2\pi/K$
degenerates to the usual (atomic) soliton crystal [14, 15].

The difference of the interaction strengths between the left and right
neighbours of a given soliton, in other words, the imbalance of the intra-cell
and inter-cell coupling rates, is controlled by the separation distance
$S_{\alpha}$. There exist two choices of $S_{\alpha}$, i.e., dimerization
choices, corresponding to the same crystal, $S_{\alpha}=S_{A}$ and
$S_{\alpha}=S_{B}=2\pi/K-S_{A}$ (see Fig. 1). In what follows, we take
$\pi/K\leqslant S_{A}<2\pi/K$ and $0<S_{B}\leqslant\pi/K$, so that the choice
of the unit-cell $A$, i.e., dimerization $A$, corresponds to the inter-cell
coupling being stronger than the intra-cell one. $S_{A,B}=\pi/K$ is the
degeneracy point with equal coupling rates.

Soliton metacrystals should modify the spectrum of small amplitude
perturbations, i.e., elementary excitations, supported by the resonator.
Physically, these excitations are slow modulations of the amplitudes and
phases of the resonator modes having frequencies in the radio- to tera-hertz
range. Using an analogy with the density waves in the solid state crystals, we
term these excitations - phonons. While the nonlinear interactions between the
crystal and phonon modes are imperative in what follows, phonon to phonon
interactions are weak and should be disregarded.

Figure 2: Phononic band structure in soliton metacrystals. Panels (a,b,c) show
the spectrum vs momentum $k$ varying across the two Brillouin zones. The
number of Bloch states in one Brillouin zone is $K=24$. Panels (b,c) zoom on
the band structure around the top edge of the continuum, see
$\beta_{k,n}/\kappa\approx 30$ in (a). Panel (d) demonstrates how the band
structure changes with the inter-soliton distance $S_{\alpha}$. The blue and
yellow shading in (b,c,d) highlight the first and second bandgaps.

The phonon band structure is computed using the Bloch theorem. This is
achieved by taking a soliton crystal, Eq. (2), and its complex conjugate, and
representing the phonon field by the cell-periodic two-component Bloch
vectors, ${\bf u}^{(\alpha)}_{k,n}(\theta)={\bf
u}^{(\alpha)}_{k,n}(\theta+2\pi/K)$,

$\displaystyle\begin{bmatrix}\phi\\\ \phi^{*}\end{bmatrix}=$
$\displaystyle\begin{bmatrix}\Phi^{(\alpha)}\\\
\Phi^{(\alpha)*}\end{bmatrix}+\sum_{k,n}e^{t\lambda_{k,n}}\times\left[{{\bf
u}^{(\alpha)}_{k,n}}e^{ik\theta-it\beta_{k,n}}\right.$ (4)
$\displaystyle~{}~{}~{}\left.+\bm{\hat{\sigma}_{x}}{{\bf
u}^{(\alpha)*}_{k,n}}e^{-ik\theta+it\beta_{k,n}}\right],\bm{\hat{\sigma}_{x}}=\begin{bmatrix}0&1\\\
1&0\end{bmatrix}.$

Every Bloch vector is characterised by (i) Bloch momentum, $k$, which can be
restricted to the first Brillouin zone, $k=1,2,\dots,K$, (ii) band index, $n$,
and (iii) dimerization index, $\alpha=A,B$. $\beta_{k,n}=\beta_{k+K,n}$ are
the phonon frequencies and $\lambda_{k,n}=\lambda_{k+K,n}$ express the balance
between the dissipation and parametric gain. The momentum varies discretely
because of the ring geometry. The condition of the crystal stability,
$\lambda_{k,n}\leqslant 0$, is satisfied for all solutions shown in Fig. 1.
Below we use the ket notation for ${\bf u}^{(\alpha)}_{k,n}=|{{\bf
u}^{(\alpha)}_{k,n}}\rangle$ and bra for its transpose and complex conjugate,
$\bm{\hat{\sigma}_{x}}{\bf u}^{(\alpha)*}_{k,n}=\langle{{\bf
u}^{(\alpha)}_{k,n}}|$.

The Bloch states solve the eigenvalue problem

$\bm{\hat{\sigma}_{z}}\widehat{H}_{k}^{(\alpha)}|{{\bf
u}^{(\alpha)}_{k,n}}\rangle=(\beta_{k,n}+i\lambda_{k,n}+i\tfrac{1}{2}\kappa)|{{\bf
u}^{(\alpha)}_{k,n}}\rangle,\bm{\hat{\sigma}_{z}}=\begin{bmatrix}1&0\\\
0&\text{-}1\end{bmatrix},$ (5)

where $\widehat{H}_{k}^{(\alpha)}$ is the system Hamiltonian, see Methods for
details. The phonon spectrum, unlike the eigenstates, does not change if
$\alpha=A$ is replaced with $B$. Fig. 2a shows the phonon spectra of the
soliton crystals with $S_{A}=S_{B}$. The values of $\beta_{k,n}$ scale with
the linewidth, $\kappa$, which, in the chip-integrated microresonators, varies
in the range from tens to hundreds of MHz. The continuum of states starts
around $\beta_{k,n}\approx\pm\delta$ and is shaped like a typical band
structure. Fig. 2b narrows the view to the top edge of the continuum, where
one can see the band-crossing feature and bandgap (yellow shading). $d_{3}\neq
0$ produces the small amplitude dispersive waves [61] with the strongly tilted
straight-line spectrum, see Fig. 2a.

Figure 3: Geometrical (Zak) phases and Wannier functions in soliton
metacrystals. (a,b) Numerically computed Zak phases for the $n=0$ (a) and
$n=1$ (b) bands. The phases flip by $\pi$ at the band-crossing points, cf.
Fig. 2d. (c) The change of the positions and relative strength of the peaks of
the $n=0$ Wannier function vs the soliton separation, $S_{\alpha}$, and
$\theta-\theta_{J}$ (restricted to the one and a half unit cell length to
boost resolution). The black-orange colour-map shows $|V_{J,0}^{(\alpha)}|$,
see Eq. (m.19). The magenta ($\alpha=B$) and blue ($\alpha=A$) vertical doted
lines mark the values of ${\cal Z}^{(\alpha)}_{0}/K$ from (a). (d) shows three
selected profiles of $|V_{J,0}^{(\alpha)}|$ for $S_{\alpha}$ marked by the
white horizontal lines in (c).

When $S_{A}$ becomes $\neq S_{B}$, either way, the band crossing opens up to
the bandgap, see the blue shading in Fig. 2c. At the same time, the yellow
shaded $S_{A}=S_{B}$ band gap starts narrowing and comes to the near band-
crossings for $S_{A,B}=\pi(1\pm 0.5)/K$, see Fig. 2d. We recall that the
textbook SSH chain with the nearest-neighbour coupling has only two bands that
cross at the Brillouin zone edges when the inter and intra-cell coupling rates
($r$ and $r^{\prime}$) are equal. The topological gap opens up for $r\neq
r^{\prime}$, and its width equals $|r-r^{\prime}|$ [4]. Thus, the first two
bands in the soliton crystal lattice behave very similarly to what happens in
the SSH chain. To some extent, the similarity with the SSH model comes
unexpectedly, since the spectrum of the Jacobian operator,
$\bm{\hat{\sigma}_{z}}\widehat{H}_{k}^{(\alpha)}-i\tfrac{1}{2}\kappa\mathbf{\hat{1}}$,
defines the phonon band structure in soliton crystals, while in the SSH model
[4], and for the non-interacting electrons in crystals [44], this is the
Hamiltonian spectrum.

### Geometrical phase and Wannier-centres

If the soliton crystal is considered from the same point of view as the ionic
crystal lattice shaping clouds of bound electrons in dielectrics, then the
next natural step would be to understand the shapes of the phononic clouds
existing around the solitons. This is achieved by developing the theory like
the theory of polarization in crystalline dielectrics, where it has also
provided a transparent interpretation of the geometrical (Zak) phase of an
electron acquired after its passage over the Brillouin zone (BZ) [62, 44, 64,
63].

The $k$-space periodicity and gauge-freedom properties of the Bloch vectors
are,

${\bf u}_{k,n}^{(\alpha)}={\bf u}_{k+K,n}^{(\alpha)}e^{iK\theta},~{}{\bf
u}_{k,n}^{(\alpha)}\to{\bf u}_{k,n}^{(\alpha)}e^{i\xi_{k}}.$ (6)

Here, $\xi_{k}$ are the phases, which are arbitrary across the first BZ and
extendable beyond it with $\xi_{k}=\xi_{k+K}$. The discreteness of $k$
requires special care when formulating the gauge invariant computational
strategy for the geometric phase. Adopting the approach of [63, 64], we have
derived the following equation for the geometric phase,

${\cal Z}^{(\alpha)}_{n}=\text{Im}\ln\prod_{k=1}^{K}\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k+1,n}}\rangle,$ (7)

of phonons belonging to a specific band in the soliton crystals, see Methods.
The simultaneous bra and ket occurrences for every $k$ make Eq. (7) gauge
invariant and applicable to any Bloch-function phases generated by the
eigenvalue solver.

We check topological properties of phonons in soliton crystals numerically by
observing the $\pi$-flipping of their geometric phase on dragging the system
through the band crossing points. Figs. 3a,b show ${\cal Z}_{n}^{(\alpha)}$ vs
$S_{\alpha}$ for $n=0,1$. The $n=0$ phase flips from $-\pi$ to zero at the
point $S_{A,B}=\pi/K$, where the $n=0$ and $n=1$ bands make the near-crossing,
see Fig. 2d. The topology of the $n=1$ band reflects on the co-existence of
one crossing with the $n=0$ band and two crossings with the $n=2$ band, and,
hence, its geometrical phase flips three times.

Figure 4: Phonon edge states in soliton metacrystals with defects. (a) The
phonon spectrum in and around the $n=0,1$ bandgap in the metacrystal with the
defect made by removing one unit cell, i.e., 2-soliton defect. (b) The
eigenfunctions of the bandgap states, $|{\bf u}_{k,n}^{(\alpha)}|^{2}$, vs
$S_{\alpha}$ and $\theta$. The stems along top axes mark locations of the
individual solitons and hence show the size of defects for $S_{A,B}=\pi/K$.
(c) is like (b), and (d) is like (a), but for the 7-soliton defect created by
removing the three unit cells plus one soliton. The red diamonds in (a) and
(d) mark the spectra of the edge states. Crystals with defects are computed
with $16384$ points across the resonator circumference, $\theta\in[0,2\pi)$.
$|{{\bf u}_{k,n}^{(\alpha)}}\rangle$ is computed using Eq. (5) with $k=0$,
$K=1$, which corresponds to $\theta\in[0,2\pi)$, see Methods for details.

If the number of cells is taken to infinity while their length is kept fixed
at $2\pi/K$, then $k$ becomes continuous and, hence, $\partial_{k}$ is well
defined. In this limit, the geometric phase acquires a more familiar form,
${\cal Z}^{(\alpha)}_{n}=\int_{0}^{K}{\cal A}^{(\alpha)}_{k,n}dk$, where
${\cal A}^{(\alpha)}_{k,n}=i\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}\partial_{k}|{{\bf
u}^{(\alpha)}_{k,n}}\rangle$. ${\cal A}^{(\alpha)}_{k,n}$ differs from the
Berry connection for the non-interacting electrons [44, 43] by the Pauli
matrix $\bm{\hat{\sigma}_{z}}$ before $\partial_{k}$. Taking a unit cell with
the coordinate $\theta=\theta_{J}$ and defining the corresponding Wannier
function as $|{{\bf w}_{J,n}^{(\alpha)}}\rangle=K^{-1}\sum_{k}|{{\bf
u}_{k,n}^{(\alpha)}}\rangle e^{ik(\theta-\theta_{J})}$, we have further
demonstrated that the geometric phase is directly proportional to the averaged
coordinate of the phononic cloud centre in a given cell,

$\frac{{\cal Z}^{(\alpha)}_{n}}{K}=\langle{{\bf
w}^{(\alpha)}_{J,n}}|\bm{\hat{\sigma}_{z}}(\theta-\theta_{J})|{{\bf
w}^{(\alpha)}_{J,n}}\rangle.$ (8)

With $\bm{\hat{\sigma}_{z}}\to\mathbf{\hat{1}}$, the above would be the
equation for the band- or Wannier-centre of the electron wave function in a
crystal [44, 62, 64, 63], see Methods for more details. For our unit cell
definition, see Fig. 1, the $n=0$ phonon Wannier centres can be located either
in the middle or at the boundary of a unit cell. Figs. 3c,d are explicit about
the matching between ${\cal Z}^{(\alpha)}_{0}/K$ and the geometric centres of
the Wannier functions. Recent papers demonstrating measurements of the
geometric phases of cold atoms in optical lattices [65] and light in waveguide
arrays [66, 67] have applied various practical methods to detect positions of
the respective wave-packets.

### Metacrystals with defects: Edge states and chirality

For the SSH model, creating the lattice defect by removing one $A$-type unit
cell, i.e., breaking two strong bonds in the chain, introduces the edge state
inside the bandgap [4]. To study the bandgap states in the soliton
metacrystals, we first extracted one pair of solitons and then tuned
$S_{\alpha}$. $\alpha=B$ corresponds to breaking the weak bond and $\alpha=A$
to the strong one, see Fig. 1. The corresponding phonon spectrum plotted as a
function of $S_{\alpha}$, see Fig. 4a, shows the near-crossing of the two
bands at $S_{\alpha}=\pi/K$, and the state inside the bandgap emerging for
$S_{\alpha}$ either $<\pi/K$ ($B$-type defect) or $>\pi/K$ ($A$-type defect).
The eigenvectors of the bandgap state are localised around the defect on
either side from $\pi/K$, see Fig. 4b. For $\alpha=A$, the state vector is
localised on the edge solitons, while, for $\alpha=B$, it is guided in the
middle of the defect, see Fig. 4b. The data presentation for the SSH edge
states similar to the style chosen in Figs. 4b,c can be found in [49].

If the defect is made by the odd number of solitons, then shifting between the
$\alpha=A$ and $\alpha=B$ dimerizations corresponds to moving the weak bond
from one side of the defect to the other, see, e.g., the structure of the
7-soliton defect in Fig. 5a. Then, the edge state also migrates together with
the weak bond, see Fig. 4c. With the defects larger than one unit cell, the
spectrum in the bandgap contains not only the edge state but also other states
guided inside the defect not at its edges, see the black doted lines inside
the blue shading in Fig. 4d. The guidance mechanism for these modes is akin to
the band-gap guidance in the hollow-core photonic crystal fibres [68].

Figure 5:  Chirality of soliton metacrystals with defects. (a,b) show
metacrystals with the differently arranged 7-soliton defects and $K=24$. (a)
and (b) geometries are called left and right, respectively. The missing
soliton locations are marked with dashed lines. Note that the
$\theta$-interval shown is $<2\pi$. (c,d) show the examples of spectra of the
achiral and chiral defects. (e) shows the normalised frequency spectra of the
crystals with the right and left defects, i.e.,
$\tilde{\omega}_{\mu}^{\text{(right)}}$ and
$\tilde{\omega}_{\mu}^{\text{(left)}}$, minus the
$\tilde{\omega}_{\mu}^{\text{(ac)}}$ spectrum corresponding to the achiral
geometry, $S_{\alpha}=\pi/K$. Red dots correspond to the right defect,
$(\tilde{\omega}_{\mu}^{\text{(right)}}-\tilde{\omega}_{\mu}^{\text{(ac)}})/\kappa$,
and blue dots to the left one,
$(\tilde{\omega}_{\mu}^{\text{(left)}}-\tilde{\omega}_{\mu}^{\text{(ac)}})/\kappa$.
The values of $\mu$ shown are restricted to $\mu\in[360,1200]$, with the step
being 120.

The defects consisting of the odd-number of solitons break the left-right
symmetry of the metacrystals and thereby appear as an interesting object to
study the spatio-temporal and spectral manifestations of chirality [55]. We
consider a crystal to be chiral, providing that the mirror image of the
solitons around the defects can not be made to coincide with the original
soliton arrangement. This is true for any $S_{A}\neq S_{B}$, while, the
$S_{A}=S_{B}=\pi/K$ case corresponds to the achiral defect. Indeed, the
unpaired soliton can be removed either on the left or right side of the
defect, and then the two crystals with the same $S_{\alpha}$ are connected by
the chiral symmetry transformation. Tuning $S_{\alpha}$ and plotting the
crystals around the defect edges makes the chirality obvious, see the tilts of
the solitons at the defect edges in Figs. 5a and 5b.

The chiral pairs of the right and left defects have the spectra which are hard
to distinguish. Therefore, we have just shown one of them in Fig. 5d. The
spectrum of the ahiral defect is shown for comparison in Fig. 5c. The envelope
of the large amplitude spectral lines match well with the spectra of the ideal
crystals, while the small amplitude dense spectrum originates from the defect.
To make the spectral signature of the chirality effect obvious, we have taken
the frequency spectra of the left and right defects, i.e.,
$\tilde{\omega}_{\mu}^{\text{(left)}}$ and
$\tilde{\omega}_{\mu}^{\text{(right)}}$, and subtracted from them the achiral
spectrum, $\tilde{\omega}_{\mu}^{\text{(ac)}}$. Here,
$\tilde{\omega}_{\mu}^{\text{(..)}}=\omega_{p}+\mu\tilde{D}_{1}^{\text{(..)}}$
is the mode-locked (dispersion free) frequency spectrum of the soliton crystal
with defect and $\tilde{D}_{1}^{\text{(..)}}$ is its numerically computed free
spectral range. $\tilde{D}_{1}^{\text{(..)}}$ is a function of $S_{\alpha}$,
pump and all the other parameters. We shall recall that the mode-locked
spectrum for the ideal crystal, $\mu=lK$, has been already defined after Eq.
(5). Fig. 5e shows how
$(\tilde{\omega}_{\mu}^{\text{(left)}}-\tilde{\omega}_{\mu}^{\text{(ac)}})/\kappa$
and
$(\tilde{\omega}_{\mu}^{\text{(right)}}-\tilde{\omega}_{\mu}^{\text{(ac)}})/\kappa$
vary with $S_{\alpha}$. The spectral differences make the butterfly pattern
with the symmetry axis at $S_{\alpha}=\pi/K$ and, thereby, reveal the spatio-
temporal chirality of the soliton metacrystals.

### Discussion

We have conjectured and confirmed numerically that the train of solitons in
microresonators can be considered as a spatio-temporal topological
metamaterial, i.e., soliton metacrystals. The optical spectra of such crystals
correspond to the octave wide frequency combs. The elementary excitations,
i.e., phonons, in metacrystals are the radio to terahertz modulations
propagating across the spectrum of the resonator modes. We have computed the
phononic Bloch states and their geometrical (Zak) phases. The Zak phase has
also been expressed via the Wannier functions and demonstrated to have the
meaning of the effective polarization of the phononic clouds around the
solitons.

Zak phase changes in the steps of $\pi$ with the tuning of the soliton
separation, which proves the non-trivial topology of the metacrystal band
structure. Further to this point, we have found the phononic edge states upon
introducing the one unit cell and larger defects. We have demonstrated that
the optical (photon) spectra of metacrystals with the left and right defects
can be used to characterize chirality in time. Subtracting the optical
spectrum of the achiral crystal from the chiral ones reveals the butterfly
structure of frequencies, see Fig. 5e, similar to the butterfly wings
illustrating chirality in nature.

Features of the proposed realisation of the SSH-like soliton lattice are in
its spatio-temporal and dissipative nature and that it does not require
spatial or temporal modulations of the material refractive index and pump
laser and fabrication of the complex arrays of coupled resonators and
waveguides. Further, we should note that one out of twenty microresonator
soliton crystals illustrated in [14] shows the SSH-like arrangement of soliton
molecules with several embedded defects, see Fig. 3j. This suggests that the
frequency combs with topological properties may have already been
unintentionally captured in the experimental measurements. Our results intend
to motivate further theory, experiments and applications so that this topic
can grow and take its place in the interdisciplinary areas of topological and
ultrafast physics. We are also forecasting applications of the soliton
metacrystals and their topological states in classical and quantum information
processing, where the microresonator solitons and topological photonics are
already making an impressive start [69, 70, 71, 72, 73].

METHODS
In order to shorten notations we have dropped the band index, $n$, and the
dimerization superscript, $(\alpha)$, in the first part of Methods. They are
subsequently reintroduced in the second and third parts.
Bloch formalism
To analyse and solve Eq. (1) we set the ansatz consisting of the time
independent solution which could be a crystal with or without defect,
$\Phi(\theta)=\Phi(\theta+2\pi)$, plus the small amplitude phonon field,
$\tilde{\phi}(\theta,t)$. Hence,

$\phi=\Phi(\theta)+\tilde{\phi}(\theta,t),~{}\tilde{\phi}(\theta,t)=\tilde{\phi}(\theta+2\pi,t).$
(m.1)

Linearising Eq. (1) for $\tilde{\phi}$ yields

$i(\partial_{t}+\tfrac{1}{2}\kappa)\begin{bmatrix}\tilde{\phi}\\\
\tilde{\phi}^{*}\end{bmatrix}=\bm{\hat{\sigma}_{z}}\widehat{H}_{0}\begin{bmatrix}\tilde{\phi}\\\
\tilde{\phi}^{*}\end{bmatrix},\theta\in[0,2\pi),$ (m.2)

where $\widehat{H}_{0}$ is specified in Eq. (m.8).

We now assume that $\Phi(\theta)$ has a period $2\pi/K$, where $K$ could also
be one,

$\Phi(\theta)=\sum_{l}\phi_{lK}e^{ilK\theta}=\Phi\left(\theta+\frac{2\pi}{K}\right).$
(m.3)

Equations for $\phi_{lK}$ have been solved using the Newton method. Parameters
used are typical for Si3N4 resonators: $D_{1}/2\pi=100$GHz,
$D_{2}/2\pi=50$kHz, $D_{3}/2\pi=50$Hz, $\gamma/2\pi=30$MHz/W,
$\kappa/2\pi=100$MHz and laser power ${\cal W}=520$mW, so that
$h^{2}=(D_{1}/2\pi\kappa){\cal W}=83$W, see Eqs. (1)-(3) and Ref. [56] for
further parameter and model discussion. The scaled values of the parameters
are given in the Fig. 1 caption. The approximate analytical expressions for
the non-topological soliton crystals in the Lugiato-Lefever equation have been
reported in, e.g., [74, 75].

The phonon field is set as [76, 77]

$\displaystyle\tilde{\phi}=\sum_{k=1}^{K}\left[x_{k}(t,\theta)e^{ik\theta}+y_{k}^{*}(t,\theta)e^{-ik\theta}\right],$
(m.4) $\displaystyle
x_{k}=X_{k}(\theta)e^{-it\beta_{k}+t\lambda_{k}},y_{k}=Y_{k}(\theta)e^{-it\beta_{k}+t\lambda_{k}}.$
(m.5)

Here, $k$ is the Bloch angular momentum. $X_{k}(\theta)$ and $Y_{k}(\theta)$
are the cell-periodic functions, i.e., have the same periodicity as
$\Phi(\theta)$, therefore their Fourier series are

$X_{k}(\theta)=\sum_{l}X_{k,l}e^{ilK\theta},~{}Y_{k}(\theta)=\sum_{l}Y_{k,l}e^{ilK\theta}.$
(m.6)

Substituting Eq. (m.4) into Eq. (m.2) gives

$\displaystyle i(\partial_{t}$ $\displaystyle+\tfrac{1}{2}\kappa)|{{\bf
Q}_{k}}\rangle=\bm{\hat{\sigma}_{z}}\widehat{H}_{k}|{{\bf
Q}_{k}}\rangle,~{}|{{\bf Q}_{k}}\rangle=\begin{bmatrix}x_{k}\\\
y_{k}\end{bmatrix},$ (m.7) $\displaystyle\widehat{H}_{k}$
$\displaystyle=\begin{bmatrix}\widehat{D}_{k}&-\gamma\Phi^{2}\\\
-\gamma\Phi^{*2}&\widehat{D}_{-k}^{*}\end{bmatrix},\theta\in[0,2\pi/K),$ (m.8)
$\displaystyle\widehat{D}_{k}$
$\displaystyle=\delta-2\gamma|\Phi|^{2}+\sum_{j\geqslant
1}\frac{d_{j}}{j!}\left(k-i\partial_{\theta}\right)^{j},$

and then, applying Eq. (m.5), we find

$(\beta_{k}+i\lambda_{k}+i\tfrac{1}{2}\kappa)|{{\bf
u}_{k}}\rangle=\bm{\hat{\sigma}_{z}}\widehat{H}_{k}|{{\bf
u}_{k}}\rangle,~{}|{{\bf u}_{k}}\rangle=\begin{bmatrix}X_{k}\\\
Y_{k}\end{bmatrix}.$ (m.9)

The above is Eq. (5) in the main text. One computational advantage of the
Bloch formalism comes from the property that

$\beta_{k}=\beta_{k+K},\lambda_{k}=\lambda_{k+K},|{{\bf u}_{k}}\rangle=|{{\bf
u}_{k+K}}\rangle e^{iK\theta},$ (m.10)

and, therefore, $k$ can be restricted to a single Brillouin zone,
$k=1,2,\dots,K$. The other is that the spectral bandwidth
$\mu\in[-\mu_{\max},\mu_{\max}]$, see Eq. (1), is now achieved by taking
$l\in[-l_{\max},l_{\max}]$ with $l_{\max}=\mu_{\max}/K$. For $K$ starting from
2 and above, this allows to work with the proportionally smaller number of
modes when Eq. (m.9) is solved in the Fourier space.

Geometrical phase and gauge invariance
The Zak phase formalism that needs to be developed for the soliton
metacrystals should reflect on two features - (A) nonlinear interaction
between the phonon and crystal and (B) the discrete sampling of the states in
the Brillouin zone. In order to make our derivations transparent, we first
neglect (B) and, hence, take the limit of a large number of unit cells, so
that $k$ is continuous in the Brillouin zone. $k$ is now assumed to vary
adiabatically in time, $k=k(t)$. The equation of motion, Eq. (m.7), for the
phonon wave function, $|{{\bf Q}^{(\alpha)}_{k,n}}\rangle$, is solved by the
substitution [43, 44]

$|{{\bf Q}^{(\alpha)}_{k,n}}\rangle=f_{k,n}^{(\alpha)}(t)|{{\bf
u}^{(\alpha)}_{k,n}}\rangle,$ (m.11)

where $f_{k,n}^{(\alpha)}$ is the function to be found and the phonons are
assumed confined to the band $n$. For
$|\omega_{k,n}|\gg|\lambda_{k,n}|,\kappa$, the left eigenvector of
$\bm{\hat{\sigma}_{z}}H_{k}^{(\alpha)}$ is $\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}$ [76]. Substituting Eq. (m.11) to
Eq. (m.8) and projecting leads to

$\displaystyle
i\frac{\text{d}f_{k,n}^{(\alpha)}}{f_{k,n}^{(\alpha)}}\approx\beta_{k,n}\text{d}t-{\cal
A}^{(\alpha)}_{k,n}\text{d}k,$ (m.12) $\displaystyle{\cal
A}^{(\alpha)}_{k,n}=\frac{i}{N_{n}}\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}\partial_{k}|{{\bf
u}^{(\alpha)}_{k,n}}\rangle,~{}N_{n}=\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf u}^{(\alpha)}_{k,n}}\rangle,$

where $N_{n}$ is the normalization constant.

The $\beta_{k,n}$ and ${\cal A}_{k,n}^{(\alpha)}$ terms in Eq. (m.12) are the
instantaneous dynamical and geometrical phases. Hence, the total geometric
phase acquired during $k(t)$ passing the first Brillouin zone (BZ) is

${\cal Z}^{(\alpha)}_{n}=\int_{BZ}{\cal A}^{(\alpha)}_{k,n}\text{d}k.$ (m.13)

Our ${\cal A}_{k,n}^{(\alpha)}$ differs from the Berry connection emerging in
the linear problems [43, 44, 4], by the Pauli matrix $\bm{\hat{\sigma}_{z}}$
before $\partial_{k}$, which has been previously reported in, e.g., photonic
crystals with parametric gain [33] and LC-circuits [34, 35]. The normalisation
condition applied by us is $N_{n}=\pm 1$, see, e.g., [76, 8] for how similar
norms appear in the theory of Bose condensates. $\bm{\hat{\sigma}_{z}}$ also
enters the Berry curvature of the Bogolyubov excitations of Bose condensates
[78].

The phases of the Bloch functions are not uniquely defined and can be changed
by the gauge transform, which, in the discrete BZ, takes the form

${\bf u}^{(\alpha)}_{k,n}\to{\bf
u}^{(\alpha)}_{k,n}e^{i\xi_{k}},~{}\xi_{k}=\xi_{k+K}.$ (m.14)

For the $k$-continuous, ${\cal Z}^{(\alpha)}_{n}$ remains invariant on
applying Eq. (m.14) [44]. When $k$ is sampled discretely, a variety of
available numerical schemes to deal with $\partial_{k}$ will not generally
comply with the gauge invariance. Therefore, designing a practical algorithm
to compute ${\cal Z}^{(\alpha)}_{n}$ is an important issue to consider.

This brings us to the point (B) and we assume that $k$ is advancing in the
unitary steps from $1$ to $K$, and each step takes the time d$t$ to make, then
Eq. (m.12) updates as

$\displaystyle
i\frac{f_{k+1,n}^{(\alpha)}-f_{k,n}^{(\alpha)}}{f_{k,n}^{(\alpha)}}\approx\beta_{k,n}\text{d}t$
(m.15) $\displaystyle-i\left(\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k+1,n}}\rangle-\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k,n}}\rangle\right)/N_{n},$

and, therefore, the total geometric phase is

${\cal Z}_{n}^{(\alpha)}=-i\sum_{k=1}^{K}\left(\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k+1,n}}\rangle-N_{n}\right)/N_{n}.$ (m.16)

Checking of Eq. (m.16) reveals the discretization scheme induced violation of
the gauge invariance. Indeed, Eq. (m.16) is invariant on applying $({\bf
u}^{(\alpha)}_{k,n},{\bf u}^{(\alpha)}_{k+1,n})\to({\bf
u}^{(\alpha)}_{k,n}e^{i\xi_{k}},{\bf u}^{(\alpha)}_{k+1,n}e^{i\xi_{k}})$ and
not relative to Eq. (m.14).

The recipe to restore the gauge invariance is given by the theory of
polarization in crystalline solids [64, 63]. We first take the exponent from
Eq. (m.16), and then use $e^{x}=1+x+\dots$,

$\displaystyle\exp\\{i{\cal Z}_{n}^{(\alpha)}\\}$
$\displaystyle=\prod_{k=1}^{K}\exp\left\\{\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k+1,n}}\rangle\frac{1}{N_{n}}-1\right\\}$ (m.17)
$\displaystyle\approx\prod_{k=1}^{K}\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k+1,n}}\rangle\frac{1}{N_{n}}.$

Taking the logarithm gives the gauge invariant expression for the geometric
phase of the soliton crystals,

$\displaystyle{\cal Z}_{n}^{(\alpha)}=\text{Im}\ln\prod_{k=1}^{K}\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{k+1,n}}\rangle\frac{1}{N_{n}}$ (m.18)
$\displaystyle=\text{Im}\ln\Big{\\{}\frac{1}{N_{n}^{K}}\times\langle{{\bf
u}^{(\alpha)}_{1,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{2,n}}\rangle\langle{{\bf
u}^{(\alpha)}_{2,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{3,n}}\rangle\dots$ $\displaystyle\dots\langle{{\bf
u}^{(\alpha)}_{K-1,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{K,n}}\rangle\langle{{\bf
u}^{(\alpha)}_{K,n}}|\bm{\hat{\sigma}_{z}}|{{\bf
u}^{(\alpha)}_{1,n}e^{-iK\theta}}\rangle\Big{\\}}.$

’$\ln$’ is defined by its principal value, such that ${\cal
Z}_{n}^{(\alpha)}\in[-2\pi,0)$. For $K$ even, $N_{n}^{K}=1$ and the above
coincides with Eq. (7) in the main text.

Wannier formalism
The direct and inverse Bloch-to-Wannier transforms are defined as

$\begin{split}&|{{\bf u}_{k,n}^{(\alpha)}}\rangle=\sum_{J}|{{\bf
w}_{J,n}^{(\alpha)}}\rangle
e^{-ik(\theta-\theta_{J})}=\begin{bmatrix}X^{(\alpha)}_{k,n}\\\
Y^{(\alpha)}_{k,n}\end{bmatrix},\\\ &|{{\bf
w}_{J,n}^{(\alpha)}}\rangle=\frac{1}{K}\sum_{k}|{{\bf
u}_{k,n}^{(\alpha)}}\rangle
e^{ik(\theta-\theta_{J})}=\begin{bmatrix}W^{(\alpha)}_{J,n}\\\
V^{(\alpha)}_{J,n}\end{bmatrix},\end{split}$ (m.19)

where $|{{\bf w}_{J,n}^{(\alpha)}}\rangle$ are Wannier functions,
$\theta_{J}=2\pi(J-1)/K$ and $J\in\mathbb{Z}$ numbers the unit cells. Our
gauge choice for the Bloch functions is specified by the condition
Im$X^{(\alpha)}_{k,n}=0$ at the point $\theta=3\pi/2K$.

We now recall Eq. (m.9) and, then, transform Eq. (m.13),

$\displaystyle{\cal Z}_{n}^{(\alpha)}=i\int_{BZ}dk\langle{{\bf
u}^{(\alpha)}_{k,n}}|\bm{\hat{\sigma}_{z}}\partial_{k}|{{\bf
u}^{(\alpha)}_{k,n}}\rangle$ (m.20)
$\displaystyle=i\int_{BZ}dk\int_{\theta_{J}-\pi/K}^{\theta_{J}+\pi/K}\left(X^{(\alpha)*}_{k,n}\partial_{k}X^{(\alpha)}_{k,n}-Y^{(\alpha)*}_{k,n}\partial_{k}Y^{(\alpha)}_{k,n}\right)d\theta$
$\displaystyle=\frac{i}{K}\int_{BZ}dk\int_{\theta_{J}-\pi}^{\theta_{J}+\pi}\left(X^{(\alpha)*}_{k,n}\partial_{k}X^{(\alpha)}_{k,n}-Y^{(\alpha)*}_{k,n}\partial_{k}Y^{(\alpha)}_{k,n}\right)d\theta.$

The last part of Eq. (m.20) uses the fact that the Bloch functions are cell
periodic and, hence, the integration can be extended to the entire ring.

If the ring is large, the problem can be well approximated by the straight-
line infinite crystal lattice [44] with the period $2\pi/K$. Then,
$\sum_{k}\to\int_{BZ}dk$ and expressing the Bloch functions via the Wannier
ones allows to take the $k$-derivative inside Eq. (m.20),

$\displaystyle{\cal Z}_{n}^{(\alpha)}$ $\displaystyle=$
$\displaystyle\sum_{J}\int_{\theta_{J}-\pi}^{\theta_{J}+\pi}\left[|W^{(\alpha)}_{J,n}|^{2}-|V^{(\alpha)}_{J,n}|^{2}\right](\theta-\theta_{J})d\theta$
(m.21) $\displaystyle=$ $\displaystyle\sum_{J}\langle{{\bf
w}^{(\alpha)}_{J,n}}|\bm{\hat{\sigma}_{z}}(\theta-\theta_{J})|{{\bf
w}^{(\alpha)}_{J,n}}\rangle.$

Wannier functions for all $J$ coincide if plotted vs $\theta-\theta_{J}$,
therefore, Eq. (m.21) simplifies to

${\cal Z}_{n}^{(\alpha)}=K\langle{{\bf
w}^{(\alpha)}_{J,n}}|\bm{\hat{\sigma}_{z}}(\theta-\theta_{J})|{{\bf
w}^{(\alpha)}_{J,n}}\rangle,$ (m.22)

see Eq. (8) in the main text. Thus, the geometric phase of soliton crystals is
determined by the coordinate of the effective centre of the phonon cloud in
the unit cell, i.e., by the Wannier-centre, $\langle{{\bf
w}^{(\alpha)}_{J,n}}|\bm{\hat{\sigma}_{z}}(\theta-\theta_{J})|{{\bf
w}^{(\alpha)}_{J,n}}\rangle$. For the ring geometry with the relatively large
number of unit cells, as we are dealing with, Eq. (m.22) remains a good
approximation.

Data availability
The data supporting the findings of this study are available from authors on
reasonable request.

## References

  * [1] F. Duncan, and M. Haldane, Nobel Lecture: Topological quantum matter, Rev. Mod. Phys. 89, 040502 (2017).
  * [2] N.R. Cooper, J. Dalibard, and I.B. Spielman, Topological bands for ultracold atoms, Rev. Mod. Phys. 91, 015005 (2019).
  * [3] T. Ozawa, H.M. Price, A. Amo, N. Goldman, M. Hafezi, L. Lu, M.C. Rechtsman, D. Schuster, J. Simon, O. Zilberberg, and I. Carusotto, Topological photonics, Rev. Mod. Phys. 91, 015006 (2019).
  * [4] J.K. Asboth, L. Oroszlany, and A. Palyi, A Short Course on Topological Insulators: Band Structure and Edge States in One and Two Dimensions (Springer, 2016).
  * [5] A. Forbes, M. de Oliveira, and M.R. Dennis, Structured light, Nat. Photon. 15, 253 (2021).
  * [6] Q. Ma, A.G. Grushin, and K.S. Burch, Topology and geometry under the nonlinear electromagnetic spotlight, Nat. Mat. 20, 1601 (2021).
  * [7] B. Deng, M. Zanaty, A.E. Forte, and K. Bertoldi, Topological solitons make metamaterials crawl, Phys. Rev. Applied 17, 014004 (2022).
  * [8] A.L. Fetter, Rotating trapped Bose-Einstein condensates, Rev. Mod. Phys. 81, 647 (2009).
  * [9] X.Z. Yu, Y. Onose, N. Kanazawa, J.H. Park, J.H. Han, Y. Matsui, N. Nagaosa, and Y. Tokura, Real-space observation of a two-dimensional skyrmion crystal, Nature 465, 901 (2010)
  * [10] D. Foster, C. Kind, P.J. Ackerman, J.B. Tai, M.R. Dennis, and I.I. Smalyukh, Two-dimensional skyrmion bags in liquid crystals and ferromagnets, Nat. Phys. 15, 655 (2019).
  * [11] D. Sugic, R. Droop, E. Otte, D. Ehrmanntraut, F. Nori, J. Ruostekoski, C. Denz, and M.R. Dennis, Particle-like topologies in light, Nat. Commun. 12, 6785 (2021).
  * [12] D. Bongiovanni, D. Jukic, Z. Hu, F. Lunic, Y. Hu, D. Song, R. Morandotti, Z. Chen, and H. Buljan, Dynamically emerging topological phase transitions in nonlinear interacting soliton lattices, Phys. Rev. Lett. 127, 184101 (2021).
  * [13] A.S. Desyatnikov, N. Sagemerten, R. Fischer, B. Terhalle, D. Trager, D.N. Neshev, A. Dreischuh, C. Denz, W. Krolikowski, and Y.S. Kivshar, Two-dimensional self-trapped nonlinear photonic lattices, Opt. Express 14, 2851 (2006).
  * [14] D.C. Cole, E.S. Lamb, P. Del’Haye, S.A. Diddams, and S.B. Papp, Soliton crystals in Kerr resonators, Nat. Photon. 11, 671 (2017).
  * [15] M. Karpov, M.H.P. Pfeiffer, H.R. Guo, W.L. Weng, J. Liu, and T.J. Kippenberg, Dynamics of soliton crystals in optical microresonators, Nat. Phys. 15, 1071 (2019).
  * [16] Z. Lu, H.J. Chen, W.Q. Wang, L. Yao, Y. Wang, Y. Yu, B.E. Little, S.T. Chu, Q. Gong, W. Zhao, X. Yi, Y.F. Xiao, and W.F. Zhang, Synthesized soliton crystals, Nat. Commun. 12, 3179 (2021).
  * [17] H. Weng, A.A. Afridi, J. Liu, J. Li, J. Dai, X. Ma, Y. Zhang, Q. Lu, W. Guo, and J.F. Donegan, Near-octave-spanning breathing soliton crystal in an AlN microresonator, Opt. Lett. 46, 3436 (2021).
  * [18] H. Taheri, A.B. Matsko, L. Maleki, and K. Sacha, All-optical dissipative discrete time crystals, Nat. Commun. 13, 848 (2022).
  * [19] H. Taheri, A.B. Matsko, T. Herr, and K. Sacha, Dissipative discrete time crystals in a pump-modulated Kerr microcavity, Commun. Phys. 5, 159 (2022).
  * [20] A. Schwache and F. Mitschke, Properties of an optical soliton gas, Phys. Rev. E 55, 7720 (1997).
  * [21] A. Gelash, D. Agafontsev, V. Zakharov, G. El, S. Randoux, and P. Suret, Bound state soliton gas dynamics underlying the spontaneous modulational instability, Phys. Rev Lett. 123, 234102 (2019).
  * [22] P. Grelu, and N. Akhmediev, Dissipative solitons for mode-locked lasers, Nat. Phot. 6, 84 (2012).
  * [23] F. Amrani, M. Salhi, P. Grelu, H. Leblond, and F. Sanchez, Universal soliton pattern formations in passively mode-locked fiber lasers, Opt. Lett. 36, 1545 (2011).
  * [24] A. Andrianov, and A. Kim, Widely stretchable soliton crystals in a passively mode-locked fiber laser, Opt. Exp. 29, 25202 (2021).
  * [25] J.K. Chana, M. Sich, F. Fras, A.V. Gorbach, D.V. Skryabin, E. Cancellieri, E.A. Cerda-Mendez, K. Biermann, R. Hey, P.V. Santos, M.S. Skolnick, and D.N. Krizhanovskii, Spatial patterns of dissipative polariton solitons in semiconductor microcavities, Phys. Rev. Lett. 115, 256401 (2015).
  * [26] H. Guo, M. Karpov, E. Lucas, A. Kordts, M.H.P. Pfeiffer, V. Brasch, G. Lihachev, V.E. Lobanov, M.L. Gorodetsky, and T.J. Kippenberg, Universal dynamics and deterministic switching of dissipative Kerr solitons in optical microresonators, Nat. Phys. 13, 94 (2017).
  * [27] Y. Xu, Y. Lin, A. Nielsen, I. Hendry, S. Coen, M. Erkintalo, H. Ma, and S.G. Murdoch, Harmonic and rational harmonic driving of microresonator soliton frequency combs, Optica 7, 940 (2020).
  * [28] Y. Lumer, Y. Plotnik, M.C. Rechtsman, and M. Segev, Self-localized states in photonic topological insulators, Phys. Rev. Lett. 111, 243905 (2013).
  * [29] Y.V. Kartashov and D.V. Skryabin, Modulational instability and solitary waves in polariton topological insulators, Optica 3, 1228 (2016).
  * [30] D. Leykam and Y.D. Chong, Edge solitons in nonlinear-photonic topological insulators, Phys. Rev. Lett. 117, 143901 (2016).
  * [31] S. Mukherjee and M.C. Rechtsman, Observation of Floquet solitons in a topological bandgap, Science 368, 856 (2020).
  * [32] Z. Yang, E. Lustig, G. Harari, Y. Plotnik, Y. Lumer, M.A. Bandres, and M. Segev, Mode-locked topological insulator laser utilizing synthetic dimensions, Phys. Rev. X 10, 011059 (2020).
  * [33] V. Peano, M. Houde, C. Brendel, F. Marquardt, and A.A. Clerk, Topological phase transitions and chiral inelastic transport induced by the squeezing of light, Nat. Commun. 7, 10779 (2016).
  * [34] T. Goren, K. Plekhanov, F. Appas, and K. Le Hur, Topological Zak phase in strongly coupled LC circuits, Phys. Rev. B 97, 041106(R) (2018).
  * [35] E. Kim, X. Zhang, V.S. Ferreira, J. Banker, J.K. Iverson, A. Sipahigil, M. Bello, A. Gonzalez-Tudela, M. Mirhosseini, and O. Painter, Quantum electrodynamics in a topological waveguide, Phys. Rev. X 11, 011015 (2021).
  * [36] Y.V. Kartashov and D.V. Skryabin, Bistable topological insulator with exciton-polaritons, Phys. Rev. Lett. 119, 253904 (2017).
  * [37] D.A. Dobrykh, A.V. Yulin, A.P. Slobozhanyuk, A.N. Poddubny, and Y.S. Kivshar, Nonlinear control of electromagnetic topological edge states, Phys. Rev. Lett. 121, 163901 (2018).
  * [38] P. St-Jean, V. Goblot, E. Galopin, A. Lemaître, T. Ozawa, L. Gratiet, I. Sagnes, J. Bloch, and A. Amo, Lasing in topological edge states of a one-dimensional lattice, Nat. Photonics 11, 651 (2017).
  * [39] M.A. Bandres, S. Wittek, G. Harari, M. Parto, J. Ren, M. Segev, D.N. Christodoulides, and M. Khajavikhan, Topological insulator laser: Experiments, Science 359, eaar4005 (2018).
  * [40] D. Leykam, and L. Yuan, Topological phases in ring resonators: recent progress and future prospects, Nanophotonics 9, 4473 (2020).
  * [41] S. Mittal, G. Moille, K. Srinivasan, Y.K. Chembo, and M. Hafezi, Topological frequency combs and nested temporal solitons, Nat. Phys. 17, 1169 (2021).
  * [42] L.J. Maczewsky, M. Heinrich, M. Kremer, S.K. Ivanov, M. Ehrhardt, F. Martinez, Y.V. Kartashov, V.V. Konotop, L. Torner, D. Bauerand, and A. Szameit, Nonlinearity-induced photonic topological insulator, Science 370, 701 (2020).
  * [43] M.V. Berry, Quantal phase factors accompanying adiabatic changes, Proc. R. Soc. Lond. A 392, 45 (1984).
  * [44] J. Zak, Berry’s phase for energy bands in solids, Phys. Rev. Lett. 62, 2747 (1989).
  * [45] F. Wilczek, Quantum time crystals, Phys. Rev. Lett. 109, 160401 (2012).
  * [46] J. Zhang, et al., Observation of a discrete time crystal, Nature 543, 217 (2017).
  * [47] S. Choi, et al., Observation of discrete time-crystalline order in a disordered dipolar many-body system, Nature 543, 221 (2017).
  * [48] E. Lustig, Y. Sharabi, and M. Segev, Topological aspects of photonic time crystals, Optica 5, 1390 (2018).
  * [49] K. Giergiel, A. Dauphin, M. Lewenstein, J. Zakrzewski, and K. Sacha, Topological time crystals, New J. Phys. 21, 052003 (2019).
  * [50] L. Guo and P. Liang, Condensed matter physics in time crystals, New J. Phys. 22, 075003 (2020).
  * [51] X. Mi et al., Time-crystalline eigenstate order on a quantum processor, Nature 601, 531 (2022).
  * [52] J.B. Pendry, A chiral route to negative refraction, Science 306, 1353 (2004).
  * [53] S.M. Morrow, A.J. Bissette, and S.P. Fletcher, Transmission of chirality through space and across length scales, Nat. Nanotechnology 12, 410 (2017).
  * [54] N.C. Zambon, P. St-Jean, M. Milicevic, A. Lemaitre, A. Harouri, L. Le Gratiet, O. Bleu, D.D. Solnyshkov, G. Malpuech, I. Sagnes, S. Ravets, A. Amo, and J. Bloch, Optically controlling the emission chirality of microlasers, Nature Photonics 13, 283 (2019).
  * [55] F. Wilczek, ”Chirality: A scientific leitmotif”, https://arxiv.org/abs/2112.06927 (2021).
  * [56] D.V. Skryabin, Z. Fan, A. Villois, and D.N. Puzyrev, Threshold of complexity and Arnold tongues in Kerr-ring microresonators, Phys. Rev. A 103, L011502 (2021).
  * [57] G. Herink, F. Kurtz, B. Jalali, D.R. Solli, and C. Ropers, Real-time spectral interferometry probes the internal dynamics of femtosecond soliton molecules, Science 356, 50 (2017).
  * [58] K. Krupa, K. Nithyanandan, U. Andral, P. Tchofo-Dinda, and P. Grelu, Real-time observation of internal motion within ultrafast dissipative optical soliton molecules, Phys. Rev. Lett. 118, 243901 (2017).
  * [59] W.L. Weng, R. Bouchand, E. Lucas, E. Obrzud, T. Herr, and T.J. Kippenberg, Heteronuclear soliton molecules in optical microresonators, Nat. Commun. 11, 2402 (2020).
  * [60] S.S. Gopalakrishnan, K. Panajotov, M. Taki, and M. Tlidi, Dissipative light bullets in Kerr cavities: Multistability, clustering, and rogue waves, Phys. Rev. Lett. 126, 153902 (2021).
  * [61] C. Milian and D.V. Skryabin, Soliton families and resonant radiation in a micro-ring resonator near zero group-velocity dispersion, Opt. Express 22, 3732 (2014).
  * [62] J. Zak, Band center - a conserved quantity in solids, Phys. Rev. Lett. 48, 359 (1982).
  * [63] R.D. King-Smith and D. Vanderbilt, Theory of polarization of crystalline solids, Phys. Rev. B 47, 1651 (1993).
  * [64] R. Resta, Macroscopic polarization in crystalline dielectrics: the geometric phase approach, Rev. Mod. Phys. 66, 899 (1994).
  * [65] M. Atala, M. Aidelsburger, J.T. Barreiro, D. Abanin, T. Kitagawa, E. Demler, and I. Bloch, Direct measurement of the Zak phase in topological Bloch bands, Nat. Phys. 9, 795 (2013).
  * [66] M. Wimmer, H.M. Price, I. Carusotto, and U. Peschel, Experimental measurement of the Berry curvature from anomalous transport, Nat. Phys. 13, 545 (2017).
  * [67] Z.Q. Jiao, S. Longhi, X.W. Wang, J. Gao, W.H. Zhou, Y. Wang, Y.X. Fu, L. Wang, R.J. Ren, L.F. Qiao, and X.M. Jin, Experimentally detecting quantized Zak phases without chiral symmetry in photonic lattices, Phys. Rev. Lett. 127, 147401 (2021).
  * [68] A. Bjarklev, J. Broeng, A.S. Bjarklev, Photonic Crystal Fibres (Springer, 2003).
  * [69] B. Corcoran, M.X. Tan, X.Y. Xu, A. Boes, J.Y. Wu, T.G. Nguyen, S.T. Chu, B.E. Little, R. Morandotti, A. Mitchell, and D.J. Moss, Ultra-dense optical data transmission over standard fibre with a single chip source, Nat. Commun. 11, 2568 (2020).
  * [70] M.A. Guidry, D.M. Lukin, K.Y. Yang, R. Trivedi, and J. Vuckovic, Quantum optics of soliton microcombs, Nature Photonics 16, 52 (2022).
  * [71] M. Kues, C. Reimer, J.M. Lukens, W.J. Munro, A.M. Weiner, D.J. Moss, and R. Morandotti, Quantum optical microcombs, Nat. Photon. 13, 170 (2019).
  * [72] A.S. Raja, S. Lange, M. Karpov, K. Shi, X. Fu, R. Behrendt, D. Cletheroe, A. Lukashchuk, I. Haller, F. Karinou, B. Thomsen, K. Jozwik, J.Q. Liu, P. Costa, T.J. Kippenberg, and H. Ballani, Ultrafast optical circuit switching for data centers using integrated soliton microcombs, Nat. Commun. 12, 5867 (2021).
  * [73] S. Mittal, E.A. Goldschmidt, and M. Hafezi, A topological source of quantum light, Nature 561, 502 (2018).
  * [74] R.D. Dikande Bitha, and A.M. Dikande, Soliton-comb structures in ring-shaped optical microresonators: generation, reconstruction and stability, Eur. Phys. J. D 73, 152 (2019).
  * [75] R.D. Dikande Bitha and A.M. Dikande, Elliptic-type soliton combs in optical ring microresonators, Phys. Rev. A 97, 033813 (2018).
  * [76] D.V. Skryabin, Instabilities of vortices in a binary mixture of trapped Bose-Einstein condensates: Role of collective excitations with positive and negative energies, Phys. Rev. A 63, 013602 (2000).
  * [77] P. Parra-Rivas, D. Gomila, L. Gelens, and E. Knobloch, Bifurcation structure of periodic patterns in the Lugiato-Lefever equation with anomalous dispersion, Phys. Rev. E 98, 042212 (2018).
  * [78] G. Engelhardt and T. Brandes, Topological Bogoliubov excitations in inversion-symmetric systems of interacting bosons, Phys. Rev. A 91, 053621 (2015).

Acknowledgements
We acknowledge the financial support received from EU Horizon 2020 Framework
Programme (812818, MICROCOMB) and UK EPSRC (2119373).