††thanks: These authors contributed equally††thanks: These authors contributed
equally

# Orbital Frustration and Emergent Flat Bands

Wenjuan Zhang  Department of Physics, Ohio State University, Columbus, OH
43210, USA  Zachariah Addison  Department of Physics, Ohio State University,
Columbus, OH 43210, USA  Nandini Trivedi  Department of Physics, Ohio State
University, Columbus, OH 43210, USA

###### Abstract

We expand the concept of frustration in Mott insulators and quantum spin
liquids to metals with flat bands. We show that when inter-orbital hopping
$t_{2}$ dominates over intra-orbital hopping $t_{1}$, in a multiband system
with strong spin-orbit coupling $\lambda$, electronic states with a narrow
bandwidth $W\sim t_{2}^{2}/\lambda$ are formed compared to a bandwidth of
order $t_{1}$ for intra-orbital hopping. We demonstrate the evolution of the
electronic structure, Berry phase distributions for time-reversal and
inversion breaking cases, and their imprint on the optical absorption, in a
tight binding model of $d$-orbital hopping on a honeycomb lattice. Going
beyond quantum Hall effect and twisted bilayer graphene, we provide an
alternative mechanism and a richer materials platform for achieving flat bands
poised at the brink of instabilities toward novel correlated and
fractionalized metallic phases.

††preprint: APS/123-QED

Introduction: Symmetry and topology have played a fundamental role in
characterizing emergent phases in quantum matter. From the well understood
emergence of long-range order in crystals, magnets and superconductors,
research has moved to the frontiers of emergence of a new type of truly exotic
order, topological order in which the quantum numbers of the excitations are
fractionalized. A key ingredient to obtain fractionalization is frustration of
the motion of the electron that leads to a small kinetic energy and hence to
flat bands in the electronic spectrum. Our main result is to provide a design
principle for creating materials in which the inter-orbital hopping $t_{2}$
dominates over intra-orbital hopping $t_{1}$, in a multiband system with
strong spin-orbit coupling $\lambda$, that generates electronic states with a
narrow bandwidth $W\sim t_{2}^{2}/\lambda$ compared to a bandwidth of order
$t_{1}$ for intra-orbital hopping. We expect these designer flat-band
materials to provide a link between frustrated metals and frustrated magnetic
insulators opening a doorway for the search and discovery of topologically
ordered states.

Emergent phases in quantum matter are understood today under two paradigms:
the Landau paradigm that has been the pillar for phases arising from the
spontaneous breaking of symmetry and the topological paradigm in which phases
are characterized by topological invariants leading to quantized response
functions. Within the topological paradigm there are broadly two classes:
phases where interactions are less important, such as the integer quantum Hall
effect, topological insulators [1, 2], Weyl semimetals [3, 4], and topological
superconductors [5, 6], and the more exotic class, where interactions dominate
along with topology, yielding a new type of order, dubbed topological order
[7, 8, 9]. These include two candidates so far: (1) the fractional quantum
Hall effect [10, 11, 12, 13] in a two-dimensional electron gas subjected to a
large magnetic field in which charge is fractionalized, and (2) quantum spin
liquids in Mott insulators in which spin is fractionalized [14, 15, 16].

Our broad motivation is to search for materials that have flatbands in the
absence of a magnetic field in which strong correlations can generate
topologically ordered phases that show fractionalization of charge,
fractionalization of spin, and quantized anomalous and topological Hall
effects.

To put this in perspective, fractionalization in FQHE arises because the
kinetic energy is quenched and the bands become completely flat in a high
magnetic field upon projecting to the lowest Landau level (see Fig. 1 (a)),
and consequently, the relatively strong electron-electron Coulomb interactions
open a bulk gap in the metallic regime with edge states that carry the
signature of fractional charge in quantized Hall and quantized thermal Hall
transport.

Quantum spin liquids are expected in strongly correlated systems, in which a
Mott gap separates the high energy charge dynamics from the low energy spin
dynamics. Further, the magnetic interactions are frustrated either from the
geometry, or from competing interactions, that can lead to spin textures and
quantum spin liquid phases [17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27]. In
Kitaev spin liquids on the honeycomb geometry, frustration arises due to bond-
dependent interactions arising from strong spin-orbit coupling. As a result of
the strong frustration, spins are fractionalized into majorana modes that have
recently been shown to produce a quantized thermal Hall conductance in a
candidate material $\alpha$–${\rm RuCl}_{3}$ [16, 28, 29, 30, 31].

One route is to investigate transition metal based materials that have narrow
bands and hence strong Coulomb interactions $U$ compared to the wider-band s-
and p- band materials. For example, in the cuprate superconductors, the
bandwidth is $W=8t\approx 2.5eV$ and U is 5 eV giving a ratio $U/W\approx 2$.
Tunability of $U$ is difficult as it is determined by the atomic correlations
in transition metal compounds, so a complimentary approach is to reduce the
bandwidth $W$ in order to enhance the ratio $U/W$.

Recently, the community has seen feverish activity in twisted bilayer graphene
in which flat bands emerge when the angle of mismatch is a very specific magic
angle of around $1^{\circ}$ yielding a large scale moiré pattern with lattice
constant much larger than that of a graphene sheet [32, 33]; (see Fig. 1 (b)).
For bilayers encapsulated by hBN the long ranged Coulomb energy scale is
$U\sim 10-15$ meV and the bandwidths of the flattest bands are $W\lesssim 5$
meV [34]. For the most ideal magic angles the flat bands are separated from
other dispersing bands by a energy gap $\Delta\sim 10$ meV leading to a
flatness ratio of $\mathcal{F}=W/\Delta\lesssim 0.5$ [35, 36]. The flat bands
have resulted in the observation of strongly correlated superconducting and
Mott insulating phases [37, 38, 39, 40, 41, 42, 43]; however so far
fractionalized metallic states have not been observed.

Figure 1:  Figure showing three mechanisms for generating flat bands. (a) In
quantum Hall systems flat bands emerge when a large magnetic field $\bm{B}$ is
applied to a two-dimensional electron gas that localizes the orbital motion of
the electrons in the bulk of the material. (b) Flat bands in twisted bilayer
graphene emerge from a periodic interlayer Moiré potential $V_{M}(\bm{r})$
whose period $a_{M}$ is much larger than the intralayer graphene lattice
constant $a_{G}$. When the angle of rotational misalignment between the
intralayer lattice vectors is equal to a magic angle , the potential acts as
an effective time preserving pseudo flux $\Phi_{B}$ that greatly reduces the
kinetic energy of the electron [44, 45, 46]. These systems have a flatness
ratio of $\mathcal{F}\lesssim 0.5$. (c) In our model flat bands are achieved
through spin-orbit assisted inter-orbital frustration whereby dominant inter-
orbital kinetic hopping can quench the kinetic energy of electrons in an
effective $j=1/2$ multiplet via a large onsite spin-orbit interaction on the
honeycomb lattice leading to a flatness ratio
$\mathcal{F}\sim(t_{2}/\lambda)^{2}$ tuned by the ratio of the inter-orbital
kinetic hopping strength $t_{2}$ and the spin orbit interaction $\lambda$. The
strong interacting limit of dominant intra-orbital or dominant inter-orbital
hopping leads to an effective Heisenberg Hamiltonian or an effective Kitaev
Hamiltonian respectively.

In this paper we demonstrate a new mechanism for achieving flat bands in
metallic systems with strong spin-orbit coupling. The basic mechanism involves
frustrating the motion of an electron by suppressing intra-orbital hopping
compared to hopping between different orbitals; we dub this mechanism spin-
orbit assisted inter-orbital frustration (see Fig. 1 (c)). We demonstrate the
emergence of flatbands in the metallic regime, with the promise of
fractionalized metals and other exotic magnetic phases upon including
correlations, in a tightbinding model of $d$-orbitals on a honeycomb lattice
with strong spin-orbit coupling.

We choose the parameters of our tightbinding Hamiltonian to ensure that with
on-site Hubbard interactions, in the strong coupling limit, we obtain the
Kitaev model with bond-dependent interactions and contrast it with the
parameters that generate the isotropic Heisenberg model (see Appendix D). Our
main result is for the metallic itinerant phase, where the presence of strong
spin-orbit interactions, $\lambda$, can lead to frustration whereby the
kinetic energy of a multiplet of bands is suppressed leading to small
bandwidths of order $(t_{2})^{2}/\lambda$ and small flatness ratios
$\mathcal{F}\sim(t_{2}/\lambda)^{2}$ in systems where intersite inter-orbital
hopping, $t_{2}$, dominates over intersite intra-orbital hopping $t_{1}$. We
calculate the electrical conductivity in two regimes: systems dominated by
$t_{2}$ and systems dominated by $t_{1}$, for which we show the crossover and
consequences of the momentum space Berry phase distributions in the emergent
flat band phase.

Figure 2: Band structures with spin orbit coupling $\lambda_{0}$ on a
honeycomb lattice: (a) Bands $\varepsilon(\bm{k})$, in units of $\lambda_{0}$,
for dominant inter-orbital hopping $t_{2}=0.1\lambda_{0}$ and intra-orbital
hopping $t_{1}=t_{3}=0$. The bandwidth of upper $j=1/2$ multiplet
$\Delta_{1/2}^{\text{inter}}\sim t^{2}/\lambda_{0}\sim 0.01\lambda_{0}$. (b)
Inset highlighting the flat bands in upper $j=1/2$ manifold of (a). (c) Bands
of intra-orbital dominated hopping $t_{1}=t_{3}=0.1\lambda_{0}$ and $t_{2}=0$.
The bandwidth of upper $j=1/2$ multiplet is $\Delta_{1/2}^{\text{intra}}\sim
t_{1}\sim 0.1\lambda_{0}$. (d) Honeycomb lattice in [001] plane showing
orthogonal Cartesian axes $(\hat{\bm{x}},\hat{\bm{y}},\hat{\bm{z}})$, bond
axes $(\hat{\bm{\gamma}}_{x},\hat{\bm{\gamma}}_{y},\hat{\bm{\gamma}}_{z})$,
and orbital axes $(\hat{\bm{e}}_{x},\hat{\bm{e}}_{y},\hat{\bm{e}}_{z})$ (see
Appendix A), and intersite intra-orbital and intersite inter-orbital
processes.

Model: The octahedral symmetry around a transition metal ion splits the
$d$-orbitals into a 3-fold $t_{2g}$ sector and a 2-fold degenerate $e_{g}$
sector separated by the crystal field $\Delta_{CF}$ energy. In the low energy
$t_{2g}$ sector, the general tight binding Hamiltonian on a honeycomb lattice
is given by:

$\displaystyle\hat{H}$
$\displaystyle=\sum_{<ij>,\alpha\beta,\sigma}t^{\alpha\beta}_{ij}\hat{d}^{\dagger}_{\alpha\sigma}(\bm{r}_{i})\hat{d}_{\beta\sigma}(\bm{r}_{j})$
$\displaystyle+\sum_{i,\alpha\beta,\sigma\sigma^{\prime}}\lambda^{\alpha\beta}_{\sigma\sigma^{\prime}}\hat{d}^{\dagger}_{\alpha\sigma}(\bm{r}_{i})\hat{d}_{\beta\sigma^{\prime}}(\bm{r}_{i})$
(1)

where $\hat{d}^{\dagger}_{\alpha\sigma}(\bm{r}_{i})$ and
$\hat{d}_{\alpha\sigma}(\bm{r}_{i})$ are the fermionic annihilation and
creation operators at site $\bm{r}_{i}$ for orbital $\alpha$ and spin
$\sigma$.

The first term in Eq. 1 describes the hopping between $d$-orbitals at adjacent
sites $\langle i,j\rangle$ on the honeycomb lattice and can be written in
terms of matrices $T_{\gamma(\bm{r}_{i},\bm{r}_{j})}$ where
$\gamma(\bm{r}_{i},\bm{r}_{j})$ takes three values
$\gamma(\bm{r}_{i},\bm{r}_{j})=x,y,z$ corresponding to the vector $\bm{v}_{s}$
that connects the sites $\bm{r}_{i}$ and $\bm{r}_{j}$ (see Appendix A). We
write the symmetry allowed components of these matrices
$t_{ij}^{\alpha\beta}\delta_{\sigma\sigma^{\prime}}=T_{\gamma(\bm{r}_{i},\bm{r}_{j})}^{\alpha\beta}\delta_{\sigma\sigma^{\prime}}=\bra{d_{\alpha\sigma}(\bm{r}_{i})}\hat{t}\ket{d_{\beta\sigma^{\prime}}(\bm{r}_{j})}$
in the basis of orbitals
$\\{\ket{d_{yz}(\bm{r}_{i})},\ket{d_{zx}(\bm{r}_{i})},\ket{d_{xy}(\bm{r}_{i})}\\}$
as

$T_{x}=\left[\begin{array}[]{ccc}t_{3}&0&0\\\ 0&t_{1}&t_{2}\\\
0&t_{2}&t_{1}\end{array}\right],T_{y}=\left[\begin{array}[]{ccc}t_{1}&0&t_{2}\\\
0&t_{3}&0\\\
t_{2}&0&t_{1}\end{array}\right],T_{z}=\left[\begin{array}[]{ccc}t_{1}&t_{2}&0\\\
t_{2}&t_{1}&0\\\ 0&0&t_{3}\end{array}\right]$ (2)

Symmetry constrains the nearest neighbor hopping matrices to have three
independent parameters $t_{1}$, $t_{2}$, and $t_{3}$. The intersite inter-
orbital coupling is described by $t_{2}$, while $t_{1}$ and $t_{3}$ describe
the intersite intra-orbital couplings.

The second term in Eq. 1 describes the onsite spin-orbit interaction
$\lambda^{\alpha\beta}_{\sigma\sigma^{\prime}}\delta_{ij}=\bra{d_{\alpha\sigma}(\bm{r}_{i})}\hat{\lambda}\ket{d_{\beta\sigma^{\prime}}(\bm{r}_{j})}$
between the $d$-orbitals in the $t_{2g}$ sector:
$\lambda=\dfrac{2\lambda_{0}}{\hbar^{2}}\sum_{i}L_{\hat{\bm{e}}_{i}}\otimes
S_{\hat{\bm{e}}_{i}}$ (see Appendix A). Here
$S_{\hat{\bm{e}}_{i}}=(\hbar/2)\sigma_{\hat{\bm{e}}_{i}}$ and
$\sigma_{\hat{\bm{e}}_{i}}$ is the Pauli matrix in the $\hat{\bm{e}}_{i}$
direction. In the absence of $\hat{t}$, the spin-orbit interactions splits the
$t_{2g}$ sector into states of total angular momentum $j=1/2$ and $j=3/2$.

Emergent Flat Bands: By using the discrete translation symmetry of the system,
the Hamiltonian in Eq. 1 yields a $12\times 12$ matrix [$(2j+1)\times 2\ {\rm
atoms}$], the Bloch Hamiltonian,
$\hat{H}(\bm{k})=e^{-i\hat{\bm{r}}\cdot\bm{k}}\hat{H}e^{i\hat{\bm{r}}\cdot\bm{k}}$
at each ${\bf k}$-point. The eigenvalues of the Hamiltonian must at least be
two-fold degenerate for all crystal momenta $\bm{k}$ due to Kramers
degeneracy. In the absence of $\hat{t}$ the bands separate into two manifolds:
a manifold with total angular momentum $j=1/2$ containing four bands and a
manifold with total angular momentum $j=3/2$ containing eight bands.

In the purely inter-orbital hopping limit, intra-orbital interactions are
turned off ($t_{1}=t_{3}=0$) and the orbital interactions are dominated by the
inter-orbital coupling $t_{2}$. The $j=3/2$ multiplet is split into four two-
fold degenerate bands, while the $j=1/2$ multiplet consists of two two-fold
degenerate bands with almost no dispersion (see Fig. 2 (a)). The intraband
kinetic energy is quenched due to an inter-orbital-spin-orbit frustration
caused by the large onsite spin-orbit interaction $\lambda$ and nonzero
$t_{2}$. This is in striking contrast to the purely intra-orbital ($t_{2}=0$)
limit where bands are seen to disperse in both the $j=1/2$ and $j=3/2$
multiplet (see Fig. 2 (c) and Appendix C).

To see the effect of nonzero $\hat{t}$ we use degenerate perturbation theory
to calculate the energy spitting of the spin-orbit split bands in orders of
the kinetic hopping coefficients $t_{1}$, $t_{2}$, and $t_{3}$. Remarkably for
purely inter-orbital hopping the first order correction to the energy bands of
the $j=1/2$ manifold vanishes. This is due to the orbital character of the
$j=1/2$ manifold and the orbital character of the purely inter-orbital hopping
matrices. The action of these matrices on the $j=1/2$ manifold transforms a
$j=1/2$ state into a mixture of $j=3/2$ states. These states are all
orthogonal to the $j=1/2$ manifold as they have different eigenvalues with
respect to $\hat{\lambda}$. In order to find the band splitting we must then
go to second order perturbation theory leading to an effective bandwidth in
the purely inter-orbital hopping limit
$\Delta^{\text{inter}}_{1/2}\sim(t_{2})^{2}/\lambda$. This is in contrast to
the band splitting of the $j=3/2$ manifold of states and of either multiplet
in the purely intra-orbital hopping limit where the first order correction to
the spin-orbit split flat bands is nonzero. In these cases the action of the
kinetic hopping, $\hat{t}$, transforms a state with eigenvalue $j=1/2$ or
$j=3/2$ into a linear combination of $j=1/2$ and $j=3/2$ states, leading to a
bandwidth $\Delta\sim t$.

These results are most easily determined by writing the Bloch Hamiltonian in
the basis of total angular momentum eigenstates $j=1/2,3/2$. In this basis the
Hamiltonian takes the form

$\hat{H}(\bm{k})=\left(\begin{array}[]{cccc}\lambda_{0}\mathds{1}_{4\times
4}+\bm{\mathcal{M}}^{1/2}_{k}(t_{1},t_{3})&\bm{G}_{k}(t_{1},t_{2},t_{3})\\\
\bm{G}_{k}^{\dagger}(t_{1},t_{2},t_{3})&-\dfrac{\lambda_{0}}{2}\mathds{1}_{8\times
8}+\bm{\mathcal{M}}^{3/2}_{k}(t_{1},t_{2},t_{3})\end{array}\right)$ (3)

where $\bm{\mathcal{M}}^{1/2}_{k}(t_{1},t_{3})$ is a $4\times 4$ matrix,
$\bm{\mathcal{M}}^{3/2}_{k}(t_{1},t_{2},t_{3})$ is a $8\times 8$ matrix, and
$\bm{G}_{k}(t_{1},t_{2},t_{3})$ is a $4\times 8$ matrix that are all linear
functions of the kinetic couplings $t_{i}$. We see that
$\bm{\mathcal{M}}^{1/2}_{k}(t_{1},t_{3})$ is independent of the interorbital
kinetic coupling $t_{2}$ and thus vanishes in the purely interorbital hopping
limit $t_{1}=t_{3}=0$, while the other matrices remain non-zero. This forces
the linear in $t$ perturbations to the eigenvalues of the $j=1/2$ manifold in
the upper-left $4\times 4$ block of $\hat{H}(\bm{k})$ to vanish, while in this
limit the lower-right $8\times 8$ block
$\bm{\mathcal{M}}_{3/2}(t_{1},t_{2},t_{3})$ is non-vanishing and leads to the
linear in $t$ perturbation to the eigenvalues of the $j=3/2$ sector to be
nonzero. In contrast for the purely intra-orbital hopping limit the matrices
$\bm{\mathcal{M}}^{1/2}_{k}(t_{1},t_{3})$,
$\bm{\mathcal{M}}^{3/2}_{k}(t_{1},t_{2},t_{3})$, and
$\bm{G}_{k}(t_{1},t_{2},t_{3})$ are all non-zero leading to nonzero first
order perturbations to the spin-orbit split flat bands (for details see
Appendix C).

In the presence of nonzero intra-orbital hopping the first order kinetic
correction to the spin-orbit split bands is of order $t_{1}$ and $t_{3}$. In
the limit $t_{1},t_{3}<<(t_{2})^{2}/\lambda$ the dispersion of the
eigenenergies of the full Bloch Hamiltonian is proportional to the inter-
orbital hopping $t_{2}$ and inversely proportional to the onsite spin-orbit
scale $\lambda$, and leads to bandwidths in the $J=1/2$ multiplet of states
$\Delta_{1/2}\sim(t_{2})^{2}/\lambda$ similar to the completely frustrated
limit ($t_{1}=t_{3}=0$). While in the limit $(t_{2})^{2}/\lambda\lesssim
t_{1},t_{3}$ the dominant intra-orbital dynamics diminishes orbital
frustration and leads to bandwidths in the $J=1/2$ multiplet $\Delta_{1/2}\sim
t_{1},t_{3}$ (see Appendix C).

The mechanism described above leading to the flat bands in the $J=1/2$
multiplet in the strong inter-orbital hopping limit is just one example of
spin-orbit assisted inter-orbital frustration. Engineering the multi-orbital
onsite potentials and synthetic multi-dimensional inter-site gauge fields that
couple onsite degrees of freedom to neighboring sites on the lattice in highly
tunable platforms, like cold atoms systems, can provide a framework for
exploring orbital frustration and flat bands in different lattice systems with
varying onsite and inter-site multi-dimensional spin-orbit potentials [47, 48,
49, 50, 51].

Quantum Band Geometry and Symmetry Breaking: The quantum geometry of a band
structure is determined by a band’s Berry potential
$\bm{A}_{n}(\bm{k})=i\bra{u_{n}(\bm{k})}\bm{\nabla}_{k}\ket{u_{n}(\bm{k})}$
and curvature $\bm{\Omega}_{n}(\bm{k})=\nabla_{k}\times\bm{A}_{n}(\bm{k})$,
whose integral across the 2D Brillouin zone dictates the winding of the phase
of the Bloch wavefunction: an integer topological invariant called the Chern
number. In the purely atomic limit, whereby the hopping of electrons across
lattice sites is strictly forbidden, the Berry curvature is trivially zero.
Here we show that in the presence of strong inter-site inter-orbital
interactions, $t_{2}$, the orbital structure of Bloch eigenstates becomes
strongly mixed across the Brillouin zone leading to unique Berry curvature
density in the presence of time reversal and inversion breaking perturbations
(see Fig 4(B) in Appendix E).

The Berry curvature is a pseudo-vector such that for inversion symmetric
systems $\bm{\Omega}^{k}_{n}(-\bm{k})=\bm{\Omega}^{k}_{n}(\bm{k})$ and for
time reversal symmetric systems
$\bm{\Omega}^{k}_{n}(-\bm{k})=-\bm{\Omega}^{k}_{n}(\bm{k})$. In the models
described above time reversal and inversion symmetry constrain the Berry
curvature to vanish in every band $n$ and at all crystal momentum $\bm{k}$, so
$\bm{\Omega}_{n}(\bm{k})=0\ {\forall\bm{k}},{\forall n}$.

In order to observe non-zero Berry curvature we perturb our model system with
a time reversal breaking perturbation that couples directly to the electronic
spin or with an inversion breaking onsite sublattice potential

$\displaystyle\hat{H}^{\prime}_{\mathcal{T}}$
$\displaystyle=\sum_{i,\sigma\sigma^{\prime},\alpha}\mu_{B}\bm{B}_{0}\cdot\bm{\sigma}_{\sigma\sigma^{\prime}}\hat{d}^{\dagger}_{\alpha\sigma}(\bm{r}_{i})\hat{d}_{\alpha\sigma^{\prime}}(\bm{r}_{i})$
(4) $\displaystyle\hat{H}^{\prime}_{\mathcal{I}}$
$\displaystyle=\sum_{i,\sigma,\alpha}m_{0}(\bm{r}_{i})\hat{d}^{\dagger}_{\alpha\sigma}(\bm{r}_{i})\hat{d}_{\alpha\sigma}(\bm{r}_{i})$
(5)

where $m_{0}(\bm{r}_{i})=+1(-1)$ if site $\bm{r}_{i}$ is in the $A(B)$ sub
lattice. Below we show the band structure and Berry curvature for the second
lowest band in the $j=1/2$ multiplet in the intra-orbital and inter-orbital
hopping dominated limits.

In Fig. 4(A) time reversal is broken and bands in the $j=1/2$ manifold are
split in both the purely intra-orbital and purely inter-orbital hopping
limits. In the intra-orbital limit internal orbital symmetry allows bands to
cross near the $\bm{K}$ and $\bm{K}^{\prime}$ points in the Brillouin zone. In
this case the Berry curvature vanishes everywhere except at the points where
bands cross and the curvature becomes ill defined. In the inter-orbital
dominated limit, peaks in the Berry curvature occur near the $\bm{M}$ points
in the Brillouin zone where the band gap between the lowest and second lowest
bands of the $j=1/2$ multiplet becomes narrow.

Fig. 4(B) shows results for systems with broken inversion symmetry. In the
intra-orbital limit, internal orbital symmetry allows bands in the $j=1/2$
manifold to remain two-fold degenerate at every point in the Brillouin zone
(see Appendix E). We see that the Berry curvature is localized at the $\bm{K}$
and $\bm{K}^{\prime}$ points where the band gap between the lower and upper
bands of the $j=1/2$ multiplet narrows. In the inversion broken inter-orbital
hopping dominated limit, the narrow bands in the $j=1/2$ multiplet are split
everywhere except at the $\bm{\Gamma}$ and $\bm{M}$ points where the bands
come together and form two two-fold degenerate points and the Berry curvature
is ill defined. Away from these points the Berry curvature disperses in a
three-fold symmetric pattern with its saddle points along high symmetry lines
connecting the $\bm{\Gamma}$ and $\bm{M}$ points in the Brillouin zone.

Optical Signatures: The topological and geometric character of the band
structures of the purely intra-orbital hopping and inter-orbital hopping lmits
can be distinguished in the system’s response to a frequency-dependent
electric field.

The optical conductivity has two contributions: The intraband contribution
$\sigma_{F}(\omega)$ from electromagnetic field induced transitions on the
Fermi surface and the interband contribution $\sigma_{I}(\omega)$ from
transitions between occupied and unoccupied Bloch states with the same crystal
momentum $\bm{k}$. Here we focus on systems where there are ten electrons per
unit cell such that the $j=3/2$ multiplet is completely filled and the $j=1/2$
multiplet is half filled.

In the intra-orbital dominated coupling limit the system is a semimetal whose
Fermi surface contains only two crystal momenta, the $\bm{K}$ and
$\bm{K}^{\prime}$ points, in the Brilloiun zone. At these points in the band
structure the energy states of the $j=1/2$ multiplet become degenerate
resulting in two four-fold degenerate points whose low energy effective theory
resembles that of two half-filled Dirac cones. In the inter-orbital dominated
limit the system is insulating and the band gap minima move away from the
$\bm{K}$ and $\bm{K}^{\prime}$ points to the $\bm{\Gamma}$ and $\bm{M}$
points. In both cases the Fermi surface is of vanishing measure leading to
negligible intraband contributions to the conductivity at zero temperature.

Symmetry constrains the interband contribution to the conductivity
$\sigma(\omega)$ (see Appendix G). Fig. 3 shows the absorption, the real part
of $\sigma(\omega)$, in the extreme intra-orbital and inter-orbital dominated
limits for the chemical potential in the center of the $j=1/2$ multiplet. In
the intra-orbital dominated coupling limit the low frequency behavior of
$\Re{(\sigma(\omega))}$ is dominated by optical transitions between the lower
bands and the upper bands of the $j=1/2$ multiplet near Bloch momenta
surrounding the Dirac cones at the $\bm{K}$ and $\bm{K}^{\prime}$ points in
the Brilloiun zone. This feature saturates at $\hbar\omega\sim
3t_{1}=\Delta^{\text{intra}}_{1/2}$ where transitions between Bloch electrons
at the band minima and maxima occur. The absorption vanishes for
$\Delta_{1/2}^{\text{intra}}<\hbar\omega<\lambda_{0}-\Delta^{\text{intra}}_{3/2}/2$
due to the large onsite spin-orbit coupling that splits the $j=1/2$ and
$j=3/2$ multiplet by a large energy gap. For
$\hbar\omega>\lambda_{0}-\Delta^{\text{intra}}_{3/2}/2$ the conductivity
vanishes due to a fine tuned extra orbital symmetry inherited from the absence
of inter-orbital hopping elements in the extreme intra-orbital dominated
coupling limit. The extra symmetry constrains the interband matrix elements of
the velocity operator $\hat{\bm{v}}(\bm{k})$ to vanish between states in
different multiplet sectors, forcing the conductivity to vanish at high
frequencies.

In the inter-orbital dominated limit the low frequency absorption
$\Re{(\sigma(\omega))}$ occurs in a narrow frequency range where light can
excite particle-hole pairs within the flat bands of the $j=1/2$ multiplet.
While at higher frequencies
$\hbar\omega>\lambda_{0}-\Delta^{\text{inter}}_{3/2}/2$ the conductivity is
finite across a large range of frequencies arising from the joint density of
states and the interband matrix elements of the velocity operator between the
$j=1/2$ and $j=3/2$ multiplet.

Figure 3: (a) $\Re(\sigma(\omega))$ for system in the purely inter-orbital
(orange) $t_{2}=0.1\lambda_{0};t_{1}=t_{3}=0$ and purely intra-orbital
(purple) $t_{1},t_{3}=0.1\lambda_{0};t_{2}=0$ hopping limits, with the
chemical potential (shown in green in (b)) set in the middle of the $j=1/2$
multiplet (with $\hbar\omega$ in units of $\lambda_{0}$). Selection rules for
purely intra-orbital hopping limit, and (c) for purely inter-orbital hopping
limit. Allowed transitions are depicted by colored arrows whose contribution
to the conductivity are shown in (a).

Conclusion: We have demonstrated a new avenue for achieving flat bands with a
flatness parameter $\mathcal{F}\sim(t_{2}/\lambda)^{2}$ in materials with
strong onsite spin-orbit interactions $\lambda$ designed to have dominant
inter-orbital hopping $t_{2}$ compared to intra-orbital hopping. Flat band
generation from electronic interference pathways usually relies on large and
complicated unit cell structure [52, 53, 54, 55]. Here instead we exploit a
particular orbital structure on the honeycomb lattice whose large spin-orbit
interaction leads to frustration in the $j=1/2$ manifold of states.

Density functional theory calculations on 4d and 5d transition metal compounds
suggest that parameter regimes where $t_{2}$ dominates over $t_{1}$ are indeed
viable [56]. Materials based on 5d transition metal ions with larger $\lambda$
and smaller $U$ compared to the 4d transition metal ions should be a
particularly attractive design platform for flat band metals that we have
focused on here.

In general, spin-orbit assisted inter-orbital frustration provides a scheme
for achieving flat bands in materials with a wide variety of different multi-
orbital structures on lattices in any dimension. Gating to control the
chemical potential and strain to tune the hopping amplitudes can be used to
engineer the orbital frustration needed to generate flat bands. In addition to
quantum materials, cold atom systems have the ability to pattern inter-site
lattice interactions in the form of synthetic gauge potentials between multi-
orbital unit cells that provide a synthetic dimension thereby creating a
definitive platform for engineering orbitally frustrated systems with emergent
flat bands in systems with a variety of lattice and multiplet on-site
structures.

We expect these designer flatband materials to provide a link between
frustrated metals and frustrated magnetic insulators opening a doorway for the
search and discovery of topologically ordered states. While the emergence of
ferromagnetism in specific flatband systems has been explored [57, 58, 59, 54,
60, 61], the nature of magnetic interactions in the model described here
provides a richer platform.

Controlling orbital frustration on a lattice can lead to material design and
discovery of systems on the edge of an instability toward unique correlated
and topological many-body phases where the orbital structure of the ground
sate plays an important role in the electronic correlations of the system.
Rich phenomenology for the proposed flatband metals to show fractionalization
of the electronic spectral function, non-obeyance of the Luttinger volume
(Luttinger breaking), and quantized anomalous and topological Hall effects
need to be further explored.

Acknowledgments: W.Z. acknowledges support from the Center for Emergent
Materials: an NSF MRSEC under award number DMR-2011876; Z.A. and N.T.
acknowledge support from National Science Foundation Grant number
DMREF-1629382.

## References

  * Hasan and Kane [2010] M. Z. Hasan and C. L. Kane, Colloquium: topological insulators, Reviews of modern physics 82, 3045 (2010).
  * Kane and Mele [2005] C. L. Kane and E. J. Mele, Z 2 topological order and the quantum spin hall effect, Physical review letters 95, 146802 (2005).
  * Yan and Felser [2017] B. Yan and C. Felser, Topological materials: Weyl semimetals, Annual Review of Condensed Matter Physics 8, 337 (2017).
  * Lv  _et al._ [2015] B. Lv, H. Weng, B. Fu, X. P. Wang, H. Miao, J. Ma, P. Richard, X. Huang, L. Zhao, G. Chen, _et al._ , Experimental discovery of weyl semimetal taas, Physical Review X 5, 031013 (2015).
  * Qi and Zhang [2011] X.-L. Qi and S.-C. Zhang, Topological insulators and superconductors, Reviews of Modern Physics 83, 1057 (2011).
  * Sato and Ando [2017] M. Sato and Y. Ando, Topological superconductors: a review, Reports on Progress in Physics 80, 076501 (2017).
  * Chen  _et al._ [2013] X. Chen, Z.-C. Gu, Z.-X. Liu, and X.-G. Wen, Symmetry protected topological orders and the group cohomology of their symmetry group, Physical Review B 87, 155114 (2013).
  * Levin and Wen [2006] M. Levin and X.-G. Wen, Detecting topological order in a ground state wave function, Physical review letters 96, 110405 (2006).
  * Wen [1990] X.-G. Wen, Topological orders in rigid states, International Journal of Modern Physics B 4, 239 (1990).
  * Stormer  _et al._ [1999] H. L. Stormer, D. C. Tsui, and A. C. Gossard, The fractional quantum hall effect, Reviews of Modern Physics 71, S298 (1999).
  * Jain [1990] J. Jain, Theory of the fractional quantum hall effect, Physical Review B 41, 7653 (1990).
  * Haldane [1983] F. D. M. Haldane, Fractional quantization of the hall effect: a hierarchy of incompressible quantum fluid states, Physical Review Letters 51, 605 (1983).
  * Wen [1995] X.-G. Wen, Topological orders and edge excitations in fractional quantum hall states, Advances in Physics 44, 405 (1995).
  * Savary and Balents [2016] L. Savary and L. Balents, Quantum spin liquids: a review, Reports on Progress in Physics 80, 016502 (2016).
  * Zhou  _et al._ [2017] Y. Zhou, K. Kanoda, and T.-K. Ng, Quantum spin liquid states, Reviews of Modern Physics 89, 025003 (2017).
  * Kitaev [2006] A. Kitaev, Anyons in an exactly solved model and beyond, Annals of Physics 321, 2 (2006).
  * Ronquillo  _et al._ [2019] D. C. Ronquillo, A. Vengal, and N. Trivedi, Signatures of magnetic-field-driven quantum phase transitions in the entanglement entropy and spin dynamics of the kitaev honeycomb model, Physical Review B 99, 140413 (2019).
  * Patel and Trivedi [2019] N. D. Patel and N. Trivedi, Magnetic field-induced intermediate quantum spin liquid with a spinon fermi surface, Proceedings of the National Academy of Sciences 116, 12199 (2019), https://www.pnas.org/content/116/25/12199.full.pdf .
  * Doretto [2014] R. Doretto, Plaquette valence-bond solid in the square-lattice j 1-j 2 antiferromagnet heisenberg model: A bond operator approach, Physical Review B 89, 104415 (2014).
  * Sachdev and Bhatt [1990] S. Sachdev and R. Bhatt, Bond-operator representation of quantum spins: Mean-field theory of frustrated quantum heisenberg antiferromagnets, Physical Review B 41, 9323 (1990).
  * Singh  _et al._ [1999] R. R. Singh, Z. Weihong, C. Hamer, and J. Oitmaa, Dimer order with striped correlations in the j 1- j 2 heisenberg model, Physical Review B 60, 7278 (1999).
  * Kotov  _et al._ [1999] V. N. Kotov, J. Oitmaa, O. P. Sushkov, and Z. Weihong, Low-energy singlet and triplet excitations in the spin-liquid phase of the two-dimensional j 1- j 2 model, Physical Review B 60, 14613 (1999).
  * Zhitomirsky and Ueda [1996] M. Zhitomirsky and K. Ueda, Valence-bond crystal phase of a frustrated spin-1/2 square-lattice antiferromagnet, Physical Review B 54, 9007 (1996).
  * Ralko  _et al._ [2009] A. Ralko, M. Mambrini, and D. Poilblanc, Generalized quantum dimer model applied to the frustrated heisenberg model on the square lattice: Emergence of a mixed columnar-plaquette phase, Physical Review B 80, 184427 (2009).
  * Capriotti  _et al._ [2001] L. Capriotti, F. Becca, A. Parola, and S. Sorella, Resonating valence bond wave functions for strongly frustrated spin systems, Physical review letters 87, 097201 (2001).
  * Jiang  _et al._ [2012] H.-C. Jiang, H. Yao, and L. Balents, Spin liquid ground state of the spin-1 2 square j 1-j 2 heisenberg model, Physical Review B 86, 024424 (2012).
  * Li  _et al._ [2012] T. Li, F. Becca, W. Hu, and S. Sorella, Gapped spin-liquid phase in the j 1- j 2 heisenberg model by a bosonic resonating valence-bond ansatz, Physical Review B 86, 075111 (2012).
  * Nasu  _et al._ [2017] J. Nasu, J. Yoshitake, and Y. Motome, Thermal transport in the kitaev model, Physical review letters 119, 127204 (2017).
  * Pidatella  _et al._ [2019] A. Pidatella, A. Metavitsiadis, and W. Brenig, Heat transport in the anisotropic kitaev spin liquid, Physical Review B 99, 075141 (2019).
  * Laurell and Okamoto [2020] P. Laurell and S. Okamoto, Dynamical and thermal magnetic properties of the kitaev spin liquid candidate $\alpha$-rucl 3, npj Quantum Materials 5, 1 (2020).
  * Do  _et al._ [2017] S.-H. Do, S.-Y. Park, J. Yoshitake, J. Nasu, Y. Motome, Y. S. Kwon, D. Adroja, D. Voneshen, K. Kim, T.-H. Jang, _et al._ , Incarnation of majorana fermions in kitaev quantum spin lattice, arXiv preprint arXiv:1703.01081 (2017).
  * Bistritzer and MacDonald [2011] R. Bistritzer and A. H. MacDonald, Moiré bands in twisted double-layer graphene, Proceedings of the National Academy of Sciences 108, 12233 (2011).
  * Dos Santos  _et al._ [2012] J. L. Dos Santos, N. Peres, and A. C. Neto, Continuum model of the twisted graphene bilayer, Physical Review B 86, 155449 (2012).
  * Cea and Guinea [2020] T. Cea and F. Guinea, Band structure and insulating states driven by coulomb interaction in twisted bilayer graphene, Phys. Rev. B 102, 045107 (2020).
  * Nam and Koshino [2017] N. N. T. Nam and M. Koshino, Lattice relaxation and energy band modulation in twisted bilayer graphene, Phys. Rev. B 96, 075311 (2017).
  * Hofmann  _et al._ [2020] J. S. Hofmann, E. Berg, and D. Chowdhury, Superconductivity, pseudogap, and phase separation in topological flat bands, Phys. Rev. B 102, 201112 (2020).
  * Cao  _et al._ [2018a] Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Jarillo-Herrero, Unconventional superconductivity in magic-angle graphene superlattices, Nature 556, 43 (2018a).
  * Cao  _et al._ [2018b] Y. Cao, V. Fatemi, A. Demir, S. Fang, S. L. Tomarken, J. Y. Luo, J. D. Sanchez-Yamagishi, K. Watanabe, T. Taniguchi, E. Kaxiras, _et al._ , Correlated insulator behaviour at half-filling in magic-angle graphene superlattices, Nature 556, 80 (2018b).
  * Xu and Balents [2018] C. Xu and L. Balents, Topological superconductivity in twisted multilayer graphene, Physical review letters 121, 087001 (2018).
  * Ochi  _et al._ [2018] M. Ochi, M. Koshino, and K. Kuroki, Possible correlated insulating states in magic-angle twisted bilayer graphene under strongly competing interactions, Physical Review B 98, 081102 (2018).
  * Wu  _et al._ [2018] F. Wu, A. MacDonald, and I. Martin, Theory of phonon-mediated superconductivity in twisted bilayer graphene, Physical Review Letters 121, 257001 (2018).
  * Zhang  _et al._ [2019] Y.-H. Zhang, D. Mao, Y. Cao, P. Jarillo-Herrero, and T. Senthil, Nearly flat chern bands in moiré superlattices, Phys. Rev. B 99, 075127 (2019).
  * Pizarro  _et al._ [2019] J. M. Pizarro, M. J. Calderón, and E. Bascones, The nature of correlations in the insulating states of twisted bilayer graphene, Journal of Physics Communications 3, 035024 (2019).
  * Tarnopolsky  _et al._ [2019] G. Tarnopolsky, A. J. Kruchkov, and A. Vishwanath, Origin of magic angles in twisted bilayer graphene, Phys. Rev. Lett. 122, 106405 (2019).
  * San-Jose  _et al._ [2012] P. San-Jose, J. Gonzalez, and F. Guinea, Non-abelian gauge potentials in graphene bilayers, Physical review letters 108, 216802 (2012).
  * Liu  _et al._ [2019] J. Liu, J. Liu, and X. Dai, Pseudo landau level representation of twisted bilayer graphene: Band topology and implications on the correlated insulating phase, Physical Review B 99, 155415 (2019).
  * Goldman  _et al._ [2014] N. Goldman, G. Juzeliūnas, P. Öhberg, and I. B. Spielman, Light-induced gauge fields for ultracold atoms, Reports on Progress in Physics 77, 126401 (2014).
  * Lin and Spielman [2016] Y.-J. Lin and I. Spielman, Synthetic gauge potentials for ultracold neutral atoms, Journal of Physics B: Atomic, Molecular and Optical Physics 49, 183001 (2016).
  * Celi  _et al._ [2014] A. Celi, P. Massignan, J. Ruseckas, N. Goldman, I. B. Spielman, G. Juzeliūnas, and M. Lewenstein, Synthetic gauge fields in synthetic dimensions, Physical review letters 112, 043001 (2014).
  * Jiménez-García  _et al._ [2015] K. Jiménez-García, L. LeBlanc, R. Williams, M. Beeler, C. Qu, M. Gong, C. Zhang, and I. Spielman, Tunable spin-orbit coupling via strong driving in ultracold-atom systems, Physical review letters 114, 125301 (2015).
  * Cooper  _et al._ [2019] N. Cooper, J. Dalibard, and I. Spielman, Topological bands for ultracold atoms, Reviews of modern physics 91, 015005 (2019).
  * Ochi  _et al._ [2015] M. Ochi, R. Arita, M. Matsumoto, H. Kino, and T. Miyake, Robust flat bands in RCo5 ($R=$ rare earth) compounds, Physical Review B 91, 165137 (2015).
  * Arita  _et al._ [2002] R. Arita, Y. Suwa, K. Kuroki, and H. Aoki, Gate-induced band ferromagnetism in an organic polymer, Physical review letters 88, 127202 (2002).
  * Shima and Aoki [1993] N. Shima and H. Aoki, Electronic structure of super-honeycomb systems: A peculiar realization of semimetal/semiconductor classes and ferromagnetism, Physical review letters 71, 4389 (1993).
  * Liu  _et al._ [2013] Z. Liu, Z.-F. Wang, J.-W. Mei, Y.-S. Wu, and F. Liu, Flat chern band in a two-dimensional organometallic framework, Physical review letters 110, 106804 (2013).
  * Kim and Kee [2016] H.-S. Kim and H.-Y. Kee, Crystal structure and magnetism in $\alpha$\- rucl 3: An ab initio study, Physical Review B 93, 155143 (2016).
  * Mielke and Tasaki [1993] A. Mielke and H. Tasaki, Ferromagnetism in the hubbard model, Communications in mathematical physics 158, 341 (1993).
  * Tasaki [1998] H. Tasaki, From nagaoka’s ferromagnetism to flat-band ferromagnetism and beyond: An introduction to ferromagnetism in the hubbard model, Progress of theoretical physics 99, 489 (1998).
  * Lieb [1989] E. H. Lieb, Two theorems on the hubbard model, Physical review letters 62, 1201 (1989).
  * Tasaki [1992] H. Tasaki, Ferromagnetism in the hubbard models with degenerate single-electron ground states, Physical review letters 69, 1608 (1992).
  * Mielke [1991] A. Mielke, Ferromagnetic ground states for the hubbard model on line graphs, Journal of Physics A: Mathematical and General 24, L73 (1991).
  * Griffiths and Schroeter [2018] D. J. Griffiths and D. F. Schroeter,  _Introduction to quantum mechanics_ (Cambridge University Press, 2018).
  * Rau  _et al._ [2014] J. G. Rau, E. K.-H. Lee, and H.-Y. Kee, Generic spin model for the honeycomb iridates beyond the kitaev limit, Phys. Rev. Lett. 112, 077204 (2014).
  * Rau [2014] J. G. Rau,  _Unconventional Symmetry Breaking in Strongly Correlated Systems_ , Ph.D. thesis (2014).
  * Vanderbilt [2018] D. Vanderbilt,  _Berry Phases in Electronic Structure Theory: Electric Polarization, Orbital Magnetization and Topological Insulators_ (Cambridge University Press, 2018).
  * Onsager [1931] L. Onsager, Reciprocal relations in irreversible processes. i., Physical review 37, 405 (1931).
  * Morimoto and Nagaosa [2018] T. Morimoto and N. Nagaosa, Nonreciprocal current from electron interactions in noncentrosymmetric crystals: roles of time reversal symmetry and dissipation, Scientific reports 8, 1 (2018).
  * Utermohlen and Trivedi [2020] F. G. Utermohlen and N. Trivedi, Symmetry analysis of tensors in the honeycomb lattice of edge-sharing octahedra, arXiv preprint arXiv:2012.11604 (2020).
  * Sipe and Ghahramani [1993] J. Sipe and E. Ghahramani, Nonlinear optical response of semiconductors in the independent-particle approximation, Physical Review B 48, 11705 (1993).
  * Passos  _et al._ [2018] D. Passos, G. Ventura, J. V. P. Lopes, J. L. dos Santos, and N. Peres, Nonlinear optical responses of crystalline systems: Results from a velocity gauge analysis, Physical Review B 97, 235446 (2018).
  * Kronig [1926] R. d. L. Kronig, On the theory of dispersion of x-rays, Josa 12, 547 (1926).
  * Kramers [1927] H. A. Kramers, La diffusion de la lumiere par les atomes, in  _Atti Cong. Intern. Fisica (Transactions of Volta Centenary Congress) Como_ , Vol. 2 (1927) pp. 545–557.

##  Appendix A Model Geometry

In the main text we demonstrate spin-orbit assisted orbital frustration and
the emergence of flat bands in a model containing d-orbital hopping on the
honeycomb lattice. The honeycomb lattice can be described by a primitive
triangular lattice with two sites $A$ and $B$ per unit cell. Each $A$ and $B$
site has three nearest neighboring sites. We orient the honeycomb such that
the primitive lattice vectors can be chosen to be
$\bm{R}_{1}=a\sqrt{3}/2\hat{\bm{x}}+a/2\hat{\bm{y}}$ and
$\bm{R}_{2}=a\sqrt{3}/2\hat{\bm{x}}-a/2\hat{\bm{y}}$, where $a$ is the lattice
constant. We denote the three vectors connecting an $A$ site to its nearest
neighboring $B$ sites as
$\bm{v}_{x}=-a/2\sqrt{3}\hat{\bm{x}}+a/2\hat{\bm{y}}$,
$\bm{v}_{y}=-a/2\sqrt{3}\hat{\bm{x}}-a/2\hat{\bm{y}}$, and
$\bm{v}_{z}=a/\sqrt{3}\hat{\bm{x}}$. These vectors can be related to the bond
axes

$\displaystyle\hat{\bm{\gamma}}_{x}$
$\displaystyle=-\dfrac{1}{\sqrt{6}}\hat{\bm{x}}+\dfrac{1}{\sqrt{2}}\hat{\bm{y}}+\dfrac{1}{\sqrt{3}}\hat{\bm{z}}$
$\displaystyle\hat{\bm{\gamma}}_{y}$
$\displaystyle=-\dfrac{1}{\sqrt{6}}\hat{\bm{x}}-\dfrac{1}{\sqrt{2}}\hat{\bm{y}}+\dfrac{1}{\sqrt{3}}\hat{\bm{z}}$
$\displaystyle\hat{\bm{\gamma}}_{z}$
$\displaystyle=\sqrt{\dfrac{2}{3}}\hat{\bm{x}}+\dfrac{1}{\sqrt{3}}\hat{\bm{z}}$
(6)

whose projection onto the plane of the honeycomb is in the direction of
nearest neighbor bond vectors $\bm{v}_{s}$ (see Fig. 2 (d)).

The tight binding model described in equation 1 can be described by the sum of
two terms: a kinetic part describing the overlap between nearest neighboring
d-orbitals on the honeycomb lattice and a spin-orbit part describing the
onsite interaction between the spin and orbital degrees of freedom of the
$t_{2g}$ sector. It is easiest to write these coupling in a maximally
symmetric coordinate system whereby the d-orbitals
$\ket{d_{\alpha}(\bm{r}_{i})}$ and spin are defined with respect to the
orthogonal orbital unit vectors

$\displaystyle\hat{\bm{e}}_{x}$
$\displaystyle=-\dfrac{1}{\sqrt{2}}\hat{\bm{x}}-\dfrac{1}{\sqrt{6}}\hat{\bm{y}}+\dfrac{1}{\sqrt{3}}\hat{\bm{z}}$
$\displaystyle\hat{\bm{e}}_{y}$
$\displaystyle=\dfrac{1}{\sqrt{2}}\hat{\bm{x}}-\dfrac{1}{\sqrt{6}}\hat{\bm{y}}+\dfrac{1}{\sqrt{3}}\hat{\bm{z}}$
$\displaystyle\hat{\bm{e}}_{z}$
$\displaystyle=\sqrt{\dfrac{2}{3}}\hat{\bm{y}}+\dfrac{1}{\sqrt{3}}\hat{\bm{z}}$
(7)

that are related to the bond axes by a $\pi/2$ counterclockwise rotation (see
Fig. 2 (d)).

In this basis of orbitals
$\\{\ket{d_{yz}(\bm{r}_{i})},\ket{d_{zx}(\bm{r}_{i})},\ket{d_{xy}(\bm{r}_{i})}\\}$
and along these coordinate axes the kinetic hopping $\hat{t}$ can be written
in terms of the matrices in equation 2, while the orbital angular momentum
operators in the onsite spin-orbit interaction $\hat{\lambda}$ can be written
as

$\displaystyle L_{\hat{\bm{e}}_{x}}=\hbar\left(\begin{array}[]{ccc}0&0&0\\\
0&0&-i\\\ 0&i&0\end{array}\right)$ (11) $\displaystyle
L_{\hat{\bm{e}}_{y}}=\hbar\left(\begin{array}[]{ccc}0&0&i\\\ 0&0&0\\\
-i&0&0\end{array}\right)$ (15) $\displaystyle
L_{\hat{\bm{e}}_{z}}=\hbar\left(\begin{array}[]{ccc}0&-i&0\\\ i&0&0\\\
0&0&0\end{array}\right)$ (19)

##  Appendix B Electronic Bloch Bands

The Hamiltonian in Eq. 1 gives rise to flat bands in the $j=1/2$ multiplet in
the purely inter-orbital coupling limit ($t_{1}=t_{3}=0$). This can be
verified by investigating the eigenvalues of the Bloch Hamiltonian
$\hat{H}(\bm{k})$. Our tight binding model has a discrete translation symmetry
due to the lattice symmetry of the honeycomb such that its energy eigenstates
are Bloch modes $\Psi_{n}(\bm{k})$ with energy eigenvalues
$\varepsilon_{n}(\bm{k})$ both of which can be labeled by their band index $n$
and crystal momentum $\bm{k}$ taking values in the Brillouin zone. The Bloch
Hamiltonian
$\hat{H}(\bm{k})=e^{-i\hat{\bm{r}}\cdot\bm{k}}\hat{H}e^{i\hat{\bm{r}}\cdot\bm{k}}$
is a $12\times 12$ matrix we write as

$\hat{H}(\bm{k})=\sum_{\mu,\alpha}t_{\mu\alpha}(\bm{k})(\Gamma_{\mu}\otimes\mathcal{D}_{\alpha}\otimes\mathds{1})+\sum_{i}\lambda_{0}(\mathds{1}\otimes
L_{\hat{\bm{e}}_{i}}\otimes S_{\hat{\bm{e}}_{i}})$ (20)

Where $\Gamma_{\mu}$ are the two dimensional Pauli matrices describing the
$A/B$ sublattice degrees of freedom, $\mathds{1}$ is the two dimensional
identity matrix, and $t_{\mu\alpha}(\bm{k})$ the three dimensional matrices
$\mathcal{D}_{\alpha}$ that satisfy

$\displaystyle\sum_{\alpha}t_{x\alpha}(\bm{k})\mathcal{D}_{\alpha}$
$\displaystyle=\Re\bigg{(}\sum_{i}e^{i\bm{k}\cdot\bm{v}_{i}}T_{i}\bigg{)}$
$\displaystyle\sum_{\alpha}t_{y\alpha}(\bm{k})\mathcal{D}_{\alpha}$
$\displaystyle=\Im\bigg{(}\sum_{i}e^{i\bm{k}\cdot\bm{v}_{i}}T_{i}\bigg{)}$
$\displaystyle t_{z\alpha}(\bm{k})$ $\displaystyle=0$ (21)

where $\bm{v}_{i}$ are the three vectors defined above that connect an $A$
site to its neighboring $B$ sites.

We now investigate the band structure of our model in the purely intra-orbital
($t_{2}=0$, $t_{1}=t_{3}\neq 0$) and inter-orbital ($t_{2}\neq 0$,
$t_{1}=t_{3}=0$) kinetic limits. For general orbital hopping our model
satisfies time reversal symmetry ($\mathcal{T}=i\sigma_{\hat{\bm{e}}_{y}}K$)
and inversion symmetry ($\mathcal{I}=\Gamma_{x}$) endowed by the honeycomb
lattice structure
($[\hat{\mathcal{T}},\hat{H}]=[\hat{\mathcal{I}},\hat{H}]=0$). These
symmetries result in a Kramers degeneracy throughout the Brilloiun zone such
that bands must be at least two-fold degenerate for all crystal momenta
$\bm{k}$. In the absence of $\hat{t}$, the bands separate into two manifolds:
a manifold with total angular momentum $j=1/2$ containing four bands, and a
manifold with total angular momentum $j=3/2$ containing eight bands. This
multiplet structure ($4+8$) remains for $\lambda\gg t$ though $j$ is no longer
a good quantum number.

Fig. 2 (a) and (c) shows the band structure along high symmetry directions in
the Brilloiun zone for hopping parameters in the extreme intra-orbital and
inter-orbital kinetic limits. In the intra-orbital dominated limit the absence
of inter-orbital hopping ($t_{2}=0$) results in an additional inter-orbital
symmetry such that each band in the $j=3/2$ sector is four-fold degenerate
throughout the Brilloiun zone, while the $j=1/2$ sector contains two two-fold
degenerate bands protected by time reversal and inversion symmetry (see Fig. 2
(c)). Band touching points occur at the $\bm{K}=(2\pi/\sqrt{3}a,2\pi/3a)$ and
$\bm{K}^{\prime}=(2\pi/\sqrt{3}a,-2\pi/3a)$ points in the Brilloiun zone. This
results in an eight-fold degenerate double Dirac point for the $j=3/2$ sector
and 4-fold degenerate Dirac points for the $j=1/2$ sector. Note that neither
of these topological degeneracies are protected by crystaline symmetries and
can be converted to manifolds containing just two-fold degenerate bands
through the introduction of other symmetry allowed inter-orbital interactions
and are simply an artifact of the finely tuned nature of the purely intra-
orbital coupling limit.

In the limit of exterme inter-orbital kinetics, intra-orbital interactions are
turned off ($t_{1}=t_{3}=0$) and the orbital interactions are dominated by the
inter-orbital coupling $t_{2}$. The $j=3/2$ multiplet is split into four two-
fold degenerate bands, while the $j=1/2$ multiplet consists of two two-fold
degenerate bands with almost no dispersion (see Fig. 2 (a)). The intraband
kinetic energy is quenched due to an inter-orbital-spin-orbit frustration
caused by the large onsite spin-orbit interaction $\lambda_{0}$ and nonzero
$t_{2}$ that leads to the emergence of flat bands (see Emergent Flat Bands in
the main text).

##  Appendix C Perturbation Theory Details

In the absence of $\hat{t}$ the Hamiltonian in equation 1 consists of a band
structure of flat band multiplets of total angular momentum $j=1/2$ and
$j=3/2$. Here we calculate the correction to these flat bands in orders of the
kinetic couplings $t_{1}$, $t_{2}$, and $t_{3}$ and show the emergent band
dispersion in the $j=1/2$ and $j=3/2$ multiplet in the purely inter-orbital
hopping ($t_{1}=t_{3}=0$) and purely intra-orbital hopping ($t_{2}=0$) limits.
We find that the first order correction to the degenerate $j=1/2$ multiplet in
the purely inter-orbital hopping limit vanishes leading to small band widths
of order $(t_{2})^{2}/\lambda$.

To calculate the energy splitting of the degenerate $j=1/2$ and $j=3/2$
multiplets to linear order in $\hat{t}$ we use degenerate perturbation theory
and first calculate the components of the matrix

$W^{j}_{\alpha\beta}(\bm{k})=\bra{\psi^{j}_{\alpha}}\sum_{\mu,\alpha}t_{\mu\alpha}(\bm{k})(\Gamma_{\mu}\otimes\mathcal{D}_{\alpha}\otimes\mathds{1})\ket{\psi^{j}_{\beta}}$
(22)

where $\ket{\psi^{j}_{\alpha}}$ are the eigenstates of $\hat{\lambda}$ with
total angular momentum $j$. The eigenvalues of the matrix
$W^{j}_{\alpha\beta}(\bm{k})$ are the first order corrections to the
eigenvalues of $\hat{\lambda}$ [62]. For the $j=3/2$ manifold the eigenvalues
of $W^{3/2}_{\alpha\beta}(\bm{k})$ are nonzero and complicated functions of
$\bm{k}$.

Intra-orbital Dominated Interactions | Eigenvalues $[t_{1}]$
---|---
$\bm{\Gamma}$ | $9/2$ | $3/2$ | $-3/2$ | $-9/2$
$\bm{K}/\bm{K}^{\prime}$ | 0 | 0 | 0 | 0
$\bm{M}$ | $3/2$ | $1/2$ | $-1/2$ | $-3/2$
Inter-orbital Dominated Interactions | Eigenvalues $[t_{2}]$
$\bm{\Gamma}$ | $\sqrt{1+\sqrt{7/16}}$ | $\sqrt{1-\sqrt{7/16}}$ | $-\sqrt{1-\sqrt{7/16}}$ | $-\sqrt{1+\sqrt{7/16}}$
$\bm{K}/\bm{K}^{\prime}$ | $\sqrt{2-\sqrt{15/16}}$ | 0 | 0 | $-\sqrt{2-\sqrt{15/16}}$
$\bm{M}$ | $\sqrt{1+\sqrt{7/16}}$ | $\sqrt{1-\sqrt{7/16}}$ | $-\sqrt{1-\sqrt{7/16}}$ | $-\sqrt{1+\sqrt{7/16}}$
Table 1: Table showing eigenvalues of $W^{3/2}_{\alpha\beta}(\bm{k})$ at high
symmetry points in the Brillouin zone in the intra-orbital and inter-orbital
dominated kinetic limits. Eigenvalues are given in units of the nonzero
kinetic coupling $t$. Each eigenvalue is two fold degenerate due to time
reversal and inversion symmetry.

Table 1 lists eigenvalues of $W^{3/2}_{\alpha\beta}(\bm{k})$ at some high
symmetry momenta in the Brilloiun zone for the intra-orbital and inter-orbital
dominated kinetic limits. Each eigenvalue is doubly degenerate because of
Kramers theorem and the presence of time reversal and inversion symmetries. We
also note the vanishing of the first order correction to the band energies at
the $\bm{K}$ and $\bm{K}^{\prime}$ momenta in the intra-orbital dominated
coupling limit. This is reflected in the full band structure by the presence
of the eight-fold degenerate double Dirac point contained in the $j=3/2$
manifold as plotted in Fig. 2 (c). In order to maintain this multifold
degeneracy higher order corrections to the band dispersion a these points must
vanish. Lastly we note that the first order correction to the band energies
predicts a bandwidth $\Delta_{3/2}$ of the $j=3/2$ manifold that is of order
the hopping strength ($\Delta\sim t$) for both the intra-orbital and inter-
orbital dominated kinetic limits.

This is in striking contrast to the $j=1/2$ manifold of states where
$W^{1/2}_{\alpha\beta}(\bm{k})$ is nonzero in the intra-orbital dominated
coupling limit, but vanishes in the inter-orbital dominated limit. For general
kinetic hopping $t$ its eigenvalues are

$\displaystyle\delta
E^{(1)}_{\pm}(\bm{k})=\pm\dfrac{2t_{1}+t_{3}}{3}\sqrt{3+4\cos\bigg{(}\dfrac{\sqrt{3}}{2}k_{x}a\bigg{)}\cos\bigg{(}\dfrac{1}{2}k_{y}a\bigg{)}+2\cos\bigg{(}k_{y}a\bigg{)}}$
(23)

where each eigenvalue $\delta E_{\pm}(\bm{k})$ is again two fold degenerate
because of Kramers theorem. In both the intra-orbital and inter-orbital
dominated coupling limit $\delta E_{\pm}(\bm{K})=\delta
E_{\pm}(\bm{K}^{\prime})=0$. In the intra-orbital dominated limit ($t_{2}=0$)
this results in the presence of Dirac cones at the $\bm{K}$ and
$\bm{K}^{\prime}$ points in the Brillouin zone (see Fig. 2 (c)) as further
higher order corrections to the band structure also vanish at these high
symmetry points. Remarkably in the inter-orbital dominated kinetic limit,
$t_{1}=t_{3}=0$, $W^{1/2}_{\alpha\beta}=0$. This is due to the orbital
character of the $j=1/2$ manifold and the orbital character of the purely
inter-orbital hopping matrices. The action of these matrices on the $j=1/2$
manifold transforms a $j=1/2$ state into a mixture of $j=3/2$ states. These
states are all orthogonal to the $j=1/2$ manifold as they have different
eigenvalues with respect to $\hat{\lambda}$ forcing
$W^{1/2}_{\alpha\beta}(\bm{k})$ to vanish. This demonstrates that the first
order correction to the energy eigenvalues vanish in the $j=1/2$ manifold and
for the extreme inter-orbital dominated limit.

To calculate any nonzero dispersion in this limit and in the $j=1/2$ manifold
we must proceed to second order degenerate perturbation theory. However, the
calculation is simplified in the inter-orbital dominated limit due to
$W^{1/2}_{\alpha\beta}(\bm{k})=0$. The second order correction to the $j=1/2$
manifold of bands is given by

$\delta
E^{(2)}_{\alpha}(\bm{k})=\sum_{\beta}\dfrac{\bra{\psi_{\alpha}^{1/2}}\hat{t}(\bm{k})\ket{\psi_{\beta}^{3/2}}\bra{\psi_{\beta}^{3/2}}\hat{t}(\bm{k})\ket{\psi_{\alpha}^{1/2}}}{E^{(0)}_{1/2}-E^{(0)}_{3/2}}$
(24)

where $E^{(0)}_{1/2}$ and $E^{(0)}_{3/2}$ are eigenvalues of $\hat{\lambda}$
for the $j=1/2$ and $j=3/2$ manifold and where

$\hat{t}(\bm{k})=e^{-i\bm{k}\cdot\hat{\bm{r}}}\hat{t}e^{i\bm{k}\cdot\hat{\bm{r}}}=\sum_{\mu,\alpha}t_{\mu\alpha}(\bm{k})(\Gamma_{\mu}\otimes\mathcal{D}_{\alpha}\otimes\mathds{1})$
(25)

The difference $E^{(0)}_{1/2}-E^{(0)}_{3/2}=3/2\lambda_{0}$ is of order the
spin-orbit coupling $\lambda_{0}$, while the numerator will be of order
$t_{2}$ squared. We see that this leads to a bandwidth in the $j=1/2$ manifold
and in the inter-orbital dominated limit of
$\Delta^{\text{inter}}_{1/2}\sim(t_{2})^{2}/\lambda_{0}$ whereas the
bandwidths in the purely intra-orbital hopping limit
$\Delta^{\text{intra}}_{1/2}$ and the bandwidth of the $j=3/2$ manifold in
either limit are all of order $t$.

Thus the ingredients that lead to the supression of band dispersion and
emergent flat bands in the $j=1/2$ manifold in the inter-orbital dominated
limit are the vanishing of the first order perturbative correction to the band
energies deriving from the orbital character of the $j=1/2$ manifold of states
and the orbital character of a purely inter-orbital coupling between sites,
and the presence of a large spin-orbit interaction for which the $j=1/2$ and
$j=3/2$ manifolds are separated by a large energy gap and for which
$t_{2}/\lambda_{0}$ is small. This ratio sets the scale for the ratio between
the expected bandwidth $\Delta\sim t$ and the actual bandwidth of the $j=1/2$
multiplet in the inter-orbital dominated limit
$\Delta^{\text{inter}}_{1/2}\sim(t_{2})^{2}/\lambda_{0}$.

##  Appendix D Strong Coupling Limit:

Upon adding a strong onsite multiorbital Hubbard interaction to Eq. 1 of the
form

$\hat{H}_{U}=\sum_{i}\bigg{(}\dfrac{U-3J_{H}}{2}(\hat{n}_{i}-1)^{2}-2J_{H}\hat{\bm{S}}_{i}\cdot\hat{\bm{S}}_{i}-\dfrac{J_{H}}{2}\hat{\bm{L}}_{i}\cdot\hat{\bm{L}}_{i}\bigg{)}$
(26)

where $\hat{n}(\bm{r}_{i})$, $\hat{\bm{S}}(\bm{r}_{i})$, and
$\hat{\bm{L}}(\bm{r}_{i})$ are the density, spin, and orbital angular momentum
operator at site $\bm{r}_{i}$, in the limit of strong spin-orbit coupling
$\lambda\gg t$ and strong Hubbard interactions an expansion in powers of $t$
leads to an effective $\mathcal{J}-K-\Gamma$ Hamiltonian for a system where
half of the $j=1/2$ multiplet is filled, given by

$\displaystyle\hat{H}_{eff}$
$\displaystyle=\sum_{<i,j>}\bigg{(}\mathcal{J}\hat{\bm{J}}(\bm{r}_{i})\cdot\hat{\bm{J}}(\bm{r}_{j})+K\hat{J}_{\hat{\bm{e}}_{\gamma}}(\bm{r}_{i})\hat{J}_{\hat{\bm{e}}_{\gamma}}(\bm{r_{j}})$
$\displaystyle+\sum_{\alpha\beta}\Gamma(\hat{J}_{\hat{\bm{e}}_{\alpha}}(\bm{r}_{i})\hat{J}_{\hat{\bm{e}}_{\beta}}(\bm{r}_{j})+\hat{J}_{\hat{\bm{e}}_{\beta}}(\bm{r}_{i})\hat{J}_{\hat{\bm{e}}_{\alpha}}(\bm{r}_{j}))\bigg{)}.$
(27)

Here $\gamma\rightarrow\gamma(\bm{r}_{i},\bm{r}_{j})$ is an implicit function
of the lattice sites, $\bm{r}_{i}$ and $\bm{r}_{j}$, and $\hat{\bm{J}_{i}}$ is
the total angular momentum operator projected onto the $j=1/2$ state space.

$\hat{\bm{J}}=\sum_{m=\pm
1/2}\ket{1/2,m_{j}}\bra{1/2,m_{j}}(\hat{\bm{S}}+\hat{\bm{L}})\ket{1/2,m_{j}}\bra{1/2,m_{j}}$
(28)

The coupling strengths $\mathcal{J}$, $K$, and $\Gamma$ are given by [63, 64]

$\displaystyle\mathcal{J}$
$\displaystyle=\dfrac{4}{27}\bigg{(}\dfrac{6t_{1}(t_{1}+2t_{3})}{U-3J_{H}}+\dfrac{2(t_{1}-t_{3})^{2}}{U-J_{H}}+\dfrac{(2t_{1}+t_{3})^{2}}{U+2J_{H}}\bigg{)}$
$\displaystyle K$
$\displaystyle=\dfrac{8J_{H}}{9}\bigg{(}\dfrac{(t_{1}-t_{3})^{2}-3t_{2}^{2}}{(U-3J_{H})(U-J_{H})}\bigg{)}$
$\displaystyle\Gamma$
$\displaystyle=\dfrac{16J_{H}}{9}\bigg{(}\dfrac{t_{2}(t_{1}-t_{3})}{(U-3J_{H})(U-J_{H})}\bigg{)}\
\cdot$ (29)

We see that in the limit of purely intra-orbital hopping ($t_{2}=0$) and
$t_{1}=t_{3}$ but non-zero, the effective Hamiltonian is that of a Heisenberg
magnetic ($K=\Gamma=0$), while in the limit of purely inter-orbital hopping
($t_{2}\neq 0$) and $t_{1}=t_{3}=0$ the effective Hamiltonian is purely Kitaev
type with $\mathcal{J}=\Gamma=0$. In this model the inter-orbital coupling
$t_{2}$ acts as an effective spin-orbit interaction where the orbital degrees
of freedom take the role of an effective spin. In the limit $t_{1}=t_{3}=0$
this effective interaction gives rise to a pseudo-spin frustration that leads
to the effective Kitaev type interaction in Eq. 27.

Figure 4: Berry curvature distribution (in units of $a^{2}$) for [Panels (A)]
time reversal broken Hamiltonian $(\mu_{B}|\mathbf{B}_{0}|=0.05\lambda_{0})$
and [Panels (B)] for inversion broken Hamiltonian $(|m_{0}|=0.05\lambda_{0})$:
(1) Berry curvature of  $10_{\text{th}}$ band; (2) band structure of
$9_{\text{th}}$ to $12_{\text{th}}$ bands in purely inter-orbital hopping
limit $(t_{2}=0.1\lambda_{0},t_{1}=t_{3}=0)$; (3) Berry curvature of
$10_{\text{th}}$ band; (4) band structure of $9_{\text{th}}$ to
$12_{\text{th}}$ bands for purely intra-orbital limit
$(t_{1}=t_{3}=0.1\lambda_{0},t_{2}=0)$. The regions outlined in green in panel
(B) figures (1) and (3) show the fractionalization of the Berry curvature
distribution around the $\bm{K}$ and $\bm{K}^{\prime}$ points in the Brillouin
zone as one interpolates between the purely intra-orbital and purely inter-
orbital hopping limits. For full band structures in the presence of both
inversion and time reversal symmetry breaking and the Berry curvature
distribution and Chern numbers of all 12 bands in the presence of time
reversal symmetry breaking and in the purely inter-orbital hopping limit see
Fig. 6 and Fig. 7 in Appendix F.

##  Appendix E Berry Curvature for Degenerate Bands

In order to understand the quantum geometry of the bands that make up the
$j=1/2$ multiplet we calculate the momentum space Berry curvature distribution
for systems with broken time-reversal and inversion symmetries in both the
purely inter-orbital hopping ($t_{1}=t_{3}=0)$ and purely intra-orbital
hopping limits ($t_{2}=0$). In the limit of purely intra-orbital hopping
($t_{2}=0$) and in the presence of an inversion symmetry breaking mass the
$j=1/2$ multiplet contains two two-fold degenerate bands (see Fig. 4 (B)(4)).
The single band Berry curvature is ill defined in the presence of degenerate
bands and thus for this system we plot the Berry phase density for a state
with chemical potential filling half the $j=1/2$ multiplet. We first partition
the Brillouin zone into small plaquettes and compute the Berry flux through
each plaquette

$\phi(\bm{k})=-\Im(\text{Ln}(\det(\bm{\mathcal{U}})))$ (30)

where $\bm{\mathcal{U}}_{nm}=\braket{u_{n}(\bm{k}_{N})}{u_{m}(\bm{k}_{0})}$
with $n$ and $m$ taking values over all bands up to the half filled $j=1/2$
multiplet, and where the state $\ket{u_{n}(\bm{k}_{N})}$ is calculated by
parallel transport of the state $\ket{u_{n}(\bm{k}_{0})}$ from $\bm{k_{0}}$
around a plaquette centered at $\bm{k}$ [65]. The Berry curvature is then
found by dividing $\phi(\bm{k})$ by the area of the plaquette.

##  Appendix F Hall Conductivity in the Presence of Time-Reversal Symmetry
Breaking

The quantum geometry of a band structure is determined by a band’s Berry
curvature, $\bm{\Omega}_{n}(\bm{k})$, whose integral across the 2D Brillouin
zone dictates the winding of the phase of the Bloch wavefunction: an integer
topological invariant called the Chern number. In the presence of a time-
reversal breaking perturbation to the Hamiltonian in equation 1 both the Berry
curvature and Chern number can be nonzero. Fig. 7 shows the momentum space
Berry curvature distributions and Chern numbers for all 12 bands in the
presence of time reversal symmetry breaking and in the purely inter-orbital
coupling limit ($t_{2}\neq 0$,$t_{1}=t_{3}=0$).

##  Appendix G Optical Conductivity

To understand the optical response of our model to external frequency
dependent electric field perturbations we calculate the optical conductivity
tensor. The conductivity tensor $\sigma_{ij}(t,t^{\prime})\in\mathcal{R}$ such
that $\sigma_{ij}(-\omega)=\sigma_{ij}^{*}(\omega)$. Furthermore time reversal
symmetry constrains the antisymmetric part of the conductivity to vanish
$\sigma_{ij}(\omega)=\sigma_{ji}(\omega)$ [66, 67]. Crystalline three-fold
rotational symmetry about a lattice point constrains the diagonal parts of the
conductivity to be equal
($\sigma(\omega)=\sigma_{xx}(\omega)=\sigma_{yy}(\omega)$) while inplane two-
fold rotational symmetry constrains
$\sigma_{xy}(\omega)=\sigma_{yx}(\omega)=0$ [68]. As described in the main
text the intra-band conductivity vanishes at $T=0$ for systems filling half of
the $j=1/2$ multiplet, and thus we focus on calculating the real symmetric
part of the interband optical conductivity,
$\Re{(\sigma_{ij}(\omega))}=\Re(\sigma_{I}^{ij}(\omega)+\sigma_{I}^{ji}(\omega))/2$,
that couples to real optical transitions between Bloch states whose energy
differs by $\omega$ [69, 70].

The real symmetric part of the interband conductivity can be written as

$\displaystyle\Re(\sigma_{ij}(\omega))$
$\displaystyle=\dfrac{e^{2}\pi\hbar}{V}\sum_{nm\bm{k}}\dfrac{f^{T}_{n}(\bm{k},\mu)-f^{T}_{m}(\bm{k},\mu)}{\varepsilon_{n}(\bm{k})-\varepsilon_{m}(\bm{k})}$
$\displaystyle\times\Re(v^{i}_{nm}(\bm{k})v^{j}_{mn}(\bm{k}))\delta(\varepsilon_{n}(\bm{k})-\varepsilon_{m}(\bm{k})+\hbar\omega)$
(31)

Here $V$ is the volume of the system, $e$ is the electric charge,
$f^{T}_{n}(\bm{k},\mu)$ are the Fermi occupation functions at temperature $T$
and chemical potential $\mu$, and $v_{nm}^{i}(\bm{k})$ are the interband
matrix elements of the velocity operator in the $\hat{\bm{i}}$-direction.

$v_{nm}^{i}(\bm{k})=\dfrac{1}{\hbar}\bra{u_{n}(\bm{k})}\dfrac{\partial\hat{H}(\bm{k})}{\partial
k_{i}}\ket{u_{m}(\bm{k})}$ (32)

Where we have chosen the phase on the periodic part of the Bloch functions
$\ket{u_{n}(\bm{k})}$ such that the full Bloch functions
$\ket{\Psi_{n}(\bm{k})}=e^{i\bm{k}\cdot\hat{\bm{r}}}\ket{u_{n}(\bm{k})}$ are
periodic in the Brilloiun zone
$\ket{\Psi_{n}(\bm{k})}=\ket{\Psi_{n}(\bm{k}+\bm{G})}$.

Figure 5: Optical conductivity: (a) real part and (b) imaginary part obtained
via Kramers-Kronig relations for purely inter-orbital (orange)
$t_{2}=0.1\lambda_{0};t_{1}=t_{3}=0$ and purely intra-orbital (purple)
$t_{1},t_{3}=0.1\lambda_{0};t_{2}=0$ hopping limits.

Fig. 3 shows the real part of the optical conductivity in the purely inter-
orbital and purely intra-orbital coupling limits. To calculate the imaginary
part of $\sigma(\omega)$ we use the Kramers-Kronig relations for complex
functions analytic in the upper-half plane of $\omega$:
$\Im(\sigma(\omega))=-\dfrac{1}{\pi}\mathcal{P}\int_{-\infty}^{\infty}\dfrac{\Re(\sigma(\omega^{\prime}))}{\omega^{\prime}-\omega}$
[71, 72]. Fig. 5(b) shows the imaginary part of $\sigma(\omega)$ in the two
limits of interest for systems with chemical potential where half of the
$j=1/2$ multiplet is filled.

Figure 6: Band structure for model with (A) time reversal symmetry breaking
($\mu_{B}|\mathbf{B}_{0}|=0.05\lambda_{0}$) and (B) inversion symmetry
breaking ($|m_{0}|=0.05\lambda_{0}$). Figure 7: Momentum space Berry curvature
distributions and Chern numbers for all 12 bands in the presence of a time
reversal breaking perturbation ($\mu_{B}|\mathbf{B}_{0}|=0.05\lambda_{0}$) and
in the purely inter-orbital hoping limit ($t_{1}=t_{3}=0$).


# Performance and Energy Balance: A Comprehensive Study of State-of-the-Art
Sound Event Detection Systems

###### Abstract

In recent years, deep learning systems have shown a concerning trend toward
increased complexity and higher energy consumption. As researchers in this
domain and organizers of one of the Detection and Classification of Acoustic
Scenes and Events challenges task, we recognize the importance of addressing
the environmental impact of data-driven SED systems. In this paper, we propose
an analysis focused on SED systems based on the challenge submissions. This
includes a comparison across the past two years and a detailed analysis of
this year’s SED systems. Through this research, we aim to explore how the SED
systems are evolving every year in relation to their energy efficiency
implications 111The data related to the submissions used for the analysis are
available at https://github.com/RonFrancesca/SED_carbon_footprint.

Index Terms—  sound event detection, machine listening, energy consumption,
carbon footprint

##  1 Introduction

Deep learning has yielded incredible achievements in a variety of audio
processing applications, including Speech Recognition [1], Machine Listening
[2, 3], and Music Generation [4, 5]. Despite the outcome of the remarkable
results, the computational overhead of deep learning remains significant and
continues to increase [6]. One particular concern is the substantial energy
usage and resulting carbon footprint tied to the computational needs of deep
learning. This situation is rapidly becoming unsustainable from both technical
and environmental perspectives [7, 8, 9]. Until very recently, the
environmental impact of deep learning models has largely been dominated by the
persistent demand for high accuracy and effectiveness [10]. In the last years,
there has been a rise of concern about the environmental impact and energy
consumption of deep learning within audio signal processing communities [11,
10, 12, 13, 14].

However, comparing accurately the energy of different models, possibly trained
on different sites is not straightforward [12]. There is no real consensus on
the metric to be used and the relation between the different potential metrics
(complexity, MACS, energy consumption) is often unclear. In the last years,
different open-source Python packages have been introduced by the community in
order to face this issue [15, 16, 17, 18]. Anyway, none of them provide a
complete overview of the environmental impact. In fact, several factors must
be taken into account when quantifying the energy consumption of deep learning
algorithms [12, 10].

|  System complexity $\downarrow$ |  Energy train (kWh) $\downarrow$ |  Energy test (kWh) $\downarrow$
---|---|---|---
|  25% |  Median |  75% |  25% |  Median |  75% |  25% |  Median |  75%
2022 Entries |  2200000 |  6676303 |  18903660 |  1.815 |  3.699 |  17.291 |  0.010 |  0.026 |  0.046
2023 Entries |  4804956 |  14662273 |  97176570 |  1.615 |  4.295 |  13.975 |  0.019 |  0.035 |  0.283
Table 1: General comparison between DCASE 2022 and DCASE 2023 submissions
entries. The table presents the median, 25th percentile, and 75th percentile
values for system complexity, training energy, and test energy.

Starting from 2018, the objective of DCASE task 4 has been to investigate SED
using a heterogeneous dataset featuring audio soundscapes with varying levels
of label detail[19]. Throughout the years, alongside the ranking procedure,
the evaluation of diverse submissions in practical scenarios has been crucial
to gaining insights into the systems’ performance [20, 21, 22, 23]. In DCASE
task 4, as in many other audio tasks, there has been a consistent trend of
models steadily increasing in parameter complexity, frequently incorporating
ensemble techniques. As also reported in Serizel et. al [12], as organizers of
the DCASE task 4, we recognized a responsibility to raise awareness about the
carbon emissions and environmental implications associated with data-driven
SED systems.

In 2022, we asked participants to report the energy consumption of their
systems both at training and test time [20]. We also introduced a new energy
consumption metric [24], based on CodeCarbon toolkit [17]. The metric has been
introduced as an optional metric due to possible biases in terms of hardware
used and fairness of comparison between systems. Since 2023, energy
consumption reporting is mandatory. We also asked every participant to report
energy consumption for 10 epoch of training of the baseline on their setup.
This aims at normalizing the energy consumption metric by accounting for
possible hardware disparities [12]. We further asked participants to report an
additional hardware-agnostic metric involving the computation of Multiply-
Accumulate operations (MACs) for ten seconds of audio prediction. We employed
THOP: PyTorch-OpCounter as a framework for MACs computation [18].

This paper presents an analysis of the general evolution trend between 2022
and 2023. Following this initial overview, we focus on SED systems submitted
in 2023. The metrics gathered in 2023 include hardware normalization and allow
for a fairer comparison. The goal of the analysis is to provide insights for
achieving a better balance between performance and energy efficiency in SED
system development.

##  2 Analysis setup and evaluation metrics

The analysis is conducted on DCASE task 4 submissions in 2022 and 2023. For
more information on system submissions, the reader is invited to visit the
DCASE Challenge website 222https://dcase.community/challenge2023/task-sound-
event-detection-with-weak-labels-and-synthetic-soundscapes. Within DCASE task
4, systems performance is evaluated with the polyphonic sound event detection
scores (PSDS) [25]. PSDS allows users to define parameter sets, defining
customized scenarios under which SED systems are evaluated. For DCASE task 4,
two scenarios are considered, as described in Ronchini et. al [20]. The PSDS
is indicated throughout the paper as PSDS_1 when evaluated on scenario 1 and
PSDS_2 when evaluated on scenario 2, regardless of the system.

From 2022, we propose a tentative, trivial energy weighted polyphonic sound
detection score (EW-PSDS):

$\mathrm{{EW-
PSDS}}=\mathrm{PSDS}*\frac{\mathrm{kWh}_{\mathrm{baseline}}}{\mathrm{kWh}_{\mathrm{submission}}}$
(1)

where PSDS is the polyphonic sound event detection scores [25],
$\mathrm{kWh}_{\mathrm{baseline}}$ is the energy consumption reported for the
baseline, and $\mathrm{kWh}_{\mathrm{submission}}$ is the energy consumption
of the submitted system [24].

The energy usage during both the training and test stages is reported using
kWh (kilowatt-hour) as the unit.

This study presents an analysis of the relation between system performance
metrics and energy consumption-related measures 333Additional results are
available at https://github.com/RonFrancesca/SED_carbon_footprint. We report a
subset of these metrics, along with MACs, system complexity (number of
parameters of the deep learning model), and energy consumption. The energy
usage is measured during both the training and inference stages.

Fig. 1: Relation between system complexity and energy consumption at training
for 2023 entries, compared with the two baselines systems. Fig. 2: Relation
between system complexity and energy consumption at test for 2023 entries,
compared with the two baselines systems.

##  3 General comparison between DCASE 2022 and DCASE 2023 systems

This section analyzes all entries from 2022 and 2023 to understand how energy-
related metrics have evolved over the past two years. In this study, we
compare median, 25th percentile, and 75th percentile, for system complexity,
energy consumption during training, and energy consumption during test.
Quartiles and the median were chosen as measures of central tendency instead
of the mean and standard deviation due to the presence of significant data
variability. We conducted a filtering process in order to remove duplicate
entries and entries with erroneous reported metrics. Initially, we had 101
total entries for 2022 and 123 total entries for 2023. After filtering, the
analysis focused on 60 entries for 2022 and 64 entries for 2023. Table 1 shows
the results for system complexity, training energy consumption, and test
energy for both 2022 and 2023 entries. Unsurprisingly, it can be observed a
similar trend as the one discussed in Section 1 for deep learning models.
There is an inclination towards increased system complexity and energy
consumption during both training and test. While this is true, it is
interesting to note that the energy consumption at training of the 75th
percentile has remained stable or even decreased.

Fig. 3: PSDS_1 and energy consumption at training for best 2023 systems,
compared with the two baselines systems. Fig. 4: PSDS_1 and energy consumption
at test for best performance 2023 systems, compared with the two baselines
systems.

However, it’s important to note that the energy consumption data for this
general analysis is not normalized, which means we can hardly draw objective
conclusions due to potential hardware biases. Additionally, not all 2022
submissions provided energy consumption values. This initial study is only
presented to observe general trends. To ensure fairness and a more insightful
study, the rest of the analysis is exclusively focused on results related to
2023 submissions.

##  4 Relation between system complexity, MACs, and energy consumption

Fig. 5: Relation between MACs and energy consumption at training for 2023
entries, compared with the two baselines systems. Fig. 6: Relation between
MACs and energy consumption at test for 2023 entries, compared with the two
baselines systems.

In this section, we present an analysis of the MACs and system complexity in
relation to energy consumption during the training and test phases of the
DCASE 2023 submissions. To ensure a fair comparison among systems, energy
consumption has been normalized by the baseline energy consumption [12].

Figure 1 illustrates the correlation between system complexity and energy
consumption during the training stage, Figure 2 presents the corresponding
relation for energy consumption during the test stage. Figure 5 and Figure 6
show the relation between MACs and energy consumed at the training phase, and
the energy consumed at the test phase, respectively. On all the plots, we also
report the performance of the two baselines proposed for the challenge.
Baseline indicates the simple baseline which does not use external dataset or
embeddings, while Baseline (A+B) stands for the baseline using Audioset as
external dataset [26] and BEATs embeddings [27] 444See also the task webpage 2
for more details..

From the results, it is possible to observe that MACs correlate slightly more
with energy at training than system complexity. It is surprising that MACs
correlate better with training energy than test energy, while the other way
around would be expected as MACs are computed at test time. This difference
might be related to different system architectures but this would have to be
verified in extensive experiments.

The relation between system complexity with the energy consumed by the system
at both training and test phases is not straightforward. These three different
metrics independently are insufficient to provide a comprehensive
understanding of the system’s footprint. In fact, as an example, from Figure
1, there are systems with a complexity of 5M parameters that consume more
energy than systems with a complexity of 100M parameters. Similar observations
apply to MACs counts and energy consumption. However, for simplicity sake, we
will focus the analysis mainly on energy consumption for the rest of the
paper.

| System complexity | MACs | Energy train (kWh) | System complexity | MACs | Energy train (kWh)
---|---|---|---|---|---|---
| Max | PSDS_1 | Max | PSDS_1 | Max | PSDS_1 | Max | PSDS_1 | Max | PSDS_1 | Max | PSDS_1
All | 1B | 0.59 | 492 B | 0.59 | 23.00 | 0.59 | 1B | 0.62 | 21 T | 0.62 | 190.00 | 0.62
25th | 5 M | 0.55 | 912 M | 0.55 | 0.99 | 0.55 | 25 M | 0.61 | 8 B | 0.58 | 4.59 | 0.60
Median | 6 M | 0.59 | 4 B | 0.55 | 2.33 | 0.56 | 67 M | 0.61 | 72 B | 0.60 | 9.34 | 0.60
Table 2: The table presents PSDS_1 when system complexity MACS and training
energy are thresholded to the median value or the 25th percentile. The left
side is related to no-ensemble systems, the right is related to ensemble
systems. Fig. 7: Relation between PSDS_1 and energy consumption at training
for the best ensemble systems. Fig. 8: Relation between PSDS_1 and energy
consumption at training for the best not-ensemble systems.

##  5 Relation between performance and energy consumption

This section presents findings regarding the relation between performance and
energy consumption. We focus on the top 15 systems in terms of PSDS_1
performance along with training and test energy consumption. If the same team
has more than one submission with values within a range of 1-2 points in terms
of PSDS_1, we selected only one submission per team, the best in terms of
PSDS_1. We removed duplicate entries for each team based on MACs, keeping the
best-performing system for each team. 555A similar analysis for PSDS_2 is
reported on the additional results.

Fig. 9: Relation between PSDS_1 and energy consumption weighted at training
for 2023 entries, compared with the two baselines systems.

Figure 3 and Figure 4 illustrate the performance metrics PSDS_1 in relation to
training energy consumption, and test energy consumption, respectively. The
figures highlight the diversity in terms of system efficiency. Particularly
for PSDS_1, some systems manage to outperform the baseline results while
consuming less energy. Additionally, should be noted that the top-performing
systems are not the systems that consume the most energy. The same observation
holds true for energy consumption during test. In fact, from the results, it
appears that energy consumed for training and test have a similar
distribution. In the remainder of the paper, we will focus on energy at
training. We know that energy at test is the most important in terms of the
footprint of deployed systems yet it also depends on the hardware target and
is difficult to tackle in the current challenge setup. Additionally, according
to previous observations energy at training can be considered as a first
(gross) indicator of what would happen at the test (even though many other
factors are involved).

##  6 Comparison between ensemble/non-ensemble systems

This section analyzes the relation between PSDS_1 and energy consumption for
ensemble and not-ensemble systems. We still focus on the 15 best systems in
terms of PSDS_1.

Figure 7 and Figure 8 show the relation between PSDS_1 and energy consumption
at training for the best ensemble-based submissions and for the top not-
ensembled-based entries, respectively. What can emerge from this analysis is
that an ensemble is useful at combining systems that alone are not so good in
achieving decent performance (this is not so efficient in terms of energy but
these not-so-good systems are already expensive) while a single system can
provide a lighter alternative to reach good performance anyway. In fact, the
best not-ensemble system is able to achieve a PSDS_1 score of 0.58, while the
best ensemble system scores 0.62 for PSDS_1.

##  7 Relation between EW-PSDS and PSDS

Figure 9 shows the relation between PSDS_1 and EW-PSDS. In particular, we can
image the plot divided into four areas: bottom left corner, showing the system
that were outperformed by the baseline and still had potentially higher
consumption; bottom right corner: systems that outperform the baseline but
consume more energy; top right: systems that outperform the baseline with
limited energy consumption increase. The area to which we should aim is the
top-right corner, which includes a system able to right high performance, not
underestimating the environmental impact they are going to have.
Unfortunately, as the figure shows, they are still minorities. This Figure
highlights how most of the systems required a higher quantity of energy at
training compared to the baseline. Some systems of the top right and the
bottom right have pretty close PSDS_1 while having pretty different energy
consumption. The systems able to outperform the baseline necessitating less
energy at training are three different submissions of the team Chen_CHT [28].

##  8 Thresholding based on energy consumption

The last part of the analysis evaluates how much the performances degrade if a
footprint cap is set. In order to do so, we define a threshold related to the
system complexity, MACs, and energy consumption. For each threshold, we select
only the systems that have a lower value of the threshold. For example, when
considering the median of the energy consumption at training, we selected the
systems with a lower energy consumption than the median consumption and
reported the best PSDS_1 score. We applied a similar approach for MACs and
system complexity. Due to space limitations, the same analysis for the 75th
percentile is reported in the additional results, considering additional
metrics 3. Table 2 reports the degradation of the PSDS_1 for the systems that
have been thresholded according to the different metrics. The left side
reports the results for non-ensemble systems, while the right reports the
results for ensemble systems. The PSDS performance remains rather stable
regardless of the threshold cap while the complexity, MACs and energy
consumption are substantially decreased. This is even clearer where
considering ensembling. This indicates that we are spending a large amount of
energy and computation to increase the performance only marginally.

##  9 Conclusions

This paper presents a comprehensive examination of energy usage and its
correlation with various metrics applied to systems participating in the DCASE
task 4 during the years 2022 and 2023. The findings make it evident that
relying on a single metric is insufficient for accurately measuring a system’s
footprint. The paper also highlights that systems consuming the most energy
(or having the most MACs) do not necessarily outperform less computationally
expressive systems. These observations highlight the pressing need for
metric(s) capable of taking into account various factors to accurately
estimate the energy consumption of deep learning models while taking into
account the task-wise performance of the systems. This would be the first
important step to effectively design sustainable SED systems.

## References

  * [1] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever,  “Robust speech recognition via large-scale weak supervision,”  in International Conference on Machine Learning. PMLR, 2023.
  * [2] Qing Wang, Jun Du, Hua-Xin Wu, Jia Pan, Feng Ma, and Chin-Hui Lee,  “A four-stage data augmentation approach to resnet-conformer based acoustic modeling for sound event localization and detection,”  IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
  * [3] Ji Won Kim, Sang Won Son, Yoonah Song, Hong Kook Kim, Il Hoon Song, and Jeong Eun Lim,  “Semi-supervsied learning-based sound event detection using freuqency dynamic convolution with large kernel attention for dcase challenge 2023 task 4,”  arXiv preprint arXiv:2306.06461, 2023.
  * [4] Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al.,  “Musiclm: Generating music from text,”  arXiv preprint arXiv:2301.11325, 2023.
  * [5] Antoine Caillon and Philippe Esling,  “Rave: A variational autoencoder for fast and high-quality neural audio synthesis,”  arXiv preprint arXiv:2111.05011, 2021.
  * [6] Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn, and Pablo Villalobos,  “Compute trends across three eras of machine learning,”  in 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022.
  * [7] Neil C Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F Manso,  “The computational limits of deep learning,”  arXiv preprint arXiv:2007.05558, 2020.
  * [8] Udit Gupta, Young Geun Kim, Sylvia Lee, Jordan Tse, Hsien-Hsin S Lee, Gu-Yeon Wei, David Brooks, and Carole-Jean Wu,  “Chasing carbon: The elusive environmental footprint of computing,”  in 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2021.
  * [9] Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni,  “Green ai,”  Communications of the ACM, 2020.
  * [10] Constance Douwes, Giovanni Bindi, Antoine Caillon, Philippe Esling, and Jean-Pierre Briot,  “Is quality enoughf integrating energy consumption in a large-scale evaluation of neural audio synthesis models,”  in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023.
  * [11] Nesrine Bannour, Sahar Ghannay, Aurélie Névéol, and Anne-Laure Ligozat,  “Evaluating the carbon footprint of nlp methods: a survey and analysis of existing tools,”  in Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing, 2021.
  * [12] Romain Serizel, Samuele Cornell, and Nicolas Turpault,  “Performance above all? energy consumption vs. performance, a study on sound event detection with heterogeneous data,”  in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023.
  * [13] Gianmarco Cerutti, Rahul Prasad, Alessio Brutti, Elisabetta Farella, et al.,  “Neural network distillation on iot platforms for sound event detection.,”  in Interspeech, 2019.
  * [14] Titouan Parcollet and Mirco Ravanelli,  “The energy and carbon footprint of training end-to-end speech recognizers,”  2021\.
  * [15] Lasse F Wolff Anthony, Benjamin Kanding, and Raghavendra Selvan,  “Carbontracker: Tracking and predicting the carbon footprint of training deep learning models,”  arXiv preprint arXiv:2007.03051, 2020.
  * [16] Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He,  “Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters,”  in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
  * [17] Victor Schmidt, Kamal Goyal, Aditya Joshi, Boris Feld, Liam Conell, Nikolas Laskaris, Doug Blank, Jonathan Wilson, Sorelle Friedler, and Sasha Luccioni,  “Codecarbon: estimate and track carbon emissions from machine learning computing,”  Cited on, 2021.
  * [18] Ligeng Zhu,  “Thop: Pytorch-opcounter,”  Lyken17/pytorch-OpCounter, 2019.
  * [19] Nicolas Turpault and Romain Serizel,  “Training sound event detection on a heterogeneous dataset,”  arXiv preprint arXiv:2007.03931, 2020.
  * [20] Francesca Ronchini and Romain Serizel,  “A benchmark of state-of-the-art sound event detection systems evaluated on synthetic soundscapes,”  in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022.
  * [21] Nicolas Turpault, Romain Serizel, Ankit Parag Shah, and Justin Salamon,  “Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,”  in Workshop on Detection and Classification of Acoustic Scenes and Events, 2019.
  * [22] Romain Serizel and Nicolas Turpault,  “Sound event detection from partially annotated data: Trends and challenges,”  in IcETRAN conference, 2019.
  * [23] Janek Ebbers, Reinhold Haeb-Umbach, and Romain Serizel,  “Post-processing independent evaluation of sound event detection systems,”  arXiv preprint arXiv:2306.15440, 2023.
  * [24] Francesca Ronchini, Samuele Cornell, Romain Serizel, Nicolas Turpault, Eduardo Fonseca, and Daniel PW Ellis,  “Description and analysis of novelties introduced in dcase task 4 2022 on the baseline system,”  arXiv preprint arXiv:2210.07856, 2022.
  * [25] Çağdaş Bilen, Giacomo Ferroni, and Francesco et al. Tuveri,  “A framework for the robust evaluation of sound event detection,”  in Proc. of ICASSP, 2020.
  * [26] Shawn Hershey, Daniel PW Ellis, Eduardo Fonseca, Aren Jansen, Caroline Liu, R Channing Moore, and Manoj Plakal,  “The benefit of temporally-strong labels in audio event classification,”  in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 366–370.
  * [27] Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, and Furu Wei,  “Beats: Audio pre-training with acoustic tokenizers,”  arXiv preprint arXiv:2212.09058, 2022.
  * [28] Wei-Yu Chen, Chung-Li Lu, Hsiang-Feng Chuang, Yu-Han Cheng Cheng, and Bo-Cheng Chan,  “Sound event detection system using pre-trained model for dcase 2023 task 4,”  Tech. Rep., DCASE2023 Challenge, June 2023.