# Hubbard subbands and superconductivity in the infinite-layer nickelate

Tharathep Plienbumrung  Maria Daghofer  Institute for Functional Matter and
Quantum Technologies, University of Stuttgart, Pfaffenwaldring 57, D-70550
Stuttgart, Germany Center for Integrated Quantum Science and Technology,
University of Stuttgart, Pfaffenwaldring 57, D-70550 Stuttgart, Germany
Michael T. Schmid  Waseda Research Institute for Science and Engineering,
Waseda University, Okubo, Shinjuku, Tokyo, 169-8555, Japan Andrzej M. Oleś
Corresponding author<EMAIL_ADDRESS>Max Planck Institute for Solid State
Research, Heisenbergstrasse 1, D-70569 Stuttgart, Germany Institute of
Theoretical Physics, Jagiellonian University, Profesora Stanisława
Łojasiewicza 11, PL-30348 Kraków, Poland

###### Abstract

An effective two-dimensional two-band model for infinite-layer nickelates
consists of bands obtained from $d_{x^{2}-y^{2}}$ and $s$–like orbitals. We
investigate whether it could be mapped onto a single-band Hubbard model and
the filling of Hubbard bands. We find that both one-band physics and a Kondo-
lattice regime emerge from the same two-orbital model, depending on the
strength of electronic correlations and the filling of the itinerant $s$-band.
Next we investigate one-particle excitations by changing the screening. First,
for weak screening the strong correlations push electrons out of the $s$-band
so that the undoped nickelate is similar to a cuprate. Second, for strong
screening the $s$ and $d_{x^{2}-y^{2}}$ bands are both partly filled and
weakly coupled. Particularly in this latter regime mapping to a one-band model
gives significant spectral weight transfer between the Hubbard subbands.
Finally we show how the symmetry of superconducting phases depends on the
interaction parameters and determine the regions of $d$-wave or $s$-wave
symmetry.

##  I Introduction

A few years ago, superconductivity was reported in infinite-layer NdNiO2 thin
films with Sr doping [1]. The lattice structure shares similarities with
cuprate superconductors, with NiO2 planes taking the place of CuO2 planes.
While both can be expected to be quite correlated and both show
antiferromagnetic (AFM) superexchange [2, 3, 4, 5, 6], there are some
microscopic differences. One is the lack of apical oxygens in the Ni case,
which affects crystal-field energies, the other is the presence of dispersive
rare-earth states close to the Fermi level. Whether one starts from isolated
NiO2 layers [2, 3, 4, 5, 6, 7, 8] or from band-structure calculations [9, 10,
11, 12, 13, 14, 15, 16], one expects that more than one orbital or band might
be relevant.

While single-band [17] and three-band [11, 18] models have also been proposed,
two bands cross the Fermi level, see Fig. 1, and many groups have accordingly
investigated two-band models [19, 20, 21, 22, 23, 18]. One of the bands has a
large contribution from the $x^{2}-y^{2}$ orbital at Ni and its dispersion is
nearly perfectly two-dimensional (2D). This band can be expected to share
features with the Cu-dominated band of the cuprates to be rather correlated.
In the second band, rare-earth states hybridize with Ni apical states, thus
obtaining some Ni($d_{3z^{2}-r^{2}}$) and Ni($d_{xy}$) character, however, its
wave function has $s$-symmetry [20], and we denote it accordingly. Previous
studies of various two-band models have yielded a large variety of potential
pairing symmetries [7, 24], among them $s$, $d$, and $s_{\pm}$-wave states
[11, 8, 18], while a one-band scenario favors $d$-wave [17].

Figure 1: Non-interacting band structure of NdNiO2: DFT bands crossing the
Fermi surface (black dashed lines) and the 2D tight-binding model obtained by
projecting a Wannier fit onto the plane along 2D path (green solid lines). The
Fermi energy $E=\mu$ corresponds to the DFT electronic structure.

The $s$-like band lies mostly above the Fermi level, however, it forms
electron pockets around the $\Gamma$ and $A$ points in the Brillouin zone. In
the DFT band structure, the pockets account for $\approx 7\,\%$ of the
occupied states [25]. Even without Sr-doping, these electrons are thus missing
from the $x^{2}-y^{2}$ band. When translating to a cuprate scenario, it should
be noted that $5\,\%$ of Sr doping suffices to destroy antiferromagnetism in
La2CuO4-y [26]. We thus have to expect partly filled bands, and at least one
of them is correlated.

The purpose of this paper is to investigate the evolution of the bands shown
in Fig. 1 with crystal-field splittings and electron correlations in both
bands. Thereby we investigate to what extent multi-band effects come into play
in nickelates. It is important to realize that partial filling of the strongly
correlated $x^{2}-y^{2}$ orbitals means that the electronic spectral weight
may be transferred from the upper Hubbard band (UHB) to the lower Hubbard band
(LHB) above the Fermi energy. The mechanism of such a weight transfer is well
known for the doped Hubbard model [27, 28]. Another mechanism which promotes
such a weight transfer is interaction screening that generates finite filling
within the weakly correlated orbitals of $s$-wave symmetry. It is remarkable
that both for very weak and for strong screening the bands mostly decouple and
the effective physics becomes similar to a single Hubbard band [17]. We then
find a Mott insulator (doped band with potential $d$-wave pairing) for strong
(weak) correlations. Correlation strength thus emerges as an important factor
in the description of nickelate superconductors.

The remaining of this paper is organized as follows. The two-band model arises
from the electronic structure calculations as described in Sec. II.1.
Electronic interactions are given by two Kanamori parameters for an atom
$\alpha$: $\\{U_{\alpha},J_{H}\\}$, and we discuss their screening in Sec.
II.2. Below we investigate a very wide range of potential regimes going from
weak to very strong correlations in Sec. III.1. There we show how the density
of states changes with screening for strong and weak interactions in Sec.
III.2. Finally, we present the essential AFM and superconducting (SC) phases
for various regimes of screening in Sec. III.3. The paper is concluded in Sec.
IV.

##  II Two-band model and Methods

###  II.1 Kinetic energy

We start from the kinetic energy in the electronic structure. The DFT band
structure, see Fig. 1, is calculated with Quantum Espresso code [29, 30, 31]
using a plane-wave pseudopotential method [32]. As discussed in Ref. [33],
many models can be constructed that differ in the shape of the apical $s$-like
orbital. Since their hopping integrals, given in [33], are nevertheless very
similar, the kinetic energy is not affected by this ambiguity in any
physically relevant way.

The Wannier90 interface [34] gives the parametrization,

$\displaystyle H_{\rm kin}$
$\displaystyle=\sum_{i,\lambda=d,s;\sigma}\epsilon_{\lambda}d^{\dagger}_{i\lambda\sigma}d^{\phantom{\dagger}}_{i\lambda\sigma}+\sum_{ij,\\{\lambda\mu\\},\sigma}{t_{ij}^{\lambda\mu}d^{\dagger}_{i\lambda\sigma}d_{j\mu\sigma}},$
(1)

where $d^{\phantom{\dagger}}_{i\lambda\sigma}$
($d^{\dagger}_{i\lambda\sigma}$) annihilates (creates) an electron at site $i$
in orbital $\lambda=d,s$, with spin $\sigma$. ($d$ and $s$ refer to the two
bands discussed above and shown in Fig. 1.) Hopping parameters
$t^{\lambda\mu}_{ij}$ and on-site energies $\epsilon_{\lambda}$ are given in
Ref. [33]. We project these three-dimensional bands onto the $(x,y)$-plane, as
we are here mostly interested in the correlated $x^{2}-y^{2}$ states, whose
band is already quite 2D to start with [33].

###  II.2 Interactions and screening effect

While the rather extended wave-function of (especially) the $s$-like state
might lead to longer-ranged Coulomb interactions, on-site terms can be
expected to dominate and we use (intraorbital and interorbital) Coulomb
elements of the form [3],

$\displaystyle\\!H_{\rm int}$
$\displaystyle=\sum_{i,\lambda=d,s}U_{\lambda}n_{i\lambda\uparrow}n_{i\lambda\downarrow}+\left(U^{\prime}-\frac{J_{H}}{2}\right)\sum_{i}{n_{id}n_{is}}$
(2)
$\displaystyle-2J_{H}\sum_{i}\vec{S}_{id}\\!\cdot\\!\vec{S}_{is}+\\!J_{H}\sum_{i}\\!\left(d^{\dagger}_{id\uparrow}d^{\dagger}_{id\downarrow}d^{\phantom{\dagger}}_{is\downarrow}d^{\phantom{\dagger}}_{is\uparrow}+{\rm
H.c.}\right)\\!.$

$n_{i\lambda\sigma}$ is the electron number operator at site $i$, in orbital
$\lambda$ and for spin $\sigma$ and $\\{\vec{S}_{i\lambda}\\}$ the
corresponding spin operator. Intraorbital Coulomb repulsion $U_{\lambda}$
depends on the band index $\lambda=d,s$. Hund’s exchange is given by $J_{H}$
and interorbital repulsion $U^{\prime}$ couples the bands.

Upper limits for the ’bare’ $U_{d}$ and $J_{0}$ are given by their atomic
values $U_{d}\approx 8$ eV and $J_{0}\approx 1.2$ eV, as one might use in
modelling an insulating NiO2 layer [2, 3]. However, when projecting out oxygen
states and using Wannier functions instead, effective values have to be
significantly reduced. In the case of $U_{s}$, atomic values for Ni cannot be
even taken as a starting point, as the $s$-orbital is mostly made up of rare-
earth states and is not centered on a Ni site [20]. The strong Nd($5d$)
character and very itinerant character of the $s$-bands suggests that their
effective interaction should be strongly screened, in fact more that can be
expected for the $d$ states. One thus expects $U_{d}>U_{s}$, which we take
into account in a phenomenological way via a screening parameter
$\alpha\in[0,1]$, so that

$\displaystyle U_{s}=\alpha U_{d},\quad J_{H}=\alpha J_{0},\quad
U^{\prime}=U_{s}-2J_{H}.$ (3)

This parametrization provides the simplest approach to discuss the interplay
of a more and a less correlated band.

We then use Lanczos exact diagonalization to treat the full Hamiltonian
$H=H_{\rm kin}+H_{\rm int}$ on an eight-site square cluster standing for a
NiO2 plane. Orbital densities are analyzed following Ref. [33] for the two
orbitals, $d$ and $s$. Below we summarize the evolution of the density of
states in the correlated $d$ band and the accompanying $s$ band. We
particularly focus on the circumstances favoring the electron transfer between
the Hubbard subbands in the correlated band.

##  III Numerical results

###  III.1 Hubbard subbands

To understand the occurrence of possible SC phase in infinite-layer nickelates
we consider first the one-particle spectra in the normal phase. Figure 2 shows
the orbital-resolved density of states, taking two values of the Coulomb
interaction $U_{d}=8.0$ and $4.0$ eV. The larger value is the same as
interactions in cuprates [35] and could be considered to be the upper limit;
the lower value stands for effective Coulomb interactions in the metallic
state in nickelates where Coulomb interactions are weaker. Here we begin with
unscreened interactions in the $s$ band, i.e., we take $\alpha=1.0$.

Figure 2:  Density of states $D(\omega)$ of undoped nickelate with unscreened
interactions ($\alpha=1$). Fermi energy is set to zero; the orbital densities
are normalized to one (per spin). Intraorbital Coulomb interaction in Eq. (2)
is selected at: (a) $U_{d}=8$ eV and (b) $U_{d}=4$ eV.

In both cases of large $U_{d}=8.0$ eV and moderate $U_{d}=4.0$ eV, one finds a
Mott insulator with two subbands separated by a gap, the occupied LHB and the
empty UHB. For large $U_{d}=8.0$ eV, the gap is $\eqsim 3.5$ eV and the Fermi
energy falls within the gap, see Fig. 2(a). This may be considered a textbook
example of a Mott insulator. Then one finds also an AFM order in the LHB.

When $U_{d}=4$ eV, the gap in the correlated band decreases to less than
$\eqsim 1.0$ eV and the tail of the $s$ band falls below the Fermi energy
which still separates the occupied and unoccupied states of the LHB, see Fig.
2(b). However, we should keep in mind that the calculations are done for a
finite system and we cannot exclude a metallic phase in the thermodynamic
limit. In any case, one finds a small fraction of electrons occupying the $s$
states and these states are just below the Fermi energy, see Fig. 2(b). Here
the correlated LHB band is less than half-filled and develops dynamics. As a
result, electron transfer from the UHB to the unoccupied part of the LHB
increases, and the total occupancy of the LHB exceeds 0.5. We conclude that
the presence of the second more itinerant band is responsible for the electron
transfer between the Hubbard subbands.

The next question to ask is where doped holes go in the quarter-filled system.
Before we have shown [36] that three regimes emerge for increasing screening
as discussed below. The reduction of the Coulomb interaction to $U_{d}=4.0$ eV
is sufficient to cause the loss of long-range AFM order in the $x^{2}-y^{2}$
orbital due to reduced electron filling.

###  III.2 One-particle spectral density

First, in the weakly screened Mott insulator and for large $U_{d}$, the ground
state is AFM and holes naturally enter only the $x^{2}-y^{2}$ orbital. In
contrast, the second regime is found at intermediate screening [$\alpha\simeq
0.5$], or for interactions that are reduced from the outset, see Fig. 2(b).
Finally, in the third regime of strong screening ($\alpha\sim 0.2$), hole
doping occurs again into the $x^{2}-y^{2}$ orbital, with the $s$ electrons
remaining unaffected [36]. This behaviour is presented in more detail below.

Figure 3:  Density of states $D(\omega)$ of undoped nickelate for $U_{d}=8.0$
eV and for different values of screening $\alpha$.  Figure 4:  Density of
states $D(\omega)$ of undoped nickelate for $U_{d}=4.0$ eV and for different
values of screening $\alpha$.

The different behaviour in the three regimes mentioned in Sec. III.1 is also
reflected in the single-particle spectra shown in Fig. 3. Filling corresponds
to doping with two holes and twisted boundary conditions (TBC) are used to
resolve more momenta [37, 38, 39]. Both for very strong $U_{d}=8.0$ eV, see
Fig. 3(a) and for moderate $U_{d}=4.0$ eV, see Fig. 4(a), the correlations
induce a gap in the $x^{2}-y^{2}$ band [40]. The lowest (occupied) states for
electrons are in the $s$ band at reduced $U_{d}=4$ eV, i.e., both bands are
partly filled. This can be seen in Fig. 2(b), where we show the density of
states for eight electrons (i.e., at quarter filling). Data were obtained by
means of TBC, integrating over five sets of boundary conditions.

At strong screening when $\alpha=0.2$, the occupied states in the
$x^{2}-y^{2}$ band are rather similar for $U_{d}=8.0$ eV and $U_{d}=4.0$ eV,
except that the curvature of the occupied states changes along the
$(\pi,0)-(0,\pi)$ line. Also the values of $n_{s}$ and of the weight
transferred to the LHB are similar. Since this implies that interactions
$U^{\prime}$ and $J_{H}$ between $d$ and $s$ states do not play here a
significant role, it supports the notion of a correlated (and doped) $d$ band
that is only affected by a metallic $s$ band via self-doping.

In contrast, for stronger correlations, i.e., weaker screening
$\alpha\in[0.5,0.7]$, the spectra shown in Figs. 3(b) and 3(c) are affected by
$U_{d}$. All electrons are here in the correlated $x^{2}-y^{2}$ states. It is
remarkable that the occupied states fall almost at the same energies,
independently of whether $U_{d}=8.0$ eV or $U_{d}=4.0$ eV [cf. Figs. 3 and 4].
However, splitting between $d$ and $s$ states is clearly affected by $U_{d}$
(via $U^{\prime}$ and $J_{H}$), which indicates that the $s$\- and $d$-bands
are in this regime directly coupled, not only via self-doping.

Analogous conclusions can be drawn from the undoped density of states shown in
Figs. 3(a) and 4(a) are extremely similar: In the regime of strong screening,
both bands are partly filled and results hardly depend on $U_{d}$ at all. In
the intermediate regime on the other hand, both bands are likewise partially
filled. A comparison between Figs. 3 and 4 indicates that the $s$ states being
close to the Fermi level could be doped away. In this regime, results depend
on $U_{d}$, indicating that correlations are here more important to describe
low-energy features close to the Fermi level.

Table 1:  Electron densities $n_{d}$ and $n_{s}$ per spin obtained in the undoped nickelate for screened interactions $(\alpha<1)$. The weight of the LHB $w_{\rm LHB}$ in increased by the kinetic weight transfer from the UHB [41, 42].  $U_{d}$ (eV) | $\alpha$ | $n_{d}$ | $n_{s}$ | $w_{\rm LHB}^{\;>}$ | $w_{\rm LHB}$
---|---|---|---|---|---
8.0 | 0.20 | 0.358 | 0.139 | 0.343 | 0.700
| 0.50 | 0.445 | 0.051 | 0.079 | 0.524
| 0.70 | 0.475 | 0.021 | 0.011 | 0.485
| 1.00 | 0.473 | 0.025 | 0.002 | 0.475
4.0 | 0.20 | 0.342 | 0.158 | 0.382 | 0.724
| 0.50 | 0.374 | 0.126 | 0.321 | 0.695
| 0.70 | 0.419 | 0.079 | 0.236 | 0.656
| 1.00 | 0.427 | 0.072 | 0.078 | 0.505

Interestingly, the screening increases the density of $s$ electrons and
simultaneously the density in the correlated band $n_{d}$ decreases as in the
undoped case the constraint $n_{d}+n_{s}=1$ is satisfied. This makes the LHB
less than half-filled and considerable spectral weight is transferred from the
UHB to the unoccupied part of the LHB (i.e., above the Fermi energy $\mu$).
The mechanism of such a spectral weight transfer is well known in the partly
filled Hubbard model [27, 28] and explains why the weight of the LHB exceeds
eventually 0.5 per spin. Here doping in the Mott insulator is mimicked by the
partial filling of the $s$ band. The largest transfer of spectral weight is
found at $U_{d}=4.0$ eV and $\alpha=0.2$, see Table I. The UHB forms only in
the correlated $x^{2}-y^{2}$ band and Hubbard subbands are absent within the
$s$ band even at $U_{d}=8.0$ eV.

The regimes of weak and strong screening differ qualitatively. The number of
correlated electrons $n_{d}$ is close to $n_{d}=0.5$ for weak screening but
decreases rapidly for large screening. As a result, the total weight in the
LHB $w_{\rm LHB}$ increases somewhat above $0.7$ both for $U_{d}=8.0$ and
$U_{d}=4.0$ and the transferred weight is large, see Table I. The condition to
activate spectral weight transfer between the Hubbard subbands is finite hole
doping in the LHB of the correlated $d$ band. Indeed, this hole doping nnd
finite density within the $s$ orbitals induces the spectral transfer towards
the LHB in the correlated band.

Altogether, the densities of states $D(\omega)$ give a metallic regime for
intermediate ($\alpha=0.5$) and strong ($\alpha=0.2$) screening of strongly
correlated $x^{2}-y^{2}$ states, see Fig. 3. A large gap between the Hubbard
subbands opens when $U_{d}=8$ eV; this gap is reduced to $\sim 0.5$ eV when
$U_{d}=4.0$ eV. Nevertheless the system has still an insulating gap which
separates the Hubbard subbands. The electronic structure for the $x^{2}-y^{2}$
band is typical for a doped Mott insulator, with the weight of the UHB reduced
by the kinetic processes in a doped system [41, 42]. Indeed, the weight in the
LHB above the Fermi energy increases by $\sim 2\delta$ where $\delta$ stands
for the doping of the LHB, what would also be the weight transferred from the
UHB to the LHB by finite doping. In this regime the $s$ band is only weakly
correlated and Hubbard subbands are poorly visible.

###  III.3 Superconductivity in the two-band model

Figure 5:  Phase diagram in $(U_{d},\alpha)$ plane for increasing Hund’s
exchange, i.e., increasing ratio $J_{0}/U_{d}$: (a) $J_{0}/U_{d}=0.15$ (as
used above); (b) $J_{0}/U_{d}=0.25$. The pairing has $s$\- and $d$-wave
symmetry, depending on the parameters. Tendency towards triplet pairing is
highlighted by gray boxes; the points mark the parameters at which the
calculations were performed.

Finally, we investigate the nature of the SC state. Therefore, we first
compute the ground state of undoped two-band model on a finite square cluster,
i.e., for 8 electrons on 8-site cluster via exact (Lanczos) diagonalization.
Next, the above cluster model is doped by 2 holes (by removing 2 electrons)
and we look again at its ground state. After that pairing operators (for $s$\-
or $d$-wave) are applied on the undoped ground state and the overlap between
the two states is obtained. We consider SC states of both symmetries by
changing the Coulomb interaction $U_{d}$ and the screening $\alpha$ in Fig. 5.

For strong and unscreened Coulomb repulsion $U_{d}>4.0$ eV and weak Hund’s
exchange coupling, see Fig. 5(a), one finds AFM order in the undoped system
with no sign of pairing. This regime resembles the state of cuprates: the
$s$-like band is empty while the strongly correlated $x^{2}-y^{2}$ band is
half-filled and Mott insulating, see Fig. 2. This changes for weaker
correlations (intermediate screening), where AFM order disappears and is
replaced by $s$-wave pairing, see Fig. 5(b).

The presence of $d$-wave pairing known from cuprates requires strong
screening. In the regime of strong screening doped holes enter the
$x^{2}-y^{2}$ band and the model becomes similar to cuprate model. Altogether,
Fig. 5 tells that stronger Hund’s exchange promotes triplet pairing, reduces
effective correlations, and suppresses AFM order.

In the phase diagram of Fig. 5(a), AFM phase and $s$-wave or $d$-wave pairings
are accompanied by some indications of triplet pairing. As expected, the
latter is more pronounced at stronger Hund’s exchange coupling, see Fig. 5(b).
Energies obtained for the doping with either one $\uparrow$ or one
$\downarrow$ hole are here degenerate with the energies obtained with two
$\uparrow$ holes, indicating that the doped hole-pair is a triplet. In order
to check the stability of this result, we used again TBC. The degeneracy is
then lifted and the $S^{z}=0$ state has lower energy, suggesting that triplet
pairing might be a finite-size effect. Finally, we remark that interaction
screening reduces the stability of AFM order in the correlated band and
broadens the range of stability of $s$-wave and $d$-wave pairings, see Fig. 5.
Moreover, the needed Hund’s exchange to stabilize $d$-wave pairing in a broad
regime is rather large ($J_{0}/U_{d}\gtrsim 0.25$).

##  IV Summary and conclusions

In summary, we have used exact diagonalization to investigate an effective
two-band model for infinite-layer nickelates, where the band with a strong
Ni($d_{x^{2}-y^{2}}$) character can be expected to be more correlated than the
one with a rather extended $s$-like wave function of mostly rare-earth
character. We focused here on the interactions in both bands, especially their
relative strength, which also tunes the interorbital interactions between the
two orbitals [20]. The latter give interband interactions and could generate
superconducting pairing.

We have established that both the very strongly correlated and the strongly
screened regimes support the mapping of the two-band model onto a single
Hubbard-like band. For (unrealistically) strong interaction $U_{d}$, we find
an antiferromagnetic Mott insulator without tendencies to superconductivity.
In the more realistic screened regime, the $s$-like band takes up some of the
charge carriers and the states from both bands contribute at the Fermi energy.
In this way the correlated $x^{2}-y^{2}$ band [17] is partly filled and the
spectral weight may be transferred to the unoccupied part of the lower Hubbard
band [41].

For intermediate screening the model is very rich and the $s$-band hosts the
doped holes forming $s$-wave pairs. We point out that this situation broadly
corresponds to a Kondo-lattice–like scenario, with the caveat that the
’localized’ $d_{x^{2}-y^{2}}$ spins can also move [4, 43, 44]. Hund’s exchange
coupling naturally yields ferromagnetic interaction between itinerant $s$
carriers and $d_{x^{2}-y^{2}}$ spins, but it is interesting to note that
$s$-wave pairing at stronger coupling was also obtained in a similar effective
model with AFM spin-spin coupling [44]. Altogether, this shows that the model
investigated here is very rich and predicts pairing of different symmetry.

###### Acknowledgements.

We thank Wojtek Brzezicki and Andres Greco for very insightful discussions. T.
Plienbumrung acknowledges Development and Promotion of Science and Technology
Talents Project (DPST). A. M. Oleś acknowledges Narodowe Centrum Nauki (NCN,
Poland) Project No. 2021/43/B/ST3/02166 and is grateful for support via the
Alexander von Humboldt Foundation Fellowskip (Humboldt-Forschungspreis).

## References

  * Li  _et al._ [2019] D. Li, K. Lee, B. Y. Wang, , M. Osada, S. Crossley, H. R. Lee, Y. Cui, Y. Hikita, and H. Y. Hwang, Superconductivity in an infinite-layer nickelate, Nature (London) 572, 624 (2019).
  * Jiang  _et al._ [2020] M. Jiang, M. Berciu, and G. A. Sawatzky, Critical nature of the Ni spin state in doped NdNiO2, Phys. Rev. Lett. 124, 207004 (2020).
  * Plienbumrung  _et al._ [2021a] T. Plienbumrung, M. Daghofer, and A. M. Oleś, Interplay between Zhang-Rice singlets and high-spin states in a model for doped ${\mathrm{NiO}}_{2}$ planes, Phys. Rev. B 103, 104513 (2021a).
  * Hepting  _et al._ [2020] M. Hepting, D. Li, C. J. Jia, H. Lu, E. Paris, Y. Tseng, X. Feng, M. Osada, E. Been, Y. Hikita, Y.-D. Chuang, Z. Hussain, K. J. Zhou, A. Nag, M. Garcia-Fernandez, M. Rossi, H. Y. Huang, D. J. Huang, Z. X. Shen, T. Schmitt, H. Y. Hwang, B. Moritz, J. Zaanen, T. P. Devereaux, and W. S. Lee, Electronic structure of the parent compound of superconducting infinite-layer nickelates, Nature Materials 19, 381 (2020).
  * Lu  _et al._ [2021] H. Lu, M. Rossi, A. Nag, M. Osada, D. F. Li, K. Lee, B. Y. Wang, M. Garcia-Fernandez, S. Agrestini, Z. X. Shen, E. M. Been, B. Moritz, T. P. Devereaux, J. Zaanen, H. Y. Hwang, K.-J. Zhou, and W. S. Lee, Magnetic excitations in infinite-layer nickelates, Science 373, 213 (2021).
  * Lin  _et al._ [2021] J. Q. Lin, P. Villar Arribi, G. Fabbris, A. S. Botana, D. Meyers, H. Miao, Y. Shen, D. G. Mazzone, J. Feng, S. G. Chiuzbăian, A. Nag, A. C. Walters, M. García-Fernández, K.-J. Zhou, J. Pelliciari, I. Jarrige, J. W. Freeland, J. Zhang, J. F. Mitchell, V. Bisogni, X. Liu, M. R. Norman, and M. P. M. Dean, Strong superexchange in a ${d}^{9-\delta}$ nickelate revealed by resonant inelastic x-ray scattering, Phys. Rev. Lett. 126, 087001 (2021).
  * Hu and Wu [2019] L.-H. Hu and C. Wu, Two-band model for magnetism and superconductivity in nickelates, Phys. Rev. Research 1, 032046 (2019).
  * Zhang and Vishwanath [2020] Y.-H. Zhang and A. Vishwanath, Type-II $t$-$J$ model in superconducting nickelate Nd1-xSrxNiO2, Phys. Rev. Research 2, 023112 (2020).
  * Jiang  _et al._ [2019] P. Jiang, L. Si, Z. Liao, and Z. Zhong, Electronic structure of rare-earth infinite-layer $R$NiO2 ($R$=La,Nd), Phys. Rev. B 100, 201106 (2019).
  * Si  _et al._ [2020] L. Si, W. Xiao, J. Kaufmann, J. M. Tomczak, Y. Lu, Z. Zhong, and K. Held, Topotactic hydrogen in nickelate superconductors and akin infinite-layer oxides ABO2, Phys. Rev. Lett. 124, 166402 (2020).
  * Wu  _et al._ [2020] X. Wu, D. Di Sante, T. Schwemmer, W. Hanke, H. Y. Hwang, S. Raghu, and R. Thomale, Robust ${d}_{{x}^{2}-{y}^{2}}$-wave superconductivity of infinite-layer nickelates, Phys. Rev. B 101, 060504 (2020).
  * Klett  _et al._ [2021] M. Klett, T. Schwemmer, S. Wolf, X. Wu, D. Riegler, A. Dittmaier, D. Di Sante, G. Li, W. Hanke, S. Rachel, and R. Thomale, From high ${T}_{c}$ to low ${T}_{c}$: Multiorbital effects in transition metal oxides, Phys. Rev. B 104, L100502 (2021).
  * Zhang  _et al._ [2021] R. Zhang, C. Lane, B. Singh, J. Nokelainen, B. Barbiellini, R. S. Markiewicz, A. Bansil, and J. Sun, Magnetic and $f$-electron effects in LaNiO2 and NdNiO2 nickelates with cuprate-like $3d_{x^{2}-y^{2}}$ band, Communications Phys. 4, 118 (2021).
  * Been  _et al._ [2021] E. Been, W.-S. Lee, H. Y. Hwang, Y. Cui, J. Zaanen, T. Devereaux, B. Moritz, and C. Jia, Electronic structure trends across the rare-earth series in superconducting infinite-layer nickelates, Phys. Rev. X 11, 011050 (2021).
  * Higashi  _et al._ [2021] K. Higashi, M. Winder, J. Kuneš, and A. Hariki, Core-level x-ray spectroscopy of infinite-layer nickelate: $\mathrm{LDA}+\mathrm{DMFT}$ study, Phys. Rev. X 11, 041009 (2021).
  * Botana  _et al._ [2021] A. S. Botana, K.-W. Lee, M. R. Norman, V. Pardo, and W. E. Pickett, Low valence nickelates: Launching the nickel age of superconductivity, Front. Phys. 9, 813532 (2021).
  * Kitatani  _et al._ [2020] M. Kitatani, L. Si, R. Arita, Z. Zhong, and K. Held, Nickelate superconductors—a renaisance of the one-band Hubbard model, npj Quantum Mat. 5, 59 (2020).
  * Kreisel  _et al._ [2022] A. Kreisel, B. M. Andersen, A. T. Rømer, I. M. Eremin, and F. Lechermann, Superconducting instabilities in strongly correlated infinite-layer nickelates, Phys. Rev. Lett. 129, 077002 (2022).
  * Nomura  _et al._ [2019] Y. Nomura, M. Hirayama, T. Tadano, Y. Yoshimoto, K. Nakamura, and R. Arita, Formation of a two-dimensional single-component correlated electron system and band engineering in the nickelate superconductor NdNiO2, Phys. Rev. B 100, 205138 (2019).
  * Adhikary  _et al._ [2020] P. Adhikary, S. Bandyopadhyay, T. Das, I. Dasgupta, and T. Saha-Dasgupta, Orbital-selective superconductivity in a two-band model of infinite-layer nickelates, Phys. Rev. B 102, 100501 (2020).
  * Gu  _et al._ [2020] Y. Gu, S. Zhu, X. Wang, J. Hu, and H. Chen, A substantial hybridization between correlated Ni-$d$ orbital and itinerant electrons in infinite-layer nickelates, Communications Physics 3, 84 (2020).
  * Lechermann [2020a] F. Lechermann, Multiorbital processes rule the Nd1-xSrxNiO2 normal state, Phys. Rev. X 10, 041002 (2020a).
  * Xie  _et al._ [2022] T. Y. Xie, Z. Liu, C. Cao, Z. F. Wang, J. L. Yang, and W. Zhu, Microscopic theory of superconducting phase diagram in infinite-layer nickelates, Phys. Rev. B 106, 035111 (2022).
  * Oleś  _et al._ [2019] A. M. Oleś, K. Wohlfeld, and G. Khaliullin, Orbital symmetry and orbital excitations in high-$T_{c}$ superconductors, Condensed Matter 4, 46 (2019).
  * Botana and Norman [2020] A. S. Botana and M. R. Norman, Similarities and differences between LaNiO2 and CaCuO2 and implications for superconductivity, Phys. Rev. X 10, 011024 (2020).
  * Budnick  _et al._ [1988] J. I. Budnick, B. Chamberland, D. P. Yang, C. Niedermayer, A. Golnik, E. Recknagel, M. Rossmanith, and A. Weidinger, Dependence of the Nèel-temperatures of La2CuO4 on Sr-doping studied by muon spin rotation, EPL (Europhysics Letters) 5, 651 (1988).
  * Eskes  _et al._ [1991] H. Eskes, M. B. J. Meinders, and G. A. Sawatzky, Anomalous transfer of spectral weight in doped strongly correlated systems, Phys. Rev. Lett. 67, 1035 (1991).
  * Meinders  _et al._ [1993] M. B. J. Meinders, H. Eskes, and G. A. Sawatzky, Spectral-weight transfer: Breakdown of low-energy-scale sum rules in correlated systems, Phys. Rev. B 48, 3916 (1993).
  * Giannozzi  _et al._ [2009] P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, G. L. Chiarotti, M. Cococcioni, I. Dabo, A. D. Corso, S. de Gironcoli, S. Fabris, G. Fratesi, R. Gebauer, U. Gerstmann, C. Gougoussis, A. Kokalj, M. Lazzeri, L. Martin-Samos, N. Marzari, F. Mauri, R. Mazzarello, S. Paolini, A. Pasquarello, L. Paulatto, C. Sbraccia, S. Scandolo, G. Sclauzero, A. P. Seitsonen, A. Smogunov, P. Umari, and R. M. Wentzcovitch, Quantum Espresso: a modular and open-source software project for quantum simulations of materials, Journal of Physics: Condensed Matter 21, 395502 (2009).
  * Giannozzi  _et al._ [2017] P. Giannozzi, O. Andreussi, T. Brumme, O. Bunau, M. B. Nardelli, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, M. Cococcioni, N. Colonna, I. Carnimeo, A. D. Corso, S. de Gironcoli, P. Delugas, R. A. DiStasio, A. Ferretti, A. Floris, G. Fratesi, G. Fugallo, R. Gebauer, U. Gerstmann, F. Giustino, T. Gorni, J. Jia, M. Kawamura, H.-Y. Ko, A. Kokalj, E. Küçükbenli, M. Lazzeri, M. Marsili, N. Marzari, F. Mauri, N. L. Nguyen, H.-V. Nguyen, A. O. de-la Roza, L. Paulatto, S. Poncé, D. Rocca, R. Sabatini, B. Santra, M. Schlipf, A. P. Seitsonen, A. Smogunov, I. Timrov, T. Thonhauser, P. Umari, N. Vast, X. Wu, and S. Baroni, Advanced capabilities for materials modelling with Quantum Espresso, Journal of Physics: Condensed Matter 29, 465901 (2017).
  * Giannozzi  _et al._ [2020] P. Giannozzi, O. Baseggio, P. Bonfà, D. Brunato, R. Car, I. Carnimeo, C. Cavazzoni, S. de Gironcoli, P. Delugas, F. Ferrari Ruffino, A. Ferretti, N. Marzari, I. Timrov, A. Urru, and S. Baroni, Quantum Espresso toward the exascale, The Journal of Chemical Physics 152, 154105 (2020).
  * Dal Corso [2014] A. Dal Corso, Pseudopotentials periodic table: From H to Pu, Comp. Mat. Science 95, 337 (2014).
  * Plienbumrung  _et al._ [2022] T. Plienbumrung, M. Daghofer, M. T. Schmid, and A. M. Oleś, Screening in a two-band model for superconducting infinite-layer nickelate, Phys. Rev. B 106, 134504 (2022).
  * Pizzi  _et al._ [2020] G. Pizzi, V. Vitale, R. Arita, S. Blügel, F. Freimuth, G. Géranton, M. Gibertini, D. Gresch, C. Johnson, T. Koretsune, J. Ibañez-Azpiroz, H. Lee, J.-M. Lihm, D. Marchand, A. Marrazzo, Y. Mokrousov, J. I. Mustafa, Y. Nohara, Y. Nomura, L. Paulatto, S. Poncé, T. Ponweiser, J. Qiao, F. Thöle, S. S. Tsirkin, M. Wierzbowska, N. Marzari, D. Vanderbilt, I. Souza, A. A. Mostofi, and J. R. Yates, Wannier90 as a community code: new features and applications, Journal of Physics: Condensed Matter 32, 165902 (2020).
  * Grant and McMahan [1992] J. B. Grant and A. K. McMahan, Spin bags and quasiparticles in doped ${\mathrm{La}}_{2}$${\mathrm{CuO}}_{4}$, Phys. Rev. B 46, 8440 (1992).
  * Plienbumrung  _et al._ [2021b] T. Plienbumrung, M. T. Schmidt, M. Daghofer, and A. M. Oleś, Character of doped holes in Nd1-xSrxNiO2, Condensed Matter 6, 33 (2021b).
  * Shiroishi and Wadati [1997] M. Shiroishi and M. Wadati, Integrable boundary conditions for the one-dimensional Hubbard model, Jpn. Phys. Soc. 66 (1997).
  * Poilblanc [1991] D. Poilblanc, Twisted boundary conditions in cluster calculations of the optical conductivity in two-dimensional lattice models, Phys. Rev. B 44, 9562 (1991).
  * Yin and Ku [2009] W.-G. Yin and W. Ku, Flavor-twisted boundary condition for simulations of quantum many-body systems, Phys. Rev. B 80, 180402 (2009).
  * Lechermann [2020b] F. Lechermann, Late transition metal oxides with infinite-layer structure: Nickelates versus cuprates, Phys. Rev. B 101, 081110 (2020b).
  * Eskes and Oleś [1994] H. Eskes and A. M. Oleś, Two Hubbard Bands: Weight Transfer in Optical and One-Particle Spectra, Phys. Rev. Lett. 73, 1279 (1994).
  * Eskes  _et al._ [1994] H. Eskes, A. M. Oleś, M. B. J. Meinders, and W. Stephan, Spectral properties of the Hubbard bands, Phys. Rev. B 50, 17980 (1994).
  * Zhang  _et al._ [2020] G.-M. Zhang, Y.-F. Yang, and F.-C. Zhang, Self-doped Mott insulator for parent compounds of nickelate superconductors, Phys. Rev. B 101, 020501 (2020).
  * Wang  _et al._ [2020] Z. Wang, G.-M. Zhang, Y.-F. Yang, and F.-C. Zhang, Distinct pairing symmetries of superconductivity in infinite-layer nickelates, Phys. Rev. B 102, 220501 (2020).


# Political Ideology and Polarization: A Multi-dimensional Approach

Barea Sinno1∗ Bernardo Oviedo2∗ Katherine Atwell3∗
Malihe Alikhani3 Junyi Jessy Li4
1 Political Science, Rutgers University
2 Computer Science, 4 Linguistics, The University of Texas at Austin
3 Computer Science, University of Pittsburgh
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Analyzing ideology and polarization is of critical importance in advancing our
grasp of modern politics. Recent research has made great strides towards
understanding the ideological _bias_ (i.e., stance) of news media along the
left-right spectrum. In this work, we instead take a novel and more nuanced
approach for the study of ideology based on its left or right positions on the
issue being discussed. Aligned with the theoretical accounts in political
science, we treat ideology as a multi-dimensional construct, and introduce the
first diachronic dataset of news articles whose ideological positions are
annotated by trained political scientists and linguists at the paragraph
level. We showcase that, by controlling for the author’s stance, our method
allows for the quantitative and temporal measurement and analysis of
polarization as a multidimensional ideological distance. We further present
baseline models for ideology prediction, outlining a challenging task distinct
from stance detection.

††* Equal contribution ordered by first name.

##  1 Introduction

Political ideology rests on a set of beliefs about the proper order of a
society and ways to achieve this order Jost et al. (2009); Adorno et al.
(2019); Campbell et al. (1980). In Western politics, these worldviews
translate into a multi-dimensional construct that includes: equal opportunity
as opposed to economic individualism; general respect for tradition, hierarchy
and stability as opposed to advocating for social change; and a belief in the
un/fairness and in/efficiency of markets Jost et al. (2009).

The divergence in ideology, i.e., polarization, is the undercurrent of
propaganda and misinformation Vicario et al. (2019); Bessi et al. (2016);
Stanley (2015). It can congest essential democratic functions with an increase
in the divergence of political ideologies. Defined as a growing ideological
distance between groups, polarization has waxed and waned since the advent of
the American Republic Pierson and Schickler (2020).111We distinguish ourselves
from work that considers other types of polarization, e.g., as a measure of
emotional distance Iyengar et al. (2019) or distance between political parties
Lauka et al. (2018). Two eras—post-1896 and -1990s—have witnessed deleterious
degrees of polarization Jenkins et al. (2004); Jensen et al. (2012). More
recently, COVID-19, the murder of George Floyd, and the Capitol riots have
exposed ideological divergences in opinion in the US through news media and
social media. With the hope of advancing our grasp of modern politics, we
study ideology and polarization through the lens of computational linguistics
by presenting a carefully annotated corpus and examining the efficacy of a set
of computational and statistical analyses.

Two dimensions: trade and economic liberalism |  The U.S. aim is to create a monetary system with enough flexibility to prevent bargain-hungry money from rolling around the world like loose ballast on a ship disrupting normal trade and currency flows. Nixon goals: dollar, trade stability. This must be accompanied, Washington says, by reduction of [trade] barriers …
---|---
One dimension: trade protectionism |  The controls program, which Mr. Nixon inaugurated Aug. 15, 1971, has helped to reduce inflation to about 3 percent yearly, and to boost annual U.S. economic growth to more than 7 percent…
Table 1:  Excerpts from news article #730567 in COHA Davies (2012). The first
paragraph advocates for liberalism and the reduction of trade barriers. It
also has a domestic economic dimension. The second paragraph, on the contrary,
advocates for protectionism and a domestic controls program.

In contrast to studying the bias or the stance of the author of the text via
linguistic framing Kulkarni et al. (2018); Kiesel et al. (2019); Baly et al.
(2019, 2020); Chen et al. (2020); Stefanov et al. (2020), we study the little
explored angle that is nonetheless critical in political science research:
ideology of the issue (e.g., policy or concept) under discussion. That is, in
lieu of examining the author’s stance, we focus on addressing the at-issue
content of the text and the ideology that it represents in the implicit social
context. The nuanced co-existence of stance and ideology can be illustrated in
the following excerpt:

> _“Republicans and Joe Biden are making a huge mistake by focusing on cost.
> The implication is that government-run health care would be a good thing–a
> wonderful thing!– if only we could afford it."_(The Federalist, 9/27/2019)__

The author is attacking a liberal social and economic policy; therefore, the
ideology being discussed is liberal on two dimensions—social and economic,
while the author’s stance is conservative. Moreover, our novel approach
acknowledges that ideology can also _vary_ within one article. In Table 1, we
show an example in which one part of an article advocates for trade
liberalism, while another advocates for protectionism.

Together, author stance and ideology inform us not only that there is bias in
the media, but also which beliefs are being supported and/or attacked. A full
analysis of _polarization_ (that reflects a growing distance of political
ideology over time) can then be derived if _diachronic_ data for both author
stance and ideology were available. However, while there has been data for the
former (with articles from recent years only) Kiesel et al. (2019), to date,
there has been no temporal data on the latter.

In this paper, we present a multi-dimensional framework, and an annotated,
diachronic, stance-neutral corpus, for the analysis of ideology in text. This
allows us to study polarization as a _state_ of ideological groups with
divergent positions on a political issue as well as polarization as a
_process_ whose magnitude grows over time DiMaggio et al. (1996). We use
proclaimed center, center-left and center-right media outlets who claim to be
objective in order to focus exclusively and more objectively on the ideology
of the issue being discussed, without the subjectivity of author stance
annotation. We study ideology within every paragraph222We use automatically
segmented paragraphs since the raw texts were not paragraph-segmented. of an
article and aim to answer the following question: which ideological dimension
is present and to which ideological position does it correspond to on the
liberal-conservative spectrum.

Our extensive annotation manual is developed by a political scientist, and the
data then annotated by three linguists after an elaborate training phase
(Section 3). After 150 hours of annotation, we present a dataset of 721 _fully
adjudicated_ annotated paragraphs, from 175 news articles and covering an
average of 7.86 articles per year (excerpts shown in Tables 1, 2, and 3).
These articles originate from 5 news outlets related the US Federal Budget
from 1947-1975 covering the center-left, center, center-right spectrum:
Chicago Tribune (CT), Christian Science Monitor (CSM), the New York Times
(NYT), Time Magazine (TM), and the Wall Street Journal (WSJ).

With this data, we reveal lexical insights on the language of ideology across
the left-right spectrum and across dimensions. We observe that linguistic use
even at word level can reveal the ideology behind liberal and conservative
policies (Section 4). Our framework also enables fine-grained, quantitative
analysis of polarization, which we demonstrate in Section 5. This type of
analysis, if scaled up using accurate models for ideology prediction, has the
potential to reveal impactful insights into the political context of our
society as a whole.

Finally, we present baselines for the automatic identification of multi-
dimensional ideology at the paragraph level (Section 6). We show that this is
a challenging task with our best baseline yielding an F measure of 0.55;
exploring pre-training with existing data in news ideology/bias
identification, we found that this task is distinct from, although correlated
with, labels automatically derived from news outlets. _We contribute our data
and code athttps://github.com/bernovie/political-polarization_.

##  2 Setup

Many political scientists and political psychologists argue for the use of _at
least_ a bidimensional ideology for domestic politics that distinguishes
between economic and social preferences Carsey and Layman (2006); Carmines et
al. (2012); Feldman and Johnston (2014).333It is important to distinguish
between ideology and several other concepts. (1) Partisanship (party identity)
Campbell et al. (1980): a partisan person changes their ideology when their
party changes its ideology, whereas an ideological person changes their party
when their party changes its ideology. Partisanship is easily conflated with
party ID using a unidimensional conceptualization of ideology, but not with a
multi-dimensional one. (2) Moral foundations: Haidt et al. (2009) gave an
evolutionary explanation of how human morals, values and traits such as
freedom, safety, harm, care, reciprocity, in-group loyalty, authority,
equality are formed. Since, some scholars have used these traits to predict
ideology whereas others have attempted to understand what traits unites people
with the same ideology. (3) Framing: frames are used in many ways in political
science. They can refer to different ways scholars describe the same
information or when scholars talk about different aspects of a single problem
Chong and Druckman (2007). We start with these two dimensions while adding a
third dimension, “Foreign”, when the article tackles foreign issues.

Specifically, our annotation task entails examining a news article and
annotating each dimension (detailed below) along three levels— _liberal,
conservative, neutral_ —for each paragraph. The neutral level for every
dimension is reserved for paragraphs related to a specific dimension but
either (a) contain both conservative and liberal elements that annotators were
unable to ascertain an ideological dimension with confidence, or (b) do not
portray any ideology. We additionally provide an _irrelevant_ option if a
dimension does not apply to the paragraph. The three dimensions are:

_Social :_ While the (1) _socially conservative_ aspect of this dimension is
defined as respect for tradition, fear of threat and uncertainty, need for
order and structure, concerns for personal and national security, and
preference for conformity, its (2) _socially liberal_ counterpart has been
associated with a belief in the separation of church and state, tolerance for
uncertainty and change Jost et al. (2009).

_Economic :_ Similarly, while the (3) _economically conservative_ aspect of
this dimension refers to motivations to achieve social rewards, power, and
prestige such as deregulation of the economy, lower taxes and privatization
(i.e., being against deficit) spending and advocating for a balanced budget,
its (4) _economically liberal_ counterpart refers to motivation for social
justice and equality such as issues related to higher taxes on rich
individuals and businesses and more redistribution.

_Foreign :_ After piloting the bidimensional approach on 300 articles, we find
that using only 2 dimensions conflates two important aspects of ideology
related to domestic economy and foreign trade. Tariffs, import quotas, and
other nontariff-based barriers to trade that are aimed at improving employment
and the competitiveness of the US on the international market did not map well
onto the bidimensional framework. After consulting several senior political
scientists, we adopted a third dimension that dealt with the markets as well
as the relations of the US with the rest of the world. While the (5)
_globalist_ counterpart of this dimension accounts for free-trade, diplomacy,
immigration and treaties such as the non-proliferation of arms, its (6)
_interventionist_ aspect is nationalist in its support for excise tax on
imports to protect American jobs and economic subsidies and anti-immigration.

With the annotated data, we demonstrate quantitative measures of polarization
(Section 5) and introduce the modeling task (Section 6) of automatically
identifying the ideology of the policy positions being discussed.

##  3 Data collection and annotation

#### Raw data

Since polarization is a _process_ that needs to be analyzed _over time_
DiMaggio et al. (1996), our annotated articles are sampled from a diachronic
corpus of 1,749 news articles across nearly 3 decades (from 1947 till 1974).
Articles in this corpus are from political news articles of Desai et al.
(2019) from the Corpus of Historical American English (COHA, Davies (2012))
covering years 1922-1986. These 1,749 articles are extracted such that: (1)
they cover broad and politically relevant topics (ranging from education and
health to economy) but still share discussions related to the federal budget
to make our annotations tractable444Because federal budget stories touch on
all aspects of the federal activity, this topic appeals to both liberal and
conservative media and thus can provide a good testing ground to showcase our
proposed ideological annotation method.; (2) balanced in the number of
articles across 5 news outlets with center-left, central, and center-right
ideology (c.f. Section 5): Chicago Tribune (CT), Wall Street Journal (WSJ),
Christian Science Monitor (CSM), the New York Times (NYT), and Time Magazine
(TM). A detailed description of our curation process is in Appendix A.

The raw texts were not segmented into paragraphs, thus we used Topic Tiling
Riedl and Biemann (2012) for automatic segmentation. Topic Tiling finds
segment boundaries again using LDA and, thus, identifies major subtopic
changes within the same article. The segmentation resulted in articles with 1
to 6 paragraphs. The average number of paragraphs per article was 4.

#### Annotation process

Our team (including a political science graduate student) developed an
annotation protocol for expert annotators using definitions in Section 2. The
annotation process is independently reviewed by four political science
professors from two universities in the US who are not authors of this paper;
the research area of two of them is ideology and polarization in the US. We
will release our full annotation interface, protocol, and procedure along with
the data upon publication.

We sampled on average 7.86 articles per year for annotation, for a total of
721 paragraphs across 175 articles. We divided the annotation task into two
batches of 45 and 130 articles, the smaller batch was for training purposes.

In addition to the political science graduate student, we recruited three
annotators, all of whom are recent Linguistics graduates in the US. The
training sessions consisted of one general meeting (all annotators met) and
six different one-on-one meetings (each annotator met with another annotator
once). During initial training, the annotators were asked to highlight
sentences based on which the annotation was performed.

Two dimensions: socially and economically liberal |  … Secretary of Defense Robert S. [Mc-Namara] threw his full support today behind the Administration’s drive against poverty. …Mr. Mc-Namara said : “It is the youth that we can expect to be the most immediate beneficiaries of the war on poverty." He said he was endorsing the “entire program" both as a citizen and as a member of the Cabinet. His endorsement came as his fellow Republicans in Congress continued to hammer away at parts of the Administration’s antipoverty program. …
---|---
Two dimensions: socially and economically conservative |  The antipoverty program, the Republicans insisted, would undercut the authority of the Cabinet members by making Sargent Shriver a "poverty czar." “I don’t see how you can lie down and be a doormat for this kind of operation. "…
Table 2:  Excerpts from article #723847 in COHA. Because the first paragraph calls for minimizing income inequality, it is socially liberal; and because advocating for such a program call for an budgetary expenditure, it is also has an economic liberal dimension. The second paragraph advocates for the exact opposites of the positions in the first paragraph. Therefore, it is socially and economically conservative. Sentences most relevant to these labels are highlighted. Zero dimension |  … “The committee is holding public hearings on President Eisenhower’s Economic Report, which he sent to Congress last week. The Secretary’s [Humphrey] appearance before the group provided an opportunity for political exchanges.
---|---
One dimension: economically liberal |  Senators Paul H. Douglas of Illinois, J. W. Fulbright of Representative Wright Patman of Texas, all Democrats, were active in questioning Mr. Humphrey. The Democrats asserted that the Administration’s tax reduction program was loaded in favor of business enterprises and shareholders in industry and against the taxpayer in the lowest income brackets….
One dimension: economically neutral |  Senator Fulbright .. declare[d] that the problem was to expand consumption rather than production. … “Production is the goose that lays the golden egg,“ Mr. Humphrey replied. “Payrolls make consumers."
Table 3:  Excerpts from article #716033 in COHA. The first paragraph is void
of ideology. In the second paragraph the topic is anti tax reduction on
businesses, thus it is economically liberal. The third paragraph is
simultaneously economically conservative and liberal because one speaker is
advocating for decreasing tax on businesses and asserting that production
gives an advantage to businesses, the other is advocating for decreasing tax
on the poor because they need the income and asserting that healthy businesses
are the ones who pay salaries for the low income bracket worker.

After the annotations of this batch were finalized, the annotators met with
the political science student to create ground truth labels in cases of
disagreement. Then, the three annotators received the second batch and each
article was annotated by 2 annotators. This annotation was composed of two
stages to account for possible subjectivity. In stage 1, each annotator worked
on a batch that overlapped with only one other annotator. In stage 2, the two
annotators examine paragraphs that they disagree, and met with the third
annotator acting as consolidator to adjudicate. Tables 1. 3 and 2 are examples
of adjudicated annotation in the data.

#### Agreement

To assess the inter-annotator agreement of stage 1, we report Krippendorf’s
$\alpha$ (Hayes and Krippendorff, 2007) for each dimension for the 135
articles after training and before any discussion/adjudication: economic
(0.44), foreign (0.68), social (0.39). The agreements among annotators for the
economic and foreign dimensions are moderate & substantial Artstein and Poesio
(2008), respectively; for social, the ‘fair’ agreement was noticed during
annotation, and additional discussion for each paragraph was then held.
Afterwards, 25 more articles were independently annotated and assessed with an
$\alpha$ of 0.53. Although the agreements were not perfect and reflected a
degree of subjectivity in this task, all dimensional labels were _adjudicated_
after discussions between annotators. In total, creating this dataset cost
$\sim$150 hours of manual multi-dimensional labeling.

#### Qualitative analysis of text highlights

For the 25 articles used in training, all annotators highlighted the sentences
that are relevant to each dimension they annotated. This helped annotators to
focus on the sentences that drived their decision, and provided insights to
the language of ideology, which we discuss here. On average, 21%–54% of the
sentences in a paragraph were highlighted.

We found entities such as “President" and "Congress" were the most prevalent
in the highlights, and they tackled social and economic issues combined. This
is not surprising as it suggests that when the media quotes or discusses the
“President" and “Congress", they do so with reference to more complex policy
issues. In contrast, individual congresspeople tackled mostly economic or
social issues. This also is not surprising as it suggests that individual
congresspeople are more concerned with specific issues. Interestingly, “House"
and “Senate" almost always figured more in social issues. This suggest that
when news media speaks about a specific chamber, they do so associating this
chamber with social issues. Finally, party affiliation was infrequent and was
mostly associate with social issues.

##  4 Ideology analyses

| #Docs | Econ. | Soc. | Fgn. | Total
---|---|---|---|---|---
CSM | 37 | 115 | 63 | 82 | 260
CT | 14 | 48 | 33 | 16 | 97
NYT | 60 | 219 | 114 | 130 | 463
TM | 52 | 134 | 60 | 89 | 283
WSJ | 12 | 42 | 21 | 21 | 84
Total | 175 | 558 | 291 | 338 | 1187
Table 4: Dimensional label counts across all 721 paragraphs in the adjudicated
data (there can be multiple dimensions per paragraph).

The number of paragraphs per dimension in total is: Economics (558), Social
(291), Forign (338), across the 175 articles. In Table 4 we tabulate this for
each of the news outlets. Figure 1 shows the dimensional label distributions
per outlet for each dimension. Expectedly, the dimensional labels often
diverge from proclaimed ideology of the news outlet.

Figure 1: Dimensional label distribution per outlet.

We also analyze the percentage of articles that contain at least one pair of
paragraph labels that lean in different directions; for instance, a paragraph
with a label of globalist (i.e., liberal) in the foreign dimension and another
paragraph with a label of conservative or neutral in the fiscal dimension. The
percentage of such articles is 78.3%. Out of these articles, we examine the
average proportion of neutral, liberal, and conservative paragraph labels, and
find neutral labels have the highest share (43.27%), followed by liberal
(33.20%) and conservative (23.53%). In Figure 2 (right), we visualize the
percentage of articles where two dimensional labels co-occur within the same
article. The figure indicates that ideology varies frequently within an
article, showing that a single article-level label will not be fine-grained
enough to capture variances within an article.

In Figure 2 (left) we also show paragraph-level label co-occurrence. Unlike
the article-level, the co-occurrences are less frequent and we are more likely
to observe co-occurrences along the same side of ideology. Still, we see
interesting nuances; for example, on both the paragraph and the article level,
the economic dimension is often neutral, and this tends to co-occur with both
liberal and conservative positions in other dimensions.

Figure 2: Co-occurrence matrices on the paragraph level (left) and article
level (right)

#### Lexical analysis

To understand ways ideology is reflected in text, we also look into the top
vocabulary that associates with conservative or liberal ideology. To do so, we
train a logistic regression model for each dimension to predict whether a
paragraph is labeled conservative or liberal on that dimension, using unigram
frequency as features. In Table 5 we show the top most left-leaning (L) or
right-leaning (R) vocabulary with their weights. The table intimately
reproduces our annotation of ideology. For example, words like federal and
Senator allude to the fact that the topic is at the federal level. The
importance of education and labor to liberals is also evident in the economic
and social dimensions in words like school, education, and wage. The
importance of the topic of taxation and defense is evident in conservative
ideology in words such as tax, business, missile, and force.

Economic |  C: mr (5.2), tax (5.0), truman (3.7), business (3.3), billion (3.2)
---|---
L: school (-4.3), education (-3.3), commission (-3.2), senator (-3.0), plan
(-2.5)
Social |  C: defense (7.5), tax (4.6), air (4.4), billion (3.9), missile (3.8)
L: federal (-3.6), wage (-3.6), would (-3.5), policy (-3.2), labor (-2.9)
Foreign |  C: defense (6.9), force (5.3), north (5.2), air (5.0), vietnam (4.6)
L: aid (-9.3), economic (-5.5), foreign (-5.3), germany (-4.3), make (-4.1)
Table 5: Words with the most positive and negative weights from a logistic
regression model trained to predict liberal/conservative ideology for each
dimension.

##  5 Polarization

In this section, we demonstrate how our framework can be used to analyze
ideological polarization, quantitatively. To say that two groups are polarized
is to say that they are moving to opposite ends of an issue on the multi-
dimensional ideological spectrum while, at the same time, their respective
political views on ideological issues converge within a group, i.e. socially
liberals become also economically liberal Fiorina and Abrams (2008). In
political science when ideology is multidimensional, polarization is often
quantified by considering three measures that capture complementary aspects
Lelkes (2016): (1) sorting Abramowitz and Saunders (1998) (the extent to which
the annotated ideology deviates from an outlet’s proclaimed ideological bias);
(2) issue constraint Baldassarri and Gelman (2008) (a correlational analysis
between pairs of ideological dimensions); (3) ideological divergence Fiorina
and Abrams (2008) (the magnitude of the distance between two groups along a
single dimension). Together these measures describe changes in the ideological
environment over time: a _concurrent_ increase in all three measures indicates
polarization in media.

_Limitations:_ We use only the fully adjudicated data and refrain from using
model predictions, since our baseline experiments in Section 6 show that
predicting ideology is challenging. Hence, the analysis are demonstrations of
what our framework _enables_ , which we discuss at the end of this section,
and the conclusions are drawn for our annotated articles only. We group our
data in four-year periods to reduce sparsity.

#### Measure 1: Sorting

We adapt the sorting principle of Abramowitz and Saunders (1998) to our data
and investigate the difference between the proclaimed ideological bias of a
news outlet and the ideology of annotated articles from the outlet. To obtain
the bias $B_{j}$ of a news outlet $j$, we average the ratings of each news
outlet across common sites that rates media bias (Adfontes, Allsides, and
MBFC), yielding: CSM (-0.07), CT (0.15), NYT (-0.36), TM (-0.4), WSJ (0.32)
(c.f. Table 8 in Appendix B for ratings from each site).

To obtain the overall ideology $I_{i}^{(j)}$ of article $i$ from outlet $j$,
we take the average of liberal (-1), neutral (0), and conservative (1) labels
across its paragraphs in all three dimensions. Thus, for each 4-year time
period with $m$ articles for outlet $j$, the sorting measure would be the
absolute distance of article vs. outlet ideology
$|\text{avg}_{i=1}^{m}(I_{i}^{(j)})-B_{j}|/B_{j}$.

Figure 3: The evolution of the sorting measure, aggregating
conservative/neutral/liberal outlets. Moving further away from the zero means
articles deviate more from the proclaimed ideology of their outlets.

In Figure 3, we plot the sorting measure, averaged across news outlets of the
same proclaimed ideological bias. The figure shows that in our sample of
articles, the left-leaning news outlets were closest to their proclaimed
ideological bias measure over time, whereas the neutral outlets were more
liberal before 1957 and after 1964. The right-leaning outlets were more
conservative at that time than their proclaimed ideological bias.

#### Measure 2: Issue constraint

This measure refers to the tightness between ideological dimensions over time
Baldassarri and Gelman (2008) so as to assess, for example, if socially
liberal dimensions are more and more associated with economic liberal
dimensions for the news outlets. Concretely, for each article we derive its
ideology along a single dimension as the average of paragraph annotations
along that dimension. We, then, calculate the Pearson correlation between the
article ideology of each pair of dimensions, over all articles from one outlet
in the same period.

Figure 4: The evolution of the issue constraint measure, stratified by pairs
of dimensions. Higher values mean some dimensions correlate more strongly than
others. Due to the lack of articles that simultaneously contains social &
economic dimensions (1st graph), and economic & foreign dimensions (3rd graph)
from conservative outlets their respective blue lines start in 1958.

Results in Figure 4, again averaged across news outlets of the same
ideological bias, show that for right-leaning media, the correlation between
any two dimensions in the annotated data are largely positive (e.g.,
economically conservative were also socially conservative) until 1967 or 1970.
However, for proclaimed left-leaning and neutral outlets, the correlations
fluctuates especially when considering the foreign dimension.

#### Measure 3: Ideological divergence

This measures the distance between two ideological groups on a single
dimension Fiorina and Abrams (2008). We follow Lelkes (2016) and calculate the
bimodality coefficient Freeman and Dale (2013); Pfister et al. (2013) per
dimension over articles from all news outlets over the same time period. The
bimodality coefficient ranges from 0 (unimodal, thus not at all polarized) to
1 (bimodal, thus completely polarized).

Figure 5 shows the evolution of the ideological divergence measure of every
dimension. A bimodality measure assesses whether this divergence attained the
threshold for the cumulative distribution to be considered bimodal.
Ideological distance, as a result, refers to the three bimodal coefficients.
We note, for example, that the foreign dimension crossed this threshold
between 1956 and 1968. This means that proclaimed left-leaning and right-
leaning outlets grew further apart on foreign issues during this time period.

Figure 5: The evolution of the ideological divergence measure stratified by
dimension. The dotted line refers to the bimodality threshold Lelkes (2016).
Higher values mean the ideology of an article along that one dimension is
bimodal.

#### Discussion

Taken together, the graphs indicate that the years between 1957 and 1967 are
the most noteworthy. During this period, from our sample of articles, we see
that polarization was only present in conservative news media because it (1)
sorted, as it was significantly more conservative than its composite bias
measure, (2) constrained its issues, as evidence by high positive correlation
values, and (3) became increasingly bimodal, as the ideological distance
between their positions and those of their liberal counterpart on foreign
issues increased over time. While this conclusion applies to only the set of
articles in our dataset, the above analysis illustrates that our framework
enables nuanced, quantitative analyses into polarization. We leave for future
work, potentially equipped with strong models for ideology prediction, to
analyze the data at scale.

##  6 Experiments

We present political ideology detection experiments as classification tasks
per-dimension on the paragraph level.

We performed an 80/10/10 split to create the train, development, and test
sets. The development and test sets contain articles uniformly distributed
from our time period (1947 to 1974) such that no particular decade is
predominant. To ensure the integrity of the modeling task, all paragraphs
belonging to the same article are present in a single split. The number of
examples in the splits for each dimension for the adjudicated data are as
follows: for the economic dimension, we had 450 training, 50 development, and
58 test examples. For the social dimension, we had 253 for training, 13 for
development, and 25 for testing. For the foreign dimension, we had 266 for
training, 33 for development, and 39 for testing.

###  6.1 Models

#### Recurrent neural networks

We trained a 2-layer bidirectional LSTM Hochreiter and Schmidhuber (1997),
with sequence length and hidden size of 256, and 100D GloVe embeddings
Pennington et al. (2014).

#### Pre-trained language models

We used BERT-base Devlin et al. (2019) from HuggingFace Wolf et al. (2020) and
trained two versions, with and without fine-tuning. In both cases we used a
custom classification head consisting of 2 linear layers with a hidden size of
768 and a ReLU between them. To extract the word embeddings we followed Devlin
et al. (2019) and used the hidden states from the second to last layer. To
obtain the embedding of the whole paragraph55599% of the paragraphs in the
dataset have $\leq$512 tokens. we averaged the word embeddings and passed this
vector to the classification head.

To find the best hyperparameters we performed a grid search in each dimension.
For the economic dimension, the best hyperparameters consisted of a learning
rate of 2e-6, 6 epochs of training, a gamma value of 2, no freezing of the
layers, a 768 hidden size, and 10% dropout. For the social dimension, the best
hyperparameters were a learning rate of 2e-5, 12 epochs, a gamma of 4, no
freezing of the layers, a 768 hidden size, and 10% dropout. Finally, for the
foreign dimension the best hyperparameters consisted of a learning rate of
2e-5, 6 epochs, a gamma of 2, no freezing of the layers, a 768 hidden size,
and a 10% dropout.

#### Focal loss.

To better address the imbalanced label distribution of this task, we
incorporated _focal loss_ Lin et al. (2017), originally proposed for dense
object detection. Focal loss can be interpreted as a dynamically scaled cross-
entropy loss, where the scaling factor is inversely proportional to the
confidence on the correct prediction. This dynamic scaling, controlled by
hyperparameter $\gamma$, leads to a higher focus on the examples that have
lower confidences on the correct predictions, which in turn leads to better
predictions on the minority classes. Since a $\gamma$ of 0 essentially turns a
focal loss into a cross entropy loss, it has less potential to hurt
performance than to improve it. We found the best $\gamma$ values to be 2 or 4
depending on the dimension.

#### Task-guided pre-training.

We also explored supervised pre-training on two adjacent tasks that can give
insights to the relationship between tasks. We used distant supervision that
labeled the ideological bias of each article according to that of its news
outlet from www.allsides.com Kulkarni et al. (2018). This procedure allowed us
to use the unannotated articles. 666We also experimented with pre-training on
the dataset from Chen et al. (2020). However, because their dataset starts
from 2006 (outside of our time domain), this setting performed poorly.

| Econ | Social | Foreign | Average
---|---|---|---|---
Majority | 0.30 | 0.23 | 0.25 | 0.26
BiLSTM | 0.44 | 0.37 | 0.33 | 0.38
BERT no-ft | 0.46 | 0.31 | 0.53 | 0.44
+pre-training | 0.42 | 0.32 | 0.46 | 0.40
BERT ft | 0.64 | 0.50 | 0.52 | 0.55
+pre-training | 0.56 | 0.47 | 0.46 | 0.49
-focal loss | 0.61 | 0.50 | 0.50 | 0.54
Table 6: Macro F1 of the models averaged across 10 runs.

###  6.2 Results

Table 6 shows the macro F1 for each configuration, averaged across 10 runs
with different random initializations. The fine-tuned BERT model, with no
task-guided pre-training shows the best performance across all 3 ideology
dimensions. It is important to note that all the models do better than
randomly guessing, and better than predicting the majority class. This shows
that the models are capturing some of the complex underlying phenomena in the
data. However, the classification tasks still remain challenging for neural
models, leaving plenty of room for improvement in future work.

The _BERT ft -focal loss_ setting ablates the effect of focal loss against a
weighted cross entropy loss. with weights inversely proportional to the
distribution of the classes in the dimension. This loss helped get a bump in
the macro F1 score of around 0.1 for each dimension compared to an unweighted
cross entropy loss. However, the focal loss gave further improvements for 2 of
the 3 dimensions. Although task-guided pre-training improved the BERT (no
fine-tuning) model for 1 of the 3 dimensions, it led to worse performance than
BERT (fine-tuned). The improvement on the no fine-tuning setting indicates
that there is a potential correlation to be exploited by the ideology of the
news outlet, but such labels are not that informative for multi-dimensional
prediction. We hope that this dataset provides a testbed for future work to
evaluate more distant supervision data/methods.

##  7 Related work

In contrast to our multi-dimensional approach that examines the ideology of
the issue being discussed instead of the author stance, much of the recent
work in computational linguistics has been dedicated to the latter (detection
of ideological bias in news media) while collapsing ideology to one dimension
Budak et al. (2016); Kulkarni et al. (2018); Kiesel et al. (2019); Baly et al.
(2019, 2020); Chen et al. (2020); Ganguly et al. (2020); Stefanov et al.
(2020). The proposed computational models classify the partiality of media
sources without quantifying their ideology Elejalde et al. (2018).

Other researchers interested in the computational analysis of the ideology
have employed text data to analyze congressional text data at the legislative
level Sim et al. (2013); Gentzkow et al. (2016) and social media text at the
electorate level Saez-Trumper et al. (2013); Barberá (2015).

In political science, the relationship between (news) media and polarization
is also an active area of research. Prior work has studied media ideological
bias in terms of coverage George and Waldfogel (2006); Valentino et al.
(2009). Prior (2013) argues there is no firm evidence of a direct causal
relationship between media and polarization and that this relationship depends
on preexisting attitudes and political sophistication. On the other hand,
Gentzkow et al. (2016) have established that polarization language snippets
move from the legislature in the direction of the media whereas Baumgartner et
al. (1997) have shown that the media has an impact on agenda settings of
legislatures.

##  8 Conclusion

We take the first step in studying multi-dimensional ideology and polarization
over time and in news articles relying on the major political science theories
and tools of computational linguistics. Our work opens up new opportunities
and invites researchers to use this corpus to study the spread of propaganda
and misinformation in tandem with ideological shifts and polarization. The
presented corpus also provides the opportunity for studying ways that social
context determines interpretations in text while distinguishing author stance
from content.

This work has several limitations. We only focus on news whereas these
dynamics might be different in other forms of communication such as social
media posts or online conversations, and the legislature. Further, our corpus
is relatively small although carefully annotated by experts. Future work may
explore semi-supervised models or active learning techniques for annotating
and preparing a larger corpus that may be used in diverse applications.

## Acknowledgements

This research is partially supported by NSF grants IIS-1850153, IIS-2107524,
and Good Systems777http://goodsystems.utexas.edu, a UT Austin Grand Challenge
to develop responsible AI technologies. We thank Cutter Dalton, Kathryn
Slusarczyk and Ilana Torres for their help with complex data annotation. We
thank Zachary Elkins, Richard Lau, Beth Leech, and Katherine McCabe for their
feedback of this work and its annotation protocol, and Katrin Erk for her
comments. Thanks to Pitt Cyber888https://www.cyber.pitt.edu for supporting
this project. We also acknowledge the Texas Advanced Computing Center
(TACC)999https://www.tacc.utexas.edu at UT Austin and The Center for Research
Computing at the University of Pittsburgh for providing the computational
resources for many of the results within this paper.

## References

  * Abramowitz and Saunders (1998) Alan I Abramowitz and Kyle L Saunders. 1998.  Ideological realignment in the us electorate.  _The Journal of Politics_ , 60(3):634–652.
  * Adorno et al. (2019) Theodor Adorno, Else Frenkel-Brenswik, Daniel J Levinson, and R Nevitt Sanford. 2019\.  _The authoritarian personality_.  Verso Books.
  * Artstein and Poesio (2008) Ron Artstein and Massimo Poesio. 2008.  Inter-coder agreement for computational linguistics.  _Computational Linguistics_ , 34(4):555–596.
  * Baldassarri and Gelman (2008) Delia Baldassarri and Andrew Gelman. 2008.  Partisans without constraint: Political polarization and trends in american public opinion.  _American Journal of Sociology_ , 114(2):408–446.
  * Baly et al. (2020) Ramy Baly, Giovanni Da San Martino, James Glass, and Preslav Nakov. 2020.  We can detect your bias: Predicting the political ideology of news articles.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4982–4991.
  * Baly et al. (2019) Ramy Baly, Georgi Karadzhov, Abdelrhman Saleh, James Glass, and Preslav Nakov. 2019\.  Multi-task ordinal regression for jointly predicting the trustworthiness and the leading political ideology of news media.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2109–2116.
  * Barberá (2015) Pablo Barberá. 2015.  Birds of the same feather tweet together: Bayesian ideal point estimation using twitter data.  _Political analysis_ , 23(1):76–91.
  * Baumgartner et al. (1997) Frank R Baumgartner, Bryan D Jones, and Beth L Leech. 1997.  Media attention and congressional agendas.  _Do the media govern_ , pages 349–363.
  * Bender et al. (2021) Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021.  On the dangers of stochastic parrots: Can language models be too big?  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , pages 610–623.
  * Bessi et al. (2016) Alessandro Bessi, Fabio Petroni, Michela Del Vicario, Fabiana Zollo, Aris Anagnostopoulos, Antonio Scala, Guido Caldarelli, and Walter Quattrociocchi. 2016\.  Homophily and polarization in the age of misinformation.  _The European Physical Journal Special Topics_ , 225(10):2047–2059.
  * Blei et al. (2003) David M Blei, Andrew Y Ng, and Michael I Jordan. 2003.  Latent dirichlet allocation.  _Journal of machine Learning research_ , 3(Jan):993–1022.
  * Budak et al. (2016) Ceren Budak, Sharad Goel, and Justin M Rao. 2016.  Fair and balanced? Quantifying media bias through crowdsourced content analysis.  _Public Opinion Quarterly_ , 80(S1):250–271.
  * Campbell et al. (1980) Angus Campbell, Philip E Converse, Warren E Miller, and Donald E Stokes. 1980.  _The american voter_.  University of Chicago Press.
  * Carmines et al. (2012) Edward G Carmines, Michael J Ensley, and Michael W Wagner. 2012.  Who fits the left-right divide? partisan polarization in the american electorate.  _American Behavioral Scientist_ , 56(12):1631–1653.
  * Carsey and Layman (2006) Thomas M Carsey and Geoffrey C Layman. 2006.  Changing sides or changing minds? party identification and policy preferences in the american electorate.  _American Journal of Political Science_ , 50(2):464–477.
  * Chen et al. (2020) Wei-Fan Chen, Khalid Al Khatib, Henning Wachsmuth, and Benno Stein. 2020.  Analyzing political bias and unfairness in news articles at different levels of granularity.  In _Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science_ , pages 149–154.
  * Chong and Druckman (2007) Dennis Chong and James N Druckman. 2007.  Framing theory.  _Annual Review of Political Science_ , 10:103–126.
  * Davies (2012) Mark Davies. 2012.  Expanding horizons in historical linguistics with the 400-million word corpus of historical american english.  _Corpora_ , 7(2):121–157.
  * Desai et al. (2019) Shrey Desai, Barea Sinno, Alex Rosenfeld, and Junyi Jessy Li. 2019.  Adaptive ensembling: Unsupervised domain adaptation for political document analysis.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4712–4724.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * DiMaggio et al. (1996) Paul DiMaggio, John Evans, and Bethany Bryson. 1996.  Have american’s social attitudes become more polarized?  _American journal of Sociology_ , 102(3):690–755.
  * Elejalde et al. (2018) Erick Elejalde, Leo Ferres, and Eelco Herder. 2018.  On the nature of real and perceived bias in the mainstream media.  _PloS one_ , 13(3):e0193765.
  * Feldman and Johnston (2014) Stanley Feldman and Christopher Johnston. 2014.  Understanding the determinants of political ideology: Implications of structural complexity.  _Political Psychology_ , 35(3):337–358.
  * Fiorina and Abrams (2008) Morris P Fiorina and Samuel J Abrams. 2008.  Political polarization in the american public.  _Annu. Rev. Polit. Sci._ , 11:563–588.
  * Freeman and Dale (2013) Jonathan B Freeman and Rick Dale. 2013.  Assessing bimodality to detect the presence of a dual cognitive process.  _Behavior research methods_ , 45(1):83–97.
  * Ganguly et al. (2020) Soumen Ganguly, Juhi Kulshrestha, Jisun An, and Haewoon Kwak. 2020.  Empirical evaluation of three common assumptions in building political media bias datasets.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 14, pages 939–943.
  * Gentzkow et al. (2016) Matthew Gentzkow, Jesse M Shapiro, and Matt Taddy. 2016.  _Measuring polarization in high-dimensional data: Method and application to congressional speech_.  National Bureau of Economic Research.
  * George and Waldfogel (2006) Lisa M George and Joel Waldfogel. 2006.  The new york times and the market for local newspapers.  _American Economic Review_ , 96(1):435–447.
  * Haidt et al. (2009) Jonathan Haidt, Jesse Graham, and Craig Joseph. 2009.  Above and below left–right: Ideological narratives and moral foundations.  _Psychological Inquiry_ , 20(2-3):110–119.
  * Hayes and Krippendorff (2007) Andrew F Hayes and Klaus Krippendorff. 2007.  Answering the call for a standard reliability measure for coding data.  _Communication methods and measures_ , 1(1):77–89.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural Comput._ , 9(8):1735–1780.
  * Iyengar et al. (2019) Shanto Iyengar, Yphtach Lelkes, Matthew Levendusky, Neil Malhotra, and Sean J Westwood. 2019.  The origins and consequences of affective polarization in the united states.  _Annual Review of Political Science_ , 22:129–146.
  * Jenkins et al. (2004) Jeffery A Jenkins, Eric Schickler, and Jamie L Carson. 2004.  Constituency cleavages and congressional parties: Measuring homogeneity and polarization, 1857–1913.  _Social Science History_ , 28(4):537–573.
  * Jensen et al. (2012) Jacob Jensen, Suresh Naidu, Ethan Kaplan, Laurence Wilse-Samson, David Gergen, Michael Zuckerman, and Arthur Spirling. 2012.  Political polarization and the dynamics of political language: Evidence from 130 years of partisan speech [with comments and discussion].  _Brookings Papers on Economic Activity_ , pages 1–81.
  * Jost et al. (2009) John T Jost, Christopher M Federico, and Jaime L Napier. 2009.  Political ideology: Its structure, functions, and elective affinities.  _Annual review of psychology_ , 60:307–337.
  * Kiesel et al. (2019) Johannes Kiesel, Maria Mestre, Rishabh Shukla, Emmanuel Vincent, Payam Adineh, David Corney, Benno Stein, and Martin Potthast. 2019.  Semeval-2019 task 4: Hyperpartisan news detection.  In _Proceedings of the 13th International Workshop on Semantic Evaluation_ , pages 829–839.
  * Kulkarni et al. (2018) Vivek Kulkarni, Junting Ye, Steven Skiena, and William Yang Wang. 2018.  Multi-view models for political ideology detection of news articles.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3518–3527.
  * Lauka et al. (2018) Alban Lauka, Jennifer McCoy, and Rengin B Firat. 2018.  Mass partisan polarization: Measuring a relational concept.  _American behavioral scientist_ , 62(1):107–126.
  * Lelkes (2016) Yphtach Lelkes. 2016.  Mass polarization: Manifestations and measurements.  _Public Opinion Quarterly_ , 80(S1):392–410.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017\.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2980–2988.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Pfister et al. (2013) Roland Pfister, Katharina A Schwarz, Markus Janczyk, Rick Dale, and Jon Freeman. 2013.  Good things peak in pairs: a note on the bimodality coefficient.  _Frontiers in psychology_ , 4:700.
  * Pierson and Schickler (2020) Paul Pierson and Eric Schickler. 2020.  Madison’s constitution under stress: A developmental analysis of political polarization.  _Annual Review of Political Science_ , 23:37–58.
  * Prior (2013) Markus Prior. 2013.  Media and political polarization.  _Annual Review of Political Science_ , 16:101–127.
  * Riedl and Biemann (2012) Martin Riedl and Chris Biemann. 2012.  Topictiling: a text segmentation algorithm based on lda.  In _Proceedings of ACL 2012 Student Research Workshop_ , pages 37–42.
  * Rijhwani and Preotiuc-Pietro (2020) Shruti Rijhwani and Daniel Preotiuc-Pietro. 2020.  Temporally-informed analysis of named entity recognition.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7605–7617.
  * Saez-Trumper et al. (2013) Diego Saez-Trumper, Carlos Castillo, and Mounia Lalmas. 2013.  Social media news communities: gatekeeping, coverage, and statement bias.  In _Proceedings of the 22nd ACM international conference on Information & Knowledge Management_, pages 1679–1684.
  * Sim et al. (2013) Yanchuan Sim, Brice DL Acree, Justin H Gross, and Noah A Smith. 2013.  Measuring ideological proportions in political speeches.  In _Proceedings of the 2013 conference on empirical methods in natural language processing_ , pages 91–101.
  * Stanley (2015) Jason Stanley. 2015.  _How propaganda works_.  Princeton University Press.
  * Stefanov et al. (2020) Peter Stefanov, Kareem Darwish, Atanas Atanasov, and Preslav Nakov. 2020.  Predicting the topical stance and political leaning of media using tweets.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 527–537.
  * Valentino et al. (2009) Nicholas A Valentino, Antoine J Banks, Vincent L Hutchings, and Anne K Davis. 2009\.  Selective exposure in the internet age: The interaction between anxiety and information utility.  _Political Psychology_ , 30(4):591–613.
  * Vicario et al. (2019) Michela Del Vicario, Walter Quattrociocchi, Antonio Scala, and Fabiana Zollo. 2019\.  Polarization and fake news: Early warning of potential misinformation targets.  _ACM Transactions on the Web (TWEB)_ , 13(2):1–22.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations_ , pages 38–45.

##  Appendix A Data curation

A diachronic corpus is required to measure and analyze polarization over time
DiMaggio et al. (1996). We collect and annotate data across a long period to
address the issue of distributional shifts across years Desai et al. (2019);
Rijhwani and Preotiuc-Pietro (2020); Bender et al. (2021) and help build
robust models that can generalize beyond certain periods.

Additionally, the raw data on top of which we annotate needs to satisfy the
following constraints: (1) for human annotation to be tractable, the articles
should share some level of topical coherence; (2) for the data to be useful
for the larger community, the content should also cover a range of common
discussions in politics across the aisle; and (3) the articles should come
from a consistent set of news outlets, forming a continuous and ideologically
balanced corpus.

We start with the diachronic corpus of political news articles of Desai et al.
(2019) which covers years 1922-1986, the longest-spanning dataset to our
knowledge. This corpus is a subset of news articles from the Corpus of
Historical American English (COHA, Davies (2012)). To extract topically
coherent articles, we investigate the topics and articles across multiple LDA
Blei et al. (2003) runs varying the number of topics (15, 20, 30, 50), aiming
to arrive at a cluster of topics that share common points of discussion and
collectively will yield a sizable number of articles each year from the same
news outlets.

The LDA models consistently showed one prominent topic—the federal
budget—across 5 news outlets with balanced ideology (c.f. Table 8): Chicago
Tribune (CT), Wall Street Journal (WSJ), Christian Science Monitor (CSM), the
New York Times (NYT), and Time Magazine (TM). Because federal budget stories
touch on all aspects of the federal activity, this topic appeals to both
liberal and conservative media and thus can provide a good testing ground to
showcase our proposed ideological annotation method. In addition to the core
federal budget topic (topic 5 of Table 7), we also include other topics such
as health and education that are integral parts of ideological beliefs in the
United States, and when discussed at the federal government level, are
typically related to the federal budget. The top vocabulary of the cluster is
shown in Table 7. In an effort to purge articles unrelated to the federal
budget, we selected only those that contain words such as “federal” and
“congress”, and excluded those that mention state budget, and letters to
editors. (Note that during annotation, we also discard articles that are
unrelated to the federal budget.) After this curation, the total number of
articles is 5,706 from the 5 outlets.

Topic1: Trade |  bank, market, farm, loan, export, agricultur, farmer, dollar, food, debt
---|---
Topic2: Business |  incom, tax, revenu, profit, corpor, financ, treasuri, pay, sale, bond
Topic3: Education |  school, univers, educ, student, colleg, professor, institut, teacher, research, graduat
Topic4: Defense |  nuclear, missil, weapon, atom, test, energi, strateg, bomb, space, pentagon
Topic5: Economy |  budget, billion, economi, inflat, economic, deficit, unemploy, cut, dollar, rate
Topic6: Health/Race |  negro, hospit, medic, health, racial, southern, discrimin, doctor, contra, black
Topic7: Industry |  compani, contract, plant, steel, coal, wage, railroad, corpor, manufactur, miner
Table 7:  Top words from topics selected in our cluster, from the 50-topic LDA
model that yielded the most well-deliminated topics.

To account for the sparsity of articles in the first decades and their density
in later decades, we narrowed down the articles to the period from 1947 to
1974. We believe this period is fitting because it includes various
ideological combinations of the tripartite composition of the American
government, Congress and presidency101010For example, between 1947-49,
Congress was Republican and the President was a Democrat while the story
flipped between 1955-57.. The total number of articles in the final corpus of
political articles on the federal budget from 1947 to 1974 is 1,749.

##  Appendix B Proclaimed ideology of news outlets

|  Adfontes |  Allsides |  MBFC |  _Average_
---|---|---|---|---
CSM |  -.06 |  0.00 |  -.16 |  -.07
CT |  -.04 |  NA |  .34 |  .15
NYT |  -.20 |  -.5 |  -.4 |  -.36
TM |  -.10 |  -.5 |  -.6 |  -.4
WSJ |  .15 |  .25 |  .58 |  .32
Table 8: Ideological bias of news outlets from common references of media
bias. We use the average in our analyses.