2015 035 19 February

Precision measurement of the mass difference between light nuclei and anti-
nuclei

ALICE Collaboration††thanks: See Appendix A for the list of collaboration
members ALICE Collaboration

The measurement of the mass differences for systems bound by the strong force
has reached a very high precision with protons and anti-protons. The extension
of such measurement from (anti-)baryons to (anti-)nuclei allows one to probe
any difference in the interactions between nucleons and anti-nucleons encoded
in the (anti-)nuclei masses. This force is a remnant of the underlying strong
interaction among quarks and gluons and can be described by effective
theories, but cannot yet be directly derived from quantum chromodynamics. Here
we report a measurement of the difference between the ratios of the mass and
charge of deuterons (d) and anti-deuterons ($\rm{\overline{d}}$), and
$\rm{{}^{3}He}$ and $\rm{{}^{3}\overline{He}}$ nuclei carried out with the
ALICE (A Large Ion Collider Experiment) detector in Pb-Pb collisions at a
centre-of-mass energy per nucleon pair of 2.76 TeV. Our direct measurement of
the mass-over-charge differences confirm CPT invariance to an unprecedented
precision in the sector of light nuclei. This fundamental symmetry of nature,
which exchanges particles with anti-particles, implies that all physics laws
are the same under the simultaneous reversal of charge(s) (charge conjugation
C), reflection of spatial coordinates (parity transformation P) and time
inversion (T).

The measurement of the mass differences for systems bound by the strong force
has reached a very high precision with protons and anti-protons[1, 2]. The
extension of such measurement from (anti-)baryons to (anti-)nuclei allows one
to probe any difference in the interactions between nucleons and anti-nucleons
encoded in the (anti-)nuclei masses. This force is a remnant of the underlying
strong interaction among quarks and gluons and can be described by effective
theories[3], but cannot yet be directly derived from quantum chromodynamics.
Here we report a measurement of the difference between the ratios of the mass
and charge of deuterons (d) and anti-deuterons ($\rm{\overline{d}}$), and
$\rm{{}^{3}He}$ and $\rm{{}^{3}\overline{He}}$ nuclei carried out with the
ALICE (A Large Ion Collider Experiment)[4] detector in Pb-Pb collisions at a
centre-of-mass energy per nucleon pair of 2.76 TeV. Our direct measurement of
the mass-over-charge differences confirm CPT invariance to an unprecedented
precision in the sector of light nuclei[5, 6]. This fundamental symmetry of
nature, which exchanges particles with anti-particles, implies that all
physics laws are the same under the simultaneous reversal of charge(s) (charge
conjugation C), reflection of spatial coordinates (parity transformation P)
and time inversion (T). Heavy ions are collided at very high energies at the
CERN Large Hadron Collider (LHC) to study matter at extremely high
temperatures and densities. Under these conditions heavy-ion collisions are a
copious source of matter and anti-matter particles and thus are suitable for
an experimental investigation of their properties such as mass and electric
charge. In relativistic heavy-ion collisions, nuclei and corresponding anti-
nuclei are produced with nearly equal rates[7]. Their yields have been
measured at the Relativistic Heavy Ion Collider (RHIC) by the STAR[8] and
PHENIX[9] experiments and at the LHC by the ALICE[4] experiment. To date, the
heaviest anti-nucleus which has been observed[7] is $\rm{{}^{4}\overline{He}}$
(anti-$\rm{\alpha}$); meanwhile, for lighter nuclei and anti-nuclei, which are
more copiously produced, a detailed comparison of their properties is
possible. This comparison represents an interesting test of CPT symmetry in an
analogous way as done for elementary fermions[10, 11] and bosons[12], and for
QED[13, 14] and QCD systems[1, 2, 15, 16, 17] (a particular example for the
latter being the measurements carried out on neutral kaon decays[18]), with
different levels of precision which span over several orders of magnitude. All
these measurements can be used to constrain, for different interactions, the
parameters of effective field theories that add explicit CPT violating terms
to the Standard Model Lagrangian, such as the Standard Model Extension[19]
(SME).

The measurements reported in this paper are based on the high-precision
tracking and identification capabilities of the ALICE experiment[20]. The main
detectors employed in this analysis are the ITS[21] (Inner Tracking System)
for the determination of the interaction vertex, the TPC[22] (Time Projection
Chamber) for tracking and specific energy loss (${\rm d}E/{\rm d}x$)
measurements, and the TOF[23] (Time Of Flight) detector to measure the time
$t_{{\rm TOF}}$ needed by each track to traverse the detector. The combined
ITS and TPC information is used to determine the track length ($L$) and the
rigidity ($p/z$, where $p$ is the momentum and $z$ the electric charge in
units of the elementary charge $e$) of the charged particles in the solenoidal
0.5 T magnetic field of the ALICE central barrel (pseudo-rapidity
$|\eta|<0.8$). Based on these measurements, we can extract the squared mass-
over-charge ratio $\mu^{2}_{\rm{TOF}}\equiv\left(m/z\right)^{2}_{{\rm
TOF}}=(p/z)^{2}\,[(t_{\rm TOF}/L)^{2}-1/c^{2}]$. The choice of this variable
is motivated by the fact that $\mu^{2}$ is directly proportional to the square
of the time of flight, allowing to better preserve its Gaussian behaviour.

The high precision of the TOF detector, which determines the arrival time of
the particle with a resolution of 80 ps[20], allows us to measure a clear
signal for (anti-)protons, (anti-)deuterons and (anti-)$\rm{{}^{3}He}$ nuclei
over a wide rigidity range ($1<p/|z|<4~{}\rm{GeV}/{\it c}$). The main source
of background, which is potentially of the same order of the signal, arises
from tracks erroneously associated to a TOF hit. To reduce this contamination,
a $2\sigma$ cut (where $\sigma$ is the standard deviation) around the expected
TPC ${\rm d}E/{\rm d}x$ signal is applied. Such a requirement strongly
suppresses (to below 4%) this background for rigidities below
$p/|z|<2.0~{}\rm{GeV/{\it c}}$ for (anti-)deuterons and for all rigidities for
(anti-)$\rm{{}^{3}He}$ (to below 1%). For each of the species under study, the
mass is extracted by fitting the mass-squared distributions in narrow $p/|z|$
and $\eta$ intervals, using a Gaussian with a small exponential tail that
reflects the time signal distribution of the TOF detector. Examples of the
mass-squared distributions for (anti-)deuterons and (anti-)$\rm{{}^{3}He}$
candidates are reported in Fig. 1 in selected rigidity intervals.

Using mass differences, rather than absolute masses, allows us to reduce the
systematic uncertainties related to tracking, spatial alignment (affecting the
measurement of the track momentum and length) and time calibration. Despite
that, residual effects are still present, due to imperfections in the detector
alignment and the description of the magnetic field, which can lead to
position-dependent systematic uncertainties. In terms of relative
uncertainties, the ones affecting the measurement of the momentum are the
largest and independent of the mass, and are the same for all positive
(negative) particles in a given momentum interval. It is therefore possible to
correct the (anti-)deuteron and the (anti-)$\rm{{}^{3}He}$ masses by scaling
them with the ratio between the (anti-)proton masses recommended by PDG[24]
($\mu_{{\rm p(\bar{p})}}^{{\rm PDG}}$) and the ones measured in the analysis
presented here ($\mu_{{\rm p(\bar{p})}}^{{\rm TOF}}$), i.e. $\mu_{{\rm
A(\bar{A})}}=\mu_{{\rm A(\bar{A})}}^{{\rm TOF}}\times(\mu_{{\rm
p(\bar{p})}}^{{\rm PDG}}/\mu_{{\rm p(\bar{p})}}^{{\rm TOF}})$. These
correction factors, which depend on the rigidity, deviate from unity by at
most 1%. Conversely, systematic effects connected to the track-length
measurement are mass dependent and cannot be completely accounted for using
the above correction. However, they are expected to be symmetric for positive
and negative particles when inverting the magnetic field. Any residual
asymmetry is therefore indicative of remaining systematic uncertainties
related to the detector conditions. In order to estimate them and keep these
effects under control, both nuclei and anti-nuclei measurements are performed
for two opposite magnetic field configurations and then averaged. Their half
difference is taken as the estimate of this systematic uncertainty. Other
sources of systematic uncertainties are evaluated by varying energy loss
corrections applied to the reconstructed momentum, the range and the shape of
the background function assumed in the fit of the mass-squared distributions
and the track selection criteria. In particular, TPC ${\rm d}E/{\rm d}x$ cuts
are varied between one and four standard deviations to probe the sensitivity
of the fit results on the residual background, and a tracking quality cut on
the distance of closest approach of the track to the vertex is varied to
evaluate the influence of secondary particles on the measurement. The sources
of systematic uncertainties are found to be fully correlated among all the
rigidity intervals, except for those due to the fit procedure and the TPC
selection criteria where the uncertainties are uncorrelated. For deuterons and
anti-deuterons, the largest relative systematic uncertainties on
$\Delta\mu/\mu$ come from the detector alignment ($\sim 0.7\times 10^{-4}$),
the TPC selection criteria ($\sim 0.7\times 10^{-4}$) and the secondaries
($\sim 1.0\times 10^{-4}$). For $\rm{{}^{3}He}$ and
$\rm{{}^{3}\overline{He}}$, they come from the energy loss corrections ($\sim
0.7\times 10^{-3}$), the fit procedure ($\sim 0.5\times 10^{-3}$) and the TPC
selection criteria ($\sim 0.4\times 10^{-3}$).

The (anti-)deuteron and (anti-)$\rm{{}^{3}He}$ masses are measured as the peak
position of the fitting curves of the mass-squared distribution. The mass-
over-charge ratio differences between the deuteron and $\rm{{}^{3}He}$ and
their respective anti-particle are then evaluated as a function of the
rigidity of the track, as shown in Fig. 2. The measurements in the individual
rigidity intervals are combined, taking into account statistical and
systematic uncertainties (correlated and uncorrelated), and the final result
is shown in the same figure with one and two standard deviation uncertainty
bands. The measured mass-over-charge ratio differences are

$\displaystyle\Delta\mu_{\rm{d\bar{d}}}=\rm{[1.7\pm 0.9(stat.)\pm
2.6(syst.)]\times 10^{-4}~{}GeV/{\it c}^{2}}\,,$ (1)
$\displaystyle\Delta\mu_{\rm{{}^{3}He^{3}\overline{He}}}=\rm{[-1.7\pm
1.2(stat.)\pm 1.4(syst.)]\times 10^{-3}~{}GeV/{\it c}^{2}}\,,$ (2)

corresponding to

$\displaystyle\frac{\Delta\mu_{\rm{d}\rm{\overline{d}}}}{\mu_{\rm{d}}}=[0.9\pm
0.5(\rm{stat.})\pm 1.4(syst.)]\times 10^{-4}\,,$ (3)
$\displaystyle\frac{\Delta\mu_{\rm{{}^{3}He}\rm{{}^{3}\overline{He}}}}{\mu_{\rm{{}^{3}He}}}=[-1.2\pm
0.9(\rm{stat.})\pm 1.0(\rm{syst.})]\times 10^{-3}\,,$ (4)

where $\mu_{\rm{d}}$ and $\mu_{\rm{{}^{3}He}}$ are the values recommended by
CODATA[25]. The mass-over-charge differences are compatible with zero within
the estimated uncertainties, in agreement with CPT invariance expectations.

Given that$z_{\bar{\rm{d}}}=-z_{\rm{d}}$ and
$z_{{}^{3}\overline{\rm{He}}}=-z_{{}^{3}\rm{He}}$ as for the proton and anti-
proton[1, 2], the mass-over-charge differences in Eq. 1 and Eq. 2 and the
measurement of the mass differences between proton and anti-proton[1, 2] and
between neutron and anti-neutron[15, 16] can be used to derive the relative
binding energy differences between the two studied particle species. We obtain

$\displaystyle\frac{\Delta\varepsilon_{\rm{d\bar{d}}}}{\varepsilon_{\rm{d}}}=-0.04\pm
0.05~{}(\rm{stat.})\pm 0.12~{}(\rm{syst.})\,,$ (5)
$\displaystyle\frac{\Delta\varepsilon_{\rm{{}^{3}He^{3}\overline{He}}}}{\varepsilon_{\rm{{}^{3}He}}}=0.24\pm
0.16~{}(\rm{stat.})\pm 0.18~{}(\rm{syst.})\,,$ (6)

where $\varepsilon_{A}=Zm_{\rm{p}}+(A-Z)m_{\rm{n}}-m_{A}$, being $m_{\rm{p}}$
and $m_{\rm{n}}$ the proton and the neutron mass values recommended by PDG[24]
and $m_{A}$ the mass value of the nucleus with atomic number $Z$ and mass
number $A$, recommended by CODATA[25]. This quantity allows one to explicitly
isolate possible violations of the CPT symmetry in the (anti-)nucleon
interaction from the one connected to the (anti-)nucleon masses, the latter
being constrained with a precision of $7\times 10^{-10}$ for the proton/anti-
proton system[1, 2]. Our results and the comparisons with previous mass
difference measurements for ($\rm{d}$-$\rm{\overline{d}}$)[26, 27] and
($\rm{{}^{3}He}$-$\rm{{}^{3}\overline{He}}$)[28], as well as binding energy
measurements for ($\rm{d}$-$\rm{\overline{d}}$)[29, 30] are reported in Fig.
3.

We have shown that the copious production of (anti-)nuclei in relativistic
heavy-ion collisions at the LHC represents a unique opportunity to test the
CPT invariance of nucleon-nucleon interaction using light nuclei. In
particular, we have measured the mass-over-charge ratio differences for
deuteron and $\rm{{}^{3}He}$. The values are compatible, within uncertainties,
with zero and represent a CPT invariance test in systems bound by nuclear
forces. The results reported here (Fig. 3, left) represent the highest
precision direct measurements of mass differences in the sector of nuclei and
they improve by one to two orders of magnitude analogous results originally
obtained more than 40 years ago[26, 27, 28], and precisely 50 years ago for
the anti-deuteron[26, 27]. Remarkably such an improvement is reached in an
experiment which is not specifically dedicated to test the CPT invariance in
nuclear systems. In the forthcoming years the increase in luminosity and
center-of-mass energy at the LHC will allow to push forward the sensitivity of
these measurements, and possibly to extend the study to
(anti-)${\rm{}^{4}He}$. Given the equivalence between mass and binding energy
differences, our results also improve (Fig. 3, right) by a factor two the
constraints on CPT invariance inferred by existing measurements[29, 30] in the
(anti-)deuteron system. The binding energy difference has been determined for
the first time in the case of (anti-)$\rm{{}^{3}He}$, with a relative
precision comparable to the one obtained in the (anti-)deuteron system.

Figure 1: Examples of squared mass-over-charge ratio distributions for
deuterons (left) and $\rm{{}^{3}He}$ (right) in selected rigidity intervals.
Particle and anti-particle spectra are in the top and bottom plots,
respectively. The fit function (red curve) also includes, for the
(anti-)deuteron case, an exponential term to describe the background. In the
rigidity intervals shown here the background is about 4% for (anti-)deuterons,
while it is 0.7% for $\rm{{}^{3}He}$ and $\rm{{}^{3}\overline{He}}$ . Figure
2: The $\rm{d}$-$\rm{\overline{d}}$ (top) and
$\rm{{}^{3}He}$-$\rm{{}^{3}\overline{He}}$ (bottom) mass-over-charge ratio
difference measurements as a function of the particle rigidity. Vertical bars
and open boxes show the statistical and the uncorrelated systematic
uncertainties (standard deviations), respectively. Both are taken into account
to extract the combined result in the full rigidity range, together with the
correlated systematic uncertainty, which is shown as a box with tilted lines.
Also shown are the $1\sigma$ and $2\sigma$ bands around the central value,
where $\sigma$ is the sum in quadrature of the statistical and systematic
uncertainties.  Figure 3: The ALICE measurements for
$\rm{d}$-$\rm{\overline{d}}$ and $\rm{{}^{3}He}$-$\rm{{}^{3}\overline{He}}$
mass-over-charge ratio differences compared with CPT invariance expectation
(dotted lines) and existing mass measurements MAS65[26], DOR65[27] and
ANT71[28] (left panel). The inset shows the ALICE results on a finer
$\Delta(m/z)/(m/z)$ scale. The right panel shows our determination of the
binding energy differences compared with direct measurements from DEN71[29]
and KES99[30]. Error bars represent the sum in quadrature of the statistical
and systematic uncertainties (standard deviations).

## Acknowledgements

The ALICE Collaboration would like to thank all its engineers and technicians
for their invaluable contributions to the construction of the experiment and
the CERN accelerator teams for the outstanding performance of the LHC complex.
The ALICE Collaboration gratefully acknowledges the resources and support
provided by all Grid centres and the Worldwide LHC Computing Grid (WLCG)
collaboration. The ALICE Collaboration acknowledges the following funding
agencies for their support in building and running the ALICE detector: State
Committee of Science, World Federation of Scientists (WFS) and Swiss Fonds
Kidagan, Armenia, Conselho Nacional de Desenvolvimento Científico e
Tecnológico (CNPq), Financiadora de Estudos e Projetos (FINEP), Fundação de
Amparo à Pesquisa do Estado de São Paulo (FAPESP); National Natural Science
Foundation of China (NSFC), the Chinese Ministry of Education (CMOE) and the
Ministry of Science and Technology of China (MSTC); Ministry of Education and
Youth of the Czech Republic; Danish Natural Science Research Council, the
Carlsberg Foundation and the Danish National Research Foundation; The European
Research Council under the European Community’s Seventh Framework Programme;
Helsinki Institute of Physics and the Academy of Finland; French CNRS-IN2P3,
the ‘Region Pays de Loire’, ‘Region Alsace’, ‘Region Auvergne’ and CEA,
France; German Bundesministerium fur Bildung, Wissenschaft, Forschung und
Technologie (BMBF) and the Helmholtz Association; General Secretariat for
Research and Technology, Ministry of Development, Greece; Hungarian Orszagos
Tudomanyos Kutatasi Alappgrammok (OTKA) and National Office for Research and
Technology (NKTH); Department of Atomic Energy and Department of Science and
Technology of the Government of India; Istituto Nazionale di Fisica Nucleare
(INFN) and Centro Fermi - Museo Storico della Fisica e Centro Studi e Ricerche
”Enrico Fermi”, Italy; MEXT Grant-in-Aid for Specially Promoted Research,
Japan; Joint Institute for Nuclear Research, Dubna; National Research
Foundation of Korea (NRF); Consejo Nacional de Cienca y Tecnologia (CONACYT),
Direccion General de Asuntos del Personal Academico(DGAPA), México, :Amerique
Latine Formation academique – European Commission(ALFA-EC) and the EPLANET
Program (European Particle Physics Latin American Network) Stichting voor
Fundamenteel Onderzoek der Materie (FOM) and the Nederlandse Organisatie voor
Wetenschappelijk Onderzoek (NWO), Netherlands; Research Council of Norway
(NFR); National Science Centre, Poland; Ministry of National
Education/Institute for Atomic Physics and Consiliul Naţional al Cercetării
Ştiinţifice - Executive Agency for Higher Education Research Development and
Innovation Funding (CNCS-UEFISCDI) - Romania; Ministry of Education and
Science of Russian Federation, Russian Academy of Sciences, Russian Federal
Agency of Atomic Energy, Russian Federal Agency for Science and Innovations
and The Russian Foundation for Basic Research; Ministry of Education of
Slovakia; Department of Science and Technology, South Africa; Centro de
Investigaciones Energeticas, Medioambientales y Tecnologicas (CIEMAT),
E-Infrastructure shared between Europe and Latin America (EELA), Ministerio de
Economía y Competitividad (MINECO) of Spain, Xunta de Galicia (Consellería de
Educación), Centro de Aplicaciones Tecnológicas y Desarrollo Nuclear (CEADEN),
Cubaenergía, Cuba, and IAEA (International Atomic Energy Agency); Swedish
Research Council (VR) and Knut $\&$ Alice Wallenberg Foundation (KAW); Ukraine
Ministry of Education and Science; United Kingdom Science and Technology
Facilities Council (STFC); The United States Department of Energy, the United
States National Science Foundation, the State of Texas, and the State of Ohio;
Ministry of Science, Education and Sports of Croatia and Unity through
Knowledge Fund, Croatia. Council of Scientific and Industrial Research (CSIR),
New Delhi, India

## References

  * [1] Hori, M. et al. Two-photon laser spectroscopy of antiprotonic helium and the antiproton-to-electron mass ratio. Nature 475, 484–488 (2011).
  * [2] Gabrielse, G. et al. Precision Mass Spectroscopy of the Antiproton and Proton Using Simultaneously Trapped Particles. Phys. Rev. Lett. 82, 3198–3201 (1999).
  * [3] van Kolck, U. Effective field theory of nuclear forces. Prog. Part. Nucl. Phys. 43, 337–418 (1999).
  * [4] Aamodt, K. et al. ALICE Collaboration. The ALICE experiment at the CERN LHC. JINST 3, S08002 (2008).
  * [5] Lüders, G. On the Equivalence of Invariance under Time Reversal and under Particle-Antiparticle Conjugation for Relativistic Field Theories. Kong. Dan. Vid. Sel. Mat. Fys. Med. 28N5, 1–17 (1954).
  * [6] Pauli, W. Exclusion Principle, Lorentz Group and Reflection of Space-Time and Charge. Niels Bohr and the Development of Physics, ed. by Pauli, W. (Pergamon Press, New York), 30–51 (1955).
  * [7] Agakishiev, H. et al. STAR Collaboration. Observation of the antimatter helium-4 nucleus. Nature 473, 353–356 (2011).
  * [8] STAR Collaboration (Harris J. W. for the collaboration). The STAR experiment at the relativistic heavy ion collider. Nucl. Phys. A 566, 277C–285C (1994).
  * [9] Gregory J. C. et al. PHENIX Collaboration. PHENIX Experiment at RHIC. Nucl. Phys. A 566, 287C–298C (1994).
  * [10] Fee, M. S. et al. Measurement of the positronium $1~{}^{3}S_{1}$–$2~{}^{3}S_{1}$ interval by continuous-wave two-photon excitation. Phys. Rev. A 48 192–219 (1993).
  * [11] Van Dyck, R. S. Jr., Schwinberg P. B. and Dehmelt H. G. New High-Precision Comparison of Electron and Positron g Factors. Phys. Rev. Lett. 59, 26–29 (1987).
  * [12] Abe, F. et al. CDF Collaboration. A Measurement of the $W$-boson mass. Phys. Rev. Lett. 65, 2243–2246 (1990).
  * [13] Amole, C. et al. Resonant quantum transitions in trapped antihydrogen atoms. Nature 483, 439–443 (2012).
  * [14] Amole, C. et al. An experimental limit on the charge of antihydrogen. Nature Commun. 5, 3955 (2014).
  * [15] Cresti, M., Pasquali, G., Peruzzo, L., Pinori, C. and Sartori, G. Measurement Of The Anti-neutron Mass. Phys. Lett. B 177, 206–210 (1986).
  * [16] Cresti, M., Pasquali, G., Peruzzo, L., Pinori, C. and Sartori, G. Errata. Phys. Lett. B 200, 587–588 (1988).
  * [17] Di Sciacca, J. et al. ATRAP Collaboration. One-Particle Measurement of the Antiproton Magnetic Moment. Phys. Rev. Lett. 110, 130801 (2013).
  * [18] Ambrosino, F. et al. KLOE Collaboration. Determination of CP and CPT violation parameters in the neutral kaon system using the Bell-Steinberger relation and data from the KLOE experiment. JHEP 0612, 011 (2006).
  * [19] Kostelecký, V. A., Russel, N. Data tables for Lorentz and CPT violation. Rev. Mod. Phys 83, 11–31 (2011).
  * [20] Abelev, B. I. et al. ALICE Collaboration. Performance of the ALICE experiment at the CERN LHC. Int. J. Mod. Phys. A 29, 1430044 (2014).
  * [21] Aamodt, K. et al. ALICE Collaboration. Alignment of the ALICE Inner Tracking System with cosmic-ray tracks. JINST 5, P03003 (2010).
  * [22] Alme, J. et al. The ALICE TPC, a large 3-dimensional tracking device with fast readout for ultra-high multiplicity events. Nucl. Instrum. Meth. A 622, 316–367 (2010).
  * [23] Akindinov, A. et al. Performance of the ALICE Time-Of-Flight detector at the LHC. Eur. Phys. J. Plus 128, 44 (2013).
  * [24] Olive, K. A. et al. Particle Data Group Collaboration. Review of Particle Physics. Chin. Phys. C 38, 090001 (2014).
  * [25] Mohr, P. J., Taylor, B. N. and Newell, D. B. CODATA Recommended Values of the Fundamental Physical Constants: 2010. Rev. Mod. Phys. 84, 1527–1605 (2012).
  * [26] Massam, T., Muller, Th., Righini, B., Schneegans, M. and Zichichi, A. Experimental observation of antideuteron production. Nuovo Cimento 39, 10–14 (1965).
  * [27] Dorfan, D. E., Eades, J., Lederman, L. M., Lee, W. and Ting, C. C. Observation of antideuterons. Phys. Rev. Lett. 14, 1003–1006 (1965).
  * [28] Antipov, Yu. M. et al. Observation of antihelium-3. Nucl. Phys. B 31, 235–252 (1971).
  * [29] Denisov, S. P. et al. Measurements of anti-deuteron absorption and stripping cross sections at the momentum 13.3 GeV/$c$. Nucl. Phys. B 31, 253–260 (1971).
  * [30] Kessler, E. G. Jr. et al. The Deuteron Binding Energy and the Neutron Mass. Phys. Lett. A 255, 221–229 (1999).

##  Appendix A The ALICE Collaboration

J. Adamorg39&D. Adamováorg82&M.M. Aggarwalorg86&G. Aglieri Rinellaorg36&M.
Agnelloorg110&N. Agrawalorg47&Z. Ahammedorg130&I. Ahmedorg16&S.U. Ahnorg67&I.
Aimoorg93,org110&S. Aiolaorg135&M. Ajazorg16&A. Akindinovorg57&S.N.
Alamorg130&D. Aleksandrovorg99,org99&B. Alessandroorg110&D. Alexandreorg101&R.
Alfaro Molinaorg63&A. Aliciorg104,org12&A. Alkinorg3&J. Almeorg37&T.
Altorg42&S. Altinpinarorg18,org18&I. Altsybeevorg129&C. Alves Garcia
Pradoorg118&C. Andreiorg77&A. Andronicorg96&V. Anguelovorg92&J.
Anielskiorg53&T. Antičićorg97&F. Antinoriorg107&P. Antonioliorg104&L.
Aphecetcheorg112&H. Appelshäuserorg52&S. Arcelliorg28&N. Armestoorg17&R.
Arnaldiorg110&T. Aronssonorg135&I.C. Arseneorg22&M. Arslandokorg52&A.
Augustinusorg36&R. Averbeckorg96&M.D. Azmiorg19,org19&M. Bachorg42&A.
Badalàorg106&Y.W. Baekorg43&S. Bagnascoorg110&R. Bailhacheorg52&R.
Balaorg89&A. Baldisseriorg15&M. Ballorg91&F. Baltasar Dos Santos
Pedrosaorg36&R.C. Baralorg60&A.M. Barbanoorg110&R. Barberaorg29&F.
Barileorg33&G.G. Barnaföldiorg134&L.S. Barnbyorg101&V. Barretorg69&P.
Bartaliniorg7&J. Bartkeorg115&E. Bartschorg52&M. Basileorg28&N. Bastidorg69&S.
Basuorg130&B. Bathenorg53&G. Batigneorg112&A. Batista Camejoorg69&B.
Batyunyaorg65&P.C. Batzingorg22&I.G. Beardenorg79&H. Beckorg52&C.
Beddaorg110&N.K. Beheraorg48,org47&I. Belikovorg54&F. Belliniorg28&H. Bello
Martinezorg2&R. Bellwiedorg120&R. Belmontorg133&E. Belmont-Morenoorg63&V.
Belyaevorg75&G. Bencediorg134&S. Beoleorg27&I. Berceanuorg77&A.
Bercuciorg77&Y. Berdnikovorg84&D. Berenyiorg134&R.A. Bertensorg56&D.
Berzanoorg36,org27&L. Betevorg36&A. Bhasinorg89&I.R. Bhatorg89&A.K.
Bhatiorg86&B. Bhattacharjeeorg44&J. Bhomorg126&L. Bianchiorg27,org120&N.
Bianchiorg71&C. Bianchinorg133,org56&J. Bielčíkorg39&J. Bielčíkováorg82&A.
Bilandzicorg79,org79&S. Biswasorg78,org78&S. Bjelogrlicorg56&F. Blancoorg10&D.
Blauorg99&C. Blumeorg52&F. Bockorg73,org92&A. Bogdanovorg75&H. Bøggildorg79&L.
Boldizsárorg134&M. Bombaraorg40&J. Bookorg52&H. Borelorg15&A. Borissovorg95&M.
Borriorg81&F. Bossúorg64&M. Botjeorg80&E. Bottaorg27&S. Böttgerorg51&P. Braun-
Munzingerorg96&M. Bregantorg118&T. Breitnerorg51&T.A. Brokerorg52&T.A.
Browningorg94&M. Brozorg39&E.J. Bruckenorg45,org45&E. Brunaorg110&G.E.
Brunoorg33&D. Budnikovorg98&H. Bueschingorg52&S. Bufalinoorg36,org110&P.
Buncicorg36&O. Buschorg92&Z. Butheleziorg64&J.T. Buxtonorg20&D.
Caffarriorg36,org30&X. Caiorg7&H. Cainesorg135&L. Calero Diazorg71&A.
Calivaorg56&E. Calvo Villarorg102&P. Cameriniorg26&F. Carenaorg36&W.
Carenaorg36&J. Castillo Castellanosorg15&A.J. Castroorg123&E.A.R.
Casulaorg25,org25&C. Cavicchioliorg36&C. Ceballos Sanchezorg9&J.
Cepilaorg39,org39&P. Cerelloorg110&B. Changorg121&S. Chapelandorg36&M.
Chartierorg122&J.L. Charvetorg15&S. Chattopadhyayorg130&S.
Chattopadhyayorg100&V. Chelnokovorg3&M. Cherneyorg85&C. Cheshkovorg128&B.
Cheynisorg128&V. Chibante Barrosoorg36&D.D. Chinellatoorg119&P.
Chochulaorg36&K. Choiorg95&M. Chojnackiorg79&S. Choudhuryorg130&P.
Christakoglouorg80&C.H. Christensenorg79&P. Christiansenorg34&T.
Chujoorg126&S.U. Chungorg95&C. Cicaloorg105&L. Cifarelliorg12,org28&F.
Cindoloorg104&J. Cleymansorg88&F. Colamariaorg33&D. Colellaorg33&A.
Colluorg25&M. Colocciorg28&G. Conesa Balbastreorg70&Z. Conesa del
Valleorg50&M.E. Connorsorg135&J.G. Contrerasorg39,org11&T.M. Cormierorg83&Y.
Corrales Moralesorg27&I. Cortés Maldonadoorg2&P. Corteseorg32&M.R.
Cosentinoorg118&F. Costaorg36&P. Crochetorg69&R. Cruz Albinoorg11&E.
Cuautleorg62&L. Cunqueiroorg36&T. Dahmsorg91&A. Daineseorg107&A. Danuorg61&D.
Dasorg100&I. Dasorg100,org50&S. Dasorg4&A. Dashorg119&S. Dashorg47&S.
Deorg130,org118&A. De Caroorg31,org12&G. de Cataldoorg103&J. de
Cuvelandorg42&A. De Falcoorg25&D. De Gruttolaorg12,org31&N. De Marcoorg110&S.
De Pasqualeorg31&A. Deistingorg96,org92&A. Delofforg76&E. Dénesorg134&G.
D’Erasmoorg33&D. Di Bariorg33&A. Di Mauroorg36&P. Di Nezzaorg71&M.A. Diaz
Corcheroorg10&T. Dietelorg88&P. Dillensegerorg52&R. Diviàorg36&Ø.
Djuvslandorg18&A. Dobrinorg56,org80&T. Dobrowolskiorg760&D. Domenicis
Gimenezorg118&B. Dönigusorg52&O. Dordicorg22&A.K. Dubeyorg130&A. Dublaorg56&L.
Ducrouxorg128&P. Dupieuxorg69&R.J. Ehlersorg135&D. Eliaorg103&H. Engelorg51&B.
Erazmusorg112,org36&F. Erhardtorg127&D. Eschweilerorg42&B. Espagnonorg50&M.
Estienneorg112&S. Esumiorg126&D. Evansorg101&S. Evdokimovorg111&G.
Eyyubovaorg39&L. Fabbiettiorg91&D. Fabrisorg107&J. Faivreorg70&A.
Fantoniorg71&M. Faselorg73&L. Feldkamporg53&D. Feleaorg61&A.
Felicielloorg110&G. Feofilovorg129&J. Ferenceiorg82&A. Fernández
Téllezorg2&E.G. Ferreiroorg17&A. Ferrettiorg27&A. Festantiorg30&J.
Figielorg115&M.A.S. Figueredoorg122&S. Filchaginorg98&D. Finogeevorg55&F.M.
Fiondaorg103&E.M. Fioreorg33&M.G. Fleckorg92&M. Florisorg36&S.
Foertschorg64&P. Fokaorg96&S. Fokinorg99&E. Fragiacomoorg109&A.
Francesconorg36,org30&U. Frankenfeldorg96&U. Fuchsorg36&C. Furgetorg70&A.
Fursorg55&M. Fusco Girardorg31&J.J. Gaardhøjeorg79&M. Gagliardiorg27&A.M.
Gagoorg102&M. Gallioorg27&D.R. Gangadharanorg73&P. Ganotiorg87&C. Gaoorg7&C.
Garabatosorg96&E. Garcia-Solisorg13&C. Gargiuloorg36&P. Gasikorg91&M.
Germainorg112&A. Gheataorg36&M. Gheataorg61,org36&P. Ghoshorg130&S.K.
Ghoshorg4&P. Gianottiorg71&P. Giubellinoorg36&P. Giubilatoorg30&E. Gladysz-
Dziadusorg115&P. Glässelorg92&A. Gomez Ramirezorg51&P. González-Zamoraorg10&S.
Gorbunovorg42&L. Görlichorg115&S. Gotovacorg114&V. Grabskiorg63&L.K.
Graczykowskiorg132&A. Grelliorg56&A. Grigorasorg36&C. Grigorasorg36&V.
Grigorievorg75&A. Grigoryanorg1&S. Grigoryanorg65&B. Grinyovorg3&N.
Grionorg109&J.F. Grosse-Oetringhausorg36&J.-Y. Grossiordorg128&R.
Grossoorg36&F. Guberorg55&R. Guernaneorg70&B. Guerzoniorg28&K.
Gulbrandsenorg79&H. Gulkanyanorg1&T. Gunjiorg125&A. Guptaorg89&R.
Guptaorg89&R. Haakeorg53&Ø. Haalandorg18&C. Hadjidakisorg50&M. Haiducorg61&H.
Hamagakiorg125&G. Hamarorg134&L.D. Hanrattyorg101&A. Hansenorg79&J.W.
Harrisorg135&H. Hartmannorg42&A. Hartonorg13&D. Hatzifotiadouorg104&S.
Hayashiorg125&S.T. Heckelorg52&M. Heideorg53&H. Helstruporg37&A.
Herghelegiuorg77&G. Herrera Corralorg11&B.A. Hessorg35&K.F. Hetlandorg37&T.E.
Hildenorg45&H. Hillemannsorg36&B. Hippolyteorg54&P. Hristovorg36&M.
Huangorg18&T.J. Humanicorg20&N. Hussainorg44&T. Hussainorg19&D.
Hutterorg42&D.S. Hwangorg21&R. Ilkaevorg98&I. Ilkivorg76&M. Inabaorg126&C.
Ionitaorg36&M. Ippolitovorg75,org99&M. Irfanorg19&M. Ivanovorg96&V.
Ivanovorg84&V. Izucheevorg111&P.M. Jacobsorg73&C. Jahnkeorg118&H.J.
Jangorg67&M.A. Janikorg132&P.H.S.Y. Jayarathnaorg120&C. Jenaorg30,org30&S.
Jenaorg120&R.T. Jimenez Bustamanteorg62&P.G. Jonesorg101&H. Jungorg43&A.
Juskoorg101&P. Kalinakorg58&A. Kalweitorg36&J. Kaminorg52&J.H. Kangorg136&V.
Kaplinorg75&S. Karorg130&A. Karasu Uysalorg68&O. Karavichevorg55&T.
Karavichevaorg55&E. Karpechevorg55&U. Kebschullorg51&R. Keidelorg137&D.L.D.
Keijdenerorg56&M. Keilorg36&K.H. Khanorg16&M.M. Khanorg19&P. Khanorg100&S.A.
Khanorg130&A. Khanzadeevorg84&Y. Kharlovorg111&B. Kilengorg37&B.
Kimorg136&D.W. Kimorg67,org43&D.J. Kimorg121&H. Kimorg136&J.S. Kimorg43&M.
Kimorg43&M. Kimorg136&S. Kimorg21&T. Kimorg136&S. Kirschorg42&I. Kiselorg42&S.
Kiselevorg57&A. Kisielorg132&G. Kissorg134&J.L. Klayorg6&C. Kleinorg52&J.
Kleinorg92&C. Klein-Bösingorg53&A. Klugeorg36&M.L. Knichelorg92,org92&A.G.
Knospeorg116&T. Kobayashiorg126&C. Kobdajorg113&M. Kofaragoorg36&M.K.
Köhlerorg96&T. Kolleggerorg96,org42&A. Kolojvariorg129&V. Kondratievorg129&N.
Kondratyevaorg75&E. Kondratyukorg111&A. Konevskikhorg55&C.
Kouzinopoulosorg36&V. Kovalenkoorg129&M. Kowalskiorg115,org36&S. Koxorg70&G.
Koyithatta Meethaleveeduorg47&J. Kralorg121&I. Králikorg58&A.
Kravčákováorg40&M. Krelinaorg39&M. Kretzorg42&M. Krivdaorg101,org58&F.
Krizekorg82&E. Kryshenorg36&M. Krzewickiorg42,org96&A.M. Kuberaorg20&V.
Kučeraorg82&Y. Kucheriaevorg990&T. Kugathasanorg36&C. Kuhnorg54&P.G.
Kuijerorg80&I. Kulakovorg42&J. Kumarorg47&L. Kumarorg78,org86&P.
Kurashviliorg76,org76&A. Kurepinorg55&A.B. Kurepinorg55&A. Kuryakinorg98&S.
Kushpilorg82&M.J. Kweonorg49&Y. Kwonorg136&S.L. La Pointeorg110&P. La
Roccaorg29&C. Lagana Fernandesorg118&I. Lakomovorg50,org36&R. Langoyorg41&C.
Laraorg51&A. Lardeuxorg15&A. Lattucaorg27&E. Laudiorg36&R. Leaorg26&L.
Leardiniorg92&G.R. Leeorg101&S. Leeorg136&I. Legrandorg36&J. Lehnertorg52&R.C.
Lemmonorg81&V. Lentiorg103&E. Leograndeorg56&I. León Monzónorg117&M.
Leoncinoorg27&P. Lévaiorg134&S. Liorg7,org69&X. Liorg14&J. Lienorg41&R.
Lietavaorg101&S. Lindalorg22&V. Lindenstruthorg42&C. Lippmannorg96&M.A.
Lisaorg20&H.M. Ljunggrenorg34&D.F. Lodatoorg56&P.I. Loenneorg18&V.R.
Logginsorg133&V. Loginovorg75&C. Loizidesorg73&X. Lopezorg69&E. López
Torresorg9&A. Loweorg134,org134&X.-G. Luorg92&P. Luettigorg52&M.
Lunardonorg30&G. Luparelloorg26,org56&A. Maevskayaorg55&M. Magerorg36&S.
Mahajanorg89&S.M. Mahmoodorg22&A. Maireorg54&R.D. Majkaorg135&M.
Malaevorg84&I. Maldonado Cervantesorg62&L. Malininaorg65&D. Mal’Kevichorg57&P.
Malzacherorg96&A. Mamonovorg98&L. Manceauorg110&V. Mankoorg99&F. Mansoorg69&V.
Manzariorg36,org103&M. Marchisoneorg27&J. Marešorg59&G.V. Margagliottiorg26&A.
Margottiorg104&J. Marguttiorg56&A. Marínorg96&C. Markertorg116&M.
Marquardorg52&I. Martashviliorg123&N.A. Martinorg96&J. Martin Blancoorg112&P.
Martinengoorg36&M.I. Martínezorg2&G. Martínez Garcíaorg112&M. Martinez
Pedreiraorg36&Y. Martynovorg3&A. Masorg118&S. Masciocchiorg96&M.
Maseraorg27&A. Masoniorg105&L. Massacrierorg112&A. Mastroserioorg33&A.
Matyjaorg115&C. Mayerorg115&J. Mazerorg123&M.A. Mazzoniorg108&D.
Mcdonaldorg120&F. Meddiorg24&A. Menchaca-Rochaorg63&E. Meninnoorg31&J. Mercado
Pérezorg92&M. Meresorg38&Y. Miakeorg126&M.M. Mieskolainenorg45&K.
Mikhaylovorg57,org65&L. Milanoorg36&J. Milosevicorg22,org131&L.M.
Minerviniorg103,org23&A. Mischkeorg56&A.N. Mishraorg48&D. Miśkowiecorg96&J.
Mitraorg130&C.M. Mituorg61&N. Mohammadiorg56&B. Mohantyorg130,org78&L.
Molnarorg54&L. Montaño Zetinaorg11&E. Montesorg10&M. Morandoorg30&D.A. Moreira
De Godoyorg112&S. Morettoorg30&A. Morrealeorg112&A. Morschorg36&V.
Mucciforaorg71&E. Mudnicorg114&D. Mühlheimorg53&S. Muhuriorg130&M.
Mukherjeeorg130&H. Müllerorg36&J.D. Mulliganorg135&M.G. Munhozorg118&S.
Murrayorg64&L. Musaorg36&J. Musinskyorg58&B.K. Nandiorg47&R. Naniaorg104&E.
Nappiorg103&M.U. Naruorg16&C. Nattrassorg123&K. Nayakorg78&T.K. Nayakorg130&S.
Nazarenkoorg98&A. Nedosekinorg57&L. Nellenorg62&F. Ngorg120&M.
Nicassioorg96,org96&M. Niculescuorg36,org61,org61&J. Niedzielaorg36&B.S.
Nielsenorg79&S. Nikolaevorg99&S. Nikulinorg99&V. Nikulinorg84&F.
Noferiniorg104,org12&P. Nomokonovorg65&G. Noorenorg56&J. Normanorg122&A.
Nyaninorg99&J. Nystrandorg18&H. Oeschlerorg92&S. Ohorg135&S.K. Ohorg66&A.
Ohlsonorg36&A. Okatanorg68&T. Okuboorg46&L. Olahorg134&J. Oleniaczorg132&A.C.
Oliveira Da Silvaorg118&M.H. Oliverorg135&J. Onderwaaterorg96&C.
Oppedisanoorg110&A. Ortiz Velasquezorg62&A. Oskarssonorg34&J.
Otwinowskiorg96,org115&K. Oyamaorg92&M. Ozdemirorg52&Y. Pachmayerorg92&P.
Paganoorg31&G. Paićorg62&C. Pajaresorg17&S.K. Palorg130&J. Panorg133&A.K.
Pandeyorg47&D. Pantorg47&V. Papikyanorg1&G.S. Pappalardoorg106&P.
Pareekorg48&W.J. Parkorg96,org96&S. Parmarorg86&A. Passfeldorg53&V.
Paticchioorg103&B. Paulorg100&T. Pawlakorg132&T. Peitzmannorg56&H. Pereira Da
Costaorg15&E. Pereira De Oliveira Filhoorg118&D. Peresunkoorg75,org99&C.E.
Pérez Laraorg80&V. Peskovorg52,org52&Y. Pestovorg5&V. Petráčekorg39&V.
Petrovorg111&M. Petroviciorg77&C. Pettaorg29&S. Pianoorg109&M. Piknaorg38&P.
Pillotorg112&O. Pinazzaorg104,org36&L. Pinskyorg120&D.B. Piyarathnaorg120&M.
Płoskońorg73&M. Planinicorg127&J. Plutaorg132&S. Pochybovaorg134&P.L.M.
Podesta-Lermaorg117,org117&M.G. Poghosyanorg85&B. Polichtchoukorg111&N.
Poljakorg127&W. Poonsawatorg113&A. Poporg77&S. Porteboeuf-Houssaisorg69&J.
Porterorg73&J. Pospisilorg82&S.K. Prasadorg4&R.
Preghenellaorg104,org104,org36&F. Prinoorg110&C.A. Pruneauorg133&I.
Pshenichnovorg55&M. Puccioorg110&G. Pudduorg25&P. Pujahariorg133&V.
Puninorg98&J. Putschkeorg133&H. Qvigstadorg22&A. Rachevskiorg109&S.
Rahaorg4&S. Rajputorg89&J. Rakorg121&A. Rakotozafindrabeorg15&L.
Ramelloorg32&R. Raniwalaorg90&S. Raniwalaorg90&S.S. Räsänenorg45&B.T.
Rascanuorg52&D. Ratheeorg86&V. Razaziorg25&K.F. Readorg123&J.S. Realorg70&K.
Redlichorg76&R.J. Reedorg133&A. Rehmanorg18&P. Reicheltorg52&M.
Reicherorg56&F. Reidtorg92,org36&X. Renorg7&R. Renfordtorg52&A.R.
Reolonorg71&A. Reshetinorg55&F. Rettigorg42&J.-P. Revolorg12&K.
Reygersorg92&V. Riabovorg84&R.A. Ricciorg72&T. Richertorg34&M.
Richterorg22,org22&P. Riedlerorg36&W. Rieglerorg36&F. Riggiorg29&C.
Risteaorg61&A. Rivettiorg110&E. Roccoorg56&M. Rodríguez
Cahuantziorg11,org2,org11&A. Rodriguez Mansoorg80&K. Røedorg22&E.
Rogochayaorg65&D. Rohrorg42&D. Röhrichorg18&R. Romitaorg122&F.
Ronchettiorg71&L. Ronfletteorg112&P. Rosnetorg69&A. Rossiorg36&F.
Roukoutakisorg87&A. Royorg48&C. Royorg54&P. Royorg100&A.J. Rubio
Monteroorg10&R. Ruiorg26&R. Russoorg27&E. Ryabinkinorg99&Y. Ryabovorg84&A.
Rybickiorg115&S. Sadovskyorg111&K. Šafaříkorg36&B. Sahlmullerorg52&P.
Sahooorg48&R. Sahooorg48&S. Sahooorg60&P.K. Sahuorg60&J. Sainiorg130&S.
Sakaiorg71&M.A. Salehorg133&C.A. Salgadoorg17&J. Salzwedelorg20&S.
Sambyalorg89&V. Samsonovorg84&X. Sanchez Castroorg54&L. Šándororg58&A.
Sandovalorg63&M. Sanoorg126&G. Santagatiorg29&D. Sarkarorg130&E.
Scapparoneorg104&F. Scarlassaraorg30&R.P. Scharenbergorg94&C. Schiauaorg77&R.
Schickerorg92&C. Schmidtorg96&H.R. Schmidtorg35&S. Schuchmannorg52&J.
Schukraftorg36&M. Schulcorg39&T. Schusterorg135&Y. Schutzorg112,org36&K.
Schwarzorg96&K. Schwedaorg96&G. Scioliorg28&E. Scomparinorg110&R.
Scottorg123&K.S. Seederorg118&J.E. Segerorg85&Y. Sekiguchiorg125&I.
Selyuzhenkovorg96,org96&K. Senosiorg64&J. Seoorg66,org95&E.
Serradillaorg10,org63&A. Sevcencoorg61&A. Shabanovorg55&A. Shabetaiorg112&O.
Shaduraorg3&R. Shahoyanorg36&A. Shangaraevorg111&A. Sharmaorg89&N.
Sharmaorg60,org123&K. Shigakiorg46&K. Shtejerorg9,org27&Y. Sibiriakorg99&S.
Siddhantaorg105&K.M. Sielewiczorg36&T. Siemiarczukorg76&D.
Silvermyrorg83,org34&C. Silvestreorg70&G. Simatovicorg127&G. Simonettiorg36&R.
Singarajuorg130&R. Singhorg89,org78&S. Singhaorg78,org130&V.
Singhalorg130&B.C. Sinhaorg130&T. Sinhaorg100&B. Sitarorg38&M. Sittaorg32&T.B.
Skaaliorg22&M. Slupeckiorg121&N. Smirnovorg135&R.J.M. Snellingsorg56&T.W.
Snellmanorg121&C. Søgaardorg34&R. Soltzorg74&J. Songorg95&M. Songorg136&Z.
Songorg7&F. Soramelorg30&S. Sorensenorg123&M. Spacekorg39&E. Spiritiorg71&I.
Sputowskaorg115&M. Spyropoulou-Stassinakiorg87&B.K. Srivastavaorg94&J.
Stachelorg92&I. Stanorg61&G. Stefanekorg76&M. Steinpreisorg20&E.
Stenlundorg34&G. Steynorg64&J.H. Stillerorg92&D. Stoccoorg112&P.
Strmenorg38&A.A.P. Suaideorg118&T. Sugitateorg46&C. Suireorg50&M.
Suleymanovorg16&R. Sultanovorg57&M. Šumberaorg82&T.J.M. Symonsorg73&A.
Szaboorg38&A. Szanto de Toledoorg1180&I. Szarkaorg38&A.
Szczepankiewiczorg36&M. Szymanskiorg132&J. Takahashiorg119&N.
Tanakaorg126&M.A. Tangaroorg33&J.D. Tapia Takakiidp5882640,org50&A. Tarantola
Peloniorg52&M. Tariqorg19&M.G. Tarzilaorg77&A. Tauroorg36&G. Tejeda
Muñozorg2&A. Telescaorg36&K. Terasakiorg125&C. Terrevoliorg30,org25&B.
Teyssierorg128&J. Thäderorg96,org73&D. Thomasorg56,org116&R.
Tieulentorg128&A.R. Timminsorg120&A. Toiaorg52&S. Trogoloorg110&V.
Trubnikovorg3&W.H. Trzaskaorg121&T. Tsujiorg125&A. Tumkinorg98&R.
Turrisiorg107&T.S. Tveterorg22&K. Ullalandorg18&A. Urasorg128,org128&G.L.
Usaiorg25&A. Utrobicicorg127&M. Vajzerorg82&M. Valaorg58&L. Valencia
Palomoorg69&S. Valleroorg27&J. Van Der Maarelorg56&J.W. Van Hoorneorg36&M. van
Leeuwenorg56&T. Vanatorg82&P. Vande Vyvreorg36&D. Vargaorg134&A. Vargasorg2&M.
Vargyasorg121&R. Varmaorg47&M. Vasileiouorg87&A. Vasilievorg99&A.
Vauthierorg70&V. Vecherninorg129&A.M. Veenorg56&M. Veldhoenorg56&A.
Velureorg18&M. Venaruzzoorg72&E. Vercellinorg27&S. Vergara Limónorg2&R.
Vernetorg8&M. Verweijorg133&L. Vickovicorg114&G. Viestiorg300&J.
Viinikainenorg121&Z. Vilakaziorg124&O. Villalobos Baillieorg101&A.
Vinogradovorg99&L. Vinogradovorg129&Y. Vinogradovorg98&T. Virgiliorg31&V.
Vislaviciusorg34&Y.P. Viyogiorg130&A. Vodopyanovorg65&M.A. Völklorg92&K.
Voloshinorg57&S.A. Voloshinorg133&G. Volpeorg36,org134&B. von Hallerorg36&I.
Vorobyevorg91&D. Vranicorg96,org36&J. Vrlákováorg40&B. Vulpescuorg69&A.
Vyushinorg98&B. Wagnerorg18&J. Wagnerorg96&H. Wangorg56&M. Wangorg7,org112&Y.
Wangorg92&D. Watanabeorg126&M. Weberorg36,org120&S.G. Weberorg96&J.P.
Wesselsorg53&U. Westerhofforg53&J. Wiechulaorg35&J. Wikneorg22&M.
Wildeorg53&G. Wilkorg76&J. Wilkinsonorg92&M.C.S. Williamsorg104&B.
Windelbandorg92&M. Winnorg92&C.G. Yaldoorg133&Y. Yamaguchiorg125&H.
Yangorg56,org56&P. Yangorg7&S. Yanoorg46&S. Yasnopolskiyorg99&Z. Yinorg7&H.
Yokoyamaorg126&I.-K. Yooorg95&V. Yurchenkoorg3&I. Yushmanovorg99&A.
Zaborowskaorg132&V. Zaccoloorg79&A. Zamanorg16&C. Zampolliorg104&H.J.C.
Zanoliorg118&S. Zaporozhetsorg65&A. Zarochentsevorg129&P. Závadaorg59&N.
Zaviyalovorg98&H. Zbroszczykorg132&I.S. Zguraorg61&M. Zhalovorg84&H.
Zhangorg7&X. Zhangorg73&Y. Zhangorg7&C. Zhaoorg22&N. Zhigarevaorg57&D.
Zhouorg7&Y. Zhouorg56&Z. Zhouorg18&H. Zhuorg7&J. Zhuorg7,org112&X. Zhuorg7&A.
Zichichiorg12,org28&A. Zimmermannorg92&M.B. Zimmermannorg53,org36&G.
Zinovjevorg3&M. Zyzakorg42

## Affiliation notes

0Deceased

idp5882640Also at: University of Kansas, Lawrence, Kansas, United States

## Collaboration Institutes

org1A.I. Alikhanyan National Science Laboratory (Yerevan Physics Institute)
Foundation, Yerevan, Armenia

org2Benemérita Universidad Autónoma de Puebla, Puebla, Mexico

org3Bogolyubov Institute for Theoretical Physics, Kiev, Ukraine

org4Bose Institute, Department of Physics and Centre for Astroparticle Physics
and Space Science (CAPSS), Kolkata, India

org5Budker Institute for Nuclear Physics, Novosibirsk, Russia

org6California Polytechnic State University, San Luis Obispo, California,
United States

org7Central China Normal University, Wuhan, China

org8Centre de Calcul de l’IN2P3, Villeurbanne, France

org9Centro de Aplicaciones Tecnológicas y Desarrollo Nuclear (CEADEN), Havana,
Cuba

org10Centro de Investigaciones Energéticas Medioambientales y Tecnológicas
(CIEMAT), Madrid, Spain

org11Centro de Investigación y de Estudios Avanzados (CINVESTAV), Mexico City
and Mérida, Mexico

org12Centro Fermi - Museo Storico della Fisica e Centro Studi e Ricerche
“Enrico Fermi”, Rome, Italy

org13Chicago State University, Chicago, Illinois, USA

org14China Institute of Atomic Energy, Beijing, China

org15Commissariat à l’Energie Atomique, IRFU, Saclay, France

org16COMSATS Institute of Information Technology (CIIT), Islamabad, Pakistan

org17Departamento de Física de Partículas and IGFAE, Universidad de Santiago
de Compostela, Santiago de Compostela, Spain

org18Department of Physics and Technology, University of Bergen, Bergen,
Norway

org19Department of Physics, Aligarh Muslim University, Aligarh, India

org20Department of Physics, Ohio State University, Columbus, Ohio, United
States

org21Department of Physics, Sejong University, Seoul, South Korea

org22Department of Physics, University of Oslo, Oslo, Norway

org23Dipartimento di Elettrotecnica ed Elettronica del Politecnico, Bari,
Italy

org24Dipartimento di Fisica dell’Università ’La Sapienza’ and Sezione INFN
Rome, Italy

org25Dipartimento di Fisica dell’Università and Sezione INFN, Cagliari, Italy

org26Dipartimento di Fisica dell’Università and Sezione INFN, Trieste, Italy

org27Dipartimento di Fisica dell’Università and Sezione INFN, Turin, Italy

org28Dipartimento di Fisica e Astronomia dell’Università and Sezione INFN,
Bologna, Italy

org29Dipartimento di Fisica e Astronomia dell’Università and Sezione INFN,
Catania, Italy

org30Dipartimento di Fisica e Astronomia dell’Università and Sezione INFN,
Padova, Italy

org31Dipartimento di Fisica ‘E.R. Caianiello’ dell’Università and Gruppo
Collegato INFN, Salerno, Italy

org32Dipartimento di Scienze e Innovazione Tecnologica dell’Università del
Piemonte Orientale and Gruppo Collegato INFN, Alessandria, Italy

org33Dipartimento Interateneo di Fisica ‘M. Merlin’ and Sezione INFN, Bari,
Italy

org34Division of Experimental High Energy Physics, University of Lund, Lund,
Sweden

org35Eberhard Karls Universität Tübingen, Tübingen, Germany

org36European Organization for Nuclear Research (CERN), Geneva, Switzerland

org37Faculty of Engineering, Bergen University College, Bergen, Norway

org38Faculty of Mathematics, Physics and Informatics, Comenius University,
Bratislava, Slovakia

org39Faculty of Nuclear Sciences and Physical Engineering, Czech Technical
University in Prague, Prague, Czech Republic

org40Faculty of Science, P.J. Šafárik University, Košice, Slovakia

org41Faculty of Technology, Buskerud and Vestfold University College,
Vestfold, Norway

org42Frankfurt Institute for Advanced Studies, Johann Wolfgang Goethe-
Universität Frankfurt, Frankfurt, Germany

org43Gangneung-Wonju National University, Gangneung, South Korea

org44Gauhati University, Department of Physics, Guwahati, India

org45Helsinki Institute of Physics (HIP), Helsinki, Finland

org46Hiroshima University, Hiroshima, Japan

org47Indian Institute of Technology Bombay (IIT), Mumbai, India

org48Indian Institute of Technology Indore, Indore (IITI), India

org49Inha University, Incheon, South Korea

org50Institut de Physique Nucléaire d’Orsay (IPNO), Université Paris-Sud,
CNRS-IN2P3, Orsay, France

org51Institut für Informatik, Johann Wolfgang Goethe-Universität Frankfurt,
Frankfurt, Germany

org52Institut für Kernphysik, Johann Wolfgang Goethe-Universität Frankfurt,
Frankfurt, Germany

org53Institut für Kernphysik, Westfälische Wilhelms-Universität Münster,
Münster, Germany

org54Institut Pluridisciplinaire Hubert Curien (IPHC), Université de
Strasbourg, CNRS-IN2P3, Strasbourg, France

org55Institute for Nuclear Research, Academy of Sciences, Moscow, Russia

org56Institute for Subatomic Physics of Utrecht University, Utrecht,
Netherlands

org57Institute for Theoretical and Experimental Physics, Moscow, Russia

org58Institute of Experimental Physics, Slovak Academy of Sciences, Košice,
Slovakia

org59Institute of Physics, Academy of Sciences of the Czech Republic, Prague,
Czech Republic

org60Institute of Physics, Bhubaneswar, India

org61Institute of Space Science (ISS), Bucharest, Romania

org62Instituto de Ciencias Nucleares, Universidad Nacional Autónoma de México,
Mexico City, Mexico

org63Instituto de Física, Universidad Nacional Autónoma de México, Mexico
City, Mexico

org64iThemba LABS, National Research Foundation, Somerset West, South Africa

org65Joint Institute for Nuclear Research (JINR), Dubna, Russia

org66Konkuk University, Seoul, South Korea

org67Korea Institute of Science and Technology Information, Daejeon, South
Korea

org68KTO Karatay University, Konya, Turkey

org69Laboratoire de Physique Corpusculaire (LPC), Clermont Université,
Université Blaise Pascal, CNRS–IN2P3, Clermont-Ferrand, France

org70Laboratoire de Physique Subatomique et de Cosmologie, Université
Grenoble-Alpes, CNRS-IN2P3, Grenoble, France

org71Laboratori Nazionali di Frascati, INFN, Frascati, Italy

org72Laboratori Nazionali di Legnaro, INFN, Legnaro, Italy

org73Lawrence Berkeley National Laboratory, Berkeley, California, United
States

org74Lawrence Livermore National Laboratory, Livermore, California, United
States

org75Moscow Engineering Physics Institute, Moscow, Russia

org76National Centre for Nuclear Studies, Warsaw, Poland

org77National Institute for Physics and Nuclear Engineering, Bucharest,
Romania

org78National Institute of Science Education and Research, Bhubaneswar, India

org79Niels Bohr Institute, University of Copenhagen, Copenhagen, Denmark

org80Nikhef, National Institute for Subatomic Physics, Amsterdam, Netherlands

org81Nuclear Physics Group, STFC Daresbury Laboratory, Daresbury, United
Kingdom

org82Nuclear Physics Institute, Academy of Sciences of the Czech Republic, Řež
u Prahy, Czech Republic

org83Oak Ridge National Laboratory, Oak Ridge, Tennessee, United States

org84Petersburg Nuclear Physics Institute, Gatchina, Russia

org85Physics Department, Creighton University, Omaha, Nebraska, United States

org86Physics Department, Panjab University, Chandigarh, India

org87Physics Department, University of Athens, Athens, Greece

org88Physics Department, University of Cape Town, Cape Town, South Africa

org89Physics Department, University of Jammu, Jammu, India

org90Physics Department, University of Rajasthan, Jaipur, India

org91Physik Department, Technische Universität München, Munich, Germany

org92Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg,
Heidelberg, Germany

org93Politecnico di Torino, Turin, Italy

org94Purdue University, West Lafayette, Indiana, United States

org95Pusan National University, Pusan, South Korea

org96Research Division and ExtreMe Matter Institute EMMI, GSI Helmholtzzentrum
für Schwerionenforschung, Darmstadt, Germany

org97Rudjer Bošković Institute, Zagreb, Croatia

org98Russian Federal Nuclear Center (VNIIEF), Sarov, Russia

org99Russian Research Centre Kurchatov Institute, Moscow, Russia

org100Saha Institute of Nuclear Physics, Kolkata, India

org101School of Physics and Astronomy, University of Birmingham, Birmingham,
United Kingdom

org102Sección Física, Departamento de Ciencias, Pontificia Universidad
Católica del Perú, Lima, Peru

org103Sezione INFN, Bari, Italy

org104Sezione INFN, Bologna, Italy

org105Sezione INFN, Cagliari, Italy

org106Sezione INFN, Catania, Italy

org107Sezione INFN, Padova, Italy

org108Sezione INFN, Rome, Italy

org109Sezione INFN, Trieste, Italy

org110Sezione INFN, Turin, Italy

org111SSC IHEP of NRC Kurchatov institute, Protvino, Russia

org112SUBATECH, Ecole des Mines de Nantes, Université de Nantes, CNRS-IN2P3,
Nantes, France

org113Suranaree University of Technology, Nakhon Ratchasima, Thailand

org114Technical University of Split FESB, Split, Croatia

org115The Henryk Niewodniczanski Institute of Nuclear Physics, Polish Academy
of Sciences, Cracow, Poland

org116The University of Texas at Austin, Physics Department, Austin, Texas,
USA

org117Universidad Autónoma de Sinaloa, Culiacán, Mexico

org118Universidade de São Paulo (USP), São Paulo, Brazil

org119Universidade Estadual de Campinas (UNICAMP), Campinas, Brazil

org120University of Houston, Houston, Texas, United States

org121University of Jyväskylä, Jyväskylä, Finland

org122University of Liverpool, Liverpool, United Kingdom

org123University of Tennessee, Knoxville, Tennessee, United States

org124University of the Witwatersrand, Johannesburg, South Africa

org125University of Tokyo, Tokyo, Japan

org126University of Tsukuba, Tsukuba, Japan

org127University of Zagreb, Zagreb, Croatia

org128Université de Lyon, Université Lyon 1, CNRS/IN2P3, IPN-Lyon,
Villeurbanne, France

org129V. Fock Institute for Physics, St. Petersburg State University, St.
Petersburg, Russia

org130Variable Energy Cyclotron Centre, Kolkata, India

org131Vinča Institute of Nuclear Sciences, Belgrade, Serbia

org132Warsaw University of Technology, Warsaw, Poland

org133Wayne State University, Detroit, Michigan, United States

org134Wigner Research Centre for Physics, Hungarian Academy of Sciences,
Budapest, Hungary

org135Yale University, New Haven, Connecticut, United States

org136Yonsei University, Seoul, South Korea

org137Zentrum für Technologietransfer und Telekommunikation (ZTT),
Fachhochschule Worms, Worms, Germany


(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext: Multimedia Laboratory, The Chinese University of Hong Kong
22institutetext: Centre for Perceptual and Interactive Intelligence (CPII)
33institutetext: Avolution AI 44institutetext: SenseTime
44email<EMAIL_ADDRESS>
# Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models

Xiaoshi Wu* 1133 Yiming Hao 22 Manyuan Zhang 11 Keqiang Sun 11
Zhaoyang Huang 33 Guanglu Song 44 Yu Liu 44 Hongsheng Li 1122

###### Abstract

Optimizing a text-to-image diffusion model with a given reward function is an
important but underexplored research area. In this study, we propose Deep
Reward Tuning (DRTune), an algorithm that directly supervises the final output
image of a text-to-image diffusion model and back-propagates through the
iterative sampling process to the input noise. We find that training earlier
steps in the sampling process is crucial for low-level rewards, and deep
supervision can be achieved efficiently and effectively by stopping the
gradient of the denoising network input. DRTune is extensively evaluated on
various reward models. It consistently outperforms other algorithms,
particularly for low-level control signals, where all shallow supervision
methods fail. Additionally, we fine-tune Stable Diffusion XL 1.0 (SDXL 1.0)
model via DRTune to optimize Human Preference Score v2.1, resulting in the
Favorable Diffusion XL 1.0 (FDXL 1.0) model. FDXL 1.0 significantly enhances
image quality compared to SDXL 1.0 and reaches comparable quality compared
with Midjourney v5.2. 111Authors with * contributed equally to this work.

Figure 1: Images generated by Favorable Diffusion XL 1.0 (FDXL 1.0). FDXL 1.0
is initialized from Stable Diffusion XL 1.0 (SDXL 1.0) and then trained to
optimize Human Preference Score v2.1 via the proposed Deep Reward Tuning.

##  1 Introduction

Diffusion models [30, 7] are generative models that sample data from a certain
distribution by iteratively denoising from random input, and have been proven
effective for image creation [30, 23, 25, 27, 20, 26, 43]. However, the
iterative denoising paradigm makes it less straightforward to train a
diffusion model with gradients from a reward model, compared with generative
adversarial networks (GANs) [5]. Training a diffusion model to optimize a
given reward is especially useful in cases where the training target can not
be easily characterized by a set of images, such as compressibility and
symmetry.

Inspired by Reinforcement Learning from Human Feedback (RLHF) [45, 32, 18, 1]
in natural language processing (NLP), previous works like ReFL [41], DRaFT
[3], and AlignProp [21] explore directly supervising the final output of a
diffusion model by a differentiable reward function. However, text-to-image
diffusion models typically require more than 20 sampling steps [31, 15, 16].
Directly back-propagating step-by-step from the output image to the input
noise results in significant memory consumption, as discussed in AlignProp
[21]. To avoid this issue, ReFL [41] and DRaFT [3] ignore earlier sampling
steps and only train the last few steps before the output image. However, we
find that this strategy is suboptimal, and it fails to optimize certain low-
level reward, such as rewards encouraging symmetry images. We call this a
depth-efficiency dilemma.

In this work, we propose DRTune as a solution to the depth-efficiency dilemma.
In DRTune, two main modifications are made comparing to the naive step-by-step
back-propagation approach: 1) we stop the gradients of the denoising network
input, and 2) in each training iteration, we sample a subset of equally spaced
steps among all $T$ steps in the sampling process for back-propagation. This
design has two advantages: 1) it skips back-propagation for untrained
intermediate steps, and enables the optimization of early denoising steps
without huge memory consumption and computation. 2) the stop gradient
operation solves the gradient explosion issue [3] when training early
timesteps, which significantly accelerates convergence.

We evaluate DRTune by comparing it with baseline methods on a benchmark of 7
rewards, and DRTune consistently achieves higher rewards given the same
computation budget, demonstrating the effectiveness of DRTune. Next, we fine-
tune Stable Diffusion XL 1.0 (SDXL 1.0) using HPS v2.1 [38], producing
Favorable Diffusion XL 1.0 (FDXL 1.0), which exhibits significantly better
visual quality compared to the base model SDXL 1.0 and even achieves
comparable quality with Midjourney v5.2.

The contributions of this work can be summarized as: 1) We propose DRTune for
efficiently and effectively supervising early denoising steps of a diffusion
model. 2) We introduce FDXL 1.0, the state-of-the-art open-source text-to-
image generative model tuned on human preferences.

##  2 Related Works

Reward-training for diffusion models. Optimizing text-to-image generative
models given a reward function is crucial for learning properties that are
difficult to define using a set of images, such as human preference [40, 41,
12], 3D consistency [33, 34, 19], and additional control [43]. Back-
propagating through the sampling process of diffusion models has been the
focus of several research studies. DiffusionCLIP [10] was the first to explore
fine-tuning a diffusion model for image manipulation. Watson _et al_. [37]
adopted a similar technique, but optimize sampler parameters. DOODL[36]
focused on optimizing the input noise to enhance classifier guidance. In
addition, Lee _et al_. [13], Wu _et al_. [39], DPOK [4], and DDPO [2] explored
fine-tuning text-to-image diffusion models using reinforcement learning to
optimize rewards. An advantage of these methods is that they do not require
rewards to be differentiable, which is beneficial when dealing with non-
differentiable reward functions. However, these approaches lead to slower
convergence compared to methods exploiting the gradients from rewards. Recent
works, such as DRaFT [3] and AlignProp [21], focuses on optimizing diffusion
models using differentiable rewards of human preference [12, 38], which
effectively improve the image quality. In this work, we identify that there is
an unsolved depth-efficiency dilemma in previous methods, and propose our
solution.

Rewards for text-to-image diffusion models. In fact, any perception model that
takes an image as input, and make a prediction can serve as a reward model.
Common reward models for tuning a text-to-image diffusion model are CLIP Score
for text-to-image alignment [22, 10, 3, 13], human preference [12, 38, 3, 39,
21], JPEG compressibility [2, 3]. In this work, we explore a new kind of
reward of symmetry. Although it is a relatively low-level reward compared with
other rewards, previous methods fail to optimize it. We identify that the key
to successfully optimize the symmetry reward is to adopt deep supervision.

Stop gradient for iterative refinement. Training a network with iterative
refinement is common in a variety of computer vision tasks. It typically
involves the formulation:

$x_{next}=x_{prev}+R(x_{prev}),$ (1)

where $x_{\text{prev}}$ represents the current output and $R$ is a network
that refines the current output by predicting a residue. This approach is
utilized in two-stage object detectors such as Faster R-CNN [24], where the
box coordinates predicted in the first stage are detached from gradients of
the second stage refinement. Similarly, state-of-the-art transformer-based
detectors like Deformable DETR [44] and DINO [42] employ intermediate
supervision for each decoder output. Optical flow estimation models like RAFT
[35] also iteratively refine a warping field following the same formulation.
In these cases, the inputs of $R$ are all detached. Given that diffusion
models also adhere to this formulation during the inference process, the stop
gradient technique may prove effective for optimizing them.

##  3 Methods

In this study, we focus on supervising a text-to-image diffusion model using a
differentiable reward model. We begin by presenting the background in Sec.3.1,
and then introduce our method, Deep Reward Tuning, in Sec. 3.2.

Figure 2: Illustration of the iterative denoising sampling pipeline. When
trained, gradients flow through the green branch and the red branch.

###  3.1 Background

Sampling of diffusion models. The sampling process of a diffusion model is
conducted as an iterative denoising procedure. Since this study does not focus
on a specific sampling algorithm, we adopt a notation for an abstracted
sampler:

$\mathbf{x}_{t-1}=a_{t}\mathbf{x}_{t}+b_{t}\epsilon_{\theta}(\mathbf{x}_{t},t)+c_{t}\epsilon.$
(2)

At each time step $t$, the denoising model $\epsilon_{\theta}$ estimates the
noise based on the current noisy input $\mathbf{x}_{t}$ and $t$ to predict the
direction towards $\mathbf{x}_{0}$. Extra conditions for $\epsilon_{\theta}$
are omitted without loss of generality. $\epsilon\sim\mathcal{N}(0,1)$
represents Gaussian noise with a mean of 0 and a variance of 1, introducing
randomness into the sampling process. The coefficients $a_{t},b_{t},c_{t}$ are
determined by sampling algorithms and noise schedules. Popular schedulers,
such as DDPM [7], DDIM [31], DPM [15, 16], can be parameterized into this
framework. In Fig. 2, the sampling process is unfolded for better
illustration.

Challenges. When fine-tuning a diffusion model with gradient on the output
image $\mathbf{\hat{x}}_{0}$, a dilemma arises regarding whether the earlier
sampling steps should be trained. Tuning the earlier sampling steps requires
back-propagation through the later steps, leading to a significant computation
overhead. A previous work [3] has shown that tuning earlier sampling steps can
also cause the gradient explosion problem, which hinders convergence. However,
given that every single sampling step contributes to the final output,
training only the last few steps may not suffice.

###  3.2 Deep Reward Tuning

To address the above challenge, we propose DRTune, an algorithm that can
supervise early sampling steps efficiently. The key idea of DRTune is to block
the gradients of denoising network inputs and train a subset of all sampling
steps.

Stop gradient. In DRTune, we address the convergence issue by blocking the
gradient of the input $\mathbf{x}_{t}$:

${\color[rgb]{0,1,0}\mathbf{x}_{t-1}}=a_{t}{\color[rgb]{0,1,0}\mathbf{x}_{t}}+b_{t}\epsilon_{\theta}({\color[rgb]{1,0,0}sg}(\mathbf{x}_{t}),t)+c_{t}\epsilon,$
(3)

where $sg(\cdot)$ denotes the stop gradient operation. With this modification,
the gradient will only flow through the green linear branch, and the gradients
of early steps can be easily computed by multiplying a scalar. We observe that
this operation helps alleviate the gradient explosion issue and accelerates
convergence.

Training a subset of sampling steps. After blocking the gradient on the input
$\mathbf{x}_{t}$, the gradients between neighboring time steps satisfy

$\partial\mathbf{x}_{t-1}=a_{t}\partial\mathbf{x}_{t}.$ (4)

Consequently, the gradient of $\mathbf{x}_{t}$ can be calculated as

$\partial\mathbf{x}_{t}=\prod_{s=1}^{t}a_{s}^{-1}\partial\mathbf{x}_{0}.$ (5)

This implies that each sampling step can be independently optimized given
$\partial\mathbf{x}_{0}$, allowing us to sample a subset of sampling steps for
training. For each optimization step, we sample $K$ equally spaced sampling
steps
$T_{train}:=\\{t_{s},t_{s}+\lfloor\frac{T}{K}\rfloor,...,t_{s}+K\lfloor\frac{T}{K}\rfloor\\}$,
where $K$ is the number of training steps, and $t_{s}$ is a random start
timestep that ensures $T_{train}\subseteq{1,...,T}$. Only the sampling steps
in $T_{train}$ are trained, which reduces the computation and memory overhead
and leads to faster convergence. We provide the pseudocode of DRTune in
Algorithm 1 and highlight the differences between our approach and relevant
algorithms.

Algorithm 1 DRTune

1:Input: pre-trained diffusion model weights $\theta$, reward $r$, number of
training timesteps $K$, range of early stop timestep $m$. $sg$ stands for the
stop gradient operation.

2:while not converged do

3:
$t_{train}=\left\\{\begin{array}[]{ll}\\{1,...,\mathrm{K}\\}&\mathbf{if}\quad\text{DRaFT-$K$}\\\
\\{i\\}_{i\geq randint(1,T)}&\mathbf{if}\quad\text{AlignProp}\\\
\end{array}\right.$

4: if DRTune then

5: # Equally spaced timesteps.

6: $s=randint(1,T-K\lfloor\frac{T}{K}\rfloor)$

7: $t_{train}=\\{s+i\lfloor\frac{T}{K}\rfloor\mid i=0,1,...,K-1\\}$

8: if ReFL or DRTune then

9: $t_{min}=randint(1,m)$

10: else

11: $t_{min}=0$

12: $\mathbf{x}_{T}\sim\mathcal{N}(0,\mathbf{I})$

13: for $t=T,...,1$ do

14: if DRTune then

15:
$\hat{\epsilon}=\epsilon_{\theta}({\color[rgb]{0,0,1}sg(\mathbf{x}_{t})},t)$

16: else

17: $\hat{\epsilon}=\epsilon_{\theta}(\mathbf{x}_{t},t)$

18: if $t\notin t_{train}$ then

19: $\hat{\epsilon}=sg(\hat{\epsilon})$

20: if $t==t_{min}$ then

21: $\mathbf{x}_{0}\approx
intermediate\\_prediction(\mathbf{x}_{t},\hat{\epsilon})$

22: break

23: $\mathbf{x}_{t-1}=a_{t}\mathbf{x}_{t}+b_{t}\hat{\epsilon}+c_{t}\epsilon$

24: $\mathbf{g}=\nabla_{\theta}r(\mathbf{x}_{0},c)$

25: $\theta\leftarrow\theta-\eta\mathbf{g}$

Figure 3: Comparison between reward training algorithms.

Comparison with other algorithms. The comparison between algorithms is shown
in Fig. 3. The key difference between DRTune and other algorithms lies in the
stop gradient operation. We will demonstrate that other algorithms can also
benefit from this simple technique in the experiment section. While all the
aforementioned algorithms train a subset of steps, these steps are sampled
differently. In ReFL [41], the step is randomly sampled near the output. The
denoising loop terminates at this step, and then uses the intermediate result
at that step for supervision. DRaFT-$K$ [3] only trains the last $K$ steps
near the output image, without involving earlier sampling steps. AlignProp
[21] trains a random subset of sampling steps, with a higher probability for
steps near the output image and a lower probability for steps near the input
noise. DRTune also trains a subset of $K$ steps, but the selected steps are
equally spaced. We find that this design results in faster convergence than
randomly sampling $K$ steps for training. We also adopt the early stop design
of ReFL, which accelerates sampling, and leads to faster convergence.

##  4 Experiments

We present a comparison between DRTune and other reward-training methods in
Sec.4.1. Following that, we analyze the design choices of DRTune through
ablation experiments in Sec.4.2. Finally, we demonstrate the effectiveness of
DRTune in tuning the larger model of SDXL 1.0 in Sec.4.3.

###  4.1 Comparison with baseline methods

We experiment with 7 different reward functions in our study: Aesthetic Score
[17, 29], CLIPScore [22, 9], PickScore [12], HPS v2.1 [38], symmetry,
compressibility, and objectness.

Aesthetic Score is evaluated using an aesthetic score predictor [29], which is
a single-modal reward model that takes an image as input and predicts a score
from 1 to 10, evaluating its aesthetic quality. This predictor is trained on a
scoring dataset [17] consisting of real images.

CLIPScore [22, 9] is defined as the cosine similarity between the text
embedding of the input prompt and the image embedding of the output image. We
use the ViT-H-14 variant of the CLIP model for evaluation.

PickScore [12], HPS v2.1 [38] are models trained to capture human preference
for images based on input prompts. Both models are fine-tuned on top of the
CLIP model [22] but with different data.

Symmetry is a low-level reward that is defined in the pixel space. The reward
function for symmetry is given by:

$r_{symmetry}(I)=\frac{||I-\text{flip}(I)||_{1}}{\text{std}(I)},$ (6)

where $\text{flip}()$ applies a mirror flip to the input image, and
$\text{std}()$ computes the standard deviation of pixel values in the image,
which helps prevent the image from degenerating into a solid color image. To
maintain the controllability of text prompts, we also incorporate the CLIP
Score [22, 9] as an additional regularization term.

Compressibility. Inspired by [2, 3], we use the reconstruction error of images
before and after JPEG compression to measure the compressibility of an image,
where the error is defined as:

$e_{compress}(I)=||I-d(c(I))||^{2}_{2},$ (7)

where $d$ and $c$ are implemented as differentiable operators.

Objectness. Inspired by [21], we minimize the objectness score of a type of
object in an image to achieve object removal. We use OWL-ViT as the object
detector, and try to remove “books” in images generated by prompts in the form
of “[concept name], and books.”. In practice, we minimize the maximum
objectness of class “book” in all the box predictions.

Experiment setup. To ensure a fair comparison, we adopt a unified
hyperparameter setting for all baselines. However, since each method has
different GPU consumption, comparing them based on the same training steps or
reward query count would be unfair for faster methods. Instead, we evaluate
the methods using the same computation budget. Specifically, we train all
methods for 2 hours on 4 A100 GPUs. The final reward is computed on the 400
prompts from the HPS v2 benchmark [38]. Since this work focuses on optimizing
a given reward, common evaluation metrics for image creation, like FiD [6] and
IS [28] are not reported.

Implementation details. We use Stable Diffusion 1.5 [25] as the base model for
our experiments. We employ a DDPM [7] sampler to perform 50 steps of sampling,
with a classifier-free guidance scale of 7.5. The output resolution of the
generated images is set to $512\times 512$, which is then down-sampled to
$224\times 224$ for reward evaluation. During training, we sample prompts from
the training prompts provided by HPS v2 [38]. The models are trained with a
batch size of 32 and a constant learning rate of $2\times 10^{-5}$. We apply
gradient clipping of 0.1 to ensure stable training. The AdamW optimizer with
default hyperparameters ($\beta_{1}=0.9,\beta_{2}=0.999$) [11, 14] is used for
parameter optimization. To save memory during training, we train LoRA [8]
weights of rank 128 instead of the entire network. Additionally, we use half-
precision (fp16) for frozen parameters and single precision for trainable
parameters. Gradient checkpointing is also applied to further reduce memory
usage. It is worth noting that AlignProp [21] requires significantly more
memory for optimization. Therefore, in order to make a fair comparison with
other methods, we perform gradient accumulation of 2 steps in AlignProp. The
Stable Diffusion 1.5 model consists of three block modules: the VAE, the
U-Net, and the text encoder. In our training process, we only train the LoRA
parameters in the U-Net module.

Figure 4: Comparison between images generated with DRTune or other baseline
methods and symmetry supervision. Figure 5: Comparison between images
generated with Stable Diffusion tuned by DRTune or other baseline methods.
Supervised by PickScore.

Comparison between reward-training methods. In Tab. 1, we compare DRTune with
variants of each baseline. ReFL-10 and ReFL-20 [41] are variants of ReFL that
stops at different range of steps. ReFL runs faster than other baselines due
to its early stop design. DRaFT-LV trains only the last step, but increases
its efficiency by sampling the last step twice with different noise. DRaFT-10
and AlignProp train more denoising steps in each parameter update, but they
are considerably slower compared to the previous three baselines, hindering
their convergence. In contrast, DRTune samples all denoising steps without
early termination. It trains 5 denoising steps and achieves the highest
efficiency among the baselines. An important feature of DRTune is its ability
to successfully optimize the symmetry loss, which sets it apart from the other
algorithms. This success can be attributed to DRTune’s efficient supervision
of early denoising steps, which is better related to the global layout of an
image. In Fig. 4 and Fig. 5, we show qualitative comparison between methods on
the reward of PickScore and symmetry. Our method successfully learns the
symmetry property, while others fail. And with supervision on the early
denoising steps, models tuned with DRTune often shows better global layout and
more natural appearance than other methods.

Method | HPS v2.1$\uparrow$ | PickScore$\uparrow$ | Aesthetic Score$\uparrow$ | Symmetry$\downarrow$ | CLIPScore $\uparrow$ | Compression error $\downarrow$ | Objectness $\downarrow$
---|---|---|---|---|---|---|---
ReFL-10 [41] | 39.18 | 0.2488 | 9.585 | 0.7512 | 0.3771 | 0.0057 | 0.0029
ReFL-20 [41] | 39.43 | 0.2491 | 9.911 | 0.7328 | 0.3844 | 0.0043 | 0.0021
DRaFT-LV [3] | 39.02 | 0.2477 | 9.615 | 0.7648 | 0.3613 | 0.0088 | 0.1024
DRaFT-10 [3] | 37.62 | 0.2456 | 8.317 | 0.7250 | 0.3707 | 0.0073 | 0.0050
AlignProp [21] | 34.04 | 0.2297 | 6.627 | 0.7840 | 0.3580 | 0.0097 | 0.9266
DRTune (outs) | 40.63 | 0.2492 | 10.360 | 0.0551 | 0.3856 | 0.0038 | 0.0001

Table 1: Comparison with baseline methods. The diffusion model is trained to
maximize or minimize the given targets, indicated by top / down arrows. We
report averaged targets computed on prompts from the HPS v2 [38] benchmark
except for the objectness. Prompts for objectness are in the form of “[concept
name], and books”, following [21].

Figure 6: AlignProp [21] and DRaFT-$T$ [3] can also benefit from early stop.
Figure 7: The gradient norm of network parameters explodes as $K$ increases,
which can be mitigated by the stop gradient technique.  Figure 8: The impact
of the number of training steps $K$ vs. various total sampling steps $T$. Some
data points are missing due to GPU memory constraint.

###  4.2 Ablation studies

Stop gradient for deep supervision. Stop gradient is a general reward-training
technique that can greatly help the convergence when training diffusion models
with a reward. In Fig. 6, we present the convergence curves of DRaFT-$T$[3]
and AlignProp [21] trained with and without the proposed stop gradient
technique. Both algorithms involve supervision on early denoising steps.
DRaFT-$T$ is a variant of DRaFT-$K$ that trains all the $T$ denoising steps,
while AlignProp trains $\frac{T}{2}$ steps on average. DRaFT-$T$ fails to
optimize without the stop gradient technique. In the case of AlignProp, the
stop gradient technique significantly improves the training efficiency, as it
only requires roughly 50% of iterations to achieve a similar reward. The
effectiveness of the stop gradient technique can be explained by the gradient
norm. In Fig. 7, we plot the gradient norm of LoRA parameters of the U-Net
module. Without the stop gradient technique, the gradient norm rapidly
increases after $K\geq 15$. However, this issue is effectively resolved by
adopting the proposed stop gradient technique.

Choice of K. $K$ is the number of denoising steps to be trained in each
iteration. While a larger $K$ value requires more computational resources for
back-propagation, it may also help to acquire more accurate gradient
estimation. In Fig. 8, we conduct an ablation study on the selection of $K$ in
DRTune, considering different total sampling steps $T$ ranging from 20 to 50.
To ensure fair comparison, experiments with the same $T$ were conducted under
identical computational budget and hyperparameters. The final reward is
plotted against the ratio of $\frac{K}{T}$. The results demonstrate that the
optimal value of $K$ exhibits a linear relationship with $T$, and a ratio of
$0.1T$ turns out to be a favorable choice.

Figure 9: The impact of maximal early stop step $m$ vs. various total sampling
steps $T$.

Choice of m. $m$ controls early stop, which is the maximal number of steps to
skip. In Fig. 9, we ablate the choice of $m$. The result shows that the best
choice of $m$ falls on the ratio of $0.4T$.

Figure 10: Qualitative comparison between SDXL, Midjourney and FDXL.

###  4.3 Training SDXL 1.0

Training details. We fine-tune the SDXL 1.0 model to optimize HPS v2.1 using
DRTune, resulting in Favorable Diffusion XL 1.0 (FDXL 1.0). Given SDXL’s
larger size compared to Stable Diffusion 1.5, we utilize the DPM++[16] sampler
for improved sampling efficiency. We adjust the sampling steps $T$ to 25 and
the training step $K$ to 3, maintaining a classifier-free guidance scale of
5.0 and a resolution of 1024 to align with SDXL 1.0’s default setting. We
train the model using a batch size of 2 and a gradient accumulation step of 4
on 8 A100 GPUs with 80G memory for 1,900 iterations. We lower the learning
rate to $5\times 10^{-6}$ for training stability. To address the discrepancy
between the input resolution of the reward model and the output image
resolution of SDXL, we apply random shifting within the range of
$\\{0,...,\lceil\frac{r_{\text{output}}}{r_{\text{input}}}\rceil\\}$ pixels to
the output image, randomizing the down-sampling pixel locations. We set the
LoRA weight to 0.7 during inference. All other settings remain consistent with
those outlined in Sec. 4.1.

(a) User study comparing images generated by SDXL 1.0 and FDXL 1.0.

(b) User study comparing images generated by Midjourney and FDXL 1.0.

###  4.4 Results

We compare images generated by FDXL 1.0 with SDXL 1.0 and the commercial
generative model Midjourney. In Fig.10, we present the visual comparison
between images generated by SDXL, Midjourney, and FDXL. FDXL produces higher-
quality and more contextually consistent images than SDXL. For quantitative
comparison, we conduct user studies on 400 randomly sampled prompts from the
HPS v2[38] benchmark, with 100 prompts from each category of “Photo”,
“Paintings”, “Concept-Art”, and “Anime”. Each prompt is evaluated by 3
participants, and the results are averaged for visualization, as depicted in
Fig.11(a), FDXL significantly outperforms SDXL, with an average winning rate
of 69%. In Fig.11(b), we illustrate the comparison between FDXL and
Midjourney, showing an average winning rate of 37% compared to Midjourney,
with the “paintings” category approaching a draw. However, we acknowledge that
the realism of images generated by FDXL still lags behind Midjourney, and the
winning rate over SDXL on prompts of “photo” category is also the lowest among
others. AS this is potentially due to bias in the reward model, the challenge
of leveraging a preference model while mitigating its bias still remains an
open question.

##  5 Limitations

In this work, we primarily focus on optimizing a given reward, but we find in
our experiments that rewards that are computed from a neural network are very
likely to suffer from the reward-hacking problem. For example, an image
generated by an over-optimized model can have a very high aesthetic score, but
the visual quality may degenerate. When training FDXL, we avoid the issue by
early stop. It is also meaningful to explore other strategies like
regularization.

Generative models potentially have negative social impacts by their nature.
The advancement in generative models often means more plausible generated
content, which may be maliciously utilized for the spread of more convincing
misinformation and fake content. Additionally, there is a risk of amplifying
existing biases and stereotypes present in the training data.

##  6 Conclusion

In this study, we address the challenge of training text-to-image diffusion
models using feedback from a reward model. We emphasize the importance of deep
supervision for optimizing global rewards and resolve convergence issues using
a stop gradient technique. Additionally, we demonstrate the potential of
reward training by fine-tuning the FDXL 1.0 model to achieve image quality
comparable to Midjourney.

## References

  * [1] Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al.: Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022)
  * [2] Black, K., Janner, M., Du, Y., Kostrikov, I., Levine, S.: Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301 (2023)
  * [3] Clark, K., Vicol, P., Swersky, K., Fleet, D.J.: Directly Fine-Tuning Diffusion Models on Differentiable Rewards. arXiv preprint arXiv:2309.17400 (2023)
  * [4] Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, M., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Lee, K., Lee, K.: DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models. arXiv preprint arXiv:2305.16381 (2023)
  * [5] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Communications of the ACM 63(11), 139–144 (2020)
  * [6] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In: NeurIPS (2017)
  * [7] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. NeurIPS 33, 6840–6851 (2020)
  * [8] Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)
  * [9] Ilharco, G., Wortsman, M., Wightman, R., Gordon, C., Carlini, N., Taori, R., Dave, A., Shankar, V., Namkoong, H., Miller, J., Hajishirzi, H., Farhadi, A., Schmidt, L.: Openclip (Jul 2021). https://doi.org/10.5281/zenodo.5143773, https://doi.org/10.5281/zenodo.5143773, if you use this software, please cite it as below.
  * [10] Kim, G., Kwon, T., Ye, J.C.: Diffusionclip: Text-guided diffusion models for robust image manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2426–2435 (2022)
  * [11] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [12] Kirstain, Y., Polyak, A., Singer, U., Matiana, S., Penna, J., Levy, O.: Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation. arXiv preprint arXiv:2305.01569 (2023)
  * [13] Lee, K., Liu, H., Ryu, M., Watkins, O., Du, Y., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Gu, S.S.: Aligning text-to-image models using human feedback. arXiv preprint arXiv:2302.12192 (2023)
  * [14] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
  * [15] Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., Zhu, J.: Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems 35, 5775–5787 (2022)
  * [16] Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., Zhu, J.: Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095 (2022)
  * [17] Murray, N., Marchesotti, L., Perronnin, F.: AVA: A large-scale database for aesthetic visual analysis. CVPR pp. 2408–2415 (2012)
  * [18] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. NeurIPS 35, 27730–27744 (2022)
  * [19] Piao, J., Sun, K., Wang, Q., Lin, K.Y., Li, H.: Inverting generative adversarial renderer for face reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15619–15628 (2021)
  * [20] Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023)
  * [21] Prabhudesai, M., Goyal, A., Pathak, D., Fragkiadaki, K.: Aligning Text-to-Image Diffusion Models with Reward Backpropagation. arXiv preprint arXiv:2310.03739 (2023)
  * [22] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning Transferable Visual Models From Natural Language Supervision. In: ICML (2021)
  * [23] Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical Text-Conditional Image Generation with CLIP Latents. ArXiv abs/2204.06125 (2022)
  * [24] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)
  * [25] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-Resolution Image Synthesis with Latent Diffusion Models. CVPR pp. 10674–10685 (2022)
  * [26] Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22500–22510 (2023)
  * [27] Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al.: Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS 35, 36479–36494 (2022)
  * [28] Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. Advances in neural information processing systems 29 (2016)
  * [29] Schuhmann, C.: CLIP+MLP Aesthetic Score Predictor. https://github.com/christophschuhmann/improved-aesthetic-predictor (2022)
  * [30] Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: International conference on machine learning. pp. 2256–2265. PMLR (2015)
  * [31] Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)
  * [32] Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., Voss, C., Radford, A., Amodei, D., Christiano, P.F.: Learning to summarize with human feedback. Advances in Neural Information Processing Systems 33, 3008–3021 (2020)
  * [33] Sun, K., Wu, S., Huang, Z., Zhang, N., Wang, Q., Li, H.: Controllable 3d face synthesis with conditional generative occupancy fields. Advances in Neural Information Processing Systems 35, 16331–16343 (2022)
  * [34] Sun, K., Wu, S., Zhang, N., Huang, Z., Wang, Q., Li, H.: Cgof++: Controllable 3d face synthesis with conditional generative occupancy fields. IEEE transactions on pattern analysis and machine intelligence (2023)
  * [35] Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. pp. 402–419. Springer (2020)
  * [36] Wallace, B., Gokul, A., Ermon, S., Naik, N.: End-to-end diffusion latent optimization improves classifier guidance. arXiv preprint arXiv:2303.13703 (2023)
  * [37] Watson, D., Chan, W., Ho, J., Norouzi, M.: Learning fast samplers for diffusion models by differentiating through sample quality. In: International Conference on Learning Representations (2021)
  * [38] Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., Li, H.: Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 (2023)
  * [39] Wu, X., Sun, K., Zhu, F., Zhao, R., Li, H.: Better aligning text-to-image models with human preference. arXiv preprint arXiv:2303.14420 (2023)
  * [40] Wu, X., Sun, K., Zhu, F., Zhao, R., Li, H.: Human Preference Score: Better Aligning Text-to-Image Models with Human Preference. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2096–2105 (2023)
  * [41] Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation (2023)
  * [42] Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L.M., Shum, H.Y.: Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605 (2022)
  * [43] Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3836–3847 (2023)
  * [44] Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)
  * [45] Ziegler, D.M., Stiennon, N., Wu, J., Brown, T.B., Radford, A., Amodei, D., Christiano, P., Irving, G.: Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019)