|  Daniel Benevides da Costa (S’04-M’08-SM’14) was born in Fortaleza, Ceará,
Brazil, in 1981. He received the B.Sc. degree in Telecommunications from the
Military Institute of Engineering (IME), Rio de Janeiro, Brazil, in 2003, and
the M.Sc. and Ph.D. degrees in Electrical Engineering, Area:
Telecommunications, from the University of Campinas, SP, Brazil, in 2006 and
2008, respectively. His Ph.D. thesis was awarded the Best Ph.D. Thesis in
Electrical Engineering by the Brazilian Ministry of Education (CAPES) at the
2009 CAPES Thesis Contest. From 2008 to 2009, he was a Postdoctoral Research
Fellow with INRS-EMT, University of Quebec, Montreal, QC, Canada. From 2010 to
2022, he was with the Federal University of Ceará, Brazil. From January 2019
to April 2019, he was Visiting Professor at Lappeenranta University of
Technology (LUT), Finland, with financial support from Nokia Foundation. He
was awarded with the prestigious Nokia Visiting Professor Grant. From May 2019
to August 2019, he was with King Abdullah University of Science and Technology
(KAUST), Saudi Arabia, as a Visiting Faculty, and from September 2019 to
November 2019, he was a Visiting Researcher at Istanbul Medipol University,
Turkey. From 2021 to 2022, he was Full Professor at the National Yunlin
University of Science and Technology (YunTech), Taiwan. Since 2022, he is
Principal Researcher of the AI and Digital Science Research Center at the
Technology Innovation Institute (TII), a global research center and the
applied pillar of Abu Dhabi’s Advanced Technology Research Council. He is
Editor of several IEEE journals and has acted as Symposium/Track Co-Chair in
numerous IEEE flagship conferences.
---|---
|  Petar Popovski (S’97–A’98–M’04–SM’10–F’16) is a Professor at Aalborg
University, where he heads the section on Connectivity and a Visiting
Excellence Chair at the University of Bremen. He received his Dipl.-Ing and M.
Sc. degrees in communication engineering from the University of Sts. Cyril and
Methodius in Skopje and the Ph.D. degree from Aalborg University in 2005. He
received an ERC Consolidator Grant (2015), the Danish Elite Researcher award
(2016), IEEE Fred W. Ellersick prize (2016), IEEE Stephen O. Rice prize
(2018), Technical Achievement Award from the IEEE Technical Committee on Smart
Grid Communications (2019), the Danish Telecommunication Prize (2020) and
Villum Investigator Grant (2021). He was a Member at Large at the Board of
Governors in IEEE Communication Society 2019-2021. He is currently an Editor-
in-Chief of IEEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS. He also serves
as a Vice-Chair of the IEEE Communication Theory Technical Committee and the
Steering Committee of IEEE TRANSACTIONS ON GREEN COMMUNICATIONS AND
NETWORKING. Prof. Popovski was the General Chair for IEEE SmartGridComm 2018
and IEEE Communication Theory Workshop 2019. His research interests are in the
area of wireless communication and communication theory. He authored the book
“Wireless Connectivity: An Intuitive and Fundamental Guide”, published by
Wiley in 2020.
---|---
|  Constantinos B. Papadias is the founding Executive Director of the
Research, Technology and Innovation Network (RTIN) of The American College of
Greece (ACG), in Athens, Greece and the Head of RTIN’s Smart Wireless Future
Technologies (SWIFT) Lab, since 2020. He is also Professor of Information
Technology at ACG’s Deree College and Alba graduate business school and the
Scientific Director of the American College of Greece Research Center (ACG-
RC). Prior to these, he was the Dean of Athens Information Technology (AIT),
in Athens, Greece, where he was also Head of the Broadband Wireless and Sensor
Networks (B-WiSE) Research Group. He currently holds Adjunct Professorships at
Aalborg University and at the University of Cyprus. He received the Diploma of
Electrical Engineering from the National Technical University of Athens (NTUA)
in 1991 and the Doctorate degree in Signal Processing (highest honors) from
the Ecole Nationale Supérieure des Télécommunications (ENST), Paris, France,
in 1995. He was a researcher at Institut Eurécom (1992-1995), Stanford
University (1995-1997) and Bell Labs (as Member of Technical Staff from
1997-2001 and as Technical Manager from 2001-2006). He was also Adjunct
Professor at Columbia University (2004-2005) and Carnegie Mellon University
(2006-2011). He has published over 220 papers and 4 books and has received
approx. 10000 citations for his work, with an h-index of 45. He has also made
standards contributions and holds 12 patents. He was a member of the Steering
Board of the Wireless World Research Forum (WWRF) from 2002-2006, a member and
industrial liaison of the IEEE’s Signal Processing for Communications
Technical Committee from 2003-2008 and a National Representative of Greece to
the European Research Council’s IDEAS program from 2007-2008. He has served as
member of the IEEE Communications Society’s Fellow Evaluation and Awards
Committees, as well as an Associate Editor for various journals, including the
IEEE Trans. on Signal Processing. He was also a member of the IEEE Signal
Processing for Communications for 2 terms. He has contributed to the
organization of several IEEE conferences, including, as General Chair, the
IEEE CTW 2016 and the IEEE SPAWC 2018 workshops. He has acted as Technical
Coordinator in several EU projects such as: CROWN in the area of cognitive
radio; HIATUS in the area of interference alignment; HARP in the area of
remote radio heads and ADEL in the area of licensed shared access. He was the
Research Coordinator of the European Training Network project PAINLESS on the
topic of energy autonomous infrastructure-less wireless networks (2018-2022)
and is currently the Technical Coordinator of the EU CHIST-ERA project FIREMAN
on the topic of predictive maintenance via machine type wireless communication
systems. His distinctions include the Bell Labs President’s Award (2002), the
IEEE Signal Processing Society’s Young Author Best Paper Award (2003), a Bell
Labs Teamwork Award (2004), his recognition as a “Highly Cited Greek
Scientist” (2011), two IEEE conference paper awards (2013, 2014) and a “Best
Booth” Award at EUCNC (2016). He has also been shortlisted twice for the Bell
Labs Prize (2014, 2019). He was a Distinguished Lecturer of the IEEE
Communications Society for 2012-2013. He is a Fellow of IEEE since 2013 and
Fellow of the European Alliance of Innovation (EAI) since 2019. From 2020-2022
he was a member of Greece’s sectorial scientific council of Engineering
Sciences, which supports the country’s National Council for Research and
Innovation. He was recently elected Director-at-Large of the IEEE Signal
Processing Society for Region 8 (Europe, Middle East and Africa).
---|---
|  Mérouane Debbah is Chief Researcher at the Technology Innovation Institute
in Abu Dhabi. He is a Professor at Centralesupelec and an Adjunct Professor
with the Department of Machine Learning at the Mohamed Bin Zayed University of
Artificial Intelligence. He received the M.Sc. and Ph.D. degrees from the
Ecole Normale Supérieure Paris-Saclay, France. He was with Motorola Labs,
Saclay, France, from 1999 to 2002, and also with the Vienna Research Center
for Telecommunications, Vienna, Austria, until 2003. From 2003 to 2007, he was
an Assistant Professor with the Mobile Communications Department, Institut
Eurecom, Sophia Antipolis, France. In 2007, he was appointed Full Professor at
CentraleSupelec, Gif-sur-Yvette, France. From 2007 to 2014, he was the
Director of the Alcatel-Lucent Chair on Flexible Radio. From 2014 to 2021, he
was Vice-President of the Huawei France Research Center. He was jointly the
director of the Mathematical and Algorithmic Sciences Lab as well as the
director of the Lagrange Mathematical and Computing Research Center. Since
2021, he is leading the AI & Digital Science Research centers at the
Technology Innovation Institute. He has managed 8 EU projects and more than 24
national and international projects. His research interests lie in fundamental
mathematics, algorithms, statistics, information, and communication sciences
research. He is an IEEE Fellow, a WWRF Fellow, a Eurasip Fellow, an AAIA
Fellow, an Institut Louis Bachelier Fellow and a Membre émérite SEE. He was a
recipient of the ERC Grant MORE (Advanced Mathematical Tools for Complex
Network Engineering) from 2012 to 2017. He was a recipient of the Mario Boella
Award in 2005, the IEEE Glavieux Prize Award in 2011, the Qualcomm Innovation
Prize Award in 2012, the 2019 IEEE Radio Communications Committee Technical
Recognition Award and the 2020 SEE Blondel Medal. He received more than 20
best paper awards, among which the 2007 IEEE GLOBECOM Best Paper Award, the
Wi-Opt 2009 Best Paper Award, the 2010 Newcom++ Best Paper Award, the WUN
CogCom Best Paper 2012 and 2013 Award, the 2014 WCNC Best Paper Award, the
2015 ICC Best Paper Award, the 2015 IEEE Communications Society Leonard G.
Abraham Prize, the 2015 IEEE Communications Society Fred W. Ellersick Prize,
the 2016 IEEE Communications Society Best Tutorial Paper Award, the 2016
European Wireless Best Paper Award, the 2017 Eurasip Best Paper Award, the
2018 IEEE Marconi Prize Paper Award, the 2019 IEEE Communications Society
Young Author Best Paper Award, the 2021 Eurasip Best Paper Award, the 2021
IEEE Marconi Prize Paper Award, the 2022 IEEE Communications Society
Outstanding Paper Award, the 2022 ICC Best paper Award as well as the
Valuetools 2007, Valuetools 2008, CrownCom 2009, Valuetools 2012, SAM 2014,
and 2017 IEEE Sweden VT-COM-IT Joint Chapter best student paper awards. He is
an Associate Editor-in-Chief of the journal Random Matrix: Theory and
Applications. He was an Associate Area Editor and Senior Area Editor of the
IEEE TRANSACTIONS ON SIGNAL PROCESSING from 2011 to 2013 and from 2013 to
2014, respectively. From 2021 to 2022, he served as an IEEE Signal Processing
Society Distinguished Industry Speaker.
---|---


# Conservation Laws for Free Particles Interacting with a Thermal Reservoir in
Open Quantum Systems

M. A. Z. Khan1<EMAIL_ADDRESS>M. Moodley1<EMAIL_ADDRESS>F.
Petruccione1<EMAIL_ADDRESS>1Quantum Research Group, School of
Chemistry and Physics, University of KwaZulu–Natal, Private Bag X54001, Durban
4000, South Africa

###### Abstract

We construct Lie point symmetries, a closed-form solution and conservation
laws using a non-Noetherian approach for a specific case of the Gorini-
Kossakowski-Sudarshan-Lindblad equation that has been recast for the study of
non-relativistic free particles in a thermal reservoir environment.
Conservation laws are constructed subsequently using the Ibragimov method via
a solution to the adjoint form of the equation of motion via its corresponding
scalaing symmetry. A general computational framework for obtaining all
conserved vectors is exhibited some triplets of conserved quantities are
calculated in full.
_Keywords_ : Open quantum systems; Quantum Brownian motion; Lie symmetries;
Conservation laws

##  I Introduction

An “open quantum system” is a quantum mechanical system which interacts with
its environment. The environment can be external noise or a quantum
environment. The quantum environment can be bosonic or fermionic in nature;
the quantum system is subjected to the effects of the environment. The
environment contains many degrees of freedom - infinite in the case of a
reservoir or a thermal bath. Of particular interest is the collective
evolution of the system as an ensemble, not the motion of the individual
degrees of freedom. Thus, the foremost intent is to ascertain the effects of
the environment on the open system.

The theoretical rationalism for studying open quantum systems is essentially
divided into six components -

  1. 1.

All quantum systems are truly open; the random and systematic superimposition
of erroneous signals is unavoidable. The Schrödinger equation, which describes
the dynamics of singleton particles, is simply an approximation and does not
epitomize reality as observed by experiments.

  2. 2.

It is the foundational fabric and substructure to assess and describe open
problems in quantum theory. For example, the quantum-to-classical transition
Schlosshauer1  and the quantum measurement problem Schlosshauer2 .

  3. 3.

Technological applications of quantum theory. Exploiting - for example,
quantum entanglement - for sensing, metrology, computation, communication and
simulation NielsenChuang . It has been observed experimentally that these
quantum features are vulnerable to noise and thus, they are quickly consumed.
As a consequence, there is a limited lifetime for which any quantum technology
can exist. Therefore, the environments are modelled in order to understand and
minimize noise.

  4. 4.

Quantum reservoir engineering and quantum optimal control. To minimize the
effects of decoherence induced by the environment and to enhance coherence or
engineer types of decoherence that allow the final states of the dynmaics to
have specific entangled states, which serve as resources of quantum
computation. Further,

     * •

to modify some of the parameters of the environment - for example, the
spectral density in order to transfigure it for quantum specific technological
purposes.

     * •

to add dissipative sources to the isolated system so as to reach some
entangled or target states that are advantageous resources for quantum
technologies.

  5. 5.

Quantum simulations. To simulate quantum systems controllably. These allow for
the addressal of open problems - for example, the analog of the Schrödinger
equation for open quantum systems.

  6. 6.

Quantum probing. Consider a small quantum system such as an atom confined in a
certain optical potential and consider a many-body system such as an optical
lattice or a complex network. Allowing the two systems to interact and being
able to turn on and turn off the interaction strength perfectly, the small
quantum system can be probed to understand the dynamics of the many-body
system because the decohering dynamics is induced upon the probe. Therefore,
the properties of the many-body system can be mapped onto the probe. By
immersing the probe, one wishes to observe that the global property, perhaps
the ground state of the many-body system, could be detected in the type of
induced decoherence on the probe. For example, the Bose-Hubbard model for the
transition of a superfluid to a Mott insulator regime MottRegime1 ;
MottRegime2 .

This paper is organized in the following manner:

In §II, we present the general equations of motion that govern the dissipative
dynamics of the system under consideration. Moreover, a discussion of the
mathematical and physical properties of the equation of motion is discussed.

In §III, we derive the Lie point symmetries of the equation of motion and
describe the algebra of these symmetries.

In §IV, we construct a closed-form solution using a scaling symmetry.

In §V, we rigorously derive the conservation laws of the underlying equation
in the chronological order: Construction of formal Lagrangian, realization of
the adjoint-form of the equation of motion via the Euler-Lagrange derivative,
acquirement of a group-invariant solution of the adjoint equation and a
conscientious statement of the general form of the conservation laws.
Subsequently, we calculate three conserved vectors.

In §VI, we discuss the results furnished in this paper

In §VII, we provide a contextual background to ancillary formulae used in this
research to construct Lie point symmetries and conservation laws.

##  II The Model

The dynamics of the open quantum system is described by the Gorini-
Kossakowski-Sudarshan-Lindblad master equation / quantum Liovillian equation
of motion GKSL1 ; GKSL2 ; GKSL3

$\frac{\partial\rho_{s}(t)}{\partial
t}=-\imath\left[\hat{H},\rho_{s}(t)\right]+\sum_{k}\gamma_{k}\left[\hat{L}_{k}\rho_{s}(t)\hat{L}_{k}^{\dagger}-\frac{1}{2}\left\\{\hat{L}_{k}^{\dagger}\hat{L}_{k},\rho_{s}(t)\right\\}\right],$
(1)

The equation (1) is an extension of the idealized Schrödinger equation for
non-interacting systems to include environmental-interaction. Equation (1) is
not the most general equation and arises because of the Born-Markov
approximations and recoupling of the system and the environment. The first
term in (1) descibes the unitary dynamics of the system and the second term
descibes the dissipative dynamics that is related to the reciprocity between
the system and the the environment.

For a non-relativistic free particle exposed to a thermal reservoir, it has
been shown in ThermalRes1 ; ThemalRes2 ; ThermalRes4 ; ThermalRes3  that
Equation (1) reduces to

$4\pi mh^{2}\frac{\partial\rho}{\partial t}+\imath
h^{3}\left(\frac{\partial^{2}\rho}{\partial
y^{2}}-\frac{\partial^{2}\rho}{\partial x^{2}}\right)+32\pi^{3}m^{2}\gamma
k_{B}T\left(y-x\right)^{2}\rho=0,$ (2)

where $h\approx 6.626\times 10^{-34}\;J.s$ is the Planck constant,
$\imath=\sqrt{-1}$, $m$ is the particle mass, $\gamma$ is the dissipation
quantifier, $k_{B}\approx 1.381\times 10^{-23}\;m^{2}.kg/s^{2}.K$ is the
Boltzmann constant, $T$ is the environmental temperature and
$\rho=\rho(x,y,t)$ is the positively-defined trace 1 reduced density matrix
for the system. Principally, Equation (2) describes the interaction of an
unbound particle having uniform potential with a system of high-temperature
coupled harmonic oscillators. Compelling physical features are realized from
the study of this model. For example, it has been observed that for a free
particle system that is in thermal equilibrium with a reservoir, the system
localizes when subjected to specific initial conditions. As a result, quantum
fluctuations in the dynamical evolution of the system become damped by
temperature variations that are present in the medium; mathematically, this
entails that the reduced density matrix approximately becomes non-defective
and can therefore be scaled anisotropically. Physically, this implies that
information from the free particle dissipates into the environment and
consequently explains the quantum decoherence phenomenon. This has the
remarkable ramification that the system’s evolution state over time traces out
a path through a higher-dimensional space which is comparatively commensurate
to a continuous-time stochastic process with stationary independent
increments, videlicet a Wiener process (Brownian motion) ThermalRes3 .

##  III Lie Point Symmetries

We consider Lie point symmetries of the form

$\Gamma=\xi^{x}(x,y,t,\rho_{s})\frac{\partial}{\partial
x}+\xi^{y}(x,y,t,\rho)\frac{\partial}{\partial
y}+\xi^{t}(x,y,t,\rho)\frac{\partial}{\partial
t}+\eta(x,y,t,\rho)\frac{\partial}{\partial\rho}.$ (3)

Acting the second-order extension operator (A.2.1) on (3) together with the
condition

$\frac{\partial\rho}{\partial
t}=\frac{\imath\hbar^{3}\left(\frac{\partial^{2}\rho}{\partial
x^{2}}-\frac{\partial^{2}\rho}{\partial y^{2}}\right)-32\pi^{3}m^{2}\gamma
k_{B}T\left(y-x\right)^{2}\rho}{4\pi m\hbar^{2}}.$ (4)

we obtain the $\left(7+\infty\right)$-dimensional subalgebra of symmetries,
spanned by the vector fields

$\displaystyle\Gamma_{1}=$ $\displaystyle\;\frac{\partial}{\partial
t},\qquad\qquad\qquad\;\;\;\Gamma_{2}=\rho\frac{\partial}{\partial t},$
$\displaystyle\Gamma_{3}=$ $\displaystyle\;\frac{\partial}{\partial
x}+\frac{\partial}{\partial
y},\qquad\qquad\Gamma_{4}=t\frac{\partial}{\partial
x}+t\frac{\partial}{\partial y}+\frac{2\pi\imath
m}{h}\left(x-y\right)\rho\frac{\partial}{\partial\rho},$
$\displaystyle\Gamma_{5}=$
$\displaystyle\;\left(\frac{x+2y}{2}\right)\frac{\partial}{\partial
x}+\left(\frac{2x+y}{2}\right)\frac{\partial}{\partial
y}+t\frac{\partial}{\partial t}-\frac{\rho}{2}\frac{\partial}{\partial\rho},$
$\displaystyle\Gamma_{6}=$ $\displaystyle\;\left(\frac{4\pi k_{B}Tt^{2}-\imath
h}{4\pi k_{B}T}\right)\frac{\partial}{\partial
x}+t^{2}\frac{\partial}{\partial y}+\frac{4\pi\imath
m}{h}t\left(x-y\right)\rho\frac{\partial}{\partial\rho},$
$\displaystyle\Gamma_{7}=$ $\displaystyle\;\left(\frac{4\pi
k_{B}Tt^{3}-3\imath ht}{4\pi k_{B}T}\right)\frac{\partial}{\partial
x}+t^{3}\frac{\partial}{\partial y}+\rho\left[\frac{mhx+6\pi mk_{B}T\imath
t^{2}\left(x-y\right)}{hk_{B}T}\right]\frac{\partial}{\partial\rho},$
$\displaystyle\Gamma_{\infty}=$
$\displaystyle\;\Sigma(x,y,t)\frac{\partial}{\partial\rho},$ (5)

where $\Sigma(x,y,t)$ is any solution that satisfies (2). The algebra of the
symmetries (5) is
$\left\\{\mathcal{A}_{1}\oplus_{s}\mathbb{W}\right\\}\oplus_{s}\mathcal{A}^{a}_{3,5}$,
where $\mathbb{W}$ is the Heisenberg-Weyl algebra.

##  IV A Group Invariant Solution

Using $\Gamma_{3}$ from (5), we obtain the invariants

$\mu=y-x,\quad\nu=t,\quad\Phi(\mu,\nu)=\rho(x,y,t).$ (6)

Substituting (6) into (2), we obtain the reduced equation

$\hbar^{2}\frac{\partial\Phi}{\partial\nu}+8\pi^{2}m^{2}\gamma
k_{B}T\mu^{2}\Phi=0.$ (7)

Solving (7) and reverting to the original coordinates in (6), we obtain the
solution

$\rho(x,y,t)=\varrho(y-x)\exp\left[-\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{\hbar^{2}}\left(y-x\right)^{2}t\right],$ (8)

where $\varrho(y-x)$ is some arbitrary function of integration.

##  V Conservation Laws

Using the Ibragimov method Ibragimov1 ; Ibragimov2 , we construct the formal
Lagrangian for (2)

$\mathscr{L}=\vartheta(x,y,t)\left[4\pi h^{2}\frac{\partial\rho}{\partial
t}+\imath h^{3}\left(\frac{\partial^{2}\rho}{\partial
y^{2}}-\frac{\partial^{2}\rho}{\partial x^{2}}\right)+32\pi^{3}m^{2}\gamma
k_{B}T\left(y-x\right)^{2}\rho\right].$ (9)

Applying the Euler-Lagrange derivative (A.5) to (9), we obtain the adjoint
equation

$\imath h^{3}\left(\frac{\partial^{2}\vartheta}{\partial
y^{2}}-\frac{\partial^{2}\vartheta}{\partial x^{2}}\right)-4\pi
h^{2}\frac{\partial\vartheta}{\partial
t}+32\pi^{3}m^{2}k_{B}T\left(y-x\right)^{2}\gamma\vartheta=0.$ (10)

Following the same procedure as in §III and §IV, it is easily verified that
Equation (10) admits the Lie point symmetry, inter alia, and corresponding
invariant solution

$\overleftarrow{\Gamma}_{3}=\frac{\partial}{\partial
x}+\frac{\partial}{\partial
y},\qquad\qquad\vartheta(x,y,t)=\varpi(y-x)\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right],$ (11)

where $\varpi(y-x)$ is some arbitrary function of integration.

We consider conservation laws of the form
$\boldsymbol{\phi}=\left\langle\phi^{x},\phi^{y},\phi^{t}\right\rangle$. From
(A.6.1) and (A.6.2), we discern that the general form of the conserved
currents are

$\displaystyle\bar{\phi}^{t}=$ $\displaystyle\;\mathscr{L}\xi^{t}+4\pi
h^{2}\left(\eta-\xi^{t}\frac{\partial\rho}{\partial
t}-\xi^{x}\frac{\partial\rho}{\partial x}-\xi^{y}\frac{\partial\rho}{\partial
y}\right)\vartheta,$ (12.1) $\displaystyle\bar{\phi}^{x}=$
$\displaystyle\;\mathscr{L}\xi^{x}+\imath
h^{3}\left(\eta-\xi^{t}\frac{\partial\rho}{\partial
t}-\xi^{x}\frac{\partial\rho}{\partial x}-\xi^{y}\frac{\partial\rho}{\partial
y}\right)\frac{\partial\vartheta}{\partial x}-\imath
h^{3}\frac{\partial}{\partial x}\left(\eta-\xi^{t}\frac{\partial\rho}{\partial
t}-\xi^{x}\frac{\partial\rho}{\partial x}-\xi^{y}\frac{\partial\rho}{\partial
y}\right)\vartheta,$ (12.2) $\displaystyle\bar{\phi}^{y}=$
$\displaystyle\;\mathscr{L}\xi^{y}-\imath
h^{3}\left(\eta-\xi^{t}\frac{\partial\rho}{\partial
t}-\xi^{x}\frac{\partial\rho}{\partial x}-\xi^{y}\frac{\partial\rho}{\partial
y}\right)\frac{\partial\vartheta}{\partial y}+\imath
h^{3}\frac{\partial}{\partial y}\left(\eta-\xi^{t}\frac{\partial\rho}{\partial
t}-\xi^{x}\frac{\partial\rho}{\partial x}-\xi^{y}\frac{\partial\rho}{\partial
y}\right)\vartheta.$ (12.3)

Using (12.1)-(12.3) and the form of the adjoint equation in (11), we
demonstrate some of the conservation laws below.

###  V.1 Conservation of Energy

$\displaystyle\phi^{t}_{1}=$
$\displaystyle\;\varpi(y-x)\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left[32k_{B}m^{2}\pi^{3}\gamma
T\left(x-y\right)^{2}+\imath h^{2}\left(\frac{\partial^{2}\rho}{\partial
y^{2}}-\frac{\partial^{2}\rho}{\partial x^{2}}\right)\right],$ (13.1)
$\displaystyle\phi^{x}_{1}=$ $\displaystyle\;\imath
h\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left\\{h^{2}\varpi^{\prime}(y-x)\frac{\partial\rho}{\partial
t}+\varpi(y-x)\left[h^{2}\frac{\partial^{2}\rho}{\partial t\partial
x}\right.\right.$ $\displaystyle\left.\left.-16k_{B}m^{2}\pi^{2}T\gamma
t\left(x-y\right)\frac{\partial\rho}{\partial t}\right]\right\\},$ (13.2)
$\displaystyle\phi^{y}_{1}=$ $\displaystyle\;-\imath
h\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left\\{-h^{2}\varpi^{\prime}(y-x)\frac{\partial\rho}{\partial
t}+\varpi(y-x)\left[h^{2}\frac{\partial^{2}\rho}{\partial t\partial
y}\right.\right.$ $\displaystyle\left.\left.\;+16k_{B}m^{2}\pi^{2}T\gamma
t\left(x-y\right)\frac{\partial\rho}{\partial t}\right]\right\\}.$ (13.3)

###  V.2 Non-trivial Conservation Law I

$\displaystyle\phi_{2}^{t}=$
$\displaystyle\;\rho\varpi(y-x)\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left[32k_{B}m^{2}\pi^{3}\gamma
T\left(x-y\right)^{2}\rho+\imath h^{3}\left(\frac{\partial^{2}\rho}{\partial
y^{2}}-\frac{\partial^{2}\rho}{\partial x^{2}}\right)\right],$ (13.4)
$\displaystyle\phi_{2}^{x}=$ $\displaystyle\;\imath
h\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left\\{h^{2}\rho\varpi^{\prime}(y-x)\frac{\partial\rho}{\partial
t}+\varpi(y-x)\left[h^{2}\frac{\partial\rho}{\partial
t}\frac{\partial\rho}{\partial x}\right.\right.$
$\displaystyle\left.\left.\;+h^{2}\rho\frac{\partial^{2}\rho}{\partial
t\partial x}-16k_{B}m^{2}\pi^{2}\gamma
Tt\left(x-y\right)\rho\frac{\partial\rho}{\partial t}\right]\right\\},$ (13.5)
$\displaystyle\phi_{2}^{y}=$ $\displaystyle\;-\imath
h\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left\\{-h^{2}\rho\varpi^{\prime}(y-x)\frac{\partial\rho}{\partial
t}+\varpi(y-x)\left[\frac{\partial\rho}{\partial
t}\frac{\partial\rho}{\partial y}\right.\right.$
$\displaystyle\left.\left.\;+h^{2}\rho\frac{\partial^{2}\rho}{\partial
t\partial y}+16k_{B}m^{2}\pi^{2}\gamma
Tt\left(x-y\right)\rho\frac{\partial\rho}{\partial t}\right]\right\\}.$ (13.6)

###  V.3 Non-trivial Conservation Law II

$\displaystyle\phi_{3}^{t}=$ $\displaystyle\;-4\pi
h^{2}\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\varpi(y-x)\left(\frac{\partial\rho}{\partial
x}+\frac{\partial\rho}{\partial y}\right),$ (13.7)
$\displaystyle\phi_{3}^{x}=$
$\displaystyle\;\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left\\{\imath
h\left[h^{2}\varpi^{\prime}(y-x)-16k_{B}m^{2}\pi^{2}\gamma
Tt\left(y-x\right)\varpi(y-x)\right]\right.$
$\displaystyle\left.\;\times\left(\frac{\partial\rho}{\partial
x}+\frac{\partial\rho}{\partial
y}\right)+\varpi(y-x)\left[32k_{B}m^{2}\pi^{3}\gamma
T\left(x-y\right)^{2}\rho+4\pi h^{2}\frac{\partial\rho}{\partial t}+\imath
h^{3}\left(\frac{\partial^{2}\rho}{\partial
y^{2}}-\frac{\partial^{2}\rho}{\partial x^{2}}\right)\right]\right.$
$\displaystyle\left.\;+\imath h^{3}\left(\frac{\partial^{2}\rho}{\partial
x\partial y}+\frac{\partial^{2}\rho}{\partial x^{2}}\right)\right\\},$ (13.8)
$\displaystyle\phi^{y}_{3}=$
$\displaystyle\;\exp\left[\frac{8\pi^{2}m^{2}\gamma
k_{B}T}{h^{2}}\left(y-x\right)^{2}t\right]\left\\{\imath
h\left[h^{2}\varpi^{\prime}(y-x)-16k_{B}m^{2}\pi^{2}\gamma
Tt\left(y-x\right)\varpi(y-x)\right]\right.$
$\displaystyle\left.\;\times\left(\frac{\partial\rho}{\partial
x}+\frac{\partial\rho}{\partial y}\right)-\imath
h^{3}\varpi(y-x)\left(\frac{\partial^{2}\rho}{\partial x\partial
y}+\frac{\partial^{2}\rho}{\partial
y^{2}}\right)+\varpi(y-x)\left[32k_{B}m^{2}\pi^{3}\gamma
T\left(x-y\right)^{2}\rho\right.\right.$ $\displaystyle\left.\left.\;+4\pi
h^{2}\frac{\partial\rho}{\partial t}+\imath
h^{3}\left(\frac{\partial^{2}\rho}{\partial
y^{2}}-\frac{\partial^{2}\rho}{\partial x^{2}}\right)\right]\right\\}.$ (13.9)

##  VI Discussion

Using the Lie point symmetry method, we obtained a
$\left(7+\infty\right)$-dimensional set of point symmetries. By examination of
the nonzero commutator actions, it was shown that, inter alia, a Heisenberg-
Weyl algebra was obtained. Upon investigation of the literature, Equation (2)
shares this ubiquitous feature with the Schrödinger equations for a
homogeneous and harmonic potential energies. Using the scaling symmetry, an
invariant solution was constructed in terms of a generic integration function,
which serves as the normalization constant.

Conservation laws were constructed using a non-Noetherian approach. This has
the added benefit that one need not derive the equation of motion from Euler-
Lagrange equations and guarantees that every symmetry leads to a conservation
law; properties that Noether’s approach lacks. Moreover, we have provided the
general form of all subsequent conservation laws that one wishes to construct
at one’s own liberty and further demonstrated the construction of three such
pairs. Interestingly, the conservation of energy law in (13.1)-(13.3) is
particularly provocative as one would think that the only energy of the system
that is conserved is the Hamiltonian operator. In principal, one would be able
to generate a large class, possibly infinite, of conserved energies,
contingent upon the choice of the normalization constant so as to ensure its
convergence. Finally, we note that equation (2) is linear and therefore
possesses any infinite number of point symmetries and conservation laws; this
advocates the appearance of $\Gamma_{\infty}$. The formulae (12.1)-(12.3) that
we have proposed gives an algorithmic and computational approach to obtaining
these conservation laws.

##  VII Appendix

In this section, we provide a cursory overview of the Lie symmetry method and
the Ibragimov method. Following the notation of Olver Olver , consider the
$n^{\text{th}}$-order PDE of the form

$\Delta\left(x,y,t,\rho,\frac{\partial\rho}{\partial
x},\frac{\partial\rho}{\partial y},\frac{\partial\rho}{\partial
t},\frac{\partial^{2}\rho}{\partial x^{2}},\frac{\partial^{2}\rho}{\partial
y^{2}},\ldots\right)=0,$ (A.1)

where $\rho=\rho(x,y,t)$. Equation (A.1) admits Lie point symmetries of the
form (2) with the $n^{\text{th}}$-order extension operator given by

$\displaystyle\text{pr}^{\left[2\right]}\Gamma\equiv\Gamma^{\left[2\right]}=$
$\displaystyle\;\Gamma+\zeta^{t}\frac{\partial}{\partial\rho_{t}}+\zeta^{xx}\frac{\partial}{\partial\rho_{xx}}+\zeta^{yy}\frac{\partial}{\partial\rho_{yy}},$
(A.2.1) $\displaystyle\zeta^{x}=$
$\displaystyle\;D_{x}\left(\eta\right)-D_{x}\left(\xi^{x}\right)\rho_{x}-D_{x}\left(\xi^{y}\right)\rho_{y}-D_{x}\left(\xi^{t}\right)\rho_{t},$
(A.2.2) $\displaystyle\zeta^{y}=$
$\displaystyle\;D_{y}\left(\eta\right)-D_{y}\left(\xi^{x}\right)\rho_{x}-D_{y}\left(\xi^{y}\right)\rho_{y}-D_{y}\left(\xi^{t}\right)\rho_{t},$
(A.2.3) $\displaystyle\zeta^{t}=$
$\displaystyle\;D_{t}\left(\eta\right)-D_{t}\left(\xi^{x}\right)\rho_{x}-D_{t}\left(\xi^{y}\right)\rho_{y}-D_{t}\left(\xi^{t}\right)\rho_{t},$
(A.2.4) $\displaystyle\zeta^{xx}=$
$\displaystyle\;D_{x}\left(\zeta^{x}\right)-D_{x}\left(\xi^{x}\right)\rho_{xx}-D_{x}\left(\xi^{y}\right)\rho_{yx}-D_{x}\left(\zeta^{t}\right)\rho_{tx},$
(A.2.5) $\displaystyle\zeta^{yy}=$
$\displaystyle\;D_{y}\left(\zeta^{y}\right)-D_{y}\left(\xi^{x}\right)\rho_{xy}-D_{y}\left(\xi^{y}\right)\rho_{yy}-D_{y}\left(\zeta^{t}\right)\rho_{ty},$
(A.2.6)

and $D_{i}$, for $i\in\left\\{x,y,t\right\\}$, is the total differential
operator. The application of (A.2.1) on (A.1) yields a system of linear PDEs

$\xi^{x}\frac{\partial\Delta}{\partial
x}+\xi^{y}\frac{\partial\Delta}{\partial
y}+\xi^{t}\frac{\partial\Delta}{\partial
t}+\zeta^{t}\frac{\partial\Delta}{\partial\rho_{t}}+\zeta^{xx}\frac{\partial\Delta}{\partial\rho_{xx}}+\zeta^{yy}\frac{\partial\Delta}{\partial\rho_{yy}}=0.$
(A.3)

Equating the various powers of the variable $\Delta$ in (A.3) to zero produces
a set of linear PDEs. Solving the system of PDEs provides functional forms for
$\xi^{x},\xi^{y},\xi^{t}$ and $\eta$ which gives each Lie point symmetry.

The formal Lagrangian is given by

$\mathcal{L}=\sum\limits_{i=1}^{m}v_{i}F_{i}\left(\mathbf{x},\boldsymbol{u_{\left(i\right)}}\right),$
(A.4)

where $\mathbf{x}=\left(x_{1},x_{2},\ldots,x_{m}\right)$ and
$\boldsymbol{u_{\left(i\right)}}=\left(u_{\left(1\right)},u_{\left(2\right)},\dots,u_{\left(s\right)}\right)$.
The Euler-Lagrange derivative is given by

$\frac{\delta}{\delta u^{\alpha}}=\frac{\partial}{\partial
u^{\alpha}}+\sum_{r\geq
1}D_{i_{1}},D_{i_{2}},\ldots,D_{i_{r}}\frac{\partial}{\partial
u^{\alpha}_{i_{1}i_{2}\ldots i_{r}}},\qquad\alpha=1,2\ldots,m.$ (A.5)

Applying (A.5) on (A.4) yields the adjoint form of the underlying equation.
Thereafter, one applies the Lie method in order to obtain a set of new
symmetries and corresponding invariant solutions of the adjoint equation.
Subsequently, one substitutes in the symmetries of the underlying equations
together with the invariant solutions of the adjoint equation into

$\displaystyle\varphi^{i}=$
$\displaystyle\;\mathcal{L}\xi^{i}+W^{\alpha}\left[\frac{\partial\mathcal{L}}{\partial
u_{i}^{\alpha}}-D_{j}\left(\frac{\partial\mathcal{L}}{\partial
u^{\alpha}_{ij}}\right)+D_{j}D_{k}\left(\frac{\partial\mathcal{L}}{\partial
u^{\alpha}_{ijk}}\right)-\ldots\right]$
$\displaystyle\;+D_{j}\left(W^{\alpha}\right)\left[\frac{\partial\mathcal{L}}{\partial
u^{\alpha}_{ij}}-D_{k}\left(\frac{\partial\mathcal{L}}{\partial
u^{\alpha}_{ijk}}\right)+\ldots\right]$
$\displaystyle\;+D_{j}D_{k}\left(W^{\alpha}\right)\left(\frac{\partial\mathcal{L}}{\partial
u^{\alpha}_{ijk}}\right)+\ldots,$ (A.6.1) $\displaystyle W^{\alpha}=$
$\displaystyle\;\eta^{\alpha}-\sum_{\mathclap{i\in\mathbf{x}=\left(x_{1},x_{2},\ldots,x_{m}\right)}}\xi^{i}u_{i}^{\alpha},$
(A.6.2)

in order to obtain the conservation laws of the form

$\sum_{\mathclap{i\in\mathbf{x}=\left(x_{1},x_{2},\ldots,x_{m}\right)}}D_{i}\varphi^{i}=0.$
(A.6.3)

###### Acknowledgements.

MAZK, MM and FP thank the University of KwaZulu-Natal of the Republic of South
Africa for making this research possible. FP acknowledges that this research
is supported by the South African Research Chair Initiative of the Department
of Science and Innovation and the National Research Foundation (NRF).

## References

  * (1) M. A. Schlosshauer, Decoherence and the Quantum-to-Classical Transition; Springer: Berlin, Germany, 2007.
  * (2) M. A. Schlosshauer, J. Koffler and A. Zeilinger, Stud. Hist. Phil. Sci. Part B. 44, 222 (2013).
  * (3) M. A. Nielsen and I. L. Chuang, Quantum Computation and Qunatum Information; Cambridge University Press: Cambridge, England, 2000.
  * (4) M. P. A. Fisher, G. Grinstein and D. S. Fisher, Phys. Rev. B. 40, 546, (1989).
  * (5) M. Ma, B. I. Halperin and P. A. Lee, Phys. Rev. B. 34, 3136, (1986).
  * (6) A. Kossakowski, Rep. Math. Phys. 3, 247 (1972).
  * (7) G. Lindblad, Commun. Math. Phys. 48, 2 (1976).
  * (8) V. Gorini, A. Kossakowski and E.C.G. Sudarshan, J. Math. Phys. 17, 821 (1976).
  * (9) A. O. Caldeira and A.J. Legget, Phys. A. 121, 587 (1983).
  * (10) N. Gisin and I. C. Percival, J. Phys. A. 25, 5677 (1992).
  * (11) J. Halliwal and A. Zoupas, Phys. Rev. D. 52, 7294 (1995).
  * (12) J. Halliwal and A. Zoupas, Phys. Rev. D. 55, 4697 (1997).
  * (13) N. H. Ibragimov, J. Math. Anal. Appl. 333, 311 (2007).
  * (14) N. H. Ibragimov, Teor. Mate. Fiz. 1, 350 (1969).
  * (15) P. Olver, Applications of Lie Groups to Differential Equations; Springer: Berlin, Germany, 1986.


# DUST FORMATION, EVOLUTION, AND OBSCURATION EFFECTS IN THE VERY HIGH-REDSHIFT
UNIVERSE

Eli Dwek11affiliation: Observational Cosmology Lab., Code 665, NASA at Goddard
Space Flight Center, Greenbelt, MD 20771 ,Johannes Staguhn11affiliation:
Observational Cosmology Lab., Code 665, NASA at Goddard Space Flight Center,
Greenbelt, MD 20771 22affiliation: Department of Physics and Astronomy, Johns
Hopkins University, Baltimore, MD 21218 , Richard G. Arendt11affiliation:
Observational Cosmology Lab., Code 665, NASA at Goddard Space Flight Center,
Greenbelt, MD 20771 33affiliation: CRESST, University of Maryland Baltimore
County, Baltimore, MD 21250 , Attila Kovacks44affiliation: Astronomy
Department, CalTech, Pasadena, CA 90025 and Astronomy Department, University
of Minnesota, MN 12345 , Ting Su22affiliation: Department of Physics and
Astronomy, Johns Hopkins University, Baltimore, MD 21218 , & Dominic J.
Benford11affiliation: Observational Cosmology Lab., Code 665, NASA at Goddard
Space Flight Center, Greenbelt, MD 20771 Observational Cosmology Lab., Code
665
NASA Goddard Space Flight Center, Greenbelt, MD 20771,
e-mail<EMAIL_ADDRESS>
###### Abstract

The evolution of dust at redshifts $z\gtrsim 9$, and consequently the dust
properties, differs greatly from that in the local universe. In contrast to
the local universe, core collapse supernovae (CCSNe) are the only source of
thermally-condensed dust. Because of the low initial dust-to-gas mass ratio,
grain destruction rates are low, so that CCSNe are net producers of
interstellar dust. Galaxies with large initial gas mass or high mass infall
rate will therefore have a more rapid net rate of dust production comported to
galaxies with lower gas mass, even at the same star formation rate. The dust
composition is dominated by silicates, which exhibit a strong rise in the UV
opacity near the Lyman break. This “silicate-UV break” may be confused with
the Lyman break, resulting in a misidentification of a galaxies’ photometric
redshift. In this paper we demonstrate these effects by analyzing the spectral
energy distribution (SED) of MACS1149-JD, a lensed galaxy at $z=9.6$. A
potential 2mm counterpart of MACS1149-JD has been identified with GISMO. While
additional observations are required to corroborate this identification, we
use this possible association to illustrate the physical processes and the
observational effects of dust in the very high redshift universe.

galaxies: high-redshift - galaxies: evolution - galaxies: individual
(MACS1149-JD) - Interstellar medium (ISM), nebulae: dust, extinction -
physical data and processes: nuclear reactions, nucleosynthesis, abundances

††slugcomment: Accepted for publication in ApJ Letters

##  1 INTRODUCTION

Galaxies are the principle tracers of the global evolution of the star
formation history, the metal enrichment, and the formation of dust in the
universe. Their detection at very high redshifts offers a unique glimpse of
these physical processes during the earliest epochs of galaxy evolution. Dust
has been detected in several high-redshift galaxies (Gall et al., 2011b), the
current record being held by the dusty quasar QSO J1120+0641 located at
redshift $z=7.12$ (Venemans et al., 2012). Dust attenuates the intrinsic
spectrum of the stellar population, affecting the various physical quantities
derived from UV-optical observations. Detection of dust through its extinction
and emission is therefore essential in determining the properties of galaxies
in the high-redshift universe.

A gravitationally magnified galaxy, designated MACS1149-JD was recently
discovered at a photometric redshift of $z=9.6\pm$0.2 (Zheng et al., 2012).
The massive lensing cluster (MACS J1149.6+2223), located at redshift 0.544,
provides a magnification factor of $\mu=14.5\pm^{+4.2}_{-1.0}$. Assuming no
extinction by dust, and correcting for magnification, the star formation rate
(SFR), the total stellar mass, and luminosity derived from the observed UV-
optical (UVO) flux are about $1.2\,\mu_{15}^{-1}$ $M_{\odot}$ yr-1, $1.5\times
10^{8}\,\mu_{15}^{-1}$ $M_{\odot}$, and $2\times 10^{10}\,\mu_{15}^{-1}$
$L_{\odot}$, respectively, where $\mu_{15}$ is the lensing magnification
normalized to the nominal value of 15 (Zheng et al., 2012). However,
uncertainties in fitting the spectral energy distribution (SED) of this galaxy
allow the possibility of a certain amount of extinction (Zheng et al., 2012).

The Goddard-IRAM Superconducting 2 Millimeter Observer (GISMO) camera at the
IRAM 30m telescope on Pico Veleta (Staguhn et al., 2013), Spain was used to
probe whether MACS1149-JD has a rest frame far-infrared counterpart. The
observations detected a 2 mm source with an intensity of $400\pm 98~{}\mu$Jy
(see Figure 1). The observed offset of $8^{\prime\prime}$ (less than half the
GISMO beam) is entirely consistent with this source being at the HST detected
position of MACS1149-JD. The rest wavelength is $\sim 190$ $\mu$m, close to
the peak of the dust emission in star forming galaxies. These observations
strongly suggest that dust is present in this galaxy, however, further
confirmations are needed to absolutely corroborate this association.
Observations using the SHARC-2 submillimeter bolometer camera at the Caltech
submillimeter observatory yielded a 350 $\mu$m upper limit of $8.6\pm 5.6$ mJy
within an area consistent with the nominal position of MACS1149-JD.

For sake of the current analysis we will assume the validity of this
association, adopt the observed 2 mm flux as originating from MACS1149-JD, and
demonstrate the unique properties of any dust that may be forming in the high
redshift universe. All fluxes and results of this paper are presented in the
galaxy’s rest frame with the adopted cosmological parameters $H_{0}=67$ km s-1
Mpc-1, $\Omega_{M}=0.315$, and $\Omega_{\Lambda}=0.685$ (Planck Collaboration
et al., 2013), and a lensing magnification factor of 15. With these
cosmological parameters the age of the universe is $\sim 500$ Myr at $z=9.6$.

##  2 MODELING THE SED OF MACS1149-JD

In the absence of any starlight, dust would be in thermal equilibrium with the
cosmic microwave background (CMB). This effect can be important at high
redshifts where starlight and the CMB can have comparable energy densities (da
Cunha et al., 2013). The GISMO flux is the differential 2 mm flux with respect
to the CMB emission. Therefore, the observed 2 mm flux consists of only
reradiated starlight. We characterize the IR spectrum by $T_{d}$, the
temperature that would be attained by the dust in the absence of the CMB, and
by a mass absorption coefficient $\kappa(\lambda)$, that has a
$\lambda^{-\beta}$ ($\beta=1.5$) dependence at far-IR wavelengths and a value
of 1.5 cm2 g-1 at $\lambda=850$ $\mu$m (Kovács et al., 2010). The specific IR
luminosity in the galaxy’s rest frame is then given by:

$L_{\nu}(\lambda)=4M_{d}\,\kappa(\lambda)\,\left[\pi
B_{\nu}(\lambda,T_{eff})-\pi B_{\nu}(\lambda,T_{cmb})\right]$ (1)

where $T_{eff}=(T_{d}^{4+\beta}+T_{cmb}^{4+\beta})^{1/4+\beta}$, $M_{d}$ is
the dust mass, and $T_{cmb}=(1+z)\,2.73\approx 29$ K is the CMB temperature at
$z=9.6$. $L_{\nu}(\lambda)$ is a function of the dust temperature and mass.
With $L_{\nu}(\lambda=190$$\mu$m) normalized to the observed GISMO flux, the
dust temperature $T_{d}$ and dust mass are inversely related.

The presence of dust significantly affects the properties of MACS1149-JD
derived under the assumption that it is dust free, and constrains viable star
formation scenarios for this galaxy. The panels of Figure 2 depict the
analysis used to constrain the star formation history (SFH), the amount of
dust obscuration and resulting infrared emission, and the mass of radiating
dust, based on the available UVO data and adopted far-IR flux. Two star
formation histories, characterized by a delayed exponential star formation
rate (SFR), $\psi(t)=\psi_{0}\,(t/t_{exp})\,\exp(-t/t_{exp}+1)$, were used in
the analysis with values of $t_{exp}=$100 and 300 Myr (hereafter models
$t_{exp}100$ and $t_{exp}300$, respectively, see Figure 2a).

Intrinsic stellar spectra were calculated using the stellar population
synthesis code PÉGASE (Fioc & Rocca-Volmerange, 1997) with low-metallicity
(LowZ) and Kroupa (Kr) stellar initial mass functions (IMFs) (Marks et al.,
2012; Kroupa, 2001). The LowZ IMF is more heavily weighted towards the
formation of massive stars, and its stellar population produces more UV
radiation compared to its Kroupa counterpart. Its choice is motivated by the
theoretical argument that inefficient cooling inhibits the fragmentation of
protostellar clouds into smaller units (Bromm & Larson, 2004), and by observed
variations of the IMF with stellar metallicity (Marks et al., 2012; Kroupa et
al., 2013). Figure 2b shows the spectra for $\psi_{0}=1$ $M_{\odot}$ yr-1.

For each star formation (SF) scenario, characterized by $t_{exp}$, $\psi_{0}$,
and a stellar IMF, we calculated the amount of dust absorption needed to
reproduce the observed UVO fluxes. The ratio between the observed and
intrinsic flux is the escape probability at that given wavelength, which for a
homogeneous sphere is given by (Lucy et al., 1991; Cox & Mathews, 1969;
Osterbrock & Ferland, 2006):

$\displaystyle P_{esc}(\tau,\omega)$ $\displaystyle=$ $\displaystyle
P_{esc}(\tau,0)\left[1-\omega+\omega\,P_{esc}(\tau,0)\right]$ (2)
$\displaystyle P_{esc}(\tau,0)$ $\displaystyle=$ $\displaystyle{3\over
4\tau}\,[1-{1\over 2\tau^{2}}+({1\over\tau}+{1\over
2\tau^{2}})\,\exp(-2\tau)]$

where $\tau(\lambda)=(3/4)M_{d}\,\kappa(\lambda)/\pi R^{2}$ is the optical
depth, and $\omega(\lambda)$ is the dust albedo. The second expression
corresponds to the case when $\omega=0$. The albedo of small dust grains
($a\lesssim 0.01$ $\mu$m) is $\lesssim 0.2$ and $P_{esc}(\tau,0)\approx 0.1$
for most scenarios. With these parameters, scattering will increase the escape
probability by at most 20%, having only a minor effect on the conclusions of
this paper.

We assumed that the IR emission is only powered by stellar radiation, so that
the dust giving rise to the IR emission is also responsible for the
attenuation of the stellar emission. The reradiated IR luminosity, given by
the integral of $L_{\nu}(\lambda)$ in eq. (1), must then be equal to that the
starlight energy absorbed by the dust:

$L_{abs}=\int\ L_{\nu}^{int}(\lambda)\,[1-P_{esc}(\lambda)]\,d\nu$ (3)

where $L_{\nu}^{int}$ is the intrinsic stellar luminosity. Figure 2c
illustrates this principle of energy balance for model $t_{exp}300$–LowZ for
values of $\psi_{0}=20$ and 40 $M_{\odot}$ yr-1. The dust must correspondingly
radiate at temperatures of 44 and 70 K, with dust masses of $1.2\times 10^{7}$
and $4.0\times 10^{6}$ $M_{\odot}$, respectively.

Not all SF scenarios can fulfill the energy constraint. Figure 2d plots the IR
luminosity as a function of $T_{d}$, and the absorbed stellar luminosity as a
function of $\psi_{0}$. Energy conservation requires $L_{IR}$ (red curve) to
be equal to $L_{abs}$ (green and blue curves). The GISMO detection sets a
lower limit on the dust luminosity regardless of the dust temperature. This
constraint sets a corresponding lower limit on $\psi_{0}$ for each SF scenario
(see Table 1 for details).

An additional constraint on viable SF scenarios is that the dust mass giving
rise to the UVO attenuation and the IR emission be produced within the
allotted time of $<500$ Myr, taking possible grain destruction processes into
account (Dwek et al., 2007).

##  3 THE EVOLUTION OF DUST AT HIGH REDSHIFTS

Dust in galaxies is produced in the ejecta of explosive core-collapse
supernovae (CCSNe) and in the quiescent winds of asymptotic giant branch (AGB)
stars which are not massive enough to end their life as CCSNe. Refractory
elements locked up in dust are returned to the gas phase by thermal and
kinetic sputtering and evaporative grain-grain collisional processes in
supernova blast waves (Jones, 2004). Star formation removes both, gas and dust
from the ISM. The equations governing the evolution of the dust and gas masses
in the ISM of galaxies were presented in Dwek & Cherchneff (2011). At high
redshifts, CCSNe dominate the rate of dust production at a rate given by
$dM_{d}/dt=\widetilde{Y_{d}}\,R_{SN}$, where $\widetilde{Y_{d}}$ is the IMF-
averaged dust yield in CCSNe (see Dwek et al., 2007, Table 2 for the maximum
attainable dust yield as a function of stellar mass), and
$R_{SN}\propto\psi_{0}$ is the IMF-dependent rate of CCSNe. CCSNe are also the
main source of grain destruction in the ISM, at a rate given by
$M_{d}/\tau_{d}$, where $\tau_{d}(t)$ is the dust lifetime, given by (Dwek &
Scalo, 1980):

$\tau_{d}={M_{d}(t)\over\widetilde{m_{d}}(t)\,R_{SN}(t)}={M_{g}(t)\over\widetilde{m_{g}}\,R_{SN}(t)}$
(4)

where $\widetilde{m_{d}}$$(t)$ is the time-dependent total mass of refractory
elements locked in dust that is returned back to the gas phase by a single
supernova remnant, $\widetilde{m_{g}}$$\equiv$
$\widetilde{m_{d}}$$(t)/Z_{d}(t)$ is the effective mass of interstellar gas
that is totally cleared of dust, where $Z_{d}(t)\equiv M_{d}/M_{g}$ is the
dust-to-gas mass ratio in the ISM, and $M_{g}$ is the total gas mass. The dust
destruction rate is then given by:

${dM_{d}\over
dt}=\widetilde{m_{d}}(t)\,R_{SN}(t)=Z_{d}(t)\widetilde{m_{g}}\,R_{SN}(t)$ (5)

The value of $\widetilde{m_{g}}$ is approximately time-independent and about
equal to $10^{3}$ $M_{\odot}$ (Dwek et al., 2007), and $Z_{d}\approx 0.007$ in
the solar neighborhood (Zubko et al., 2004). The total mass of dust destroyed
by a single SNR is therefore $\sim 7$ $M_{\odot}$, significantly larger than
the total $\sim 1$ $M_{\odot}$ of condensible elements in their ejecta. In the
local universe, CCSN are therefore net destroyers of interstellar dust.

In a dust-free galaxy, CCSNe are obviously net sources of interstellar dust.
There is therefore a limiting value of $Z_{d}\approx 1\times 10^{-4}$ for
which the grain destruction rate by SNR is balanced by their formation rate,
assuming a 10% condensation efficiency in the ejecta. The initial rate of dust
enrichment in a galaxy depends therefore on: (1) the stellar IMF - a top-heavy
IMF will have a higher IMF-averaged dust yield; and (2) the evolution of the
gas mass in the galaxy, which in the instantaneous recycling approximation is
given by:

$M_{g}(t)=M_{g}(0)-\left[{\left<m_{ej}\right>\over\left<m\right>}\right]\,\int_{0}^{t}\psi(t^{\prime})\,dt^{\prime}+\left({dM_{g}\over
dt}\right)_{inf}$ (6)

where $M_{g}(0)$ is the initial gas mass, $\left<m_{ej}\right>$ and
$\left<m\right>$ are, respectively, the IMF-averaged ejecta and initial
stellar mass, and $(dM_{g}/dt)_{inf}$ is the infall rate of the gas into the
galaxy.

Dust masses derived from evolutionary models depend on $\psi_{0}$ and
$\widetilde{m_{g}}$, whereas dust masses derived from fitting the UVO-IR SED
depend only on $\psi_{0}$. Figures 2e-f depict these two distinctly derived
dust masses as a function of the stellar mass derived from the corresponding
population synthesis and dust evolution models at $t=500$ Myr, the epoch of
observations. Figure 2e represents the results for the LowZ IMF and an initial
gas mass of $1\times 10^{10}$ $M_{\odot}$, and Figure 2f for the Kroupa IMF
and an initial gas mass of $5\times 10^{10}$ $M_{\odot}$ with no infall. The
solid black lines show the relation between $M_{d}$ and stellar mass for the
dust evolutionary models with selected lines labeled by the value of
$\widetilde{m_{g}}$ (in $M_{\odot}$). Tick marks along the lines indicate the
values of $\psi_{0}$. The models were calculated assuming that all refractory
elements condensed in the mass outflow from AGB stars, but that only 10% did
in the ejecta of CCSN. The thick dashed lines show the derived dust masses in
the absence of any grain destruction. The rate of grain destruction depends on
the dust-to-gas mass ratio in the ISM, and therefore on the choice of the
initial mass of the gas reservoir. The line connecting the filled and open
diamonds represents the relation between the dust mass derived from the SED
fitting to the stellar mass derived from the population synthesis models. The
diamonds are labeled by the value of $\psi_{0}$. The hatched region gives the
dust masses for all combinations of $\psi_{0}$ and $\widetilde{m_{g}}$
considered in this paper. Viable SF scenarios are those for which dust masses
derived from SED fitting are equal to those derived from the evolutionary
models (hatched regions). The resulting constraints on the SF scenarios and
dust and stellar masses are summarized in the 3rd and 4th column of Table 1.

In general, SF scenarios characterized by a LowZ IMF produce the inferred dust
mass at lower SFR and lower stellar masses. Scenarios that require low values
of $\widetilde{m_{g}}$ imply dust lifetimes that are significantly larger than
those expected for a homogeneous ISM, implying a clumpy ISM (Dwek & Scalo,
1979; Dwek et al., 2007).

Figure 3 depicts the evolution of the dust for different values of the SFR and
the stellar IMF, highlighting the role of the gas mass in determining the dust
production rate. The models assume no infall, so the gas mass is only
determined by its initial value and net consumption rate by stars. Models with
large initial gas masses produce dust at a higher rate than those with a lower
initial gas mass. Eventually, the dust-to-gas mass ratio will reach the
critical value of $\sim 0.001$ after which the rate of grain destruction by
CCSN exceeds the rate of dust formation in their ejecta. Galaxies with large
initial gas mass have therefore a faster rate of dust enrichment. However,
they will also be rarer objects since lower mass galaxies are more readily
assembled at $z\approx 10$ than the more massive ones (Somerville et al.,
2008; Behroozi et al., 2013).

##  4 THE ULTRAVIOLET SILICATE BREAK

The galaxy’s intrinsic spectrum was forced to reproduce the observed UVO
emission (see Figure 2c). It is therefore of interest to examine how the
derived mass absorption coefficient of the dust compares with that of
astrophysical dust grains. Figure 4 compares the wavelength dependence of
$\kappa(\lambda)$ with that of 0.01 $\mu$m radii silicate and amorphous carbon
grains, with optical constants from Li & Draine (2001). The mass absorption
coefficient shows a sharp rise at $\lambda\lesssim 0.15$ $\mu$m. The rise is
similar to that of the silicate grains but at shorter wavelengths. A good
match would require the galaxy to be at a somewhat lower redshift than that
inferred from the UVO SED fitting. This suggests that young galaxies, in which
silicates are the dominant dust component, will exhibit a sharp drop in their
spectrum around 0.15 $\mu$m caused by this rising silicate UV opacity. This
“UV silicate break” could be mistaken for an intrinsic Lyman break at
$\lambda=0.0912$ $\mu$m, leading to an overestimate in the photometric
redshift determination of galaxies. Such galaxies may also exhibit a break in
their IR spectrum that is associated with the 9.7 and 18 $\mu$m silicate
absorption features. This “IR silicate break” can be used to photometrically
select galaxies in the $z\approx 1-2$ range (Takagi & Pearson, 2005), or to
estimate the photometric redshift of high-$z$ galaxies.

##  5 SUMMARY

The evolution of dust at redshifts above $\sim 10$ differs fundamentally from
that at lower redshifts: (1) at these redshifts, CCSNe are the dominant
sources of interstellar dust. The enrichment of the ISM with carbon and carbon
dust is delayed until the most efficient carbon producing AGB stars have
evolved off the main sequence. The composition of the ISM and its dust may
therefore be significantly different from that in lower redshift galaxies
(Dwek, 1998). The redshift interval over which this effect can be observable
may be quite narrow, depending on the stellar IMF and the mass range of carbon
rich stars; (2) at low metallicities, hence low dust-to-gas mass ratios, CCSNe
are net producers of interstellar dust. This is in sharp contrast to lower
redshift galaxies, where CCSNe destroy more dust than they produce, which has
led to the suggestions that most of the interstellar dust in galaxies must be
reconstituted by cold accretion in the ISM (Dwek & Scalo, 1980; Zhukovska et
al., 2008; Draine, 2009; Calura et al., 2010; Valiante et al., 2011; Gall et
al., 2011a; Zhukovska, 2014); (3) the efficient production of dust in the
high-z universe is also greatly facilitated by the fact that at low-
metallicities the stellar IMF is biased towards the formation of massive
stars, further reducing the relative importance of lower mass AGB stars to the
production of carbon in the early universe.

Analysis of the SED of MACS1149-JD, including the 2 mm (190 $\mu$m restframe)
flux from its GISMO counterpart, shows that dust formation proceeds at rapid
rate at the earliest stages of galaxy evolution. Dust masses of
$\sim(2-7)\times 10^{7}$ $M_{\odot}$ can be readily produced within 500 Myr of
evolution with SFR as low as $5-8$ $M_{\odot}$ yr-1, without the need to
resort to growth in the ISM. This dust exhibits a sharp rise in opacity at UV
wavelengths around 0.15 $\mu$m, characteristic of very small silicate grains
expected to form in the ejecta of CCSNe (Nozawa et al., 2010; Cherchneff &
Dwek, 2010). The sharp rise produces a ”UV silicate break” in the spectra of
very young galaxies, which can be mistaken for a Lyman break, leading to an
overestimate of their photometric-based redshifts. The existence of high-
redshift, actively star-forming, galaxies with little dust, such as Himiko at
$z=6.62$ (Ouchi et al., 2013), shows that the dust abundance in galaxies
strongly depends on their star formation history, the efficiency of grain
destruction processes, and the presence of gas infalls and dusty outflows. Our
analysis shows that dust can form following the death of the first massive
stars, but its survival and abundance will depend on the particular physical
processes operating in each individual galaxy. Finally, independent
confirmation of the association of the GISMO 2mm source with MACS1149-JD, will
render this galaxy the youngest dust forming galaxy, with profound
implications for the onset of radiative, thermal, and chemical processes in
the early universe.

Figure 1: Left: The GISMO 2 mm signal-to-noise map of the MACS1149-JD field.
The $4.1\sigma$ source near the center of the field is is $\sim
8^{\prime\prime}$ (or $<0.5$ beam widths [FWHM] = $17.5^{\prime\prime}$) from
the Hubble location of MACS1149-JD (red cross), and therefore consistent with
being the same source. Two other sources with S/N $>$ 3 are present in the
field. Right: HST WFC3 image of MACS1149-JD at 0.85, 1.05, and 1.60 $\mu$m
(blue, green, red respectively). MACS1149-JD (arrowed) can be seen as the only
extremely red source (a dropout at 0.85 and 1.05$\mu$m). The field shown here
is much smaller than the GISMO field. The blue circle indicates the
$24.7^{\prime\prime}$ FWHM of the smoothed GISMO beam centered on the nominal
location of the GISMO source.

Figure 2: (a) The star formation rates as a function of cosmic age considered
in this study have time scales of $t_{exp}=100$ or 300 Myr. At the time when
MACS1149-JD is observed ($t=500$ Myr) the SFR has fallen to 0.092 or 0.86 of
the peak SFR, $\psi_{0}$; (b) the spectrum of the galaxy at $t=500$ Myr for
the different SFRs shown in (a) for the Low-metallicity and Kroupa IMF. (c)
models for the intrinsic spectrum must satisfy the energy constraint,
requiring the absorbed stellar luminosity to be equal to the reradiated IR
Luminosity. The two examples depicted in the figure are characterized by
values of $\psi_{0}=10$ and 40 $M_{\odot}$ yr-1, and corresponding dust
temperatures of $T_{d}=44$ and 70 K; (d) depiction of SF scenarios that
satisfy the energy constraint. The scenarios are characterized by their value
of $t_{exp}$, $\psi_{0}$, and the stellar IMF. The IR luminosity (red curve)
is characterized by its dust temperature. The dashed red line indicates the
lower limit on $\psi_{0}$ for the different SF scenarios (see Table 1). The
filled circles on the two horizontal lines illustrate the connection between
$\psi_{0}$ and $T_{d}$ for the two cases shown in Figure 2c; (e)-(f)
Comparison of modeled stellar and dust masses (hatched regions) and SED
determined masses (diamonds) as a function of $\psi_{0}$ (vertical ticks) and
$\widetilde{m_{g}}$ (black lines). See text for details. Figure 3:  The
evolution of the dust mass for a closed box model with a low-metallicity IMF
and $\psi_{0}=2$ $M_{\odot}$ yr-1. The bold solid lines assume no grain
destruction. The thin solid and dashed lines are models with different initial
gas mass. The evolution of the gas mass determines the rate of grain
destruction. Large gas masses imply low dust-to-gas mass ratios and lower rate
of grain destruction [eqs. (5) and (6)]. The figure also shows that AGB stars
do not contribute to the dust mass. Figure 4:  The wavelength dependence of
$\kappa(\lambda)$, derived from forcing the intrinsic spectra to reproduce the
observed galaxy’s UVO spectrum, is compared to the mass absorption coefficient
of 0.01 $\mu$m radius astronomical silicate and amorphous carbon grains. The
curves are labeled by the value of $\psi_{0}$ in units of $M_{\odot}$ yr-1.
For a given optical depth and dust mass, $\kappa$ is proportional to the
projected area of the disk. Plotted values assume an area of 1 kpc2. The
derived mass absorption coefficient exhibits a similar rise at UV wavelengths
as the small silicates grains. Young galaxies with a silicate dominated
population of interstellar dust grains will therefore have a break in their
spectrum around 0.15 $\mu$m, which can be confused with the Lyman break.

Table 1: Viable star formation scenarios following successive model
constraints 1,21,2footnotemark:

|  | Energy constraint | Mass production constraint: $M_{d}(SEDfit)=M_{d}(evol)$
---|---|---|---
SF scenario |  | $L_{abs}=L_{IR}$ |  $M_{g}(0)=1\times 10^{10}$ $M_{\odot}$ |  $M_{g}(0)=5\times 10^{10}$ $M_{\odot}$
$t_{exp}=100$ Myr | Low-Z IMF | $\psi_{0}\gtrsim 30$ | $\psi_{0}\gtrsim 40$ | $\psi_{0}\approx 40-50$
|  |  | M⋆$\gtrsim 7\times 10^{8}$ | M⋆$\approx(7-10)\times 10^{8}$
|  |  | $M_{d}\lesssim 6\times 10^{7}$ | $M_{d}\approx(7-3)\times 10^{7}$
| Kroupa IMF | $\psi_{0}\gtrsim 45$ | no viable models | no viable models
$t_{exp}=300$ Myr | Low-Z IMF | $\psi_{0}\gtrsim 4$ | $\psi_{0}\approx 5-35$ | $\psi_{0}\approx 5-8$
|  |  | M⋆$\approx(2-12)\times 10^{8}$ | M⋆$\approx(2-3)\times 10^{8}$
|  |  | $M_{d}\approx(70-5)\times 10^{6}$ | $M_{d}\approx(7-2)\times 10^{7}$
| Kroupa IMF | $\psi_{0}\gtrsim 10$ | no viable models | $\psi_{0}\gtrsim 35$
|  |  |  | M⋆$\gtrsim 1\times 10^{10}$
|  |  |  | $M_{d}\lesssim 9\times 10^{6}$

11footnotetext: Values of $\psi_{0}$ are in units of $M_{\odot}$ yr-1.

22footnotetext: Values of the stellar and dust masses, M⋆ and $M_{d}$,
respectively, and are in units of $M_{\odot}$.

Acknowledgements \- This work was supported through NSF ATI grants 1020981 and
1106284 (J.S., T.S., A.K. and the GISMO observations). IRAM is supported by
INSU/CNRS (France), MPG (Germany) and IGN (Spain). ED and R.G.A. acknowledges
support of NASA-ROSES-ATP2012. We acknowledge the comments made by the referee
which have led to a more detailed discussion on the origin of dust in the
early universe. E.D. thanks Rachel Somerville for a helpful discussion.

## References

  * Behroozi et al. (2013) Behroozi, P. S., Wechsler, R. H., & Conroy, C. 2013, ApJ, 770, 57
  * Bromm & Larson (2004) Bromm, V. & Larson, R. B. 2004, ARA&A, 42, 79
  * Calura et al. (2010) Calura, F., Recchi, S., Matteucci, F., & Kroupa, P. 2010, MNRAS, 406, 1985
  * Cherchneff & Dwek (2010) Cherchneff, I. & Dwek, E. 2010, ApJ, 713, 1
  * Cox & Mathews (1969) Cox, D. P. & Mathews, W. G. 1969, ApJ, 155, 859
  * da Cunha et al. (2013) da Cunha, E., Groves, B., Walter, F., et al. 2013, ApJ, 766, 13
  * Draine (2009) Draine, B. T. 2009, in Astronomical Society of the Pacific Conference Series, Vol. 414, Cosmic Dust - Near and Far, ed. T. Henning, E. Grün, & J. Steinacker, 453
  * Dwek (1998) Dwek, E. 1998, ApJ, 501, 643
  * Dwek & Cherchneff (2011) Dwek, E. & Cherchneff, I. 2011, ApJ, 727, 63
  * Dwek et al. (2007) Dwek, E., Galliano, F., & Jones, A. P. 2007, ApJ, 662, 927
  * Dwek & Scalo (1979) Dwek, E. & Scalo, J. M. 1979, ApJ, 233, L81
  * Dwek & Scalo (1980) Dwek, E. & Scalo, J. M. 1980, ApJ, 239, 193
  * Fioc & Rocca-Volmerange (1997) Fioc, M. & Rocca-Volmerange, B. 1997, A&A, 326, 950
  * Gall et al. (2011a) Gall, C., Andersen, A. C., & Hjorth, J. 2011a, A&A, 528, A13+
  * Gall et al. (2011b) Gall, C., Hjorth, J., & Andersen, A. C. 2011b, A&A Rev., 19, 43
  * Jones (2004) Jones, A. P. 2004, in ASP Conf. Ser. 309: Astrophysics of Dust, ed. A. N. Witt, G. C. Clayton, & B. T. Draine, 347–+
  * Kovács et al. (2010) Kovács, A., Omont, A., Beelen, A., et al. 2010, ApJ, 717, 29
  * Kroupa (2001) Kroupa, P. 2001, MNRAS, 322, 231
  * Kroupa et al. (2013) Kroupa, P., Weidner, C., Pflamm-Altenburg, J., et al. 2013, in Planets, Stars and Stellar Systems. Volume 5: Galactic Structure and Stellar Populations, ed. T. D. Oswalt & G. Gilmore (Springer Netherlands), 115
  * Li & Draine (2001) Li, A. & Draine, B. T. 2001, ApJ, 554, 778
  * Lucy et al. (1991) Lucy, L. B., Danziger, I. J., Gouiffes, C., & Bouchet, P. 1991, in Supernovae. The Tenth Santa Cruz Workshop in Astronomy and Astrophysics, held July 9-21, 1989, Lick Observatory. Editor, S.E. Woosley; Publisher, Springer-Verlag, New York, 1991. LC # QB856 .S26 1989. ISBN # 0387970711. P. 82, 1991, ed. S. E. Woosley, 82–+
  * Marks et al. (2012) Marks, M., Kroupa, P., Dabringhausen, J., & Pawlowski, M. S. 2012, MNRAS, 422, 2246
  * Nozawa et al. (2010) Nozawa, T., Kozasa, T., Tominaga, N., et al. 2010, ApJ, 713, 356
  * Osterbrock & Ferland (2006) Osterbrock, D. E. & Ferland, G. J. 2006, Astrophysics of gaseous nebulae and active galactic nuclei (University Science Books)
  * Ouchi et al. (2013) Ouchi, M., Ellis, R., Ono, Y., et al. 2013, ApJ, 778, 102
  * Planck Collaboration et al. (2013) Planck Collaboration, Ade, P. A. R., Aghanim, N., et al. 2013, ArXiv e-prints
  * Somerville et al. (2008) Somerville, R. S., Hopkins, P. F., Cox, T. J., Robertson, B. E., & Hernquist, L. 2008, MNRAS, 391, 481
  * Staguhn et al. (2013) Staguhn, J. G., Kovacs, A., Arendt, R. G., et al. 2013, ArXiv e-prints
  * Takagi & Pearson (2005) Takagi, T. & Pearson, C. P. 2005, MNRAS, 357, 165
  * Valiante et al. (2011) Valiante, R., Schneider, R., Salvadori, S., & Bianchi, S. 2011, MNRAS, 416, 1916
  * Venemans et al. (2012) Venemans, B. P., McMahon, R. G., Walter, F., et al. 2012, ApJ, 751, L25
  * Zheng et al. (2012) Zheng, W., Postman, M., Zitrin, A., et al. 2012, Nature, 489, 406
  * Zhukovska (2014) Zhukovska, S. 2014, A&A, 562, A76
  * Zhukovska et al. (2008) Zhukovska, S., Gail, H., & Trieloff, M. 2008, A&A, 479, 453
  * Zubko et al. (2004) Zubko, V., Dwek, E., & Arendt, R. G. 2004, ApJS, 152, 211


# Zero-Shot Language Transfer vs Iterative Back Translation for Unsupervised
Machine Translation

Aviral Joshi, Chengzhi Huang, Har Simrat Singh
(aviralj, chengzhh<EMAIL_ADDRESS>
###### Abstract

This work focuses on comparing different solutions for machine translation on
low resource language pairs, namely, with zero-shot transfer learning and
unsupervised machine translation. We discuss how the data size affects the
performance of both unsupervised MT and transfer learning. Additionally we
also look at how the domain of the data affects the result of unsupervised MT.
The code to all the experiments performed in this project are accessible on
Github.

##  1 Introduction

Unsupervised Machine translation(UMT) is the task of translating sentences
from a source language to a target language without the help of parallel data
for training. UMT is especially useful when translating to and from low
resource languages for example English to Nepali or vice-versa. Since it is
assumed that there is no access to parallel data between two languages,
unsupervised MT techniques focus on leveraging monolingual data to learn a
translation model. More generally there are three data availability scenarios
that can dictate the choice of the training procedure and the translation
model to be used. These scenarios are as follows:

  1. 1.

We have no parallel data for X-Z but have a large monolingual corpus.
Unsupervised Machine translation is very well suited for this scenario.

  2. 2.

We do not have parallel data for X-Z but we have parallel data for Y-Z and a
good amount of monolingual data. Zero-Shot language transfer can be more
effective in this scenario.

  3. 3.

Finally, we have some parallel data for X-Z and monolingual data for X and Z.
In this scenario semi-supervised approaches that leverage both monolingual and
parallel data can outperform the 2 other approaches mentioned.

Here X represents the source language (the language to transfer from), Z
represents the target language (the language to transfer to) and Y represents
a pivot language (an intermediate language) which is similar to the target
language Z. In a low resource scenario there is little to no parallel data
between X and Z.

The first 2 data availability scenarios mentioned above are generally more
challenging due to lack of presence of parallel data between source and target
languages and are explored in greater detail in this work. Futhermore, our
report also discuss the impact of mismatch in dataset domains for training and
evaluation and its effect on translation. Our report is structured as follows.
We first introduce related work in the domain of unsupervised machine
translation with back translation and language transfer while also discussing
works that detail the impact of domain mismatch in training and evaluation
datasets for UMT in Section 2. After which we describe the experimental setup
in Section 3. Moving on we describe the results of our experiments on low-
resource language pairs with varying training data sizes and also analyze the
impact of out of domain datasets on the quality of translation in Section 4.
Finally, we conclude with possible directions for future work in the domain of
UMT for low-resource languages.

##  2 Related Work

###  2.1 Unsupervised Machine Translation Using Monolingual Data

Recent advancement in Machine Translation can be attributed to the development
of training techniques that allow deep learning models to utilize large-scale
monolingual data for improving performance on translation while reducing
dependency on massive amounts of parallel data. Of the several attempts to
leverage monolingual data three techniques – back-translation, language
modelling and Auto-Encoding – in particular standout as the most effective and
widely accepted strategies to do so.

Sennrich et. al. [20] first introduced the idea of iteratively improving the
quality of a machine translation model in a semi-supervised setting by using
“back-translation”, the idea revolved around training an initial sub-optimal
translation system with the help of available parallel data and then utilizing
it to translate many sentences from a large monolingual corpus on the target
side to the source language to generate augmented parallel-data for training
the original translation system.

Gulcehre et. al [6] demonstrated an effective strategy to integrate language
modelling as an auxiliary task with Machine Translation improving the
performance on translation quality between both high resource languages and
low resource language pairs. A more recent work by Lample. et. al [14]
introduced a Cross Lingual Model(XLM) that utilized a Cross Lingual Language
modelling pretraining objective to improve performance on translation.

Previous works have also looked at the task of translation from the
perspective of an Auto-Encoder to utilize monolingual data. Cheng et. al.[3]
introduced an approach which utilized an Auto-Encoder to reconstruct sentences
in monolingual data where the encoder translates input from source to target
language and the decoder works to translate sentences back to the source
language. Parallel data is utilized in conjunction to monolingual data to
learn the translation system. This method was shown to outperform the original
back-translation system proposed by Sennrich et. al.

Unlike the previously described semi-supervised Lample et. al. [15] introduce
an effective strategy to perform completely unsupervised machine translation
in the presence of a large monolingual corpus. Their work leverages a
Denoising Auto-Encoder (DAE) to reconstruct denoised sentences from the
sentences in monolingual corpora which are systematically corrupted to train
encoder and decoders for both source and target language. To map the latent
DAE representations of the two languages into the same space. To aid
translation the authors also introduce an additional adversarial loss term.
The DAE along with iterative back-translation is utilized to train a
completely unsupervised translation system.

###  2.2 Transfer Learning - Zero shot Translation

Even though UMT has shown to perform well on similar language pairs with good
amount of monolingual data, Artetxe et al. argued a scenario without any
parallel data and abundant monolingual data is unrealistic in practice. One
common alternative in this scenario is to avoid using monolingual data with
zero-shot transfer learning. Zero-shot translation concerns the case where
direct (src-tgt) parallel data is lacking but there is parallel data between
the target language and a similar language to source language.

Transfer Learning is firstly proposed for NMT by Zoph et al., who leverage a
high resourced parent model to initialize the low-resourced child model.
Nguyen and Chiang and Kocmi and Bojar utilizes shared vocabularies for source
and target language to improve transfer learning, in addition, mitigate the
vocabulary mismatch by using cross-lingual word embedding. These methods show
great performance in transfer learning for low resource languages, they show
limited effects on zero-shot transfer learning.

Multilinguality has been extensively studied and has been applied to the
related problem of zero-shot translation (Johnson et al.; Firat et al.;
Arivazhagan et al.). Liu et al. showed some initial results on multilingual
unsupervised translation in the low-resource setting. Namely, Pretrained
mBART25 is finetuned on transfer language to target language pair, and is
directly to the source to target pairs. mBART obtained up to 34.1 (Nl-En) with
zero-shot translation and works well when fine-tuning is also conducted in the
same language family. Specifically, it acheives 23 BLEU points for Ro-En using
Cs as the transfer language, and 18.9 BLUE points for Ne-En using Hi as the
transfer language.

###  2.3 Unsupervised Machine Translation on Out-of-Domain Datasets

Owing to increasing popularity of unsupervised MT and the techniques involved,
adaptation of UMT to translate data from domains where a large amount of
parallel data is not obtainable is also gaining importance. One of the main
issues associated with this is the feasibility of using pre-trained models.
Most of the pre-trained models, even though trained under supervised learning
paradigm, fail to achieve significant translation accuracy on a domain which
was non-existent in the training data. Unsupervised models also follow the
same pattern. Marchisio et al. [18] show some significant results and comment
on the domain similarity of test and training data in State of the Art models.
The authors use United Nations Parallel Corpus (UN) [22], News Crawl and
CommonCrawl datasets for their work. The model was trained solely on NewsCrawl
while being evaluated on all 3 datasets. A general trend of reduction in
performance was observed across various language pairs. Extensive work has
been done to address this problem. Dou et al. [4] propose a Domain-Aware
feature embeddings based approach and achieve improvements over previous
existing methods. Luong and Manning [17] adopt an attention based approach to
fine tune a pretrained model on out of domain data. Sennrich et al. [20] show
that for out of domain data back translation helps the models learn a whole
lot of words previously non existent in the vocabulary which could be the
reason why out of domain back translation based UMT fails. Our experiments
follow a similar line of work and show that fine tuning a pre-trained
translation model does not effectively help with out of domain data.

##  3 Experiments

###  3.1 Proposed Experiments

In our experiments, we focus on how the size of training data will affect the
performance of both zero-shot transfer learning and Unsupervised Machine
Translation with iterative back translation. Additionally, we conduct an
extensive empirical evaluation of unsupervised machine translation using
dissimilar domains for monolingual training data between source and target
languages and with diverse datasets between training data and test data.

  1. 1.

Unsupervised MT using iterative back translation: Our first experiment
involves UMT using back translation(BT) with the XLM model. Here we
demonstrate the feasibility of UMT with BT for low-resource language pairs and
also how variation in the monolingual data size effects the translation
quality for similar and dissimilar language pairs.

  2. 2.

Zero Shot Language Transfer: Our second experiement involves zero-shot
translation using mBART model. We finetune on related language (X
$\rightarrow$ En) and apply it directly to the source-target (Y $\rightarrow$
En) language pair. Here we illustrate how the size of finetuning data affect
both finetuning scores (finetuned on X $\rightarrow$ En and tested on X
$\rightarrow$ En) and transferring scores (finetuned on X $\rightarrow$ En and
tested on Y $\rightarrow$ En).

  3. 3.

Domain Dissimilarity Our third experiment analyses the performance of a model
which is trained on a different dataset and is tested on sentences from an
entirely different domain. We use XML model and perform back translation on
pre-trained models for all language paris X $\rightarrow$ En to fine tune it
on the required dataset. Back translation steps include X $\rightarrow$ En and
En $\rightarrow$ X. The model is then tested for the previously mentioned
steps on a dataset sourced from a completely different domain.

###  3.2 Models

mBART [16] is used for experiments with zero-shot transfer learning, XLM [14]
is used for experiments with unsupervised machine translation.

  1. 1.

mBART is the first method for pre-training a complete sequence to sequence
model by denoising full texts in multiple languages. Pre-training is done on
CC25 data set and pretrained model can be directly fine-tuned for both
supervised and unsupervised MT. It also enables a smooth zero-shot transfer to
language pairs with no bi-text data. In our experiments, we use mBART25, which
is pretrained on all 25 lagnauges. Ideally, for a fair comparison unsupervised
BT should be also done using mBART25, however, the authors did not provide
their implementation for BT and so we shift to the XLM model for experiments
with BT since its implementation was more accessible.

  2. 2.

XLM is a cross lingual language model developed for both supervised and
unsupervised machine translation tasks. XLM model uses shared sub word
vocabulary and is trained on 3 different language modelling objectives of
Causal language modeling, Masked Language modeling and Translation language
modeling. The model is essentially Transformer based with 1024 sub units and 8
heads. Cross lingual XLM is trained on 15 languages and is shown to achive
SoTA on all the different languages. Our motivation behind using XLM was that
a pretrained XLM model was available for the languages of our choice. It
supported 2 way backtranslation and was comparatively easier to train and fine
tune than most other multilingual models.

###  3.3 Dataset

Two language pairs were chosen for the experiments where we conduct an
investigation of the impact of size of monolingual training data on
translation quality between a similar and dissimilar language pair (Ro
(Romanian) - En (English) and Ne (Nepali) - En respectively. For Ro & En, data
from wikipedia was used. We perform random sampling to obtain monolingual
sentences from the monolingual data provided by the authors of the XLM model
(This was originally obtained by scraping articles from Wikipedia). For Nepali
the monolingual data was obtained from the Flores data corpus.

For zero-shot transfer learning, we conducted experiments on Ro to En with Cs
(Czech) being the transfer language and Ne to En with Hi (Hindi) being the
transfer language. Cs - En parallel training data is taken from Europarl v7
[11] with 668,595 sentence pairs, Ro - En test data is taken from WMT16 test
set for Romanian and English, consisting of 1999 sentence pairs; Hi-En
parallel training data is taken from IITB Bombay English-Hindi corpus[13] with
1,609,682 sentence pairs, Ne-En test data is taken from [7] with 2835 test
sentences.

To analyse the effect of domain mismatch training and testing, we use Fr
(French), Genrman (De) and Romanian (Ro). In all the experiments the target
language is English (En). Back translation is performed both ways. The choice
for these specific languages was motivated by the criterion on language
similarity and availability of data. French and German are related to English,
albeit not closely but still possess a similarity in terms of language family
and closeness of vocabulary. Romanian was chosen as a low resource language,
distantly related to English. To simulate domain mismatch, the XLM model was
pretrained on WMT’16 which mostly has parallel data extracted from news
articles. The model was then fine tuned on monolingual data of the four
languages obtained from the Tatoeba challenge [21]. For French, English and
German 50K monolingual sentences were used, and for Romanian only 1999
sentences were available. Tatoeba dataset has sentences with minimum overlap
with any other available machine translation dataset. We used the sentences
extracted from Wikipedia pages present within the dataset. The validation and
testing set was extracted from WMT’16.

####  3.3.1 Experiment Details

We use the following tokenizers for both preprocessing training data and
evaluations for all three experiments:

  * •

Ne, Hi: We use Indic-NLP Library to tokenize both Indic language inputs and
outputs.[12]

  * •

Ro, Cs, En: We apply Moses tokenization and special normalization for English,
Czech and Romanian texts.

We conduct all the experiments for transfer learning on AWS `g4dn.xlarge`
instance, with `max-tokens` 512, `total-num-update` 80000 and dropout 0.3.
During the inference time, beam size is set to 5.

Similar to experiments with transfer learning all experiments for Back
Translation were conducted on a g4dn.xlarge instance the batch size was set to
32 and multiple iterations of back translation were run. The number of tokens
per batch were set to a maximum of 1500 and a embedding dimension of 1024 was
used. Other hyper-parameters were kept the same as the implementation provided
by the authors of XLM.

For domain mismatch experiment we use XLM model as well. We follow the Byte
Pair Encoding technique as used by the authors of XLM, and use fastBPE for the
same. Tokenization is done using Moses tokenization. The experiments were
conducted on AWS `p3.2xlarge` instances. The model used 6 layers of
transformers with 8 heads. The embedding dimension was 1024.

###  3.4 Evaluation Metrics

For all of our tasks, we use BLEU score calculated by `sacrebleu` as the
automatic metric to evaluate the translation performance. Specifically, we
compute the BLEU scores over tokenized text for both hypothesis outputs and
references as mentioned in Liu et al.

##  4 Results and Discussions

In this section we describe the results of the proposed experiments as
mentioned previously in Section 3 and discuss our finding to illustrate the
the impact of dataset size and dataset domain for Unsupervised Machine
Translation and data size for Transfer Learning.

###  4.1 Back Translation

As expected with most deep learning approaches, the trained model scales with
increase in training data (monolingual) as the BLEU show a general upward
trend as more data is made accessible to the XLM model. This is evident in the
results shown in Table 1 especially for the high similarity English-Romanian
language pair. Whereas, the performance on low similarity English-Nepali pair
is degenerate in all data availability scenarios. Even when the entire
training corpus is provided to the model. This result indicates that it might
not be feasible to perform unsupervised MT when language pairs have little or
no similarity using Back Translation along. In which case other approaches for
Unsupervised MT such as language transfer are better suited. More about
language transfer is discussed in Section 4.2.

A big improvement in BLEU score is seen when training data size is increased
from 10,000 to 100,000 for the high similarity language pair. The performance
increases steadily as more data is provided to the model while the rate of
increase slows down. A reasonable BLEU for the English-Nepali language pair
was reached with just 600,000 sentences from each language.

BLEU - Back Translation
---
Data Size | High Similarity | Low Similarity
| en | ro | en | ne
| $\rightarrow$ | $\leftarrow$ | $\rightarrow$ | $\leftarrow$
10K | 2.76 | 3.93 | 0.0 | 0.0
100K | 23.6 | 23.6 | 0.0 | 0.0
300K | 27.7 | 26.9 | 0.1 | 0.1
600K | 28.8 | 28.3 | 0.1 | 0.2
$>$10M | 33.3 | 31.8 | 0.1 | 0.5
Table 1: Comparison of Translation Quality between Similar and Dissimilar
language pairs and how they scale with increase in monolingual data.

(a)

(b)

Figure 1: Figures depicting the variation of translation quality with the
availability of monolingual data for English and Romanian

###  4.2 Transfer Learning

Zero shot transfer learning for mBART [16] works as following: pretrained
mBART is finetuned on a related language Y to target En and is directly
applied for testing on source X to target language En.

In order to investigate on how the size of training data will affect the zero-
shot BLEU points, we randomly sample 1K, 10K, 100K and 300K sentences from
Europart v7 [11] of Cs - En parallel training corpus and test the finetuning
model directly on Ro - En test data; We also randomly sample 10K, 100K, 300K
and 600K sentences from ITTB Bombay-English corpus [13] and test the
finetuning model directly on Ne - En test data. We present BLEU points of
finetuning models on both source-target language pairs (X \- En) and related-
target language pairs (Y \- En) for better observing the correlation between
them. Each experiment is trained using the same hyper-parameters.

Data Size |  Transfer Ro $\rightarrow$ En |  Finetune Cs $\rightarrow$ En
---|---|---
0 | 0.0 | 0.0
1K | 17.6 | 16.2
10K | 23.3 | 21.9
100K | 23.0 | 25.7
300K | 20.3 | 26.5
668K | 14.5 | 27.5
Table 2: BLEU points for Transfer Learning and directly finetuning of different data size, pretrained mBART25 is finetuned on Cs to En and directly transferred to Ro to En, we choose the best checkpoint weights validated on Cs to En dev set Data Size |  Transfer Ne $\rightarrow$ En |  Finetune Hi $\rightarrow$ En
---|---|---
0 | 0.0 | 0.0
10K | 11.7 | 12.8
100K | 14.0 | 18.7
300K | 13.6 | 19.1
600K | 13.2 | 19.3
1.68M | 15.1 | 23.4
Table 3: BLEU points for Transfer Learning and directly finetuning of
different data size, pretrained mBART25 is finetuned on Hi to En and directly
transferred to Ne to En, we choose the best checkpoint weights validated on Hi
to En dev set

It is worth noting that for the model finetuned on Cs - En, we observed a
generally negative correlation between transferring scores and finetuning
scores, as well as a negative correlation between the data size and
transferring scores. In contrast, for the model finetuned on Hi - En there is
a positive correlation between transferring scores and finetuning scores and a
positive correlation between the data size and transferring scores. One
hypothesis is that even though mBART achieves the highest transferring scores
for Ro - En when finetuned on Cs - En and they both belong to Indo-European
language family, they are still vastly different languages using orthographic
distance metric Heeringa et al. developed, with Czech being in the Slavic
Language sub-group and Romanian being in the Romance language sub-group.
Hence, finetuning on a larger data set would make mBART overfit towards the
finetuned language, thus hurts the performance of zero-shot translation on
source-target pair due to the limited capacity of mBART model itself. For Ne
and Hi are more closely related to each other and have large overlap in their
vocabularies, so finetuning on Hi will greatly benefit the performance of
zero-shot translation.

###  4.3 Comparison between Back Translation with Zero-Shot Transfer Learning

When does back translation work?  In Table 3, we do not have much increase in
performance when we increase the data size for monolingual data when training
on dissimilar language pairs (Ne and En). If there exists high quality similar
languages bi-text data, language transfer is able to beat the conventional
methods with BT. For similar language pairs (Ro and En), we do see the gain in
BLEU points when data size increases and can easily beat the zero-shot
transfer learning with 100K monolingual data.

When does transfer learning work? Note that in Table 2, the performance for
zero-shot transfer learning suffers from ”the curse of data size”, which means
simply increasing the size of finetuning data does not contribute to the
performance of transferring scores, especially when finetuning language (Cs)
is not close enough to the target language (Ro). Romanian is closest to
Catalan [8], however, not so much bi-text exists between Catalan and English
and it is also a out of vocabulary language for mBART, thus does not benefit
much from multilingual pretraining. On the contrary, for languages in Indic
Language family, increase in data size does help in both transferring scores
and direct finetuning scores. Compared with Back Translation, zero-shot
transfer learning outperforms it with just a small amount of bi-text for Hi to
En.

###  4.4 Domain Dissimilarity

Language Pair | Training Dataset
---|---
WMT’16 | Tatoeba(Wikipedia)
Fr $\rightarrow$ En | 32.8 | 27.38
De $\rightarrow$ En | 34.3 | 27.44
Ro $\rightarrow$ En | 18.3 | 1.74
En $\rightarrow$ Fr | 33.4 | 28.33
En $\rightarrow$ De | 26.4 | 21.93
En $\rightarrow$ Ro | 18.9 | 0.39
Table 4: Domain Mismatch

We present the results of domain mismatch experiments in table 4. As expected
there is a consistent decrease in the performance of the model across all
languages. The decrease is comparatively less for Fr and De then Ro. French
and German are borh high resource languages and the XLM model for both these
languages was pre trained as well as finetuned on 50K sentences. Whereas for
Romanian only 1999 sentences were available in the Tatoeba challenge. The size
of the validation and test data was also smaller. Using back translation in
this scenario did not prove to be ideal. This also suggests that even for high
resource languages the domain of the dataset is a big factor to determine the
overall translation efficacy of the model. XLM uses shared sub word vocabulary
which means there would be a lot of words which are differently used in
different domains and are thus harder for the model to predict. The results
for Romanian could have been better if transfer learning was used as the UMT
technique. The results are particularly interesting as for low resource
languages like Romanian, getting parallel domain specific data is very tough
and adapting a pre-trained model does not help in the same way as it does in
cases of high resource languages. We wish to explore this further with some
extra experiments as well.

##  5 Conclusion and Future-Work

In this work, we compare two different solutions for low resource language MT:
unsupervised MT with BT and zero-shot transfer learning. We focus on how the
size affects the performance for both unsupervised MT and transfer and how the
domain matters in unsupervised MT.

For transfer learning, we find that if the model is transferred from a ”not so
similar” language, like Cs $\rightarrow$ En in our experiment, the performance
for transfer learning can not be scaled up by simply increasing the size of
finetuning data. In contrast, for the closely related language, like languages
in Indic Language Family, we observe an opposite trend. However, the capacity
of mBART model has not yet been well investigated, and we still do not have a
clear definition of the similarity between transfer language and target
language. These could be the potential topics conducted in the future
resesarch.

Talking about domain mismatch, we wish to explore if transfer learning is a
better UMT technique for adapting a model to a different domain in case of low
resource languages. We also want to try more experiments to further solidify
and generalize our findings. There have been numerous researches done in this
field where attempts have been made to learn domain specific embeddings or
even model based changes to generalize across mismatching domains. We hope to
use some of the existing methods and try to boost the performance of models
specifically in case of low resource languages.

## References

  * Arivazhagan et al. [2019] Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Roee Aharoni, Melvin Johnson, and Wolfgang Macherey.  The missing ingredient in zero-shot neural machine translation, 2019.
  * Artetxe et al. [2020] Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, and Eneko Agirre.  A call for more rigor in unsupervised cross-lingual learning, 2020.
  * Cheng [2019] Yong Cheng.  Semi-supervised learning for neural machine translation.  In _Joint Training for Neural Machine Translation_ , pages 25–40. Springer, 2019.
  * Dou et al. [2019] Zi-Yi Dou, Junjie Hu, Antonios Anastasopoulos, and Graham Neubig.  Unsupervised domain adaptation for neural machine translation with domain-aware feature embeddings, 2019.
  * Firat et al. [2016] Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, and Kyunghyun Cho.  Zero-resource translation with multi-lingual neural machine translation, 2016.
  * Gulcehre et al. [2015] Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.  On using monolingual corpora in neural machine translation.  _arXiv preprint arXiv:1503.03535_ , 2015.
  * Guzmán et al. [2019] Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato.  Two new evaluation datasets for low-resource machine translation: Nepali-english and sinhala-english.  2019\.
  * Heeringa et al. [2013] Wilbert Heeringa, Jelena Golubovic, Charlotte Gooskens, Anja Schüppert, Femke Swarte, and Stefanie Voigt.  _Lexical and orthographic distances between Germanic, Romance and Slavic languages and their relationship to geographic distance_ , pages 99–137.  P.I.E. - Peter Lang, 2013.
  * Johnson et al. [2017] Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean.  Google’s multilingual neural machine translation system: Enabling zero-shot translation.  _Transactions of the Association for Computational Linguistics_ , 5:339–351, 2017.  doi: 10.1162/tacl˙a˙00065.  URL https://www.aclweb.org/anthology/Q17-1024.
  * Kocmi and Bojar [2018] Tom Kocmi and Ondřej Bojar.  Trivial transfer learning for low-resource neural machine translation.  _Proceedings of the Third Conference on Machine Translation: Research Papers_ , 2018.  doi: 10.18653/v1/w18-6325.  URL http://dx.doi.org/10.18653/v1/W18-6325.
  * Koehn [2005] Philipp Koehn.  Europarl: A Parallel Corpus for Statistical Machine Translation.  In _Conference Proceedings: the tenth Machine Translation Summit_ , pages 79–86, Phuket, Thailand, 2005. AAMT, AAMT.  URL http://mt-archive.info/MTS-2005-Koehn.pdf.
  * Kunchukuttan [2020] Anoop Kunchukuttan.  The IndicNLP Library.  https://github.com/anoopkunchukuttan/indic_nlp_library/blob/master/docs/indicnlp.pdf, 2020\.
  * Kunchukuttan et al. [2017] Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhattacharyya.  The IIT bombay english-hindi parallel corpus.  _CoRR_ , abs/1710.02855, 2017.  URL http://arxiv.org/abs/1710.02855.
  * Lample and Conneau [2019] Guillaume Lample and Alexis Conneau.  Cross-lingual language model pretraining, 2019.
  * Lample et al. [2017] Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato.  Unsupervised machine translation using monolingual corpora only.  _arXiv preprint arXiv:1711.00043_ , 2017.
  * Liu et al. [2020] Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer.  Multilingual denoising pre-training for neural machine translation, 2020\.
  * Luong and Manning [2015] Minh-Thang Luong and Christopher D Manning.  Stanford neural machine translation systems for spoken language domains.  In _Proceedings of the International Workshop on Spoken Language Translation_ , pages 76–79, 2015.
  * Marchisio et al. [2020] Kelly Marchisio, Kevin Duh, and Philipp Koehn.  When does unsupervised machine translation work?, 2020.
  * Nguyen and Chiang [2017] Toan Q. Nguyen and David Chiang.  Transfer learning across low-resource, related languages for neural machine translation, 2017.
  * Sennrich et al. [2016] Rico Sennrich, Barry Haddow, and Alexandra Birch.  Improving neural machine translation models with monolingual data, 2016\.
  * Tiedemann [2020] Jörg Tiedemann.  The tatoeba translation challenge – realistic data sets for low resource and multilingual mt, 2020.
  * Ziemski et al. [2016] Michał Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen.  The United Nations parallel corpus v1.0.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 3530–3534, Portorož, Slovenia, May 2016. European Language Resources Association (ELRA).  URL https://www.aclweb.org/anthology/L16-1561.
  * Zoph et al. [2016] Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight.  Transfer learning for low-resource neural machine translation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1568–1575, Austin, Texas, November 2016. Association for Computational Linguistics.  doi: 10.18653/v1/D16-1163.  URL https://www.aclweb.org/anthology/D16-1163.