# Equality of Opportunity and Opportunity Pluralism††thanks: This work is a
generalization of the Master’s dissertation I defended in September 2019 under
the supervision of Professor François Maniquet at UCLouvain. I am heavily
indebted to Professor Maniquet for his many suggestions and comments.

Giovanni Valvassori Bolgè
Center of Economic Research
at ETH Zürich
Leonhardstrasse 21
8092 Zürich
Switzerland
<EMAIL_ADDRESS>
(Last updated: )

This paper seeks to explore the potential trade-off arising between the
theories of Equality of Opportunity and Opportunity Pluralism. Whereas the
first theory has received much attention in the literature on Welfare
Economics, the second one has only recently been introduced with the
publication of the book by Joseph Fishkin, Bottlenecks: A New Theory of Equal
Opportunity. After arguing extensively that any notion of human flourishing is
incompatible with traditional theories of Equality of Opportunity, the author
proposes an alternative theory squarely based on a broad notion of human
development. This paper seeks to formalize the argument made in this book
through the lens of economic theory. My analysis suggests that traditional
theories of Equality of Opportunity are not incompatible with Opportunity
Pluralism.

Keywords: Equality of Opportunity, Education and Inequality, Distributive
Justice

JEL Classification: D60, D63, I24, I3

##  1 Introduction

The theoretical foundations of Equality of Opportunity (EOp, henceforth) might
be traced back to the pioneering contributions of the political philosopher
John Rawls (Rawls, (1958), Rawls, (1971)). By crucially integrating personal
responsibility with egalitarianism, Rawls paved the way for a large literature
in Political Philosophy; soon, other major contributions followed, most
notably, from Sen, (1980) and Dworkin, (1981).

More recently, the theory of EOp has found its way into normative economics,
owing to the work of John Roemer (Roemer, (1993), Roemer, (2000)). Ever since,
there has been a strong development both in the theoretical underpinnings and
in the empirical applications of EOp.111For a detailed survey, see Roemer and
Trannoy, (2016). According to Maniquet, (2017), these developments can be
rationalized in three different approaches within Economics.

The first approach is constituted by the theory of fair resource allocation.
Starting from the fact that individuals are heterogeneous along several
characteristics, economic theorists have focused on devising resource
allocation mechanisms which are compatible with different ethical principles
of social justice.

Building on the work of John Roemer, Fleurbaey and Maniquet, (2011) propose to
divide the relevant individual characteristics into two sets: the compensation
characteristics are those whose inequality between individuals is deemed as
ethically objectionable (e.g. innate abilities); the responsibility
characteristics are constituted by those individual traits for which
individuals should be held accountable (effort variables; e.g. schooling).
Several theories have been advanced to identify the cut between compensation
and responsibility variables. Although they differ in the particular ethical
principles which inspire them, they all agree that society should redistribute
resources in order to compensate the inequality deriving from the first set of
characteristics, while remaining neutral with respect to the inequality
stemming from the second set.222For a thorough discussion, see, inter alia,
Fleurbaey, (2008).

The second approach relates to the theory of income inequality measurement.
Following the groundbreaking contributions of Pigou, (1912) and Dalton,
(1920), economists have been able to decompose the income distribution as the
sum of different components, some of which calling for compensation and others
for neutrality by society (see Peragine, (2002) and Bourguignon et al.,
(2007)).

Measuring the extent to which the income distributions differ from one of
perfect equality due to ethically objectionable characteristics is the basis
of many different empirical studies, most notably the World Bank Report on
Equity and Development (World Bank, (2006)) as well as the thorough empirical
analysis for Latin America in Paes de Barros et al., (2009).

The third approach has been devised within the theory of inequality of
opportunity measurement. By considering the set of opportunities open to each
individual, economists have focused their analysis not only on the elements of
the sets eventually chosen by individuals, but also on the cardinality of the
sets themselves (e.g. Thomson, (2011); Kranich, (1998)).

More recently, an insightful critique of the theoretical foundations of EOp
has been published by the political theorist Joseph Fishkin in the book
Bottlenecks: A New Theory of Equal Opportunity (Fishkin, (2014)).

The author claims to advance a radically new way of conceptualizing EOp —
which he refers to as Opportunity Pluralism — building a novel theory squarely
on an account of human development.

Unlike traditional theories of EOp which — the author argues — have narrowly
focussed on the instrumental role of opportunities, Opportunity Pluralism
ascribes a central role to the developmental features of opportunities, namely
the broadly defined way in which opportunities shape individual development.

In particular, the first condition of an ideal society satisfying Opportunity
Pluralism is the presence of a “plurality of values and goals”, which I refer
to as Plurality of Opportunity (POp, henceforth). In the words of Fishkin,
(2014),

> “People in this society hold diverse conceptions of the good, of what kinds
> of lives and forms of flourishing they value, and of what specific goods and
> roles they want to pursue […].”333Fishkin, (2014), p. 135.

Importantly, Fishkin argues that POp is incompatible with traditional EOp
theories. His line of reasoning suggests the existence of a trade-off faced by
society: either society intervenes to equalize opportunities on a chosen scale
in the traditional sense or it promotes Opportunity Pluralism, by removing the
different types of bottlenecks which constrain the individual development of
its citizens.

The present paper aims at evaluating two salient critiques moved by Joseph
Fishkin to EOp theories through the lens of basic economic reasoning. First,
the purpose is to evaluate his claim regarding the impossibility of
incorporating the developmental role of opportunities into the standard
framework of EOp adopted by economists. Second, I investigate whether this
conceptualization of EOp is incompatible with POp, and therefore with
Opportunity Pluralism, as Fishkin argues.

In order to do this, I build a simple two-period, two-sector model in an
economy populated by agents with heterogeneous productive skills over the two
sectors. An egalitarian government seeks to equalize opportunities among
agents through educational investments in either of two schools, where skills
in each sector are nurtured.

The preferences of the agents are uncertain and endogenous to educational
investments. Agents do not know their preferences over the two sectors when
educational investments take place. Moreover, the realization of preferences
will depend on the bundles of opportunities received.

I show that a pluralistic opportunity structure, namely a society with two
schools, yields a higher social welfare than a unitary opportunity structure,
a society with a single school.

Therefore, my analysis shows that EOp implies a pluralistic opportunity
structure, the notion at the heart of POp, and consequently of Opportunity
Pluralism as well. As a result of this, I claim that the theories of EOp and
Opportunity Pluralism are not incompatible.

I structure the discussion as follows. In Section 2 and Section 3, I briefly
describe the main lines of reasoning of Fishkin’s contribution, articulating
them into two distinct critiques to EOp. Section 4 lays out the theoretical
framework through which I frame those critiques. Section 5 and Section 6
consider the nature and evaluation of the different education policies.
Section 7 concludes.

##  2 Opportunities and Human Development

Fishkin envisions human flourishing as a complex, multi-stage process in which
three sets of factors interact with each other: the environment, comprehensive
of developmental opportunities; the person, as constituted by his/her
capacities, traits and genetic heritage; his/her ambitions, goals and efforts,
namely the “multidimensional path" a person is embarking on as a result of the
previous layer of interactions between the environment and his/her personal
and social background.

Moreover, this process is iterative. Some unusual capacities at an early age
will affect the range of developmental opportunities provided by one’s family
and environment; importantly, also the converse holds. In the author’s own
words:

> “… our ambitions, goals and efforts do not emerge fully formed from the
> ether, but are instead products of our lived experience; they, in turn,
> influence other aspects of the processes by which we develop traits and
> capacities, convince others to recognize our capacities, prove “our" merit,
> and secure jobs and other social roles. Our decisions about how to direct
> our efforts are in part a function of the choice set of paths and options we
> see before us at each stage - as well as our own conclusions, mediated by
> the conclusions of others, about our merit and potential."444Ibid. p.115.

According to Fishkin, if we embrace this perspective with regard to human
development, the theoretical foundations of any framework for equalizing
opportunities begin to crumble. In fact, the author argues:

> “Equalizing developmental opportunities means that we ought to arrange the
> different bundles of developmental opportunities that children receive on
> some sort of scale; then we aim for state of affairs in which everyone has a
> bundle of opportunities of equal value, or as close as possible to equal
> value, on that scale."555Ibid. p.116.

This line of reasoning is fatally flawed because:

> “… the more carefully we think through the iterative interactions that make
> up human development, the less clear it becomes what ‘equal’ developmental
> opportunities could possibly mean."666Ibid. p.116.

As a result of this,

> “… it becomes impossible either to arrange the different bundles of
> opportunities on a single scale or to identify any one set of developmental
> opportunities that could function as a fair baseline of equality for
> everyone."777Ibid. p.116.

This difficulty, the common scale problem, is due to two other problems: the
incommensurability of bundles of opportunity and the endogeneity of
preferences.

By incommensurability, Fishkin means “the problem that different developmental
opportunities are valuable for different reasons, and some of those reasons
are incommensurable with one another."888Ibid. p.119.

This problem is compounded by the one of the endogeneity of preferences. This
builds directly on the model of iterative human development discussed above.
The layers of interactions between the environment, the genetic heritage and
the set of personal traits all contribute to shaping one’s values, goals and
ambitions. As a consequence,

> “… the endogeneity of our preferences and values — specifically, their
> dependence on our developmental opportunities and experiences — can make it
> difficult or impossible to say ex ante which bundles of developmental
> opportunities are best for a person."999Ibid. p.124.

I summarize this discussion as the first fundamental critique of Fishkin:

######  Critique 1.

It is impossible to build a metric of opportunities so as to take their
developmental role into account.

##  3 The Role of the Opportunity Structure and the Trade-off

The shape of a society’s opportunity structure — Fishkin argues — greatly
affects the degree to which the common scale problem binds. In the author’s
own words,

> “… the unitary or pluralistic character of the opportunity structure has an
> interesting effect on the commensurability of different bundles of
> opportunities. The more unitary the opportunity structure, the more often we
> will be able to say objectively that a given bundle of opportunities is
> better or worse than another."101010Ibid. p.126.

In the extreme case of the “big test society", to Fishkin refers to many times
throughout the book — namely, a fictitious society in which access to all
sought-after jobs and positions in adulthood depends on the score on a single
test undertaken at age 18 — only those skills essential to passing the test
are crucial. In this regard, deciding which bundles of developmental
opportunities are better or worse becomes much easier for an ideal, benevolent
planner. However, such a planner should advocate exactly the opposite, since a
unitary opportunity structure exacerbates all the intrinsic problems of equal
opportunity which Fishkin discusses in Part I of his book.

In the limiting case of the big test society, in fact, the problem of the
family binds even more tightly because parents have every incentive to
maximize the development opportunities for their children to score well on the
test. Hence, more affluent families will have an unfair advantage over poorer
ones.

This, in turn, triggers the problem of the merit and of the starting gate: in
a fascinating treatment of the subject, Fishkin argues extensively that there
will be “no fair way to equalize opportunities among people whose
developmental opportunities were unequal."111111Ibid. p.128.

Lastly, a unitary opportunity structure is the worst possible society as far
as the problem of the individuality is concerned: there is only one goal
everyone agrees about; life is really a race. Therefore, there is no space for
“anyone to develop in other directions, devote themselves to other goals, or
to carve out new paths of their own."121212Ibid. p.128.

All this discussion leads the author to formulate the following bold
conclusion:

> “… even though a unitary opportunity structure might help make the problem
> of equalizing opportunity conceptually more tractable, its substantive
> effect is just the opposite: it makes all the things that made us value
> equal opportunity in the first place that much harder to
> achieve."131313Ibid. p.128.

This constitutes the second fundamental critique moved by Fishkin to
traditional EOp theories:

######  Critique 2.

Equalizing opportunities within a pluralistic opportunity structure is
intractable.

Political philosophers and economic theorists seem to be stuck: either they
focus on a unitary opportunity structure — a “unidimensional society" which
severely hinders human flourishing — or defining EOp within a pluralistic
opportunity structure becomes conceptually intractable, leading to the
necessity of elaborating the new theory of Opportunity Pluralism.

In fact, the two critiques point to the existence of a fundamental trade-off
between EOp and Opportunity Pluralism.

##  4 The Theoretical Framework

Since the language and the tools of economists and political philosophers
differ, I make two fundamental remarks about the way I frame the theoretical
claims moved by Fishkin through the lens of economic theory. The first one
relates to the respect of preferences. The second one deals with the way I
model human development.

One of the normative values which provide the basis of economists’ toolbox is
the respect of preferences. People are autonomous moral agents who have the
right to choose their own notion of good life, without it being imposed from
an external authority. Therefore, an individual’s choices can reasonably be
interpreted as revealing information about the nature of this notion. Under
this assumption, economists construct metrics of well-being to measure
different bundles of opportunities. The key point is that this is not
incompatible with viewing opportunities as a crucial factor in shaping one’s
values, goals and ambitions. Even if economists held the very same view as
Fishkin, it would still be justifiable to construct measures of individual
well-being, building on what revealed (and well-informed) preferences induce
individuals to choose.141414For a thorough discussion, see Fleurbaey and
Maniquet, (2011) and Maniquet, (2017).

It seems that Fishkin’s argument that it is impossible to come up with a
measure of well-being for different bundles of opportunities is actually
incompatible with the respect of preferences.151515See the discussion leading
up to 1. In this regard, I depart from Fishkin’s view and I adhere to the
economic principle of respecting individual preferences. This enables to adopt
utility theory and model an economy in which bundles of opportunities can be
arranged according to a common scale.

Second, I model human development by assuming that preferences are both
uncertain and endogenous. In this way, I try to follow Fishkin’s claim that
one’s preferences evolves over time, depending on those multi-stage
interactions between one’s genetic endowment, the environment and the
opportunities provided in a way that cannot be thoroughly predicted ex ante,
by talents at birth.

###  4.1 The Model

The economy is populated by a continuum of individuals, and an egalitarian
government seeks to equalize opportunities among them. There are two
occupations, a and b, each of which requires a different skill. Individuals
are heterogeneous with respect to the talents (or marginal productivities) in
the two sectors; individual $i$ ’s talents are represented by a vector
$\bm{t}^{i}=(t_{a}^{i},t_{b}^{i})\in\Re_{+}^{2}$.

######  Definition 1.

A distribution of talents is pluralistic if there are at least two individuals
— $i$ and $j$ — such that $t_{a}^{i}>t_{b}^{i}$ and $t_{b}^{j}>t_{a}^{j}$.

As a minimal requirement of heterogeneity among individuals, we make the
following

######  Assumption 1.

The distribution of talents is pluralistic.

Bundles of opportunities are represented by educational investments in either
of two schools, $s_{a}$ and $s_{b}$, depending on whether the marginal
productivity in occupation $a$ or $b$ is nurtured. Individuals are also
heterogeneous with respect to a taste parameter $\theta$, a random variable
with distribution $F$ and density function $f$. This parameter represents the
preference of each individual for occupation $a$ relative to $b$. As it will
be explained below, this preference parameter captures the uncertainty and the
endogeneity of preferences.

#### Timing

There are two periods, $t=1,2$. At $t=1$, the egalitarian government observes
the talents of the individuals and assigns them to either of two schools,
$s_{a}$ and $s_{b}$, where, depending on the amount of educational expenditure
received, they will increase their productivity in either sector a or b,
respectively.161616I assume that, within the same school — $a$ or $b$ — the
amount of educational expenditure is individual-specific.

Then, at $t=2$, individuals observe the realization of $\theta$ and decide
whether to work in sector a or b, depending on the value of $\theta$ and the
wages that they will earn in occupation a or b.

#### The Technology of Educational Investments

The wages each individual would earn on the labour market at $t=2$ are a
function of his/her talents and the amount of educational expenditure
received. Within each occupation, denote
$\textit{w}_{j}^{i}:\Re^{2}_{+}\rightarrow\Re_{+}$ as the wage earned by
individual i endowed with talent $t_{j}^{i}$, after receiving an amount
$e_{j}^{i}\in\Re_{+}$ of educational expenditure. For simplicity, additive
separability between talent and expenditure within each sector is assumed;
i.e. for each $j\in(a,b)$,171717All results go through for more general
technologies of educational investments in which talent and educational
investments are complementary.

$\textit{w}_{j}^{i}=\textit{t}_{j}^{i}+\textit{e}_{j}^{i}\ .$

Each individual receives educational inputs in only one of the two sectors. In
the sector for which s/he is not educated, the wage at $t=2$ would be equal to
his/her talent (or marginal productivity).

#### Uncertainty of Preferences

During the schooling process, the individual does not know what his or her
preferences will be at $t=2$. I follow Arrow, (1995) in modelling uncertainty
about one’s own preferences. Two features of this framework seem to fit
particularly well within Fishkin’s account of opportunities.

First, I adopt a view of opportunity as a component of well-being or, in
Arrow’s terms, a view of freedom as autonomy: freedom of choice — represented
by opportunity sets — itself constitutes part of well-being, independently of
the actual choice made by the individual.

Secondly, this freedom of choice is represented by uncertainty about one’s
future preferences; i.e. autonomy is interpreted as freedom to choose one’s
own preferences. The conceptual framework devised by Arrow, (1995) is as
follows.
Each individual is endowed with an opportunity set $A^{i}$ whose generic
element is denoted by $x^{i}$. Individual i’s utility at $t=1$ is represented
by a utility function $U(x^{i},\theta)$, where $\theta$ is a random variable
with a (known) distribution which governs how the preferences of the
individual will be at $t=2$.
For any given opportunity set $A^{i}$, the achieved utility of the individual
would be:

$\mathcal{P}(\theta,A^{i})=[\max U(x^{i},\theta)\ |\ x^{i}\in A^{i}]\ .$

However, since $\theta$ is not known at $t=1$ when $A^{i}$ is assigned, the
expected value of this payoff is:

$\mathcal{V}(A^{i})=\mathbb{E}_{\theta}[\max U(x^{i},\theta)\ |\ x^{i}\in
A^{i}]=\mathbb{E}_{\theta}[\mathcal{P}(\theta,A^{i})]\ .$

This function, $\mathcal{V}(A^{i})$ is referred to in Arrow, (1995) as freedom
evaluation function. It has two basic properties.

######  Property 1.

If the opportunity set $A^{i}$ is included in $B^{i}$, then, for each
$\theta$,

$\max[U(x^{i},\theta)\ |\ x^{i}\in A^{i}]\leq\max[U(x^{i},\theta)\ |\ x^{i}\in
B^{i}]$

or, equivalently,

$\mathcal{V}(A^{i})\leq\mathcal{V}(B^{i}),\ \text{if}\ A^{i}\subseteq B^{i}.$

######  Property 2.

Only those elements which maximize $U(x^{i},\theta)$ for some $\theta$ matter.
Formally, let

$\xi(\theta,A^{i})=\operatorname*{arg\,max}[U(x^{i},\theta)\ |\ x^{i}\in
A^{i}]$

and let

$E(A^{i})=\text{range}_{\theta}\xi(\theta,A^{i}).$

Then,

$\mathcal{V}(A^{i})=\mathcal{V}(B^{i})\iff E(A^{i})=E(B^{i})\ .$

Thus, adding an element to the opportunity set which is not better than all
existing elements for at least one possible utility function parameter
$\theta$ does not increase freedom nor, consequently, utility.

#### The Opportunity Sets

I assume that preferences are represented by the following utility function

$U(l_{a}^{i},l_{b}^{i},\theta)=\theta l_{a}^{i}+(1-\theta)l_{b}^{i}\ ,$ (1)

where $l_{a}^{i}$ and $l_{b}^{i}$ denote occupations $a$ and $b$ for
individual $i$.181818This choice is motivated by the assumption that each
individual can work in a single occupation, not in both. Usually, labour time
enters negatively in the utility function as disutility from labour, or
positively as leisure. Instead, in this model $l_{a}^{i}$ and $l_{b}^{i}$
represent two normal goods (e.g., “being a mathematician” and “being a
musician”) or simply the extensive margin of labour in each occupation. It is
straightforward to generalize the model to include consumption and labour time
entering negatively into the utility function (the intensive margin).

I assume that each individual $i$ is endowed with an opportunity set $A^{i}$
with constant transformation rates given by:

$(l_{a}^{i}/w_{a}^{i})+(l_{b}^{i}/w_{b}^{i})\leq 1\ .$ (2)

This is displayed in Figure 1.

$l_{b}^{i}$$l_{a}^{i}$$w_{b}^{i}$$w_{a}^{i}$
Figure 1: The Opportunity Sets

Maximizing (1) over (2) yields the indirect utility:

$\mathcal{P}(w_{a}^{i},w_{b}^{i};\theta)=\max\big{\\{}\theta\textit{w}_{a}^{i},(1-\theta)\textit{w}_{b}^{i}\big{\\}}\
.$

The freedom evaluation function is defined analogously as

$\mathcal{V}(w_{a}^{i},w_{b}^{i})=\mathbb{E}_{\theta}[\max\big{\\{}\theta\textit{w}_{a}^{i},(1-\theta)\textit{w}_{b}^{i}\big{\\}}]\
.$ (3)

At $t=2$, when $\theta$ is realized, individual i will decide whether to work
in occupation a or b depending on the inequality:

$\theta\textit{w}_{a}^{i}>(1-\theta)\textit{w}_{b}^{i}\ .$

#### Endogeneity of Preferences

Preferences are endogenous in the following sense. The distribution of the
taste parameter $\theta$ is allowed to depend on wages in the two sectors, and
hence on the educational investments received by individuals.

Formally, denote $F(\theta;w_{a}^{i},w_{b}^{i})$ the distribution of the taste
parameter, and $f(\theta;w_{a}^{i},w_{b}^{i})$ the density function. I make
the following

######  Assumption 2.

The density $f$ satisfies, $\forall(w_{a}^{i},w_{b}^{i})\in\Re^{2}_{+}$,

  1. (i)

$f(\theta;w_{a}^{i},w_{b}^{i})=f(1-\theta;w_{b}^{i},w_{a}^{i})$ ;

  2. (ii)

If $w_{a}^{i}=w_{b}^{i}$, then $f$ is symmetric on $[0,1]$ ;

  3. (iii)

If $w_{a}^{i}\geq(\leq)\ w_{b}^{i}$, $f$ is skewed to the left (right).

Condition (i) simply means that relabelling the two sectors and the taste
parameter does not affect the (expected) utility of individuals.

The other two conditions provide a very minimal constraint on the “direction"
of the endogeneity of preferences. To see why this is the case, recall the
freedom evaluation function in (3), and suppose that $w_{a}^{i}=w_{b}^{i}$.

Condition (ii) says that it is equally likely to observe a realization of
$\theta$ above or below $\frac{1}{2}$; in other words, it is ex ante equally
likely for $i$ to choose occupation $a$ or $b$ at time $t=2$.

If $w_{a}^{i}\geq w_{b}^{i}$, then by Condition (iii), $f$ puts at least as
much mass to the right of $\frac{1}{2}$ as it puts on its left. Intuitively,
this means that individual $i$ is at least as likely to observe a realization
of $\theta$ which will induce him/her to choose occupation $a$ as occupation
$b$.

Moreover, 2 ensures that the freedom evaluation function satisfies two
important properties.

######  Property 3.

$\mathcal{V}$ is symmetric.

In addition to this, because of the specification of opportunity sets in (2),
we also have the following

######  Property 4.

$\mathcal{V}$ is quasiconvex.

These properties will play an important role in the characterisation of the
education policies in Section 5 and Section 6. Their proofs are in Appendix A.

#### The Egalitarian Government

The government is egalitarian; hence, it cares only about the least well-off.
Moreover, the government has to make educational investments at $t=1$, but
individuals will learn their preferences only at $t=2$. Therefore, the social
welfare function is given by

$\mathcal{W}=\max_{e_{a},e_{b}}\min_{i}\mathcal{V}(w_{a}^{i},w_{b}^{i})=\max_{e_{a},e_{b}}\min_{i}\mathbb{E}_{\theta}\Big{[}\max\big{\\{}\theta
w_{a}^{i},(1-\theta)w_{b}^{i}\big{\\}}\Big{]}$

Finally, I assume that the budget $\bar{R}$ allocated to educational
investment is large, but finite.

######  Assumption 3.

$0\ll\bar{R}<\infty\ .$

Therefore, all education policies presented in Section 5 and Section 6 can be
financed.

##  5 The Education Policies: Equalization of Resources

Suppose that the egalitarian government seeks to equalize wages among
individuals. Given that there is fixed educational budget, $\bar{R}$, the
government faces three alternatives. Either it equalizes wages within sector
a, or sector b, or between both.

######  Definition 2.

The one-school policy in sector $j=\\{a,b\\}$ equalizes wages in sector $j$ by
investing the per-capita amount $e_{j}^{i}=|{w}^{*}-t_{j}^{i}|$, for each
individual $i$ and some $w^{*}\in\Re_{+}$.

The third possibility is sending individuals to the two different schools,
depending on whether they have more talent for sector a or sector b.

######  Definition 3.

The two-school policy equalizes wages in each sector $j=a,b$ by investing the
per-capita amount
$e_{j}^{i}=|\tilde{w}-\max_{j}\left\\{t_{a}^{i},t_{b}^{i}\right\\}|$, for each
individual $i$ and some $\tilde{w}\in\Re_{+}$.

It is straightforward to derive the following

######  Lemma 1.

For any pluralistic distribution of talents and a given budget $\bar{R}$, the
two-school policy enables to equalize wages at a higher level than either of
the one-school policies within each sector.

###  5.1 The Evaluation of the Three Education Policies

It is straightforward to show that an egalitarian government will choose the
two-school policy. To illustrate the argument, consider the distribution of
talents given by $t_{a}^{i}+t_{b}^{i}=1$ in Figure 2.

$t_{b}+e_{b}$$t_{a}+e_{a}$1101.251.25
Figure 2: Evaluating the three policies .

If wages were equalized in sector $a$, then the most talented individual in
the distribution, located at (1,0), would become ex post the least well-off:
everyone will have the same wage in sector $a$ as him/her, but would have
strictly higher wage in sector $b$. This is represented by the horizontal blue
segment.

The case for sector $b$ would be analogous, but in that case, the individual
located at (0,1) would be the least well-off ex post. This is given by the
vertical blue segment. Considering these two individuals, either of the one-
school policies would bring one of the two at (1,1) while leaving the other
one at his/her own original coordinates.

However, the two-school policy — given by the two red segments — would yield
an expected indirect utility level to both individuals which would be higher
than their worst-case scenario. This follows from 1. Lemma 1 generalises the
result to any pluralistic distribution of talents. Thus, we have the following

######  Theorem 1.

The two-school policy yields a higher social welfare than any of the one-
school policies.

##  6 The Education Policies: Equalization of Expected Utilities

The above set of education policies might seem highly questionable on at least
two levels. On the one hand, it was assumed that everyone could be educated to
reach any marginal productivity level. This assumption — sometimes called
educability in the literature (e.g., see Kranich, (1998)) — might well be
deemed as highly unrealistic, even after weighting the education cost of low-
talent individuals more heavily.

In fact, it would not be unreasonable to assume that some individuals might
never reach the marginal productivity levels of the high-talent ones, not even
with an arbitrarily large amount of educational expenditure.

On the other hand, the education policies presented above imply that
individuals with equal taste parameter $\theta$ are likely end up with a
different utility level, not only ex ante (at $t=1$), but also ex post (at
$t=2$).

In the literature regarding the axiomatisation of EOp, one of the commonly-
used axioms is referred to as Equal Welfare for Equal Preference (e.g.,
Fleurbaey, (2008)). Under the assumption that skills and talents are
individual characteristics that call for compensation, whereas taste
differences should not, the axiom requires that the welfare level of
individuals with the same taste parameter be the same. This axiom is likely to
be violated by the education policies discussed above.

Therefore, I now assume that the government aims at investing in individual
education in such a way as to bring everybody to the same expected indirect
utility level curve. Denote $\mathcal{V}^{-1}(k^{*})$ the highest expected
indirect utility level curve that all individuals can reach, given the
available educational budget $\bar{R}$.191919Formally, this means defining the
indifference map $\\{\mathcal{V}^{-1}(k):k\in\Re_{+}\\}$, with
$\mathcal{V}^{-1}(k)=\\{(w_{a},w_{b})\in\Re^{2}_{+}:\mathcal{V}(w_{a},w_{b})=k\\}$.
Then, $k^{*}$ is the unique value of the level curve such that the total cost
of the policy is equal to $\bar{R}$.

######  Definition 4.

The one-school policy in sector $j=\\{a,b\\}$ equalizes expected utilities in
sector $j$ by investing the per-capita amount
$e_{j}^{i}=|\mathcal{V}^{-1}(k^{*})-t_{j}^{i}|$, for each individual $i$.

######  Definition 5.

The two-school policy equalizes utilities in each sector $j=a,b$ by investing
the per-capita amount
$e_{j}^{i}=|\mathcal{V}^{-1}(k^{*})-\max_{j}\\{t_{a}^{i},t_{b}^{i}\\}|$ for
each individual $i$.

###  6.1 The Evaluation of The Education Policies

Consider again the case in which the distribution of talents is given by
$t_{a}^{i}+t_{b}^{i}=1$. It is straightforward to show that the two-school
policy is (ex ante) Pareto efficient: everybody prefers it to either of the
one-school policies.202020A caveat applies: different educational policies
would yield different distributions of $\theta$ and hence, different utility
functions. Therefore, this efficiency criterion is different from the usual
Pareto efficiency. I refer to it as ex ante Pareto efficiency.

To see this, suppose that the highest level curve allowed by the educational
budget in both one-school policies is the one in blue in Figure 3.

$t_{b}+e_{b}$$t_{a}+e_{a}$110 Figure 3: Evaluation of the three policies

Each one-school policy entails investing in the education in one of the two
sectors so as to move every individual from the distribution of talents to the
blue curve, either horizontally or vertically.

However, the two-school policy would yield the same result, but at a lower
cost; in fact, the red area would be saved. Hence, given that the budget is
fixed, the government could increase educational expenditures such that
everyone reaches a strictly higher indifference curve. This result again
generalises to any pluralistic distribution of talents. In fact, we have the
following

######  Theorem 2.

Any of the one-school policies is Pareto-dominated by the two-school policy.

##  7 Conclusions

In this paper, I tried to provide an answer to the following question : Is
there a trade-off between EOp and Opportunity Pluralism?

In pursuing this task, I tried to embed Fishkin’s view about opportunities
into a simple economic model in which an egalitarian government seeks to
equalize opportunities among individuals whose preferences are uncertain at
the time of educational investment, as well as endogenous to it.

Under very mild assumptions, my analysis clearly goes against this claim.
Moreover, the results seem to suggest that economic efficiency, both in terms
of utility maximization and exhaustion of the available budget, implies POp.
In fact, an egalitarian government will always prefer a pluralistic
opportunity structure (two-school society) over a unitary opportunity
structure (one-school society).

Whereas the present analysis questions the fundamental ideas at the heart of
Opportunity Pluralism, it does not consider the new theory itself. The main
idea is to define EOp in a negative way; opportunities are equal among
individuals when society is able to remove the different types of bottlenecks
which hinder the full realization of each person’s ambitions, goals and
attitudes. This is left for future research.

## References

  * Arrow, (1995) Arrow, K. (1995).  A Note on Freedom and Flexibility.  In Basu, K., Pattanaik, P. K., Suzumura, K., Basu, K., Pattanaik, P. K., and Suzumura, K., editors, Choice, Welfare, and Development: A Festschrift for Amartya K. Sen. Oxford University Press, Oxford, New York.
  * Bourguignon et al., (2007) Bourguignon, F., Ferreira, F. H. G., and Menéndez, M. (2007).  Inequality of Opportunity in Brazil.  Review of Income and Wealth, 53(4):585–618.
  * Dalton, (1920) Dalton, H. (1920).  The Measurement of the Inequality of Incomes.  The Economic Journal, 30(119):348–361.
  * Dworkin, (1981) Dworkin, R. (1981).  What is Equality? Part 2: Equality of Resources.  Philosophy & Public Affairs, 10(4):283–345.
  * Fishkin, (2014) Fishkin, J. (2014).  Bottlenecks: A New Theory of Equal Opportunity.  Oxford University Press, Oxford, New York.
  * Fleurbaey, (2008) Fleurbaey, M. (2008).  Fairness, Responsibility, and Welfare.  Oxford University Press, Oxford, New York.
  * Fleurbaey and Maniquet, (2011) Fleurbaey, M. and Maniquet, F. (2011).  A Theory of Fairness and Social Welfare.  Econometric Society Monographs. Cambridge University Press, Cambridge.
  * Kranich, (1998) Kranich, L. (1998).  Equalizing opportunities through public education when innate abilities are unobservable.  Discussion papers, University at Albany, SUNY, Department of Economics.
  * Maniquet, (2017) Maniquet, F. (2017).  Opportunities, Welfare, and Social Justice: A Review of Fishkin’s _Bottlenecks_.  Journal of Economic Literature, 55(2):580–591.
  * Paes de Barros et al., (2009) Paes de Barros, R., Ferreira, F. H. G., Molinas Vega, J. R., and Saavedra Chanduvi, J. (2009).  Measuring Inequality of Opportunities in Latin America and the Caribbean.  Technical report, World Bank, Washington, DC.
  * Peragine, (2002) Peragine, V. (2002).  Opportunity egalitarianism and income inequality.  Mathematical Social Sciences, 44(1):45–64.
  * Pigou, (1912) Pigou, A. C. (1912).  Wealth and Welfare.  Macmillan and co., limited, London.
  * Rawls, (1958) Rawls, J. (1958).  Justice as Fairness.  The Philosophical Review, 67(2):164–194.
  * Rawls, (1971) Rawls, J. (1971).  A Theory of Justice.  Belknap Press, Cambridge, MA.
  * Roemer, (1993) Roemer, J. E. (1993).  A Pragmatic Theory of Responsibility for the Egalitarian Planner.  Philosophy & Public Affairs, 22(2):146–166.
  * Roemer, (2000) Roemer, J. E. (2000).  Equality of Opportunity:.  Harvard University Press, Cambridge, MA.
  * Roemer and Trannoy, (2016) Roemer, J. E. and Trannoy, A. (2016).  Equality of Opportunity: Theory and Measurement.  Journal of Economic Literature, 54(4):1288–1332.
  * Sen, (1980) Sen, A. (1980).  Tanner Lectures on Human Values, Volume 1.  Cambridge University Press, Cambridge.
  * Thomson, (2011) Thomson, W. (2011).  Chapter Twenty-One - Fair Allocation Rules.  In Arrow, K. J., Sen, A., and Suzumura, K., editors, Handbook of Social Choice and Welfare, volume 2 of Handbook of Social Choice and Welfare, pages 393–506. Elsevier.
  * World Bank, (2006) World Bank (2006).  World Development Report 2006: Equity and Development.  Technical report, World Bank, Washington, DC.

##  Appendix A Proofs

Proof of 3. By 2 and the definition of $\mathcal{V}$ we have that

$\displaystyle\mathcal{V}(w_{a}^{i},w_{b}^{i})$
$\displaystyle=\int\mathcal{P}(w_{a}^{i},w_{b}^{i};\theta)f(\theta;w_{a}^{i},w_{b}^{i})\
d\theta$
$\displaystyle=\int\max\big{\\{}\theta\textit{w}_{a}^{i},(1-\theta)\textit{w}_{b}^{i}\big{\\}}f(\theta;w_{a}^{i},w_{b}^{i})\
d\theta$
$\displaystyle=\int\max\big{\\{}(1-\theta)\textit{w}_{b}^{i},\theta\textit{w}_{a}^{i}\big{\\}}f(1-\theta;w_{b}^{i},w_{a}^{i})\
d\theta$
$\displaystyle=\int\mathcal{P}(w_{b}^{i},w_{a}^{i};1-\theta)f(1-\theta;w_{b}^{i},w_{a}^{i})\
d\theta$ $\displaystyle=\mathcal{V}(w_{b}^{i},w_{a}^{i})\ .$

∎

Proof of 4. Consider two vectors, $(w_{a}^{i},w_{b}^{i})$,
$(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\in\Re_{+}^{2}$, and any convex
combination between them,
$(\tilde{w}_{a}^{i},\tilde{w}_{b}^{i})\in\Re_{+}^{2}$. Quasiconvexity requires
that

$\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}\geq\mathcal{V}(\tilde{w}_{a}^{i},\tilde{w}_{b}^{i})$
(4)

for all $(w_{a}^{i},w_{b}^{i})$,
$(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})$, and
$(\tilde{w}_{a}^{i},\tilde{w}_{b}^{i})$. We need to distinguish two cases.

If $w_{a}^{i}\geq(\leq)\ w_{a}^{{}^{\prime}i}$ and $w_{b}^{i}\geq(\leq)\
w_{b}^{{}^{\prime}i}$, then (4) follows from 1.
Consider now the case $w_{a}^{i}\geq w_{a}^{{}^{\prime}i}$ and $w_{b}^{i}\leq
w_{b}^{{}^{\prime}i}$. We have the following

######  Claim 1.

$\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}\geq\mathcal{V}\big{(}\max\\{w_{a}^{i},w_{a}^{{}^{\prime}i}\\},\max\\{w_{b}^{i},w_{b}^{{}^{\prime}i}\\}\big{)}\
.$

Proof. If $f$ does not depend on wages, then

$\displaystyle\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}=$
$\displaystyle\max\bigg{\\{}\int\max\big{\\{}\theta\textit{w}_{a}^{i},(1-\theta)\textit{w}_{b}^{i}\big{\\}}f(\theta;w_{a}^{i},w_{b}^{i})\
d\theta,\int\max\big{\\{}\theta\textit{w}_{a}^{{}^{\prime}i},(1-\theta)\textit{w}_{b}^{{}^{\prime}i}\big{\\}}f(\theta;w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\
d\theta\bigg{\\}}=$
$\displaystyle\max\bigg{\\{}\int\max\big{\\{}\theta\textit{w}_{a}^{i},(1-\theta)\textit{w}_{b}^{i}\big{\\}}f(\theta)\
d\theta,\int\max\big{\\{}\theta\textit{w}_{a}^{{}^{\prime}i},(1-\theta)\textit{w}_{b}^{{}^{\prime}i}\big{\\}}f(\theta)\
d\theta\bigg{\\}}=$
$\displaystyle\int\max\bigg{\\{}\max\big{\\{}\theta\textit{w}_{a}^{i},(1-\theta)\textit{w}_{b}^{i}\big{\\}},\max\big{\\{}\theta\textit{w}_{a}^{{}^{\prime}i},(1-\theta)\textit{w}_{b}^{{}^{\prime}i}\big{\\}}\bigg{\\}}f(\theta)\
d\theta=$
$\displaystyle\int\max\bigg{\\{}\theta\max\\{\textit{w}_{a}^{i},w_{a}^{{}^{\prime}i}\\},(1-\theta)\max\\{\textit{w}_{b}^{i},w_{b}^{{}^{\prime}i}\\}\bigg{\\}}f(\theta)\
d\theta=$
$\displaystyle\mathcal{V}\big{(}\max\\{w_{a}^{i},w_{a}^{{}^{\prime}i}\\},\max\\{w_{b}^{i},w_{b}^{{}^{\prime}i}\\}\big{)}\
.$

Consider now the case in which the two densities differ.

Since $w_{b}^{i}\leq w_{b}^{{}^{\prime}i}$, whenever
$\theta\leq\frac{w_{b}^{{}^{\prime}i}}{w_{a}^{i}+w_{b}^{{}^{\prime}i}}$, then
$\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}=\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})$.
Moreover, since $w_{a}^{i}\geq w_{a}^{{}^{\prime}i}$, whenever
$\theta\geq\frac{w_{b}^{{}^{\prime}i}}{w_{a}^{i}+w_{b}^{{}^{\prime}i}}$, then
$\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}=\mathcal{V}(w_{a}^{i},w_{b}^{i})$.
In other words, the same reasoning applies inside the integrals, but now we
are comparing values of $\theta$ from two different distributions.

However, by 2

$1-F\bigg{(}\frac{w_{b}^{{}^{\prime}i}}{w_{a}^{i}+w_{b}^{{}^{\prime}i}};\
w_{a}^{i},w_{b}^{i}\bigg{)}\geq
1-F\bigg{(}\frac{w_{b}^{{}^{\prime}i}}{w_{a}^{i}+w_{b}^{{}^{\prime}i}};\
w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i}\bigg{)}$

Therefore, the skewness of the densities makes the total mass of
$\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}$
add up to more than 1. The claim then follows. ∎
Then, we have that

$\max\big{\\{}\mathcal{V}(w_{a}^{i},w_{b}^{i}),\mathcal{V}(w_{a}^{{}^{\prime}i},w_{b}^{{}^{\prime}i})\big{\\}}\geq\mathcal{V}\big{(}\max\\{w_{a}^{i},w_{a}^{{}^{\prime}i}\\},\max\\{w_{b}^{i},w_{b}^{{}^{\prime}i}\\}\big{)}\geq\mathcal{V}(\tilde{w}_{a}^{i},\tilde{w}_{b}^{i})\
,$

where the last inequality again follows from 1. ∎


# Do You See What I See?
Coordinating Multiple Aerial Cameras for Robot Cinematography

Arthur Bucker∗1, Rogerio Bonatti2, Sebastian Scherer2 1University of São
Paulo, Brazil<EMAIL_ADDRESS>Robotics Institute, Carnegie Mellon
University, Pittsburgh PA {rbonatti<EMAIL_ADDRESS>Supported by the CMU
Robotics Institute Summer Scholars program

###### Abstract

Aerial cinematography is significantly expanding the capabilities of film-
makers. Recent progress in autonomous unmanned aerial vehicles (UAVs) has
further increased the potential impact of aerial cameras, with systems that
can safely track actors in unstructured cluttered environments. Professional
productions, however, require the use of multiple cameras simultaneously to
record different viewpoints of the same scene, which are edited into the final
footage either in real time or in post-production. Such extreme motion
coordination is particularly hard for unscripted action scenes, which are a
common use case of aerial cameras. In this work we develop a real-time multi-
UAV coordination system that is capable of recording dynamic targets while
maximizing shot diversity and avoiding collisions and mutual visibility
between cameras. We validate our approach in multiple cluttered environments
of a photo-realistic simulator, and deploy the system using two UAVs in real-
world experiments. We show that our coordination scheme has low computational
cost and takes only 1.17 ms on average to plan for a team of 3 UAVs over a 10
s time horizon. Supplementary video: https://youtu.be/m2R3anv2ADE

##  I Introduction

Flying cameras are revolutionizing industries that require live and dynamic
camera viewpoints such as entertainment, sports, and security. The use of
unmanned aerial vehicles (UAVs) has several advantages in comparison with
traditional techniques such as dollies and cranes in terms of cost, efficiency
and safety [1, 2]. Furthermore, recent woks both in industry [3, 4] and
academia [5, 6, 7] now allow UAVs to track targets autonomously in cluttered
environments.

High-quality cinematographic productions, however, require the composition of
multiple viewpoints to form the final footage [8, 9], which can be edited on-
the-fly in the case of live sports events, or more commonly in post-
production. Camera arrangement is a complex problem that requires the
production crew to reason about what is going to happen in the scene in terms
of actor movements, increase viewpoint diversity, obey cinematographic
guidelines (e.g. the $180^{\circ}$ rule, staying on a single side of the
scene), and avoid mutual visibility between cameras. The problem becomes even
more challenging in the case of unscripted scenes in sports or journalistic
coverage, where cameras need to be re-arranged more frequently while being
subjected to velocity and acceleration constraints, and the event cannot be
re-enacted in the case of mistakes.

Despite the large recent progress in single-UAV filming solutions, multi-UAV
systems are still scarce. Existing approaches focus mainly on coordinating
vehicles to follow predefined shot types [10] or pre-planned filming missions
[11, 12], restricting the use of this technology to scripted applications. We
also find work on maximizing actor coverage [13], partially addressing the
shot diversity objective, but this approach is restricted to 2D reasoning and
static targets. None of the existing systems can autonomously coordinate
multiple cameras in dynamic and unscripted applications.

As seen in Figure LABEL:fig:main, our work aims to tackle the full multi-UAV
coordination problem, targeting cinematography in unscripted and dynamic
scenarios. We propose a system that can generate diverse and artistic shots in
real-time, empowering operators with full artistic capabilities of aerial
cameras. Our main contributions are:

1) Problem formulation: We formalize the multi-UAV coordination problem in
term of its principal cost functions. We start with a single-UAV formulation
[14] that considers trajectory smoothness, obstacle avoidance and
environmental occlusions, and add new cost terms to maximize shot diversity,
avoid mutual visibility between cameras, and to respect high-level user-
specified cinematography guidelines;

2) Camera coordination: We propose a greedy framework for multi-UAV
coordination that can run in real-time in dynamic scenes. First, a fast
centralized planner computes a set of desired viewpoints for each camera, and
a decentralized planning system computes the final trajectory for each UAV
based on an occupancy map of the environment and the predicted actor
trajectory;

3) Experimental validation: We validate our approach in multiple environments
using a photo-realistic simulator, and deploy the system in real-world
experiments with two UAVs.

##  II Related Work

Single-Drone Cinematography: There is vast body of academic literature and
consumer products on the topic of single-drone cinematography. For instance,
products such as the DJI Mavic [15] and Skydio R1 [4] can detect and track
targets autonomously. In addition, works such as [16, 17, 18, 19, 20, 21, 22]
can follow user-specific artistic guidelines during motion. We also find works
that try to automate the artistic decision-making guidelines as well. For
instance, [23, 14] use deep reinforcement learning to train a deep policy to
automatically select the best shot types for a given scene, conditioned on the
current actor actions and obstacle field. Also, [7] uses the actor’s skeleton
configuration to automatically position the camera, maximizing the body’s
projection on the image.

Multi-Robot Systems: There is a rich selection of work on multi-robot systems,
ranging from safety and controls [24, 25], planning [26, 27, 28], target
localization [29, 30, 31], exploration [32, 33], robot swarm task planning
[34], and even theatrical performances [35, 36]. We also find important
theoretical work on multi-sensor coordination using efficient greedy methods
[37, 38] that enjoy bounds on sub-optimality when compared to the full
exploration of the search space.

Multi-Drone Cinematography: In the field of UAV cinematography we find a few
pioneer works that employ multiple vehicles. For instance, [10] proposes an
optimization-based algorithm to coordinate multiple UAVs while accounting for
inter-drone collisions and mutual visibility, and requires user-defined paths
as guidelines for the motion of each drone. The work of [12] takes on
additional constraints into a greedy optimization, and maximizes target
visibility over time using a team of drones subject to limited battery life.
[13] simplifies the actor coverage problem to a 2D space, maximizing target
visibility. Also focusing on multi-UAV coverage, [39] optimizes multiple
flying cameras trajectories for efficient 3D reconstruction. In the context of
filming outdoor events with dynamic targets, [11] provides an overview of
cinematography principles that can be used for filming with multiple drones.

##  III Problem formulation

Our overall task is to optimally control a team of UAVs to film an actor who
is moving through an environment with obstacles. Similarly to [14], we
formulate a trajectory optimization problem using cost functions that measure
environmental occlusion of the actor, jerkiness of motion and safety. In
addition, we introduce new cost components for maximizing shot diversity
between UAVs, avoid inter-vehicle visibility, and to allow user-specified
cinematographic guidelines.

Let $\xi_{qi}:[0,t_{f}]\rightarrow^{3}\times SO(2)$ be the trajectory of the
i-th UAV, i.e., $\xi_{qi}(t)=\\{x(t),y(t),z(t),\psi_{q}(t)\\}$, and
$\Xi=\\{\xi_{q1},...,\xi_{qn}\\}$ be the set of trajectories from $n$ UAVs.
Let $\xi_{a}:[0,t_{f}]\rightarrow^{3}\times SO(2)$ be the trajectory of the
actor, $\xi_{a}(t)=\\{x(t),y(t),z(t),\psi_{a}(t)\\}$, which is inferred using
the onboard cameras. Let grid $\mathcal{G}:^{3}\rightarrow[0,1]$ be a voxel
occupancy grid that maps every point in space to a probability of occupancy.
Let $\mathcal{M}:^{3}\rightarrow$ be the signed distance values of a point to
the nearest obstacle. Our mathematical objective is to minimize a cost
function $J\left(\Xi\right)$ that tends to the following objectives:

1) Smoothness: Penalizes jerky motions that may lead to camera blur and
unstable flight. Calculated as the sum of costs from individual trajectories:
$J_{\mathrm{smooth}}\left(\Xi\right)=\sum_{i}J_{\mathrm{smooth}}\left(\xi_{qi}\right)$
2) Occlusion: Penalizes occlusion of the actor by obstacles in the environment
for each camera:
$J_{\mathrm{occ}}\left(\Xi\right)=\sum_{i}J_{\mathrm{occ}}\left(\xi_{qi},\xi_{a},\mathcal{M}\right)$
3) Safety: Penalizes proximity to obstacles that are unsafe for each UAV:
$J_{\mathrm{obs}}\left(\Xi\right)=\sum_{i}J_{\mathrm{obs}}\left(\xi_{qi},\mathcal{M}\right)$
4) Diversity: Penalizes viewpoints similarities between UAVs. Calculated over
the entire set of trajectories: $J_{\mathrm{div}}\left(\Xi,\xi_{a}\right)$
5) Inter-visibility: Avoids visibility between UAVs:
$J_{\mathrm{vis}}\left(\Xi\right)$
6) Cinematography guidelines: Penalizes user-specified undesired viewpoints
(e.g. high tilt angles): $J_{\mathrm{cine}}\left(\Xi,\xi_{a}\right)$

We then compose the overall cost function as a linear combination between each
component, with relative weights $\lambda$. The solution $\Xi^{*}$ is then
tracked by each UAV:

$\displaystyle J\left(\Xi\right)$
$\displaystyle=\begin{bmatrix}1&\lambda_{1}&\lambda_{2}&\lambda_{3}&\lambda_{4}&\lambda_{5}\end{bmatrix}\begin{bmatrix}J_{\mathrm{smooth}}\left(\Xi\right)\\\
J_{\mathrm{occ}}\left(\Xi\right)\\\ J_{\mathrm{obs}}\left(\Xi\right)\\\
J_{\mathrm{div}}\left(\Xi\right)\\\ J_{\mathrm{vis}}\left(\Xi\right)\\\
J_{\mathrm{cine}}\left(\Xi\right)\end{bmatrix}$ (1) $\displaystyle\Xi^{*}$
$\displaystyle=\underset{}{\operatorname*{arg\,min}}\quad J\left(\Xi\right)$

##  IV Multi-Camera Coordination

We now detail the methods we use for camera coordination in the multi-UAV
system. As displayed in Equation 1, our overall objective function involves
the minimization of $6$ sub-objectives, which often conflict with one another.
Our goal is to formulate an algorithm that works in real-time, in unscripted
scenes, and that can deal with a state space which grows exponentially in
complexity with the number of UAVs. Given this challenge, we prefer to find
fast solutions which are only locally optimal as opposed to globally optimal
trajectories that take a long time to compute.

To address the time complexity issue, we break down our method into three main
subsystems that operate together. First, a centralized motion planner (Sec.
IV-A) coordinates desired positions for all cameras simultaneously. Next, a
decentralized motion planner network (Sec. IV-B) computes the final
trajectories for each specific UAV. Finally, an image selection module (Sec.
IV-C) chooses the best live image to be displayed out of all cameras.
Alternatively, the final image selection can be manually performed in post-
production. Fig. 1 depicts the system diagram.

Figure 1: System overview: a centralized waypoint planner computes discrete
camera positions. Next, a decentralized planning system optimizes smooth
trajectories for each UAV.

###  IV-A Centralized Multi-Camera Planning

Our centralized planning system parametrizes trajectories as waypoints, i.e.,
$\xi\in^{T\times 3}$, where $T$ is the number of time steps. We assume that
the heading dimension $\psi(t)$ is set to always point the drone from
$\xi_{qi}(t)$ towards the actor in $\xi_{a}(t)$, which can be achieved
independently of the aircraft’s translation by rotating the UAV’s body and
camera gimbal. We parametrize the state-space of all possible camera positions
using spherical coordinates $\\{\rho,\theta,\phi\\}$ centered on the actor’s
location:

$\displaystyle\xi_{qi}(t)=\xi_{a}(t)+\rho\begin{bmatrix}cos(\psi_{a}+\theta)sin(\phi)\\\
sin(\psi_{a}+\theta)cos(\phi)\\\ cos(\phi)\end{bmatrix}$ (2)

Space discretization:  We define a discrete state-space lattice $S$ that
contains all possible camera positions distributed as $|S|=576$ points in a
half-sphere above ground.

Figure 2: Discrete spherical state space centered on actor.

Based on cinematographic guidelines [9, 8] we equally divide the yaw
coordinates $\theta$ into $16$ values between $[0,2\pi]$, the tilt angles
$\phi$ into $6$ values between $[0,\pi/2]$, and the distance to actor within
the set $\rho\subseteq\\{2,3,4,5,6,7\\}$, ranging from close-up to long shots.
In addition, we discretize the trajectory’s time into $5$ steps equally spaced
every $2$ seconds, forming a $10$-second planning time horizon.

Cost functions:  Next we mathematically formulate the cost functions that
optimized by the centralized camera planner for the set of UAV trajectories
$\Xi$:

i) Shot diversity cost:  The shot diversity cost prevents the planner to
allocate the $n$ camera positions close to each other up until a maximum
distance $d_{max}^{div}$:

$\displaystyle
J_{\mathrm{div}}\left(\Xi\right)=\sum_{\tau=1}^{T}\sum_{i=1}^{n}\sum_{j=1}^{n}D({\xi_{qi}(\tau),\xi_{qj}(\tau)}),$
(3) where $\displaystyle D(p_{1},p_{2})=\begin{cases}1&\mbox{if
}d<d_{min}^{div}\\\ \frac{d}{d_{max}^{div}-d_{min}^{div}}&\mbox{if
}d_{min}^{div}<d<d_{max}^{div}\\\ 0&\mbox{if }d>d_{max}^{div}\end{cases}$

ii) Inter-drone collision cost:  Analogously to $J_{\mathrm{div}}$ in Eq. 3,
we define the inter-drone collision cost as a measure of distance between
UAVs. The user can, however, choose a different cost weight $\lambda$ and
safety distance $d_{max}^{col}$ to penalize collisions.

iii) Inter-drone visibility: We express the inter-drone visibility cost as a
binary measure $V(p_{i},p_{j})\subseteq\\{0,1\\}$ of whether point $j$ is
visible from position $i$. We model the visible area as a cone in space using
the camera’s diagonal field-of-view angle, and position its center line
towards the actor’s position:

$J_{\mathrm{vis}}\left(\Xi\right)=\sum_{\tau=1}^{T}\sum_{i=1}^{n}\sum_{j=1}^{n}V({\xi_{qi}(\tau),\xi_{qj}(\tau)})\vspace{-2mm}$
(4)

We pre-compute all mutual visibility values into a look-up table to reduce
planning times, which is possible in our discrete state-space model.

iv) Obstacle and occlusion avoidance:  In order to keep all UAVs safe and to
maintain visibility of the actor at all times, we must reason about the role
of obstacles in the environment. First, we transform the environment’s
occupancy grid into a time-dependent spherical domain centered around the
actor $\mathcal{G}\rightarrow\mathcal{G}_{s}^{t}\in[0,1]$, as shown in Fig.
3(a). We then compute the obstacle avoidance cost by summing the occupancy
likelihood of all cells within a radius $r_{max}$ of each UAV. We also
calculate the occlusion cost as a measure of occupancy along a line
$l_{i}(\tau)=\tau\xi_{qi}(t)+(1-\tau)\xi_{a}(t)$ between UAV and actor:

$\displaystyle J_{\mathrm{obs}}\left(\Xi\right)$
$\displaystyle=\sum_{\tau=1}^{T}\sum_{i=1}^{n}\int_{0}^{r_{max}}\mathcal{G}_{s}^{\tau}({\xi_{qi}(\tau)})\
d(\text{volume})$ (5) $\displaystyle J_{\mathrm{occ}}\left(\Xi\right)$
$\displaystyle=\sum_{\tau=1}^{T}\sum_{i=1}^{n}\int_{0}^{1}\mathcal{G}_{s}^{\tau}({l_{i}(\tau)})\
d\tau\vspace{-2mm}$

v) Cinematography guidelines as cost prior:  We also allow operators to
manually specify a cost prior $J_{\mathrm{cine}}$ for each cell, in case they
wish to follow specific cinematographic guidelines. For example, Fig. 3(b)
shows an example where we define overhead shots as undesired.

(a)

(b)

Figure 3: (a) Visualization of occupancy and occlusion avoidance costs in the
cells of the spherical grid $\mathcal{G}_{s}^{t}$. (b) Example of a
cinematography guideline prior that penalizes overhead shots. Red spheres
represent regions of high cost.

Greedy optimization:  The centralized planning problem for multi-camera
optimization over multiple time steps proves to be NP-hard, similarly to other
optimal sensor placement works [40, 41, 42]. In order to make the computation
tractable in real-time and avoid a combinatorial problem, we develop a greedy
optimization approach that computes the optimal trajectory
$\xi_{q\,i}^{*\text{greedy}}$ for each UAV sequentially, fixing all previously
calculated trajectories $\\{\xi_{q\,1}^{*\text{greedy}}$, …,
$\xi_{q\,i-1}^{*\text{greedy}}\\}$.

Figure 4: Greedy planner.

At each stage $i$ we employ a backward induction dynamic programming algorithm
[43, 44, 45] to find the optimal cost-to-go for of each state $s\subseteq S$
at all time steps, analogously to a value iteration algorithm. To do so, we
build a cost map $C:S\rightarrow^{|S|}$ that contains the cost of all states,
and a cost-to-go map $V:S\rightarrow^{|S|}$. In order to make transitions
between cells dynamically feasible for the real vehicle, we only allow
expansions to neighboring cells in the spherical grid. Since we operate in a
discrete state-space with a small branching factor and deterministic
transitions, a single backwards pass quickly yields the optimal solution.
Finally, we build the full trajectory $\xi_{q\,i}^{*\text{greedy}}$ by
selecting neighboring cells with the least cost-to-go at consecutive time
steps, starting at the UAV’s initial position $S_{i}^{0}$. We update the cost
manifold after each drone is added, since inter-visibility, shot diversity,
and collision costs are recalculated. Algorithm 1 details the process:

Initialize $\Xi_{\textnormal{greedy}}=\\{\\}$ $\triangleright$ empty set

1 for  _each UAV $i$_ do

$C$ $\leftarrow$ $J(S,\Xi_{\textnormal{greedy}})$ $\triangleright$ update
entire cost map

$V$ $\leftarrow$ Backward_Induction($C$) $\triangleright$ update cost-to-go

2 $\xi_{q\,i}^{*\text{greedy}}$ $\leftarrow$ Optimal_Path($V$, $S_{i}^{0}$);

3 Append $\xi_{q\,i}^{*\text{greedy}}$ to $\Xi_{\textnormal{greedy}}$;

4

5 end for

return $\Xi_{\textnormal{greedy}}$

Algorithm 1 Compute greedy traj. set $\Xi=\\{\xi_{q1},...,\xi_{qn}\\}$

###  IV-B Decentralized UAV Trajectory Optimization and Tracking

After calculating the greedy-optimal set of UAV trajectories
$\Xi_{\textnormal{greedy}}$ using the centralized planner, we post-process
each output using a decentralized planning system. Our objective here is to
obtain smoother individual trajectories at a finer time discretization. While
the original waypoints were spaced every $2$ seconds over a $10$-second
horizon, here we achieve finer resolutions with $0.5$ s granularity in local
planning, and $0.02$ s for trajectory tracking. We employ the single-UAV local
planner described in [14], which uses covariant gradient descent to produce
locally optimal trajectories while considering the costs of smoothness,
obstacle and occlusions avoidance, and a desired artistic trajectory, which we
define as $\xi_{q\,i}^{*\text{greedy}}$. In addition, each local planner
receives the expected waypoints of all remaining vehicles, and avoids
positioning its trajectory within 1m of other UAVs. We run the local planner
at $5$ Hz, and use a PID controller at $50$ Hz.

###  IV-C Live Image Selection

Even though our system produces multiple image streams, most filming
applications display a single camera to the viewer at a time. The final movie
can be produced with manual post-processing, or in the case of live streams it
requires a human to make selections while viewing all videos simultaneously.
Here, we propose a method for automatic selection of live images based on a
subset of our cost functions. We compute a image quality score $Q_{i}$ for
each camera stream, calculated as a weighted sum of the inter-drone visibility
cost $J_{\mathrm{vis}}$ and of the prior cinematographic guidelines
$J_{\mathrm{cine}}$. We select the current camera with the highest score, and
once a camera is chosen we exponentially decay its score to foster viewpoint
diversity, and gradually return it to the original value once another image is
chosen. Based on cinematography literature we set minimum and maximum limits
for shot lengths of $3$ and $8$ seconds respectively, which are reasonable
units of length for individual action shots [46, 47].

##  V Experimental Results

Here we detail the simulated and real-world experiments that validate our
multi-UAV cinematography system. Additional visualizations are shown in the
supplementary video.

###  V-A Simulation experiments

Experimental setup:  We record all simulated data using a drone in a photo-
realistic environment, AirSim [48], coupled with a custom ROS interface [49].
As seen in the supplementary video, our simulated scenes consists of an
animated character walking around a suburban environment, surrounded by
obstacles such as trees, buildings, and posts.

(a)

(b)

(c)

Figure 5: Diverse range of simulated environments where we tested our multi-
UAV system: a) narrow gaps, b) suburban environment with sparse tall
obstacles, and c) dense foliage.

E1) Viewpoint diversity validation:  This first experiment’s objective was to
validate the assumption of the relation between shot diversity and artistic
quality by quantifying the benefits of the employing multiple UAVs for aerial
cinematography as opposed to the use of a single camera. To do so, we
generated a set of $5$ videos with $12$ seconds of length of an actor walking
around a park. The first clip featured a single-camera back shot for its
entire duration, while all other clips alternated between the back shot and a
secondary viewpoint every $3$ seconds. We chose the secondary viewpoints to be
either $45^{\circ}$, $90^{\circ}$, $135^{\circ}$, or $180^{\circ}$ for each of
the other four videos respectively.

For this survey we recruited $15$ participants using Amazon Mechanical Turk
(MTurk) [50], who were compensated for their time. After being approved on a
short qualifying task, each participant viewed a total of 12 pairs of videos,
one being the constant back shot clip, and the other being one of the multi-
viewpoint clips. Videos were played synchronously three times, and after
watching at least once participants answered: “Which video is more enjoyable?”
Each clip was compared 15 times against the baseline single-camera shot.

Figure 6 displays the survey results, where the height of each bar represents
the percentage of users that rated the multi-UAV clip as being more enjoyable
than the single-UAV clip. As expected, we found that an overwhelming majority
of users prefer to watch a scene with viewpoint diversity.

Figure 6: User study to evaluate viewer preference for videos with diverse
viewpoints. We ask users to compare how enjoyable a single back shot view is
against clips with additional viewpoints. Vertical bar shows percentage of
users that prefer the video with a secondary viewpoint, where each bar
corresponds to an angle value.

E2) Robustness across environments:  We tested our approach in a diverse set
of photo-realistic simulations, ranging from open to cluttered environments,
and with groups of UAVs containing 2 to 5 vehicles. As shown in Figure 5, our
approach is able to successfully coordinate the teams of UAVs in a wide
spectrum of scenarios. For instance, in 5(a) the actor passes trough a narrow
gap (2m) that the UAVs need to avoid, and in 5(b) she walks next to a
extensive line of trees (1m apart), which forces the UAVs to move to the right
side of the actor. In addition, in 5(c) we see the UAVs navigating within
dense and unstructured foliage tunnels.

The tests indicate that our system is able to adapt to a diverse range of
environments and actor motions. Furthermore, we verify that the final clips
follow our objective functions and present diverse viewpoints with little
inter-drone visibility.

E3) Scaling performance:  We conducted a set of experiments to quantify our
centralized planner’s performance as it scales to larger teams of UAVs and
larger state-spaces. This analysis is important because keeping low
computational costs is vital for real-time performance.

First, we measured the planning time and computer resources consumption for
finer discretizations of the state space (Table I). We note that memory usage
grows proportionally to the square of the number of cells in the state space
because we employ a precomputed lookup table for the inter-drone visibility
and shot diversity costs. However, our implementation offers large benefits in
terms of computing time and CPU usage, which grow linearly.

| State space
---
($n_{\psi}$,$n_{\phi}$,$n_{\rho}$)
| Computed
---
states
| Planning time
---
for 3 drones [ms]
| CPU[%]
---
(1 core)
| Memory
---
use [MB]
(3,3,8) | 360 | 0.17+-0.03 | 22 | 35
(16,6,6) | 2880 | 1.17+-0.18 | 25 | 37.5
(24,9,9) | 9720 | 4.65+-0.47 | 28 | 64
(32,12,12) | 23040 | 16.70+-1.26 | 34 | 197
(40,15,15) | 45000 | 19.56+-0.83 | 38 | 655
(48,18,18) | 77760 | 51.71+-2.04 | 52 | 1840
(52,21,21) | 114660 | 116.28+-6.98 | 75 | 4693
(64,24,24) | 184320 | 228.01+-8.85 | 100 | 10150
TABLE I: Performance of the greedy planner for different discretizations of
the State space. We use 3 UAVs with a 5 time-steps horizon, and re-plan at 5
Hz.

The discretization we use for most of our experiments
($n_{\psi}$=16,$n_{\phi}$=6 and $n_{\rho}$=6) presents a great trade-off, with
a reasonable space discretization and a low computation time of 1.17ms, while
only consuming 37.5 MB of RAM. As a comparison, we implemented an optimal
brute-force planner and tested it using the same spatial discretization and
number of drones. Using only 2 time steps, the non-greedy planner took 16.4
seconds to find the optimal solution. The planning time jumps to 2h:30min when
we consider 3 time steps. These results show the importance of adopting a
greedy strategy to solve a NP-hard problem in real-time.

We also evaluated how the planning time increase with a larger number of
drones and time steps. As expected, our greedy solution has a linear growth of
complexity (Fig. 7).

Figure 7: Time required by the high-level planner to compute the trajectories
of the drones: (a) given the number of vehicles, (b) given the number of time
steps computed an (c) given the number of states in the artistic domain
($n_{\psi}\cdot n_{\phi}\cdot n_{\rho}\cdot n_{t}$)

###  V-B Real-world results

Experimental setup:  For the real-world experiments we used two unmodified
Parrot Bebop 2 UAVs with an integrated electronic gimbal, a WiFi router and a
standard desktop PC (Intel i7-8700 CPU and Nvidia GTX1060 GPU). All
communication between the drones and PC were handled via a ROS interface with
the Bebop SKD. We transmitted images from the UAVs to the base station for the
actor’s visual detection and trajectory forecasting, and sent velocity
commands back to the UAV after planning.

E4) Flight among obstacle:  We tested the system by recording a moving actor
with 2 drones in an environment with a virtual obstacle. Figure 8 depicts how
the high-level central planner interacts with the local trajectory
optimization method to calculate the final UAV paths. Drone 1 is optimized
first, and takes a trajectory close to, but sufficiently far from the
obstacle. Drone 2 is optimized next by the central planner. When it reaches
the vicinity of the virtual obstacle, the planner brings its ideal path to a
higher elevation, above the first drone’s path, in order to avoid colliding
with the obstacle while keeping the actor visible and avoiding visibility of
drone 1. The central planner’s output for drones 1 and 2 are colored in blue
and red respectively. The UAV’s respective local planners receive the discrete
high-level path and optimize it, creating a smooth trajectory output. After
drone 2 reaches the left side of the actor, it switches to a slightly tilted
and distant shot due to the shot diversity cost.

E5) Dynamic actor:  We evaluated the system’s capability to track and record
an actor performing dynamic movements with abrupt motion changes, such as a
person playing soccer. As seen in the supplementary video, both drones were
able to safely keep the actor in frame through the entire test while exploring
diverse viewpoints with low inter-drone visibility.

Figure 8: Real-life flight among a simulated obstacle. On the left, the
outputted sequence waypoints for each drone (red and blue) guide the locally
optimized paths (yellow) towards a region free from obstacles and free of
inter-drone visibility. Diverse shots can be seen during flight.  Figure 9:
Real-life flight following a highly dynamic actor playing soccer. The drones
are able to keep up with abrupt motion motion changes while filming the actor.

##  VI Conclusion and Discussion

In this paper we present a system for real-time coordination of aerial cameras
for autonomous cinematography in dynamic and unscripted scenarios. First, we
formalize the multi-UAV filming problem in terms of its main objectives:
maximizing shot diversity, avoiding inter-vehicle visibility and obeying high-
level cinematographic guidelines. Next, we develop a two-step approach for
calculating the trajectory of each UAV, based on an efficient centralized
greedy planner for viewpoint selection coupled with a decentralized trajectory
optimizer to calculate smooth trajectories. We validate our system in multiple
simulated and real-world experiments, and show that it can successfully
control a team of UAVs. Additionally, we provide insights into how our methods
can scale for larger numbers of UAVs in terms of planning time and memory.

We note that the nature of trajectories generated with our system is highly
dependent on the combination of relative weights between costs. Different
applications may require distinct weighing schemes. For instance, for a
journalistic coverage the operator would likely increase the penalty on inter-
drone visibility, while this weight could be negligible for footage generated
in a 3D motion reconstruction application.

We are interested in extending this research in multiple future directions.
Despite using a decentralized trajectory optimization framework, our
experiments in this paper were performed on a single desktop PC, and commands
were then transmitted to each UAV individually in real time. We are currently
working on a new multi-UAV system with onboard processors, where one master
drone will run the fast centralized planner and then communicate with all
other UAVs via a mesh network so that each drone can optimize its own
trajectories. In addition, we plan to employ our system in novel applications
such as 3D reconstruction of scenes in the wild. Even though multi-view
systems are already well developed for indoors environments [51], creating
such systems for capturing images in natural environments is still an open
research field.

## ACKNOWLEDGMENTS

We thank CITI-Poli, Grupo Turing and the Skyrats team for the help with robot
experiments. RB also gratefully acknowledges the support from the Waibel
Education Fund.

## References

  * [1] M. De-Miguel-Molina, _Ethics and Civil Drones: European Policies and Proposals for the Industry_ , 2018.
  * [2] V. Santamarina-Campos and M. Segarra-Oña, “Introduction to drones and technology applied to the creative industry. airt project: An overview of the main results and actions,” in _Drones and the Creative Industry_. Springer, 2018, pp. 1–17.
  * [3] DJI. (2018) Dji mavic, https://www.dji.com/mavic. [Online]. Available: https://www.dji.com/mavic
  * [4] Skydio. (2018) Skydio r1 self-flying camera, https://www.skydio.com/technology/. [Online]. Available: https://www.skydio.com/technology/
  * [5] R. Bonatti, W. Wang, C. Ho, A. Ahuja, M. Gschwindt, E. Camci, E. Kayacan, S. Choudhury, and S. Scherer, “Autonomous aerial cinematography in unstructured environments with learned artistic decision-making,” _Journal of Field Robotics_ , 2020.
  * [6] B. F. Jeon, D. Shim, and H. J. Kim, “Detection-aware trajectory generation for a drone cinematographer,” _arXiv preprint arXiv:2009.01565_ , 2020.
  * [7] C. Huang, F. Gao, J. Pan, Z. Yang, W. Qiu, P. Chen, X. Yang, S. Shen, and K.-T. T. Cheng, “Act: An autonomous drone cinematography system for action scenes,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 7039–7046.
  * [8] C. J. Bowen and R. Thompson, _Grammar of the Shot_. Taylor & Francis, 2013.
  * [9] D. Arijon, “Grammar of the film language,” 1976.
  * [10] T. Nageli, L. Meier, A. Domahidi, J. Alonso-Mora, and O. Hilliges, “Real-time planning for automated multi-view drone cinematography,” _ACM Transactions on Graphics_ , 2017.
  * [11] I. Mademlis, V. Mygdalis, N. Nikolaidis, M. Montagnuolo, F. Negro, A. Messina, and I. Pitas, “High-level multiple-uav cinematography tools for covering outdoor events,” _IEEE Transactions on Broadcasting_ , vol. 65, no. 3, pp. 627–635, 2019.
  * [12] L.-E. Caraballo, Ángel Montes-Romero, J.-M. Díaz-Báñez, J. Capitán, A. Torres-González, and A. Ollero, “Autonomous planning for multiple aerial cinematographers,” 2020.
  * [13] A. Saeed, A. Abdelkader, M. Khan, A. Neishaboori, K. A. Harras, and A. M. S. Mohamed, “On realistic target coverage by autonomous drones,” _ACM Transactions on Sensor Networks_ , 2019.
  * [14] R. Bonatti, W. Wang, C. Ho, A. Ahuja, M. Gschwindt, E. Camci, E. Kayacan, S. Choudhury, and S. Scherer, “Autonomous aerial cinematography in unstructured environments with learned artistic decision-making,” _Journal of Field Robotics_ , vol. 37, no. 4, pp. 606–641, Jan. 2020. [Online]. Available: https://doi.org/10.1002/rob.21931
  * [15] “Dji mavic,” https://www.dji.com/br/mavic, August 2020.
  * [16] R. Bonatti, Y. Zhang, S. Choudhury, W. Wang, and S. Scherer, “Autonomous drone cinematographer: Using artistic principles to create smooth, safe, occlusion-free trajectories for aerial filming,” 2018.
  * [17] Q. Galvane, J. Fleureau, F.-L. Tariolle, and P. Guillotel, “Automated cinematography with unmanned aerial vehicles,” _arXiv preprint arXiv:1712.04353_ , 2017.
  * [18] Q. Galvane, C. Lino, M. Christie, J. Fleureau, F. Servant, F. o.-l. Tariolle, and P. Guillotel, “Directing cinematographic drones,” _ACM Transactions on Graphics (TOG)_ , vol. 37, no. 3, pp. 1–18, 2018.
  * [19] N. Joubert, D. B. Goldman, F. Berthouzoz, M. Roberts, J. A. Landay, P. Hanrahan _et al._ , “Towards a drone cinematographer: Guiding quadrotor cameras using visual composition principles,” _arXiv preprint arXiv:1610.01691_ , 2016.
  * [20] T. Nageli, L. Meier, A. Domahidi, J. Alonso-Mora, and O. Hilliges, “Real-time planning for automated multi-view drone cinematography,” _ACM Transactions on Graphics (TOG)_ , vol. 36, no. 4, pp. 1–10, 2017.
  * [21] C. Gebhardt, B. Hepp, T. Nageli, S. Stevšić, and O. Hilliges, “Airways: Optimization-based planning of quadrotor trajectories according to high-level user goals,” in _Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems_ , 2016, pp. 2508–2519.
  * [22] M. Roberts and P. Hanrahan, “Generating dynamically feasible trajectories for quadrotor cameras,” _ACM Transactions on Graphics (SIGGRAPH 2016)_ , vol. 35, no. 4, 2016.
  * [23] M. Gschwindt, E. Camci, R. Bonatti, W. Wang, E. Kayacan, and S. Scherer, “Can a robot become a movie director? learning artistic principles for aerial cinematography,” _arXiv preprint arXiv:1904.02579_ , 2019.
  * [24] W. Luo, N. Chakraborty, and K. Sycara, “Distributed dynamic priority assignment and motion planning for multiple mobile robots with kinodynamic constraints,” in _2016 American Control Conference (ACC)_. IEEE, 2016, pp. 148–154.
  * [25] W. Luo and A. Kapoor, “Multi-robot collision avoidance under uncertainty with probabilistic safety barrier certificates,” _arXiv preprint arXiv:1912.09957_ , 2019.
  * [26] N. Karnad and V. Isler, “Modeling human motion patterns for multi-robot planning,” in _2012 IEEE International Conference on Robotics and Automation_. IEEE, 2012, pp. 3161–3166.
  * [27] Y. Jiang, H. Yedidsion, S. Zhang, G. Sharon, and P. Stone, “Multi-robot planning with conflicts and synergies,” _Autonomous Robots_ , vol. 43, no. 8, pp. 2011–2032, 2019.
  * [28] L. Liu and N. Michael, “An mdp-based approximation method for goal constrained multi-mav planning under action uncertainty,” in _2016 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2016, pp. 56–62.
  * [29] S. Engin and V. Isler, “Active localization of multiple targets using noisy relative measurements,” _arXiv preprint arXiv:2002.09850_ , 2020.
  * [30] H. Bayram, N. Stefas, K. S. Engin, and V. Isler, “Tracking wildlife with multiple uavs: System design, safety and field experiments,” in _2017 International symposium on multi-robot and multi-agent systems (MRS)_. IEEE, 2017, pp. 97–103.
  * [31] B. Charrow, V. Kumar, and N. Michael, “Approximate representations for multi-robot control policies that maximize mutual information,” _Autonomous Robots_ , vol. 37, no. 4, pp. 383–400, 2014.
  * [32] M. Corah and N. Michael, “Distributed matroid-constrained submodular maximization for multi-robot exploration: Theory and practice,” _Autonomous Robots_ , vol. 43, no. 2, pp. 485–501, 2019.
  * [33] ——, “Efficient online multi-robot exploration via distributed sequential greedy assignment.” in _Robotics: Science and Systems_ , vol. 13, 2017.
  * [34] M. Chandarana, W. Luo, M. Lewis, K. Sycara, and S. Scherer, “Decentralized method for sub-swarm deployment and rejoining,” in _2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)_. IEEE, 2018, pp. 1209–1214.
  * [35] E. A. Cappo, A. Desai, and N. Michael, “Robust coordinated aerial deployments for theatrical applications given online user interaction via behavior composition,” in _Distributed Autonomous Robotic Systems_. Springer, 2018, pp. 665–678.
  * [36] E. A. Cappo, A. Desai, M. Collins, and N. Michael, “Online planning for human–multi-robot interactive theatrical performance,” _Autonomous Robots_ , vol. 42, no. 8, pp. 1771–1786, 2018.
  * [37] A. Krause, A. Singh, and C. Guestrin, “Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies,” _Journal of Machine Learning Research_ , vol. 9, no. Feb, pp. 235–284, 2008\.
  * [38] M. Roberts, D. Dey, A. Truong, S. Sinha, S. Shah, A. Kapoor, P. Hanrahan, and N. Joshi, “Submodular trajectory optimization for aerial 3d scanning,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 5324–5333.
  * [39] X. Zheng, F. Wang, and Z. Li, “A multi-uav cooperative route planning methodology for 3d fine-resolution building model reconstruction,” _ISPRS journal of photogrammetry and remote sensing_ , vol. 146, pp. 483–494, 2018.
  * [40] A. Krause, A. Singh, and C. Guestrin, “Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies,” _Journal of Machine Learning Research_ , vol. 9, no. Feb, pp. 235–284, 2008\.
  * [41] A. Krause, C. Guestrin, A. Gupta, and J. Kleinberg, “Near-optimal sensor placements: Maximizing information while minimizing communication cost,” in _Proceedings of the 5th international conference on Information processing in sensor networks_ , 2006, pp. 2–10.
  * [42] S. Arora and S. Scherer, “Randomized algorithm for informative path planning with budget constraints,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 4997–5004.
  * [43] R. S. Sutton and A. G. Barto, _Reinforcement learning: An introduction_. MIT press, 2018.
  * [44] D. P. Bertsekas, D. P. Bertsekas, D. P. Bertsekas, and D. P. Bertsekas, _Dynamic programming and optimal control_. Athena scientific Belmont, MA, 1995, vol. 1, no. 2.
  * [45] S. M. LaValle, _Planning algorithms_. Cambridge university press, 2006.
  * [46] B. Mahadani and B. MahadaniI, “How film shot length decrease in last century,” Feb 2015. [Online]. Available: http://bhushanmahadani.com/film-shot-length-decreased-in-last-century/
  * [47] J. E. Cutting, J. E. DeLong, and C. E. Nothelfer, “Attention and the evolution of hollywood film,” _Psychological science_ , vol. 21, no. 3, pp. 432–439, 2010.
  * [48] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in _Field and service robotics_. Springer, 2018, pp. 621–635.
  * [49] ROS. (2018) Robot operating system (ros). [Online]. Available: http://www.ros.org/
  * [50] Amazon. Amazon mechanical turk. [Online]. Available: https://www.mturk.com
  * [51] H. Joo, H. Liu, L. Tan, L. Gui, B. Nabbe, I. Matthews, T. Kanade, S. Nobuhara, and Y. Sheikh, “Panoptic studio: A massively multiview system for social motion capture,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2015, pp. 3334–3342.