Circular Figures in Elamite Mathematics

Nasser Heydari[Email<EMAIL_ADDRESS>and  Kazuo Muroi[Email<EMAIL_ADDRESS>
In this article, we study a particular group of plane figures whose    constants are listed in the Susa Mathematical Tablet No. 3 (SMT No. 3).  We explain   possible ways to define these figures and seek to demonstrate that the Susa scribes   used  complicated calculations to obtain  such numbers. We also give examples of circular figures used for    Elamite artifacts  through which one can appreciate the significance of these figures   in   Elamite art.

§ INTRODUCTION

In Babylonian mathematics  there are mathematical tablets whose texts list  many geometrical coefficients regarding geometric plane figures.  Many scholars call them  coefficient lists or constant lists.  In these texts, a list of numbers and modifiers are listed together. However, no information regarding the related calculations are provided.  By checking  the possible calculations  required   to find   some of these constants, one might better understand both the complexity of the calculations and  the level of mathematical skill employed by  the  scribes who recorded them.

One of the famous coefficient lists in Babylonian mathematics is     SMT No. 3, which   is one of   26 clay tablets excavated from Susa in  southwest Iran by French archaeologists in 1933 and now held in the collection of the Louvre. The texts of all the Susa mathematical tablets (SMT) along with their interpretations were first published in 1961 (see [4]).

This  tablet[The reader can see the new  photos of this tablet on the website of the Louvre's collection. Please see <https://collections.louvre.fr/en/ark:/53355/cl010185653> for obverse  and   reverse.]  contains a list of constants with 68 entries, almost all of which are preserved in good condition. The content of this tablet is as follows: line 1 is the headline, lines 2-32 contain the mathematical constants regarding areas and dimensions of geometrical figures  and  lines 33-69 contain the non-mathematical constants  concerning work quotas, metals, and so on. The last two lines (lines 70-71)  are an example of how to use the constants  of work quotas.

Some of the figures mentioned in this text are rather complex geometric figures and finding their area  requires  mathematical skill and experience.  This clearly shows the considerable interest of the Susa scribes in geometry and their  ability to compute the area and other geometric constants of these figures.

§ CIRCULAR FIGURES

Circular figures are usually made up of     arcs of different circles. In other words, these figures can be considered as polygons but their sides instead of line segments are arcs of circles. Unlike polygons, the numbers of sides can be any natural number $n\geq 1$, because we can consider the circle itself as a circular figure with only 1 arc (the whole circumference). For example, intersections     of   circles (not necessarily equal) produce different circular figures. <ref> shows  examples of circular figures with 2, 3 and 6 arcs.   We call the intersection of  two arcs in a circular figure a vertex.    If the circular figure has $n$  arcs, we call it  a polyarc with $n$ arcs or an  $n$-arc    and denote it by   $\Psi_n$.

Although the definition of polyarcs encompasses a wide variety of figures, we are mostly interested in those  where all   arcs are equal and come from the same circle (the constructive circle). We prefer to use the term regular polyarcs for  such specific  circular figures. For example, the polyarc $\Psi_6$ in <ref> is   regular    while  $\Psi_2$ and $\Psi_3$ are not.  We usually use $\Psi_n(a)$ for a regular polyarc with $n$ arcs  the common length of whose equal arcs   is $a$. Note that this common value is a fraction of the circumference of the constructive circle.

In this article, we only consider  regular polyarcs whose arcs are all convex or all concave. We call them regular convex polyarcs and   regular concave polyarcs, respectively.   In <ref>, cases 1, 2 and 4 are regular  convex polyarcs, while case 3 is a regular  concave polyarc. Note that cases 3 and 6 are regular polyarcs but neither is convex nor concave.

Regular polyarcs

An  interesting problem  might be the computation of the area of a regular convex or a  regular concave  polyarc $\Psi_n(a)$. Some of these geometric figures have     properties that   make the accomplishment of this task  possible. In this article, we consider   two groups of regular polyarcs whose areas can be computed because of the way we construct them.  We first discuss the concave case.

§.§ Regular concave polyarcs

One  efficient way to construct this group of regular concave  polyarcs with $n$ arcs   is to consider a   circle  of radius $r$   and repeat (rotate) it $n$ times  along a bigger circle in a way that the obtained $n$ copies   form a chain of pairwise tangent circles.[It can be shown that the radius of the bigger circle must be $R=\frac{r}{\sin(\frac{\pi}{n})} $.]   If we connect the centers of adjacent circles together, we get a regular $n$-gon whose sides are of length $2r$, where $r$ is the radius of the constructive circle. <ref> shows this situation for the case $n=8$. The symmetry of this figure helps us to obtain a formula for the area of this polyarc with respect to its arc  $a$. We use $\Psi^c_n(a)$ for the  polyarcs in this group.

Construction of a   regular concave polyarc

Let us compute the area of this regular concave  polyarc $\Psi^c_n(a)$. It is clear from <ref> that the area of   $\Psi^c_n(a)$ is the area of the regular $n$-gon with side $2r$ minus $n$ times the area of a sector $\Lambda_n$ of the constructive circle and internal angle $\alpha_n=\frac{(n-2)\pi}{n}$.  Since the circumference is $c=2\pi r$  and  $a$ is the arc length of the internal angle $\alpha_n$,   the  values of circumference $c$ and radius $r$ with respect to $a$  are

\begin{equation}\label{equ-a}
c =\frac{2na}{n-2}
\end{equation}

\begin{equation}\label{equ-b}
r =\frac{na}{(n-2)\pi}.
\end{equation}

It follows from <ref> that the area of the sector $\Lambda_n$ is

\[ S_{\Lambda_n}=\frac{1}{2}\alpha_n r^2= \frac{1}{2}\times \frac{(n-2)\pi}{n} \times \left(\frac{na}{(n-2)\pi}\right)^2 \]

which can be simplified as

\begin{equation}\label{equ-c}
S_{\Lambda_n}= \frac{na^2}{2\pi(n-2)}.
\end{equation}

On the other hand, the area of the regular $n$-gon  with side $2r$, say $ \Gamma_n(2r) $, is

\[ S_{\Gamma_n(2r)}= \frac{n}{4}\times   (2r)^2\times \cot\left(\frac{\pi}{n}\right)= nr^2     \cot\left(\frac{\pi}{n}\right). \]

If we use <ref> in the last equation, we get

\begin{equation}\label{equ-d}
S_{\Gamma_n(2r)}= \frac{n^3a^2}{\pi^2(n-2)^2} \times \cot\left(\frac{\pi}{n}\right).
\end{equation}

Now, it follows from <ref> and <ref> that

\begin{align*}
S_{\Psi^c_n(a)}& = S_{\Gamma_n(2r)} - n\times S_{\Lambda_n}\\
& =  \frac{n^3a^2}{\pi^2(n-2)^2} \times \cot\left(\frac{\pi}{n}\right) -n\times   \frac{na^2}{2\pi(n-2)}\\
& =  \left(\frac{n^3}{\pi^2(n-2)^2} \times \cot\left(\frac{\pi}{n}\right) -  \frac{n^2}{2\pi(n-2)}\right)a^2.
\end{align*}

Hence, the area of the  regular concave polyarc   $\Psi^c_n(a)$ with $n$ arcs of the same length $a$ is

\begin{equation}\label{equ-e}
S_{\Psi^c_n(a)}=  \left(\frac{n^3}{\pi^2(n-2)^2} \times \cot\left(\frac{\pi}{n}\right) -  \frac{n^2}{2\pi(n-2)}\right)a^2.
\end{equation}

§.§ Regular convex polyarcs

A  group of regular convex  polyarcs    with $n$ sides denoted by $\Psi^v_n(a)$ can be constructed by repeating (rotating) a constructive circle  $n$ times   around a smaller circle. In this case, the innermost $n$ equal arcs form a regular convex  polyarc. Unlike the concave case, finding the area of such polyarcs is not so easy in general and one may use different approaches such as integration. However, in some special cases ($n=2,3,4,5,6$) one can use the symmetry of the figure and use elementary mathematics to find a formula with respect to the arc length $a$ and $n$.  The key part in this process is to observe that the area of $\Psi^v_n(a)$  is the area  of the inscribed regular $n$-gon plus the total area of $n$ equal circular segments (see <ref>).

Construction of a regular convex  polyarc

For even numbers $n=2m$, there is an alternate way to define this group of regular convex  polyarcs. In this method, instead of   circles we use a special regular  convex 2-arc. This polyarc is obtained by intersecting   two equal circles such that the length of each arc is the quarter of the circumference of the constructive circle. The obtained polyarc looks like a “convex lens”.   If we rotate a convex lens $m$ times  around   its center, then the  intersection of the obtained $n$ copies forms a regular convex polyarc with $n=2m$ arcs (see <ref>).

Construction of a regular convex  polyarc with $n=2m$ arcs

To compute the area of this regular convex  $n$-arc $\Psi^v_n(a)$, one needs to use the symmetry of the figure and perform some    complicated calculations. We skip the related calculations and only give the formula with respect to $n$ and $r$, the radius of the quadrants:

\begin{equation}\label{equ-f}
\begin{split}
S_{\Psi^v_n(a)}&=  n\left[\arcsin\left(\frac{1}{\sqrt{2}}\sin\left(\frac{\pi}{n}\right) \sqrt{1+\cos^2\left(\frac{\pi}{n}\right)}-\frac{1}{2\sqrt{2}}\sin\left(\frac{2\pi}{n}\right)\right)\right]r^2\\
& - n\left[\frac{1}{2}\sin\left(\frac{\pi}{n}\right)\sqrt{1+\cos^2\left(\frac{\pi}{n}\right)}-\frac{1}{4}\sin\left(\frac{2\pi}{n}\right)\right]r^2.\\
\end{split}
\end{equation}

§ CIRCULAR FIGURES IN SMT NO. 3

In SMT No. 3, the constants of several geometric figures are listed some of which have not been fully understood since the publishing of the text of this tablet. The figures we are going to consider here are listed in lines 5,6 and 16-25.  The terms used by the Susa scribe to name these figures might be the reason for this lack of comprehension. There are  six circular figures in these lines which we explain in the following sections.

§.§ A barley-field

The transliteration and translation of   lines 16-18 are as follows:

Obverse: Lines 16-18

(L16) 13,20 igi-g[ub] šà a-šà še

(L17) 56,40 dal šà a-šà še

(L18) 23,20 [pi]-ir-ku šà a-šà še

(L16) 0;13,20 is the constant of a “barley-field”.

(L17) 0;56,40 is the length of a “barley-field”.

(L18) 0;23,20 is the width of a “barley-field”.

The scribe has used the  term barley-field for this geometric figure which might be derived  from the shape of a barley seed (see <ref>).

Barley seeds (photo credit: Encyclopædia Britannica)

In fact, the figure is   a regular convex 2-arc   (or a convex lens). As mentioned before, it is the intersection of two equal circles such that they separate one-fourth of the circumference of one  another (see <ref>). Beside the figure itself, the Susa scribe has listed two constants for its  length and its width. The former is the line segment connecting  the vertices of the figure and the latter is the line segment connecting the midpoints of the two arcs. We try to verify the scribe's numbers.   To see that, consider a convex lens, say $\Sigma$, whose common arcs have length $a$.  Assume that the radius of the equal quadrants is $r$ and the  length   and the width  are   $2x$ and $2y,$ respectively (see <ref>). We   compute the values of $x$, $y$  and  the area of the barley-field figure $\Sigma$ with respect to $a$.

A barley-field figure

It is clear from  <ref> that   $\overline{DB}=2x $ and $\overline{EF}=2y $. Since  the arc  $\arc{DFB} $   is one-fourth of the circumference $c$ of  a circle with radius $r$  and   $c=2\pi r  $ it follows that

\[ a=\frac{c}{4}=\frac{2\pi r}{4}=\frac{\pi r}{2}  \]

and consequently

\begin{equation}\label{equ-aa}
r =\frac{2a}{\pi}.
\end{equation}

In the right triangle  $\triangle DAB $, two sides $AD$ and $AB$ (radii of the circle) are equal, so two angles $ \angle ADH$ and $ \angle ABH$ are equal and their common measure is $45^{\circ}$. This follows that two angle $ \angle DAH$ and $ \angle  BAH$ have the same measure $45^{\circ}$.    So in  the   right triangle   $\triangle AHD $, we have  $\overline{AH}=\overline{DH}  $. A similar argument shows that $\overline{AH}=\overline{BH}  $, $\overline{BH}=\overline{CH}  $ and $\overline{CH}=\overline{DH}  $. In other words, they all have the same length $x$.  Also note  that by the symmetry we have $\overline{EH}=\overline{FH}=y$.

Now, by <ref> and   the Pythagorean theorem in the right triangle $\triangle AHD$, we can write

\begin{align*}
\Longrightarrow~~&~~2x^2=r^2 \\
\Longrightarrow~~&~~x=\frac{\sqrt{2}}{2}r \\
\Longrightarrow~~&~~x= \frac{\sqrt{2}}{2}  \times\frac{2a}{\pi},
\end{align*}

and therefore

\begin{equation}\label{equ-ab}
\end{equation}

On the other hand, since $ \overline{AF}=\overline{AH}+ \overline{FH}$, we obtain $r=x+y $. So, it follows from <ref> that

\[ y=r-x=\frac{2a}{\pi}-\dfrac{\sqrt{2}a}{\pi}, \]

or equivalently

\begin{equation}\label{equ-ac}
\end{equation}

To compute the area of the figure, we should note that it is two times the area of the segment with chord $DB$ and arc $\arc{DFB}$. This area in turn is the area of the  quadrant $ADFB$ with radius $r$ minus that of the triangle $\triangle ABD$. So it follows from <ref>   that

\begin{align*}
S_{\Sigma}&=2\times \left(S_{ADFB}-S_{\triangle ABD}\right)\\
&=2\times \left(\frac{1}{4}\pi r^2-\frac{\overline{AB}\times \overline{AD}}{2}\right)\\
&=2\times \left(\frac{1}{4}\pi r^2-\frac{r^2}{2}\right)\\
&=  \left(\frac{\pi}{2}  -1\right)r^2\\
&=  \left(\frac{\pi}{2}  -1\right) \times \left(\frac{2a}{\pi}\right)^2 \\
&=   \left(\frac{2(\pi-2)}{\pi^2}\right) \times a^2,
\end{align*}

\begin{equation}\label{equ-ad}
S_{\Sigma}=  \frac{2(\pi-2)a^2}{\pi^2}.
\end{equation}

Note that by setting $n=2$ in <ref> and using <ref>, we get the same formula.

Let us verify the scribe's numbers by assuming $a=1$ and    using  <ref>, <ref> and <ref>. Since in Babylonian mathematics  the approximate values $\pi\approx 3 $ and $\sqrt{2}\approx \frac{17}{12}=1;25 $  were usually used in  calculations, we have

\[\overline{DB}=2x=  \dfrac{2\sqrt{2}a}{\pi}\approx \dfrac{2\times (1;25)\times 1}{3} = (2;50)\times (0;20)=0;56,40  \]

which is the number listed in line 17. On the other hand, for the width of $\Sigma$, we have

\begin{align*}
\overline{EF} =2y =  2\left(\dfrac{2-\sqrt{2}}{\pi}\right)a \approx  \frac{2}{3}\times(2-1;25 )\times 1=(0;40)\times (0;35)   = 0;23,20
\end{align*}

which is the number in line 18. Finally, for approximate  area  of the  barley-field  $\Sigma: DEBF$, we can use <ref> to write

\begin{align*}
S_{\Sigma} = \frac{2(\pi-2)a^2}{\pi^2}  \approx \frac{2(3-2)\times 1^2}{3^2} = \frac{2}{9} = 0;13,20
\end{align*}

which is the   number   in line 16. So,  it seems that  the Susa scribe was capable of performing such calculations  to obtain these numbers.

§.§ An ox-eye

The transliteration and translation of   lines 19-21 are as follows:

Obverse: Lines 16-18

(L19) 16,52,30 igi-gub šà igi-gud

(L20) 52,30 dal šà igi-gud

(L21) 30 pi-ir-ku šà igi-gud

(L19) 0;16,52,30 is the constant of an “ox-eye”.

(L20) 0;52,30 is the length of an “ox-eye”.

(L21) 0;30 is the width  of an “ox-eye”.

The    term ox-eye for this geometric figure clearly refers to the shape of the eye of an ox. The reason for this naming might be that the figure    is similar to  a convex lens but  it is wider.  It can be made by intersecting   two equal circles such that their arcs are one-third of the circumference of the circle (see <ref>).

An ox-eye

Three numbers for the area, the  length and the width of the figure has been listed in lines 19-21.     To verify these numbers, consider   the regular convex 2-arc $\Pi$ with arc length $a$ as shown in <ref>.   Set $\overline{DH}=x$ and $\overline{AH}=y$.  All sides $AD $, $AB$, $BC$, $DC$ and $AC$ are radii of equal circles, so   the triangles $\triangle ADC $ and $\triangle ABC$ are equilateral and the measures of their internal angles are $60^{\circ} $. Also, the central angles of two arcs   $\arc{DAB} $ and $\arc{DCB} $ are $120^{\circ} $ and the common length $a$ of two arcs   $\arc{DAB} $ and $\arc{DCB} $ is the one-third of the circumference of the circle    with radius  $r$. So $c=2\pi r=3a$ and

\begin{equation}\label{equ-3-ga}
r =\frac{3a}{2\pi}.
\end{equation}

Since $DB$   perpendicularly bisects $\angle ADC $,  the height $DH$ bisects $AC$ and thus $r=\overline{AC}=2\overline{AH}=2y  $. So it follows from <ref> that

\begin{equation}\label{equ-3-gb}
y =\frac{3a}{4\pi}.
\end{equation}

Area of an ox-eye

On the other hand, by the Pythagorean theorem in the right triangle $\triangle AHD $ and using <ref> and <ref>, we have

\begin{align*}
&=\dfrac{\sqrt{3}}{2}\times \frac{3a}{2\pi},
\end{align*}

which  implies that

\begin{equation}\label{equ-3-gc}
\end{equation}

Since $\overline{AC}=2y$ and $\overline{BD}=2x$, it follows from <ref>, <ref>   that

\begin{equation}\label{equ-3-gca}
\begin{dcases}
\overline{AC}=\frac{3a}{2\pi},\\
\overline{BD}=\frac{3\sqrt{3}a}{2\pi}.
\end{dcases}
\end{equation}

The area  of the ox-eye  $\Pi: ABCD $   is double    the difference between the area  of circular sector  $\Gamma_1: ABCD$ and  the area  of the isosceles triangle $\triangle ABD $. Since the area of the sector $\Gamma_1: ABCD$ is the one-third of the area of a circle with radius $r$, it follows from <ref>, <ref> and <ref>   that

\begin{align*}
S_{\Pi}&=2\times \left(S_{\Gamma_1}-S_{\triangle ABD}\right)\\
& =2\times \left[\left(\frac{1}{3}\times \pi r^2\right)-\frac{2xy}{2}\right]\\
&\approx 2\times \left[ \left(\frac{1}{3}  \times \frac{9\pi a^2}{4\pi^2}\right)   - \left(\frac{3\sqrt{3}a}{4\pi}\times \frac{3a}{4\pi}\right) \right]\\
&=2\times \left[\frac{3\pi a^2}{4\pi^2}   -  \frac{9\sqrt{3}a^2}{16\pi^2} \right]\\
&=2 \times\left(\frac{(12\pi-9\sqrt{3}) a^2}{16\pi^2} \right)\\
& = \frac{(12\pi-9\sqrt{3}) a^2}{8\pi^2},
\end{align*}

or equivalently

\begin{equation}\label{equ-3-gcb}
S_{\Pi}= \left(\dfrac{3}{2\pi}-\frac{9\sqrt{3}}{8\pi^2}\right) a^2.
\end{equation}

Now, set $a=1$ and use the Babylonian approximations $ \sqrt{3} \approx \frac{7}{4}$ and $ \pi \approx 3 $. It follows from <ref> that

\[ \overline{AC}=\frac{3a}{2\pi}\approx \frac{3\times 1}{2\times 3}=\frac{1}{2}=0;30, \]

which is listed in line 21. Also

\[ \overline{BD}=\frac{3\sqrt{3}a}{2\pi}\approx \frac{3\times (7/4)\times 1}{2\times 3}=\frac{7}{8}=0;52,30. \]

which is written in line 20.  Finally, the approximate area is obtained from <ref> as follows:

\[ S_{\Pi}= \left(\dfrac{3}{2\times 3}-\frac{9\times(7/4)}{8\times 9}\right) \times 1\approx   \frac{1}{2} - \frac{7}{32}=\frac{9}{32}=0;16,52,30. \]

This is the very number in line 19 which is given as the constant   of the “ox-eye”. Thus, the numbers of the Susa scribe are verified.

§.§ Overlap of two barley-fields

The transliteration and translation of   line  5 are as follows:

Obverse: Line 5

(L5)   16(sic) [igi]-gub šà gúr šà 2 še i-na šà gúr gar

(L5)  0;16  is the constant of a circular figure  with two barley figures  in the middle.

In  this line, a circular figure  is described as the overlap of  two  barley-field figures. This is exactly the intersection of two convex lenses such that their centers   coincide and their lengths (diagonals) form a right angle. So, it is  a regular convex polyarc with 4 arcs (see <ref>).

Overlap of two barley-fields

The number 0;16 is written as the constant of this regular polyarc in line 5.  Apparently, the Susa scribe has assumed the common radius of all quadrants forming the figure to be 1.   Let us compute the area of the figure  for a general case when the radius of all quadrants is   $r$. We denote this polyarc  by $\Lambda_4 $. As   in      <ref>, this  figure is the union of a quadrilateral $\Gamma_4$ and four   circular segments $\Gamma_4^1, \Gamma_4^2,\Gamma_4^3$  and $\Gamma_4^4 $.

Partition of a regular convex  4-arc

Now, consider <ref> and connect vertices  $H,G$ to the vertex  $A$ to get the isosceles triangle  $\triangle AGH$. Connect $H$ to $B$. Since three line segments $AH$, $AB$ and $HB$ are the radius of equal quadrants, the triangle $\triangle AHB $ is equilateral and all of  its internal angles are $60$ degrees. This implies that

\[  \angle DAH=\angle  DAB-\angle  HAB =90^{\circ}-60^{\circ}=30^{\circ}. \]

Because of symmetry, the same is true for angles $\angle GAB$ and $\angle HAG $. Therefore, three arcs $\arc{DH}$, $\arc{HG}$ and $\arc{GB}$ have the same length which is one-twelfth of the circumference of a circle with radius $r$.  Similar arguments can be used to show that all other arcs   $\arc{GF}$, $\arc{FE}$ and $\arc{EH}$ have the common length of $\arc{HG}$, so they all    are equal implying that  their corresponding chords are equal too. Thus,  the sides of the quadrilateral $\Gamma_4:  EFGH $ have the same length.

Partition of a  regular convex 4-arc

Next, consider  the isosceles triangle $\triangle AGH$ in which $\angle HAG=30^{\circ} $. So  the other two inner angles are

$$\angle AGH= \angle AHG=\frac{180^{\circ}-30^{\circ}}{2} =75^{\circ} $$

which implies that

$$\angle BHG= \angle AHG - \angle AHO=75^{\circ}-60^{\circ}=15^{\circ}. $$

Similarly, we get $\angle EHA=  15^{\circ} $ and thus

$$ \angle EHG=\angle EHA + \angle AHB+\angle BHG =15^{\circ}+60^{\circ}+15^{\circ}=90^{\circ}.$$

By symmetry, we can see that all other three inner angles of the quadrilateral $\Gamma_4(r):  EFGH $ are also right-angled. So far, we have shown that   the quadrilateral $\Gamma_4(r):  EFGH $ has equal sides and equal right angles, therefore it   must be a square.

On the other hand, since $G$ is in the middle of the arc $ \arc{HGB}$, the line segment $AG$ perpendicularly bisects   its chord $HB$. So   $\angle HOG=90^{\circ} $  and since $ \overline{HB}=r$,    we get $ \overline{HO}=\frac{\overline{HB}}{2}= \frac{r}{2}$. By the Pythagorean theorem in the right triangle $\triangle HOA $, we have

\[ \overline{AO}=\sqrt{\overline{HA}^2-\overline{HO}^2}=\sqrt{r^2-\frac{r^2}{4}}=\frac{\sqrt{3}}{2}r. \]

Since $ \overline{GO}=\overline{AG}-\overline{AO}$, we get

\begin{equation}\label{equ-3-ba}
\begin{dcases}
\overline{HO}= \dfrac{r}{2}\\
\overline{GO}=\left(1-\dfrac{\sqrt{3}}{2}\right)r.
\end{dcases}
\end{equation}

Note that  the area of the  square  $\Gamma_4 :  EFGH $ in   <ref>  is equal to $S_{\Gamma_4}=\overline{HG}^2  $, so we can apply <ref> and   the Pythagorean  theorem  in the right triangle $\triangle HOG $  to obtain $\overline{HG} $  as follows:

\begin{align*}
\overline{HG}  = \sqrt{\overline{HO}^2+\overline{GO}^2 } = \sqrt{\left(\frac{r}{2}\right)^2+\left(r-\frac{\sqrt{3}r}{2}\right)^2}   = \left(\sqrt{2-\sqrt{3}}\right)  r.
\end{align*}

\begin{equation}\label{equ-3-bc}
\overline{HG} =\left(\sqrt{2-\sqrt{3}}\right)  r,
\end{equation}

and   the  area of the inner square $\Gamma_4:  EFGH $  is

\begin{equation}\label{equ-3-b}
\end{equation}

Let us    compute the exact area of the convex polyarc $\Lambda_4$. To do so, we first need to compute the areas of the   circular segments  $\Gamma_4^1, \Gamma_4^2,\Gamma_4^3$  and $\Gamma_4^4 $.   It suffices to compute the area of one of them, say $\Gamma_4^3$, because as we saw above they have the same arcs and chords. It is clear from <ref> that $ S_{\Gamma_4^3}  $ is the area of the one-twelfth of the circle with radius $r$ minus the area of the isosceles triangle $\triangle AHG $. So, we can write

\begin{align*}
S_{\Gamma_4^3}&=\frac{\pi r^2}{12}- S_{\triangle AHG}\\
&=\frac{\pi r^2}{12}- \dfrac{1}{2}\times \overline{HO}\times \overline{AG}\\
&=\frac{\pi r^2}{12}- \dfrac{1}{2}\times \dfrac{r}{2}\times r\\
&=\frac{\pi r^2}{12}- \dfrac{r^2}{4},
\end{align*}

\begin{equation}\label{equ-3-bd}
S_{\Gamma_4^3}=\left(\frac{\pi}{3}- 1\right) \dfrac{r^2}{4}.
\end{equation}

Now, we can use  <ref> and <ref> to compute the exact area of the convex polyarc $\Lambda_4$ as follows:

\begin{align*}
S_{\Lambda_4}&=S_{\Gamma_4}+4\times S_{\Gamma_4^3}\\
&=  (2-\sqrt{3})r^2+4\times  \left(\frac{\pi}{3}- 1\right) \times \dfrac{r^2}{4}\\
&=(2-\sqrt{3})r^2+\left(\frac{\pi}{3}- 1\right)  r^2\\
&= \left(\frac{\pi}{3}+1-\sqrt{3}\right)  r^2.
\end{align*}

Thus, we obtain

\begin{equation}\label{equ-3-be}
S_{\Lambda_4}= \left(\frac{\pi}{3}+1-\sqrt{3}\right)  r^2.
\end{equation}

Note that if we set $n=2$ in the general formula <ref>, we get <ref>.

Finally, let us verify the scribe's number. According to Babylonian tradition,  if we set $r=1$ and use the approximation $ \sqrt{3}\approx \frac{7}{4}$ in <ref>, the approximate area  of $\Gamma_4 $  is

\[ S_{\Gamma_4}=(2-\sqrt{3})\times 1^2\approx 2-\frac{7}{4}=2-1;45=0;15. \]

By comparing this value with the value $0;16$ given in line 5, one might get the impression  that  the Susa scribe     has made a  mistake here, but there may be something hidden here. If we   only set $r=1$   in <ref> and assume that  $S_{\Gamma_4}\approx 0;16$ then we get

\begin{align*}
~~&~~2-\sqrt{3}  \approx 0;16\\
\Longrightarrow~~&~~ \sqrt{3}  \approx 2-\dfrac{16}{60}\\
\Longrightarrow~~&~~\sqrt{3}  \approx 2-\dfrac{4}{15}\\
\Longrightarrow~~&~~\sqrt{3}  \approx  \dfrac{30-4}{15}\\
\Longrightarrow~~&~~\sqrt{3}  \approx  \dfrac{26}{15}.
\end{align*}

In other words, the scribe of this tablet   has considered the less common approximation $\sqrt{3}  \approx  \frac{26}{15} $ to approximate the area of the convex polyarc $ \Lambda_4$. Although this approximate value seems to be rare, there are other mathematical texts using  this approximation in their calculations. One specific example is given in the Achaemenid mathematical tablet   W 23291[This mathematical tablet  belongs to the Late Babylonian period (ca. the first millennium BC). It contains around 26 problems regarding the areas and dimensions of geometric figures like equilateral triangles.] (see [6], for details). It should be noted that this more accurate approximation to $ \sqrt{3} $ can be obtained by using the usual Babylonian method or the approximate formula  $\sqrt{a^2\pm b}\approx a\pm \frac{b}{2a}$ by setting $a=\frac{5}{3}$ and $b=\frac{2}{9}$:

\begin{align*}
\sqrt{3}&=\sqrt{\frac{25}{9}+\dfrac{2}{9}}\\
&\approx \frac{5}{3}+\frac{\frac{2}{9}}{2\times (\frac{5}{3})}\\
\end{align*}

Therefore, the approximate area of the inscribed square $\Gamma_4$ is

\begin{equation}\label{equ-3-ab}
\end{equation}

which is the number in line 5.

It is interesting that if we   use the Babylonian approximations  $\pi\approx 3  $  and   $\sqrt{3}\approx\frac{26}{15} $  in <ref>,  the approximate area of the convex polyarc $\Lambda_4$ is

\[S_{\Lambda_4}\approx\left(\dfrac{3}{3}+1-\dfrac{26}{15}\right) \times 1^2 =   \dfrac{4}{15}=0;16\]

which is the approximate area of the inscribed square  $\Gamma_4$ given in <ref>. Therefore, the scribe of this tablet might have hit two birds with one stone!

§.§ Overlap of three barley-fields

The transliteration and translation of   line  6 are as follows:

Obverse: Line 6

(L6)   16,26,46,40 igi-gub šà gúr šà 3 še i-na šà gúr gar

(L6)  0;16,26,46,40 is the constant of a circular figure  with three barley figures in the middle.

The  scribe refers to the circular figure in this line   as the overlap of  three  barley-field figures. In fact, it is the intersection of three  rotated convex lenses which produces a regular convex polyarc with 6 arcs (see <ref>).  The arrangement of barley figures  are  symmetrical meaning that each one is obtained from another   if we rotate them by $60^{\circ}$.

Overlap of three barley-fields

In line 6, the area of this circular figure, say $ \Lambda_6 $,  is assumed to be $S_{\Lambda_6}=  0;16,26,46,40$.    To calculate the area,   the Susa scribe  of this tablet seems to have assumed the radius  of each quadrant  to be 1   and has omitted the areas of the six small circular segments from his calculations. To verify this, we first compute the area of this regular  convex   polyarc for the general case in which the radius of each quadrant is $r$. If we connect each vertex of the regular polyarc  first  to adjacent vertices and then to its center, we get a partition of the circular figure  consisting  of six   equilateral triangles  and six   small circular segments as shown in  <ref>.

Partition of a  regular convex 6-arc

In line 6, the area of this circular figure $ \Lambda_6 $  is assumed to be $S_{\Lambda_6}=  0;16,26,46,40$.    To calculate the area,   the Susa scribe  of this tablet seems to have assumed the radius  of each quadrant  to be 1   and has omitted the areas of  six small circular segments from his calculations. To verify this, we first compute the area of this convex hexagon for the general case in which the radius of each quadrant is $r$.

According to  <ref>, this convex polyarc is the union of six triangles and six circular segments. We first show that all these triangles are equal and equilateral and also that all the  segments are   equal. In such a case, to find the area we just need to compute the area of the equilateral triangle $\Gamma_3$ and the segment $\Gamma_0$. To do so, we need to consider some auxiliary lines as shown in <ref>.

Partition of a regular convex  6-arc

By our assumption, each barley figure  (convex lens) is obtained from the other one by a 60-degree rotation. So, all central angles of the hexagon $\Gamma_6=ABCDEF$ are equal and have the same measure $60^{\circ}$. By symmetry, all line segments connecting $O$ to the vertices of $\Gamma_6$ are equal. Therefore, the hexagon $\Gamma_6$ is regular and all interior triangles partitioning $\Gamma_6$ are equilateral. Let the common length of all sides of all these triangles be $x$. Since $GI$ and $PL$ are perpendicular, $\angle HOE=90^{\circ}-60^{\circ}=30^{\circ}$ which implies that  $\angle OHE=90^{\circ} $. It follows   that in the equilateral triangle $ \triangle OED$, the line segment $OH$ bisects $\angle EOD$. Thus, $OH$ also bisects the base $ED$ and so

\begin{equation}\label{equ-3-ea}
\overline{EH} = \frac{\overline{ED}}{2}=\frac{x}{2}.
\end{equation}

Now, by the Pythagorean theorem in the right triangle $\triangle OHE $, we have

\[ \overline{HO} = \sqrt{\overline{OE}^2- \overline{EH}^2}=\sqrt{x^2-\frac{x^2}{4}}=\frac{\sqrt{3}}{2}x.\]

Since $ \overline{GP}= \overline{GL}=r$, the triangle $ \triangle PGL$  is isosceles and thus the height $GO$ bisects the base $PL$  and we have  $\overline{OK}=\overline{OL} $. By the Pythagorean theorem in the right triangle  $\triangle POG $ we get

\[ \overline{OG}=\sqrt{\overline{OG}^2}=\sqrt{\dfrac{\overline{OP}^2+\overline{OG}^2}{2}} =\sqrt{\dfrac{\overline{PG}^2}{2}} =\dfrac{r}{\sqrt{2}}= \frac{\sqrt{2}}{2}r.\]

It follows from these calculations that

\begin{equation}\label{equ-3-eb}
\overline{GH}=\frac{\sqrt{3}}{2}x+\frac{\sqrt{2}}{2}r.
\end{equation}

Next, by the Pythagorean theorem in the right triangle $ \triangle EHG $ and equations <ref> and <ref>, we obtain the following quadratic equation

\[\left(\frac{x}{2}\right)^2+\left(\frac{\sqrt{3}x}{2}+\frac{\sqrt{2}}{2}r\right)^2 = r^2,\]

which can be simplified and written as the standard form

\begin{equation}\label{equ-3-ec}
x^2+\left(\frac{\sqrt{6}}{2}r\right)x = \frac{r^2}{2}.
\end{equation}

Note that $x$ is unknown and $r$ is a constant.

By completing the square, which is a standard method in Babylonian mathematics and called takīltum in Akkadian, the solution of the equation  <ref>  is obtained as follows:

\begin{align*}
~~&~~x^2+\left(\frac{\sqrt{6}}{2}\right)x  = \frac{r^2}{2}\\
\Longrightarrow~~&~~x^2+2\times \left(\frac{\sqrt{6}}{4}r\right)x= \frac{r^2}{2}\\
\Longrightarrow~~&~~x^2+2\times \left(\frac{\sqrt{6}}{4}r\right)x+\left(\frac{\sqrt{6}}{4}r\right)^2  = \frac{r^2}{2}+\left(\frac{\sqrt{6}}{4}r\right)^2\\
\Longrightarrow~~&~~\left(x+\frac{\sqrt{6}}{4}\right)^2  = \frac{14}{16}r^2\\
\Longrightarrow~~&~~x+\frac{\sqrt{6}}{4}r  = \frac{\sqrt{14}}{4}r,
\end{align*}

which implies that

\begin{equation}\label{equ-3-f}
x =\left(\frac{\sqrt{14}-\sqrt{6}}{4}\right)r.
\end{equation}

It follows from    equation  <ref>    that the area  of the equilateral triangle   $\triangle OED$  with side $x$ is

\begin{align*}
S_{\triangle OED}&= \frac{\sqrt{3}}{4}x^2\\
&=\frac{\sqrt{3}}{4} \times \left(\frac{\sqrt{14}-\sqrt{6}}{4}\right)^2r^2\\
&=\frac{\sqrt{3}}{4} \times \left(\frac{14+6-2\sqrt{4\times21}}{16}\right)r^2\\
&=\frac{\sqrt{3}}{4} \times \left(\frac{5-\sqrt{21}}{4}\right)r^2\\
&=  \frac{\sqrt{3}}{16}  \left(5-\sqrt{21}\right)r^2,
\end{align*}

\begin{equation}\label{equ-3-g}
S_{\triangle OED}=\frac{\sqrt{3}}{16}  \left(5-\sqrt{21}\right)r^2.
\end{equation}

It remains to compute the area of the segment $\Gamma_0$ which is a part of a circle with radius $r$ separated by the   arc $\arc{ED}$. Let $\alpha$ be the central angle of the segment $\Gamma_0$. It follows from <ref> and <ref> that

\[ \sin\left(\frac{\alpha}{2}\right)=\frac{\overline{EH}}{\overline{EG}}=\frac{\left(\frac{\sqrt{14}-\sqrt{6}}{4}\right)r}{2r}=\frac{\sqrt{14}-\sqrt{6}}{8}, \]

\begin{equation}\label{equ-3-i}
\alpha= 2\arcsin\left(\frac{\sqrt{14}-\sqrt{6}}{8}\right).
\end{equation}

The area of the segment $\Gamma_0$ is the difference between the area of the sector $\Gamma_1$  in quadrant $GPEDL$ separated by the arc $\arc{ED}$ and that of the triangle $\triangle GED$. First note that the area of the sector $\Gamma_1$ is equal to the area of the quadrant times the ratio of their central angles, i.e., $ \frac{2\alpha}{\pi}$.   So, it follows from   <ref>, <ref> and <ref> that

\begin{align*}
S_{\Gamma_0}&= \left(\dfrac{2\alpha}{\pi}\right)\times S_{GPEDL} - S_{\triangle GED}\\
&= \left(\dfrac{2\alpha}{\pi}\right)\times \dfrac{\pi r^2}{4} - \frac{\overline{ED}\times \overline{GH}}{2}\\
&=  \dfrac{\alpha r^2}{2}   -     \left(\frac{\sqrt{3}}{2} x + \dfrac{\sqrt{2}}{2}r\right)\times \left(\dfrac{x}{2}\right) \\
&=  \dfrac{\alpha r^2}{2}  -  \frac{\sqrt{3}}{4} x^2 - \dfrac{\sqrt{2}}{4}   xr \\
&=  \dfrac{\alpha r^2}{2}  -  \frac{\sqrt{3}}{4} \left(\frac{\sqrt{14}-\sqrt{6}}{4}\right)^2 r^2 - \dfrac{\sqrt{2}}{4}   \left(\frac{\sqrt{14}-\sqrt{6}}{4}\right)r^2 \\
&=  \dfrac{\alpha r^2}{2}  -  \frac{\sqrt{3}}{4} \left(\frac{5-\sqrt{21}}{4}\right)  r^2 -   \left(\frac{\sqrt{7}-\sqrt{3}}{8}\right)r^2 \\
&=  \dfrac{\alpha r^2}{2}  -    \left(\frac{5\sqrt{3}-3\sqrt{7}}{16}\right)  r^2 -   \left(\frac{\sqrt{7}-\sqrt{3}}{8}\right)r^2 \\
&=  \dfrac{\alpha r^2}{2}  +    \left(\frac{\sqrt{7}-3\sqrt{3}}{16}\right)  r^2.
\end{align*}

Therefore, we get

\begin{equation}\label{equ-3-j}
S_{\Gamma_0}= \dfrac{\alpha r^2}{2}  +   \left(\frac{\sqrt{7}-3\sqrt{3}}{16}\right)  r^2.
\end{equation}

If we denote the circular part consisting of $\Gamma_3$ and  $\Gamma_0$ by $\Lambda_1$, then  it follows  from <ref>  and <ref> that

\[ 	S_{\Lambda_1}= \dfrac{\alpha r^2}{2}  +   \left(\frac{\sqrt{7}-3\sqrt{3}}{16}\right)  r^2+ \left(\dfrac{5\sqrt{3}-3\sqrt{7}}{16}\right)r^2, \]

or equivalently

\begin{equation}\label{equ-3-k}
S_{\Lambda_1}= \left[\arcsin\left(\frac{\sqrt{14}-\sqrt{6}}{8}\right) +   \frac{\sqrt{3}-\sqrt{7}}{8}\right] r^2.
\end{equation}

Now, we can use <ref> to compute the area of   regular  convex 6-gon $\Lambda_6$. Since $S_{\Lambda_6} =6\times S_{\Lambda_1}$, we obtain

\begin{equation}\label{equ-3-l}
S_{\Lambda_6}= \left[6\arcsin\left(\frac{\sqrt{14}-\sqrt{6}}{8}\right)  +   \frac{3}{4}\left(\sqrt{3}-\sqrt{7}\right)\right]   r^2.
\end{equation}

This formula is obtained from the general formula <ref> if we set $n=6$.  Similarly, the area of the inscribed regular hexagon $\Gamma_6$ is computed from <ref> as follows:

\begin{equation}\label{equ-3-m}
S_{\Gamma_6}= \frac{3\sqrt{3}}{8}  \left(5-\sqrt{21}\right)r^2.
\end{equation}

It should be noted that the general formula <ref> can be obtained using a similar argument to the one   just discussed for the case $n=6$. Another method might be the application of definite integrals.

Now, it is   time to analyze the number  given    in line 6 by the scribe as the constant of the  regular convex polyarc $\Lambda_6$. As usual, the scribe has assumed the radius $r$ to be 1 and approximated the area of $\Lambda_6$ to that of   the inscribed regular hexagon $\Gamma_6$. It should be noted that to compute the area of   $\Lambda_6$ one must  compute  the value of angle $\alpha$ which was impossible for the Susa scribe.  So the Susa scribe of this tablet has had to ignore the areas of the six circular segments in his calculations.  As we can see from <ref>, there are two irrational square roots in the   formula  for the area of the inscribed regular  hexagon $\Gamma_6$. This must have made  the scribe to apply the  approximate values  in his calculations.   It is better to consider different possible approximations to  these two square roots.

Although    the most common approximation $ \sqrt{3}\approx \frac{7}{4}$ must have been applied here,   we  need to  check different approximations for the square root of 21. To do so, we   apply the usual Babylonian method for approximating square roots     using the formula $x_{n+1}=\frac{1}{2}(x_n+\frac{21}{x_n})$ when $x_0$ is known.[For more details, see [5, 6, 7, 8, 10, 11].] Since $4<\sqrt{21}<5$, we may use $x_0=4,5,4\frac{1}{2}$. The first three terms  for each choice of $x_0$   are shown     in <ref>. Note that the decimal fractions are written to five sexagesimal places. Also, the exact value of $\sqrt{21}$ is $ 4;34,57,16,21,0,23,51,\ldots $.

$x_1 $     $x_2$     $x_3 $

$x_0= 4$    $\frac{37}{8}\approx4;37,30 $     $ \frac{2713}{592}\approx4;34,57,58,22,42$     $ \frac{14720113}{3212192}\approx4;34,57,16,21,3 $

$x_0= 5$    $\frac{23}{5}=4;36$     $ \frac{527}{115}\approx 4;34,57,23,28,41$     $ \frac{277727}{60605}\approx 4;34,57,16,21,0 $

$x_0= 4;30$    $\frac{55}{12}\approx 4;35 $     $ \frac{6049}{1320}\approx 4;34,57,16,21,49$     $ \frac{73180801}{15969360}\approx 4;34,57,16,21,0 $

Possible approximations of $ \sqrt{21}  $

By using the approximations of $\sqrt{21}$ given in  <ref> and the usual approximation $\sqrt{3}\approx\frac{7}{4}$, we can  compute   the  possible  approximations  of $\frac{3\sqrt{3}}{8}  \left(5-\sqrt{21}\right)$ which are  shown in <ref>.

$x_1 $     $x_2$     $x_3 $

$x_0=4$   	$0;14,45,56,13$    $0;16,25,18,51,4 $     $ 0;16,26,9,53,40$

$x_0=5$   	$0;15,45$         $0;16,26,5,13,2 $     $0;16,26,9,53,42 $

$x_0=4;30$   	$0;16,24,22,30$     $0;16,26,9,53,10$      $0;16,26,9,53,42 $

Possible approximations of $\frac{3\sqrt{3}}{8}  \left(5-\sqrt{21}\right)$

It follows from the data of  <ref> that a reasonable approximate value of $S_{\Gamma_6} $ up to five sexagesimal places is $0;16,26,9,53$. One  might expect  the scribe to have obtained this value, however,  the scribe's value  is $0;16,26,46,40$ which is  a little bigger than the obtained approximation. Two reasonable conjectures for this error are either that (1) the scribe has used different approximations of $ \sqrt{3}$ and $\sqrt{21}$ or  (2) he  has made a mistake in his calculations.

Let us consider different approximations for two square roots $ \sqrt{3}$ and $\sqrt{21}$ and compute the corresponding approximations of  $\frac{3\sqrt{3}}{8}  \left(5-\sqrt{21}\right)$. We have considered 11 initial values  for $\sqrt{21}$ and 13 initial values for $\sqrt{3}$. The following table shows the corresponding approximate value for each square root and $\frac{3\sqrt{3}}{8}  \left(5-\sqrt{21}\right)$. As we can see from  <ref>, none of the obtained values equal to the scribe's value $0.274104938$, but the closest one is given in the shaded cell, i.e., $ 0.2740178571$.

Different approximations of $\frac{3\sqrt{3}}{8} \left(5-\sqrt{21}\right)$

From the data of   <ref> and the discussion above we can come to the conclusion that the scribe must have made a mistake in the process of approximating $\sqrt{21}$, because he   had to perform rather complicated calculations in the sexagesimal system.

Finally, we may compute the exact area of the  regular  convex 6-gon. If we set $r=1$ in <ref>, then  $S_{\Lambda_6}=  0;17,17,12,44,20,29,36,54,\cdots $. Hence, the error percentage of scribe's approximation is

\[ e=\left|\frac{0;17,17,12,44,20,29,36,54,\cdots -0;16,26,46,40}{0;17,17,12,44,20,29,36,54,\cdots}\right|\times 100\%\approx 1.4\% \]

which, despite the scribe's mistake,  is a relatively good approximation.

§.§ An apusamikkum: a hole of lyre

The transliteration and translation of   lines  22-24 are as follows:

Obverse: Lines 22-24

(L22) 26,40 igi-gub šà a-pu-sà-am-mi-ki

(L23) 1,20 bar-dá šà a-pu-sà-mi-ki

(L24) 33,20 pi-ir-ku šà a-pu-sà-mi-ki

(L22) 0;26,40 is the constant of (a figure called) apusamikkum.

(L23) 1;20 is the diagonal  of (a figure called) apusamikkum.

(L24) 0;33,20 is the transversal of (a figure called) apusamikkum.

The figure  described in these lines is called an apusamikkum, which is a  regular  concave 4-arc. It is constructed by repeating (rotating) a circle  of radius $r$ four times around a bigger circle   such that they are pairwise tangent. It also can be made by removing four equal quarters of radius $r$ from a square of side $2r$ (see <ref>).  It should be noted that the Akkadian word  apusamikkum  is derived from the Sumerian word  qpláb-zà-mí “the hole of a lyre”, where qpláb $ \approx $ ab means “hole” (see [9]).

Construction of a  regular concave 4-arc: apusamikkum

In lines 22-24, three numbers are listed as the  area, the diagonal and the transversal of the   apusamikkum $\Phi$. We computed the area for the general case in Section 2. If we set $n=4$ in <ref> and simplify, we get

\begin{equation}\label{equ-3-gee}
\end{equation}

where $a$ is the arc length of each quarter. The other two numbers are called the diagonal and the transversal of the polyarc $\Phi$. The  transversal seems to be the intersection of the diagonal   of the circumscribed square with the  apusamikkum $\Phi$, which is the line segment $KL$ in <ref>. The diagonal is the horizontal or vertical diagonals $HF$  or $EG$ whose length is $2r$. So,

\begin{equation}\label{equ-3-ge}
\overline{HF}=2r.
\end{equation}

Area of an apusamikkum

To compute the transversal $KL$, first note that
$\overline{KL}=\overline{AC}-(\overline{AK}+\overline{CL}), $
and since $\overline{AK}=\overline{CL}=r$, we get

\begin{equation}\label{equ-3-gea}
\overline{KL}=\overline{AC}-2r.
\end{equation}

It remains to find $\overline{AC}$.  By the Pythagorean theorem in the right triangle $ \triangle ABC$, since $\overline{AB}=\overline{CB}=2r $,   we have

\begin{equation}\label{equ-3-gf}
\overline{AC}=2\sqrt{2}r.
\end{equation}

Now, since  $a$ is the quarter of the circumference of a circle with radius $r$, we have$a=\frac{\pi r}{2}$ or $r=\frac{2a}{\pi}$. So, <ref> and <ref> imply that

\begin{align*}
\overline{KL} &=\overline{AC}-2r\\
&=  2\sqrt{2}r - 2r \\
& = 2(\sqrt{2} -1)r \\
&=  2(\sqrt{2} -1)\times \frac{2a}{\pi},
\end{align*}

\begin{equation}\label{equ-3-geb}
\overline{KL}= \dfrac{4(\sqrt{2}-1)a}{\pi}.
\end{equation}

Now, we can check the scribe's numbers by assuming $a=1$ and using the Babylonian approximations $\pi\approx 3$ and $\sqrt{2}\approx \frac{17}{12}=1;25$. It follows from <ref> that

\[ \overline{HF}=\dfrac{4a}{\pi}\approx\frac{4\times 1}{3}=4\times (0;20)=1;20 \]

which is the number in line 23. By <ref>, we have

\[ 	\overline{KL}= \dfrac{4(\sqrt{2}-1)a}{\pi}\approx   \dfrac{4\times (1;25-1)\times 1}{3}=\frac{4\times (0;25)}{3}=(1;20)\times (0;25)=0;33,20 \]

which is the number in line 24. The approximate area is obtained from <ref>:

\[ S_{\Phi}=\frac{(16-4\pi)a^2}{\pi^2} \approx \frac{(16-12)\times 1}{9}=\frac{4}{9}= 0;26,40 \]

which is listed in line 22. Again, the numbers of the Susa scribe here are correct approximate values of the  area, the diagonal and the transversal of an   apusamikkum whose arc length is unit.

§.§ An apusamikkum with three vertices

The transliteration and translation of   line   25 are as follows:

Obverse: Line 25

(L25) 15 igi-gub šà a-pu-sà-mi-ik-ki šà 3

(L25) 0;15 is the constant of (a figure called) apusamikkum  of three vertices.

The figure    in this line   is a regular concave 3-arc. Beside the standard way, it can be made by removing three equal one-sixths of a circle with radius $r$ from a equilateral triangle with side $2r$ (see <ref>).

Construction of an apusamikkum with three vertices

In line 25, the area of this figure $\Omega$ is listed assuming that the radius of the circle is $r=1$. The exact value of the area is obtained from <ref> by setting $n=3$ and $a=\frac{\pi r}{3}$:

\begin{align*}
S_{\Omega}&= \left[\frac{27}{\pi^2}\times \cot\left(\frac{\pi}{3}\right)-\frac{9}{2\pi}\right]a^2\\
&= \left(\frac{27}{\pi^2\sqrt{3}}-\frac{9}{2\pi}\right)\times\frac{\pi^2r^2}{9} \\
&=  \sqrt{3}r^2 -  \frac{\pi r^2}{2},
\end{align*}

\begin{equation}\label{equ-3-gef}
S_{\Omega}=\left(\sqrt{3}  -  \frac{\pi}{2}\right)r^2.
\end{equation}

If we set $r=1$ in <ref> and use the Babylonian approximations  $ \sqrt{3}\approx\frac{7}{4} $  and $\pi\approx3 $,  we get

\begin{align*}
S_{\Omega}=\left(\sqrt{3}  -  \frac{\pi}{2}\right)r^2 \approx  \left(\frac{7}{4} -  \frac{3}{2}\right)\times 1= 1;45-1;30=0;15
\end{align*}

which is the number listed in line 25   verifying the Scribe's number.

§ SIGNIFICANCE OF CIRCULAR FIGURES

All the calculations and explanations we have given in the previous sections demonstrate the mathematical importance of these constants. Every mathematician or   student of mathematics  understands the high level of mathematical skill needed to deal with these geometric figures. The Susa scribes were  the skilled “geometers” of their age and we believe this remarkable list of geometric constants identifies the special importance of geometry in   Elamite mathematic.

Surprisingly, some of these circular figures were used to decorate Elamite artifacts. In the book “The Art of Elam” published in 2020, there are  pictures of many artifacts excavated from ancient sites   (mostly Susa) in the southwest of Iran. Some of these artifacts use the circular figures that we discussed here.  The following table gives the photo reference for some of these usages.

Circular Figure	    Photo Reference

barley-field 	    [1]:  Plate 12-c,  Plate 12-g

apusamikkum  	   [1]:   Plate 12-b, Plate 49-f

with four vertices                          	     Louvre Collection: SB23397, A58786

apusamikkum	   [1]: Plate 11-e, Plate 11-j,  Plate 11-m

with three vertices                                                   	    Louvre Collection: SB19519, SB23369, SB23386, SB23462

Examples of circular figures in Elamite artifacts

It is interesting that the decoration on some artifacts use similar constructions to those of the  circular figures we have discussed in this article. For example, inside two terracotta vases[The reference numbers of these item are SB23369 and SB23386. The reader can see their photos on the Louvre collection website: <https://collections.louvre.fr/ark:/53355/cl010174158> and <https://collections.louvre.fr/ark:/53355/cl010174136>.] dated to ca. 4000–3500 BC,  three almost pairwise tangent circles   enclose a small  regular concave 3-arc.

A similar theme  is  found inside another terracotta vase[The reference number of this item is SB23397 and the reader can see its photo on the Louvre collection website: <https://collections.louvre.fr/ark:/53355/cl010173875>.] from the same era. In this pattern, a chain of five equal repeating circles encloses another circle and the space  between five surrounding  circles and the middle one  is occupied by    five        regular  concave 3-arcs whose union forms the corners of a     regular concave 5-arc.  A reconstruction of this pattern is shown in <ref>.

Reconstruction of the pattern inside a terracotta vase: SB23397

Besides these examples, there are many other Elamite artifacts whose decorations contain  different geometric patterns. The examples given  in this section may be followed up by the interested reader  either through visiting    the Louvre collection website or consulting the book “The Art of Elam” to see additional such patterns (see [1]). In an ongoing book project on Elamite mathematics, the authors have attributed two chapters to categorize the geometric patterns used in   Elamite art. This part of the book demonstrates the wide variety of geometric shapes and  patterns that Elamite artists used in their artworks implying a  close connection between geometry and Elamite art.

§ CONCLUSION

The detailed analysis of some of the geometric constants in  this tablet  clearly suggests  that the Susa scribes  enjoyed   considerable knowledge about two-dimensional figures and   plane geometry. In particular, computing the area   of       complex geometric  figures   reveals  that the scribes of Susa  knew  those  figures very well and the methods needed to approximate their areas.  As we saw, some of these calculations are  complicated and performing them requires a high level of mathematical skill and experience.

As Bruins   rightly has  pointed out   [3], the list of geometric  constants  in   this text provides a documentary evidence that Susa scribes    performed some of their calculations from a purely  mathematical point of view, or in his words “science for science”.      This observation     suggests  that  in  ancient Susa      mathematical skills were most likely taught   in  an  educational setting   which might reasonably be named  “The Susa School of Mathematics”[As is known from Mecquenem's report  (see the introduction of the TMS [4]), all the  26 Susa mathematical  tablets   were collected in 1933 under a large paving $30m$ wide and $15m$ long, at the  Ville Royale, where, in 1934–35, father Van der Meer also collected some school texts which he published in  the M'emoires de la Mission Arch'eologique de Iran: Volume XVII. All these artifacts taken together suggest the existence of a special educational establishment  in  ancient Susa.].

Elamite artists and craftsmen  were  obviously interested in using various complicated and beautiful geometric patterns to decorate their works and this application of geometric patterns in Elamite art is strong evidence for the relationship between geometry and art in   ancient Elam.

[1]
J. 'Alvarez-Mon, The Art of Elam,  CRC Press,   2020.

[3]
E. M. Bruins, Quelques Textes Math'ematiques De La Mission De Suse, Proceedings of the Amsterdam Academy, volume 53, deel 7 (1950), pp. 1025–1033: <https://www.dwc.knaw.nl/DL/publications/PU00018846.pdf>.

[4]
E. M. Bruins and M. Rutten,    Textes Math'ematiques de Suse [TMS], Librairie Orientaliste Paul Geuthner,   Paris, 1961.

[5]
D. Fowler  and E. Robson, Square Root Approximations in Old Babylonian Mathematics: YBC 7289 in Context, Historia Mathematica  25 (1998), pp. 366-378.

[6]
J. Friberg, Amazing Traces of a Babylonian Origin in Greek Mathematics, World Scientific, 2007.

[7]
J. Høyrup,   Lengths, Widths, Surfaces: A Portrait of Old Babylonian  Algebra and Its Kin, Springer,  Studies and Sources in the History of Mathematics and Physical Sciences, New York,   2002.

[8]
K. Muroi, Extraction of Square Roots in Babylonian Mathematics, Historia Scientiarum Vol. 9-2 (1999), pp. 127–133.

[9]
K. Muroi, Mathematics Hidden Behind the Two Coefficients of Babylonian Mathematics, 	arXiv:1305.0865 [math.HO], 2013, 10 pages: <https://arxiv.org/pdf/1305.0865.pdf>.

[10]
O. Neugebauer, Astronomical Cuneiform Texts: Babylonian Ephemerides of the Seleucid Period for the Motion of the Sun, the Moon, and the Planets, Sources in the History of Mathematics and Physical Sciences, Vol. 5, Springer,    1955.
[11]
O. Neugebauer, The Exact Science in Antiquity, Second Edition, Dover Publication Inc., New York,   1969.


# Learning Safety Constraints From Demonstration Using One-Class Decision
Trees

Mattijs Baert, Sam Leroux, Pieter Simoens

###### Abstract

The alignment of autonomous agents with human values is a pivotal challenge
when deploying these agents within physical environments, where safety is an
important concern. However, defining the agent’s objective as a reward and/or
cost function is inherently complex and prone to human errors. In response to
this challenge, we present a novel approach that leverages one-class decision
trees to facilitate learning from expert demonstrations. These decision trees
provide a foundation for representing a set of constraints pertinent to the
given environment as a logical formula in disjunctive normal form. The learned
constraints are subsequently employed within an oracle constrained
reinforcement learning framework, enabling the acquisition of a safe policy.
In contrast to other methods, our approach offers an interpretable
representation of the constraints, a vital feature in safety-critical
environments. To validate the effectiveness of our proposed method, we conduct
experiments in synthetic benchmark domains and a realistic driving
environment.

## Introduction

Reinforcement Learning (RL) has made significant strides in training
autonomous agents, but as these systems become more advanced, ensuring their
safety and alignment with human intentions, often referred to as the alignment
problem (Russell 2019), is becoming a critical concern. As a result, the
objectives of autonomous agents extend beyond their primary task goals, also
encompassing safety constraints, human values, legal regulations, and various
other factors. For instance, an autonomous vehicle’s objective is not only to
get passengers quickly to their destination but also to abide by traffic laws
and ensure passenger comfort. To address these challenges and enhance the
safety of RL agents, Constrained Reinforcement Learning (CRL) methods have
emerged. CRL is designed to obtain safe RL agents by incorporating constraints
into the learning process. CRL methods help ensure that the RL agent operates
within predefined boundaries, reducing the risk of harmful or unintended
actions. However, one key limitation of CRL methods is that they assume the
availability of a well-defined set of constraints. Specifying these
constraints correctly and completely is a complex task on its own (Krakovna et
al. 2020). It often requires a deep understanding of the environment,
potential risks, and ethical considerations. Therefore, addressing the
alignment problem in RL goes beyond the development of CRL methods; it also
involves addressing the challenge of correctly specifying constraints as
necessary. This research paper introduces a novel approach to reduce the need
for manual specification of constraints by learning from expert
demonstrations. First, a model of the expert behavior is learned as a one-
class decision tree (Itani, Lecron, and Fortemps 2020). From this tree a
logical formula in disjunctive normal form is extracted defining the
constraints. Next, we demonstrate that the learned constraints can be used by
a CRL method to learn a constraint abiding (i.e. safe) policy in synthetic and
realistic environments. A notable advantage of this approach is the
interpretability of both the learned expert model and the constraints
themselves. By monitoring the evaluation-violation ratio of the constraints
during training, we can refine the set of constraints after training, further
enhancing interpretability. Additionally, as constraints are frequently shared
across multiple agents and tasks, once they have been acquired, they can be
seamlessly applied to other agents undertaking diverse tasks. This eliminates
the need to separately learn constraints for each individual agent and task.
We assess the effectiveness of the proposed approach through an evaluation on
a series of synthetic benchmarks for safe RL (Ray, Achiam, and Amodei 2019)
and in a real-world driving scenario (Krajewski et al. 2018).

## Background

### Markov Decision Process

A Markov decision process (MDP) is characterized by a state space
$\mathcal{S}$, an action space $\mathcal{A}$, a discount factor $\gamma$
within the range [0,1], a transition distribution denoted as $p(s^{\prime}\mid
s,a)$, an initial state distribution represented by $\mathcal{I}(s)$, and a
reward function $R:\mathcal{S}\times\mathcal{A}\mapsto[r_{\min},r_{\max}]$.
Within this framework, an agent interacts with the environment at discrete
time steps, generating a sequence of state-action pairs known as a trajectory
$\tau=((s_{0},a_{0}),...,(s_{T-1},a_{T-1}))$, where $T$ is the length of the
trajectory. The cumulative reward of a trajectory is calculated as the sum of
rewards, each discounted by a factor of $\gamma^{t}$, where t represents the
time step: $R(\tau)=\sum_{t=0}^{T-1}\gamma^{t}R(s_{t},a_{t})$. At each
discrete time step, the agent’s choice of action is governed by a policy
$\pi$, which is a function that maps states from the state space $\mathcal{S}$
to a probability distribution over actions from the action space
$\mathcal{A}$. The primary objective of forward reinforcement learning is to
find a policy $\pi$ that maximizes the expected sum of discounted rewards,
expressed as $J_{r}(\pi)=\mathbb{E}_{\tau\sim\pi}R(\tau)$.

### Constrained Reinforcement Learning

Constraints provide a natural and widely applicable means of specifying safety
requirements in various contexts (Ray, Achiam, and Amodei 2019). Within the
domain of Constrained Reinforcement Learning (CRL), the prevailing framework
for problem modeling is the Constrained Markov Decision Process (CMDP) (Altman
1999). CMDPs extend the traditional Markov Decision Processes (MDPs) by
introducing a non-negative bounded cost function, denoted as
$C:\mathcal{S}\times\mathcal{A}\mapsto[c_{\min},c_{\max}]$, and a budget
parameter $\alpha\geq 0$. In this context, $C(s,a)$ quantifies the cost
associated with taking action $a$ in state $s$, while the cumulative cost of a
trajectory is defined as the summation of discounted costs:
$C(\tau)=\sum_{t=0}^{T-1}\gamma^{t}C(s_{t},a_{t})$. The objective in CRL is to
find an optimal policy that maximizes the expected sum of discounted rewards,
as denoted by $J_{r}(\pi)$, while adhering to specific constraints defined by
the cost function $C$. Formally, the optimal policy $\pi^{*}$ is obtained
through the following optimization problem:

$\pi^{*}=\arg\max_{\pi}J_{r}(\pi)\;\textrm{s.t.}\;J_{c}(\pi)<\alpha.$ (1)

Here, $J_{c}(\pi)=\mathbb{E}_{\tau\sim\pi}C(\tau)$ represents the expected sum
of discounted costs, and $\alpha$ signifies a limit on the allowable cost.

### One-Class Decision Tree

One-class classification (OCC) is a machine learning paradigm that involves
training a model to classify instances into a single well-defined class,
treating all other data points as anomalies or outliers. One-class
classification trees (OC-trees) (Itani, Lecron, and Fortemps 2020) provide an
interpretable approach for addressing OCC problems. Building on kernel density
estimation, OC-trees aim to represent target areas in the input space that
describe the training data. We consider the training data as a set of $M$
instances $X=\\{x_{0},x_{1},\dotsc x_{M-1}\\}$. Each instance is characterized
by a k-dimensional feature vector, with $x^{j}_{i}$ representing the $j$-th
feature of the $i$-th instance. We define $\chi\subset\mathbb{R}^{k}$ as a
k-dimensional hyper-rectangle that encompasses all training instances. The
primary objective is to partition the initial hyper-rectangle $\chi$ into
distinct subspaces $\chi_{t}$, represented by tree nodes $t$, such that the
learned sub-spaces encompass the training data. The sub-space $\chi_{t}$
associated with node $t$ is divided into one or more sub-spaces
$\chi_{t_{n}}=\\{x\in\chi_{t}:\textrm{L}_{t_{n}}\leq
x^{j}\leq\textrm{R}_{t_{n}}\\}$ with $L_{t_{n}}$, $R_{t_{n}}\in\mathbb{R}$,
$n\in\\{0,...,N-1\\}$ and $N$ the number of children of $t$. At a given node
$t$, several steps are carried out for each dimension to determine the
dimension $j$ which best cuts the data in multiple sub-spaces:

  * •

Estimate the probability density function $\hat{f}_{j}(x)$ along dimension $j$
using kernel density estimation, based on the available training samples
$x\in\chi_{t}$.

  * •

Divide the subspace $\chi_{t}$ along dimension $j$ based on the modes of the
estimated density function $\hat{f}_{j}(x)$ into $N$ intervals defined by
their left and right bound $L_{t_{n}}$ and $R_{t_{n}}$. Note, that $N$ depends
on the number of modes of $\hat{f}_{j}(x)$.

  * •

Evaluate the quality of the division using a measure of impurity.

The dimension that yields the best purity score is selected to partition the
subspace $\chi_{t}$.

## Method

Figure 1: Simple navigation environment with a two dimensional feature space.
The red region represents the ground truth constraints. The solid lines
represent expert trajectories and the dashed line, trajectories a learning
agent would take which is unaware of the constraints.

Our approach builds upon the insight, as discussed by Lindner et al. (2023),
that feature expectations of policies adhering to the genuine constraints form
a convex set (i.e. safe set) within the feature vector space. If, for some
policy, we can guarantee that the feature expectations are enclosed by the
safe set then we can guarantee this policy is safe. Our method encompasses
four key steps: firstly, a safe convex set is established through constructing
an OC-tree; following this, constraints are extracted in the form of a logical
formula; then, we employ a CRL method to train a policy that conforms to these
constraints; and lastly, we propose an approach for pruning the learned
formula after training.

### Learning a Safe Set

In this section, we discuss the process of acquiring a representation of safe
behavior. We start with a collection of expert trajectories denoted as
$\mathcal{T}$. We define a fixed mapping from state-action tuples to a bounded
k-dimensional feature space as
$\phi:\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R}^{k}$. Our goal is to
learn a safe convex set in this feature space. To accomplish this, we
construct a dataset $\mathcal{D}$ from the set of trajectories, defined as:
$\mathcal{D}=\\{\phi(s,a):\forall~{}(s,a)\in\tau;\forall~{}\tau\in\mathcal{T}\\}$.
Training an OC-tree (Itani, Lecron, and Fortemps 2020) on the dataset
$\mathcal{D}$ constructs a region in the feature space by taking the union of
hyper-rectangles. Since hyper-rectangles themselves are convex, and the union
operation is known to maintain convexity, it follows that the OC-tree
effectively defines a convex set. To illustrate this concept, let’s consider a
simplified environment characterized by a two-dimensional feature space. In
this scenario, the first dimension, denoted as $\phi^{0}$, corresponds to the
agent’s x-coordinate, while the second dimension, $\phi^{1}$, represents the
agent’s y-coordinate. Figure 1 visually represents this environment along with
three expert trajectories depicted as solid lines. The agent is tasked with a
straightforward navigation assignment, starting from an initial feature vector
of $[0.1,0.1]$ and aiming to reach a goal vector of $[0.9,0.9]$. The red
rectangle in Figure 1 represents a constrained area within the feature space.
Figure 2 showcases the OC-tree that has been learned from the provided
trajectories, which we refer to as the expert tree or expert model. In this
specific instance, the expert behavior is enclosed by two rectangles within
the feature space.

Figure 2: Expert behavior tree learned from trajectories presented in figure
1. The root of tree encompasses the complete feature space. Every other node
defines an interval along one dimension such that the complete tree represents
the safe set as multiple hyper-rectangles. This tree represents two rectangles
in the two dimensional feature space.

### Formula Extraction

Our objective is to acquire a logic formula $\varphi$ in Disjunctive Normal
Form (DNF) that captures the constraints governing the expert’s behavior.
$\varphi$ should evaluate true for the portion of the feature space not
covered by the safe set represented by the expert tree. For a tree defined by
its root node $t$ and a given feature vector $\phi$, the corresponding formula
$\varphi_{t}$ is recursively defined as follows:

$\displaystyle\begin{split}&\varphi_{t}=\perp\vee\bigvee_{t_{\textrm{child}}\in
t}\bigg{(}(\phi^{j}<L_{t_{\textrm{child}}})\vee(\phi^{j}>R_{t_{\textrm{child}}})\\\
&\vee\big{(}(\phi^{j}\geq L_{t_{\textrm{child}}})\wedge(\phi^{j}\leq
R_{t_{\textrm{child}}})\wedge\varphi_{t_{\textrm{child}}})\big{)}\bigg{)}.\end{split}$
(2)

In this context, $j$ denotes the split dimension corresponding to node $t$.
Note that $\varphi_{t}$ evaluates to $\perp$ if $t$ is a leaf node. When
$\varphi$ evaluates to $\top$, it signifies that the input feature vector
corresponds to a state-action tuple that violates the constraints. For the
simple navigation examples the following formula can be extracted from the
expert tree presented in Figure 2:

$\displaystyle\varphi=(\phi^{0}<0.1)\vee(\phi^{0}>0.9)$
$\displaystyle\vee\left((\phi^{0}>0.1)\wedge(\phi^{0}<0.7)\wedge(\phi^{1}<0.1)\right)$
$\displaystyle\vee\left((\phi^{0}>0.1)\wedge(\phi^{0}<0.7)\wedge(\phi^{1}>0.3)\right)$
$\displaystyle\vee\left((\phi^{0}>0.7)\wedge(\phi^{0}<0.9)\wedge(\phi^{1}<0.1)\right)$
$\displaystyle\vee\left((\phi^{0}>0.7)\wedge(\phi^{0}<0.9)\wedge(\phi^{1}>0.9)\right).$

Additionally, for each dimension $j$, two more rules are incorporated to
invalidate feature representations where $\phi^{j}$ is lower than the observed
minimum in the expert trajectories:
$\min\,(\phi^{j}\>:\>\forall\phi\in\mathcal{D})$ or higher than the observed
maximum: $\max\,(\phi^{j}\>:\>\forall\phi\in\mathcal{D})$.

### Constraint-Based Cost Function and Optimization

The extracted formula $\varphi$ gives rise to a cost function that can be
utilized within a CRL framework. In cases where a constraint is violated, the
agent incurs a cost of 1; conversely, if all constraints are adhered to, the
cost remains at 0. To tackle this constrained problem, we employ the
Lagrangian method in conjunction with Proximal Policy Optimization (PPO)
(Schulman et al. 2017). This constrained problem can then be addressed as an
unconstrained max-min optimization problem, and we employ the implementation
provided by the Omnisafe framework (Ji et al. 2023). It’s noteworthy that the
PPO-Lagrangian approach, while conservative, has been shown to yield
comparable or superior results to other methods such as Constrained Policy
Optimization (CPO) (Achiam et al. 2017), Projected Constrained Policy
Optimization (PCPO) (Yang et al. 2020), and First-Order Constrained Policy
Optimization with Penalty (FOCOPS) (Zhang, Vuong, and Ross 2020) on various
safety gym benchmark environments, as discussed by Ji et al. (2023) and Ray,
Achiam, and Amodei (2019).

(a) PointGoal1 (b) PointPush1

(c) CarGoal1 (d) CarPush1

Figure 3: Reward (top) and ground truth cost (bottom) during training of
agents in the synthetic benchmark environments.

### Refining Constraint Definitions

It is important to mention that the learned tree could serve directly as a
cost function. However, if our intention is to prune the set of constraints to
improve interpretability, the DNF description of the constraints becomes more
practical. We interpret each conjunction in $\varphi$ as an individual rule or
constraint. During the training of the CRL agent, we maintain a record of the
number of times each conjunction is evaluated and how many times it evaluates
to true (i.e. the corresponding constraint is violated). Conjunctions that
exhibit a violation-evaluation ratio below a predefined threshold are pruned
from the constraint definition. The rationale behind this approach lies in the
fact that during training, the learning agent strives to maximize the
discounted sum of rewards. When rules are violated during the execution of
highly rewarding behavior, it suggests that these rules likely correspond to
genuine constraints since the expert agent actively avoided these states.
Conversely, when rules are never violated, they are less likely to be true
constraints, as they pertain to regions in the feature space that are scarcely
visited by the learning agent, indicating low reward potential. When
evaluating a feature representation, the conjunctions are assessed in
ascending order of complexity. As a result, conjunctions with fewer literals
are given preference over more intricate rules. For the simple navigation
example, the pruned constraints are defined as follows:

$\varphi=(\phi^{0}>0.1)\wedge(\phi^{0}<0.7)\wedge(\phi^{1}>0.3).\\\ $

It is worth highlighting that if we examine the trajectories taken by an
initial learning agent, as depicted by the dashed lines in Figure 1, we will
observe a notably high violation-evaluation ratio for this rule. On the
contrary the rules which are pruned describe areas within the feature space
where the potential rewards are limited, and consequently, these regions are
seldom explored by the learning agent.

## Results

We evaluate our method on a set of synthetic safe RL benchmark domains
proposed by Ray, Achiam, and Amodei (2019) and a realistic highway environment
(Krajewski et al. 2018). All results are averaged over 10 trials with random
seeds. In the appendix, we provide examples of the learned formulas.

source/target | PointGoal1 | PointPush1 | CarGoal1 | CarPush1
---|---|---|---|---
PointGoal1 | $13.9\pm 1.7$ | $0.6\pm 0.3$ | $4.9\pm 2.1$ | $0.7\pm 0.3$
PointPush1 | $0.5\pm 0.2$ | $0.6\pm 0.1$ | $0.1\pm 0.2$ | $0.4\pm 0.1$
CarGoal1 | $12.2\pm 3.6$ | $0.9\pm 0.3$ | $15.1\pm 3.1$ | $0.7\pm 0.2$
CarPush1 | $0.1\pm 0.1$ | $0.5\pm 0.1$ | $0.3\pm 0.3$ | $0.5\pm 0.1$
Table 1: Transferring constraints between agents and tasks: reward source/target | PointGoal1 | PointPush1 | CarGoal1 | CarPush1
---|---|---|---|---
PointGoal1 | $27.9\pm 5.0$ | $54.5\pm 16.7$ | $35.1\pm 11.6$ | $47.4\pm 9.3$
PointPush1 | $4.5\pm 1.7$ | $26.2\pm 7.3$ | $6.5\pm 3.9$ | $18.7\pm 8.2$
CarGoal1 | $41.0\pm 8.0$ | $50.3\pm 12.8$ | $33.7\pm 5.3$ | $48.7\pm 9.9$
CarPush1 | $13.9\pm 4.4$ | $25.7\pm 8.3$ | $14.4\pm 3.5$ | $24.6\pm 10.6$
Table 2: Transferring constraints between agents and tasks: ground truth cost

### Synthetic Environments

We assess two categories of agents: The Point agent, a basic robot confined to
a 2D plane, equipped with two actuators for rotation and forward/backward
movement, and the Car agent, a somewhat more complex robot with two
independently driven parallel wheels and a free-rolling rear wheel. In the
case of the Car robot, both steering and forward/backward movement necessitate
coordination between the two drives. Our evaluation encompasses two distinct
tasks: Goal and Push. In the Goal task, the agent’s aim is to reach a
specified goal location while circumventing hazards. The Push task involves
the agent pushing a box to the designated goal position while avoiding
hazards. The feature space for the various agents is characterized by the
agent’s acceleration and velocity along both the x and y axes, in addition to
the data from 16 lidar sensors: ${a_{x},a_{y},v_{x},v_{y},d_{l0:15}}$. The
lidar sensor readings indicate the distance in each of the 16 directions
around the agent to a hazard. The lidar values range from 0 to 1, with a
reading of 1 signifying that the agent is in a hazardous area.
Figure 3 illustrates the acquired rewards and costs during the training of
different agents. These agents include one provided with ground truth
constraints and trained using PPO-Lagrangian (i.e. expert), another trained
with expert demonstrations using Maximum Causal Entropy Inverse Constrained
Reinforcement Learning (MCE-ICRL) (Baert et al. 2023) as a state-of-the-art
inverse constrained reinforcement learning method, and an agent trained with
expert demonstrations using our proposed method (i.e. ours). We leverage the
expert agent’s policy, trained using the ground truth constraints, to sample
trajectories, which serve as input expert trajectories for both MCE-ICRL and
our method. For both the Goal and Push tasks our method approximates the
reward and cost achieved by the expert. The Push task presents a significantly
greater challenge compared to the Goal task, as is reflected by the lower
rewards obtained. Even when provided with ground truth constraints, current
CRL methods struggle to learn a satisfactory policy. Consequently, the expert
trajectories will be sub-optimal, inevitably impacting the performance of our
method, as these trajectories serve as its fundamental input. In essence, the
efficacy of our method is upper bounded by the quality of the expert
demonstrations. Furthermore, the same oracle CRL method is part of our
proposed approach, which raises concerns that its failure in cases where
ground truth constraints are available may also extend to situations where it
operates with learned constraints. It is notable that MCE-ICRL fails to
recover a satisfactory policy in our experiments. This can be attributed to
the complexity of the environments which significantly surpasses those
utilized in the original paper (Baert et al. 2023).
One key advantage of defining or learning constraints is that they often are
the same for various agents and tasks. This is beneficial because we can learn
the constraints once and use them in multiple settings. To this end, we assess
to what extent the constraints learned from demonstrations of an agent of type
A performing task X can be transferred to agents of type B performing task Y.
In table 1 and 2, we present the reward and cost, respectively, acquired by
agents after training. The rows indicate the environment in which the rules
are learned (source domain) and the columns correspond with the environment in
which the agent is trained (target domain). The main diagonal contains the
results when the source and target domain are the same. The best results for a
given target domain are achieved when the source and target domains are the
same. Notably, the most favorable outcomes are observed when transferring
constraints between different agents performing the same task (e.g. CarGoal to
PointGoal). However, when constraints are transferred between distinct tasks
for the same agent, there is a noticeable decline in performance (e.g.
PointPush to PointGoal). Analyzing the learned formula from expert
trajectories performing the Push task, we find that more restrictive
constraints are learned compared to when expert trajectories optimize the Goal
task. This discrepancy is reflected in the results, where employing
constraints learned in the Goal domain within the Push domain leads to a
higher frequency of constraint violations, suggesting that the constraints are
excessively lenient. Conversely, employing constraints acquired in the Push
domain within the Goal domain results in lower costs but also reduced rewards,
indicating that the constraints are overly restrictive.
For each environment, we determine the ideal tree depth for achieving optimal
results. It’s important to note that as the tree depth increases, the learned
formula becomes more restrictive. A more restrictive formula used as a cost
function tends to produce trajectories that closely resemble expert
trajectories, often leading to lower overall costs. However, it’s essential to
consider that in such scenarios, there is a risk of overfitting the learned
tree to the training data. This can result in a situation where the model does
not allow for regions within the feature space which are not observed in the
expert trajectories but not necessarily invalid. This could limit the agent’s
exploration and generalization capabilities and consequently may lead to lower
rewards. This reasoning is exemplified in Figure 4 where we depict the
acquired reward and cost by agents trained on cost functions originating from
trees with various depths.

Figure 4: Reward (left) and cost (right) during training in the CarPush
environment for constraints extracted from trees with various depths.

### Realistic Highway Environment

(a) Simple (b) Full

Figure 5: Reward (top), collision rate (middle) and off road ratio (bottom)
during training in a realistic highway environment.

Traffic is a real-world environment where agent behavior is heavily governed
by both explicit and implicit rules. We extract expert demonstrations from the
highD dataset (Krajewski et al. 2018), a comprehensive repository of annotated
vehicle trajectories recorded on German highways. To facilitate the training
of Reinforcement Learning (RL) agents within this environment, we transform
this dataset into multiple scenarios adhering to the CommonRoad framework
(Althoff, Koschi, and Manzinger 2017). Subsequently, we employ the CommonRoad-
RL framework for agent training (Wang, Krasowski, and Althoff 2021). All the
scenarios derived from the highD dataset encompass either two or three lanes
in both directions. The feature space is characterized by the velocity along
the x- and y-axes, the relative position of agents leading and following the
ego-agent in the left, same and right lane, and the proximity to the left and
right road edges
$\\{v_{x},v_{y},p_{\textrm{rel}_{0}},p_{\textrm{rel}_{1}},p_{\textrm{rel}_{2}},p_{\textrm{rel}_{3}},p_{\textrm{rel}_{4}},p_{\textrm{rel}_{5}},d_{\textrm{left}},d_{\textrm{right}}\\}$.
The output of the distance sensor, which gauges the distance to surrounding
vehicles, spans a range from 0 to 500. Our experiments entail two distinct
settings: one where an agent is trained on a restricted set of 52 scenarios
from the highD dataset (i.e. highD simple) and another where an agent is
trained on all 3000 scenarios (i.e. highD full). Due to the absence of ground
truth constraints in this environment, it is not feasible to report the
frequency of constraint violations. Instead, we present two key performance
metrics: the off-road ratio (indicating the percentage of trajectories in
which the agent deviates from the road) and the collision ratio (reflecting
the percentage of trajectories in which the agent collides with another
vehicle). For the purpose of comparison, our approach is evaluated against two
other agent types: a standard RL agent without any provided constraints (i.e.
RL) and an RL agent equipped with an engineered reward function, as proposed
by Wang, Krasowski, and Althoff (2021), which issues negative rewards when the
agent deviates off-road or collides with another vehicle (i.e. RL w. optimized
reward). The results displayed in Figure 5 clearly demonstrate that our method
excels in preventing agents from driving off-road and reducing the incidence
of collisions with other vehicles. These outcomes strongly suggest that our
rule-learned agents are not only safer but also achieve higher rewards
compared to both the conventional RL agents and those equipped with the hand-
engineered optimized reward function.

## Related Work

The constraint learning problem is by definition ill-posed as many set of
constraints can describe the same set of expert trajectories. In order to
prevent the acquisition of overly cautious constraints, many approaches rely
on the principle of maximum entropy to derive the most concise set of
constraints that align with expert demonstrations (Ziebart et al. 2008).
Scobee and Sastry (2020) introduced an approach focused on learning a set of
constraints that maximize the likelihood of expert demonstrations. This
method, when combined with inductive logic programming, allows for the
acquisition of a logical formula representing the learned constraints (Baert,
Leroux, and Simoens 2023). By approximating the maximum likelihood objective,
this approach can be effectively applied to environments with continuous
state-action spaces (Malik et al. 2021; Gaurav et al. 2023; Liu et al. 2023).
However, it’s important to emphasize that these methods are primarily well-
suited for environments with deterministic dynamics. For environments with
stochastic dynamics, constraint learning is made possible through methods
based on the principle of maximum causal entropy (Ziebart, Bagnell, and Dey
2010), as demonstrated by McPherson, Stocking, and Sastry (2021) for discrete
environments and by Baert et al. (2023) for continuous environments. Unlike
our methodology, the aforementioned methods all necessitate knowledge of the
agent’s goal and the availability of a simulator of the environment for
learning the constraints. Of particular relevance to our research is the work
of Lindner et al. (2023), who also define constraints as a convex set in the
environment’s feature vector space. Nevertheless, their definition of the safe
set involves the convex hull of the feature expectations derived from the
demonstrations. In contrast, our approach models the safe set using a decision
tree, offering the distinct advantage of interpretability for both the safe
set and the constraints extracted. Additionally, we hypothesize that our
method is less prone to overfitting, as it employs only the most informative
feature dimensions for defining the safe set. Furthermore, it is important to
note that they learn a linear cost function in feature space. To allow their
method to work in environments with non-linear constraints, they adopt a one-
hot encoding of the state-action space as features. The latter is only
possible when the state-action space is relatively small and finite. Because
of this the method proposed by Lindner et al. (2023) is unsuitable for
learning constraints in the environments we evaluated. Another benefit of our
method is that the OC-tree naturally provides robustness when dealing with a
limited number of negative examples in the training data, a common scenario
when learning from human demonstrations.

## Conclusion

We have introduced a novel approach to acquire constraints modeled by a logic
formula in disjunctive normal form from expert demonstrations. These acquired
rules can be employed as an easily interpretable cost function in the context
of constrained reinforcement learning. We evaluated our method on multiple
synthetic environments and realistic autonomous driving scenarios. In both,
our method improves safety and performance of the learning agents. In
addition, we provided a post-training pruning mechanism to simplify to learned
formula. Nevertheless, there exists substantial potential for enhancing the
transferability of constraints across various domains. This improvement will
alleviate the necessity for expert trajectories to be available in every
target domain. In future work, we plan to investigate the relation between the
tree depth and the transferability of the learned constraints. We would also
like to investigate whether more effective constraints can be learned when a
simulator of the environment is provided. At last, we want to extend the
proposed method for learning constraints expressed in temporal logic. This
extension brings the significant benefit of enhanced expressiveness, enabling
us to effectively describe more intricate and complex constraints.

##  Appendix A Examples of Learned Rules

We present some learned formulas describing the constraints in an environment.
The presented formulas are already pruned using a violation-evaluation
threshold of $0.001$.

Rules extracted from the expert trajectories in the PointGoal environment
after pruning. $\displaystyle\varphi=a_{x}<-5.42\vee v_{y}<-1.05\vee
v_{x}>1.46\vee d_{l0}>0.88\vee d_{l3}>1.0\vee d_{l5}>0.99$ $\displaystyle\vee
d_{l0}>=0.0\wedge d_{l0}<=0.22\wedge d_{l1}>0.81$ $\displaystyle\vee
d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge d_{l7}>0.87$ $\displaystyle\vee
d_{l0}>=0.0\wedge d_{l0}<=0.22\wedge d_{l1}>=0.0\wedge d_{l1}<=0.27\wedge
d_{l15}>0.66$ $\displaystyle\vee d_{l0}>=0.0\wedge d_{l0}<=0.22\wedge
d_{l1}>=0.27\wedge d_{l1}<=0.81\wedge d_{l15}>0.81$ $\displaystyle\vee
d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge d_{l7}>=0.0\wedge d_{l7}<=0.27\wedge
d_{l6}>0.85$ $\displaystyle\vee d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge
d_{l7}>=0.27\wedge d_{l7}<=0.87\wedge d_{l14}>0.94$ $\displaystyle\vee
d_{l0}>=0.0\wedge d_{l0}<=0.22\wedge d_{l1}>=0.0\wedge d_{l1}<=0.27\wedge
d_{l15}>=0.0\wedge d_{l15}<=0.66\wedge d_{l2}>0.83$ $\displaystyle\vee
d_{l0}>=0.0\wedge d_{l0}<=0.22\wedge d_{l1}>=0.27\wedge d_{l1}<=0.81\wedge
d_{l15}>=0.0\wedge d_{l15}<=0.81\wedge d_{l14}>0.94$ $\displaystyle\vee
d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge d_{l7}>=0.0\wedge d_{l7}<=0.27\wedge
d_{l6}>=0.0\wedge d_{l6}<=0.31\wedge d_{l8}>0.78$ $\displaystyle\vee
d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge d_{l7}>=0.0\wedge d_{l7}<=0.27\wedge
d_{l6}>=0.31\wedge d_{l6}<=0.85\wedge d_{l8}>0.89$ $\displaystyle\vee
d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge d_{l7}>=0.27\wedge d_{l7}<=0.87\wedge
d_{l14}>=0.0\wedge d_{l14}<=0.28\wedge d_{l13}>0.94$ $\displaystyle\vee
d_{l0}>=0.22\wedge d_{l0}<=0.88\wedge d_{l7}>=0.27\wedge d_{l7}<=0.87\wedge
d_{l14}>=0.28\wedge d_{l14}<=0.94\wedge d_{l4}>0.94$

PointGoal: The first rules define a maximum acceleration and velocity. The
following rules define a safe boundary between the agent and hazards in
different directions around the agent.

Rules extracted from the human trajectories in the highD dataset after
pruning. $\displaystyle\varphi=v_{y}<-0.95\vee v_{y}>0.96\vee v_{x}<20.67\vee
v_{x}>47.47\vee p_{\textrm{rel}_{1}}<18.83\vee p_{\textrm{rel}_{4}}<18.83\vee
d_{\textrm{left edge}}<4.42\vee d_{\textrm{right edge}}<2.02$
$\displaystyle\vee v_{y}>=-0.95\wedge v_{y}<=0.96\wedge d_{\textrm{right
edge}}>=4.65\wedge d_{\textrm{right edge}}<=7.38\wedge
p_{\textrm{rel}_{1}}>=18.83\wedge p_{\textrm{rel}_{1}}<=328.76\wedge
p_{\textrm{rel}_{0}}<236.11$ $\displaystyle\vee v_{y}>=-0.95\wedge
v_{y}<=0.96\wedge d_{\textrm{right edge}}>=4.65\wedge d_{\textrm{right
edge}}<=7.38\wedge p_{\textrm{rel}_{1}}>=328.76\wedge
p_{\textrm{rel}_{1}}<=500.0\wedge p_{\textrm{rel}_{0}}<155.9$
$\displaystyle\vee v_{y}>=-0.95\wedge v_{y}<=0.96\wedge d_{\textrm{right
edge}}>=4.65\wedge d_{\textrm{right edge}}<=7.38\wedge
p_{\textrm{rel}_{1}}>=18.83\wedge p_{\textrm{rel}_{1}}<=328.76\wedge
p_{\textrm{rel}_{0}}>=236.11$ $\displaystyle\qquad\wedge
p_{\textrm{rel}_{0}}<=500.0\wedge p_{\textrm{rel}_{3}}<235.66$
$\displaystyle\vee v_{y}>=-0.95\wedge v_{y}<=0.96\wedge d_{\textrm{right
edge}}>=4.65\wedge d_{\textrm{right edge}}<=7.38\wedge
p_{\textrm{rel}_{1}}>=328.76\wedge p_{\textrm{rel}_{1}}<=500.0\wedge
p_{\textrm{rel}_{0}}>=155.9$ $\displaystyle\qquad\wedge
p_{\textrm{rel}_{0}}<=500.0\wedge p_{\textrm{rel}_{3}}<136.43$
$\displaystyle\vee v_{y}>=-0.95\wedge v_{y}<=0.96\wedge d_{\textrm{right
edge}}>=2.02\wedge d_{\textrm{right edge}}<=4.65\wedge
p_{\textrm{rel}_{0}}>=0.04\wedge p_{\textrm{rel}_{0}}<=325.83$
$\displaystyle\qquad\wedge p_{\textrm{rel}_{3}}>=0.03\wedge
p_{\textrm{rel}_{3}}<=332.09\wedge p_{\textrm{rel}_{4}}<26.81$
$\displaystyle\vee v_{y}>=-0.95\wedge v_{y}<=0.96\wedge d_{\textrm{right
edge}}>=2.02\wedge d_{\textrm{right edge}}<=4.65\wedge
p_{\textrm{rel}_{0}}>=325.83\wedge p_{\textrm{rel}_{0}}<=500.0\wedge
p_{\textrm{rel}_{1}}>=33.84$ $\displaystyle\qquad\wedge
p_{\textrm{rel}_{1}}<=219.6\wedge p_{\textrm{rel}_{4}}<29.48$
$\displaystyle\vee v_{y}>=-0.95\wedge v_{y}<=0.96\wedge d_{\textrm{right
edge}}>=2.02\wedge d_{\textrm{right edge}}<=4.65\wedge
p_{\textrm{rel}_{0}}>=325.83\wedge p_{\textrm{rel}_{0}}<=500.0\wedge
p_{\textrm{rel}_{1}}>=219.6$ $\displaystyle\qquad\wedge
p_{\textrm{rel}_{1}}<=500.0\wedge p_{\textrm{rel}_{3}}>344.91$

highD: The first rules define a minimum and maximum speed limit along the x-
and y-axis and a limit on the distance between the car in the same lane in
front of the agent ($p_{\textrm{rel}_{4}}$) and behind the agent
($p_{\textrm{rel}_{1}}$). The next rules define a minimum distance to the road
edges. The following rules distinguish the agent being on the left or right
lane based on the distance to the right road edge. The last rule states the
distance to the vehicle on the left lane in front of the agent should be
smaller than 344.91 if the distance to the vehicle on the left lane behind the
agent is bigger than 325.83. This will force the agent to move one lane to the
left. It is debatable whether this last rule corresponds to desired behavior
as the agent will have a preference to drive on the left or middle lane.

##  Appendix B Acknowledgement

This research was partially funded by the Flemish Government (Flanders AI
Research Program).

## References

  * Achiam et al. (2017) Achiam, J.; Held, D.; Tamar, A.; and Abbeel, P. 2017.  Constrained policy optimization.  In _International conference on machine learning_ , 22–31. PMLR.
  * Althoff, Koschi, and Manzinger (2017) Althoff, M.; Koschi, M.; and Manzinger, S. 2017.  CommonRoad: Composable benchmarks for motion planning on roads.  In _2017 IEEE Intelligent Vehicles Symposium (IV)_ , 719–726. IEEE.
  * Altman (1999) Altman, E. 1999.  _Constrained Markov decision processes: stochastic modeling_.  Routledge.
  * Baert, Leroux, and Simoens (2023) Baert, M.; Leroux, S.; and Simoens, P. 2023.  Inverse reinforcement learning through logic constraint inference.  _Machine Learning_ , 1–26.
  * Baert et al. (2023) Baert, M.; Mazzaglia, P.; Leroux, S.; and Simoens, P. 2023.  Maximum Causal Entropy Inverse Constrained Reinforcement Learning.  _arXiv preprint arXiv:2305.02857_.
  * Gaurav et al. (2023) Gaurav, A.; Rezaee, K.; Liu, G.; and Poupart, P. 2023.  Learning Soft Constraints From Constrained Expert Demonstrations.  In _The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023_. OpenReview.net.
  * Itani, Lecron, and Fortemps (2020) Itani, S.; Lecron, F.; and Fortemps, P. 2020.  A one-class classification decision tree based on kernel density estimation.  _Applied soft computing_ , 91: 106250.
  * Ji et al. (2023) Ji, J.; Zhou, J.; Zhang, B.; Dai, J.; Pan, X.; Sun, R.; Huang, W.; Geng, Y.; Liu, M.; and Yang, Y. 2023.  OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research.  _arXiv preprint arXiv:2305.09304_.
  * Krajewski et al. (2018) Krajewski, R.; Bock, J.; Kloeker, L.; and Eckstein, L. 2018.  The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems.  In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_ , 2118–2125.
  * Krakovna et al. (2020) Krakovna, V.; Uesato, J.; Mikulik, V.; Rahtz, M.; Everitt, T.; Kumar, R.; Kenton, Z.; Leike, J.; and Legg, S. 2020.  Specification gaming: the flip side of AI ingenuity.
  * Lindner et al. (2023) Lindner, D.; Chen, X.; Tschiatschek, S.; Hofmann, K.; and Krause, A. 2023.  Learning Safety Constraints from Demonstrations with Unknown Rewards.  _arXiv preprint arXiv:2305.16147_.
  * Liu et al. (2023) Liu, G.; Luo, Y.; Gaurav, A.; Rezaee, K.; and Poupart, P. 2023.  Benchmarking Constraint Inference in Inverse Reinforcement Learning.  In _The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023_. OpenReview.net.
  * Malik et al. (2021) Malik, S.; Anwar, U.; Aghasi, A.; and Ahmed, A. 2021.  Inverse constrained reinforcement learning.  In _International conference on machine learning_ , 7390–7399. PMLR.
  * McPherson, Stocking, and Sastry (2021) McPherson, D. L.; Stocking, K. C.; and Sastry, S. S. 2021.  Maximum likelihood constraint inference from stochastic demonstrations.  In _2021 IEEE Conference on Control Technology and Applications (CCTA)_ , 1208–1213. IEEE.
  * Ray, Achiam, and Amodei (2019) Ray, A.; Achiam, J.; and Amodei, D. 2019.  Benchmarking safe exploration in deep reinforcement learning.  _arXiv preprint arXiv:1910.01708_ , 7(1): 2.
  * Russell (2019) Russell, S. 2019.  _Human compatible: Artificial intelligence and the problem of control_.  Penguin.
  * Schulman et al. (2017) Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017.  Proximal Policy Optimization Algorithms.  _CoRR_ , abs/1707.06347.
  * Scobee and Sastry (2020) Scobee, D. R. R.; and Sastry, S. S. 2020.  Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Wang, Krasowski, and Althoff (2021) Wang, X.; Krasowski, H.; and Althoff, M. 2021.  CommonRoad-RL: A Configurable Reinforcement Learning Environment for Motion Planning of Autonomous Vehicles.  In _IEEE International Conference on Intelligent Transportation Systems (ITSC)_.
  * Yang et al. (2020) Yang, T.; Rosca, J.; Narasimhan, K.; and Ramadge, P. J. 2020.  Projection-Based Constrained Policy Optimization.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Zhang, Vuong, and Ross (2020) Zhang, Y.; Vuong, Q.; and Ross, K. 2020.  First order constrained optimization in policy space.  _Advances in Neural Information Processing Systems_ , 33: 15338–15349.
  * Ziebart, Bagnell, and Dey (2010) Ziebart, B. D.; Bagnell, J. A.; and Dey, A. K. 2010.  Modeling Interaction via the Principle of Maximum Causal Entropy.  In Fürnkranz, J.; and Joachims, T., eds., _Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel_ , 1255–1262. Omnipress.
  * Ziebart et al. (2008) Ziebart, B. D.; Maas, A. L.; Bagnell, J. A.; Dey, A. K.; et al. 2008.  Maximum entropy inverse reinforcement learning.  In _Aaai_ , volume 8, 1433–1438. Chicago, IL, USA.