# An explicit Plancherel formula for line bundles over the one-sheeted
hyperboloid

Frederik Bang–Jensen & Jonathan Ditlevsen

###### Abstract.

In this paper we consider $G=\operatorname{SL}(2,\mathbb{R})$ and $H$ the
subgroup of diagonal matrices. Then $X=G/H$ is a unimodular homogeneous space
which can be identified with the one-sheeted hyperboloid. For each unitary
character $\chi$ of $H$ we decompose the induced representations
$\operatorname{Ind}_{H}^{G}(\chi)$ into irreducible unitary representations,
known as a Plancherel formula. This is done by studying explicit intertwining
operators between $\operatorname{Ind}_{H}^{G}(\chi)$ and principal series
representations of $G$. These operators depends holomorphically on the
induction parameters.

## Introduction

The Plancherel formula for a unimodular homogeneous space $X=G/H$ of a Lie
group $G$ describes the decomposition of the left-regular representation of
$G$ on $L^{2}(X)$ into irreducible unitary representations. More generally,
one can ask for the decomposition of $L^{2}(G\times_{H}V_{\chi})$, the
$L^{2}$-sections of a homogeneous vector bundle associated with a unitary
representation $(\chi,V_{\chi})$ of $H$. In representation theoretic language,
this corresponds to the induced representation
$\operatorname{Ind}_{H}^{G}(\chi)$ of $G$, and for the trivial representation
$\chi={\bf 1}$ we recover $L^{2}(G/H)$.

By abstract theory, the unitary representation
$\operatorname{Ind}_{H}^{G}(\chi)$ decomposes into a direct integral of
irreducible unitary representations of $G$, i.e. there exists a measure $\mu$
on the unitary dual $\widehat{G}$ of $G$ and a multiplicity function
$m:\widehat{G}\to\mathbb{N}\cup\\{\infty\\}$ such that

$\operatorname{Ind}_{H}^{G}(\chi)\simeq\int^{\oplus}_{\widehat{G}}m(\pi)\cdot\pi\,d\mu(\pi).$

An _abstract_ Plancherel formula describes the support of the Plancherel
measure $\mu$ as well as the multiplicity function $m$. Such abstract
Plancherel formulas have been established for certain classes of homogeneous
spaces such as semisimple symmetric spaces (see e.g. [16]).

However, for some applications an abstract Plancherel formula is not
sufficient, and a more _explicit_ version is needed (see e.g. [5, 17]). By
this, we mean an explicit formula for the measure $\mu$ as well as explicit
linearly independent intertwining operators
$A_{\pi,j}:\operatorname{Ind}_{H}^{G}(\chi)^{\infty}\to\pi^{\infty}$,
$j=1,\ldots,m(\pi)$, for $\mu$-almost every $\pi\in\widehat{G}$ such that

$\|f\|_{\operatorname{Ind}_{H}^{G}(\chi)}^{2}=\int_{\widehat{G}}\sum_{j=1}^{m(\pi)}\|A_{\pi,j}f\|_{\pi}^{2}\,d\mu(\pi)\qquad(f\in\operatorname{Ind}_{H}^{G}(\chi)^{\infty}).$

Such an explicit Plancherel formula is for instance known for Riemannian
symmetric spaces $X=G/K$, where the Plancherel measure $\mu$ is explicitly
given in terms of Harish-Chandra’s $c$-function and the intertwining operators
$A_{\pi,j}$ can be described in terms of spherical functions (see e.g. [9],
and also [14] for the case of line bundles over Hermitian symmetric spaces).
This explicit Plancherel formula has recently been applied in the context of
branching problems for unitary representations where the explicit Plancherel
measure and in particular its singularities play a crucial role (see e.g. [5,
17]). In order to apply the same strategy to other branching problems,
explicit Plancherel formulas are needed for more general homogeneous spaces.

In this paper, we determine the explicit Plancherel formula for line bundles
over the one-sheeted hyperboloid $X=G/H$, where
$G=\operatorname{SL}(2,\mathbb{R})$ and $H$ the subgroup of diagonal matrices.
This specific Plancherel formula has direct applications to branching problems
for the pairs
$(\operatorname{SL}(2,\mathbb{R})\times\operatorname{SL}(2,\mathbb{R}),\operatorname{diag}(\operatorname{SL}(2,\mathbb{R}))$
and $(\operatorname{GL}(3,\mathbb{R}),\operatorname{GL}(2,\mathbb{R}))$. The
homogeneous Hermitian line bundles over $X$ are parameterized by
$\varepsilon\in\mathbb{Z}/2\mathbb{Z}$ and $\lambda\in i\mathbb{R}$, the
corresponding unitary character of $H$ being

$\chi_{\varepsilon,\lambda}\begin{pmatrix}t&0\\\
0&t^{-1}\end{pmatrix}=\operatorname{sgn}(t)^{\varepsilon}|t|^{\lambda}\qquad(t\in\mathbb{R}^{\times}).$

We find intertwining operators
$A_{\lambda,\mu}^{\xi}:\operatorname{Ind}_{H}^{G}(\chi_{\lambda,\varepsilon})\to\operatorname{Ind}_{P}^{G}(\varepsilon\otimes
e^{\mu}\otimes 1),$ $\xi=0,1$ between the line bundles over $X$ and the
principal series representation (see Proposition 5.1).

######  Theorem 0.1 (See Theorem 7.6).

For $f\in\operatorname{Ind}_{H}^{G}(\chi_{\lambda,\varepsilon})$, $\lambda\in
i\mathbb{R}$ and $\varepsilon\in\\{0,1\\}$ we have

$\|f\|^{2}=\int_{i\mathbb{R}}\sum_{\xi=0}^{1}\|\mathbf{A}_{\lambda,\mu}^{\xi}f\|^{2}\frac{d\mu}{|a(\mu,\varepsilon)|^{2}}+\sum_{\mu\in
1-\varepsilon-2\mathbb{N}}c(\mu,\varepsilon)\|\mathbb{A}_{\lambda,\mu}^{\varepsilon}f\|^{2},$
(1)

where $\mathbf{A}_{\lambda,\mu}$ and $\mathbb{A}_{\lambda,\mu}$ are some
combinations of $A_{\lambda,\mu}^{0}$ and $A_{\lambda,\mu}^{1}$.

The proof of (1) consists of two steps. First, we prove (1) in the case
$\lambda=0$ separately for each $K$-isotypic component. On a fixed
$K$-isotypic component, the intertwining operators $A_{\lambda,\mu}^{\xi}$ are
essentially Fourier–Jacobi transforms, and the Plancherel formula follows from
the spectral decomposition of the corresponding ordinary second order
differential operator by Sturm–Liouville theory. The main difficulty is that
the continuous spectrum occurs with multiplicity two, while the discrete part
occurs with multiplicity-one, and it is non-trivial to find the right linear
combination of $A_{\mu}^{0}$ and $A_{\mu}^{1}$ that corresponds to a direct
summand. In fact, this linear combination is very different for the cases
$\varepsilon=0$ and $\varepsilon=1$. In the second step, we show that, as a
representation of $G$, $L^{2}(G/H,\mathcal{L}_{\varepsilon,\lambda})$ is
independent of $\lambda$, and by finding an explicit unitary isomorphism
$L^{2}(G/H,\mathcal{L}_{\varepsilon,\lambda})\to
L^{2}(G/H,\mathcal{L}_{\varepsilon,0})$ we deduce the claimed formula.

We remark that for $\varepsilon=0$ and general $\lambda\in i\mathbb{R}$ the
Plancherel formula was recently obtained by Zhu [18]. Moreover, for
$\varepsilon=0$ and $\lambda=0$ our Plancherel formula can be viewed as a
special case of the one for pseudo-Riemannian real hyperbolic spaces
$\operatorname{O}(p,q)/\operatorname{O}(p,q-1)$ with $p=1$ and $q=2$ which was
obtained by Faraut [2], Rossmann [13] and Strichartz [15]. Note also that the
corresponding _abstract_ Plancherel formula, i.e. the description of the
representations occurring in the direct integral decomposition, also follows
from the general theory (see e.g. [16]).

### Acknowledgements

We would like to thank our supervisor Jan Frahm for his help and input on the
topics of this paper.

### Notation

$\mathbb{N}=\\{1,2,3\dots\\}$, $\mathbb{N}_{0}=\mathbb{N}\cup\\{0\\}$. For
$A\subseteq\mathbb{R}$ and $b,c\in\mathbb{R}$ we denote by
$b+cA=\\{b+ca\,|\,a\in A\\}$. The Pochhammer symbol is
$(x)_{n}=x(x+1)\cdots(x+n-1)$. We denote Lie groups by Roman capitals and
their corresponding Lie algebras by the corresponding Fraktur lower cases. For
$m\in\mathbb{Z}$ we let $[m]_{2}\in\\{0,1\\}$ be the remainder of $m$ after
division by 2.

##  1\. The principal series of $\operatorname{SL}(2,\mathbb{R})$

In this section we recall some results about the representation theory of
$\operatorname{SL}(2,\mathbb{R})$ following [1]. Let
$G=\operatorname{SL}(2,\mathbb{R})$ and consider the following subgroups

$M=\\{\pm I\\},\quad A=\left\\{\begin{pmatrix}t&0\\\
0&t^{-1}\end{pmatrix}\,:\,t\in\mathbb{R}_{>0}\right\\},\quad
N=\left\\{\begin{pmatrix}1&x\\\ 0&1\end{pmatrix}\,:\,x\in\mathbb{R}\right\\},$

then $P=MAN$ is a minimal parabolic subgroup of $G$. Identify
$\widehat{M}\cong\mathbb{Z}/2\mathbb{Z}$ by mapping
$\varepsilon\in\mathbb{Z}/2\mathbb{Z}$ to the character

$M\to\\{\pm 1\\},\,\quad\begin{pmatrix}\pm 1&0\\\ 0&\pm
1\end{pmatrix}\mapsto(\pm 1)^{\varepsilon}.$

Further, we identify $\mathfrak{a}_{\mathbb{C}}^{*}\cong\mathbb{C}$ by mapping
$\lambda\mapsto\lambda\big{(}\operatorname{diag}(1,-1)\big{)}.$ We can then
observe that any character of $H:=MA$ is of the form
$\chi_{\varepsilon,\lambda}=\varepsilon\otimes e^{\lambda}$ where

$\chi_{\varepsilon,\lambda}\begin{pmatrix}t&0\\\
0&t^{-1}\end{pmatrix}=|t|^{\lambda}_{\varepsilon}:=\operatorname{sgn}(t)^{\varepsilon}|t|^{\lambda},\quad\quad(t\in\mathbb{R}^{\times}).$

As the commutator subgroup of $P$ is $N$ the characters of $P$ is of the form
$\varepsilon\otimes e^{\mu}\otimes 1$ and these characters are unitary exactly
when $\lambda\in i\mathbb{R}$.

Let $\varepsilon\in\mathbb{Z}/2\mathbb{Z}$ and $\mu\in\mathbb{C}$. For any
character $\varepsilon\otimes e^{\mu}\otimes 1$ of $P$ define the principal
series representation $\pi_{\varepsilon,\mu}$induced by it to be the left
regular representation of $G$ on

$\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes 1)=\left\\{f\in
C^{\infty}(G)\,\mid f(gman)=|t|_{\varepsilon}^{-\mu-1}f(g),\,m\in M,\,a\in
A,\,n\in N\right\\},$

where $ma=(\begin{smallmatrix}t&0\\\ 0&t^{-1}\end{smallmatrix})\in MA$. We
introduce the notation

$k_{\theta}=\begin{pmatrix}\cos\theta&\sin\theta\\\
-\sin\theta&\cos\theta\end{pmatrix},$

and $\zeta_{m}(k_{\theta})=e^{im\theta}$. According to the theory of Fourier
series we have the $K$-type decomposition

$\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes
1)\cong\mathop{\widehat{{\bigoplus}}}_{m\in
2\mathbb{Z}+\varepsilon}\mathbb{C}\zeta_{m}.$

We let $\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes 1)_{m}$
denote the set of functions contained in the $K$-type given by
$m\in\mathbb{Z}$, that is $\operatorname{Ind}_{P}^{G}(\varepsilon\otimes
e^{\mu}\otimes 1)_{m}=\mathbb{C}\zeta_{m}$.

A basis of $\mathfrak{g}$ is given by

$\displaystyle H=\begin{pmatrix}1&0\\\ 0&-1\end{pmatrix},\quad
E=\begin{pmatrix}0&1\\\ 0&0\end{pmatrix},\quad F=\begin{pmatrix}0&0\\\
1&0\end{pmatrix}.$

Consider the Casimir operator

$\Delta_{\mu}=d\pi(H)^{2}+d\pi(E+F)^{2}-d\pi(E-F)^{2},$

where $\pi=\pi_{\varepsilon,\mu}$.

######  Proposition 1.1 (See e.g. [1, Prop. 10.7]).

For $f\in\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes 1)$ we
have

$\displaystyle\Delta_{\mu}f=(\mu^{2}-1)f.$

######  Proposition 1.2 (See [1, Prop. 10.8]).

The representation $\operatorname{Ind}_{P}^{G}(\varepsilon\otimes
e^{\mu}\otimes 1)$ is irreducible except when $\mu\in
1-\varepsilon-2\mathbb{Z}$ . If $\mu\in 1-\varepsilon-2\mathbb{N}$ then
$\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes 1)$ decomposes
as $V_{0}\oplus V_{1}\oplus V_{2}$ where $V_{0}$ is an irreducible
representation containing exactly the $K$-types with $|m|\leq-\mu$. The
quotient $\pi_{\varepsilon,\mu}^{\text{ds}}$ is a direct sum of two infinite
dimensional representations $\pi_{\varepsilon,\mu}^{\textup{hol}}$ and
$\pi_{\varepsilon,\mu}^{\textup{ahol}}$.

Let $w_{0}=(\begin{smallmatrix}0&1\\\ -1&0\end{smallmatrix}),$ a
representative of the longest Weyl group element of $G$. Recall the definition
of the Knapp–Stein intertwining operator

$\displaystyle
T_{\mu}^{\varepsilon}:\operatorname{Ind}_{P}^{G}(\varepsilon\otimes
e^{\mu}\otimes 1)\rightarrow\operatorname{Ind}_{P}^{G}(\varepsilon\otimes
e^{-\mu}\otimes 1),\quad
T_{\mu}^{\varepsilon}f(g)=\frac{1}{\Gamma(\frac{\mu+\varepsilon}{2})}\int_{\overline{N}}f(gw_{0}\overline{n})d\overline{n},$

for $\operatorname{Re}{(\mu)}>0$. The normalization is chosen such that
$T_{\mu}^{\varepsilon}$ extends holomorphically to $\mu\in\mathbb{C}$.

######  Proposition 1.3.

For $f\in\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes 1)_{m}$
we have

$T^{\varepsilon}_{\mu}f=b_{m}^{\varepsilon}(\mu)f,$

where

$b_{m}^{\varepsilon}(\mu)=\sqrt{\pi}i^{[\varepsilon]_{2}}(-1)^{\frac{m+|m|}{2}-[\varepsilon]_{2}}\frac{\big{(}\frac{1+\varepsilon-\mu}{2}\big{)}_{\frac{|m|-\varepsilon}{2}}}{\Gamma\big{(}\frac{\mu+1+|m|}{2}\big{)}}.$

For $\varepsilon=0$ and $\mu\in 1-2\mathbb{N}$ we have $b_{m}^{0}(\mu)\geq 0$
for all $m\in 2\mathbb{Z}$. Whereas for $\varepsilon=1$, $m$ odd and
$\mu\in-2\mathbb{N}$ we have $-ib_{m}^{1}(\mu)\geq 0$ for $m>0$ and
$ib_{m}^{1}(\mu)\geq 0$ for $m<0$.

###### Proof.

As $T^{\varepsilon}_{\mu}$ maps $K$-types to $K$-types we have
$T_{\mu}^{\varepsilon}f=T_{\mu}^{\varepsilon}f(e)f$. Now decompose

$w_{0}\overline{n}_{x}=kan=\frac{1}{\sqrt{1+x^{2}}}\begin{pmatrix}x&1\\\
-1&x\end{pmatrix}\begin{pmatrix}\sqrt{x^{2}+1}&0\\\
0&\frac{1}{\sqrt{1+x^{2}}}\end{pmatrix}\begin{pmatrix}1&\frac{x}{x^{2}+1}\\\
0&1\end{pmatrix},$

then applying $f$’s equivariance properties, we arrive at

$\int_{\mathbb{R}}f(w_{0}\overline{n}_{x})\,dx=\int_{\mathbb{R}}(x+i)^{\frac{m-\mu-1}{2}}(x-i)^{\frac{-m-\mu-1}{2}}\,dx=\frac{2^{1-\mu}\pi
i^{m}\Gamma(\mu)}{\Gamma\big{(}\frac{\mu-|m|+1}{2}\big{)}\Gamma\big{(}\frac{\mu+|m|+1}{2}\big{)}},$

where in the last equality we used Lemma A.1. Now dividing by
$\Gamma(\frac{\mu+\varepsilon}{2})$ and shuffling around Gamma-factors we
arrive at the result. ∎

For $\mu\in i\mathbb{R}$ we equip the space
$\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes 1)$ with the
usual $L^{2}$-norm. Using Proposition 1.3 we can for $\varepsilon=0$ and
$\mu\in 1-2\mathbb{N}$ equip $\operatorname{Ind}_{P}^{G}(0\otimes
e^{\mu}\otimes 1)$ with the norm

$\|f\|^{2}=\int_{K}f(k)\overline{T_{\mu}^{0}f(k)}\,dk.$

Similarly for $\varepsilon=1$ and $\mu\in-2\mathbb{N}$ we can equip
$\operatorname{Ind}_{P}^{G}(1\otimes e^{\mu}\otimes 1)$ with the norm

$\|f\|^{2}=\int_{K}f(k)\overline{\hat{T}_{\mu}^{1}f(k)}\,dk$

where

$\hat{T}_{\mu}^{1}f=\begin{cases}iT_{\mu}^{1}f,&\text{for }m>0,\\\
-iT_{\mu}^{1}f,&\text{for }m<0\\\ \end{cases}$

for $f\in\operatorname{Ind}_{P}^{G}(1\otimes e^{\mu}\otimes 1)_{m}$. The
operator $\hat{T}_{\mu}^{1}$ is still an intertwining operator as it vanishes
on $V_{0}$ per Proposition 1.3 and thus we just altered it by a scalar on each
of the summands in Proposition 1.2.

##  2\. The homogeneous space $G/H$

For a unitary character $\chi_{\varepsilon,\lambda}=\varepsilon\otimes
e^{\lambda}$ with $\lambda\in i\mathbb{R}$ the left-regular action
$\tau_{\varepsilon,\lambda}$ of $G$ on the space of $L^{2}$-sections
associated to the line bundle
$G\times_{H}\mathbb{C}_{\varepsilon,\lambda}\rightarrow G/H$, given by

$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})=\left\\{f:G\rightarrow\mathbb{C},\textit{ measurable}\mid
f(gh)=\chi_{\varepsilon,\lambda}(h)^{-1}f(g),\,\int_{G/H}|f(g)|^{2}d(gH)<\infty\right\\},$

defines a unitary representation of $G$. The goal of this paper is to
decompose this space. Furthermore we consider the subspace of compactly
supported smooth functions

$C^{\infty}_{c}\text{-}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})=\left\\{f\in
C^{\infty}(G)\cap\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\mid\text{supp}(f)\subseteq\Omega,\,\Omega H\text{ is compact in
}G/H\right\\}.$

We will denote the smooth vectors in
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})$ by
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})^{\infty}$. We
introduce the notation

$b_{u}=\begin{pmatrix}\cosh u&\sinh u\\\ \sinh u&\cosh
u\end{pmatrix},\quad\overline{n}_{x}=\begin{pmatrix}1&0\\\ x&1\end{pmatrix}.$

Using the decomposition $G=KBA$, where $B=\\{b_{u}\,|\,u\in\mathbb{R}\\},$ we
consider $G/H$ in the global coordinates
$(\theta,u)\in[0,\pi)\times\mathbb{R}$ where $xH=k_{\theta}b_{u}H$ and the
invariant measure is $d(xH)=\cosh(2u)dud\theta$, see e.g. [11]. Now in terms
of these coordinates we have the $K$-type decomposition

$\displaystyle
C^{\infty}_{c}\text{-}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})=\mathop{\widehat{\bigoplus}}_{m\in
2\mathbb{Z}+\varepsilon}\mathbb{C}\zeta_{m}\otimes C_{c}^{\infty}(\mathbb{R})$
(2)

with $\zeta_{m}(k_{\theta})=e^{im\theta}$. We let
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})_{m}$ denote the
set of functions contained in the $K$-type given by $m\in
2\mathbb{Z}+\varepsilon$, that is
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})_{m}=\mathbb{C}\zeta_{m}\otimes C_{c}^{\infty}(\mathbb{R})$.

We denote by $\Delta_{\lambda}$ the Casimir operator for the representation
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})$ defined in a
similar fashion as for the principal series.

######  Proposition 2.1.

Written in the coordinates $(\theta,u)$ the Casimir operator
$\Delta_{\lambda}$ is given by

$\displaystyle\Delta_{\lambda}=\frac{\lambda^{2}}{\cosh^{2}(2u)}+2\lambda\frac{\tanh(2u)}{\cosh(2u)}\partial_{\theta}+2\tanh(2u)\partial_{u}-\frac{1}{\cosh^{2}(2u)}\partial_{\theta}^{2}+\partial_{u}^{2}.$

###### Proof.

This is a standard computation. ∎

Another set of coordinates can be obtained by using the Iwasawa decomposition
$G=KA\overline{N}$ with $(\theta,y)\in[0,\pi)\times\mathbb{R}$ where
$xH=k_{\theta}\overline{n}_{y}H$. The invariant measure is given by
$d(xH)=\frac{1}{2}dyd\theta$ see [10, Chap. 5, §6].

##  3\. Constructing an isomorphism

The goal of this section is to construct the explicit isomorphism in the
following theorem, of which the proof was in large presented to us by Jan
Frahm.

######  Theorem 3.1.

For $\nu,\lambda\in i\mathbb{R}$ the map

$\operatorname{\text{\Large\Ankh}}_{\lambda}^{\nu}:\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\rightarrow\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\nu})$

given by

$\operatorname{\text{\Large\Ankh}}_{\lambda}^{\nu}f(g)=\frac{1}{\sqrt{\pi}2^{\frac{\lambda-\nu}{4}}{}}\frac{\Gamma\left(\frac{2+\nu-\lambda}{4}\right)}{\Gamma\left(\frac{\lambda-\nu}{4}\right)}\int_{\mathbb{R}}|x|^{\frac{\lambda-\nu}{2}-1}f(g\overline{n}_{x})\,dx$

defines a unitary isomorphism intertwining
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})$ and
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\nu})$.

To this extent we consider the minimal parabolic subgroup
$\overline{P}=\overline{N}AM\subset G$ and let

$\operatorname{Ind}_{MA}^{P}(\varepsilon\otimes
e^{\lambda})=\left\\{f:\overline{N}AM\to\mathbb{C}\,|\,f(gma)=\operatorname{sgn}(m)^{\varepsilon}a^{-\lambda-1}f(g),\,\&\,\int_{P/MA}|f(g)|^{2}\,dg<\infty\right\\},$

where $a^{\lambda}:=e^{\lambda(X)}$ for $a=e^{X}$ and
$\lambda\in\mathfrak{a}_{\mathbb{C}}^{*}\cong\mathbb{C}$.

######  Lemma 3.2 (Induction in stages).

$\operatorname{Ind}_{MA}^{G}(\varepsilon\otimes
e^{\lambda})\simeq\operatorname{Ind}_{MA\overline{N}}^{G}(\operatorname{Ind}_{MA}^{MA\overline{N}}(\varepsilon\otimes
e^{\lambda})),$

where the map is given by $f\mapsto F$ where
$F(g)(\overline{p})=f(g\overline{p})$ and thus the inverse is given by
$f(g)=F(g)(1)$.

###### Proof.

See e.g. [6, Chapter VI, section 9] ∎

###### Proof of Theorem 3.1.

We first show the isomorphism claim of Theorem 3.1. By Lemma 3.2 it suffices
to show that $\operatorname{Ind}_{MA}^{\overline{P}}(\varepsilon\otimes
e^{\lambda})\simeq\operatorname{Ind}_{MA}^{\overline{P}}(\varepsilon\otimes
e^{\nu})$. Let
$\text{rest}_{\overline{N}}:\operatorname{Ind}_{MA}^{\overline{P}}(\varepsilon\otimes
e^{\lambda})\to L^{2}(\overline{N})$ be the restriction from $\overline{P}$ to
$\overline{N}$. We let $\Phi$ be the inverse map which is given by $\Phi
F(\overline{n}am)=\operatorname{sgn}(m)^{\varepsilon}a^{-\lambda-1}F(\overline{n})$.
Let $\pi_{\varepsilon,\lambda}$ be the left regular representation on
$\operatorname{Ind}_{MA}^{\overline{P}}(\varepsilon\otimes e^{\lambda})$ and
define
$\tilde{\pi}_{\varepsilon,\lambda}(g)=\text{rest}_{\overline{N}}\circ\pi_{\varepsilon,\lambda}(g)\circ\Phi$.
Then $\overline{P}$ acts on $L^{2}(\overline{N})$ via
$\tilde{\pi}_{\varepsilon,\lambda}$, and the above statement reduces to
showing that
$\tilde{\pi}_{\varepsilon,\lambda}\cong\tilde{\pi}_{\varepsilon,\nu}$ for
$\lambda,\nu\in i\mathbb{R}$.

To construct an isomorphism $H:L^{2}(\mathbb{R})\rightarrow L^{2}(\mathbb{R})$
intertwining $\tilde{\pi}_{\varepsilon,\lambda}$ and
$\tilde{\pi}_{\varepsilon,\nu}$ we note that the action of $P=\overline{N}AM$
on $f\in L^{2}(\overline{N})$ is given by

$\displaystyle\tilde{\pi}_{\varepsilon,\lambda}(\overline{n})f(\overline{n}^{\prime})=f(\overline{n}^{-1}\overline{n}^{\prime})\qquad\qquad\qquad\qquad\qquad\quad\overline{n},\overline{n}^{\prime}\in\overline{N},$
$\displaystyle\tilde{\pi}_{\varepsilon,\lambda}(ma)f(\overline{n})=\operatorname{sgn}(m)^{\varepsilon}a^{\lambda+1}f\big{(}(ma)^{-1}\overline{n}(ma)\big{)},\quad
m\in M,\,a\in A,\,\overline{n}\in\overline{N}.$

Identifying $\overline{N}\simeq\mathbb{R}$, $M\simeq\\{\pm 1\\}$ and
$A\simeq\mathbb{R}_{>0}$, the above becomes

$\displaystyle\tilde{\pi}_{\varepsilon,\lambda}(y)f(x)=f(x-y),\qquad
x,y\in\mathbb{R},$
$\displaystyle\tilde{\pi}_{\varepsilon,\lambda}(t)f(x)=t^{\lambda+1}f(t^{2}x),\quad
x\in\mathbb{R},\ t\in\mathbb{R}_{>0}.$

Since $\overline{N}$ acts by translation any such intertwining operator $H$
must be a translation invariant operator on $L^{2}(\mathbb{R})$, hence there
exists some tempered distribution $u\in\mathcal{S}^{\prime}(\mathbb{R})$ such
that $H$ is given by convolution with $u$, that is $HF(x)=\langle
u,\tau_{x}\check{F}\rangle$, where $\tau_{x}F(y)=F(y-x)$ and
$\check{F}(x)=F(-x)$. Furthermore let $d_{t}f$ denote the dilation of $f$ by
$t\in\mathbb{R}\setminus\\{0\\}$, i.e. $d_{t}f(x)=f(tx)$, then

$\displaystyle
H\circ\tilde{\pi}_{\varepsilon,\lambda}(t)F(x)=\tilde{\pi}_{\varepsilon,\nu}(t)\circ
HF(x).$

Evaluating at $x=0$ then yields $\langle
d_{t^{-2}}u,\check{F}\rangle=t^{\nu-\lambda+2}\langle u,\check{F}\rangle$.
Hence $u$ is a homogeneous distribution of degree $\frac{\lambda-\nu-2}{2}$
and we conclude that

$Hf=f*|x|^{\frac{\lambda-\nu-2}{2}}_{\delta}\quad\text{ for some
}\delta\in\\{0,1\\}.$

Comparing with Lemma B.2 we see that these are both necessary and sufficient
conditions for $H$ to establish an isomorphism between
$\operatorname{Ind}_{MA}^{P}(\varepsilon\otimes e^{\lambda})$ and
$\operatorname{Ind}_{MA}^{P}(\varepsilon\otimes e^{\nu})$. Putting $\delta=0$,
composing with the map from Lemma 3.2 then yields the desired isomorphism. To
see that the normalization indeed makes
$\operatorname{\text{\Large\Ankh}}_{\lambda}^{\nu}$ unitary, it suffices to
note that for $\lambda,\nu\in i\mathbb{R}$ Lemma B.2 gives

$\left|\frac{1}{\sqrt{\pi}2^{\frac{\lambda-\nu}{4}}{}}\frac{\Gamma\left(\frac{2+\nu-\lambda}{4}\right)}{\Gamma\left(\frac{\lambda-\nu}{4}\right)}\mathcal{F}(|x|^{\frac{\lambda-\nu}{2}-1})\right|=1.$

∎

##  4\. Eigenfunctions for the Casimir operator

By Theorem 3.1 a Plancherel formula on
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})$ for some fixed
$\lambda\in i\mathbb{R}$ can be extended to all $\nu\in i\mathbb{R}$ by
compositon with the unitary isomorphism
$\operatorname{\text{\Large\Ankh}}^{\nu}_{\lambda}$ from Theorem 3.1.
Following this we will therefore mostly consider the cases of which
$\lambda=0$, which often simplifies matters considerably.

For $f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})_{m}$ with
$f(k_{\theta}b_{u})=e^{im\theta}\cdot h(u),\ h\in C_{c}^{\infty}(\mathbb{R})$
we have

$\displaystyle\Delta_{0}f=e^{im\theta}\tilde{\Delta}_{m}h(u),$

for some differential operator $\tilde{\Delta}_{m}$.

######  Lemma 4.1.

Let $h\in C_{c}^{\infty}(\mathbb{R})$ and $m\in\mathbb{Z}$. Then we have

$\tilde{\Delta}_{m}\cosh^{\frac{m}{2}}(2u)h\big{(}-\sinh^{2}(2u)\big{)}=\cosh^{\frac{m}{2}}(2u)\square_{m}h\big{(}-\sinh^{2}(2u)\big{)}.$

For $t=-\sinh^{2}(2u)$ the operator $\square_{m}$ is given by

$\displaystyle\square_{m}=m(m+2)+8(-1+\left(3+m\right)t)\frac{d}{dt}-16t\cdot(1-t)\frac{d^{2}}{dt^{2}}.$

###### Proof.

Follows directly from Proposition 2.1. ∎

Recall that a hypergeometric differential equation has the form

$t(1-t)\frac{d^{2}}{dt^{2}}+[c-(a+b+1)t]\frac{d}{dt}-ab=0.$

If $c$ is not a non-positive integer there are two independent solutions
(around $t=0$)

$\ {}_{2}F_{1}(a,b;c;t)\quad\text{and}\quad t^{1-c}\
_{2}F_{1}(1+a-c,1+b-c;2-c;t),$

expressed in terms of the hypergeometric function $\,{}_{2}F_{1}(a,b;c;t)$.

We note that the eigenvalue problem $\square_{m}f=(\mu^{2}-1)f$ is a
hypergeometric differential equation thus giving us two linearly independent
solutions $\varphi^{m}_{\mu}$ and $\psi_{\mu}^{m}$. Using the notation from
appendix C we can express these solutions as

$\varphi^{m}_{\mu}(u)=\phi_{\frac{-i\mu}{2}}^{-\frac{1}{2},\frac{m}{2}}(u),\quad\psi_{\mu}^{m}(u)=i\sinh(u)\cdot\phi_{\frac{-i\mu}{2}}^{\frac{1}{2},\frac{m}{2}}(u),$

where $u\in[0,\infty)$. Note that these functions allow for natural extensions
from $[0,\infty)$ to $\mathbb{R}$. We now restate the results from Appendix C
in terms of $\varphi_{\mu}^{m}$ and $\psi_{\mu}^{m}$.

######  Proposition 4.2.

For $f\in C^{\infty}_{c}([0,\infty))$, let $(J_{j}f)(\mu)$, $j=0,1,$ denote
the Fourier–Jacobi transforms of $f$ given by

$\displaystyle J_{0}f(\mu)$
$\displaystyle=\int_{0}^{\infty}f(t)\varphi_{\mu}^{m}(t)\cosh^{m+1}(t)\,dt,$
$\displaystyle J_{1}f(\mu)$
$\displaystyle=\int_{0}^{\infty}f(t)\psi_{\mu}^{m}(t)\sinh(t)\cosh^{m+1}(t)\,dt.$

Then we have the following inversion formulas

$\displaystyle f(t)$
$\displaystyle=\frac{1}{4\pi^{2}}\int_{i\mathbb{R}}J_{0}f(\mu)\varphi^{m}_{\mu}(t)\frac{d\mu}{|\ell_{0}(\mu)|^{2}}-\frac{1}{2\pi}\sum\limits_{\mu\in
D_{0}}J_{0}f(\mu)\varphi_{\mu}^{m}(t)\underset{\nu=\mu}{\operatorname{Res}}(\ell_{0}(\nu)\ell_{0}(-\nu))^{-1}$
$\displaystyle\sinh(t)f(t)$
$\displaystyle=\frac{-1}{\pi^{2}}\int_{i\mathbb{R}}J_{1}f(\mu)\psi^{m}_{\mu}(t)\frac{d\mu}{|\ell_{1}(\mu)|^{2}}+\frac{2}{\pi}\sum\limits_{\mu\in
D_{1}}J_{1}f(\mu)\psi_{\mu}^{m}(t)\underset{\nu=\mu}{\operatorname{Res}}(\ell_{1}(\nu)\ell_{1}(-\nu))^{-1}.$

with
$D_{j}=\\{\eta\in\mathbb{R}\mid\eta=4k+1+2j-|m|<0,\,k\in\mathbb{N}_{0}\\}$ and

$\displaystyle\ell_{j}(\mu)=\frac{\Gamma(\frac{\mu}{2})}{\Gamma(\frac{\mu+1+2j+|m|}{4})\Gamma(\frac{\mu+1+2j-|m|}{4})}.$

######  Remark 4.3.

As $\varphi_{\mu}^{m}$ and $\psi_{\mu}^{m}$ are given in terms of
hypergeometric functions we get

$\varphi_{\mu}^{m}=\varphi_{-\mu}^{m},\quad\text{and}\quad\psi_{\mu}^{m}=\psi_{-\mu}^{m},$

as$\ {}_{2}F_{1}(a,b;c;t)=\ _{2}F_{1}(b,a;c;t)$. The Euler transformation$\
{}_{2}F_{1}(a,b;c;t)=(1-t)^{c-a-b}\ _{2}F_{1}(c-a,c-b;c;t)$ amounts to

$\displaystyle\varphi_{\mu}^{m}(u)=\cosh^{-m}(u)\varphi_{\mu}^{-m}(u),\quad\text{and}\quad\psi_{\mu}^{m}(u)=\cosh^{-m}(u)\psi_{\mu}^{-m}(u).$

##  5\. Intertwining operators

To obtain an explicit Plancherel formula for representation theoretic
purposes, we require expressions for intertwining operators between the
representation spaces introduced in earlier sections. More explicitly we
consider intertwining operators

$\displaystyle P:\operatorname{Ind}_{P}^{G}(\delta\otimes e^{\mu}\otimes
1)\rightarrow\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})^{\infty},$ $\displaystyle
A:C_{c}^{\infty}\mbox{--}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\rightarrow\operatorname{Ind}_{P}^{G}(\delta\otimes e^{\mu}\otimes
1)$

and their realizations in terms of the coordinates introduced in earlier
sections. Such operators only exist when $\varepsilon=\delta$ as $M$ lies in
the center of $G$.

We fix $\varepsilon\in\mathbb{Z}/2\mathbb{Z}$ and supress it in the notation
for the rest of this section. When $\varepsilon$ appear in formulas we will
consider it as number in $\\{0,1\\}$ where we will use the notation
$[\,\cdot\,]_{2}$ when confusions can occur.

For $\xi\in\mathbb{Z}/2\mathbb{Z}$ and $\lambda\in i\mathbb{R}$ consider the
kernel

$K_{\lambda,\mu}^{\xi}(g)=|g_{11}|^{\frac{\lambda+\mu-1}{2}}_{\xi+\varepsilon}|g_{21}|^{\frac{\mu-\lambda-1}{2}}_{\xi},\quad
g\in G,$

where $g_{ij}$ is the $(i,j)$’th entry in $G$. As $g_{11}$ and $g_{21}$ does
not simultaneously vanish in $G$ this kernel enjoys many of the similar
properties as a Riesz distribution (see Appendix B).
$\Gamma(\frac{\mu+1}{2})^{-1}K_{\lambda,\mu}^{\xi}$ is locally integrable for
$\operatorname{Re}{(\mu)}>-1$ and admits a holomorphic continuation as a
distribution to $\mu\in\mathbb{C}.$

######  Proposition 5.1.

The map given by

$P_{\lambda,\mu}^{\xi}f(g)=\int_{K}K_{\lambda,\mu}^{\xi}(g^{-1}k)f(k)\,dk,$

defines an intertwining operator
$\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu}\otimes
1)\rightarrow\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})^{\infty}$. Similarly the map given by

$A^{\xi}_{\lambda,\mu}f(g)=\int_{G/H}K_{-\lambda,-\mu}^{\xi}(x^{-1}g)f(x)\,d(xH),$

defines an intertwining operator
$C_{c}^{\infty}\mbox{--}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\rightarrow\operatorname{Ind}_{P}^{G}(\varepsilon\otimes
e^{\mu}\otimes 1)$. Both integrals should be understood in the distributional
sense.

###### Proof.

The equivariance properties follows by direct verification. ∎

######  Proposition 5.2.

For $\xi,\varepsilon\in\mathbb{Z}/2\mathbb{Z}$ we have the following relation

$\displaystyle T_{\mu}^{\varepsilon}\circ
A_{\lambda,\mu}^{\xi}=d^{\xi}_{\lambda,\mu}A_{\lambda,-\mu}^{\xi+\varepsilon},$

where

$d_{\lambda,\mu}^{\xi}=(-1)^{\lfloor\frac{\varepsilon+\xi}{2}\rfloor}\sqrt{\pi}\frac{\Gamma\big{(}\frac{1-\lambda-\mu+2[\xi+\varepsilon]_{2}}{4}\big{)}\Gamma\big{(}\frac{1+\lambda-\mu+2\xi}{4}\big{)}}{\Gamma\big{(}\frac{1+\lambda+\mu+2[\xi+\varepsilon]_{2}}{4}\big{)}\Gamma\big{(}\frac{1-\lambda+\mu+2\xi}{4}\big{)}\Gamma\big{(}\frac{1-\mu+\varepsilon}{2}\big{)}}.$

###### Proof.

Fix $g\in G$ and put $z=x^{-1}g$. Then the set $\\{xH\in G/H\mid
z_{11}z_{21}=0\\}$ is a $d(xH)$-null set. Using Lemma B.2 for
$0<\operatorname{Re}{(\mu)}<1$ we get

$\displaystyle\int_{\overline{N}}K_{-\lambda,-\mu}^{\xi}(zw_{0}\overline{n})d\overline{n}$
$\displaystyle=|z_{11}|_{\xi+\varepsilon}^{\frac{-\mu-\lambda-1}{2}}|z_{21}|_{\xi}^{\frac{\lambda-\mu-1}{2}}\int_{\mathbb{R}}|x|_{\xi+\varepsilon}^{{\frac{-\lambda-\mu-1}{2}}}|x-\frac{1}{z_{11}z_{21}}|_{\xi}^{\frac{\lambda-\mu-1}{2}}dx$
$\displaystyle=\Gamma\big{(}\frac{\mu+\varepsilon}{2}\big{)}d^{\xi}_{\lambda,\mu}K_{-\lambda,\mu}^{\xi+\varepsilon}(z),\quad
a.e.$

The claim then follows by analytic continuation. ∎

We introduce the notation

$\omega_{m}^{\xi}=(-1)^{\xi}+(-1)^{m}i^{m}.$

Note that $\omega_{-m}^{\xi}=\overline{\omega}_{m}^{\xi}$ and as
$\varepsilon\equiv m\mod 2$ we have

$\omega_{m}^{\xi}=\begin{cases}(-1)^{\xi}+(-1)^{\frac{m}{2}},&\varepsilon=0,\\\
(-1)^{\xi}+i(-1)^{\frac{m+1}{2}},&\varepsilon=1,\end{cases}\quad\text{and}\quad\omega_{m}^{0}\overline{\omega}_{m}^{1}=\begin{cases}0,&\varepsilon=0,\\\
2i(-1)^{\frac{m-1}{2}},&\varepsilon=1.\end{cases}$

######  Proposition 5.3.

For $\mu\in\mathbb{C}$ and $m\in\mathbb{Z}$ we have

$\frac{1}{\Gamma\big{(}\frac{\mu+1}{2}\big{)}}P_{\mu}^{\xi}\zeta_{m}(k_{\theta}b_{u})=\zeta_{m}(k_{\theta})\cosh^{\frac{m}{2}}(2u)\left(\omega_{m}^{\xi}c_{m}(\mu)\varphi_{\mu}^{m}(2u)+\frac{i}{2}\omega_{m}^{\xi+1}c_{m}(\mu-2)\psi^{m}_{\mu}(2u)\right),$

where

$\displaystyle c_{m}(\mu)=\frac{2^{1-\mu}\pi
e^{i\frac{m\pi}{4}}}{\Gamma(\frac{\mu+3+|m|}{4})\Gamma(\frac{\mu+3-|m|}{4})}.$

###### Proof.

As $P_{\mu}^{\xi}$ intertwines $\pi_{\varepsilon,\mu}$ and
$\tau_{\varepsilon,0}$ it also intertwines the derived representations
$d\pi_{\varepsilon,\mu}$ and $d\pi_{\varepsilon,0}$. Hence $P_{\mu}^{\xi}$
intertwines $\Delta_{0}$ and $\Delta_{\mu}$ and therefore the image of
$P_{\mu}^{\xi}$ is contained in the eigenspace of $\Delta_{0}$ to the
eigenvalue $\mu^{2}-1$ by Proposition 1.1. Fix $\mu$ with
$\operatorname{Re}{(\mu)}>1$. From Lemma 4.1 it follows for generic $\mu$ that

$\displaystyle
P_{\mu}^{\xi}\zeta_{m}(k_{\theta}b_{u})=\cosh^{\frac{m}{2}}(2u)\zeta_{m}(k_{\theta})\big{(}a_{m}^{\xi}(\mu)\cdot\varphi_{\mu}^{m}(2u)+b^{\xi}_{m}(\mu)\cdot\psi_{\mu}^{m}(2u)\big{)}$

for some $a_{m}^{\xi}(\mu),b_{m}^{\xi}(\mu)\in\mathbb{C}$. Hence, it only
remains to compute $a_{m}^{\xi}(\mu)$ and $b^{\xi}_{m}(\mu)$. Note that
$\varphi_{\mu}^{m}(0)=1$ and $\psi_{\mu}^{m}(0)=0$ and hence
$P_{\mu}^{\xi}\zeta_{m}(k_{\theta})=\zeta_{m}(k_{\theta})a_{m}^{\xi}(\mu)$ so

$\displaystyle a_{m}^{\xi}(\mu)$
$\displaystyle=P_{\mu}^{\xi}\zeta_{m}(e)=\int_{K}K^{\xi}_{\mu}(k_{\theta})\zeta_{m}(k_{\theta})dk_{\theta}=2\int_{0}^{\pi}|\cos\theta|_{\xi+\varepsilon}^{\frac{\mu-1}{2}}|-\sin\theta|^{\frac{\mu-1}{2}}_{\xi}e^{im\theta}\,d\theta$
$\displaystyle=2^{\frac{1-\mu}{2}}\omega_{m}^{\xi}\int_{0}^{\pi}(\sin\theta)^{\frac{\mu-1}{2}}e^{i\frac{m}{2}\theta}d\theta=\frac{2^{1-\mu}\pi\omega_{m}^{\xi}e^{i\frac{m\pi}{4}}\Gamma(\frac{\mu+1}{2})}{\Gamma(\frac{\mu+3+|m|}{4})\Gamma(\frac{\mu+3-|m|}{4})},$

by Lemma A.4.

To compute $b_{m}^{\xi}(\mu)$ it suffices to note that
$\frac{d}{du}\varphi_{\mu}^{m}(2u)\mid_{u=0}=0$ and
$\frac{d}{du}\psi_{\mu}^{m}(2u)\mid_{u=0}\\-=2i$, hence

$\displaystyle 2i\cdot b_{m}^{\xi}(\mu)$
$\displaystyle=\frac{d}{du}P_{\mu}^{\xi}\zeta_{m}(b_{u})\big{|}_{u=0}=\int_{K}\frac{d}{du}K_{\mu}^{\xi}(b_{-u}k_{\theta})\big{|}_{u=0}\zeta_{m}(k_{\theta})dk_{\theta}$
$\displaystyle=\frac{1-\mu}{2}\int_{K}K_{\mu-2}^{\xi+1}(k_{\theta})\zeta_{m}(k_{\theta})dk_{\theta}$
$\displaystyle=\frac{1-\mu}{2}a_{m}^{\xi+1}(\mu-2),$

from which the result follows by analytic continuation. ∎

Let $f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})_{m}$ and write
$f(k_{\theta}b_{u})=\zeta_{m}(k_{\theta})h(u)$ for some $h\in
L^{2}(\mathbb{R},\cosh(2u)du)$. Now let $h_{e}(u)$ be the even part of $h$ and
$h_{o}(u)$ the odd part. We introduce the following notation

$\displaystyle J_{0}f(\mu,\theta)$
$\displaystyle=\zeta_{m}(k_{\theta})J_{0}(\cosh^{-\frac{m}{2}}(x)h_{e}(b_{\frac{x}{2}}))(\mu),$
$\displaystyle J_{1}f(\mu,\theta)$
$\displaystyle=\zeta_{m}(k_{\theta})J_{1}(\sinh^{-1}(x)\cosh^{-\frac{m}{2}}(x)h_{o}(b_{\frac{x}{2}}))(\mu)$

where the $x$ denotes the variable the Fourier–Jacobi transform is done with
respect to.

######  Proposition 5.4.

Let $f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})_{m}$ then

$\frac{1}{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}A_{\mu}^{\xi}f(k_{\theta})=\frac{\overline{\omega}_{m}^{\xi}}{2}c_{-m}(-\mu)J_{0}f(\mu,\theta)+i\frac{\overline{\omega}_{m}^{\xi+1}}{4}c_{-m}(-\mu-2)J_{1}f(\mu,\theta).$

###### Proof.

As $A^{\xi}_{\mu}$ is an intertwining operator, it maps $K$-types to
$K$-types, thus
$A_{\mu}^{\xi}f(k_{\theta})=A_{\mu}^{\xi}f(e)\times\zeta_{m}(k_{\theta})$. Now

$A_{\mu}^{\xi}f(e)=\int_{0}^{\pi}\int_{\mathbb{R}}K_{-\mu}^{\xi}(b_{u}^{-1}k_{\theta}^{-1})f(k_{\theta}b_{u})\cosh(2u)dud\theta=\frac{1}{2}\int_{\mathbb{R}}\cosh(2u)h(u)P_{-\mu}^{\xi}\zeta_{-m}(b_{u})du.$

From Proposition 5.3 and Remark 4.3 we have

$\displaystyle\frac{A_{\mu}^{\xi}f(e)}{\Gamma(\frac{1-\mu}{2})}$
$\displaystyle=\frac{1}{2}\int_{\mathbb{R}}\\!h(u)\cosh^{-\frac{m}{2}+1}(2u)\Big{(}\omega_{-m}^{\xi}c_{-m}(-\mu)\varphi_{-\mu}^{-m}(2u)+\frac{i}{2}\omega^{\xi+1}_{-m}c_{-m}(-\mu-2)\psi^{-m}_{-\mu}(2u)\Big{)}du$
$\displaystyle=\int_{0}^{\infty}h_{e}(u)\cosh^{\frac{m}{2}+1}(2u)\overline{\omega}^{\xi}_{m}c_{-m}(-\mu)\varphi_{\mu}^{m}(2u)du$
$\displaystyle\phantom{=}+\frac{i}{2}\int_{0}^{\infty}h_{o}(u)\cosh^{\frac{m}{2}+1}(2u)\overline{\omega}^{\xi+1}_{m}c_{-m}(-\mu-2)\psi^{m}_{\mu}(2u)du$
$\displaystyle=\frac{1}{2}\overline{\omega}^{\xi}_{m}c_{-m}(-\mu)(J_{0}\cosh^{-\frac{m}{2}}(u)h_{e}(\tfrac{x}{2}))(\mu)$
$\displaystyle\phantom{=}+\frac{i}{4}\overline{\omega}_{m}^{\xi+1}c_{-m}(-\mu-2)(J_{1}\sinh^{-1}(x)\cosh^{-\frac{m}{2}}(x)h_{o}(\tfrac{x}{2}))(\mu).\qed$

Combining Proposition 5.3 and Proposition 5.4 yields an explicit intertwining
operator

$\displaystyle\frac{P_{\mu}^{\xi}A^{\xi^{\prime}}_{\mu}}{\Gamma(\frac{1+\mu}{2})\Gamma(\frac{1-\mu}{2})}:C_{c}^{\infty}\mbox{--}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{0})_{m}\rightarrow\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{0})_{m}^{\infty}.$

By Propositions 5.3 and 5.4, the above intertwining operator is holomorphic in
$\mu$, i.e. the above defines a holomorphic family of intertwining operators,
intertwining $\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})_{m}$ with
itself.

##  6\. Combining intertwining operators

In this section we consider a function $f\in
C_{c}^{\infty}\mbox{--}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{0})_{m}$ and then write

$f(k_{\theta}b_{\frac{u}{2}})=\cosh^{\frac{m}{2}}(u)\bigg{[}\big{(}\cosh^{-\frac{m}{2}}(u)f_{e}(k_{\theta}b_{\frac{u}{2}})\big{)}+\sinh(u)\big{(}\cosh^{-\frac{m}{2}}(u)\sinh^{-1}(u)f_{o}(k_{\theta}b_{\frac{u}{2}})\big{)}\bigg{]},$

then apply the two inversion formulas from Proposition 4.2 to each of the two
terms giving

$f(k_{\theta}b_{\frac{u}{2}})=\cosh^{\frac{m}{2}}(u)\bigg{[}\frac{1}{\pi^{2}}\int_{i\mathbb{R}}J_{0}f(\mu,\theta)\varphi_{\mu}^{m}(u)\frac{d\mu}{4|\ell_{0}(\mu)|^{2}}-\frac{1}{\pi^{2}}\int_{i\mathbb{R}}J_{1}f(\mu,\theta)\psi_{\mu}^{m}(u)\frac{d\mu}{|\ell_{1}(\mu)|^{2}}\\\
-\frac{1}{2\pi}\\!\sum_{\mu\in
D_{0}}\\!J_{0}f(\mu,\theta)\varphi_{\mu}^{m}(u)\underset{\nu=\mu}{\operatorname{Res}}\big{(}\ell_{0}(\nu)\ell_{0}(-\nu)\big{)}^{-1}+\frac{2}{\pi}\\!\sum_{\mu\in
D_{1}}\\!J_{1}f(\mu,\theta)\psi_{\mu}^{m}(u)\underset{\nu=\mu}{\operatorname{Res}}\big{(}\ell_{1}(\nu)\ell_{1}(-\nu)\big{)}^{-1}\bigg{]}.$

The goal is then to express this decomposition in terms of some combination of
the operators
$P_{\mu}^{\xi}A^{\xi^{\prime}}_{\mu}f(k_{\theta}b_{\frac{u}{2}})$ which by a
quick glance at Propositions 5.3 and 5.4 appears plausible. The following
identity will be used multiple times in the following subsections

$\displaystyle 2^{4}c_{m}(\nu)c_{-m}(-\nu)\ell_{0}(\nu)\ell_{0}(-\nu)$
$\displaystyle=c_{m}(\nu-2)c_{-m}(-\nu-2)\ell_{1}(\nu)\ell_{1}(-\nu)$
$\displaystyle=\frac{2^{5}(-1)^{1+\varepsilon}}{\pi}\frac{\cos^{2}\big{(}\frac{\pi(\nu+\varepsilon)}{2}\big{)}}{\nu\sin\big{(}\frac{\pi\nu}{2}\big{)}},$
(3)

which follows from Gamma-function identities and recalling that
$m\equiv\varepsilon\mod 2$.

###  6.1. The continuous part

For $\mu\in\mathbb{C}$ we introduce the following maps

$\displaystyle\textbf{P}^{\xi}_{\mu}=\frac{P_{\mu}^{\xi}}{\Gamma(\frac{\mu+1}{2})},\quad\textbf{A}_{\mu}^{\xi}=\frac{A_{\mu}^{\xi}}{\Gamma(\frac{1-\mu}{2})},$

which are holomorphic in $\mu$.

######  Proposition 6.1.

We have

$\sum_{\xi=0}^{1}\mathbf{P}_{\mu}^{\xi}\mathbf{A}_{\mu}^{\xi}f(k_{\theta}b_{u})\\\
=\cosh^{\frac{m}{2}}(2u)\bigg{[}2c_{m}(\mu)c_{-m}(-\mu)J_{0}f(\mu,\theta)\varphi_{\mu}^{m}(2u)-\frac{1}{2}c_{m}(\mu-2)c_{-m}(-\mu-2)J_{1}f(\mu,\theta)\psi_{\mu}^{m}(2u)\bigg{]}.$

Combining this with (6) we get

$\int_{i\mathbb{R}}\sum_{\xi=0}^{1}\mathbf{P}_{\mu}^{\xi}\mathbf{A}_{\mu}^{\xi}f(k_{\theta}b_{\frac{u}{2}})\frac{d\mu}{|a(\mu)|^{2}}\\\
=\cosh^{\frac{m}{2}}(u)\bigg{[}\frac{1}{\pi^{2}}\int_{i\mathbb{R}}J_{0}f(\mu,\theta)\varphi_{\mu}^{m}(u)\frac{d\mu}{4|\ell_{0}(\mu)|^{2}}-\frac{1}{\pi^{2}}\int_{i\mathbb{R}}J_{1}f(\mu,\theta)\psi_{\mu}^{m}(u)\frac{d\mu}{|\ell_{1}(\mu)|^{2}}\bigg{]}$

where

$\displaystyle
a(\mu)=2^{\frac{3}{2}}\pi\frac{\Gamma(\frac{\mu}{2})}{\Gamma(\frac{1+\mu+\varepsilon}{2})\Gamma(\frac{1+\mu-\varepsilon}{2})}.$

###### Proof.

When computing
$\sum_{\xi=0}^{1}\mathbf{P}_{\mu}^{\xi}\mathbf{A}_{\mu}^{\xi}f(k_{\theta}b_{\frac{u}{2}})$
we apply Proposition 5.3 and 5.4. We obtain some cross-terms, containing
factors like $J_{0}f(\mu,\theta)\psi_{\mu}^{m}(u)$, but since

$\displaystyle\sum\limits_{\xi=0}^{1}\omega_{m}^{\xi}\overline{\omega}_{m}^{\xi+1}=0\text{
and }\sum\limits_{\xi=0}^{1}\omega_{m}^{\xi}\overline{\omega}_{m}^{\xi}=4,$

no cross-terms survive and the assertion follows. ∎

To express the discrete part in terms of
$P_{\mu}^{\xi}A_{\mu}^{\xi^{\prime}}f$ is a bit more delicate as we cannot
simply take a sum to make the cross terms disappear thus we need to make a
suitable choice of normalization. The cases for $\varepsilon=0$ and
$\varepsilon=1$ will be treated differently and the main culprit as to why is
the factor $\omega_{m}^{\xi}$ which for $\varepsilon=0$ vanishes depending on
the parity of $\frac{m}{2}$ and for $\varepsilon=1$ never vanishes.

###  6.2. The discrete part for $\varepsilon=0$

In this subsection we fix $\varepsilon=0$. Consider the following
normalizations

$\displaystyle\widehat{P}^{\xi}_{\mu}=\frac{\Gamma(\frac{\mu+3-2\xi}{4})}{\Gamma(\frac{\mu+1}{2})\Gamma(\frac{\mu+1+2\xi}{4})}P^{\xi}_{\mu},\quad\widehat{A}^{\xi}_{\mu}=\frac{\Gamma(\frac{-\mu+3-2\xi}{4})}{\Gamma(\frac{-\mu+1}{2})\Gamma(\frac{-\mu+1+2\xi}{4})}A^{\xi}_{\mu},$

which, by the duplication formula for the Gamma-function, does not introduce
any poles. Now introduce the operators

$\displaystyle\mathbb{P}_{\mu}:=\widehat{P}^{0}_{\mu}+\widehat{P}^{1}_{\mu}\quad\text{and}\quad\mathbb{A}_{\mu}:=\widehat{A}^{0}_{\mu}+\widehat{A}^{1}_{\mu}.$

######  Lemma 6.2.

For a fixed $m\in 2\mathbb{Z}$ we have

$\mathbb{P}_{\mu}\zeta_{m}(k_{\theta}b_{u})=\zeta_{m}(k_{\theta})\cosh^{\frac{m}{2}}(2u)\big{(}\alpha_{m}(\mu)\varphi_{\mu}^{m}(2u)+\beta_{m}(\mu)\psi_{\mu}^{m}(2u)\big{)},$

where

$\displaystyle\alpha_{m}(\mu)=c_{m}(\mu)\left(\omega_{m}^{0}\frac{\Gamma(\frac{\mu+3}{4})}{\Gamma(\frac{\mu+1}{4})}+\omega_{m}^{1}\frac{\Gamma(\frac{\mu+1}{4})}{\Gamma(\frac{\mu+3}{4})}\right),$

and

$\displaystyle\beta_{m}(\mu)=\frac{i}{2}c_{m}(\mu-2)\left(\omega_{m}^{1}\frac{\Gamma(\frac{\mu+3}{4})}{\Gamma(\frac{\mu+1}{4})}+\omega_{m}^{0}\frac{\Gamma(\frac{\mu+1}{4})}{\Gamma(\frac{\mu+3}{4})}\right).$

Furthermore if $\mu\in 1-2\mathbb{N}$ then $\alpha_{m}(\mu)$ is only non-zero
when $\mu\in D_{0}$ and $\beta_{m}(\mu)$ is only non-zero when $\mu\in D_{1}$.

###### Proof.

The first identity is a direct consequence of Proposition 5.3. To see the
second assertion rewrite

$\alpha_{m}(\mu)=\frac{2^{1-\mu}\pi e^{\frac{\pi
im}{4}}}{\Gamma\big{(}\frac{\mu+3+|m|}{4}\big{)}}\bigg{(}\omega_{m}^{0}\frac{\big{(}\frac{\mu+3-|m|}{4}\big{)}_{\frac{|m|}{4}}}{\Gamma\big{(}\frac{\mu+1}{4}\big{)}}+\omega_{m}^{1}\frac{\big{(}\frac{\mu+3-|m|}{4}\big{)}_{\frac{|m|-2}{4}}}{\Gamma\big{(}\frac{\mu+3}{4}\big{)}}\bigg{)}.$

As either $\omega_{m}^{0}$ or $\omega_{m}^{1}$ is vanishing this makes sense
term by term. When $\mu$ is of the form $\mu=4k+3-|m|$ for $k\in\mathbb{Z}$
then term by term $\Gamma(\frac{\mu+1}{4})^{-1}$ and
$\Gamma(\frac{\mu+3}{4})^{-1}$ vanishes. When $\mu$ has the form
$\mu=4k+1-|m|$ for $k\in-\mathbb{N}$ then $\Gamma(\frac{\mu+3+|m|}{4})^{-1}$
vanishes. A similarly argument applies to $\beta_{m}(\mu)$. ∎

######  Lemma 6.3.

We have

$\mathbb{A}_{\mu}f(k_{\theta})=\tilde{\alpha}_{m}(\mu)J_{0}f(\mu,\theta)+\tilde{\beta}_{m}(\mu)J_{1}f(\mu,\theta),$

where

$\tilde{\alpha}_{m}(\mu)=\frac{1}{2}c_{-m}(-\mu)\left(\overline{\omega}_{m}^{0}\frac{\Gamma(\frac{3-\mu}{4})}{\Gamma(\frac{1-\mu}{4})}+\overline{\omega}_{m}^{1}\frac{\Gamma(\frac{1-\mu}{4})}{\Gamma(\frac{3-\mu}{4})}\right),$
$\tilde{\beta}_{m}(\mu)=\frac{i}{4}c_{-m}(-\mu-2)\left(\overline{\omega}_{m}^{1}\frac{\Gamma(\frac{-\mu+3}{4})}{\Gamma(\frac{-\mu+1}{4})}+\overline{\omega}_{m}^{0}\frac{\Gamma(\frac{-\mu+1}{4})}{\Gamma(\frac{-\mu+3}{4})}\right).$

Furthermore, if $\mu\in 1-2\mathbb{N}$ of the form $\mu=4k+1-|m|$ we have
$\tilde{\beta}(\mu)=0$ and similarly for $\mu$ of the form $\mu=4k+3-|m|$ we
have $\tilde{\alpha}_{m}(\mu)=0.$

###### Proof.

This follows from Proposition 5.3 and considerations similar to those in the
proof of Lemma 6.2. ∎

######  Lemma 6.4.

For $\mu\in D_{0}$:

$\mathbb{P}_{\mu}\mathbb{A}_{\mu}f(k_{\theta}b_{u})=\cosh^{\frac{m}{2}}(2u)\alpha_{m}(\mu)\widetilde{\alpha}_{m}(\mu)J_{0}f(\mu,\theta)\varphi^{m}_{\mu}(2u),$

and for $\mu\in D_{1}$:

$\displaystyle\mathbb{P}_{\mu}\mathbb{A}_{\mu}f(k_{\theta}b_{u})=\cosh^{\frac{m}{2}}(2u)\beta_{m}(\mu)\tilde{\beta}_{m}(\mu)J_{1}f(\mu,\theta)\psi^{m}_{\mu}(2u).$

Furthermore, if $\mu\in(1-2\mathbb{N})\setminus(D_{0}\cup D_{1})$ then
$\mathbb{P}_{\mu}\mathbb{A}_{\mu}f(k_{\theta}b_{u})=0$.

###### Proof.

This is a direct consequence of the two preceding lemmas. ∎

Consider the non-vanishing entire analytic function

$\operatorname{\text{\Capricorn}}(\mu)=\frac{1}{\Gamma\big{(}\frac{1+\mu}{4}\big{)}^{2}\Gamma\big{(}\frac{1-\mu}{4}\big{)}^{2}}+\frac{1}{\Gamma\big{(}\frac{3+\mu}{4}\big{)}^{2}\Gamma\big{(}\frac{3-\mu}{4}\big{)}^{2}}.$

######  Lemma 6.5.

For $\mu\in D_{0}$

$(-2\pi)\alpha_{m}(\mu)\widetilde{\alpha}_{m}(\mu)\ell_{0}(\mu)\ell_{0}(-\mu)=16\pi^{2}\frac{\cot\big{(}\frac{\pi\mu}{2}\big{)}\operatorname{\text{\Capricorn}}(\mu)}{\mu}.$

For $\mu\in D_{1}$

$\frac{\pi}{2}\beta_{m}(\mu)\widetilde{\beta}_{m}(\mu)\ell_{1}(\mu)\ell_{1}(-\mu)=16\pi^{2}\frac{\cot\big{(}\frac{\pi\mu}{2}\big{)}\operatorname{\text{\Capricorn}}(\mu)}{\mu}.$

###### Proof.

This follows from (6). One trick is used which arises when a term like

$\frac{|\omega_{m}^{0}|^{2}}{\Gamma\big{(}\frac{\mu+1}{2}\big{)}^{2}},$

is obtained. As $\omega_{m}^{0}$ is either 0 or 2 we can set
$\omega_{m}^{0}=2$ as $\Gamma(\frac{\mu+1}{2})^{-1}$ vanishes in the same
cases as $\omega_{m}^{0}$. ∎

######  Proposition 6.6.

We have

$\displaystyle\frac{1}{32\pi}\sum_{\mu\in
1-2\mathbb{N}}\\!\frac{-\mu}{\operatorname{\text{\Capricorn}}(\mu)}\mathbb{P}_{\mu}\mathbb{A}_{\mu}f(k_{\theta}b_{u})=\cosh^{\frac{m}{2}}(2u)\bigg{[}$
$\displaystyle\frac{-1}{2\pi}\sum_{\mu\in
D_{0}}J_{0}f(\mu,\theta)\varphi_{\mu}^{m}(2u)\underset{\nu=\mu}{\operatorname{Res}}\big{(}\ell_{0}(\nu)\ell_{0}(-\nu)\big{)}^{-1}$
$\displaystyle+\frac{2}{\pi}\sum_{\mu\in
D_{1}}J_{1}f(\mu,\theta)\psi_{\mu}^{m}(2u)\underset{\nu=\mu}{\operatorname{Res}}\big{(}\ell_{1}(\nu)\ell_{1}(-\nu)\big{)}^{-1}\bigg{]}.$

###### Proof.

Apply Lemma 6.4 to the right hand side. Now note that $c_{m}(\mu)c_{-m}(-\mu)$
is regular for $\mu\in D_{0}$ and $c_{m}(\mu-2)c_{-m}(-\mu-2)$ is regular for
$\mu\in D_{1}$, thus they can be moved inside the residues. Then everything
follows from Lemma 6.5 after recalling that
$\operatorname{Res}_{\nu=\mu}\tan(\frac{\pi\nu}{2})=-\frac{\pi}{2}$. ∎

###  6.3. The discrete part for $\varepsilon=1$

In this subsection we fix $\varepsilon=1$. The proof will proceed using the
same ideas as for $\varepsilon=0$. For $\mu\in-2\mathbb{N}$ let

$\mathcal{A}_{\mu}=\frac{1}{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}A_{\mu}^{0},\quad\text{and}\quad\mathcal{P}_{\mu}\zeta_{m}=\frac{(-1)^{\frac{m+|m|-2}{2}}}{\Gamma\big{(}\frac{1+\mu}{2}\big{)}}P_{\mu}^{1}\zeta_{m},$

that is we define $\mathcal{P}_{\mu}$ by its eigenvalues on $K$-types. By
Proposition 5.3 we get $\mathcal{P}_{\mu}$ is intertwining by the same
argument we used for $\hat{T}^{1}_{\mu}$ in section 1.

######  Lemma 6.7.

For $\mu\in D_{0}$ we have

$\mathcal{P}_{\mu}\mathcal{A}_{\mu}f(k_{\theta}b_{u})=\alpha_{m}(\mu)\cosh^{\frac{m}{2}}(2u)\varphi_{\mu}^{m}(2u)J_{0}f(\mu,\theta),$

where

$\alpha_{m}(\mu)=i(-1)^{\frac{|m|+1}{2}}c_{m}(\mu)c_{-m}(-\mu)$

For $\mu\in D_{1}$ we have

$\mathcal{P}_{\mu}\mathcal{A}_{\mu}f(k_{\theta}b_{u})=\beta_{m}(\mu)\cosh^{\frac{m}{2}}(2u)\psi_{\mu}^{m}(2u)J_{1}f(\mu,\theta),$

where

$\beta_{m}(\mu)=\frac{1}{4}i(-1)^{\frac{|m|+1}{2}}c_{m}(\mu-2)c_{-m}(-\mu-2).$

Furthermore if $\mu\in-2\mathbb{N}$ then $\alpha_{m}(\mu)$ is only non-zero if
$\mu\in D_{0}$ and $\beta_{m}(\mu)$ is only non-zero if $\mu\in D_{1}$.

###### Proof.

The proof is an application of Propositions 5.3 and 5.4. ∎

######  Lemma 6.8.

For $\mu\in D_{0}$

$(-2\pi)\alpha_{m}(\mu)\ell_{0}(\mu)\ell_{0}(-\mu)=4i(-1)^{\frac{|m|-1}{2}}\frac{\sin\big{(}\frac{\pi\mu}{2}\big{)}}{\mu},$

and for $\mu\in D_{1}$

$\frac{\pi}{2}\beta_{m}(\mu)\ell_{1}(\mu)\ell_{1}(-\mu)=4i(-1)^{\frac{|m|+1}{2}}\frac{\sin\big{(}\frac{\pi\mu}{2}\big{)}}{\mu}.$

###### Proof.

This is a direct consequence of (6). ∎

######  Proposition 6.9.

$\displaystyle\frac{1}{2\pi
i}\sum_{\mu\in-2\mathbb{N}}\mu\mathcal{P}_{\mu}\mathcal{A}_{\mu}f(k_{\theta}b_{u})=\cosh^{\frac{m}{2}}(2u)\bigg{[}$
$\displaystyle-\frac{1}{2\pi}\sum_{\mu\in
D_{0}}J_{0}f(\mu,\theta)\varphi_{\mu}^{m}(2u)\underset{\nu=\mu}{\operatorname{Res}}\big{(}\ell_{0}(\nu)\ell_{0}(-\nu)\big{)}^{-1}$
$\displaystyle+\frac{2}{\pi}\sum_{\mu\in
D_{1}}J_{1}f(\mu,\theta)\psi_{\mu}^{m}(2u)\underset{\nu=\mu}{\operatorname{Res}}\big{(}\ell_{1}(\nu)\ell_{1}(-\nu)\big{)}^{-1}\bigg{]}.$

###### Proof.

This follows in the same manner as the proof for Proposition 6.6, where we
here note for $\mu=4k+1-|m|\in D_{0}$ that
$\operatorname{Res}_{\nu=\mu}\sin\big{(}\frac{\pi\nu}{2}\big{)}^{-1}=\frac{2}{\pi}(-1)^{\frac{|m|-1}{2}},$
and for $\mu\in D_{1}$ we have
$\operatorname{Res}_{\nu=\mu}\sin\big{(}\frac{\pi\nu}{2}\big{)}^{-1}=\frac{2}{\pi}(-1)^{\frac{|m|+1}{2}}$.
∎

##  7\. The Plancherel formula

The intertwining operators $P_{\mu}^{\xi}$ and $A_{\mu}^{\xi}$ are continuous
maps and hence the intertwining operators introduced in the previous section
are also continuous. This allows for an extension of the results obtained for
K-types, described by the first theorem of this section. We then extend this
theorem to arbitrary $\lambda\in i\mathbb{R}$ by virtue of Theorem 3.1

Recall that

$\displaystyle
a(\mu)=2^{\frac{3}{2}}\pi\frac{\Gamma(\frac{\mu}{2})}{\Gamma(\frac{1+\mu+\varepsilon}{2})\Gamma(\frac{1+\mu-\varepsilon}{2})},\quad\text{and}\quad\operatorname{\text{\Capricorn}}(\mu)=\frac{1}{\Gamma\big{(}\frac{1+\mu}{4}\big{)}^{2}\Gamma\big{(}\frac{1-\mu}{4}\big{)}^{2}}+\frac{1}{\Gamma\big{(}\frac{3+\mu}{4}\big{)}^{2}\Gamma\big{(}\frac{3-\mu}{4}\big{)}^{2}}.$
(4)

######  Theorem 7.1 (Plancherel formula for $\lambda=0$).

For $\varepsilon=0$ and $f\in
C_{c}^{\infty}\mbox{--}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})$
we have the following inversion formula

$f(k_{\theta}b_{u})=\int_{i\mathbb{R}}\sum\limits_{\xi=0}^{1}\textbf{P}_{\mu}^{\xi}\textbf{A}_{\mu}^{\xi}f(k_{\theta}b_{u})\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{32\pi}\sum_{\mu\in
1-2\mathbb{N}}\frac{-\mu}{\operatorname{\text{\Capricorn}}(\mu)}\mathbb{P}_{\mu}\mathbb{A}_{\mu}f(k_{\theta}b_{u}),$

and the corresponding Plancherel formula

$\displaystyle\|f\|^{2}$
$\displaystyle=\int_{i\mathbb{R}}\sum_{\xi=0}^{1}\|\textbf{A}^{\xi}_{\mu}f\|^{2}\,\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{16\pi}\sum_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma\big{(}\frac{-\mu}{2}\big{)}\operatorname{\text{\Capricorn}}(\mu)}\|\mathbb{A}_{\mu}f\|^{2}.$

For $\varepsilon=1$ and $f\in
C_{c}^{\infty}\mbox{--}\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})$
we have the following inversion formula

$f(k_{\theta}b_{u})=\int_{i\mathbb{R}}\sum\limits_{\xi=0}^{1}\textbf{P}_{\mu}^{\xi}\textbf{A}_{\mu}^{\xi}f(k_{\theta}b_{u})\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{2\pi
i}\sum_{\mu\in-2\mathbb{N}}\mu\mathcal{P}_{\mu}\mathcal{A}_{\mu}f(k_{\theta}b_{u}),$

and the corresponding Plancherel formula

$\displaystyle\|f\|^{2}=\int_{i\mathbb{R}}\sum_{\xi=0}^{1}\|\textbf{A}^{\xi}_{\mu}f\|^{2}\,\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{2\pi}\sum_{\mu\in-2\mathbb{N}}\frac{\Gamma(-\mu)\mu^{2}}{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}\|\mathcal{A}_{\mu}f\|^{2}.$

###### Proof.

The inversion formulas follow directly from the introduction and results of
Section 6. To get the Plancherel formula write

$\|f\|^{2}=\int_{0}^{\pi}\int_{\mathbb{R}}f(k_{\theta}b_{u})\overline{f(k_{\theta}b_{u})}\cosh(2u)\,dud\theta,$

and use the inversion formula on $f(k_{\theta}b_{u})$ and apply that

$\int_{G/H}P^{\xi}_{\mu}f(xH)\overline{g(xH)}\,d(xH)=\int_{K}f(k)\overline{A_{-\overline{\mu}}^{\xi}g(k)}\,dk,$

for $f\in\operatorname{Ind}_{P}^{G}(\varepsilon\otimes e^{\mu})$ and
$g\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})$. Lastly for the
discrete part, we apply Proposition 5.2 to get

$T_{\mu}^{0}\mathbb{A}_{\mu}=\frac{\sqrt{\pi}2^{\mu}}{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}\mathbb{A}_{-\mu}\quad\text{and}\quad
T_{\mu}^{1}\mathcal{A}_{\mu}=\frac{\sqrt{\pi}2^{\mu}}{\Gamma\big{(}\frac{2-\mu}{2}\big{)}\Gamma\big{(}\frac{1+\mu}{2}\big{)}}A_{-\mu}^{1},$

giving the final result. ∎

We now extend the previous result from $\lambda=0$ to $\lambda\in i\mathbb{R}$
using Theorem 3.1. We want to compose $A_{\mu}$ and
$\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}$ but as we cannot ensure the
regularity of the functions in the image of
$\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}$ we end up doing this in an
$L^{2}$-sense using direct integrals. Consider the following operators

$\mathbb{A}_{\lambda,\mu}:=\frac{2^{\frac{1+\mu}{2}}\sqrt{\pi}\Gamma\big{(}\frac{1+\mu}{4}+\frac{\lambda}{4}\big{)}}{\Gamma\big{(}\frac{1-\mu}{4}\big{)}\Gamma\big{(}\frac{1+\mu}{4}\big{)}\Gamma\big{(}\frac{1-\mu}{4}-\frac{\lambda}{4}\big{)}}A_{\lambda,\mu}^{0}+\frac{2^{\frac{1+\mu}{2}}\sqrt{\pi}\Gamma\big{(}\frac{3+\mu}{4}+\frac{\lambda}{4}\big{)}}{\Gamma\big{(}\frac{3-\mu}{4}\big{)}\Gamma\big{(}\frac{3+\mu}{4}\big{)}\Gamma\big{(}\frac{3-\mu}{4}-\frac{\lambda}{4}\big{)}}A_{\lambda,\mu}^{1},$
$\mathbf{A}_{\lambda,\mu}^{\xi}:=\frac{A_{\lambda,\mu}^{\xi}}{\Gamma(\frac{1-\mu}{2})},\quad\text{and}\quad\mathcal{A}_{\lambda,\mu}:=\frac{2^{\frac{1+\mu}{2}}\sqrt{\pi}\Gamma\big{(}\frac{1+\mu}{4}+\frac{\lambda}{4}\big{)}}{\Gamma\big{(}\frac{1-\mu}{4}\big{)}\Gamma\big{(}\frac{3+\mu}{4}\big{)}\Gamma\big{(}\frac{1-\mu}{4}-\frac{\lambda}{4}\big{)}}A_{\lambda,\mu}^{0},$

which are extensions of $\mathbb{A}_{\mu}$, $\textbf{A}_{\mu}^{\xi}$ and
$\mathcal{A}_{\mu}$ e.g. $\mathbb{A}_{0,\mu}=\mathbb{A}_{\mu}$. Furthermore
let

$\displaystyle\mathcal{H}_{\varepsilon}=\int_{i\mathbb{R}}^{\oplus}\pi_{\varepsilon,\mu}\otimes\mathbb{C}^{2}\,d\mu\oplus\bigoplus\limits_{\mu\in
1-\varepsilon-2\mathbb{N}}\pi^{\text{ds}}_{\varepsilon,\mu}.$

where $\int^{\oplus}_{U}H_{\mu}d\mu$ denotes a direct integral of Hilbert
spaces, see e.g [4] for a short exposition. The inner-product on
$\mathcal{H}_{\varepsilon}$ is given by Theorem 7.1, i.e. for $\varepsilon=0$
and $f,h\in\mathcal{H}_{0}$

$\displaystyle\langle
g,h\rangle_{\mathcal{H}}=\int_{i\mathbb{R}}\sum\limits_{\xi=0}^{1}\langle
g^{\xi}_{\mu},h_{\mu}^{\xi}\rangle_{L^{2}(K)}\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{16\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma(\frac{-\mu}{2})\operatorname{\text{\Capricorn}}(\mu)}\langle
T^{0}_{\mu}g_{\mu},h\rangle_{L^{2}(K)}.$

For simplicity we shall assume that $\varepsilon=0$ for the remainder of the
section. All arguments made can be done for $\varepsilon=1$ as well using the
corresponding results from the previous section.

Abusing notation, Theorem 7.1 defines an isometry
$A_{0}:C_{c}^{\infty}(G/H)\rightarrow\mathcal{H}_{0}$ which extends to an
isometry

$A_{0}:\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
1)\rightarrow\mathcal{H}_{0}.$

For $f\in\mathcal{H}_{0}$ with $f=(f^{0},f^{1},f^{d})$ we introduce the
following map

$\displaystyle P_{0}:\mathcal{H}_{0}\rightarrow L^{2}(G/H)$ $\displaystyle
f=(f^{0},f^{1},f^{d})\mapsto\int_{i\mathbb{R}}$
$\displaystyle\sum\limits_{\xi=0}^{1}\textbf{P}^{\xi}_{\mu}f^{\xi}_{\mu}\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{32\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{-\mu}{\operatorname{\text{\Capricorn}}(\mu)}\mathbb{P}_{\mu}f^{d}_{\mu}.$

######  Lemma 7.2.

For $f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})$ and
$h\in\mathcal{H}_{0}$ we have the following relation:

$\displaystyle\langle A_{0}f,h\rangle_{\mathcal{H}}=\langle
f,P_{0}h\rangle_{L^{2}(G/H)}.$

###### Proof.

Let $f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})$ and $h\in
C^{\infty}_{c}(\mathcal{H}_{0})$. We then find

$\displaystyle\langle
A_{0}f,h\rangle_{\mathcal{H}}=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\int_{K}\textbf{A}_{\mu}^{\xi}f(k)\overline{h^{\xi}_{\mu}(k)}dk\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{16\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma(\frac{-\mu}{2})\operatorname{\text{\Capricorn}}(\mu)}\langle
T^{0}_{\mu}\circ\mathbb{A}_{\mu}f,h^{\xi}_{\mu}\rangle_{L^{2}(K)}$
$\displaystyle=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\int_{G/H}f(x)\overline{\textbf{P}_{\mu}^{\xi}h^{\xi}_{\mu}(x)}d(xH)+\frac{1}{16\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma(\frac{-\mu}{2})\operatorname{\text{\Capricorn}}(\mu)}\frac{\sqrt{\pi}2^{\mu}}{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}\langle\mathbb{A}_{-\mu}f,h^{d}_{\mu}\rangle_{L^{2}(K)}$
$\displaystyle=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\langle
f,\textbf{P}_{\mu}^{\xi}h^{\xi}_{\mu}\rangle_{L^{2}(G/H)}\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{32\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{-\mu}{\operatorname{\text{\Capricorn}}(\mu)}\langle
f,\mathbb{P}_{\mu}h^{d}_{\mu}\rangle_{L^{2}(G/H)}$ $\displaystyle=\langle
f,P_{0}h\rangle_{L^{2}(G/H)}.\qed$

######  Lemma 7.3 (See [12, Theorem 1]).

Suppose $S:\mathcal{H}^{\infty}\rightarrow\mathcal{H}^{\infty}$ is a
continuous intertwining operator for $\mathcal{H}^{\infty}$. Then for a.e
$\mu\in i\mathbb{R}\cup(1-\varepsilon-2\mathbb{N})$ there exists unique
$\mathcal{H}^{\infty}$ intertwining operators $S_{\mu}$ for
$\pi_{\varepsilon,\mu}^{\infty}\otimes\mathbb{C}^{2}$ if $\mu\in i\mathbb{R}$
and for $\pi_{\varepsilon,\mu}^{\text{ds}}$ if $\mu\in
1-\varepsilon-2\mathbb{N}$ such that

$(Sf)_{\mu}=S_{\mu}f_{\mu}\quad\text{a.e }\mu\in i\mathbb{R}\cup(\mu\in
1-\varepsilon-2\mathbb{N}),\,f\in\mathcal{H}^{\infty}.$

######  Proposition 7.4.

The map $A^{\varepsilon}_{0}:\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{0})\rightarrow\mathcal{H}_{0}$ is surjective. In particular $A_{0}$ is an
isometric isomorphism.

###### Proof.

Since the discrete and continuous part of $\mathcal{H}_{0}$ consist of
pairwise inequivalent representations of $G$ it suffices to show that the
projection of $W=\overline{\text{Image}(A_{0})}$ onto the continuous part and
the discrete part respectively, is surjective. For the projection to the
discrete part we can consider the projection of $W$ onto each summand
$\pi^{\text{ds}}_{0,\mu}$. Proposition 1.2 gives
$\pi_{0,\mu}^{\text{ds}}=\pi^{\textup{hol}}_{0,\mu}\oplus\pi^{\textup{ahol}}_{0,\mu}$
and since these representations are inequivalent it again suffices to show
that the projection on each of them are onto. Lemma 6.3 then shows that
$\operatorname{proj}_{\pi_{0,\mu}^{\textup{hol}}}(W)\neq 0$ and
$\operatorname{proj}_{\pi^{\textup{ahol}}_{0,\mu}}(W)\neq 0$. But since the
projection is $G$-equivariant the image is a subrepresentation and it follows
that both projections must be onto.

Since the projection onto an integrand of the continuous part of
$\mathcal{H}_{0}$ is in general not point-wise defined, the proof differs to
that of the discrete part. Abusing notation slightly we shall write
$f_{\mu}=(f_{\mu}^{0},f_{\mu}^{1})$ when $\mu\in i\mathbb{R}$ and
$f\in\mathcal{H}_{0}$, omitting the discrete part.

By Lemma 7.2 $(A_{0})^{*}=P_{0}$ and since the adjoint is $G$-equivariant we
have

$P_{0}(\mathcal{H}_{0}^{\infty})\subseteq L^{2}(G/H)^{\infty}\quad\text{ and
}\quad A_{0}(L^{2}(G/H)^{\infty})\subseteq\mathcal{H}_{0}^{\infty}.$

Let $A_{0}^{\infty}=A_{0}|_{L^{2}(G/H)^{\infty}}$ and
$P^{\infty}_{0}=P_{0}|_{\mathcal{H}_{0}^{\infty}}$. Then

$S=A^{\infty}_{0}\circ
P^{\infty}_{0}:\mathcal{H}_{0}^{\infty}\rightarrow\mathcal{H}_{0}^{\infty}$

is a $\mathcal{H}_{0}^{\infty}$ intertwining map and by Lemma 7.3 there exists
a family of $\mathcal{H}_{0}^{\infty}$ intertwining operators $(S_{\mu})$ such
that $((A^{\infty}_{0}\circ P^{\infty}_{0})f)_{\mu}=S_{\mu}f_{\mu}$ for a.e
$\mu\in i\mathbb{R}$ and all $f\in\mathcal{H}_{0}^{\infty}$. By Schur’s lemma
this implies $S_{\mu}=(\operatorname{id}\otimes B_{\mu})$ for a.e $\mu\in
i\mathbb{R}$, with $B_{\mu}:\mathbb{C}^{2}\rightarrow\mathbb{C}^{2}$ a linear
map. Let $N$ denote the corresponding null-set, we then show that for $\mu\in
i\mathbb{R}\setminus N$ we have $S_{\mu}=\operatorname{id}$.

To this extent let $\mu\in i\mathbb{R}\setminus N$ and let $f$ be a $K$-finite
vector in $\mathcal{H}_{0}^{\infty}$ and note that this implies $f_{\mu}$ must
be a $K$-finite vector in $\pi_{\mu}^{\infty}$. Assume therefore without loss
of generality that $f_{\mu}=(c_{1}\zeta_{m},c_{2}\zeta_{n})$ for some $m,n\in
2\mathbb{Z}$ and pick by Proposition 5.4 a $K$-finite vector in
$L^{2}(G/H)^{\infty}$ such that $A_{0}(w)_{\mu}=f_{\mu}$. One can e.g pick $w$
of the form $w=\zeta_{m}f_{1}+\zeta_{n}f_{2}$ with $f_{1}$ an even function
and $f_{2}$ an odd function of correct regularity and growth. Then we have

$\displaystyle(A^{\infty}_{0}\circ P^{\infty}_{0}f)_{\mu}=A^{\infty}_{0}\circ
P^{\infty}_{0}\circ
A^{\infty}_{0}(w)_{\mu}=A_{0}^{\infty}(w)_{\mu}=(A_{\mu}^{0}w,A_{\mu}^{1}w),$

where the second equality follows from the inversion formula given by Theorem
7.1. On the other hand we have

$\displaystyle(A^{\infty}_{0}\circ
P^{\infty}_{0}f)_{\mu}=S_{\mu}f_{\mu}=(\operatorname{id}\otimes
B_{\mu})A_{0}^{\infty}(w)_{\mu}=((b_{11})_{\mu}A_{\mu}^{0}w+(b_{12})_{\mu}A^{1}_{\mu}w,(b_{21})_{\mu}A_{\mu}^{0}w+(b_{22})_{\mu}A^{1}_{\mu}w)$

hence

$\displaystyle(A_{\mu}^{0}w,A_{\mu}^{1}w)=(a_{\mu}A_{\mu}^{0}w+b_{\mu}A^{1}_{\mu}w,c_{\mu}A_{\mu}^{0}w+d_{\mu}A^{1}_{\mu}w).$

Since $A_{\mu}^{0}$ and $A_{\mu}^{1}$ are linearly independent for $\mu\in
i\mathbb{R}$ it follows that $B_{\mu}=\operatorname{id}$ and hence
$S_{\mu}=\operatorname{id}$ on the $K$-finite vectors of $\mathcal{H}_{0}$,
for a.e $\mu\in i\mathbb{R}$ and since the $K$-finite vectors form a dense
subset the result follows. ∎

######  Theorem 7.5.

For $\lambda\in i\mathbb{R}$ and $\varepsilon\in\mathbb{Z}/2\mathbb{Z}$ we
have

$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\cong\int_{i\mathbb{R}}^{\oplus}\pi_{\varepsilon,\mu}\oplus\pi_{\varepsilon,\mu}\,\frac{d\mu}{|a(\mu)|^{2}}\,\oplus\bigoplus_{\mu\in
1-\varepsilon-2\mathbb{N}}\pi_{\varepsilon,\mu}^{\textup{hol}}\oplus\pi_{\varepsilon,\mu}^{\textup{ahol}},$

where the map is given by

$\displaystyle
f\mapsto(p_{\lambda,\mu}^{0}\textbf{A}_{\lambda,\mu}^{0}f,p_{\lambda,\mu}^{1}\textbf{A}_{\lambda,\mu}^{1}f,\mathbb{A}_{\lambda,\mu}f),\qquad\text{for
}f\in\operatorname{Ind}_{H}^{G}(0\otimes e^{\lambda})$ $\displaystyle
f\mapsto(p_{\lambda,\mu}^{0}\textbf{A}_{\lambda,\mu}^{0}f,p_{\lambda,\mu}^{1}\textbf{A}_{\lambda,\mu}^{1}f,\mathcal{A}_{\lambda,\mu}f),\qquad\text{for
}f\in\operatorname{Ind}_{H}^{G}(1\otimes e^{\lambda})$

###### Proof.

For $\lambda\in i\mathbb{R}$ the map
$A_{0}:\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
1)\rightarrow\mathcal{H}_{0}$ gives rise to an isometric isomorphism
$\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\rightarrow\mathcal{H}_{0}$ by composition with
$\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}:\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})\rightarrow\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{0})$
from Theorem 3.1. Let $\lambda\in i\mathbb{R}$,
$f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes e^{\lambda})$ and $h\in
C_{c}^{\infty}(\mathcal{H}_{0})$. Then by Lemma 7.2 we have

$\displaystyle\langle
A_{0}\circ\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f,h\rangle_{\mathcal{H}}=\langle\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f,P_{0}h\rangle_{L^{2}(G/H)}$
$\displaystyle=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\langle\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f,\textbf{P}_{\mu}^{\xi}h^{\xi}_{\mu}\rangle_{L^{2}(G/H)}\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{32\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{-\mu}{\operatorname{\text{\Capricorn}}(\mu)}\langle\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f,\mathbb{P}_{\mu}h^{d}_{\mu}\rangle_{L^{2}(G/H)}$
$\displaystyle=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\int_{K}\int_{G/H}\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f(x)\frac{K^{\xi}_{-\mu}(x^{-1}k)}{\Gamma(\frac{1-\mu}{2})}\overline{h_{\mu}^{\xi}(k)}d(xH)dk\frac{d\mu}{|a(\mu)|^{2}}$
$\displaystyle\quad+\frac{1}{32\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{-\mu}{\operatorname{\text{\Capricorn}}(\mu)}\langle\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f,\mathbb{P}_{\mu}h^{d}_{\mu}\rangle_{L^{2}(G/H)}.$

Using the coordinates $xH=k_{\theta}\overline{n}_{y}H$ and applying Lemma B.2
in the distributional sense, we find

$\displaystyle\frac{1}{\Gamma(\frac{1-\mu}{2})}\int_{G/H}K_{-\mu}^{\xi}(x^{-1}k)\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f(x)d(xH)$
$\displaystyle=\frac{\Gamma(\frac{2-\lambda}{4})}{\sqrt{\pi}2^{\frac{\lambda}{4}}\Gamma(\frac{1-\mu}{2})\Gamma(\frac{\lambda}{4})}\int_{0}^{\pi}\\!\int_{\mathbb{R}}|\cos\theta|_{\varepsilon}^{-\mu-1}f(gk_{\theta}\overline{n}_{z})\int_{\mathbb{R}}|x|^{\frac{-\lambda-2}{2}}_{\varepsilon}|z-\tan\theta-x|^{\frac{-\mu-1}{2}}_{\xi}\frac{1}{2}dxdzd\theta$
$\displaystyle=p_{\lambda,\mu}^{\xi}\textbf{A}_{\lambda,\mu}^{\xi}f(k).$

An analogous calculation applies to the discrete part after applying that
$\mathbb{A}_{-\mu}=\frac{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}{\sqrt{\pi}2^{\mu}}T_{\mu}^{0}\mathbb{A}_{\mu}$.
In conclusion we find

$\displaystyle\langle
A_{0}\circ\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}f,h\rangle_{\mathcal{H}}=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\langle
p_{\lambda,\mu}^{\xi}\textbf{A}^{\xi}_{\lambda,\mu}f,h^{\xi}_{\mu}\rangle_{L^{2}(K)}\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{16\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma(\frac{-\mu}{2})\operatorname{\text{\Capricorn}}(\mu)}\langle
T_{\mu}^{0}\mathbb{A}_{\lambda,\mu}f,h^{d}_{\mu}\rangle_{L^{2}(K)}$

Hence $A_{0}\circ\operatorname{\text{\Large\Ankh}}_{\lambda}^{0}=A_{\lambda}$
with $A_{\lambda}:\operatorname{Ind}_{H}^{G}(\lambda\otimes
e^{\lambda})\rightarrow\mathcal{H}_{0}$ given by

$\displaystyle\langle
A_{\lambda}f,h\rangle_{\mathcal{H}}=\sum\limits_{\xi=0}^{1}\int_{i\mathbb{R}}\langle
p_{\lambda,\mu}^{\xi}\textbf{A}^{\xi}_{\lambda,\mu}f,h^{\xi}_{\mu}\rangle_{L^{2}(K)}\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{16\pi}\sum\limits_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma(\frac{-\mu}{2})\operatorname{\text{\Capricorn}}(\mu)}\langle
T_{\mu}^{0}\mathbb{A}_{\lambda,\mu}f,h^{d}_{\mu}\rangle_{L^{2}(K)}$

∎

######  Corollary 7.6.

For $\varepsilon=0$ and $\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})$ we have the following Plancherel formula

$\displaystyle\|f\|^{2}$
$\displaystyle=\int_{i\mathbb{R}}\sum_{\xi=0}^{1}\|\textbf{A}^{\xi}_{\lambda,\mu}f\|^{2}\,\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{16\pi}\sum_{\mu\in
1-2\mathbb{N}}\frac{\Gamma(1-\mu)}{\Gamma\big{(}\frac{-\mu}{2}\big{)}\operatorname{\text{\Capricorn}}(\mu)}\|\mathbb{A}_{\lambda,\mu}f\|^{2}.$

For $\varepsilon=1$ and $f\in\operatorname{Ind}_{H}^{G}(\varepsilon\otimes
e^{\lambda})$ we have the following Plancherel formula

$\displaystyle\|f\|^{2}=\int_{i\mathbb{R}}\sum_{\xi=0}^{1}\|\textbf{A}^{\xi}_{\lambda,\mu}f\|^{2}\,\frac{d\mu}{|a(\mu)|^{2}}+\frac{1}{2\pi}\sum_{\mu\in-2\mathbb{N}}\frac{\mu^{2}\Gamma(-\mu)}{\Gamma\big{(}\frac{1-\mu}{2}\big{)}}\|\mathcal{A}_{\lambda,\mu}f\|^{2}.$

with $a(\mu)$ and $\operatorname{\text{\Capricorn}}(\mu)$ given by (4).

###### Proof.

Since $|p_{\lambda,\mu}^{\xi}|=1$ for $\lambda,\mu\in i\mathbb{R}$ the
assertion follows. ∎

##  Appendix A Integral formulas

######  Lemma A.1 (See [1, Proposition 16.8]).

For $\operatorname{Re}{(\alpha+\beta)}>0$ we have

$\int_{\mathbb{R}}\frac{dx}{(x-i)^{\alpha}(x+i)^{\beta}}=\frac{2^{2-\alpha-\beta}i^{\alpha-\beta}\Gamma(\alpha+\beta-1)}{\Gamma(\alpha)\Gamma(\beta)}$

######  Lemma A.2 (See [8, Section 3.631]).

For $\operatorname{Re}{\nu}>0$ we have

$\int_{0}^{\pi}\sin^{\nu-1}(x)\cos(ax)\,dx=\frac{2^{1-\nu}\pi\cos\big{(}\frac{a\pi}{2}\big{)}\Gamma(\nu)}{\Gamma\big{(}\frac{\nu+1-a}{2}\big{)}\Gamma\big{(}\frac{\nu+1+a}{2}\big{)}}.$

######  Lemma A.3 (See [8, Section 3.631]).

For $\operatorname{Re}{\nu}>0$ we have

$\int_{0}^{\pi}\sin^{\nu-1}(x)\sin(ax)\,dx=\frac{2^{1-\nu}\pi\sin\big{(}\frac{a\pi}{2}\big{)}\Gamma(\nu)}{\Gamma\big{(}\frac{\nu+1-a}{2}\big{)}\Gamma\big{(}\frac{\nu+1+a}{2}\big{)}}.$

######  Lemma A.4.

For $\operatorname{Re}{\nu}>0$ we have

$\int_{0}^{\pi}\sin^{\nu-1}(x)e^{iax}\,dx=\frac{2^{1-\nu}\pi
e^{\frac{ai\pi}{2}}\Gamma(\nu)}{\Gamma\big{(}\frac{\nu+1-a}{2}\big{)}\Gamma\big{(}\frac{\nu+1+a}{2}\big{)}}.$

######  Lemma A.5 (See [8, Section 3.251]).

For $\operatorname{Re}{\beta}>-1$ and $\operatorname{Re}{(\alpha+\beta)}<-1$
we have

$\int_{1}^{\infty}x^{\alpha}(x-1)^{\beta}\,dx=B\big{(}-\alpha-\beta-1,\beta+1\big{)}.$

######  Lemma A.6 (See [8, Section 3.194]).

For $\operatorname{Re}{\beta}>-1$ and $\operatorname{Re}{(\alpha+\beta)}<-1$
we have

$\int_{0}^{\infty}x^{\alpha}(x+1)^{\beta}\,dx=B\big{(}-\alpha-\beta-1,\alpha+1\big{)}.$

##  Appendix B The Fourier Transform and Riesz distributions

Define the Fourier transform of $\varphi\in C_{c}(\mathbb{R})$ as

$\mathcal{F}[\varphi](\xi)=\int_{\mathbb{R}}\varphi(x)e^{ix\xi}\,dx$

which makes the inversion formula
$\mathcal{F}\mathcal{F}[\varphi](x)=2\pi\varphi(-x)$. Extend this to
distributions in the usual way.

For $\alpha\in\mathbb{C}$ with $\operatorname{Re}{\alpha}>-1$ and
$\varepsilon\in\\{0,1\\}$ the function

$u_{\alpha}^{\varepsilon}(x)=\frac{1}{2^{\frac{\alpha}{2}}\Gamma\big{(}\frac{\alpha+1+\varepsilon}{2}\big{)}}|x|^{\alpha}_{\varepsilon},$

is locally integrable and can thus be considered as a distribution.

######  Lemma B.1.

The family of distributions $u_{\alpha}^{\varepsilon}$ extends analytically to
a holomorpic family in $\alpha\in\mathbb{C}$. For $\alpha=1-\varepsilon-2n\in
1-\varepsilon-2\mathbb{N}$ we have

$u_{1-\varepsilon-2n}^{\varepsilon}(x)=\frac{(-1)^{n+\varepsilon-1}(n-1)!}{2^{\frac{1-\varepsilon}{2}-n}(2n+\varepsilon-2)!}\delta^{(2n+\varepsilon-2)}(x),$

where $\delta(x)$ is the Dirac $\delta$–function.

######  Lemma B.2.

For $\alpha\in\mathbb{C}$ we have

$\mathcal{F}[u_{\alpha}^{\varepsilon}]=\sqrt{2\pi}i^{\varepsilon}u_{-\alpha-1}^{\varepsilon}.$

Furthermore for $\alpha,\beta\in\mathbb{C}$ with
$\operatorname{Re}{\alpha},\operatorname{Re}{\beta}>-1$ and
$\operatorname{Re}{(\alpha+\beta)}<-1$ we get

$\int_{\mathbb{R}}u_{\alpha}^{\varepsilon}(x)u_{\beta}^{\xi}(y-x)\,dx=(-1)^{\lfloor\frac{\varepsilon+\xi}{2}\rfloor}\sqrt{2\pi}\frac{\Gamma\big{(}\frac{-1-\alpha-\beta+[\varepsilon+\xi]_{2}}{2}\big{)}}{\Gamma\big{(}\frac{-\alpha+\varepsilon}{2}\big{)}\Gamma\big{(}\frac{-\beta+\xi}{2}\big{)}}u_{\alpha+\beta+1}^{\varepsilon+\xi}(y),$

for $y\neq 0$.

###### Proof.

The first assertion can be found in [7, p.170]. For $y\neq 0$ we have

$\int_{\mathbb{R}}|x|^{\alpha}_{\varepsilon}|y-x|^{\beta}_{\xi}\,dx=|y|^{\alpha+\beta+1}_{\xi+\varepsilon}\int_{\mathbb{R}}|x|^{\alpha}_{\varepsilon}|1-x|^{\beta}_{\xi}\,dx,$

by change of variables. Now writing

$\int_{\mathbb{R}}|x|^{\alpha}_{\varepsilon}|1-x|^{\beta}_{\xi}\,dx=(-1)^{\varepsilon}\int_{0}^{\infty}x^{\alpha}(1+x)^{\beta}\,dx+\int_{0}^{1}x^{\alpha}(1-x)^{\beta}\,dx+(-1)^{\xi}\int_{1}^{\infty}x^{\alpha}(x-1)^{\beta}\,dx,$

we can use the integral formula for the Beta-function, apply Lemma A.5, A.6
and then do some Gamma-function yoga. ∎

##  Appendix C Fourier-Jacobi transform

This section is a condensed form of [3, Appendix 1]. For
$\alpha,\beta\in\mathbb{C}$ with $\alpha\notin-\mathbb{N}$ and
$\operatorname{Re}{\beta}>-1$, define the Fourier–Jacobi transform of $f\in
C_{c}^{\infty}(\mathbb{R}_{\geq 0})$ by

$J_{\alpha,\beta}f(\mu)=\int_{0}^{\infty}f(t)\phi^{\alpha,\beta}_{\mu}(t)\cosh^{2\alpha+1}(t)\sinh^{2\beta+1}(t)\,dt,$

where $\phi^{\alpha,\beta}_{\mu}$ are the Jacobi functions given by

$\phi^{\alpha,\beta}_{\mu}(t)=\,_{2}F_{1}\left(\frac{\alpha+\beta+1+\mu}{2},\frac{\alpha-\beta+1+\mu}{2};\alpha+1;-\sinh^{2}(t)\right).$

Then we have the following inversion formula:

$f(t)=\frac{1}{4\pi}\int_{i\mathbb{R}}J_{\alpha,\beta}(\mu)\phi^{\alpha,\beta}_{\mu}(t)\frac{d\mu}{|c_{\alpha,\beta}(\mu)|^{2}}-\sum_{\mu\in
D_{\alpha,\beta}}J_{\alpha,\beta}(\mu)\phi^{\alpha,\beta}_{\mu}(t)\underset{\nu=\mu}{\operatorname{Res}}\big{(}c_{\alpha,\beta}(\nu)c_{\alpha,\beta}(-\nu)\big{)}^{-1},$

where

$c_{\alpha,\beta}(\mu)=\frac{\Gamma(\mu)\Gamma(\alpha+1)}{\Gamma\big{(}\frac{\alpha+|\beta|+1+\mu}{2}\big{)}\Gamma\big{(}\frac{\alpha-|\beta|+1+\mu}{2}\big{)}},$

and

$D_{\alpha,\beta}=\\{x\in\mathbb{R}\,|\,k\in\mathbb{N}_{0},\,x=2k+1+\alpha-|\beta|<0\\}.$

## References

  * [1] Bill Casselman, _Representations of $\textup{SL}_{2}(\mathbb{R})$_, 2020, available here.
  * [2] Jacques Faraut, _Distributions sphériques sur les espaces hyperboliques_ , J. Math. Pures Appl. (9) 58 (1979), no. 4, 369–444.
  * [3] Mogens Flensted-Jensen, _Spherical functions on a simply connected semisimple Lie group_ , Math. Ann. 228 (1977), 65–92.
  * [4] Jan Frahm, _On the direct integral decomposition in branching laws for real reductive groups_ , Journal of Lie Theory 32 (2022), no. 1, 191–196.
  * [5] Jan Frahm and Clemens Weiske, _Branching laws for unitary representations of $\textup{U}(1,n+1)$ in the scalar principal series_, in preparation.
  * [6] Steven A Gaal, _Linear analysis and representation theory_ , vol. 198, Springer Science & Business Media, 2012.
  * [7] I. M. Gel’fand and G.E. Shilov, _Generalized functions_ , Academic Press, New York and London, 1964.
  * [8] I. S. Gradshteyn and I. M. Ryzhik, _Table of integrals, series, and products_ , 5 ed., Academic Press, 1994.
  * [9] Sigurdur Helgason, _Geometric analysis on symmetric spaces_ , second ed., Mathematical Surveys and Monographs, vol. 39, American Mathematical Society, Providence, RI, 2008.
  * [10] Anthony W Knapp, _Representation theory of semisimple groups_ , Princeton university press, 2016.
  * [11] V. F. Molchanov, _Plancherel formula for pseudo-riemannian symmetric spaces of the universal covering group of $\textup{SL}(2,\mathbb{R})$_, Sibirskii Matematicheskii Zhurnal 25 (1984), no. 6, 89–105.
  * [12] R. Penney, _Decomposition of $\textup{C}^{\infty}$ Intertwining Operators for Lie Groups_, Proceedings of the American Mathematical Society 54 (1976), no. 1, 368–370.
  * [13] Wulf Rossmann, _Analysis on real hyperbolic spaces_ , J. Functional Analysis 30 (1978), no. 3, 448–477.
  * [14] Nobukazu Shimeno, _The Plancherel formula for spherical functions with a one-dimensional $K$-type on a simply connected simple Lie group of Hermitian type_, J. Funct. Anal. 121 (1994), no. 2, 330–388.
  * [15] Robert S. Strichartz, _Harmonic analysis on hyperboloids_ , J. Functional Analysis 12 (1973), 341–383.
  * [16] Erik P. van den Ban, _The Plancherel theorem for a reductive symmetric space_ , Lie theory, Progr. Math., vol. 230, Birkhäuser Boston, Boston, MA, 2005, pp. 1–97.
  * [17] Clemens Weiske, _Branching of unitary $\textup{O}(1,n+1)$-representations with non-trivial $(\mathfrak{g},K)$-cohomology_, 2021, preprint, available at arXiv:2106.13562.
  * [18] Li Zhu, _The Plancherel formula for the line bundles on ${\rm SL}(n+1,\mathbb{\mathbb{R}})/S({\rm GL}(1,\mathbb{\mathbb{R}})\times{\rm GL}(n,\mathbb{\mathbb{R}}))$_, Acta Math. Sci. Ser. B (Engl. Ed.) 38 (2018), no. 1, 248–268.