Interpolation and linear prediction of data - three kernel selection criteria

Azzouz Dermoune111Laboratoire Paul Painlevé, USTL-UMR-CNRS 8524. UFR de
Mathématiques, Bât. M2. 59655 Villeneuve d’Ascq Cédex, France. Email:
<EMAIL_ADDRESS>Mohammed Es.Sebaiy222National School of
Applied Sciences-Marrakech, Cadi Ayyad University, Marrakesh, Morocco. E-mail:
<EMAIL_ADDRESS>and Jabrane Moustaaid333National School of Applied
Sciences-Marrakech, Cadi Ayyad University, Marrakesh, Morocco. E-mail:
<EMAIL_ADDRESS>
Lille University and Cadi Ayyad University

###### Abstract

Interpolation and prediction have been useful approaches in modeling data in
many areas of applications. The aim of this paper is the prediction of the
next value of a time series (time series forecasting) using the techniques in
interpolation of the spatial data, for the two approaches kernel interpolation
and kriging. We are interested in finding some sufficient conditions for the
kernels and provide a detailed analyse of the prediction using kernel
interpolation. Finally, we provide a natural idea to select a good kernel
among a given family of kernels using only the data. We illustrate our results
by application to the data set on the mean annual temperature of France and
Morocco recorded for a period of 115 years (1901 to 2015).

Keyword: Kernel interpolation, stochastic interpolation, linear algebra
interpolation, cubic spline interpolation, climate change detection.

##  1 Introduction

Interpolation and prediction have been useful approaches in modelling data in
many areas of applications such as the prediction of the meteorological
variables, surface reconstruction and Interpolation of spatial data [1] among
many more. For more details see [5], [6], [7] and [8].
In this work we extend the results of Scheuerer [1] to the linear prediction
approach of time series. We also cite the work of Dermoune et all [2] where
the parametrizations and the cubic spline were used as a model of prediction
and we extend this results to the kernel interpolation framework.

Interpolation of spatial data is a very general mathematical problem and it’s
precise mathematical formulation as defined in [1] is to reconstruct a
function $f:T\to\mathbb{R}$ with $T$ is is a domain in $\mathbb{R}^{d}$, based
on its values at a finite set of data points
$X=\\{x_{1},\ldots,x_{n}\\}\subset T$, the values $f(x_{1}),\ldots,f(x_{n})$
assumed to be known. But, in our case we are interested in the time series
forecasting problem we have $T=\\{x_{1},\ldots,x_{n},x_{n+1}\\}$ represent the
time and the time series is $f(x_{1}),\ldots,f(x_{n})$ with the unknown value
is $f(x_{n+1})$. In other words, we want to predict effectively the value
$f(x_{n+1})$ using the known values $f(x_{1})$, $\ldots$, $f(x_{n})$. From [1]
we have that both approaches kernel interpolation and kriging have the same
approximant for the interpolation of spatial data problem, even with the
different model assumption, a general overview in both approaches can be fond
in [9].

##  2 Linear prediction and kernel interpolation

Let $\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}$ be the Hilbert space of real
functions on $\\{x_{1},\ldots,x_{n+1}\\}$ with inner product $(.,.)$ and norm
$N(.)$. The dual of $\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}$ is spanned by
the point evaluation linear forms $\delta_{x}:f\to f(x)$,
$x\in\\{x_{1},\dots,x_{n+1}\\}$, that is

$\displaystyle(\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}})^{*}=$$\displaystyle(\delta_{x_{1}},\ldots,\delta_{x_{n+1}}).$

Moreover, the dual norm $N^{*}$ is defined by

$\displaystyle(N^{*}(\mu))^{2}=\sup\\{|\mu(f)|^{2}:\quad N(f)\leq 1\\},$

for all $\mu\in(\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}})^{*}$.
Now, for any function $f\in\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}$ and any
sequence of real numbers $(w_{1},\ldots,w_{n})$, we define the linear
prediction of $f(x_{n+1})$

$\displaystyle\hat{f}(x_{n+1})=\sum_{i=1}^{n}w_{i}f(x_{i}),$

with the error

$\displaystyle\mathcal{E}rr_{n}(f):=|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|,$

and the worst error in the unit ball w.r.t. the norm $N(.)$

$\displaystyle\mathcal{W}err(f):=\sup\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}:\quad
N(f)\leq
1\\}=(N^{*}(\delta_{x_{n+1}}-\sum_{i=1}^{n}w_{i}\delta_{x_{i}}))^{2}.$ (1)

In the rest oh this paper, we endow the vector space
$\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}$ with the scalar inner product

$\displaystyle(f,f)=(f,f)_{\mathbf{K}^{-1}}$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{n+1}\sum_{j=1}^{n+1}f(x_{i})f(x_{j})k^{(-1)}(x_{i},x_{j})$
$\displaystyle=$
$\displaystyle\boldsymbol{f}^{\top}\mathbf{K}^{-1}\boldsymbol{f},$

with $\boldsymbol{f}=(f(x_{1}),\ldots,f(x_{n+1}))^{\top}$ and
$\mathbf{K}=[k(x_{i},x_{j}):\quad i,j=1,\ldots,n+1]$ is a fixed
$(n+1)\times(n+1)$ symmetric positive definite matrix, with
$k^{(-1)}(x_{i},x_{j})$ denotes the $(i,j)$ entry of $\mathbf{K}^{-1}$. The
norm defined by $\mathbf{K}$ is given by
$N(f)=\|\mathbf{K}^{-1/2}\boldsymbol{f}\|$, with $\|\cdot\|$ denotes the
Euclidean norm.

###  2.1 Min-max prediction and kernel interpolation

######  Definition 2.1 (Min-max prediction).

A linear prediction $f^{*}(x_{n+1})$ of $f(x_{n+1})$ is called min-max if

$\displaystyle f^{*}(x_{n+1})=\sum_{i=1}^{n}w_{i}^{*}f(x_{i}),$ (2)

where $\left(w_{1}^{*},...,w_{n}^{*}\right)$ are given by the minimization of
the $\mathcal{W}err(f)$ 1.

The following result give us the optimal weights associate to the min-max
prediction w.r.t. to the norm $\|\mathbf{K}^{-1/2}\cdot\|$.

######  Proposition 2.2.

The the worst error in the unit ball, $\mathcal{W}err(f)$, w.r.t. to the norm
$\|\mathbf{K}^{-1/2}\cdot\|$ is equals

$\displaystyle\mathcal{W}err(f)=\|\delta_{x_{n+1}}-\sum_{i=1}^{n}w_{i}\delta_{x_{i}})\|_{\mathbf{K}^{1/2}}^{2}$
(3)

where $\|\cdot\|_{\mathbf{K}^{1/2}}$ denotes the dual norm defined by the dual
scalar inner product

$\displaystyle(\delta_{x_{i}},\delta_{x_{j}})_{\mathbf{K}}=k(x_{i},x_{j}),\quad
i,j=1,\ldots,n+1.$

###### Proof.

From the general theory of reproducing kernel Hilbert spaces,see [9, 1], we
have

$\displaystyle\sup_{\|\mathbf{K}^{-1/2}\boldsymbol{f}\|\leq
1}\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}\\}$
$\displaystyle=\sup_{\|\mathbf{K}^{-1/2}\boldsymbol{f}\|\leq
1}\\{[\mathbf{K}^{-1/2}\boldsymbol{f}]^{\top}[\mathbf{K}^{1/2}(-w_{1},\ldots,-w_{n},1)^{\top}(-w_{1},\ldots,-w_{n},1)\mathbf{K}^{1/2}][\mathbf{K}^{-1/2}\boldsymbol{f}]\\}$
$\displaystyle=\mbox{the largest eigenvalue of
$[\mathbf{K}^{1/2}(-w_{1},\ldots,-w_{n},1)^{\top}(-w_{1},\ldots,-w_{n},1)\mathbf{K}^{1/2}]$}$
$\displaystyle=\|\mathbf{K}^{1/2}(-w_{1},\ldots,-w_{n},1)^{\top}\|^{2}=(-w_{1},\ldots,-w_{n},1)\mathbf{K}(-w_{1},\ldots,-w_{n},1)^{\top}$
$\displaystyle=\|\delta_{x_{n+1}}-\sum_{i=1}^{n}w_{i}\delta_{x_{i}})\|_{\mathbf{K}^{1/2}}^{2}.$

∎

######  Corollary 2.3.

The optimal weights of the min-max linear prediction of $f(x_{n+1})$ are given
by

$\displaystyle\boldsymbol{w}^{*}=(w_{1}^{*},\ldots,w_{n}^{*})=[k(x_{n+1},x_{1}),\ldots,k(x_{n+1},x_{n})][k(x_{i},x_{j}):\quad
i,j=1,\ldots,n]^{-1}.$ (4)

###### Proof.

The optimal weights are given by the minimization

$\displaystyle\arg\min\\{\|\delta_{x_{n+1}}-\sum_{i=1}^{n}w_{i}\delta_{x_{i}}\|^{2}_{\mathbf{K}^{1/2}}:\quad
w_{1},\ldots,w_{n}\in\mathbb{R}\\},$ (5)

which is the solution of the system

$\displaystyle\sum_{j=1}^{n}w_{j}k(x_{i},x_{j})=k(x_{n+1},x_{i}),\quad
i=1,\ldots,n,$ (6)

it follows easily that $\boldsymbol{w}^{*}$ is given by 4. ∎

######  Remarks 2.4.

  * 1)

The worst case linear prediction error in the ball with the radius $r>0$
w.r.t. to the norm $\|\mathbf{K}^{-1/2}\cdot\|$ is equal to

$\displaystyle\sup_{\|\mathbf{K}^{-1/2}\boldsymbol{f}\|\leq
r}\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}\\}$
$\displaystyle=r^{2}(-w_{1},\ldots,-w_{n},1)\mathbf{K}(-w_{1},\ldots,-w_{n},1)^{\top},$

as a result the optimal weights (4) do not depend on the radius of the ball.

  * 2)

The prediction using the spline interpolating w.r.t. the norm
$\|\mathbf{K}^{-1/2}\cdot\|$ (see, e.g., [9]) defined by the minimizer :

$\displaystyle S(f)=\arg\min\\{\|\mathbf{K}^{-1/2}\boldsymbol{f}\|:\quad
f(x_{1}),\ldots,f(x_{n})\,\mbox{are fixed}\\},$

coincide with the prediction (2).

  * 3)

The min-max prediction (2) is equal to

$\displaystyle f^{*}(x_{n+1})=\sum_{j=1}^{n}\alpha_{j}^{*}k(x_{n+1},x_{j}),$
(7)

where $\alpha_{j}^{*},j=1,...,n$ is the solution of the system

$\displaystyle\sum_{j=1}^{n}\alpha_{j}^{*}\boldsymbol{k}(x_{i},x_{j})=f(x_{i}),\quad
i=1,\ldots,n,$ (8)

Now, we turn to the interpolation of the function $f$ at the set
$\\{x_{1},\ldots,x_{n}\\}$ using
$span(\boldsymbol{k}_{1},\ldots,\boldsymbol{k}_{n})$ where
$\boldsymbol{k}_{j}$ denotes the $j$-th column of the matrices $\mathbf{K}$.
Then the interpolation of the function $f$ equals

$\displaystyle I(f)=\sum_{j=1}^{n}\alpha_{j}^{*}\boldsymbol{k}_{j}$

with the weights $\alpha^{*}$ are given by (8). The following Proposition
gives the error of interpolation.

######  Proposition 2.5 (Interpolation error ).

The error of interpolation, $\mathcal{I}\mathcal{E}rr$, is given by

$\displaystyle\mathcal{I}\mathcal{E}rr(f):=f(x_{n+1})-f^{*}(x_{n+1})=[\boldsymbol{k}_{n+1}^{(-1)}\boldsymbol{f}][k(x_{n+1},x_{n+1})-\sum_{i=1}^{n}w_{i}^{*}k(x_{i},x_{n+1})].$
(9)

###### Proof.

First, observe that we can write the coordinates of $\boldsymbol{f}$ in the
basis $\mathbf{K}$ as

$\displaystyle\boldsymbol{f}=\sum_{j=1}^{n+1}[\boldsymbol{k}_{j}^{(-1)}\boldsymbol{f}]\boldsymbol{k}_{j}.$

with $\boldsymbol{k}_{j}^{(-1)}$ denotes the $j$-th row of $\mathbf{K}^{-1}$.
Therefore

$\displaystyle I(f)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{n+1}[\boldsymbol{k}_{j}^{(-1)}\boldsymbol{f}]I(\boldsymbol{k}_{j})$
$\displaystyle=$
$\displaystyle\sum_{j=1}^{n}[\boldsymbol{k}_{j}^{(-1)}\boldsymbol{f}]\boldsymbol{k}_{j}+[\boldsymbol{k}_{n+1}^{(-1)}\boldsymbol{f}]I(\boldsymbol{k}_{n+1}),$

because the interpolation of $\boldsymbol{k}_{j}$ is exact for $j=1,\ldots,n$.
Thus,

$\displaystyle f-I(f)$ $\displaystyle=$
$\displaystyle[\mathbf{K}_{n+1}^{-1}\boldsymbol{f}](\boldsymbol{k}_{n+1}-I(\boldsymbol{k}_{n+1}))$
$\displaystyle=$
$\displaystyle[\boldsymbol{k}_{n+1}^{(-1)}\boldsymbol{f}](0,\ldots,0,[k(x_{n+1},x_{n+1})-\sum_{i=1}^{n}w_{i}^{*}k(x_{i},x_{n+1})])^{\top},$

which completes the proof. ∎

###  2.2 Min-max linear prediction with constraint

In this section, we consider the optimization (5) under the constraint

$\displaystyle\sum_{i=1}^{n}w_{i}\boldsymbol{p}_{k}(x_{i})=\boldsymbol{p}_{k}(x_{n+1}),\quad
k=1,\ldots,q,$ (10)

where
$\boldsymbol{p}_{1},\ldots,\boldsymbol{p}_{q}\in\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}$
are given.
Solve the minimization (5) under the constraint (10) is equivalent to solve
the system

$\displaystyle\begin{cases}\sum_{j=1}^{n}k(x_{i},x_{j})w_{j}+\sum_{k=1}^{q}\lambda_{k}\boldsymbol{p}_{k}(x_{i})=k(x_{i},x_{n+1}),\quad
i=1,\ldots,n,\\\
\sum_{j=1}^{n}w_{j}\boldsymbol{p}_{k}(x_{j})=\boldsymbol{p}_{k}(x_{n+1}),\quad
k=1,\ldots,q,\end{cases}$ (11)

where $\lambda_{1}$, $\ldots$, $\lambda_{q}$ are the Lagrange multiplier. The
solution is unique if the homogeneous system

$\displaystyle\sum_{j=1}^{n}k(x_{i},x_{j})w_{j}+\sum_{k=1}^{q}\lambda_{k}p_{k}(x_{i})=0,\quad
i=1,\ldots,n,$ $\displaystyle\sum_{j=1}^{n}w_{j}p_{k}(x_{j})=0,\quad
k=1,\ldots,q,$

has a unique solution $w_{1}=\ldots=w_{n}=0$,
$\lambda_{1}=\ldots=\lambda_{q}=0$. This is equivalent to say that the columns
$(p_{k}(x_{1}),\ldots,p_{k}(x_{n}))^{\top}$, $k=1,\ldots,q$, are linearly
independent and that $\mathbf{K}$ is conditionally positive w.r.t.
$\boldsymbol{p}_{1}$, $\ldots$, $\boldsymbol{p}_{q}$, i.e. the system

$\displaystyle\sum_{j=1}^{n}\sum_{j=1}^{n}k(x_{i},x_{j})w_{j}w_{i}=0,\quad\sum_{i=1}^{n}w_{i}p_{k}(x_{i})=0,\quad
k=1,\ldots,q,$

has a unique solution $w_{1}=\ldots=w_{n}=0$. Observe that this is true if
$\mathbf{K}$ is definite positive, but it is not necessary.
Let $w_{1}^{*}$, $\ldots$, $w_{n}^{*}$, $\lambda_{1}^{*}$, $\ldots$,
$\lambda_{q}^{*}$ be the solution of the system (11). Then the optimal
prediction under the constraint (10) is

$\displaystyle f^{*}(x_{n+1})=\sum_{i=1}^{n}w_{i}^{*}f(x_{i}).$ (12)

### Constraint’s parametrization

Now, let $\boldsymbol{z}^{(1)}=(z_{1}^{(1)},\ldots,z_{n}^{(1)})^{\top}$ be a
particular solution of the system (10) and
$\boldsymbol{z}_{1}=(z_{11},\ldots,z_{n1})^{\top}$, $\ldots$,
$\boldsymbol{z}_{n-q}=(z_{1n-q},\ldots,z_{nn-q})^{\top}$, $n-q$ independent
solutions of the corresponding homogeneous system. Then the general solution
of the system (10) has the form

$\displaystyle\boldsymbol{w}=\boldsymbol{z}^{(1)}+\sum_{l=1}^{n-q}\tilde{w}_{l}\boldsymbol{z}_{l}.$

Let us consider the basis

$\displaystyle\mathbf{B}=[\boldsymbol{p}_{1},\ldots,\boldsymbol{p}_{q},\sum_{i=1}^{n}z_{i1}\boldsymbol{k}_{i},\ldots,\sum_{i=1}^{n}z_{in-q}\boldsymbol{k}_{i},\boldsymbol{b}_{n+1}]$
(13) $\displaystyle=:[\boldsymbol{b}_{1},\ldots,\boldsymbol{b}_{n+1}]$

such that the expansion of $\boldsymbol{f}$ in the basis $\mathbf{B}$ is given
by

$\displaystyle\boldsymbol{f}=\sum_{l=1}^{q}\theta_{l}\boldsymbol{f}\boldsymbol{p}_{l}+\sum_{l=1}^{n-q}\theta_{q+l}\boldsymbol{f}\boldsymbol{b}_{q+l}+\theta_{n+1}\boldsymbol{f}\boldsymbol{b}_{n+1},$

with $\theta_{n+1}=(-(\boldsymbol{z}^{(1)})^{\top},1)$.

The unknows are the rows $\theta_{1}$, $\ldots$, $\theta_{n}$, and the column
$\boldsymbol{b}_{n+1}$. They are solution of the system

$\displaystyle\delta(i=j)=\sum_{l=1}^{q}p_{l}(x_{i})\theta_{lj}+\sum_{l=1}^{n-q}b_{q+l}(x_{i})\theta_{(q+l)j}+b_{n+1}(x_{i})\theta_{(n+1)j}$
$\displaystyle i,j=1,\ldots,n+1.$ (14)

The interpolation of any function $g$ at the set $\\{x_{1},\ldots,x_{n}\\}$
using $span(\boldsymbol{b}_{1},\ldots,\boldsymbol{b}_{n})$ is given by the map

$\displaystyle I(g)=\sum_{j=1}^{n}\beta_{j}\boldsymbol{b}_{j},$

with $\beta$ is the solution of the system

$\displaystyle\sum_{j=1}^{n}\beta_{j}\boldsymbol{b}_{j}(x_{i})=g(x_{i}),\quad
i=1,\ldots,n.$

Using similar arguments as in proposition 2.5, we can deduce the following
result.

######  Proposition 2.6.

The value

$\displaystyle\sum_{j=1}^{n}z_{j}^{(1)}k(x_{j},x_{n+1})+I(f-\sum_{j=1}^{n}z_{j}^{(1)}\boldsymbol{k}_{j})(x_{n+1})$

coincides with $f^{*}(x_{n+1})$ given by (12). In addition, the error is equal
to

$\displaystyle
f(x_{n+1})-f^{*}(x_{n+1})=f(x_{n+1})-\sum_{j=1}^{n}z_{j}^{(1)}k(x_{j},x_{n+1})-I(f-\sum_{j=1}^{n}z_{j}^{(1)}\boldsymbol{k}_{j})(x_{n+1})$
$\displaystyle=[\boldsymbol{b}_{n+1}^{(-1)}(f-\sum_{j=1}^{n}z_{j}^{(1)}\boldsymbol{k}_{j})][b_{n+1}(x_{n+1})-I(\boldsymbol{b}_{n+1})(x_{n+1})].$

If $\mathbf{K}$ is invertible and $\boldsymbol{p}_{l}=\boldsymbol{k}_{l}$ with
$l=1,\ldots,n$, then $\theta_{l}=\boldsymbol{k}_{l}^{(-1)}$ for
$l=1,\ldots,n$, and the basis (14) is given by
$\boldsymbol{b}_{l}=\boldsymbol{k}_{l}$ with $l=1,\ldots,n$, and
$\boldsymbol{b}_{n+1}=\frac{\boldsymbol{k}_{n+1}}{k(x_{n+1},x_{n+1})-I(\boldsymbol{k}_{n+1})(x_{n+1})}$.

### Constraint’s effect on the kernel

From the notations above the general solution of the system (10) has the form

$\displaystyle\boldsymbol{w}=\boldsymbol{z}^{(1)}+\sum_{l=1}^{n-q}\tilde{w}_{l}\boldsymbol{z}_{l}.$

As a consequence the quadratic form

$\displaystyle\|\delta_{x_{n+1}}-\sum_{i=1}^{n}w_{i}\delta_{x_{i}}\|_{\mathbf{K}}^{2}=\|\mu_{n+1-q}-\sum_{l=1}^{n-q}\tilde{w}_{l}\mu_{l}\|_{\tilde{\mathbf{K}}}^{2},$

with $\mu_{1}=\sum_{i=1}^{n}z_{i1}\delta_{x_{i}}$, $\ldots$,
$\mu_{n-q}=\sum_{i=1}^{n}z_{in-q}\delta_{x_{i}}$,
$\mu_{n+1-q}=\delta_{x_{n+1}}-\sum_{i=1}^{n}z_{i}^{(1)}\delta_{x_{i}}$, and
the entries of the $(n+1-q)\times(n+1-q)$ kernel $\tilde{\mathbf{K}}$ are
given by

$\displaystyle\tilde{k}(l_{1},l_{2})=(\mu_{l_{1}},\mu_{l_{2}})_{\mathbf{K}},\quad
l_{1},l_{2}=1,\ldots,n+1-q.$

Observe that $\tilde{\mathbf{K}}$ is positive definite if and only if the
columns $(p_{k}(x_{1}),\ldots,p_{k}(x_{n}))^{\top}$, $k=1,\ldots,q$, are
linearly independent and $\mathbf{K}$ is conditionally positive w.r.t.
$\boldsymbol{p}_{1}$, $\ldots$, $\boldsymbol{p}_{q}$.
It follows that

$\displaystyle\sup\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}:\quad\tilde{f}^{\top}\tilde{\mathbf{K}}^{-1}\tilde{f}\leq
1\\}$
$\displaystyle=\|\mu_{n+1-q}-\sum_{l=1}^{n-q}\tilde{w}_{l}\mu_{l}\|_{\tilde{\mathbf{K}}}^{2},$

where
$\tilde{f}=(\tilde{f}(1),\ldots,\tilde{f}(n+1-q))^{\top}\in\mathbb{R}^{n+1-q}$
are defined by

$\displaystyle\tilde{f}(1)=\sum_{i=1}^{n}z_{i1}f(x_{i}),\ldots,\tilde{f}(n-q)=\sum_{i=1}^{n}z_{in-q}f(x_{i}),$
$\displaystyle\tilde{f}(n+1-q)=f(x_{n+1})-\sum_{i=1}^{n}z_{i}^{(1)}f(x_{i}).$

The map
$f\in\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}\to\tilde{f}^{\top}\tilde{\mathbf{K}}^{-1}\tilde{f}$
is a semi kernel having the null space spanned by $\boldsymbol{p}_{1}$,
$\ldots$, $\boldsymbol{p}_{q}$.
That being the case, the optimal weights $\tilde{w}^{*}$ are given by

$\displaystyle\tilde{w}^{*}=\arg\min\\{\|\mu_{n+1-q}-\sum_{l=1}^{n-q}\tilde{w}_{l}\mu_{l}\|_{\tilde{\mathbf{K}}}^{2}:\quad\tilde{w}_{1},\ldots,\tilde{w}_{n-q}\in\mathbb{R}\\},$

and then predict $f(x_{n+1})$ is equal to

$\displaystyle\sum_{i=1}^{n}z_{i}^{(1)}f(x_{i})+\sum_{l=1}^{n-q}\tilde{w}_{l}^{*}(\sum_{i=1}^{n}z_{il}f(x_{i})).$

The latter predictor coincides with (12). Moreover, the spline

$\displaystyle
S(\tilde{f})=\arg\min_{\tilde{f}(n+1-q)}\\{\tilde{f}^{\top}\tilde{K}^{-1}\tilde{f}:\quad\tilde{f}(1),\ldots,\tilde{f}(n-q)\,\mbox{are
fixed}\\}$

is such that

$\displaystyle
S(\tilde{f})(n+1-q)=f^{*}(x_{n+1})-\sum_{i=1}^{n}z_{i}^{(1)}f(x_{i})$

with $f^{*}(x_{n+1})$ is the optimal prediction under the constraint (12).
From the expansion of $\boldsymbol{f}$ in the basis
$\mathbf{B}=[\boldsymbol{b}_{1},\ldots,\boldsymbol{b}_{n+1}]$ (13), we can
conclude the following result.

######  Proposition 2.7.

If the weights $\boldsymbol{w}$ satisfy the constraint (10), then

$\displaystyle|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}=|\sum_{l=1}^{n+1-q}\tilde{f}_{l}\\{b_{q+l}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{q+l}(x_{i})\\}|^{2}.$

It follows that

$\displaystyle\sup\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}:\quad\tilde{f}^{\top}\tilde{\mathbf{K}}^{-1}\tilde{f}\leq
1\\}$
$\displaystyle=(b_{q+1}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{q+1}(x_{i}),\ldots,b_{n+1}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{n+1-q}(x_{i}))\tilde{\mathbf{K}}^{-1}$
$\displaystyle(b_{1+q}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{1+q}(x_{i}),\ldots,b_{n+1}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{n+1}(x_{i}))^{\top}$
$\displaystyle=(-w_{1},\ldots,-w_{n},1)\mathbf{R}\tilde{K}\mathbf{R}^{\top}(-w_{1},\ldots,-w_{n},1)^{\top}$
$\displaystyle=(-\tilde{w}_{1},\ldots,-\tilde{w}_{n-q},1)\tilde{K}(-\tilde{w}_{1},\ldots,-\tilde{w}_{n-q},1)^{\top}$
$\displaystyle=\|\mu_{n+1-q}-\sum_{l=1}^{n-q}\tilde{w}_{l}\mu_{l}\|_{\tilde{\mathbf{K}}}^{2},$

with the $(n+1)\times(n+1-q)$ matrix

$\displaystyle\mathbf{R}=[\boldsymbol{b}_{q+1},\ldots,\boldsymbol{b}_{n+1}].$

###  2.3 Semi-kernel and constraint

Now, conversely we consider a semi-kernel $Q$ on
$\mathbb{R}^{\\{x_{1},\ldots,x_{n+1}\\}}$ with the null space spanned by $q$
functions $p_{1}$, $\ldots$, $p_{q}$ and let

$\displaystyle S(f)=\arg\min\\{Q(f,f):\quad f(x_{1}),\ldots,f(x_{n})\mbox{ are
fixed}\\}$

be the spline defined by the semi-norm $Q$, and

$S(f)(x_{n+1})=\sum_{i=1}^{n}w_{i}^{*}f(x_{i}).$

We consider a basis
$\mathbf{B}=[\boldsymbol{b}_{1},\ldots,\boldsymbol{b}_{n+1}]$ such that
$\boldsymbol{b}_{l}=\boldsymbol{p}_{l}$ with $l=1,\ldots,q$ and let
$(\theta_{1},\ldots,\theta_{q},u_{1},$ $\ldots,u_{n+1-q})$ be the coordinates
of $\boldsymbol{f}$, i.e.

$\displaystyle\boldsymbol{f}=\sum_{l=1}^{q}\theta_{l}\boldsymbol{p}_{l}+\sum_{l=1}^{n+1-q}u_{l}\boldsymbol{b}_{q+l}.$

It follows that

$\displaystyle
Q(f,f)=\sum_{l_{1}=1}^{n+1-q}\sum_{l_{2}=1}^{n+1-q}u_{l_{1}}u_{l_{2}}Q(b_{q+l_{1}},b_{q+l_{2}})=\|Q^{1/2}\boldsymbol{u}\|^{2},$

and the kernel $Q=:[Q_{l_{1},l_{2}}:\quad l_{1},l_{2}=1,\ldots,n+1-q]$ is
invertible. If the weights $\boldsymbol{w}$ satisfy the constraint (10), then

$\displaystyle|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}=|\sum_{l=1}^{n+1-q}u_{l}\\{b_{l}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{l}(x_{i})\\}|^{2},$

therefore,

$\displaystyle\sup\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}:\quad
Q(f,f)\leq 1\\}$
$\displaystyle=\sup\\{|f(x_{n+1})-\sum_{i=1}^{n}w_{i}f(x_{i})|^{2}:\quad\boldsymbol{u}^{\top}Q\boldsymbol{u}\leq
1\\}$
$\displaystyle=(b_{q+1}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{q+1}(x_{i}),\ldots,b_{n+1}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{n+1-q}(x_{i}))Q^{-1}$
$\displaystyle(b_{1+q}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{1+q}(x_{i}),\ldots,b_{n+1}(x_{n+1})-\sum_{i=1}^{n}w_{i}b_{n+1}(x_{i}))^{\top}$
$\displaystyle=(-w_{1},\ldots,-w_{n},1)\mathbf{R}Q^{-1}\mathbf{R}^{\top}(-w_{1},\ldots,-w_{n},1)^{\top},$

where the $(n+1)\times(n+1-q)$ matrix

$\displaystyle\mathbf{R}=[\boldsymbol{b}_{q+1},\ldots,\boldsymbol{b}_{n+1}].$

##  3 Stochastic approach

The statistical counterpart to the kernel interpolation is known as kriging
(see e.g. [1]). It is based on the modeling assumption that
$(f(x_{1}),\ldots,f(x_{n}),f(x_{n+1}))$ is a realization of random vector
$Y_{x_{1}}$, $\ldots$, $Y_{x_{n+1}}$ over the same probability space
$(\Omega,\mathcal{F},\mathbf{P})$. To predict $Y_{x_{n+1}}$ known $Y_{x_{1}}$,
$\ldots$, $Y_{x_{n}}$ we need the mean and the covariance matrix of the random
vector $(Y_{x_{1}},\ldots,Y_{x_{n+1}})$.
We assume that the mean $(m(x_{1}),\ldots,m(x_{n+1}))$ (also called the trend)
and the covariance function

$\displaystyle k(x_{i},x_{j})=cov(Y_{x_{i}},Y_{x_{j}})$

of the random vector $(Y_{x_{1}},\ldots,Y_{x_{n+1}})$ exist.
If $Y_{x_{1}}$, $\ldots$, $Y_{x_{n}}$ are assumed to be known, then the best
linear unbiased predictor (BLUP) of $Y_{x_{n+1}}$ is given by

$\displaystyle\hat{Y}_{x_{n+1}}=\sum_{i=1}^{n}w_{i}^{*}Y_{x_{i}},$

where the weights $w_{i}^{*}$ are the solution of the following optimization
problem

$\displaystyle\min\\{var(Y_{x_{n+1}}-\sum_{i=1}^{n}w_{i}Y_{x_{i}}):\quad
w_{1},\ldots,w_{n}\in\mathbb{R},\sum_{i=1}^{n}w_{i}m(x_{i})=m(x_{n+1})\\}.$
(15)

If the mean function $m$ is modeled as

$\displaystyle m(x_{i})=\sum_{k=1}^{q}\beta_{k}p_{k}(x_{i}):\quad
i=1,\ldots,n+1,$

and if we consider the weights such that

$\displaystyle\sum_{i=1}^{n}w_{i}p_{l}(x_{i})=p_{l}(x_{n+1}),\quad
l=1,\ldots,q,$

then the optimal predictor

$\displaystyle\hat{f}(x_{n+1})=\sum_{i=1}^{n}w_{i}^{*}f(x_{i})$

of $f(x_{n+1})$ in stochastic sense coincides with the interpolation (12).

##  4 Three kernel selection criteria

Kernel interpolation and prediction approaches are based on the knowledge of a
symmetric positive definite matrix $\mathbf{K}$ and the trend
$\boldsymbol{p}_{1}$, $\ldots$, $\boldsymbol{p}_{q}$. To apply kernel
interpolation it amounts to the assumption that one knows the degree of
smoothness of the function $f$. In the context of partial differential
equations, the function $f$ belongs to some Sobolev space. In stochastic
approach the covariance matrix and the trend are chosen using the maximum
likelihood method or the Bayesian method.
Here we propose three natural criteria to compare two kernels
$\mathbf{K}^{(1)}$ and $\mathbf{K}^{(2)}$. Known $f(x_{1})$, $\ldots$,
$f(x_{r})$, we predict $f(x_{r+1})$ using the kernel $[k^{(l)}(i,j):\quad
i,j=1,\ldots,r]$, and we obtain the predictor $\hat{f}^{(l)}(x_{r+1})$, with
$l=1,2$, and $r=2$, $\ldots$, $n-1$. We propose the following three criteria
to measure the performance of the Kernel $\mathbf{K}^{(l)}$:

1)
$MSPE(l)=:\frac{\sum_{j=1}^{n-1}|f(x_{j+1})-\hat{f}^{(l)}(x_{j+1})|^{2}}{n-1}$.
We say that $\mathbf{K}^{(1)}$ is better than $\mathbf{K}^{(2)}$ w.r.t. the
MSPE criterion if

$\displaystyle MSPE(1)<MSPE(2).$

2) $MAXPE(l)=:\max\\{|f(x_{j+1})-\hat{f}^{(l)}(x_{j+1})|:\quad
j=1,\ldots,n-1\\}$. We say that $\mathbf{K}^{(1)}$ is better than
$\mathbf{K}^{(2)}$ w.r.t. the MAXPE criterion if

$\displaystyle MAXPE(1)<MAXPE(2).$

3) We say that $\mathbf{K}^{(1)}$ is statistically better than
$\mathbf{K}^{(2)}$ if

$\displaystyle\frac{\sum_{j=1}^{n-1}{\bf
1}_{[|f(x_{j+1})-\hat{f}^{(1)}(x_{j+1})|<|f(x_{j+1})-\hat{f}^{2)}(x_{j+1})|]}}{n-1}>1/2.$

These criteria were also used in [DEM].

##  5 Application

In the climate change problem we are interested in the mean temperature $f(t)$
at the time $t$. The data are the years taken into account
$t_{1}<\ldots<t_{n+1}$ and the mean temperature $f(t_{1})$, $\ldots$,
$f(t_{n})$, and we are interested in the prediction of $f(t_{n+1})$. We recall
that

$\displaystyle\arg\min\\{\int_{t_{1}}^{t_{n+1}}|g^{\prime\prime}(t)|^{2}dt:\quad
g(t_{1})=f(t_{1}),\ldots,g(t_{n+1})=f(t_{n+1})\quad\mbox{are fixed}\\}$

is the natural $C^{2}$ cubic spline $s$ which interpolates the points
$(t_{i},f(t_{i})),i=1,\ldots,n+1$. See [10, 11]. We assume that $f(t_{1})$,
$\ldots$, $f(t_{n+1})$ are the values of a natural $C^{2}$ cubic spline. We
are going to predict $f(t_{n+1})$ using three kernels, and we need some
notations.

###  5.1 Kernel and semikernels using cubic splines

Let $S=S_{3}(t_{1},\ldots,t_{n+1})$ be the set of $C^{2}$ cubic splines having
the knots $t_{1}<\cdots<t_{n+1}$. Note that every element $s\in S$ is a
$C^{2}$ map on $[t_{1},t_{n+1}]$ and is a polynomial of degree three on each
interval $[t_{i},t_{i+1})$ for $i=1$,…, $n$.

More precisely, let

$\displaystyle p_{1}=s(t_{1}),\ldots,p_{n+1}=s(t_{n+1}),\quad
q_{1}=s^{\prime}(t_{1}),\ldots,q_{n+1}=s^{\prime}(t_{n+1}),$ $\displaystyle
u_{1}=s^{\prime\prime}(t_{1}),\ldots,u_{n+1}=s^{\prime\prime}(t_{n+1}),\quad
v_{1}=s^{\prime\prime\prime}(t_{1}+),\ldots,v_{n}=s^{\prime\prime\prime}(t_{n}+)$

be respectively the values of $s$ and its derivatives up to order three at the
knots. We have for every $i=1,\ldots,n$,

$\displaystyle
s(t)=p_{i}+q_{i}(t-t_{i})+(t-t_{i})^{2}u_{i}/2+(t-t_{i})^{3}v_{i}/6,\quad
t\in[t_{i},t_{i+1}).$

The following constraint for $h_{i}=t_{i+1}-t_{i}$, $i=1,\ldots,n$ ensures the
hypothesis that $s$ is $C^{2}$:

$\displaystyle
p_{i}+q_{i}h_{i}+u_{i}h_{i}^{2}/2+v_{i}h_{i}^{3}/6=p_{i+1},\quad$ (16)
$\displaystyle q_{i}+u_{i}h_{i}+v_{i}h_{i}^{2}/2=q_{i+1},\quad$ (17)
$\displaystyle v_{i}=s^{(3)}(t_{i})=(u_{i+1}-u_{i})/h_{i}.\quad$ (18)

It is well known (see [deBoor]) that $S$ has the dimension $n+3$, and the set
of natural spline $S_{nat}$ has the dimension $n+1$. Hence an element $s\in S$
(respectively $s\in S_{nat}$) is completely defined by $n+3$ (respectively
$n+1$ parameters) independent parameters.

Now we need to parametrize the set $S$ in order to define properly an element
$s\in S$. A parametrization of $S$ is a one-to-one linear map

$\displaystyle\Theta:s\in S\to\theta\in\mathbb{R}^{n+3}.$

Defining a parametrization $\Theta$ is equivalent to the existence of the
basis $\mathbf{B}=(\boldsymbol{b}_{1},\ldots,\boldsymbol{b}_{n+3})$ of $S$
such that, for all $s\in S$,

$\displaystyle
s=\sum_{i=1}^{n+3}\theta_{i}\boldsymbol{b}_{i}=\mathbf{B}\theta.$

The parametrization $\Theta_{002}=(p_{1},p_{2},u_{1},\ldots,u_{n+1})$ defines
the basis
$\mathbf{B}_{002}=(\boldsymbol{b}_{1}^{002},\ldots,\boldsymbol{b}_{n+3}^{002})$.
The subscript notation 002 is justified by the fact that

$\displaystyle p_{1}=s(t_{1})=s^{(0)}(t_{1}),p_{2}=s(t_{2})=s^{(0)}(t_{2}),$
$\displaystyle
u_{1}=s^{\prime\prime}(t_{1})=s^{(2)}(t_{1}),\ldots,u_{n+1}=s^{\prime\prime}(t_{n+1})=s^{(2)}(t_{n+1}).$

See [2, 3, 4] for more details.

It follows for $s\in S_{nat}$ that

$\displaystyle
s=p_{1}\boldsymbol{b}_{1}^{002}+p_{2}\boldsymbol{b}_{2}^{002}+\sum_{i=2}^{n}u_{i}\boldsymbol{b}_{2+i}^{002},$

and then $\boldsymbol{s}=(s(t_{1}),\ldots,s(t_{n+1})^{\top}$ is given by

$\displaystyle\boldsymbol{s}=[\boldsymbol{b}_{1}^{002}(\boldsymbol{t}),\boldsymbol{b}_{2}^{002}(\boldsymbol{t})](p_{1},p_{2})^{\top}+\sum_{i=2}^{n}\mathbf{R}(u_{2},\ldots,u_{n})^{\top},$

Here the column
$\boldsymbol{b}_{i}^{002}(\boldsymbol{t})=(b_{i}^{002}(t_{1}),\ldots,b_{i}^{002}(t_{n+1}))^{\top}$,
with $i=4,\ldots,n+2$, and the $n+1\times n+1$ matrix

$\displaystyle\mathbf{R}=[\boldsymbol{b}_{4}^{002}(\boldsymbol{t}),\ldots,\boldsymbol{b}_{n+2}^{002}(\boldsymbol{t})].$

Observe that
$span(\boldsymbol{b}_{1}^{002},\boldsymbol{b}_{2}^{002})=span({\bf 1},{\bf
t})$ with the column ${\bf 1}=(1,\ldots,1)^{\top}$, ${\bf
t}=(t_{1},\ldots,t_{n+1})^{\top}$.

We can show that

$\displaystyle\int_{t_{1}}^{t_{n+1}}|s^{\prime\prime}(t)|^{2}dt=$ (19)
$\displaystyle\sum_{i=1}^{n}\int_{t_{i}}^{t_{i+1}}|u_{i}+t(u_{i+1}-u_{i})/h_{i}|^{2}dt$
$\displaystyle=$
$\displaystyle\sum_{i=1}^{n}(u_{i}^{2}+u_{i}u_{i+1}+u_{i+1}^{2})h_{i}/3$
$\displaystyle=$
$\displaystyle(u_{2},\ldots,u_{n})\mathbf{Q}(u_{2},\ldots,u_{n})^{\top},$

with $\mathbf{Q}$ is a known $n-1\times n-1$ invertible matrix see [2]. We
also recall that

$\displaystyle(u_{2},\ldots,u_{n})^{\top}=\mathbf{U}(p_{1},\ldots,p_{n+1})^{\top},$

with $\mathbf{U}$ is a known $n-1\times n+1$ matrix see [2]. Therefore

$\displaystyle(u_{2},\ldots,u_{n})\mathbf{Q}(u_{2},\ldots,u_{n})^{\top}=(p_{1},\ldots,p_{n+1})\mathbf{U}^{\top}\mathbf{Q}\mathbf{U}(p_{1},\ldots,p_{n+1})^{\top}$
(20)
$\displaystyle=:(p_{1},\ldots,p_{n+1})\mathbf{P}(p_{1},\ldots,p_{n+1})^{\top}$
(21)

Now we propose the following predictors for $f(t_{n+1})$.

0) We assume that $\boldsymbol{s}$ is Gaussian centred with the covariance
matrix $\mathbf{K}^{(0)}=(\mathbf{Q}^{(0)})^{-1}$ with $\mathbf{Q}^{(0)}$ is
defined by

$\displaystyle\int_{t_{1}}^{t_{n+1}}|s(t)|^{2}dt=\boldsymbol{s}^{\top}\mathbf{Q}^{(0)}\boldsymbol{s}.$

1) We consider the spline

$\displaystyle
S(f)=\arg\min\\{(f(t_{1}),\ldots,f(t_{n+1}))\mathbf{P}(f(t_{1}),\ldots,f(t_{n+1}))^{\top}:\quad
f(t_{1}),\ldots,f(t_{n})\quad\mbox{are fixed}\\},$ (23)

defined by the kernel $\mathbf{P}$ (21) and the predictor
$f^{*}(t_{n+1})=S(f)(t_{n+1})$ of $f(t_{n+1})$. We assume that
$\boldsymbol{s}$ is Gaussian with the mean
$p_{1}\boldsymbol{b}_{1}^{002}(\boldsymbol{t})+p_{2}\boldsymbol{b}_{2}^{002}(\boldsymbol{t})=\beta_{1}{\bf
1}+\beta_{1}{\bf t}$ and the covariance matrix
$\mathbf{K}^{(1)}=\mathbf{R}\mathbf{Q}^{-1}\mathbf{R}^{\top}$ with the kernel
$\mathbf{Q}$ is given by (20). The predictor $\hat{f}^{(1)}(t_{n+1})$ of
$f(t_{n+1})$ (12) using the kernel $\mathbf{K}^{(1)}$ coincides with
$S(f)(t_{n+1})$.

2) We assume that $\boldsymbol{s}$ is Gaussian with the mean
$p_{1}\boldsymbol{b}_{1}^{002}(\boldsymbol{t})+p_{2}\boldsymbol{b}_{2}^{002}(\boldsymbol{t})=\beta_{1}{\bf
1}+\beta_{1}{\bf t}$ and the covariance matrix
$\mathbf{K}^{(2)}=\mathbf{R}\mathbf{Q}\mathbf{R}^{\top}$.

Let $\hat{f}^{(i)}(t_{n+1})$ be the predictor of $f(t_{n+1})$ (12) using the
kernel $\mathbf{K}^{(i)}$ with $i=0,1,2$. Using real data, we are going to
compare these three predictors.

###  5.2 Real data Application

As application in the climate change area we are interested in the annual mean
temperature observed in France and Morocco from 1901 to 2015, the data are
presented in Figure 1. We illustrate the importance of the kernel choice by
considering the kernels $\mathbf{K}^{(0)}$, $\mathbf{K}^{(1)}$,
$\mathbf{K}^{(2)}$. The three kernel selection criteria are presented in Table
1. The mean annual temperature of the year 2015 and 2016 (i.e.
$\hat{f}^{(i)}(t_{n})$ and $\hat{f}^{(i)}(t_{n+1})$, $n=114$) are given in
Tables (2, 3), as for Figure 2 it shows the splines of the predictors
$\hat{f}^{(0)}$, $\hat{f}^{(1)}$, $\hat{f}^{(2)}$ and the true temperature.
The $w_{1}^{*}$, $\ldots$, $w_{n}^{*}$ of (12) for the kernels
$\mathbf{K}^{(0)}$, $\mathbf{K}^{(1)}$, $\mathbf{K}^{(2)}$ are presented in
Figure 3.

Figure 1: Mean annual temperatures in France and Morocco from 1901 to 2015.

Table 1:  The three kernel selection criteria for the kernels $\mathbf{K}^{(0)}$,$\mathbf{K}^{(1)}$, $\mathbf{K}^{(2)}$ using Morroco and France data. Country | France | Morocco
---|---|---
Kernel | $\mathbf{K}^{(0)}$ | $\mathbf{K}^{(1)}$ | $\mathbf{K}^{(2)}$ | $\mathbf{K}^{(0)}$ | $\mathbf{K}^{(1)}$ | $\mathbf{K}^{(2)}$
$MSPE$ | 0.3301302 | 2.090779 | 5.21788 | 0.7727975 | 5.110724 | 11.70042
$MAXPE$ | 1.3961 | 3.344106 | 5.312125 | 2.251341 | 6.171007 | 9.438383
Statistically |  $\mathbf{K}^{(0)}$ with 0.8198198 for $\mathbf{K}^{(1)}$ |  $\mathbf{K}^{(0)}$ with 0.8288288 for $\mathbf{K}^{(1)}$
| and 0.8288288 for $\mathbf{K}^{(2)}$ | and 0.8468468 for $\mathbf{K}^{(2)}$
Figure 2: The splines of the predictors $\hat{f}^{(0)}$, $\hat{f}^{(1)}$,
$\hat{f}^{(2)}$ and the true temperature.

Table 2:  The predictors $\hat{f}^{(i)}(t_{n})$, $n=114$ (the mean annual temperature of the year 2015). Country | France | Morocco
---|---|---
Kernel | $\mathbf{K}^{(0)}$ | $\mathbf{K}^{(1)}$ | $\mathbf{K}^{(2)}$ | $\mathbf{K}^{(0)}$ | $\mathbf{K}^{(1)}$ | $\mathbf{K}^{(2)}$
Prediction | 13.17656 | 15.48813 | 15.61992 | 18.06526 | 21.18307 | 20.41619
True temperature | 13.5 | 18.9008
Table 3:  The predictors $\hat{f}^{(i)}(t_{n+1})$, $n=114$ (the mean annual temperature of the year 2016). Country | France | Morocco
---|---|---
Kernel | $\mathbf{K}^{(0)}$ | $\mathbf{K}^{(1)}$ | $\mathbf{K}^{(2)}$ | $\mathbf{K}^{(0)}$ | $\mathbf{K}^{(1)}$ | $\mathbf{K}^{(2)}$
Prediction | 12.91553 | 12.54049 | 11.40698 | 17.86737 | 18.99740 | 18.49113

######  Remark 5.1.

Table 1 shows that the kernel $\mathbf{K}^{(0)}$ wins against
$\mathbf{K}^{(1)}$ and $\mathbf{K}^{(2)}$ with respect to the three kernel
selection criteria.

Figure 3: The $w_{1}^{*}$, $\ldots$, $w_{n}^{*}$ of (12) for the kernels
$\mathbf{K}^{(0)}$, $\mathbf{K}^{(1)}$, $\mathbf{K}^{(2)}$.

###  5.3 Concluding remarks

The numerical results shows the three kernel selection criteria are stable,
form Table 1 we have that the best kernel among the three kernels is
$\mathbf{K}^{(0)}$ w.r.t. all the three criteria for both France and Morocco
data. Moreover, the representation of the splines (Figure 2) shows that too.

From Table 1 and Figure 2 we have that the kernel $\mathbf{K}^{(1)}$ wins
against $\mathbf{K}^{(2)}$. Considering the second derivative
$(u_{2},\ldots,u_{n})$ as Gaussian with the covariance matrix $Q^{-1}$ is a
good stochastic modelization, at least is better than the assumption that
$(u_{2},\ldots,u_{n})$ as Gaussian with the covariance matrix $Q$.
Equivalently measuring the worst error in the unit ball using the norm
$\|Q^{1/2}\boldsymbol{u}\|$ is better than the norm
$\|Q^{-1/2}\boldsymbol{u}\|$.

## References

  * [1] M. Scheuerer, R. Schaback, M. Schlather, Interpolation of spatial data-a stochastic or a deterministic problem, European Journal of Applied Mathematics 24 (4) (2013) 601–629.
  * [2] Azzouz Dermoune, Khalifa Es-Sebaiy, Mohammed Es.Sebaiy, Jabrane Moustaaid. Parametrizations, weights, and optimal prediction (2021). Communication in Statistics-Theory and Methods, 50(4), 815-836. https://www.tandfonline.com/doi/full/10.1080/03610926.2019.1642489
  * [3] A. Dermoune, C. Preda, Parametrizations, fixed and random effects, Journal of Multivariate Analysis 154 (2017) 162–176.
  * [4] A. Dermoune, C. Preda, Estimation of noisy cubic spline using a natural basis, Annals of the University 265 of Craiova, Mathematics and Computer Science Series 43 (1) (2016) 33–52.
  * [5] J.P. Chilès, How to adapt kriging to non-classical problems: three case studies, In: M. Guarascio, M. David and C. Huijbregts (editors), Advanced Geostatistics in the Mining Industry, D. Reidel, Dordrecht, Holland (1976) 69–89.
  * [6] J.P. Chilès, P. Delfiner, Geostatistics, Modeling Spatial Uncertainty, John Wiley, New York (2009).
  * [7] H. Wendland, Scattered data approximation, Cambridge Monographs on Applied and Computational Mathematics, Cambridge University Press, Cambridge, UK (2005).
  * [8] M. G. Mardikis, D. P. Kalivas, V. J. Kollias, Comparison of interpolation methods for the prediction of reference evapotranspiration—an application in greece, Water Resources Management 19 (3) (2018) 250 251–278.
  * [9] A. Berlinet, C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics, Kluwer, Berlin, Germany, (2004).
  * [10] I. J. Schoenberg, Contributions to the problem of approximation of equidistant data by analytic functions, quart, Appl. Math. 4 (1946) 44–99 and 112–141.
  * [11] C. H. Reinsch, Smoothing by spline functions, Numerische Mathematik 10 (1967) 177– 183.
  * [12] C. de Boor, A practical guide to splines, Applied Mathematical Sciences. 27 (1978) xxiv+392.