How to find least-mean-square error quadratic upper bound?
<p>I have some data of the form</p>

<p><code>x1[i], x2[i], x3[i], z[i]</code>,</p>

<p>where <code>z[i]</code> is an unknown deterministic function of <code>x1[i], x2[i], and x3[i]</code>.  I would like to find a quadratic function <code>u(x1, x2, x3)= a11*x1^2 + a22*x2^2 + a33*x3^2 + a12*x1*x2 + ... + a0</code> that overbounds the data, i.e., <code>u(x1[i], x2[i], x3[i]) &gt;= z[i]</code> for all <code>i</code>, and that minimizes the sum of the squared errors subject to the constraints.  </p>

<p>Is there a computationally efficient solution approach in either Python or Matlab?</p>

<p>There is very simple solution. Just use polynomial regression in Mathlab (<a href="http://www.matrixlab-examples.com/polynomial-regression.html" rel="nofollow">http://www.matrixlab-examples.com/polynomial-regression.html</a>).
You will get a certain function P(x1[i],x2[i],x3[i]).
1. Then for each i compute expression Diff[i] = P(x1[i],x2[i],x3[i]) - z[i].
   You will get some array Diff. 
2. Select all the negative values.
3. Find the minimum value in Diff: M = Min(Diff).
4. The desired function is F(x1[i],x2[i],x3[i]) = P(x1[i],x2[i],x3[i]) + Abs(M), where Abs(M) - it's value excluding the sign of M.</p>

<p>But if you're not just limited to quadratic functions, you can vary the degree of the polynomial and eventually get a more precise solution.</p>

python matlab least-squares