General optimization - implementation in python
<p>I am currently struggling with a gradient descent implementation problem, more on the math side of it. I have a matrix of input values, for example - <code>[[1,1,0,2],[2,3,5,1],[2,1,8,0]]</code>. I want to calculate weights which will minimize the error against output vector, the minimized function is standard linear model so my hypothesis is minimize -> <code>np.dot(input,weights)-y</code>. The problem is - values of weight vector should add to specific number, say 2. Also vector of outputs is normalized such as <code>np.dot(input,weights)/sum(np.dot(input,weights))</code> - this result is then compared to a desired output vector. How should I define this task in python/numpy? </p>

<p>Example of human-tuned procedure:</p>

<p>1) input matrix 
<code>[[4,0,2,0,2,0],[2,0,0,2,2,0],[2,0,0,2,2,0],[4,0,2,0,0,0],[0,0,2,0,0,2],[0,4,0,0,0,2],[0,2,0,0,0,2],[0,2,2,0,0,0],[0,0,2,0,0,2],[4,0,2,0,0,0]]</code></p>

<p>2) desired output <code>[12.94275893,8.07054252,9.281123898,10.53654162,8.698251382,14.67643103,7.158870124,10.26752354,8.324615155,10.0433418]</code></p>

<p>3) weights which transform input vectors in such way that np.dot(input,weights)/sum(np.dot(input,weights)) are okay
<code>[11,21,18,0,20,14]</code>- sum fixed at 84</p>

<p>4) the final output, reasonably deviated from 2) <code>[15.15,7.83,7.83,10.10,8.08,14.14,8.84,9.85,8.08,10.10]</code></p>

<p>For the scale of your example data, here is the solution:</p>

<pre><code>import numpy as np
from scipy import optimize

a = np.array([[1,1,0,2],[2,3,5,1],[2,1,8,0]], dtype=float)
target = np.random.randn(3)
target /= target.sum()

def f(p):
    p = np.r_[p, 2 - p.sum()]
    res = a.dot(p)
    res /= res.sum()
    return res - target

r, _ = optimize.leastsq(f, np.zeros(3))
print(target)
print(np.r_[r, 2 - r.sum()])
</code></pre>

<p>output:</p>

<pre><code>[-0.21987606  0.70869974  0.51117632]
[ 2.15713915  7.47554671  0.38959227 -8.02227813]
</code></pre>

<p>Here is the code for your real data:</p>

<pre><code>import numpy as np
from scipy import optimize

a = np.array([[4,0,2,0,2,0],
              [2,0,0,2,2,0],
              [2,0,0,2,2,0],
              [4,0,2,0,0,0],
              [0,0,2,0,0,2],
              [0,4,0,0,0,2],
              [0,2,0,0,0,2],
              [0,2,2,0,0,0],
              [0,0,2,0,0,2],
              [4,0,2,0,0,0]], dtype=float)

target = np.array([12.94275893,8.07054252,9.281123898,10.53654162,8.698251382,
                   14.67643103,7.158870124,10.26752354,8.324615155,10.0433418])

target /= target.sum()

def make_vector(x):
    return np.r_[x, 84 - x.sum()]

def calc_target(x):
    res = a.dot(make_vector(x))
    res /= res.sum()
    return res

def error(x):
    return calc_target(x) - target

x, _ = optimize.leastsq(error, np.zeros(a.shape[1] - 1))
print(make_vector(x))
print(calc_target(x) * 100)
print((calc_target(x) - target) * 100)
</code></pre>

<p>the output:</p>

<pre><code>[  9.40552097  20.32874298  19.8199082   13.13991088  10.00062863
  11.30528834]
[ 12.90025777   8.63333209   8.63333209  10.2474406    8.25642656
  13.78390749   8.39140263  10.65003363   8.25642656  10.2474406 ]
[-0.04250116  0.56278957 -0.64779181 -0.28910102 -0.44182483 -0.89252354
  1.23253251  0.38251009 -0.0681886   0.2040988 ]
</code></pre>

<p>It seems that the problem can also be solve by <code>numpy.linalg.lstsq()</code>, but it need to simplify your problem to a linear euqations.</p>

python numpy optimization gradient