Scipy.optimize COBYLA constraint violation
<p>I have an optimization problem with constraints, but the COBYLA solver doesn't seem to respect the constraints I specify.</p>

<p>My optimization problem:</p>

<pre><code>cons = ({'type':'ineq', 'fun':lambda t: t},) # all variables must be positive
minimize(lambda t: -stateEst(dict(zip(self.edgeEvents.keys(),t)), (0.1,)*len(self.edgeEvents), constraints=cons, method='COBYLA')
</code></pre>

<p>and <code>stateEst</code> is defined as:</p>

<pre><code>def stateEst(t):
    val = 0
    for edge,nextState in self.edgeEvents.iteritems():
        val += edge_probability(self,edge,ts) * estimates[nextState]
        val += node_probability(self, edge.head, ts, edge_list=[edge])* cost
    for node,nextState in self.nodeEvents.iteritems():
        val += node_probability(self, node, ts) * \
            (estimates[nextState] + cost*len([e for e in node.incoming if e in self.compEdges])
    return val
</code></pre>

<p>The probability functions are only defined for positive <code>t</code> values. The dictionary is necessary because the probabilities are calculated with respect to the 'named' t-values.</p>

<p>When I run this, I notice that COBYLA tries a value of -0.025 for one of the t-values. Why is the optimization not respecting the constraints?</p>

<p><strong>COBYLA</strong> is technically speaking an <strong>infeasible method</strong>, which means, that the <strong>iterates might not be always feasible in regards to your constraints!</strong> (it's only about the final convergence, where feasibility matters for these algorithms).</p>

<p>Using an objective-function which is not defined everywhere will be problematic. Maybe you are forced to switch to some <strong>feasible method</strong>.</p>

<p>Alternatively you could think about generalizing your objective, so that there are penalties introduced for negative t's. But this is problem-dependent and could introduce other problems as well (convergence; numeric-stability).</p>

<p>Try using <a href="http://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html#optimize-minimize-lbfgsb" rel="nofollow">L-BFGS-B</a>, which is limited to bound-constraints, which is not a problem here (for your current problem!).</p>

python optimization scipy constraints