Dealing with SciPy fmin_bfgs precision loss
<p>I'm currently trying to solve numerically a minimization problem and I tried to use the optimization library available in SciPy. </p>

<p>My function and derivative are a bit too complicated to be presented here, but they are based on the following functions, the minimization of which do not work either:</p>

<pre><code>def func(x):
    return np.log(1 + np.abs(x))
def grad(x):
    return np.sign(x) / (1.0 + np.abs(x))
</code></pre>

<p>When calling the fmin_bfgs function (and initializing the descent method to x=10), I get the following message:</p>

<pre><code>Warning: Desired error not necessarily achieved due to precision loss.
     Current function value: 2.397895
     Iterations: 0
     Function evaluations: 24
     Gradient evaluations: 22
</code></pre>

<p>and the output is equal to 10 (i.e. initial point). I suppose that this error may be caused by two problems:</p>

<ul>
<li><p>The objective function is not convex: however I checked with other non-convex functions and the method gave me the right result.</p></li>
<li><p>The objective function is "very flat" when far from the minimum because of the log. </p></li>
</ul>

<p>Are my suppositions true? Or does the problem come from anything else? 
Whatever the error can be, what can I do to correct this? In particular, is there any other available minimization method that I could use?</p>

<p>Thanks in advance.</p>

<p><code>abs(x)</code> is always somewhat dangerous as it is non-differentiable. Most solvers expect problems to be smooth. Note that we can drop the <code>log</code> from your objective function and then drop the <code>1</code>, so we are left with minimizing <code>abs(x)</code>. Often this can be done better by the following.</p>

<p>Instead of <code>min abs(x)</code> use </p>

<pre><code>min t
-t &lt;= x &lt;= t
</code></pre>

<p>Of course this requires a solver that can solve (linearly) constrained NLPs.</p>

python scipy mathematical-optimization