how to efficiently cythonize the "vectorize" function (numpy library) - python
<p>as the title suggets, I'd like to efficiently cythonize the <code>numpy.vectorize</code> function, which, to the core, is simplyfying this piece below (the complete function is way too long to post but the majority of the time is spent here):</p>

<pre><code>    def func(*vargs):
        for _n, _i in enumerate(inds):
            the_args[_i] = vargs[_n]
        kwargs.update(zip(names, vargs[len(inds):]))
        return self.pyfunc(*the_args, **kwargs)
</code></pre>

<p>I have read these guides (<a href="http://cython.readthedocs.io/en/latest/src/tutorial/numpy.html" rel="nofollow">http://cython.readthedocs.io/en/latest/src/tutorial/numpy.html</a> and <a href="http://pandas.pydata.org/pandas-docs/stable/enhancingperf.html" rel="nofollow">http://pandas.pydata.org/pandas-docs/stable/enhancingperf.html</a>) which are very useful but my knowledge of C is way too narrow to use them to a fraction of their potential.</p>

<p>how would you go about it ? [Python 3.5.1, Cython 0.25a, Numpy 1.10.4]</p>

<p>The function you show is just a bit of dancing to deal with <code>kwargs</code>.  Note the comment at the head of that block in <code>vectorize.__call__</code>.  With simpler arguments it just sets <code>func = self.pyfunc</code>.</p>

<p>The actual work is done in the last line:</p>

<pre><code>self._vectorize_call(func=func, args=vargs)
</code></pre>

<p>which does</p>

<pre><code>outputs = ufunc(*inputs)
&lt; return dtype conversion &gt;
</code></pre>

<p><code>ufunc</code> is, in most cases, <code>frompyfunc(func, len(args), nout)</code>.</p>

<p>So stripped of all this Python cover,  it comes down to</p>

<pre><code>np.frompyfunc(your_func, n, m)(args)
</code></pre>

<p>and <code>frompyfunc</code> is a compiled function.  I suspect that function uses <code>nditer</code> (the <code>c</code> version) to broadcast the arguments, and feed the values as scalars to <code>your_func</code>.  I discussed the use of <code>nditer</code> with <code>cython</code> in another recent SO.</p>

<p>In sum, as long as <code>your_func</code> is an impentrable (or general) python function, there's nothing <code>cython</code> can do to improve on this.  The iteration is already being handled in compiled code.</p>

python numpy vectorization cython