How much overhead does python numpy tolist() add?
<p>I am using a python program that uses numpy array as standard data type for arrays. For the heavy computation I pass the arrays to a C++ library. In order to do so, I use <a href="https://github.com/pybind/pybind11" rel="nofollow">pybind</a>. However, I am required to use python <code>list</code>. I do the conversion from <code>numpy</code> array and <code>list</code> via:</p>

<pre><code>NativeSolver.vector_add(array1.tolist(), array2.tolist(), ...)
</code></pre>

<p>How much overhead does this conversion generate? I hope it doesn't create a whole new copy. Numpy reference says:</p>

<blockquote>
  <p><strong>ndarray.tolist()</strong></p>
  
  <p>Return a copy of the array data as a (nested) Python list. Data items
  are converted to the nearest compatible Python type.</p>
</blockquote>

<p>Yes it will be new copy.  The data layout for an array is very different from that of a list.</p>

<p>An array has attributes like shape and strides, and a 1d data buffer that contains the elements - just a contiguous set of bytes.  It's the other attributes and code that treats them as floats, int, strings, 1d, 2d etc.</p>

<p>A list is a buffer of pointers, with each pointer pointing to an object else where in memory.  It may point to a number, a string, or another list.  It is not going to point to the array's databuffer or elements in it.</p>

<p>There are interfacing numpy arrays with compiled code and C arrays that make use of the array data buffer. <code>cython</code> is a common on.  There is also a whole documentation section on the C API for numpy.  I know anything about <code>pbind</code>. If it requires a list interface it may not be the best.</p>

<p>When I've done <code>timeit</code> tests with <code>tolist()</code> it hasn't appeared to be that expensive.</p>

<p>=======================</p>

<p>But looking at the <code>pybind11</code> github I find a number of references to <code>numpy</code>, and this</p>

<p><a href="http://pybind11.readthedocs.io/en/latest/advanced.html#numpy-support" rel="nofollow">http://pybind11.readthedocs.io/en/latest/advanced.html#numpy-support</a></p>

<p>documentation page.  It supports the buffer protocol, and numpy arrays.  So you shouldn't have to go through the <code>tolist</code> step.</p>

<pre><code>#include &lt;pybind11/numpy.h&gt;
void f(py::array_t&lt;double&gt; array);
</code></pre>

python c++ arrays numpy wrapper