How to insert NaN array into a numpy 2D array
<p>I'm trying to insert an arbitrary number of rows of NaN values within a 2D array at specific places. I'm logging some data from a microcontroller in a .csv file and parsing with python.</p>

<p>The data is stored in a 3 column 2D array like this</p>

<pre><code>[(122.0, 1.0, -47.0) (123.0, 1.0, -47.0) (125.0, 1.0, -44.0) ..., 
(39.0, 1.0, -47.0) (40.0, 1.0, -45.0) (41.0, 1.0, -47.0)]
</code></pre>

<p>The first column is an sequence counter. What I'm trying to do is iterate through the sequence values, diff current and previous sequence number  and insert as many rows with nan as there are missing sequences.</p>

<p>Basically, </p>

<pre><code>[(122.0, 1.0, -47.0) (123.0, 1.0, -47.0) (125.0, 1.0, -44.0)]
</code></pre>

<p>would become</p>

<pre><code>[(122.0, 1.0, -47.0) (123.0, 1.0, -47.0) (nan, nan, nan) (125.0, 1.0, -44.0)]
</code></pre>

<p>However the following implementation of <code>np.insert</code> produces an error</p>

<pre><code>while (i &lt; len(list[1])):
     pid = list[i][0]
     newMissing = (pid - LastGoodId + 255) % 256
     TotalMissing = TotalMissing + newMissing
     np.insert(list,i,np.zeros(newMissing,1) + np.nan)   
     i = i + newMissing
     list[i][0] = TotalMissing
     LastGoodId = pid 
</code></pre>

<blockquote>
  <p>---> 28                 np.insert(list,i,np.zeros(newMissing,1) + np.nan)
       29                 i = i + newMissing
       30                 list[i][0] = TotalMissing</p>
  
  <p>TypeError: data type not understood</p>
</blockquote>

<p>Any ideas on how I can accomplish this?</p>

<p><strong>Approach #1</strong></p>

<p>We can use an initialization based approach to handle multiple gaps and gaps of any lengths -</p>

<pre><code># Pre-processing step to create monotonically increasing array for first col
id_arr = np.zeros(arr.shape[0])
id_arr[np.flatnonzero(np.diff(arr[:,0])&lt;0)+1] = 256
a0 = id_arr.cumsum() + arr[:,0]

range_arr = np.arange(a0[0],a0[-1]+1)
out = np.full((range_arr.shape[0],arr.shape[1]),np.nan)
out[np.in1d(range_arr,a0)] = arr
</code></pre>

<p>Sample run -</p>

<pre><code>In [233]: arr      # Input array
Out[233]: 
array([[ 122.,    1.,  -47.],
       [ 123.,    1.,  -47.],
       [ 126.,    1.,  -44.],
       [  39.,    1.,  -47.],
       [  40.,    1.,  -45.],
       [  41.,    1.,  -47.]])

In [234]: out
Out[234]: 
array([[ 122.,    1.,  -47.],
       [ 123.,    1.,  -47.],
       [  nan,   nan,   nan],
       [  nan,   nan,   nan],
       [ 126.,    1.,  -44.],
       [  nan,   nan,   nan], (168 NaN rows)
       .....
       [  nan,   nan,   nan],
       [  nan,   nan,   nan],
       [  39.,    1.,  -47.],
       [  40.,    1.,  -45.],
       [  41.,    1.,  -47.]])
</code></pre>

<p><strong>Approach #2</strong></p>

<p>An alternative approach could be suggested to handle such generic cases using <code>np.insert</code> instead of initialization, like so -</p>

<pre><code>idx = np.flatnonzero(~np.in1d(range_arr,a0))
out = np.insert(arr,idx - np.arange(idx.size),np.nan,axis=0)
</code></pre>

python arrays numpy