How to break numpy array into smaller chunks/batches, then iterate through them
<p>Suppose i have this numpy array</p>

<pre><code>[[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
[10, 11, 12]]
</code></pre>

<p>And i want to split it in 2 batches and then iterate:</p>

<pre><code>[[1, 2, 3],      Batch 1
[4, 5, 6]]

[[7, 8, 9],      Batch 2
[10, 11, 12]]
</code></pre>

<p>What is the simplest way to do it?</p>

<p>EDIT: <strong>I'm deeply sorry i missed putting such info</strong>: Once i intend to carry on with the iteration, the original array would be destroyed due to splitting and iterating over batches. Once the batch iteration finished, i need to restart again from the first batch hence <strong>I should preserve that the original array wouldn't be destroyed</strong>. The whole idea is to be consistent with Stochastic Gradient Descent algorithms which require iterations over batches. In a typical example, I could have a 100000 iteration For loop for just 1000 batch that should be replayed again and again.</p>

<p>You can use <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.split.html" rel="nofollow"><code>numpy.split</code></a> to split along the first axis <code>n</code> times, where <code>n</code> is the number of desired batches. Thus, the implementation would look like this -</p>

<pre><code>np.split(arr,n,axis=0) # n is number of batches
</code></pre>

<p>Since, the default value for <code>axis</code> is <code>0</code> itself, so we can skip setting it. So, we would simply have -</p>

<pre><code>np.split(arr,n)
</code></pre>

<p>Sample runs -</p>

<pre><code>In [132]: arr  # Input array of shape (10,3)
Out[132]: 
array([[170,  52, 204],
       [114, 235, 191],
       [ 63, 145, 171],
       [ 16,  97, 173],
       [197,  36, 246],
       [218,  75,  68],
       [223, 198,  84],
       [206, 211, 151],
       [187, 132,  18],
       [121, 212, 140]])

In [133]: np.split(arr,2) # Split into 2 batches
Out[133]: 
[array([[170,  52, 204],
        [114, 235, 191],
        [ 63, 145, 171],
        [ 16,  97, 173],
        [197,  36, 246]]), array([[218,  75,  68],
        [223, 198,  84],
        [206, 211, 151],
        [187, 132,  18],
        [121, 212, 140]])]

In [134]: np.split(arr,5) # Split into 5 batches
Out[134]: 
[array([[170,  52, 204],
        [114, 235, 191]]), array([[ 63, 145, 171],
        [ 16,  97, 173]]), array([[197,  36, 246],
        [218,  75,  68]]), array([[223, 198,  84],
        [206, 211, 151]]), array([[187, 132,  18],
        [121, 212, 140]])]
</code></pre>

python pandas numpy