IndexError: index is out of bounds for axis 0 with size
<p>I have array <code>x_train</code> and <code>targets_train</code>. I want to shuffle the training data and split it into smaller batches and use the batches as training data. My original data has 1000 rows and each time I try to use 250 rows of them :</p>

<pre><code>    x_train = np.memmap('/home/usr/train', dtype='float32', mode='r', shape=(1000, 1, 784))
# print(x_train)
targets_train = np.memmap('/home/usr/train_label', dtype='int32', mode='r', shape=(1000, 1))
train_idxs = [i for i in range(x_train.shape[0])]
np.random.shuffle(train_idxs)


num_batches_train = 4
def next_batch(start, train, labels, batch_size=250):
    newstart = start + batch_size
    if newstart &gt; train.shape[0]:
        newstart = 0
    idxs = train_idxs[start:start + batch_size]
    # print(idxs)
    return train[idxs, :], labels[idxs, :], newstart


# x_train_lab = x_train[:200]
# # x_train = np.array(targets_train)
# targets_train_lab = targets_train[:200]
for i in range(num_batches_train):
    x_train, targets_train, newstart = next_batch(i*batch_size, x_train, targets_train, batch_size=250)
</code></pre>

<p>The problem is, when I shuffle the training data and try to access to batches I get error saying: </p>

<pre><code>    return train[idxs, :], labels[idxs, :], newstart
    IndexError: index 250 is out of bounds for axis 0 with size 250
</code></pre>

<p>Is there anybody who knows what am I doing wrong?</p>

<p>(edit - first guess about <code>newstart</code> deleted)</p>

<p>In this line:</p>

<pre><code>x_train, targets_train, newstart = next_batch(i*batch_size, x_train, targets_train, batch_size=250)
</code></pre>

<p>you change the size of <code>x_train</code> with each iteration, yet you continue to use the <code>train_idxs</code> array that you created for the full size array.</p>

<p>It's one thing to pull out random values from <code>x_train</code> in batches, but you have to keep the selection arrays consistent.</p>

<p>This question probably should have been closed for lack of a minimal, and verifiable example.  It's frustrating to have to guess and make a small testable example in hopes of recreating the problem.</p>

<p><a href="http://stackoverflow.com/help/mcve">http://stackoverflow.com/help/mcve</a></p>

<p>If my current guess is wrong, just a few intermediate print statements would have made the problem clear. </p>

<p>========================</p>

<p>Reducing your code to a simple case</p>

<pre><code>import numpy as np
x_train = np.arange(20).reshape(20,1)
train_idxs = np.arange(x_train.shape[0])
np.random.shuffle(train_idxs)

num_batches_train = 4
batch_size=5
def next_batch(start, train):
    idxs = train_idxs[start:start + batch_size]
    print(train.shape, idxs)
    return train[idxs, :]

for i in range(num_batches_train):
    x_train = next_batch(i*batch_size, x_train)
    print(x_train)
</code></pre>

<p>a run produces:</p>

<pre><code>1658:~/mypy$ python3 stack39919181.py 
(20, 1) [ 7 18  3  0  9]
[[ 7]
 [18]
 [ 3]
 [ 0]
 [ 9]]
(5, 1) [13  5  2 15  1]
Traceback (most recent call last):
  File "stack39919181.py", line 14, in &lt;module&gt;
    x_train = next_batch(i*batch_size, x_train)
  File "stack39919181.py", line 11, in next_batch
    return train[idxs, :]
IndexError: index 13 is out of bounds for axis 0 with size 5
</code></pre>

<p>I fed the (5,1) <code>x_train</code> back to the <code>next_batch</code> but tried to index it as though it were the original.</p>

<p>Changing the iteration to:</p>

<pre><code>for i in range(num_batches_train):
    x_batch = next_batch(i*batch_size, x_train)
    print(x_batch)
</code></pre>

<p>lets it run through producing 4 batches of 5 rows.</p>

python arrays list numpy machine-learning