Python itertools.product freezes with higher numbers
<p>I have 6 variables with different ranges. I want to create possibilities pool with my code. In this example i gave 10 range for every variable but i have to give them about 200 range. But whenever i'm trying to exceed 20 range (for example 30 range) Python kills itself, and sometimes it freezes computer. Is there anyway to make it faster and stable?</p>

<p>Thanks.</p>

<pre><code>import itertools

a = [x for x in range(400,411)]
b = [x for x in range(400,411)]
c = [x for x in range(400,411)]
d = [x for x in range(400,411)]
e = [x for x in range(400,411)]
f = [x for x in range(400,411)]

fl = lambda x: x

it = filter(fl, itertools.product(a,b,c,d,e,f))

posslist = [x for x in it]

print(len(posslist))
</code></pre>

<p>There are 6 lists of 11 elements each: <code>[400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410]</code>.</p>

<p>A cartesian product of 6 such lists is a list of 11<sup>6</sup> tuples of 6 integers each (the first tuple is: <code>(400, 400, 400, 400, 400, 400)</code>).</p>

<p>The size of each tuple is 6*8 bytes in 64-bit Python<sup>*</sup>.</p>

<p>So the total size of <code>posslist</code> is 6 * 8 * 11<sup>6</sup> = 81 GB!</p>

<p>Do you have enough RAM for that? Probably not, so the OS is going to start swapping RAM, which is extremely slow. Therefore, in addition to calculating 81 GB of data, thecomputer will have to constantly swap data from RAM to HDD and back, so it will do the job even slower.</p>

<hr>

<p><sup>* Note that while it is half that size in a 32-bit Python, a 32-bit Python cannot address enough memory at all </sup></p>

python python-3.x itertools