Why can I make a numpy array that's (apparently) bigger than my computer's memory?
<p>When I run code below:</p>

<pre><code>import scipy.sparse
x = scipy.sparse.random(100000, 100000, 1e-4)
y = x.toarray()
print(y.nbytes)
</code></pre>

<p>I get an output of 80000000000 bytes = 80 GB. And yet I am using a Macbook Air with only 4 GB of RAM. Can someone explain how I am (apparently) creating a NumPy array larger than my memory size? Is <code>y</code> somehow a view of <code>x</code>, rather than a copy? I didn't find anything about this in the <code>scipy.sparse</code> documentation. Unsurprisingly if I do something like <code>y.copy()</code> I crash Python... I can't expect to do something to an array of size 10^10. Thanks!</p>

<p>Versions:
Python 3.5.2 via Anaconda 4.1.1, SciPy 0.17.1, NumPy 1.11.1.</p>

<p>This is because numpy doesn't actually allocate all that space. Most likely sparse arrays and matrices are represented with <a href="http://btechsmartclass.com/DS/U1_T14.html" rel="nofollow">triplets, linked nodes</a> or some other means of ignoring all the empty space between. The bytes are calculated based on the assigned dimensions of the matrix/array, not the actual data in memory.</p>

python numpy memory scipy sparse-matrix