Most efficient way to store list of integers
<p>I have recently been doing a project in which one of the aims is to use as little memory as possible to store a series of files using Python 3. Almost all of the files take up very little space, apart from one list of integers that is roughly <code>333,000</code> integers long and has integers up to about <code>8000</code> in size. </p>

<p>I'm currently using <code>pickle</code> to store the list, which takes up around <code>7mb</code>, but I feel like there must be a more memory efficient way to do this.</p>

<p>I have tried storing it as a text file and <code>csv</code>, bur both of these used in excess of <code>10mb</code> of space.</p>

<p>One <code>stdlib</code> solution you could use is arrays from <a href="https://docs.python.org/3/library/array.html" rel="nofollow"><code>array</code></a>, from the docs:</p>

<blockquote>
  <p>This module defines an object type which can compactly represent an array of basic values: characters, integers, floating point numbers. Arrays are sequence types and behave very much like lists, except that the type of objects stored in them is constrained. </p>
</blockquote>

<p>This generally sheds a bit of memory of large lists, for example, with a 10 million element a list, the array trims up <code>11mb</code>:</p>

<pre><code>import pickle    
from array import array

l = [i for i in range(10000000)]
a = array('i', l)

# tofile can also be used.
with open('arrfile', 'wb') as f:  
    pickle.dump(a, f)

with open('lstfile', 'wb') as f:
    pickle.dump(l, f)
</code></pre>

<p>Sizes:    </p>

<pre><code>!du -sh ./*
39M     arrfile
48M     lstfile
</code></pre>

python list python-3.x memory integer