Libraries in "virtualenv" much bigger than system libraries
<p>I use virtualenv through pew (which I think is a fantastic tool), but I noticed something strange.</p>

<p>I have scipy system-side installed:</p>

<pre><code>    7,7 MiB [##########] /sparse
    5,1 MiB [######    ] /special
    5,1 MiB [######    ] /stats
    5,0 MiB [######    ] /linalg
    3,5 MiB [####      ] /spatial
    3,0 MiB [###       ] /optimize
    2,5 MiB [###       ] /signal
    2,3 MiB [###       ] /interpolate
    2,3 MiB [##        ] /misc
    2,2 MiB [##        ] /io
    1,5 MiB [##        ] /integrate
    1,3 MiB [#         ] /ndimage
    1,0 MiB [#         ] /fftpack
  744,0 KiB [          ] /cluster
  512,0 KiB [          ] /odr
  464,0 KiB [          ] /constants
  252,0 KiB [          ] /_lib
   44,0 KiB [          ] /_build_utils
   36,0 KiB [          ] /__pycache__
   24,0 KiB [          ]  HACKING.rst.txt
   12,0 KiB [          ]  THANKS.txt
    8,0 KiB [          ]  INSTALL.rst.txt
    4,0 KiB [          ]  __init__.py
    4,0 KiB [          ]  __config__.py
    4,0 KiB [          ]  LICENSE.txt
    4,0 KiB [          ]  setup.py
    4,0 KiB [          ]  BENTO_BUILD.txt
    4,0 KiB [          ]  version.py
    4,0 KiB [          ]  linalg.pxd
</code></pre>

<p>And this is scipy virtualenv-side installed (same scipy version):</p>

<pre><code>51,0 MiB [##########] /sparse
   37,6 MiB [#######   ] /.libs
   12,9 MiB [##        ] /linalg
   10,6 MiB [##        ] /spatial
    9,7 MiB [#         ] /special
    6,0 MiB [#         ] /interpolate
    5,9 MiB [#         ] /stats
    5,1 MiB [#         ] /optimize
    4,2 MiB [          ] /signal
    3,2 MiB [          ] /io
    3,0 MiB [          ] /integrate
    3,0 MiB [          ] /ndimage
    2,3 MiB [          ] /misc
    2,1 MiB [          ] /cluster
    1,7 MiB [          ] /fftpack
  884,0 KiB [          ] /odr
  328,0 KiB [          ] /constants
  204,0 KiB [          ] /_lib
   32,0 KiB [          ] /_build_utils
   24,0 KiB [          ]  HACKING.rst.txt
   20,0 KiB [          ] /__pycache__
   12,0 KiB [          ]  THANKS.txt
    8,0 KiB [          ]  INSTALL.rst.txt
    4,0 KiB [          ]  __init__.py
    4,0 KiB [          ]  LICENSE.txt
    4,0 KiB [          ]  setup.py
    4,0 KiB [          ]  __config__.py
    4,0 KiB [          ]  BENTO_BUILD.txt
    4,0 KiB [          ]  version.py
    4,0 KiB [          ]  pip-delete-this-directory.txt
    4,0 KiB [          ]  linalg.pxd
</code></pre>

<p>Needless to say there is a huge size difference. It would normally not bother me much, but I'm trying to bundle an executable file with pyinstaller and the resulting executable file is unrealistically too big.</p>

<p>Can someone explain such a difference ? It is not specific to scipy, I also see it for numpy, and maybe for other libraries.</p>

<p>EDIT:</p>

<p>The <em>files</em> inside the directories have different sizes:</p>

<p>System-wide:</p>

<pre><code>3,1 MiB [##########]  _sparsetools.cpython-35m-x86_64-linux-gnu.so
</code></pre>

<p>Virtualenv-wide:</p>

<pre><code>38,5 MiB [##########]  _sparsetools.cpython-35m-x86_64-linux-gnu.so
</code></pre>

<p>The shared library files in the Python wheels distributed for Scipy aren't stripped, so they are bigger than what your package manager installs:</p>

<pre><code>$ file _sparsetools.cpython-35m-x86_64-linux-gnu.so
_sparsetools.cpython-35m-x86_64-linux-gnu.so: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=ed7b8e00c558f234620423315fa9b53274393e72, not stripped
$ du -h _sparsetools.cpython-35m-x86_64-linux-gnu.so
39M     _sparsetools.cpython-35m-x86_64-linux-gnu.so
</code></pre>

<p>If you <a href="https://unix.stackexchange.com/questions/2969/what-are-stripped-and-not-stripped-executables-in-unix">strip it</a>, the file size shrinks:</p>

<pre><code>$ strip _sparsetools.cpython-35m-x86_64-linux-gnu.so
$ file _sparsetools.cpython-35m-x86_64-linux-gnu.so
_sparsetools.cpython-35m-x86_64-linux-gnu.so: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=ed7b8e00c558f234620423315fa9b53274393e72, stripped
$ du -h _sparsetools.cpython-35m-x86_64-linux-gnu.so
3.7M    _sparsetools.cpython-35m-x86_64-linux-gnu.so
</code></pre>

<p>PyInstaller can do this for you with the <a href="https://pythonhosted.org/PyInstaller/usage.html#how-to-generate"><code>--strip</code> flag</a>.</p>

python numpy scipy virtualenv