Are Python modules compiled?
<p>Trying to understand whether python libraries are compiled because I want to know if the interpreted code I write will perform the same or worse.</p>

<p>e.g. I saw it mentioned somewhere that numpy and scipy are efficient because they are compiled. I don't think this means byte code compiled so how was this done? Was it compiled to c using something like cython? Or was it written using a language like c and compiled in a compatible way?</p>

<p>Does this apply to all modules or is it on a case-by-case basis?</p>

<p>Python can execute functions written in Python (interpreted) and compiled functions.  There are whole API docs about writing code for integration with Python.  <code>cython</code> is one of the easier tools for doing this. </p>

<p>Libraries can be any combination - pure Python, Python plus interfaces to compiled code, or all compiled.  The interpreted files end with <code>.py</code>, the compiled stuff usually is <code>.so</code> or <code>.dll</code> (depending on the operating system).  It's easy to install pure Python code - just load, unzip if needed, and put the right directory.  Mixed code requires a compilation step (and hence a c compiler, etc), or downloading a version with binaries.</p>

<p>Typically developers get the code working in Python, and then rewrite speed sensitive portions in <code>c</code>.  Or they find some external library of working <code>c</code> or <code>Fortran</code> code, and link to that.</p>

<p><code>numpy</code> and <code>scipy</code> are mixed.  They have lots of Python code, core compiled portions, and use external libraries.  And the <code>c</code> code can be extraordinarily hard to read.</p>

<p>As a <code>numpy</code> user, you should first try to get as much clarity and performance with Python code.  Most of the optimization SO questions discuss ways of making use of the compiled functionality of <code>numpy</code> - all the operations that work on whole arrays.  It's only when you can't express your operations in efficient numpy code that you need to resort to using a tool like <code>cython</code> or <code>numba</code>.</p>

<p>In general if you have to iterate extensively then you are using low level operations.  Either replace the loops with array operations, or rewrite the loop in cython.</p>

python numpy compilation