PYTHONPATH order on Ubuntu 14.04
<p>I have two computers running Ubuntu 14.04 server (let's call them A and B). B was initially a 10.04 but it has received two upgrades to 12.04 and 14.04. I do not understand why the python path is different on the two computers.</p>

<p>As you can see on the two paths below, the pip installation path <code>/usr/local/lib/python2.7/dist-packages</code> comes <strong>before</strong> the apt python packages path <code>/usr/lib/python2.7/dist-packages</code> on Ubuntu A, but it comes <strong>after</strong> on Ubuntu B.</p>

<p>This leads to several problems if a python package is installed both via apt and pip. As you can see below, if both <code>python-six</code> apt package and <code>six</code> pip package are installed, they may be two different library versions.</p>

<p>The installation of packages system is not always my choice, but might be some dependencies of other packages that are installed.</p>

<p>This problem could probably be solved with a virtualenv, but for reasons I will not detail, <strong>I cannot use virtualenv</strong> here, and must install pip packages system-wide.</p>

<h2>Ubuntu A</h2>

<pre><code>&gt;&gt;&gt; import sys, six
&gt;&gt;&gt; sys.path
['',
 '/usr/local/bin',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-x86_64-linux-gnu',
 '/usr/lib/python2.7/lib-tk',
 '/usr/lib/python2.7/lib-old',
 '/usr/lib/python2.7/lib-dynload',
 '/usr/local/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages/PILcompat',
 '/usr/local/lib/python2.7/dist-packages/IPython/extensions']
&gt;&gt;&gt; six
&lt;module 'six' from '/usr/local/lib/python2.7/dist-packages/six.pyc'&gt;
</code></pre>

<h2>Ubuntu B</h2>

<pre><code>&gt;&gt;&gt; import sys, six
&gt;&gt;&gt; sys.path
['',
 '/usr/local/bin',
 '/usr/lib/python2.7/dist-packages',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-x86_64-linux-gnu',
 '/usr/lib/python2.7/lib-tk',
 '/usr/lib/python2.7/lib-old',
 '/usr/lib/python2.7/lib-dynload',
 '/usr/local/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages/PILcompat',
 '/usr/local/lib/python2.7/dist-packages/IPython/extensions']
&gt;&gt;&gt; six
&gt;&gt;&gt; &lt;module 'six' from '/usr/lib/python2.7/dist-packages/six.pyc'&gt;
</code></pre>

<p>For both machines <code>$PATH</code> is the same, and <code>$PYTHONPATH</code> is empty.</p>

<ul>
<li><p>Why are those PYTHONPATHS different?</p></li>
<li><p>How can I fix the pythonpath order in "Ubuntu B" so it will load pip
packages before the system ones, in a system-wide way? Is there a apt package I should reinstall or reconfigure so the
PYTHONPATH would prioritize pip packages ?</p></li>
</ul>

<p>As we cannot explore into your system, I am trying to analysis your first question by illustrating how <code>sys.path</code> is initialized. Available references are <a href="http://mikeboers.com/blog/2014/05/23/where-does-the-sys-path-start" rel="nofollow">where-does-sys-path-starts</a> and <a href="http://pyvideo.org/pycon-us-2011/pycon-2011--reverse-engineering-ian-bicking--39-s.html" rel="nofollow">pyco-reverse-engineering</a>(python2.6).</p>

<p>The <code>sys.path</code> comes from the following variables(in order):</p>

<ol>
<li><code>$PYTHONPATH</code> (highest priority)</li>
<li><code>sys.prefix</code>-ed stdlib</li>
<li><code>sys.exec_prefix</code>-ed stdlib</li>
<li><code>site-packages</code></li>
<li><code>*.pth</code> in site-packages (lowest priority)</li>
</ol>

<p>Now let's describe each of these variables:</p>

<ol>
<li><code>$PYTHONPATH</code>, this is just a system environment variable.</li>
<li>&amp; 3. <code>sys.prefix</code> and <code>sys.exec_prefix</code> are determined before any python script is executed. It is actually coded in the source <a href="https://hg.python.org/cpython/file/e5d963cb6afc/Modules/getpath.c#l314" rel="nofollow">Module/getpath.c</a>. </li>
</ol>

<p>The logic is like this: </p>

<pre><code>IF $PYTHONHOME IS set:
    RETURN sys.prefix AND sys.exec_prefix as $PYTHONHOME
ELSE:
    current_dir = directory of python executable;
    DO:
        current_dir = parent(current_dir)
        IF FILE 'lib/pythonX.Y/os.py' EXSITS:
            sys.prefix = current_dir
        IF FILE 'lib/pythonX.Y/lib-dynload' EXSITS:
            sys.exec_prefix = current_dir
        IF current_dir IS '/':
            BREAK
    WHILE(TRUE)
    IF sys.prefix IS NOT SET:
       sys.prefix = BUILD_PREFIX
    IF sys.exec_prefix IS NOT SET:
       sys.exec_prefix = BUILD_PREFIX
</code></pre>

<ol start="4">
<li><p>&amp; 5. <code>site-packages</code> and <code>*.pth</code> are added by import of <code>site.py</code>. In this module you will find the docs:</p>

<p>This will append site-specific paths to the module search path.  On
Unix (including Mac OSX), it starts with sys.prefix and
sys.exec_prefix (if different) and appends
lib/python/site-packages as well as lib/site-python.
... ...</p>

<p>For Debian and derivatives, this sys.path is augmented with directories
for packages distributed within the distribution. Local addons go
into /usr/local/lib/python/dist-packages, Debian addons
install into /usr/{lib,share}/python/dist-packages.
/usr/lib/python/site-packages is not used.</p>

<p>A path configuration file is a file whose name has the form
.pth; its contents are additional directories (one per line)
to be added to sys.path.
... ...</p></li>
</ol>

<p>And a code snippet for important function <code>getsitepackages</code>:</p>

<pre><code>sitepackages.append(os.path.join(prefix, "local/lib",
                            "python" + sys.version[:3],
                            "dist-packages"))
sitepackages.append(os.path.join(prefix, "lib",
                            "python" + sys.version[:3],
                            "dist-packages"))
</code></pre>

<p>Now I try to fig out where may be this odd problem comes from:</p>

<ol>
<li><code>$PYTHONPATH</code>, impossible, because it is empty both A and B</li>
<li><code>sys.prefix</code> and <code>sys.exec_prefix</code>, maybe, please check them and as well as <code>$PYTHONHOME</code></li>
<li><code>site.py</code>, maybe, check the file.</li>
</ol>

<p>The <code>sys.path</code> output of B is quite odd, <code>dist-package</code> (<code>site-package</code>) goes before <code>sys.exec_prefix</code> (<code>lib-dynload</code>). Please try to investigate  each step of <code>sys.path</code> initialization of machine B, you may find out something.</p>

<p>Very sorry that I cannot replicate your problem. By the way, about your question title, I think <code>SYS.PATH</code> is better than <code>PYTHONPATH</code>, which makes me misinterpretation as <code>$PYTHONPATH</code> at first glance.</p>

python ubuntu ubuntu-14.04 pythonpath ubuntu-server