What's the closest I can get to calling a Python function using a different Python version?
<p>Say I have two files:</p>

<pre><code># spam.py
import library_Python3_only as l3

def spam(x,y)
    return l3.bar(x).baz(y)
</code></pre>

<p>and</p>

<pre><code># beans.py
import library_Python2_only as l2

...
</code></pre>

<p>Now suppose I wish to call <code>spam</code> from within <code>beans</code>. It's not directly possible since both files depend on incompatible Python versions. Of course I can <code>Popen</code> a different python process, but how could I pass in the arguments and retrieve the results without too much stream-parsing pain?</p>

<p>It's possible to use the <a href="https://docs.python.org/2/library/multiprocessing.html#managers" rel="nofollow"><code>multiprocessing.managers</code></a> module to achieve what you want. It does require a small amount of hacking though.</p>

<p>Given a module that has functions you want to expose then you need to create a <code>Manager</code> that can create proxies for those functions.</p>

<p>manager process that serves proxies to the py3 functions:</p>

<pre><code>from multiprocessing.managers import BaseManager
import spam

class SpamManager(BaseManager):
    pass
# Register a way of getting the spam module.
# You can use the exposed arg to control what is exposed.
# By default only "public" functions (without a leading underscore) are exposed,
# but can only ever expose functions or methods.
SpamManager.register("get_spam", callable=(lambda: spam), exposed=["add", "sub"])

# specifying the address as localhost means the manager is only visible to  
# processes on this machine
manager = SpamManager(address=('localhost', 50000), authkey=b'abc', 
    serializer='xmlrpclib')
server = manager.get_server()
server.serve_forever()
</code></pre>

<p>I've redefined <code>spam</code> to contain two function called <code>add</code> and <code>sub</code>.</p>

<pre><code># spam.py
def add(x, y):
    return x + y

def sub(x, y):
    return x - y
</code></pre>

<p>client process that uses the py3 functions exposed by the <code>SpamManager</code>.</p>

<pre><code>from __future__ import print_function
from multiprocessing.managers import BaseManager

class SpamManager(BaseManager):
    pass
SpamManager.register("get_spam")

m = SpamManager(address=('localhost', 50000), authkey=b'abc', 
    serializer='xmlrpclib')
m.connect()

spam = m.get_spam()
print("1 + 2 = ", spam.add(1, 2)) # prints 1 + 2 = 3
print("1 - 2 = ", spam.sub(1, 2)) # prints 1 - 2 = -1
spam.__name__ # Attribute Error -- spam is a module, but its __name__ attribute 
# is not exposed
</code></pre>

<p>Once set up, this form gives an easy way of accessing functions and values. It also allows these functions and values to be used them in a similar way that you might use them if they were not proxies. Finally, it allows you to set a password on the server process so that only authorised processes can access the manager. That the manager is long running, also means that a new process doesn't have to be started for each function call you make.</p>

<p>One limitation is that I've used the <code>xmlrpclib</code> module rather than <code>pickle</code> to send data back and forth between the server and the client. This is because python2 and python3 use different protocols for <code>pickle</code>. You could fix this by adding your own client to <code>multiprocessing.managers.listener_client</code> that uses an agreed upon protocol for pickling objects.</p>

python compatibility popen