Nesting mpi calls with mpi4py
<p>I am trying to use mpi4py to call a second instance of an mpi executable.</p>

<p>I am getting the error:</p>

<pre><code>Open MPI does not support recursive calls of mpirun
</code></pre>

<p>But I was under the impression that is exactly what Spawn is supposed to be able to handle - i.e. setting up a new communicator within which another mpi command could be launched.</p>

<p>The test code:</p>

<p>parent.py:</p>

<pre><code>#!/usr/bin/env python
from mpi4py import MPI
import numpy
import sys


rank = MPI.COMM_WORLD.Get_rank()
new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
print(new_comm.Get_rank())
new_comm.Spawn(sys.executable,
                           args=['test.py'],
                           maxprocs=4)
</code></pre>

<p>which calls test.py:</p>

<pre><code>#!/usr/bin/env python
from mpi4py import MPI
import numpy
import os
import sys

comm = MPI.Comm.Get_parent()
rank = comm.Get_rank()

cwd=os.getcwd()
directory=os.path.join(cwd,str(rank))
os.chdir(directory)


os.system('{}'.format('mpirun -np 4 SOME_MPI_EXECUTABLE_HERE'))



print("Finished in "+directory)
os.chdir(cwd)

comm.Disconnect()
</code></pre>

<p>I'm running with:</p>

<pre><code>mpirun --oversubscribe -np 1 parent.py
</code></pre>

<p>Using openmpi 2.0.0 with gcc, and python/3.4.2</p>

<p>Anyone have any bright ideas as to why this is happening.....</p>

<p>Thanks!</p>

<p>The following code seems to perform the way I wanted.</p>

<pre><code>#!/usr/bin/env python
from mpi4py import MPI
import numpy
import sys
import os

rank = MPI.COMM_WORLD.Get_rank()
new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
print(new_comm.Get_rank())

cwd=os.getcwd()
os.mkdir(str(rank))
directory=os.path.join(cwd,str(rank))
print(rank,directory)
os.chdir(directory)


new_comm.Spawn("SOME_MPI_EXECUTABLE_HERE",
                  args=[""],
                           maxprocs=4)
</code></pre>

<p>run with:</p>

<p>mpirun --oversubscribe -np 4 parent.py</p>

<p>Seems to start 4 instances of SOME_MPI_EXECUTABLE each running on 4 cores.</p>

<p>(Thanks to Zulan)</p>

python parallel-processing mpi python-3.4 mpi4py