Barrier after spawned mpi4py process
<p>I have a piece of code that uses mpi4py to spawn several instances of an mpi exectuable. I want the code to halt while these processes complete, and then call a second group of the same executable.</p>

<p>The problem is that all calls to the mpi executable get spawned immediately.</p>

<p>It seems like there is no way to use a barrier to prevent this from happening. Does anyone know if this is correct, and if so does anyone have a bright idea to get the result I need.</p>

<pre><code>#!/usr/bin/env python
from mpi4py import MPI
import numpy
import sys
import os

rank = MPI.COMM_WORLD.Get_rank()
new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
print(new_comm.Get_rank())

cwd=os.getcwd()
directory=os.path.join(cwd,str(rank))
print(rank,directory)
os.chdir(directory)


new_comm.Spawn("SOME_F90_MPI_EXECUTABLE",
                  args=None,
                           maxprocs=4)

'''I want to pause here until the spawned processes finish running...'''
new_comm.Barrier()
MPI.COMM_WORLD.Barrier()

print(new_comm.Get_rank())

cwd=os.getcwd()
directory=os.path.join(cwd,str(rank))
print(rank,directory)
os.chdir(directory+"_2")


new_comm.Spawn("SOME_F90_MPI_EXECUTABLE",
                  args=["&gt; output"],
                           maxprocs=4)

new_comm.Disconnect()

print("All finished here.....")
</code></pre>

<p>Thanks!</p>

<p>You must use the <em>intercommunicator</em> returned by <code>Spawn</code>:</p>

<pre><code>child_comm = MPI.COMM_WORLD.Spawn("./spawnchild.py", args=None, maxprocs=2)
child_comm.Barrier()
</code></pre>

<p>For the child, get the parent <em>intercommunicator</em> (similarly in Fortran):</p>

<pre><code>parent = MPI.COMM_WORLD.Get_parent()
assert(parent != MPI.COMM_NULL)
parent.Barrier();
</code></pre>

<p>Note that an <em>intercommunicator</em>, which consists of processes from two groups, behaves different than a traditional <em>intercommunicator</em>. E.g. for the Barrier, the following semantics apply:</p>

<blockquote>
  <p>If comm is an intercommunicator, <code>MPI_BARRIER</code> involves two groups. The call returns at processes in one group (group A) of the intercommunicator only after all members of the other group (group B) have entered the call (and vice versa). A process may return from the call before all processes in its own group have entered the call.</p>
</blockquote>

<p>(<a href="http://mpi-forum.org/docs/" rel="nofollow">MPI 3.1 Standard</a>, 5.3)</p>

python parallel-processing mpi mpi4py