* Drew Parsons <dpars...@debian.org> [2020-01-23 19:02]:
On 2020-01-23 18:15, Thibaut Paumard wrote:
Hi,
Le 22/01/2020 à 16:49, Drew Parsons a écrit :
Might be worth bringing to the attention of the subprocess
developers. I
figure they'll either be interested to know about the problem or will
have a simple explanation.
I'm not sure subprocess is involved at all, it may be that MPI_Init()
simply dies because h5py has already called it earlier.
Hi have no time for testing right now, but a simple printf() before
calling MPI_Init() should tell.
That sounds plausible.
I am not sure this would be an explanation for the problem.
At any rate, I found a minimal example that exposes the bug. Consider
these two scripts:
#############################################################
### File name: bugtest-without-h5py.py
import subprocess
simulationProc = subprocess.Popen("mpirun",
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(stdout, stderr) = simulationProc.communicate()
returnCode = simulationProc.wait()
print("stdout = ", stdout)
print("stderr = ", stderr)
print("return code = ", returnCode)
#############################################################
#############################################################
### File name: bugtest-with-h5py.py
import subprocess
import h5py
simulationProc = subprocess.Popen("mpirun",
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(stdout, stderr) = simulationProc.communicate()
returnCode = simulationProc.wait()
print("stdout = ", stdout)
print("stderr = ", stderr)
print("return code = ", returnCode)
#############################################################
The only difference between them is the line containing "import h5py" in
the second.
Here is the result for the first script:
$ python3 bugtest-without-h5py.py
stdout = b''
stderr =
b'--------------------------------------------------------------------------\nmpirun
could not find anything to do.\n\nIt is possible that you forgot to specify how many
processes to run\nvia the "-np"
argument.\n--------------------------------------------------------------------------\n'
return code = 1
and here is the result for the second script:
$ python3 bugtest-with-h5py.py
stdout = b''
stderr = b''
return code = 1
It seems that, when h5py is imported, the mpirun command is not even
launched by subprocess.Popen. Notice that there is not even a program
that is launched by mpirun that would call MPI_Init.
This is quite weird.
Best,
Rafael
