Just wanted to follow up on my own post.
Turns out there was a missing symlink (much embarrassment) on by build host.
That’s why you don’t see “pmix_v1” in the “srun —mpi=list” output (previous
post).
Once I fixed that and rebuilt SLURM, I was able to launch existing OpenMPI 3.x
apps with,
srun —mpi=pmix_v1
Apologies for the wasted bandwidth.
Regards,
Charlie
> On Jun 28, 2018, at 8:14 AM, Charles A Taylor <chas...@ufl.edu> wrote:
>
> There is a name for my pain and it is “OpenMPI + PMIx”. :)
>
> I’m looking at upgrading SLURM from 16.05.11 to 17.11.05 (bear with me, this
> is not a SLURM question).
>
> After building SLURM 17.11.05 with
> ‘--with-pmix=/opt/pmix/1.1.5:/opt/pmix/2.1/1’ and installing a test instance,
> I see
>
> $ srun --mpi=list
> srun: MPI types are...
> srun: pmix
> srun: pmi2
> srun: pmix_v2
> srun: none
> srun: openmpi
>
> Seems reasonable.
>
> Now, we have applications built with OpenMPI 3.0.0 and 3.1.0 linked against
> /opt/pmix/1.1.5 (--with-pmix=/opt/pmix/1.1.5). When I attempt to launch
> these applications using,
>
> srun —mpi=pmix <some mpi app>
>
> I get the following ...
>
> [c1a-s18.ufhpc:17995] Security mode none is not available
> [c1a-s18.ufhpc:17995] PMIX ERROR: UNREACHABLE in file
> src/client/pmix_client.c at line 199
> --------------------------------------------------------------------------
> The application appears to have been direct launched using "srun",
> but OMPI was not built with SLURM's PMI support and therefore cannot
> execute. There are several options for building PMI support under
> SLURM, depending upon the SLURM version you are using:
>
> version 16.05 or later: you can use SLURM's PMIx support. This
> requires that you configure and build SLURM --with-pmix.
>
> Versions earlier than 16.05: you must use either SLURM's PMI-1 or
> PMI-2 support. SLURM builds PMI-1 by default, or you can manually
> install PMI-2. You must then build Open MPI using --with-pmi pointing
> to the SLURM PMI library location.
>
> Please configure as appropriate and try again.
> --------------------------------------------------------------------------
> *** An error occurred in MPI_Init
> *** on a NULL communicator
> *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
> *** and potentially your MPI job)
> ———————————————————————————————————————————————————————
>
> So slurm/srun appear to have library support for both pmix and pmix_v2 and
> OpenMPI 3.0.0 and OpenMPI 3.1.0 both have pmix support (1.1.5) since we
> launch them every day with “srun —mpi=pmix” under slurm 16.05.11.
>
> Is this a bug? Am I overlooking something? Is it possible to transition to
> OpenMPI 3.x + PMIx 2.x + SLURM 17.x without rebuilding (essentially)
> everything (including all applications)?
>
> Charlie Taylor
> UF Research Computing
>
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
