Hello everybody, we are seeing the sypmptoms described in https://github.com/open-mpi/ompi/issues/11557
However, according to the systems package manager (dnf) all munge related packages on the build node and the execution node are identical, see details at the bottom. So, the explanation given by Ralph Castain in the git issue, which I read to refer to a missing munge library, does not appear to explain the warning. I would like to note that a PMIX_MCA_psec=native is sufficient to make the warning go away, it is not necessary to disable munge completely. Launching with srun --mpi=pmi2 instead works fine, anyway. Related to that: pmix has a configure switch to disable munge support. Would it be possible and/or adviseable to disable munge in the pmix build ? I have to note that we are seeing also PMIX ERROR: ERROR in file gds_ds12_lock_pthread.c at line 168 for each rank launched. This might be a separate issue or related. Setting PMIX_MCA_gds=^ds12 makes this go away. With or without setting PMIX_MCA_psec and PMIX_MCA_gds the job launches and is executed. Still, I would like to understand this better in case we have a broken slurm or mpi setup which would need to be corrected. Best Regards Christof System Details: Rocky Linux 9.1, slurm 23.02.2, open pmix 4.2.4rc1, openmpi 4.1.5. slurm and pmix were compiled in the same step to rpms in a mock build environment using the spec files included with them. I can provide the build logs of the rpms. Both are not from distribution repositories. openmpi was configured with ./configure --enable-mpi1-compatibility --enable-orterun-prefix-by-default --with-ofi=/cluster/libraries/libfabric/1.18.0/ --with-slurm --with-pmix --with-pmix-libdir=/usr/lib64 --with-pmi --with-pmi-libdir=/usr/lib64 Installed munge packages from distribution according to dnf on build node munge.x86_64 0.5.13-13.el9 munge-devel.x86_64 0.5.13-13.el9 munge-libs.x86_64 0.5.13-13.el9 on execution node munge.x86_64 0.5.13-13.el9 munge-devel.x86_64 0.5.13-13.el9 munge-libs.x86_64 0.5.13-13.el9 -- Dr. rer. nat. Christof Köhler email: c.koeh...@uni-bremen.de Universitaet Bremen/FB1/BCCMS phone: +49-(0)421-218-62334 Am Fallturm 1/ TAB/ Raum 3.06 fax: +49-(0)421-218-62770 28359 Bremen
