Hello, Earlier this week I finally managed to Spack build OpenMPI 5.0 on an LSF cluster I've got access to. I used to use a manually build version before for years. After working with Spack version for a while, I noticed my program arguments are passed to mpirun instead e.g.
mpirun hostname -i
--------------------------------------------------------------------------
An unrecognized option was included on the mpirun command line:
Option: -i
Please use the "mpirun --help" command to obtain a list of all
supported options.
--------------------------------------------------------------------------
This is not consistent with what man pages say about SPMD model:
mpirun [ options ] <program> [ <args> ]
Adding any option to mpirun or using '--' before the program make the
problem go away, but OpenMPI compiled manually doesn't have this
behaviour. I then also compiled Spack without LSF integration and the
problem persisted so I suspect Spack has some default options which
force this behaviour. Are you able to advise what is happening here?
Also, these are the Spack specs I used
ucx@1.15+cma+dc+knem+mlx5_dv+rc+ud+verbs
openmpi+internal-pmix fabrics=ucx schedulers=lsf
I am attaching output of ompi_info too. Spack is quite explicit about
options for Automake packages so I know I don't have some of those
switches in my manual installations.
Please let me know.
Best wishes,
Robert
--
For those seeking to enter the felicity of matrimonial alliance n/e is the
answer.
http://en.wikipedia.org/wiki/Secretary_problem
Package: Open MPI rrs59-uxq05@sqg1cintr19.bullx Distribution
Open MPI: 5.0.3
Open MPI repo revision: v5.0.3
Open MPI release date: Apr 08, 2024
MPI API: 3.1.0
Ident string: 5.0.3
Prefix:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-13.2.0/openmpi-5.0.3-6fhewxt5qt4wu7alowbi4ryjfkb7tgs6
Configured architecture: x86_64-pc-linux-gnu
Configured by: rrs59-uxq05
Configured on: Wed Jun 5 20:29:43 UTC 2024
Configure host: sqg1cintr19.bullx
Configure command line:
'--prefix=/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-13.2.0/openmpi-5.0.3-6fhewxt5qt4wu7alowbi4ryjfkb7tgs6'
'--enable-shared' '--disable-silent-rules' '--disable-sphinx'
'--disable-builtin-atomics' '--disable-static' '--enable-mpi1-compatibility'
'--without-psm' '--without-ucc' '--without-hcoll' '--without-mxm'
'--without-fca' '--without-psm2' '--without-ofi'
'--with-ucx=/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-13.2.0/ucx-1.15.0-tkkt6na2mz3recxoyihcjlezb3zrdnbq'
'--without-xpmem' '--without-verbs' '--without-cma' '--without-knem'
'--without-cray-xpmem' '--without-alps' '--without-tm' '--without-sge'
'--with-lsf=/opt/lsf/10.1' '--without-slurm' '--without-loadleveler'
'--with-lsf-libdir=/opt/lsf/10.1/linux3.10-glibc2.17-x86_64/lib'
'--disable-memchecker'
'--with-libevent=/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-13.2.0/libevent-2.1.12-bccpnnmloxaxlquqsz5xrcymklbip5xe'
'--with-pmix=internal'
'--with-zlib=/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-13.2.0/zlib-ng-2.1.6-tu6xy2c56voldrdswzdz75nph6yc4krn'
'--with-hwloc=/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-13.2.0/hwloc-2.9.1-4foxjecwuixe3e4gxbtskec33spf36cy'
'--disable-java' '--disable-mpi-java' '--disable-io-romio' '--with-gpfs=no'
'--without-cuda' '--enable-wrapper-rpath' '--disable-wrapper-runpath'
'--with-wrapper-ldflags=-Wl,-rpath,/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-8.1.0/gcc-13.2.0-p6hcdy6c2wnf3blobrbm2hlqki4pdajw/lib/gcc/x86_64-pc-linux-gnu/13.2.0
-Wl,-rpath,/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/opt/spack/linux-rhel7-broadwell/gcc-8.1.0/gcc-13.2.0-p6hcdy6c2wnf3blobrbm2hlqki4pdajw/lib64'
'CFLAGS=-DYY_BUF_SIZE=1048576'
Built by: rrs59-uxq05
Built on: Wed Jun 5 20:37:03 UTC 2024
Built host: sqg1cintr19.bullx
C bindings: yes
Fort mpif.h: yes (all)
Fort use mpi: yes (full: ignore TKR)
Fort use mpi size: deprecated-ompi-info-value
Fort use mpi_f08: yes
Fort mpi_f08 compliance: The mpi_f08 module is available, but due to
limitations in the
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/gfortran
compiler and/or Open MPI, does not support the following: array subsections,
direct passthru (where possible) to underlying Open MPI's C functionality
Fort mpi_f08 subarrays: no
Java bindings: no
Wrapper compiler rpath: rpath
C compiler:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/gcc
C compiler absolute:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/gcc
C compiler family name: GNU
C compiler version: 13.2.0
C++ compiler:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/g++
C++ compiler absolute:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/g++
Fort compiler:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/gfortran
Fort compiler abs:
/netfs/smain01/scafellpike/local/HT05650/uxq05/shared/spack/lib/spack/env/gcc/gfortran
Fort ignore TKR: yes (!GCC$ ATTRIBUTES NO_ARG_CHECK ::)
Fort 08 assumed shape: yes
Fort optional args: yes
Fort INTERFACE: yes
Fort ISO_FORTRAN_ENV: yes
Fort STORAGE_SIZE: yes
Fort BIND(C) (all): yes
Fort ISO_C_BINDING: yes
Fort SUBROUTINE BIND(C): yes
Fort TYPE,BIND(C): yes
Fort T,BIND(C,name="a"): yes
Fort PRIVATE: yes
Fort ABSTRACT: yes
Fort ASYNCHRONOUS: yes
Fort PROCEDURE: yes
Fort USE...ONLY: yes
Fort C_FUNLOC: yes
Fort f08 using wrappers: yes
Fort MPI_SIZEOF: yes
C profiling: yes
Fort mpif.h profiling: yes
Fort use mpi profiling: yes
Fort use mpi_f08 prof: yes
Thread support: posix (MPI_THREAD_MULTIPLE: yes, OPAL support: yes,
OMPI progress: no, Event lib: yes)
Sparse Groups: no
Internal debug support: no
MPI interface warnings: yes
MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
dl support: yes
Heterogeneous support: no
MPI_WTIME support: native
Symbol vis. support: yes
Host topology support: yes
IPv6 support: no
MPI extensions: affinity, cuda, ftmpi, rocm, shortfloat
Fault Tolerance support: yes
FT MPI support: yes
MPI_MAX_PROCESSOR_NAME: 256
MPI_MAX_ERROR_STRING: 256
MPI_MAX_OBJECT_NAME: 64
MPI_MAX_INFO_KEY: 36
MPI_MAX_INFO_VAL: 256
MPI_MAX_PORT_NAME: 1024
MPI_MAX_DATAREP_STRING: 128
MCA accelerator: null (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA allocator: basic (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA allocator: bucket (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA backtrace: execinfo (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA btl: self (MCA v2.1.0, API v3.3.0, Component v5.0.3)
MCA btl: sm (MCA v2.1.0, API v3.3.0, Component v5.0.3)
MCA btl: tcp (MCA v2.1.0, API v3.3.0, Component v5.0.3)
MCA btl: uct (MCA v2.1.0, API v3.3.0, Component v5.0.3)
MCA dl: dlopen (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA if: linux_ipv6 (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA if: posix_ipv4 (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA installdirs: env (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA installdirs: config (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA memory: patcher (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA mpool: hugepage (MCA v2.1.0, API v3.1.0, Component v5.0.3)
MCA patcher: overwrite (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA rcache: grdma (MCA v2.1.0, API v3.3.0, Component v5.0.3)
MCA reachable: weighted (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA shmem: mmap (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA shmem: posix (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA shmem: sysv (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA smsc: knem (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA threads: pthreads (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA timer: linux (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA bml: r2 (MCA v2.1.0, API v2.1.0, Component v5.0.3)
MCA coll: adapt (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: basic (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: han (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: inter (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: libnbc (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: self (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: sync (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: tuned (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: ftagree (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: monitoring (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA coll: sm (MCA v2.1.0, API v2.4.0, Component v5.0.3)
MCA fbtl: posix (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA fcoll: dynamic (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA fcoll: dynamic_gen2 (MCA v2.1.0, API v2.0.0, Component
v5.0.3)
MCA fcoll: individual (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA fcoll: vulcan (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA fs: lustre (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA fs: ufs (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA hook: comm_method (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA io: ompio (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA op: avx (MCA v2.1.0, API v1.0.0, Component v5.0.3)
MCA osc: sm (MCA v2.1.0, API v3.0.0, Component v5.0.3)
MCA osc: monitoring (MCA v2.1.0, API v3.0.0, Component v5.0.3)
MCA osc: rdma (MCA v2.1.0, API v3.0.0, Component v5.0.3)
MCA osc: ucx (MCA v2.1.0, API v3.0.0, Component v5.0.3)
MCA part: persist (MCA v2.1.0, API v4.0.0, Component v5.0.3)
MCA pml: cm (MCA v2.1.0, API v2.1.0, Component v5.0.3)
MCA pml: monitoring (MCA v2.1.0, API v2.1.0, Component v5.0.3)
MCA pml: ob1 (MCA v2.1.0, API v2.1.0, Component v5.0.3)
MCA pml: ucx (MCA v2.1.0, API v2.1.0, Component v5.0.3)
MCA pml: v (MCA v2.1.0, API v2.1.0, Component v5.0.3)
MCA sharedfp: individual (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA sharedfp: lockedfile (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA sharedfp: sm (MCA v2.1.0, API v2.0.0, Component v5.0.3)
MCA topo: basic (MCA v2.1.0, API v2.2.0, Component v5.0.3)
MCA topo: treematch (MCA v2.1.0, API v2.2.0, Component v5.0.3)
MCA vprotocol: pessimist (MCA v2.1.0, API v2.0.0, Component v5.0.3)
signature.asc
Description: PGP signature
