Hi, I'm packaging UCX for Debian and uploading it now. Hopefully it will be ready soon and OpenMPI can be built against it.
Regards Alastair On 04/06/2020 10:26, Patrice RINGOT wrote: > Package: openmpi-bin > Version: 4.0.3-8 > Severity: wishlist > > Dear Maintainer, > > We use the mpiGraph (https://github.com/LLNL/mpiGraph) program to test > regularly openmpi basic functionalities on our clusters. > > For Infiniband networks, and since version 4.0.3 of openmpi available in > testing, we use the following command line to run this test: > > mpirun -v --allow-run-as-root -mca mtl_base_verbose 1 -mca btl_base_verbose 1 > -mca btl openib,self -hostfile /root/hostfile --map-by ppr:1:node ./mpiGraph > 20000000 50 1 > > along with the following configuration file > (/etc/openmpi/openmpi-mca-params.conf): > > mtl_ofi_provider_exclude = shm,sockets,tcp,udp,rstream,verbs > btl_openib_allow_ib = true > > We must add the "btl_openib_allow_ib = true" instruction to the > configuration, because: > > "By default, for Open MPI 4.0 and later, infiniband ports on a device > are not used by default. The intent is to use UCX for these devices. > You can override this policy by setting the btl_openib_allow_ib MCA parameter > to true." > > Mpirun outputs this message when this parameter is not set, and "-mca btl > ^openib" is not specified on the command line. > (in this case the IB interface is not used, and for one of our cluster, the > test runs at Gigabit speed instead of Infiniband speed) > > As UCX is the recommended solution now, we wonder why ucx is not packaged in > the testing distribution (and used for building openmpi). > > Best regards > > Patrice > > -- System Information: > Debian Release: bullseye/sid > APT prefers testing > APT policy: (500, 'testing') > Architecture: amd64 (x86_64) > > Kernel: Linux 5.6.0-2-amd64 (SMP w/32 CPU cores) > Locale: LANG=en_US.UTF8, LC_CTYPE=en_US.UTF8 (charmap=UTF-8), > LANGUAGE=en_US.UTF8 (charmap=UTF-8) > Shell: /bin/sh linked to /usr/bin/dash > Init: systemd (via /run/systemd/system) > LSM: AppArmor: enabled > > Versions of packages openmpi-bin depends on: > ii libc6 2.30-8 > ii libevent-2.1-7 2.1.11-stable-1 > ii libopenmpi3 4.0.3-8 > ii openmpi-common 4.0.3-8 > ii openssh-client [ssh-client] 1:8.2p1-4 > > openmpi-bin recommends no packages. > > Versions of packages openmpi-bin suggests: > ii gfortran [fortran-compiler] 4:9.2.1-3.1 > > -- Configuration Files: > /etc/openmpi/openmpi-mca-params.conf changed: > mtl_ofi_provider_exclude = shm,sockets,tcp,udp,rstream,verbs > btl_openib_allow_ib = true > > > -- no debconf information -- Alastair McKinstry, email: [email protected], matrix: @alastair:sceal.ie, phone: 087-6847928 Green Party Councillor, Galway County Council
