Jorge, pml/ucx used to be selected when no fast interconnect were detected (since ucx provides driver for both TCP and shared memory). These providers are now disabled by default, so unless your machine has a supported fast interconnect (such as Infiniband), pml/ucx cannot be used out of the box anymore.
if you really want to use pml/ucx on your notebook, you need to manually re-enable these providers. That being said, your best choice here is really not to force any pml, and let Open MPI use pml/ob1 (that has support for both TCP and shared memory) Cheers, Gilles On Sat, May 29, 2021 at 11:19 AM Jorge D'Elia via users <users@lists.open-mpi.org> wrote: > > Hi, > > We routinely build OpenMPI on x86_64-pc-linux-gnu machines from > the sources using gcc and usually everything works fine. > > In one case we recently installed Fedora 34 from scratch on an > ASUS G53SX notebook (Intel Core i7-2630QM CPU 2.00GHz ×4 cores, > without any IB device). Next we build OpenMPI using the file > openmpi-4.1.1.tar.gz and the GCC 12.0.0 20210524 (experimental) > compiler. > > However, when trying to experiment OpenMPI using UCX > with a simple test, we get the runtime errors: > > No components were able to be opened in the btl framework. > PML ucx cannot be selected > > while the test worked fine until Fedora 33 on the same > machine using the same OpenMPI configuration. > > We attach below some info about a simple test run. > > Please, any clues where to check or maybe something is missing? > Thanks in advance. > > Regards > Jorge. > > -- > $ cat /proc/version > Linux version 5.12.7-300.fc34.x86_64 > (mockbu...@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 11.1.1 20210428 (Red > Hat 11.1.1-1), GNU ld version 2.35.1-41.fc34) #1 SMP Wed May 26 12:58:58 UTC > 2021 > > $ mpifort --version > GNU Fortran (GCC) 12.0.0 20210524 (experimental) > Copyright (C) 2021 Free Software Foundation, Inc. > > $ which mpifort > /usr/beta/openmpi/bin/mpifort > > $ mpifort -o hello_usempi_f08.exe hello_usempi_f08.f90 > > $ mpirun --mca orte_base_help_aggregate 0 --mca btl self,vader,tcp --map-by > node --report-bindings --machinefile ~/machi-openmpi.dat --np 2 > hello_usempi_f08.exe > [verne:200650] MCW rank 0 bound to socket 0[core 0[hwt 0]]: [B/././.] > [verne:200650] MCW rank 1 bound to socket 0[core 1[hwt 0]]: [./B/./.] > Hello, world, I am 0 of 2: Open MPI v4.1.1, package: Open MPI bigpack@verne > Distribution, ident: 4.1.1, repo rev: v4.1.1, Apr 24, 2021 > Hello, world, I am 1 of 2: Open MPI v4.1.1, package: Open MPI bigpack@verne > Distribution, ident: 4.1.1, repo rev: v4.1.1, Apr 24, 2021 > > $ mpirun --mca orte_base_help_aggregate 0 --mca pml ucx --mca btl > ^self,vader,tcp --map-by node --report-bindings --machinefile > ~/machi-openmpi.dat --np 2 hello_usempi_f08.exe > [verne:200772] MCW rank 0 bound to socket 0[core 0[hwt 0]]: [B/././.] > [verne:200772] MCW rank 1 bound to socket 0[core 1[hwt 0]]: [./B/./.] > -------------------------------------------------------------------------- > No components were able to be opened in the btl framework. > > This typically means that either no components of this type were > installed, or none of the installed components can be loaded. > Sometimes this means that shared libraries required by these > components are unable to be found/loaded. > > Host: verne > Framework: btl > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > No components were able to be opened in the btl framework. > > This typically means that either no components of this type were > installed, or none of the installed components can be loaded. > Sometimes this means that shared libraries required by these > components are unable to be found/loaded. > > Host: verne > Framework: btl > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > No components were able to be opened in the pml framework. > > This typically means that either no components of this type were > installed, or none of the installed components can be loaded. > Sometimes this means that shared libraries required by these > components are unable to be found/loaded. > > Host: verne > Framework: pml > -------------------------------------------------------------------------- > [verne:200777] PML ucx cannot be selected > -------------------------------------------------------------------------- > No components were able to be opened in the pml framework. > > This typically means that either no components of this type were > installed, or none of the installed components can be loaded. > Sometimes this means that shared libraries required by these > components are unable to be found/loaded. > > Host: verne > Framework: pml > -------------------------------------------------------------------------- > [verne:200772] PMIX ERROR: UNREACHABLE in file > ../../../../../../../opal/mca/pmix/pmix3x/pmix/src/server/pmix_server.c at > line 2198 > > > $ ompi_info | grep ucx > Configure command line: '--enable-ipv6' '--enable-sparse-groups' > '--enable-mpi-ext' '--enable-mpi-cxx' '--enable-oshmem' > '--with-libevent=internal' '--with-ucx' '--with-pmix=internal' > '--without-libfabric' '--prefix=/usr/beta/openmpi' > MCA osc: ucx (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA pml: ucx (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > > $ ompi_info --param all all --level 9 | grep ucx > MCA osc: ucx (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA pml: ucx (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA osc ucx: --------------------------------------------------- > MCA osc ucx: parameter "osc_ucx_priority" (current value: "60", > data source: default, level: 3 user/all, type: unsigned_int) > Priority of the osc/ucx component (default: 60) > MCA osc ucx: parameter "osc_ucx_verbose" (current value: "0", > data source: default, level: 3 user/all, type: int, synonym of: > opal_common_ucx_verbose) > MCA osc ucx: parameter "osc_ucx_progress_iterations" (current > value: "100", data source: default, level: 3 user/all, type: int, synonym of: > opal_common_ucx_progress_iterations) > MCA osc ucx: parameter "osc_ucx_opal_mem_hooks" (current value: > "false", data source: default, level: 3 user/all, type: bool, synonym of: > opal_common_ucx_opal_mem_hooks) > MCA osc ucx: parameter "osc_ucx_tls" (current value: > "rc_verbs,ud_verbs,rc_mlx5,dc_mlx5,cuda_ipc,rocm_ipc", data source: default, > level: 3 user/all, type: string, synonym of: opal_common_ucx_tls) > MCA osc ucx: parameter "osc_ucx_devices" (current value: "mlx*", > data source: default, level: 3 user/all, type: string, synonym of: > opal_common_ucx_devices) > MCA pml ucx: --------------------------------------------------- > MCA pml ucx: parameter "pml_ucx_priority" (current value: "51", > data source: default, level: 3 user/all, type: int) > MCA pml ucx: parameter "pml_ucx_num_disconnect" (current value: > "1", data source: default, level: 3 user/all, type: int) > MCA pml ucx: parameter "pml_ucx_verbose" (current value: "0", > data source: default, level: 3 user/all, type: int, synonym of: > opal_common_ucx_verbose) > MCA pml ucx: parameter "pml_ucx_progress_iterations" (current > value: "100", data source: default, level: 3 user/all, type: int, synonym of: > opal_common_ucx_progress_iterations) > MCA pml ucx: parameter "pml_ucx_opal_mem_hooks" (current value: > "false", data source: default, level: 3 user/all, type: bool, synonym of: > opal_common_ucx_opal_mem_hooks) > MCA pml ucx: parameter "pml_ucx_tls" (current value: > "rc_verbs,ud_verbs,rc_mlx5,dc_mlx5,cuda_ipc,rocm_ipc", data source: default, > level: 3 user/all, type: string, synonym of: opal_common_ucx_tls) > MCA pml ucx: parameter "pml_ucx_devices" (current value: "mlx*", > data source: default, level: 3 user/all, type: string, synonym of: > opal_common_ucx_devices) > > $ ompi_info > Package: Open MPI bigpack@verne Distribution > Open MPI: 4.1.1 > Open MPI repo revision: v4.1.1 > Open MPI release date: Apr 24, 2021 > Open RTE: 4.1.1 > Open RTE repo revision: v4.1.1 > Open RTE release date: Apr 24, 2021 > OPAL: 4.1.1 > OPAL repo revision: v4.1.1 > OPAL release date: Apr 24, 2021 > MPI API: 3.1.0 > Ident string: 4.1.1 > Prefix: /usr/beta/openmpi > Configured architecture: x86_64-pc-linux-gnu > Configure host: verne > Configured by: bigpack > Configured on: Tue May 25 17:16:38 UTC 2021 > Configure host: verne > Configure command line: '--enable-ipv6' '--enable-sparse-groups' > '--enable-mpi-ext' '--enable-mpi-cxx' > '--enable-oshmem' '--with-libevent=internal' > '--with-ucx' '--with-pmix=internal' > '--without-libfabric' '--prefix=/usr/beta/openmpi' > Built by: bigpack > Built on: Tue 25 May 17:57:46 UTC 2021 > Built host: verne > C bindings: yes > C++ bindings: yes > Fort mpif.h: yes (all) > Fort use mpi: yes (full: ignore TKR) > Fort use mpi size: deprecated-ompi-info-value > Fort use mpi_f08: yes > Fort mpi_f08 compliance: The mpi_f08 module is available, but due to > limitations in the gfortran compiler and/or Open > MPI, does not support the following: array > subsections, direct passthru (where possible) to > underlying Open MPI's C functionality > Fort mpi_f08 subarrays: no > Java bindings: no > Wrapper compiler rpath: runpath > C compiler: gcc > C compiler absolute: /usr/beta/gcc-trunk/bin/gcc > C compiler family name: GNU > C compiler version: 12.0.0 > C++ compiler: g++ > C++ compiler absolute: /usr/beta/gcc-trunk/bin/g++ > Fort compiler: gfortran > Fort compiler abs: /usr/beta/gcc-trunk/bin/gfortran > Fort ignore TKR: yes (!GCC$ ATTRIBUTES NO_ARG_CHECK ::) > Fort 08 assumed shape: yes > Fort optional args: yes > Fort INTERFACE: yes > Fort ISO_FORTRAN_ENV: yes > Fort STORAGE_SIZE: yes > Fort BIND(C) (all): yes > Fort ISO_C_BINDING: yes > Fort SUBROUTINE BIND(C): yes > Fort TYPE,BIND(C): yes > Fort T,BIND(C,name="a"): yes > Fort PRIVATE: yes > Fort PROTECTED: yes > Fort ABSTRACT: yes > Fort ASYNCHRONOUS: yes > Fort PROCEDURE: yes > Fort USE...ONLY: yes > Fort C_FUNLOC: yes > Fort f08 using wrappers: yes > Fort MPI_SIZEOF: yes > C profiling: yes > C++ profiling: yes > Fort mpif.h profiling: yes > Fort use mpi profiling: yes > Fort use mpi_f08 prof: yes > C++ exceptions: no > Thread support: posix (MPI_THREAD_MULTIPLE: yes, OPAL support: yes, > OMPI progress: no, ORTE progress: yes, Event lib: > yes) > Sparse Groups: yes > Internal debug support: no > MPI interface warnings: yes > MPI parameter check: runtime > Memory profiling support: no > Memory debugging support: no > dl support: yes > Heterogeneous support: no > mpirun default --prefix: no > MPI_WTIME support: native > Symbol vis. support: yes > Host topology support: yes > IPv6 support: yes > MPI1 compatibility: no > MPI extensions: affinity, cuda, pcollreq > FT Checkpoint support: no (checkpoint thread: no) > C/R Enabled Debugging: no > MPI_MAX_PROCESSOR_NAME: 256 > MPI_MAX_ERROR_STRING: 256 > MPI_MAX_OBJECT_NAME: 64 > MPI_MAX_INFO_KEY: 36 > MPI_MAX_INFO_VAL: 256 > MPI_MAX_PORT_NAME: 1024 > MPI_MAX_DATAREP_STRING: 128 > MCA allocator: basic (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA allocator: bucket (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA backtrace: execinfo (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA btl: self (MCA v2.1.0, API v3.1.0, Component v4.1.1) > MCA btl: openib (MCA v2.1.0, API v3.1.0, Component v4.1.1) > MCA btl: tcp (MCA v2.1.0, API v3.1.0, Component v4.1.1) > MCA btl: vader (MCA v2.1.0, API v3.1.0, Component v4.1.1) > MCA compress: bzip (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA compress: gzip (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA crs: none (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA dl: dlopen (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA event: libevent2022 (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA hwloc: external (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA if: linux_ipv6 (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA if: posix_ipv4 (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA installdirs: env (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA installdirs: config (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA memory: patcher (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA mpool: hugepage (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA patcher: overwrite (MCA v2.1.0, API v1.0.0, Component > v4.1.1) > MCA pmix: isolated (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA pmix: flux (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA pmix: pmix3x (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA pstat: linux (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA rcache: grdma (MCA v2.1.0, API v3.3.0, Component v4.1.1) > MCA reachable: weighted (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA shmem: mmap (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA shmem: posix (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA shmem: sysv (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA timer: linux (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA errmgr: default_app (MCA v2.1.0, API v3.0.0, Component > v4.1.1) > MCA errmgr: default_hnp (MCA v2.1.0, API v3.0.0, Component > v4.1.1) > MCA errmgr: default_orted (MCA v2.1.0, API v3.0.0, Component > v4.1.1) > MCA errmgr: default_tool (MCA v2.1.0, API v3.0.0, Component > v4.1.1) > MCA ess: env (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA ess: hnp (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA ess: pmi (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA ess: singleton (MCA v2.1.0, API v3.0.0, Component > v4.1.1) > MCA ess: tool (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA ess: slurm (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA filem: raw (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA grpcomm: direct (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA iof: hnp (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA iof: orted (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA iof: tool (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA odls: default (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA odls: pspawn (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA oob: tcp (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA plm: isolated (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA plm: rsh (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA plm: slurm (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA ras: simulator (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA ras: slurm (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA regx: fwd (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA regx: naive (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA regx: reverse (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA rmaps: mindist (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA rmaps: ppr (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA rmaps: rank_file (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA rmaps: resilient (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA rmaps: round_robin (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA rmaps: seq (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA rml: oob (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA routed: binomial (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA routed: direct (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA routed: radix (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA rtc: hwloc (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA schizo: flux (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA schizo: ompi (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA schizo: orte (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA schizo: jsm (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA schizo: slurm (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA state: app (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA state: hnp (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA state: novm (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA state: orted (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA state: tool (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA bml: r2 (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: adapt (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: basic (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: han (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: inter (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: libnbc (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: self (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: sm (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: sync (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: tuned (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA coll: monitoring (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA fbtl: posix (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA fcoll: dynamic (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA fcoll: dynamic_gen2 (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA fcoll: individual (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA fcoll: two_phase (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA fcoll: vulcan (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA fs: ufs (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA io: ompio (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA io: romio321 (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA op: avx (MCA v2.1.0, API v1.0.0, Component v4.1.1) > MCA osc: sm (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA osc: monitoring (MCA v2.1.0, API v3.0.0, Component > v4.1.1) > MCA osc: pt2pt (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA osc: rdma (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA osc: ucx (MCA v2.1.0, API v3.0.0, Component v4.1.1) > MCA pml: v (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA pml: cm (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA pml: monitoring (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA pml: ob1 (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA pml: ucx (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA rte: orte (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA sharedfp: individual (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA sharedfp: lockedfile (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > MCA sharedfp: sm (MCA v2.1.0, API v2.0.0, Component v4.1.1) > MCA topo: basic (MCA v2.1.0, API v2.2.0, Component v4.1.1) > MCA topo: treematch (MCA v2.1.0, API v2.2.0, Component > v4.1.1) > MCA vprotocol: pessimist (MCA v2.1.0, API v2.0.0, Component > v4.1.1) > > #end
