Andrej, it seems only flux is a PMIx option, which is very suspicious.
can you check other components are available? ls -l /usr/local/lib/openmpi/mca_pmix_*.so will list them. Cheers, Gilles On Mon, Feb 1, 2021 at 10:53 PM Andrej Prsa via devel <devel@lists.open-mpi.org> wrote: > > Hi Gilles, > > > what is your mpirun command line? > > is mpirun invoked from a batch allocation? > > I call mpirun directly; here's a full output: > > andrej@terra:~/system/tests/MPI$ mpirun --mca ess_base_verbose 10 --mca > pmix_base_verbose 10 -np 4 python testmpi.py > [terra:203257] mca: base: components_register: registering framework ess > components > [terra:203257] mca: base: components_register: found loaded component slurm > [terra:203257] mca: base: components_register: component slurm has no > register or open function > [terra:203257] mca: base: components_register: found loaded component env > [terra:203257] mca: base: components_register: component env has no > register or open function > [terra:203257] mca: base: components_register: found loaded component pmi > [terra:203257] mca: base: components_register: component pmi has no > register or open function > [terra:203257] mca: base: components_register: found loaded component tool > [terra:203257] mca: base: components_register: component tool register > function successful > [terra:203257] mca: base: components_register: found loaded component hnp > [terra:203257] mca: base: components_register: component hnp has no > register or open function > [terra:203257] mca: base: components_register: found loaded component > singleton > [terra:203257] mca: base: components_register: component singleton > register function successful > [terra:203257] mca: base: components_open: opening ess components > [terra:203257] mca: base: components_open: found loaded component slurm > [terra:203257] mca: base: components_open: component slurm open function > successful > [terra:203257] mca: base: components_open: found loaded component env > [terra:203257] mca: base: components_open: component env open function > successful > [terra:203257] mca: base: components_open: found loaded component pmi > [terra:203257] mca: base: components_open: component pmi open function > successful > [terra:203257] mca: base: components_open: found loaded component tool > [terra:203257] mca: base: components_open: component tool open function > successful > [terra:203257] mca: base: components_open: found loaded component hnp > [terra:203257] mca: base: components_open: component hnp open function > successful > [terra:203257] mca: base: components_open: found loaded component singleton > [terra:203257] mca: base: components_open: component singleton open > function successful > [terra:203257] mca:base:select: Auto-selecting ess components > [terra:203257] mca:base:select:( ess) Querying component [slurm] > [terra:203257] mca:base:select:( ess) Querying component [env] > [terra:203257] mca:base:select:( ess) Querying component [pmi] > [terra:203257] mca:base:select:( ess) Querying component [tool] > [terra:203257] mca:base:select:( ess) Querying component [hnp] > [terra:203257] mca:base:select:( ess) Query of component [hnp] set > priority to 100 > [terra:203257] mca:base:select:( ess) Querying component [singleton] > [terra:203257] mca:base:select:( ess) Selected component [hnp] > [terra:203257] mca: base: close: component slurm closed > [terra:203257] mca: base: close: unloading component slurm > [terra:203257] mca: base: close: component env closed > [terra:203257] mca: base: close: unloading component env > [terra:203257] mca: base: close: component pmi closed > [terra:203257] mca: base: close: unloading component pmi > [terra:203257] mca: base: close: component tool closed > [terra:203257] mca: base: close: unloading component tool > [terra:203257] mca: base: close: component singleton closed > [terra:203257] mca: base: close: unloading component singleton > [terra:203257] mca: base: components_register: registering framework > pmix components > [terra:203257] mca: base: components_register: found loaded component flux > [terra:203257] mca: base: components_register: component flux register > function successful > [terra:203257] mca: base: components_open: opening pmix components > [terra:203257] mca: base: components_open: found loaded component flux > [terra:203257] mca:base:select: Auto-selecting pmix components > [terra:203257] mca:base:select:( pmix) Querying component [flux] > [terra:203257] mca:base:select:( pmix) No component selected! > [terra:203257] [[47344,0],0] ORTE_ERROR_LOG: Not found in file > ess_hnp_module.c at line 320 > -------------------------------------------------------------------------- > It looks like orte_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during orte_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_pmix_base_select failed > --> Returned value Not found (-13) instead of ORTE_SUCCESS > -------------------------------------------------------------------------- > > Thanks, > Andrej >
