date:20180228

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread William T Jones

Jeff,

Thanks so much for the help and the phone call.

We have built 1.10.7 on CentOS 7 without the need to explicitly turn off
either the usnic or verbs-usnic. Everything configured and built
correctly out of the box with rdma-core-devel-13-7.el7.x86_64.

We will now hopefully be able to run our regression tests with no
outcome changes due to the patch level increment. Depending on that
result we will make a decision to update to a newer OpenMPI.

Thanks again for your help and to others on this list who pointed the
way clear.

All the best and thanks for all the development too!

On 02/28/2018 03:20 PM, Jeff Squyres (jsquyres) wrote:

I took advantage of the fact that Bill's phone number is in his signature and
gave him a call (gasp! Talk to someone from the interwebs -- what craziness is
that?!).

The real issue here is that Open MPI's use of verbs in v1.10.2 pre-dates the
use of the rdma-core packaging. Various header file and other changes were
made in the transition from libiverbs to rdma-core, the Open MPI v1.10.2 simply
doesn't handle them correctly. Later versions in the v1.10.x series fix all
these issues such that both pre-rdma-core libibverbs and post-rdma-core
libibverbs are handled properly.

Compounding this was the fact that we had another bug in v1.10.2 that it wasn't
possible to fully disable the common/usnic component. Sad panda.

Unfortunately, the only way to move forward is to either apply a patch to
v1.10.2 (in this case, just to disable all the usNIC stuff, since this user is
not using usNIC at all), move forward to v1.10.7, or move forward to the latest
Open MPI (both v3.0.1 and v3.1.0 are literally immanently about to be released).

On Feb 28, 2018, at 2:41 PM, William T Jones wrote:

Thanks for the suggestions.

On 02/28/2018 12:10 PM, Jeff Squyres (jsquyres) wrote:

Oops; it looks like there's 2 chunks of usNIC code in the 1.10.x code base, and
--without-usnic only disables one of them.
I do believe we fixed that in a later 1.10.x release -- I am guessing you don't
want to upgrade to v3.0.x for compatibility/testing reasons, but do you think
you could move to a later v1.10.x release (which should be just bug fixes
compared to 1.10.2)?
If you can upgrade to v1.10.7, this should work:
./configure --without-usnic --without-verbs-usnic ...

This works. But will be tough because it will require lots of re-validation
work with the codes that depend on OpenMPI even though it is only a patch level
increment.

Sidenote: it's actually also possible that v1.10.7 will either correctly ignore
or correctly compile usNIC on your system (without any extra command line
options) -- we may have fixed that bug by v1.10.7; I honestly don't remember
offhand.
If you can't move to v1.10.7, I think you should be able to use the following
with v1.10.2 too disable the BTL usNIC component and the verbs_usnic common
component:
./configure --without-usnic --enable-mca-no-build=common-verbs_usnic ...

Sadly this does not work. Linker fails with undefined reference to
`ompi_common_verbs_usnic_register_fake_drivers'.

On Feb 28, 2018, at 11:55 AM, William T Jones wrote:

Unfortunately, that does not work.

% ./configure --enable-static \
--with-tm=/usr/local/pkgs/PBSPro_64 \
--enable-mpi-thread-multiple \
--with-verbs=/usr \
--without-usnic \
--enable-mpi-cxx \
FC=ifort \
F77=ifort \
CC=icc \
CXX=icpc \
CFLAGS="-O3 -ip" \
FCFLAGS="-O3 -ip" \
LIBS="-lcrypto -lpthread"

...
Making all in mca/common/verbs_usnic
make[2]: Entering directory
`/misc/home2/wtjones1/GIT/fun3d/misc/module-builder/k/openmpi/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
CC libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field "ops"
.ops = {
^

common_verbs_usnic_fake.c(89): warning #266: function "ibv_read_sysfs_file"
declared implicitly
if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
^

common_verbs_usnic_fake.c(133): warning #266: function "ibv_register_driver"
declared implicitly
ibv_register_driver("usnic_verbs", fake_driver_init);
^

compilation aborted for common_verbs_usnic_fake.c (code 2)

On 02/28/2018 11:10 AM, r...@open-mpi.org wrote:

Not unless you have a USNIC card in your machine!

On Feb 28, 2018, at 8:08 AM, William T Jones wrote:

Thank you!

Will that have any adverse side effects?
Performance penalties?

On 02/28/2018 10:57 AM, r...@open-mpi.org wrote:

Add --without-usnic

On Feb 28, 2018, at 7:50 AM, William T Jones wrote:

I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
am attempting to compile it after a system upgrade to CentOS 7.

This system does

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread Jeff Squyres (jsquyres)

I took advantage of the fact that Bill's phone number is in his signature and 
gave him a call (gasp! Talk to someone from the interwebs -- what craziness is 
that?!).

The real issue here is that Open MPI's use of verbs in v1.10.2 pre-dates the 
use of the rdma-core packaging.  Various header file and other changes were 
made in the transition from libiverbs to rdma-core, the Open MPI v1.10.2 simply 
doesn't handle them correctly.  Later versions in the v1.10.x series fix all 
these issues such that both pre-rdma-core libibverbs and post-rdma-core 
libibverbs are handled properly.

Compounding this was the fact that we had another bug in v1.10.2 that it wasn't 
possible to fully disable the common/usnic component.  Sad panda.

Unfortunately, the only way to move forward is to either apply a patch to 
v1.10.2 (in this case, just to disable all the usNIC stuff, since this user is 
not using usNIC at all), move forward to v1.10.7, or move forward to the latest 
Open MPI (both v3.0.1 and v3.1.0 are literally immanently about to be released).



> On Feb 28, 2018, at 2:41 PM, William T Jones  wrote:
> 
> Thanks for the suggestions.
> 
> 
> On 02/28/2018 12:10 PM, Jeff Squyres (jsquyres) wrote:
>> Oops; it looks like there's 2 chunks of usNIC code in the 1.10.x code base, 
>> and --without-usnic only disables one of them.
>> I do believe we fixed that in a later 1.10.x release -- I am guessing you 
>> don't want to upgrade to v3.0.x for compatibility/testing reasons, but do 
>> you think you could move to a later v1.10.x release (which should be just 
>> bug fixes compared to 1.10.2)?
>> If you can upgrade to v1.10.7, this should work:
>> ./configure --without-usnic --without-verbs-usnic ...
> 
> This works.  But will be tough because it will require lots of re-validation 
> work with the codes that depend on OpenMPI even though it is only a patch 
> level increment.
> 
>> Sidenote: it's actually also possible that v1.10.7 will either correctly 
>> ignore or correctly compile usNIC on your system (without any extra command 
>> line options) -- we may have fixed that bug by v1.10.7; I honestly don't 
>> remember offhand.
>> If you can't move to v1.10.7, I think you should be able to use the 
>> following with v1.10.2 too disable the BTL usNIC component and the 
>> verbs_usnic common component:
>> ./configure --without-usnic --enable-mca-no-build=common-verbs_usnic ...
> 
> Sadly this does not work.  Linker fails with undefined reference to 
> `ompi_common_verbs_usnic_register_fake_drivers'.
> 
>>> On Feb 28, 2018, at 11:55 AM, William T Jones  wrote:
>>> 
>>> Unfortunately, that does not work.
>>> 
>>> % ./configure --enable-static \
>>>  --with-tm=/usr/local/pkgs/PBSPro_64 \
>>>  --enable-mpi-thread-multiple \
>>>  --with-verbs=/usr \
>>>  --without-usnic \
>>>  --enable-mpi-cxx \
>>>  FC=ifort \
>>>  F77=ifort \
>>>  CC=icc \
>>>  CXX=icpc \
>>>  CFLAGS="-O3 -ip" \
>>>  FCFLAGS="-O3 -ip" \
>>>  LIBS="-lcrypto -lpthread"
>>> 
>>> ...
>>> Making all in mca/common/verbs_usnic
>>> make[2]: Entering directory 
>>> `/misc/home2/wtjones1/GIT/fun3d/misc/module-builder/k/openmpi/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
>>>  CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
>>> common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field "ops"
>>>  .ops = {
>>>   ^
>>> 
>>> common_verbs_usnic_fake.c(89): warning #266: function "ibv_read_sysfs_file" 
>>> declared implicitly
>>>  if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
>>>  ^
>>> 
>>> common_verbs_usnic_fake.c(133): warning #266: function 
>>> "ibv_register_driver" declared implicitly
>>>  ibv_register_driver("usnic_verbs", fake_driver_init);
>>>  ^
>>> 
>>> compilation aborted for common_verbs_usnic_fake.c (code 2)
>>> 
>>> 
>>> On 02/28/2018 11:10 AM, r...@open-mpi.org wrote:
 Not unless you have a USNIC card in your machine!
> On Feb 28, 2018, at 8:08 AM, William T Jones  wrote:
> 
> Thank you!
> 
> Will that have any adverse side effects?
> Performance penalties?
> 
> On 02/28/2018 10:57 AM, r...@open-mpi.org wrote:
>> Add --without-usnic
>>> On Feb 28, 2018, at 7:50 AM, William T Jones  wrote:
>>> 
>>> I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
>>> am attempting to compile it after a system upgrade to CentOS 7.
>>> 
>>> This system does include infiniband and I have configured as follows
>>> using Intel 2017.2.174 compilers:
>>> 
>>> % ./configure --enable-static \
>>>  --with-tm=/usr/local/pkgs/PBSPro_64 \
>>>  --enable-mpi-thread-multiple \
>>>  --with-verbs=/usr \
>>>  --enable-mpi-cxx \
>>>

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread William T Jones


Thanks for the suggestions.


On 02/28/2018 12:10 PM, Jeff Squyres (jsquyres) wrote:

Oops; it looks like there's 2 chunks of usNIC code in the 1.10.x code base, and 
--without-usnic only disables one of them.

I do believe we fixed that in a later 1.10.x release -- I am guessing you don't 
want to upgrade to v3.0.x for compatibility/testing reasons, but do you think 
you could move to a later v1.10.x release (which should be just bug fixes 
compared to 1.10.2)?

If you can upgrade to v1.10.7, this should work:

./configure --without-usnic --without-verbs-usnic ...


This works.  But will be tough because it will require lots of 
re-validation work with the codes that depend on OpenMPI even though it 
is only a patch level increment.




Sidenote: it's actually also possible that v1.10.7 will either correctly ignore 
or correctly compile usNIC on your system (without any extra command line 
options) -- we may have fixed that bug by v1.10.7; I honestly don't remember 
offhand.

If you can't move to v1.10.7, I think you should be able to use the following 
with v1.10.2 too disable the BTL usNIC component and the verbs_usnic common 
component:

./configure --without-usnic --enable-mca-no-build=common-verbs_usnic ...


Sadly this does not work.  Linker fails with undefined reference to 
`ompi_common_verbs_usnic_register_fake_drivers'.






On Feb 28, 2018, at 11:55 AM, William T Jones  wrote:

Unfortunately, that does not work.

% ./configure --enable-static \
  --with-tm=/usr/local/pkgs/PBSPro_64 \
  --enable-mpi-thread-multiple \
  --with-verbs=/usr \
  --without-usnic \
  --enable-mpi-cxx \
  FC=ifort \
  F77=ifort \
  CC=icc \
  CXX=icpc \
  CFLAGS="-O3 -ip" \
  FCFLAGS="-O3 -ip" \
  LIBS="-lcrypto -lpthread"

...
Making all in mca/common/verbs_usnic
make[2]: Entering directory 
`/misc/home2/wtjones1/GIT/fun3d/misc/module-builder/k/openmpi/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
  CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field "ops"
  .ops = {
   ^

common_verbs_usnic_fake.c(89): warning #266: function "ibv_read_sysfs_file" 
declared implicitly
  if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
  ^

common_verbs_usnic_fake.c(133): warning #266: function "ibv_register_driver" 
declared implicitly
  ibv_register_driver("usnic_verbs", fake_driver_init);
  ^

compilation aborted for common_verbs_usnic_fake.c (code 2)


On 02/28/2018 11:10 AM, r...@open-mpi.org wrote:

Not unless you have a USNIC card in your machine!

On Feb 28, 2018, at 8:08 AM, William T Jones  wrote:

Thank you!

Will that have any adverse side effects?
Performance penalties?

On 02/28/2018 10:57 AM, r...@open-mpi.org wrote:

Add --without-usnic

On Feb 28, 2018, at 7:50 AM, William T Jones  wrote:

I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
am attempting to compile it after a system upgrade to CentOS 7.

This system does include infiniband and I have configured as follows
using Intel 2017.2.174 compilers:

% ./configure --enable-static \
  --with-tm=/usr/local/pkgs/PBSPro_64 \
  --enable-mpi-thread-multiple \
  --with-verbs=/usr \
  --enable-mpi-cxx \
  FC=ifort \
  F77=ifort \
  CC=icc \
  CXX=icpc \
  CFLAGS="-O3 -ip" \
  FCFLAGS="-O3 -ip" \
  LIBS=-lcrypto -lpthread

However, when I compile I get the following error:

  ...
  Making all in mca/common/verbs_usnic
  make[2]: Entering directory
`/usr/src/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
  common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field
"ops"
.ops = {
 ^

  common_verbs_usnic_fake.c(89): warning #266: function
"ibv_read_sysfs_file" declared implicitly
if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
^

  common_verbs_usnic_fake.c(133): warning #266: function
"ibv_register_driver" declared implicitly
ibv_register_driver("usnic_verbs", fake_driver_init);
^

  compilation aborted for common_verbs_usnic_fake.c (code 2)


Unfortunately, my /usr/include/infiniband/verbs.h file defines the
"ibv_device" structure but does not include "ops" member.  Instead the
structure is defined as follows:

  /* Obsolete, never used, do not touch */
  struct _ibv_device_ops {
  struct ibv_context *(*_dummy1)(struct ibv_device *device,
int cmd_fd);
  void(*_dummy2)(struct ibv_context *context);
  };

  enum {
  IBV_SYSFS_NAME_MAX  = 64,
  IBV_SYSFS_PATH_MAX  = 256
  };

  struct

Re: [OMPI users] OpenMPI 3.0.0 Failing To Compile

2018-02-28 Thread Jeff Squyres (jsquyres)

On Feb 28, 2018, at 1:31 PM, Justin Luitjens  wrote:
>  Here is the error I see:
>  
> make[2]: Entering directory 
> '/tmpnfs/jluitjens/libs/src/openmpi-3.0.0/opal/mca/crs'
>   CC   base/crs_base_open.lo
>   GENERATE opal_crs.7
>   CC   base/crs_base_select.lo
>   CC   base/crs_base_close.lo
>   CC   base/crs_base_fns.lo
> Option package-version requires an argument
> Usage: ../../../ompi/mpi/man/make_manpage.pl --package-name= 
> --package-version= --ompi-date= --opal-date= date> --orte-date= --input= --output= 
> [--nocxx] [ --nofortran] [--nof08]
> Makefile:2199: recipe for target 'opal_crs.7' failed

The only way that I can see that it would complain about --package-version not 
getting a value is if configure failed (and did therefore did not provide a 
value for @PACKAGE_VERSION@).

Can you send all the information listed here:

https://www.open-mpi.org/community/help/

-- 
Jeff Squyres
jsquy...@cisco.com

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] OpenMPI 3.0.0 Failing To Compile

2018-02-28 Thread Justin Luitjens


I'm trying to build OpenMPI on Ubuntu 16.04.3 and I'm getting an error.


Here is how I configure and build:
./configure --with-cuda=$CUDA_HOME --prefix=$MPI_HOME && make clean &&  make -j 
&& make install


Here is the error I see:

make[2]: Entering directory 
'/tmpnfs/jluitjens/libs/src/openmpi-3.0.0/opal/mca/crs'
  CC   base/crs_base_open.lo
  GENERATE opal_crs.7
  CC   base/crs_base_select.lo
  CC   base/crs_base_close.lo
  CC   base/crs_base_fns.lo
Option package-version requires an argument
Usage: ../../../ompi/mpi/man/make_manpage.pl --package-name= 
--package-version= --ompi-date= --opal-date= --orte-date= --input= --output= 
[--nocxx] [ --nofortran] [--nof08]
Makefile:2199: recipe for target 'opal_crs.7' failed
make[2]: *** [opal_crs.7] Error 1
make[2]: *** Waiting for unfinished jobs
make[2]: Leaving directory 
'/tmpnfs/jluitjens/libs/src/openmpi-3.0.0/opal/mca/crs'
Makefile:2364: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/tmpnfs/jluitjens/libs/src/openmpi-3.0.0/opal'
Makefile:1885: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1


Any suggestions on what might be going on?

---
This email message is for the sole use of the intended recipient(s) and may 
contain
confidential information.  Any unauthorized review, use, disclosure or 
distribution
is prohibited.  If you are not the intended recipient, please contact the 
sender by
reply email and destroy all copies of the original message.
---
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread Jeff Squyres (jsquyres)

Oops; it looks like there's 2 chunks of usNIC code in the 1.10.x code base, and 
--without-usnic only disables one of them.

I do believe we fixed that in a later 1.10.x release -- I am guessing you don't 
want to upgrade to v3.0.x for compatibility/testing reasons, but do you think 
you could move to a later v1.10.x release (which should be just bug fixes 
compared to 1.10.2)?

If you can upgrade to v1.10.7, this should work:

./configure --without-usnic --without-verbs-usnic ...

Sidenote: it's actually also possible that v1.10.7 will either correctly ignore 
or correctly compile usNIC on your system (without any extra command line 
options) -- we may have fixed that bug by v1.10.7; I honestly don't remember 
offhand.

If you can't move to v1.10.7, I think you should be able to use the following 
with v1.10.2 too disable the BTL usNIC component and the verbs_usnic common 
component:

./configure --without-usnic --enable-mca-no-build=common-verbs_usnic ...


> On Feb 28, 2018, at 11:55 AM, William T Jones  wrote:
> 
> Unfortunately, that does not work.
> 
> % ./configure --enable-static \
>  --with-tm=/usr/local/pkgs/PBSPro_64 \
>  --enable-mpi-thread-multiple \
>  --with-verbs=/usr \
>  --without-usnic \
>  --enable-mpi-cxx \
>  FC=ifort \
>  F77=ifort \
>  CC=icc \
>  CXX=icpc \
>  CFLAGS="-O3 -ip" \
>  FCFLAGS="-O3 -ip" \
>  LIBS="-lcrypto -lpthread"
> 
> ...
> Making all in mca/common/verbs_usnic 
> make[2]: Entering directory 
> `/misc/home2/wtjones1/GIT/fun3d/misc/module-builder/k/openmpi/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
>  
>  CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo 
> common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field "ops"
>  .ops = { 
>   ^ 
> 
> common_verbs_usnic_fake.c(89): warning #266: function "ibv_read_sysfs_file" 
> declared implicitly 
>  if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor", 
>  ^ 
> 
> common_verbs_usnic_fake.c(133): warning #266: function "ibv_register_driver" 
> declared implicitly 
>  ibv_register_driver("usnic_verbs", fake_driver_init); 
>  ^ 
> 
> compilation aborted for common_verbs_usnic_fake.c (code 2)
> 
> 
> On 02/28/2018 11:10 AM, r...@open-mpi.org wrote:
>> Not unless you have a USNIC card in your machine!
>>> On Feb 28, 2018, at 8:08 AM, William T Jones  wrote:
>>> 
>>> Thank you!
>>> 
>>> Will that have any adverse side effects?
>>> Performance penalties?
>>> 
>>> On 02/28/2018 10:57 AM, r...@open-mpi.org wrote:
 Add --without-usnic
> On Feb 28, 2018, at 7:50 AM, William T Jones  wrote:
> 
> I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
> am attempting to compile it after a system upgrade to CentOS 7.
> 
> This system does include infiniband and I have configured as follows
> using Intel 2017.2.174 compilers:
> 
> % ./configure --enable-static \
>  --with-tm=/usr/local/pkgs/PBSPro_64 \
>  --enable-mpi-thread-multiple \
>  --with-verbs=/usr \
>  --enable-mpi-cxx \
>  FC=ifort \
>  F77=ifort \
>  CC=icc \
>  CXX=icpc \
>  CFLAGS="-O3 -ip" \
>  FCFLAGS="-O3 -ip" \
>  LIBS=-lcrypto -lpthread
> 
> However, when I compile I get the following error:
> 
>  ...
>  Making all in mca/common/verbs_usnic
>  make[2]: Entering directory
> `/usr/src/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
>CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
>  common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field
> "ops"
>.ops = {
> ^
> 
>  common_verbs_usnic_fake.c(89): warning #266: function
> "ibv_read_sysfs_file" declared implicitly
>if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
>^
> 
>  common_verbs_usnic_fake.c(133): warning #266: function
> "ibv_register_driver" declared implicitly
>ibv_register_driver("usnic_verbs", fake_driver_init);
>^
> 
>  compilation aborted for common_verbs_usnic_fake.c (code 2)
> 
> 
> Unfortunately, my /usr/include/infiniband/verbs.h file defines the
> "ibv_device" structure but does not include "ops" member.  Instead the
> structure is defined as follows:
> 
>  /* Obsolete, never used, do not touch */
>  struct _ibv_device_ops {
>  struct ibv_context *(*_dummy1)(struct ibv_device *device,
> int cmd_fd);
>  void(*_dummy2)(struct ibv_context *context);
>  };
> 
>  enum {
>  IBV_SYSFS_NAME_MAX  = 64,
>

Re: [OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread r...@open-mpi.org

Not unless you have a USNIC card in your machine!


> On Feb 28, 2018, at 8:08 AM, William T Jones  wrote:
> 
> Thank you!
> 
> Will that have any adverse side effects?
> Performance penalties?
> 
> On 02/28/2018 10:57 AM, r...@open-mpi.org wrote:
>> Add --without-usnic
>>> On Feb 28, 2018, at 7:50 AM, William T Jones  wrote:
>>> 
>>> I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
>>> am attempting to compile it after a system upgrade to CentOS 7.
>>> 
>>> This system does include infiniband and I have configured as follows
>>> using Intel 2017.2.174 compilers:
>>> 
>>> % ./configure --enable-static \
>>>  --with-tm=/usr/local/pkgs/PBSPro_64 \
>>>  --enable-mpi-thread-multiple \
>>>  --with-verbs=/usr \
>>>  --enable-mpi-cxx \
>>>  FC=ifort \
>>>  F77=ifort \
>>>  CC=icc \
>>>  CXX=icpc \
>>>  CFLAGS="-O3 -ip" \
>>>  FCFLAGS="-O3 -ip" \
>>>  LIBS=-lcrypto -lpthread
>>> 
>>> However, when I compile I get the following error:
>>> 
>>>  ...
>>>  Making all in mca/common/verbs_usnic
>>>  make[2]: Entering directory
>>> `/usr/src/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
>>>CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
>>>  common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field
>>> "ops"
>>>.ops = {
>>> ^
>>> 
>>>  common_verbs_usnic_fake.c(89): warning #266: function
>>> "ibv_read_sysfs_file" declared implicitly
>>>if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
>>>^
>>> 
>>>  common_verbs_usnic_fake.c(133): warning #266: function
>>> "ibv_register_driver" declared implicitly
>>>ibv_register_driver("usnic_verbs", fake_driver_init);
>>>^
>>> 
>>>  compilation aborted for common_verbs_usnic_fake.c (code 2)
>>> 
>>> 
>>> Unfortunately, my /usr/include/infiniband/verbs.h file defines the
>>> "ibv_device" structure but does not include "ops" member.  Instead the
>>> structure is defined as follows:
>>> 
>>>  /* Obsolete, never used, do not touch */
>>>  struct _ibv_device_ops {
>>>  struct ibv_context *(*_dummy1)(struct ibv_device *device,
>>> int cmd_fd);
>>>  void(*_dummy2)(struct ibv_context *context);
>>>  };
>>> 
>>>  enum {
>>>  IBV_SYSFS_NAME_MAX  = 64,
>>>  IBV_SYSFS_PATH_MAX  = 256
>>>  };
>>> 
>>>  struct ibv_device {
>>>  struct _ibv_device_ops  _ops;
>>>  enum ibv_node_type  node_type;
>>>  enum ibv_transport_type transport_type;
>>>  /* Name of underlying kernel IB device, eg "mthca0" */
>>>  charname[IBV_SYSFS_NAME_MAX];
>>>  /* Name of uverbs device, eg "uverbs0" */
>>>  chardev_name[IBV_SYSFS_NAME_MAX];
>>>  /* Path to infiniband_verbs class device in sysfs */
>>>  chardev_path[IBV_SYSFS_PATH_MAX];
>>>  /* Path to infiniband class device in sysfs */
>>>  charibdev_path[IBV_SYSFS_PATH_MAX];
>>>  };
>>> 
>>> 
>>> OpenMPI was previously compiled successfully under CentOS 6 and every
>>> indication is that the /usr/include/infiniband/verbs.h was defined
>>> similarly (again without the "ops" member).
>>> 
>>> Is it possible that there is a configure option that prevents this source 
>>> from being included in the build?
>>> 
>>> Any help is appreciated,
>>> 
>>> 
>>> -- 
>>> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>> 
>>>Bill Jones   w.t.jo...@nasa.gov
>>>Mail Stop 128 Computational AeroSciences Branch
>>>15 Langley Boulevard   Research Directorate
>>>NASA Langley Research Center   Building 1268, Room 1044
>>>Hampton, VA  23681-2199   Phone +1 757 864-5318
>>>Fax +1 757 864-8816
>>> http://fun3d.larc.nasa.gov
>>> ___
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://lists.open-mpi.org/mailman/listinfo/users
>> ___
>> users mailing list
>> users@lists.open-mpi.org
>> https://lists.open-mpi.org/mailman/listinfo/users
> 
> -- 
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 
>Bill Jones   w.t.jo...@nasa.gov
>Mail Stop 128 Computational AeroSciences Branch
>15 Langley Boulevard   Research Directorate
>NASA Langley Research Center   Building 1268, Room 1044
>Hampton, VA  23681-2199   Phone +1 757 864-5318
>

Re: [OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread William T Jones


Thank you!

Will that have any adverse side effects?
Performance penalties?

On 02/28/2018 10:57 AM, r...@open-mpi.org wrote:

Add --without-usnic



On Feb 28, 2018, at 7:50 AM, William T Jones  wrote:

I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
am attempting to compile it after a system upgrade to CentOS 7.

This system does include infiniband and I have configured as follows
using Intel 2017.2.174 compilers:

% ./configure --enable-static \
  --with-tm=/usr/local/pkgs/PBSPro_64 \
  --enable-mpi-thread-multiple \
  --with-verbs=/usr \
  --enable-mpi-cxx \
  FC=ifort \
  F77=ifort \
  CC=icc \
  CXX=icpc \
  CFLAGS="-O3 -ip" \
  FCFLAGS="-O3 -ip" \
  LIBS=-lcrypto -lpthread

However, when I compile I get the following error:

  ...
  Making all in mca/common/verbs_usnic
  make[2]: Entering directory
`/usr/src/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
  common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field
"ops"
.ops = {
 ^

  common_verbs_usnic_fake.c(89): warning #266: function
"ibv_read_sysfs_file" declared implicitly
if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
^

  common_verbs_usnic_fake.c(133): warning #266: function
"ibv_register_driver" declared implicitly
ibv_register_driver("usnic_verbs", fake_driver_init);
^

  compilation aborted for common_verbs_usnic_fake.c (code 2)


Unfortunately, my /usr/include/infiniband/verbs.h file defines the
"ibv_device" structure but does not include "ops" member.  Instead the
structure is defined as follows:

  /* Obsolete, never used, do not touch */
  struct _ibv_device_ops {
  struct ibv_context *(*_dummy1)(struct ibv_device *device,
int cmd_fd);
  void(*_dummy2)(struct ibv_context *context);
  };

  enum {
  IBV_SYSFS_NAME_MAX  = 64,
  IBV_SYSFS_PATH_MAX  = 256
  };

  struct ibv_device {
  struct _ibv_device_ops  _ops;
  enum ibv_node_type  node_type;
  enum ibv_transport_type transport_type;
  /* Name of underlying kernel IB device, eg "mthca0" */
  charname[IBV_SYSFS_NAME_MAX];
  /* Name of uverbs device, eg "uverbs0" */
  chardev_name[IBV_SYSFS_NAME_MAX];
  /* Path to infiniband_verbs class device in sysfs */
  chardev_path[IBV_SYSFS_PATH_MAX];
  /* Path to infiniband class device in sysfs */
  charibdev_path[IBV_SYSFS_PATH_MAX];
  };


OpenMPI was previously compiled successfully under CentOS 6 and every
indication is that the /usr/include/infiniband/verbs.h was defined
similarly (again without the "ops" member).

Is it possible that there is a configure option that prevents this source from 
being included in the build?

Any help is appreciated,


--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Bill Jones   w.t.jo...@nasa.gov
Mail Stop 128 Computational AeroSciences Branch
15 Langley Boulevard   Research Directorate
NASA Langley Research Center   Building 1268, Room 1044
Hampton, VA  23681-2199   Phone +1 757 864-5318
Fax +1 757 864-8816
 http://fun3d.larc.nasa.gov
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users


___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users



--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Bill Jones   w.t.jo...@nasa.gov
Mail Stop 128 Computational AeroSciences Branch
15 Langley Boulevard   Research Directorate
NASA Langley Research Center   Building 1268, Room 1044
Hampton, VA  23681-2199   Phone +1 757 864-5318
Fax +1 757 864-8816
 http://fun3d.larc.nasa.gov
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread r...@open-mpi.org

Add --without-usnic


> On Feb 28, 2018, at 7:50 AM, William T Jones  wrote:
> 
> I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
> am attempting to compile it after a system upgrade to CentOS 7.
> 
> This system does include infiniband and I have configured as follows
> using Intel 2017.2.174 compilers:
> 
> % ./configure --enable-static \
>  --with-tm=/usr/local/pkgs/PBSPro_64 \
>  --enable-mpi-thread-multiple \
>  --with-verbs=/usr \
>  --enable-mpi-cxx \
>  FC=ifort \
>  F77=ifort \
>  CC=icc \
>  CXX=icpc \
>  CFLAGS="-O3 -ip" \
>  FCFLAGS="-O3 -ip" \
>  LIBS=-lcrypto -lpthread
> 
> However, when I compile I get the following error:
> 
>  ...
>  Making all in mca/common/verbs_usnic
>  make[2]: Entering directory
> `/usr/src/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
>CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
>  common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field
> "ops"
>.ops = {
> ^
> 
>  common_verbs_usnic_fake.c(89): warning #266: function
> "ibv_read_sysfs_file" declared implicitly
>if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
>^
> 
>  common_verbs_usnic_fake.c(133): warning #266: function
> "ibv_register_driver" declared implicitly
>ibv_register_driver("usnic_verbs", fake_driver_init);
>^
> 
>  compilation aborted for common_verbs_usnic_fake.c (code 2)
> 
> 
> Unfortunately, my /usr/include/infiniband/verbs.h file defines the
> "ibv_device" structure but does not include "ops" member.  Instead the
> structure is defined as follows:
> 
>  /* Obsolete, never used, do not touch */
>  struct _ibv_device_ops {
>  struct ibv_context *(*_dummy1)(struct ibv_device *device,
> int cmd_fd);
>  void(*_dummy2)(struct ibv_context *context);
>  };
> 
>  enum {
>  IBV_SYSFS_NAME_MAX  = 64,
>  IBV_SYSFS_PATH_MAX  = 256
>  };
> 
>  struct ibv_device {
>  struct _ibv_device_ops  _ops;
>  enum ibv_node_type  node_type;
>  enum ibv_transport_type transport_type;
>  /* Name of underlying kernel IB device, eg "mthca0" */
>  charname[IBV_SYSFS_NAME_MAX];
>  /* Name of uverbs device, eg "uverbs0" */
>  chardev_name[IBV_SYSFS_NAME_MAX];
>  /* Path to infiniband_verbs class device in sysfs */
>  chardev_path[IBV_SYSFS_PATH_MAX];
>  /* Path to infiniband class device in sysfs */
>  charibdev_path[IBV_SYSFS_PATH_MAX];
>  };
> 
> 
> OpenMPI was previously compiled successfully under CentOS 6 and every
> indication is that the /usr/include/infiniband/verbs.h was defined
> similarly (again without the "ops" member).
> 
> Is it possible that there is a configure option that prevents this source 
> from being included in the build?
> 
> Any help is appreciated,
> 
> 
> -- 
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 
>Bill Jones   w.t.jo...@nasa.gov
>Mail Stop 128 Computational AeroSciences Branch
>15 Langley Boulevard   Research Directorate
>NASA Langley Research Center   Building 1268, Room 1044
>Hampton, VA  23681-2199   Phone +1 757 864-5318
>Fax +1 757 864-8816
> http://fun3d.larc.nasa.gov
> ___
> users mailing list
> users@lists.open-mpi.org
> https://lists.open-mpi.org/mailman/listinfo/users

___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

[OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

2018-02-28 Thread William T Jones


I realize that OpenMPI 1.10.2 is quite old, however, for compatibility I
am attempting to compile it after a system upgrade to CentOS 7.

This system does include infiniband and I have configured as follows
using Intel 2017.2.174 compilers:

% ./configure --enable-static \
  --with-tm=/usr/local/pkgs/PBSPro_64 \
  --enable-mpi-thread-multiple \
  --with-verbs=/usr \
  --enable-mpi-cxx \
  FC=ifort \
  F77=ifort \
  CC=icc \
  CXX=icpc \
  CFLAGS="-O3 -ip" \
  FCFLAGS="-O3 -ip" \
  LIBS=-lcrypto -lpthread

However, when I compile I get the following error:

  ...
  Making all in mca/common/verbs_usnic
  make[2]: Entering directory
`/usr/src/openmpi-1.10.2/ompi/mca/common/verbs_usnic'
CC   libmca_common_verbs_usnic_la-common_verbs_usnic_fake.lo
  common_verbs_usnic_fake.c(72): error: struct "ibv_device" has no field
"ops"
.ops = {
 ^

  common_verbs_usnic_fake.c(89): warning #266: function
"ibv_read_sysfs_file" declared implicitly
if (ibv_read_sysfs_file(uverbs_sys_path, "device/vendor",
^

  common_verbs_usnic_fake.c(133): warning #266: function
"ibv_register_driver" declared implicitly
ibv_register_driver("usnic_verbs", fake_driver_init);
^

  compilation aborted for common_verbs_usnic_fake.c (code 2)


Unfortunately, my /usr/include/infiniband/verbs.h file defines the
"ibv_device" structure but does not include "ops" member.  Instead the
structure is defined as follows:

  /* Obsolete, never used, do not touch */
  struct _ibv_device_ops {
  struct ibv_context *(*_dummy1)(struct ibv_device *device,
int cmd_fd);
  void(*_dummy2)(struct ibv_context *context);
  };

  enum {
  IBV_SYSFS_NAME_MAX  = 64,
  IBV_SYSFS_PATH_MAX  = 256
  };

  struct ibv_device {
  struct _ibv_device_ops  _ops;
  enum ibv_node_type  node_type;
  enum ibv_transport_type transport_type;
  /* Name of underlying kernel IB device, eg "mthca0" */
  charname[IBV_SYSFS_NAME_MAX];
  /* Name of uverbs device, eg "uverbs0" */
  chardev_name[IBV_SYSFS_NAME_MAX];
  /* Path to infiniband_verbs class device in sysfs */
  chardev_path[IBV_SYSFS_PATH_MAX];
  /* Path to infiniband class device in sysfs */
  charibdev_path[IBV_SYSFS_PATH_MAX];
  };


OpenMPI was previously compiled successfully under CentOS 6 and every
indication is that the /usr/include/infiniband/verbs.h was defined
similarly (again without the "ops" member).

Is it possible that there is a configure option that prevents this 
source from being included in the build?


Any help is appreciated,


--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Bill Jones   w.t.jo...@nasa.gov
Mail Stop 128 Computational AeroSciences Branch
15 Langley Boulevard   Research Directorate
NASA Langley Research Center   Building 1268, Room 1044
Hampton, VA  23681-2199   Phone +1 757 864-5318
Fax +1 757 864-8816
 http://fun3d.larc.nasa.gov
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

Re: [OMPI users] OpenMPI 3.0.0 Failing To Compile

[OMPI users] OpenMPI 3.0.0 Failing To Compile

Re: [OMPI users] [WARNING: UNSCANNABLE EXTRACTION FAILED]Re: Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

Re: [OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

Re: [OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

Re: [OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

[OMPI users] Cannot compile 1.10.2 under CentOS 7 rdma-core-devel-13-7.el7.x86_64

10 matches

Site Navigation

Mail list logo

Footer information