Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
That's unlikely to help. You either need to do an OS upgrade (which hopefully brings in a newer OFED stack) or install the MOFED stack separately. The libibverbs packages are tightly coupled to the (M)OFED drivers installed on the system, i.e. either the ones from the OS distro or the ones from a MOFED stack. From: easybuild-requ...@lists.ugent.be on behalf of Joaquim Jornet Somoza Sent: Friday, March 15, 2024 14:06 To: easybuild@lists.ugent.be Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb OK... I will try to updatet libibverbs , isn't it ? I will let you know ! Thanks ! quim Missatge de Åke Sandgren mailto:ake.sandg...@umu.se>> del dia dv., 15 de març 2024 a les 13:49: ConnectX-3 gets dropped in MOFED 5.1, but MOFED 4.9 LTS will still work for those. From: easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be> mailto:easybuild-requ...@lists.ugent.be>> on behalf of Ole Holm Nielsen mailto:ole.h.niel...@fysik.dtu.dk>> Sent: Friday, March 15, 2024 13:21 To: easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be> Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb Hi Joaquim, I think that old ConnectX-3 are no longer supported with recent Linux kernels (can anyone confirm this?). You may have to use an ancient (insecure) OS version, or stop using the ConnectX-3 adapters altogether :-( /Ole On 3/15/24 12:45, Joaquim Jornet Somoza wrote: > Dear Ake, > > Indeed we have Mellanox IB, > > [:~]$ lspci | grep -i mellanox > > 02:00.0 Network controller: *Mellanox*Technologies MT27500 Family [ConnectX-3] > > > [:~]$ ibstatus > > Infiniband device 'mlx4_0' port 1 status: > > default gid: fe80::::7079:9003:0007:f538 > > base lid:0x76 > > sm lid:0x1 > > state: 4: ACTIVE > > phys state:5: LinkUp > > rate:56 Gb/sec (4X FDR) > > link_layer:InfiniBand > > > > > I had no problem installing foss-2023a.eb in another cluster with : > > $lspci | grep -i mellanox > > 86:00.0 Infiniband controller: *Mellanox*Technologies MT27800 Family > [ConnectX-5] > > > > Thank you for any hint ! > > quim > > > Missatge de Åke Sandgren mailto:ake.sandg...@umu.se> > <mailto:ake.sandg...@umu.se<mailto:ake.sandg...@umu.se>>> del dia dv., 15 de > març 2024 a les 12:18: > > lspci | grep -i mellanox > will show if you have any mellanox devices on the system, some of > these could be used as normal ethernet devices though > > ibstatus > will show if any of those are running in Infiniband mode > > If ibstatus doesn't exist then you probably don't have infiniband, or > at least lack the packages for using it. > > > From: > easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be> > > <mailto:easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be>> > mailto:easybuild-requ...@lists.ugent.be> > > <mailto:easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be>>> > on behalf of Joaquim Jornet > Somoza mailto:j.jornet.som...@gmail.com> > <mailto:j.jornet.som...@gmail.com<mailto:j.jornet.som...@gmail.com>>> > Sent: Friday, March 15, 2024 11:44 > To: easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be> > <mailto:easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>> > Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when > installing foss-2023a.eb > > Dear Ake, > > How can I check this? > > Thank you! > > El vie, 15 mar 2024, 7:58, Åke Sandgren > mailto:ake.sandg...@umu.se> > > <mailto:ake.sandg...@umu.se<mailto:ake.sandg...@umu.se>><mailto:ake.sandg...@umu.se<mailto:ake.sandg...@umu.se> > <mailto:ake.sandg...@umu.se<mailto:ake.sandg...@umu.se>>>> escribió: > No there is no bug there. > > Which MOFED stack version are you using? > Or does your system lack Infiniband? > > > From: > easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be> > > <mailto:easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be>><mailto:easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be> > > <mailto:easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be>>> > mailto:easybuild-requ...@lists.ugent.be> >
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
OK... I will try to updatet libibverbs , isn't it ? I will let you know ! Thanks ! quim Missatge de Åke Sandgren del dia dv., 15 de març 2024 a les 13:49: > ConnectX-3 gets dropped in MOFED 5.1, but MOFED 4.9 LTS will still work > for those. > > > From: easybuild-requ...@lists.ugent.be > on behalf of Ole Holm Nielsen > Sent: Friday, March 15, 2024 13:21 > To: easybuild@lists.ugent.be > Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when > installing foss-2023a.eb > > Hi Joaquim, > > I think that old ConnectX-3 are no longer supported with recent Linux > kernels (can anyone confirm this?). You may have to use an ancient > (insecure) OS version, or stop using the ConnectX-3 adapters altogether :-( > > /Ole > > On 3/15/24 12:45, Joaquim Jornet Somoza wrote: > > Dear Ake, > > > > Indeed we have Mellanox IB, > > > > [:~]$ lspci | grep -i mellanox > > > > 02:00.0 Network controller: *Mellanox*Technologies MT27500 Family > [ConnectX-3] > > > > > > [:~]$ ibstatus > > > > Infiniband device 'mlx4_0' port 1 status: > > > > default gid: fe80::::7079:9003:0007:f538 > > > > base lid:0x76 > > > > sm lid:0x1 > > > > state: 4: ACTIVE > > > > phys state:5: LinkUp > > > > rate:56 Gb/sec (4X FDR) > > > > link_layer:InfiniBand > > > > > > > > > > I had no problem installing foss-2023a.eb in another cluster with : > > > > $lspci | grep -i mellanox > > > > 86:00.0 Infiniband controller: *Mellanox*Technologies MT27800 Family > > [ConnectX-5] > > > > > > > > Thank you for any hint ! > > > > quim > > > > > > Missatge de Åke Sandgren > <mailto:ake.sandg...@umu.se>> del dia dv., 15 de març 2024 a les 12:18: > > > > lspci | grep -i mellanox > > will show if you have any mellanox devices on the system, some of > > these could be used as normal ethernet devices though > > > > ibstatus > > will show if any of those are running in Infiniband mode > > > > If ibstatus doesn't exist then you probably don't have infiniband, or > > at least lack the packages for using it. > > > > ____________________ > > From: easybuild-requ...@lists.ugent.be > > <mailto:easybuild-requ...@lists.ugent.be> > > > <mailto:easybuild-requ...@lists.ugent.be>> on behalf of Joaquim > Jornet > > Somoza mailto:j.jornet.som...@gmail.com > >> > > Sent: Friday, March 15, 2024 11:44 > > To: easybuild@lists.ugent.be <mailto:easybuild@lists.ugent.be> > > Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when > > installing foss-2023a.eb > > > > Dear Ake, > > > > How can I check this? > > > > Thank you! > > > > El vie, 15 mar 2024, 7:58, Åke Sandgren > <mailto:ake.sandg...@umu.se><mailto:ake.sandg...@umu.se > > <mailto:ake.sandg...@umu.se>>> escribió: > > No there is no bug there. > > > > Which MOFED stack version are you using? > > Or does your system lack Infiniband? > > > > > > From: easybuild-requ...@lists.ugent.be > > <mailto:easybuild-requ...@lists.ugent.be> easybuild-requ...@lists.ugent.be <mailto:easybuild-requ...@lists.ugent.be>> > mailto:easybuild-requ...@lists.ugent.be > ><mailto:easybuild-requ...@lists.ugent.be easybuild-requ...@lists.ugent.be>>> on behalf of Joaquim Jornet Somoza < > j.jornet.som...@gmail.com <mailto:j.jornet.som...@gmail.com> j.jornet.som...@gmail.com <mailto:j.jornet.som...@gmail.com>>> > > Sent: Thursday, March 14, 2024 16:02 > > To: easybuild@lists.ugent.be > > <mailto:easybuild@lists.ugent.be><mailto:easybuild@lists.ugent.be > > <mailto:easybuild@lists.ugent.be>> > > Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when > > installing foss-2023a.eb > > > > Dear easybuilders, > > > > I am trying to install foss-2023a.eb on a RH7.7 servers, but when > > installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with > > the following error: > > ... > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. > > "-DCPU_FLAGS=|avx" > > -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src >
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
ConnectX-3 gets dropped in MOFED 5.1, but MOFED 4.9 LTS will still work for those. From: easybuild-requ...@lists.ugent.be on behalf of Ole Holm Nielsen Sent: Friday, March 15, 2024 13:21 To: easybuild@lists.ugent.be Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb Hi Joaquim, I think that old ConnectX-3 are no longer supported with recent Linux kernels (can anyone confirm this?). You may have to use an ancient (insecure) OS version, or stop using the ConnectX-3 adapters altogether :-( /Ole On 3/15/24 12:45, Joaquim Jornet Somoza wrote: > Dear Ake, > > Indeed we have Mellanox IB, > > [:~]$ lspci | grep -i mellanox > > 02:00.0 Network controller: *Mellanox*Technologies MT27500 Family [ConnectX-3] > > > [:~]$ ibstatus > > Infiniband device 'mlx4_0' port 1 status: > > default gid: fe80::::7079:9003:0007:f538 > > base lid:0x76 > > sm lid:0x1 > > state: 4: ACTIVE > > phys state:5: LinkUp > > rate:56 Gb/sec (4X FDR) > > link_layer:InfiniBand > > > > > I had no problem installing foss-2023a.eb in another cluster with : > > $lspci | grep -i mellanox > > 86:00.0 Infiniband controller: *Mellanox*Technologies MT27800 Family > [ConnectX-5] > > > > Thank you for any hint ! > > quim > > > Missatge de Åke Sandgren <mailto:ake.sandg...@umu.se>> del dia dv., 15 de març 2024 a les 12:18: > > lspci | grep -i mellanox > will show if you have any mellanox devices on the system, some of > these could be used as normal ethernet devices though > > ibstatus > will show if any of those are running in Infiniband mode > > If ibstatus doesn't exist then you probably don't have infiniband, or > at least lack the packages for using it. > > > From: easybuild-requ...@lists.ugent.be > <mailto:easybuild-requ...@lists.ugent.be> > <mailto:easybuild-requ...@lists.ugent.be>> on behalf of Joaquim Jornet > Somoza mailto:j.jornet.som...@gmail.com>> > Sent: Friday, March 15, 2024 11:44 > To: easybuild@lists.ugent.be <mailto:easybuild@lists.ugent.be> > Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when > installing foss-2023a.eb > > Dear Ake, > > How can I check this? > > Thank you! > > El vie, 15 mar 2024, 7:58, Åke Sandgren <mailto:ake.sandg...@umu.se><mailto:ake.sandg...@umu.se > <mailto:ake.sandg...@umu.se>>> escribió: > No there is no bug there. > > Which MOFED stack version are you using? > Or does your system lack Infiniband? > > > From: easybuild-requ...@lists.ugent.be > > <mailto:easybuild-requ...@lists.ugent.be><mailto:easybuild-requ...@lists.ugent.be > <mailto:easybuild-requ...@lists.ugent.be>> <mailto:easybuild-requ...@lists.ugent.be><mailto:easybuild-requ...@lists.ugent.be > <mailto:easybuild-requ...@lists.ugent.be>>> on behalf of Joaquim Jornet > Somoza <mailto:j.jornet.som...@gmail.com><mailto:j.jornet.som...@gmail.com > <mailto:j.jornet.som...@gmail.com>>> > Sent: Thursday, March 14, 2024 16:02 > To: easybuild@lists.ugent.be > <mailto:easybuild@lists.ugent.be><mailto:easybuild@lists.ugent.be > <mailto:easybuild@lists.ugent.be>> > Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when > installing foss-2023a.eb > > Dear easybuilders, > > I am trying to install foss-2023a.eb on a RH7.7 servers, but when > installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with > the following error: > ... > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. > "-DCPU_FLAGS=|avx" > -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src > -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1 > -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src > > -I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include > -I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include > -I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include > -I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include > -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers > -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels > -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch > -Wno-pointer-sign -Werror-implicit-function-dec
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Hi Joaquim,
I think that old ConnectX-3 are no longer supported with recent Linux
kernels (can anyone confirm this?). You may have to use an ancient
(insecure) OS version, or stop using the ConnectX-3 adapters altogether :-(
/Ole
On 3/15/24 12:45, Joaquim Jornet Somoza wrote:
Dear Ake,
Indeed we have Mellanox IB,
[:~]$ lspci | grep -i mellanox
02:00.0 Network controller: *Mellanox*Technologies MT27500 Family [ConnectX-3]
[:~]$ ibstatus
Infiniband device 'mlx4_0' port 1 status:
default gid: fe80::::7079:9003:0007:f538
base lid:0x76
sm lid:0x1
state: 4: ACTIVE
phys state:5: LinkUp
rate:56 Gb/sec (4X FDR)
link_layer:InfiniBand
I had no problem installing foss-2023a.eb in another cluster with :
$lspci | grep -i mellanox
86:00.0 Infiniband controller: *Mellanox*Technologies MT27800 Family
[ConnectX-5]
Thank you for any hint !
quim
Missatge de Åke Sandgren <mailto:ake.sandg...@umu.se>> del dia dv., 15 de març 2024 a les 12:18:
lspci | grep -i mellanox
will show if you have any mellanox devices on the system, some of
these could be used as normal ethernet devices though
ibstatus
will show if any of those are running in Infiniband mode
If ibstatus doesn't exist then you probably don't have infiniband, or
at least lack the packages for using it.
From: easybuild-requ...@lists.ugent.be
<mailto:easybuild-requ...@lists.ugent.be>
mailto:easybuild-requ...@lists.ugent.be>> on behalf of Joaquim Jornet
Somoza mailto:j.jornet.som...@gmail.com>>
Sent: Friday, March 15, 2024 11:44
To: easybuild@lists.ugent.be <mailto:easybuild@lists.ugent.be>
Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
installing foss-2023a.eb
Dear Ake,
How can I check this?
Thank you!
El vie, 15 mar 2024, 7:58, Åke Sandgren mailto:ake.sandg...@umu.se><mailto:ake.sandg...@umu.se
<mailto:ake.sandg...@umu.se>>> escribió:
No there is no bug there.
Which MOFED stack version are you using?
Or does your system lack Infiniband?
From: easybuild-requ...@lists.ugent.be
<mailto:easybuild-requ...@lists.ugent.be><mailto:easybuild-requ...@lists.ugent.be
<mailto:easybuild-requ...@lists.ugent.be>> mailto:easybuild-requ...@lists.ugent.be><mailto:easybuild-requ...@lists.ugent.be
<mailto:easybuild-requ...@lists.ugent.be>>> on behalf of Joaquim Jornet Somoza mailto:j.jornet.som...@gmail.com><mailto:j.jornet.som...@gmail.com <mailto:j.jornet.som...@gmail.com>>>
Sent: Thursday, March 14, 2024 16:02
To: easybuild@lists.ugent.be
<mailto:easybuild@lists.ugent.be><mailto:easybuild@lists.ugent.be
<mailto:easybuild@lists.ugent.be>>
Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
installing foss-2023a.eb
Dear easybuilders,
I am trying to install foss-2023a.eb on a RH7.7 servers, but when
installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with
the following error:
...
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../..
"-DCPU_FLAGS=|avx"
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include -O3 -g -Wall
-Werror -mavx -funwind-tables -Wno-missing-field-initializers -Wno-unused-parameter
-Wno-unused-label -Wno-long-long -Wno-endif-labels -Wno-sign-compare -Wno-multichar
-Wno-deprecated-declarations -Winvalid-pch -Wno-pointer-sign
-Werror-implicit-function-declaration -Wno-format-zero-length -Wnested-externs -Wshadow
-Werror=declaration-after-statement -O2 -ftree-vectorize -march=native -fno-math-errno
-fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
base/ib_md.c: In function 'uct_ib_md_access_flags':
base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first
use in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
638 | access_flags |= IBV_ACCESS_ON_DEMAND;
| ^~~~
| IBV_EXP_ACCESS_ON_DEMAND
base/ib_md.c:638:25: note: each undeclared identifier is reported only
once for each function it appears in
base/ib_md.c: In function 'uct_ib_mem_reg_internal':
base
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Well, RH7 is deprecated, and that's an OFED 3.3 stack. They (UCX devs) probably
didn't even come close to developing on something that old.
So IBV_ACCESS_ON_DEMAND doesn't exist in that old distro.
I strongly suggest updating, either both OS and OFED stack or just upgrade to
the latest MOFED stack supported on that OS
From: easybuild-requ...@lists.ugent.be on
behalf of Joaquim Jornet Somoza
Sent: Friday, March 15, 2024 12:49
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
installing foss-2023a.eb
Hi !
We are running on a RH7.2 (apology for the typo ..)
This is what I get:
[easybuild@nd48:~]$ rpm -qf /usr/include/infiniband/verbs.h
libibverbs-devel-1.1.8mlnx1-OFED.3.3.0.0.9.33100.x86_64
[easybuild@nd48:~]$ grep IBV_ACCESS_ON_DEMAND /usr/include/infiniband/verbs.h
[easybuild@nd48:~]$
Soi...there is no entry for IBV_ACCESS_ON_DEMAND
[easybuild@nd48:~]$ grep IBV_ACCESS /usr/include/infiniband/verbs.h
IBV_ACCESS_LOCAL_WRITE = 1,
IBV_ACCESS_REMOTE_WRITE = (1<<1),
IBV_ACCESS_REMOTE_READ = (1<<2),
IBV_ACCESS_REMOTE_ATOMIC= (1<<3),
IBV_ACCESS_MW_BIND = (1<<4)
Thank you !
quim
Missatge de Bart Oldeman
mailto:bart.olde...@calculquebec.ca>> del dia
dv., 15 de març 2024 a les 12:45:
Hi,
On 7.9 it's defined here:
$ rpm -qf /usr/include/infiniband/verbs.h
rdma-core-devel-22.4-6.el7_9.x86_64
$ grep IBV_ACCESS_ON_DEMAND /usr/include/infiniband/verbs.h
IBV_ACCESS_ON_DEMAND= (1<<6),
if you have MOFED it would be somewhere else, not sure.
Le jeu. 14 mars 2024, à 11 h 03, Joaquim Jornet Somoza
mailto:j.jornet.som...@gmail.com>> a écrit :
>
> Dear easybuilders,
>
> I am trying to install foss-2023a.eb on a RH7.7 servers, but when installing
> UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the following
> error:
> ...
> libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include
> -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
> -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
> -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
> -Wno-pointer-sign -Werror-implicit-function-declaration
> -Wno-format-zero-length -Wnested-externs -Wshadow
> -Werror=declaration-after-statement -O2 -ftree-vectorize -march=native
> -fno-math-errno -fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
> rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
> rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
> base/ib_md.c: In function 'uct_ib_md_access_flags':
> base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
> this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 638 | access_flags |= IBV_ACCESS_ON_DEMAND;
> | ^~~~
> | IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c:638:25: note: each undeclared identifier is reported only once
> for each function it appears in
> base/ib_md.c: In function 'uct_ib_mem_reg_internal':
> base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
> this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
> |^~~~
> |IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c: In function 'uct_ib_md_global_odp_init':
> base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
> this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 1449 |UCT_IB_MEM_ACCESS_FLAGS |
> IBV_ACCESS_ON_DEMAND,
> |
> ^~~~
> |
> IBV_EXP_ACCESS_ON_DEMAND
>
>
> Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND variable?
>
> Thank you !
>
> Best regards
> quim
> --
>
> Dr. Jo
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Hi !
We are running on a RH7.2 (apology for the typo ..)
This is what I get:
[easybuild@nd48:~]$ rpm -qf /usr/include/infiniband/verbs.h
libibverbs-devel-1.1.8mlnx1-OFED.3.3.0.0.9.33100.x86_64
[easybuild@nd48:~]$ grep IBV_ACCESS_ON_DEMAND
/usr/include/infiniband/verbs.h
[easybuild@nd48:~]$
Soi...there is no entry for IBV_ACCESS_ON_DEMAND
[easybuild@nd48:~]$ grep IBV_ACCESS /usr/include/infiniband/verbs.h
*IBV_ACCESS*_LOCAL_WRITE = 1,
*IBV_ACCESS*_REMOTE_WRITE = (1<<1),
*IBV_ACCESS*_REMOTE_READ = (1<<2),
*IBV_ACCESS*_REMOTE_ATOMIC= (1<<3),
*IBV_ACCESS*_MW_BIND = (1<<4)
Thank you !
quim
Missatge de Bart Oldeman del dia dv., 15 de
març 2024 a les 12:45:
> Hi,
>
> On 7.9 it's defined here:
> $ rpm -qf /usr/include/infiniband/verbs.h
> rdma-core-devel-22.4-6.el7_9.x86_64
>
> $ grep IBV_ACCESS_ON_DEMAND /usr/include/infiniband/verbs.h
> IBV_ACCESS_ON_DEMAND= (1<<6),
>
> if you have MOFED it would be somewhere else, not sure.
>
> Le jeu. 14 mars 2024, à 11 h 03, Joaquim Jornet Somoza
> a écrit :
> >
> > Dear easybuilders,
> >
> > I am trying to install foss-2023a.eb on a RH7.7 servers, but when
> installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the
> following error:
> > ...
> > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include
> -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
> -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
> -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
> -Wno-pointer-sign -Werror-implicit-function-declaration
> -Wno-format-zero-length -Wnested-externs -Wshadow
> -Werror=declaration-after-statement -O2 -ftree-vectorize -march=native
> -fno-math-errno -fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
> rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
> rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
> > base/ib_md.c: In function 'uct_ib_md_access_flags':
> > base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> > 638 | access_flags |= IBV_ACCESS_ON_DEMAND;
> > | ^~~~
> > | IBV_EXP_ACCESS_ON_DEMAND
> > base/ib_md.c:638:25: note: each undeclared identifier is reported only
> once for each function it appears in
> > base/ib_md.c: In function 'uct_ib_mem_reg_internal':
> > base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> > 751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
> > |^~~~
> > |IBV_EXP_ACCESS_ON_DEMAND
> > base/ib_md.c: In function 'uct_ib_md_global_odp_init':
> > base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first
> use in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> > 1449 |UCT_IB_MEM_ACCESS_FLAGS |
> IBV_ACCESS_ON_DEMAND,
> > |
> ^~~~
> > |
> IBV_EXP_ACCESS_ON_DEMAND
> >
> >
> > Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND
> variable?
> >
> > Thank you !
> >
> > Best regards
> > quim
> > --
> >
>
> > Dr. Joaquim Jornet Somoza
> > Técnico Superior de Cálculo Científico
> > Servicios Generales a la Investigación (SGIker)
> > Universidad del País Vasco (UPV/EHU)
> > email: j.jornet.som...@gmail.com
> > Edificio Joxe Maria Korta (Campus Gipuzkoa)
> > Av. Tolosa 72, 4a planta
> > 20018 Donostia-San Sebastián,
> > Gipuzkoa, Spain
> >
> > External Collaborator.
> > Nano-Bio Spectroscopy group
> > Departamento de Física de Materiales
> > Universidad del País Vasco (UPV/EHU)
> > Donostia-San Sebastián, Gipuzkoa, Spain
> >
> > The Max Planck Institute for the Structure and Dynamics of Matter (MPSD)
> > Bldg. 99 (CFEL)
> > Luruper Chaussee 149
> > 22761 Hamburg, Germany
>
>
>
> --
> Bart E. Oldeman, Ph.D. | bart.olde...@mcgill.ca |
> bart.olde...@calculquebec.ca
> Scientific Analyst / Analyste scientifique
> McGill HPC Centre / Centre de Calcul Haute Performance de McGill
> Calcul Québec | https://www.calculquebec.ca
> Digital Research Alliance
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Dear Ake,
Indeed we have Mellanox IB,
[:~]$ lspci | grep -i mellanox
02:00.0 Network controller: *Mellanox* Technologies MT27500 Family
[ConnectX-3]
[:~]$ ibstatus
Infiniband device 'mlx4_0' port 1 status:
default gid: fe80::::7079:9003:0007:f538
base lid:0x76
sm lid: 0x1
state: 4: ACTIVE
phys state: 5: LinkUp
rate:56 Gb/sec (4X FDR)
link_layer: InfiniBand
I had no problem installing foss-2023a.eb in another cluster with :
$ lspci | grep -i mellanox
86:00.0 Infiniband controller: *Mellanox* Technologies MT27800 Family
[ConnectX-5]
Thank you for any hint !
quim
Missatge de Åke Sandgren del dia dv., 15 de març 2024
a les 12:18:
> lspci | grep -i mellanox
> will show if you have any mellanox devices on the system, some of these
> could be used as normal ethernet devices though
>
> ibstatus
> will show if any of those are running in Infiniband mode
>
> If ibstatus doesn't exist then you probably don't have infiniband, or at
> least lack the packages for using it.
>
>
> From: easybuild-requ...@lists.ugent.be
> on behalf of Joaquim Jornet Somoza
> Sent: Friday, March 15, 2024 11:44
> To: easybuild@lists.ugent.be
> Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
> installing foss-2023a.eb
>
> Dear Ake,
>
> How can I check this?
>
> Thank you!
>
> El vie, 15 mar 2024, 7:58, Åke Sandgren ake.sandg...@umu.se>> escribió:
> No there is no bug there.
>
> Which MOFED stack version are you using?
> Or does your system lack Infiniband?
>
>
> From: easybuild-requ...@lists.ugent.be easybuild-requ...@lists.ugent.be> <mailto:easybuild-requ...@lists.ugent.be>> on behalf of Joaquim Jornet
> Somoza mailto:j.jornet.som...@gmail.com>>
> Sent: Thursday, March 14, 2024 16:02
> To: easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>
> Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
> installing foss-2023a.eb
>
> Dear easybuilders,
>
> I am trying to install foss-2023a.eb on a RH7.7 servers, but when
> installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the
> following error:
> ...
> libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include
> -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
> -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
> -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
> -Wno-pointer-sign -Werror-implicit-function-declaration
> -Wno-format-zero-length -Wnested-externs -Wshadow
> -Werror=declaration-after-statement -O2 -ftree-vectorize -march=native
> -fno-math-errno -fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
> rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
> rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
> base/ib_md.c: In function 'uct_ib_md_access_flags':
> base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 638 | access_flags |= IBV_ACCESS_ON_DEMAND;
> | ^~~~
> | IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c:638:25: note: each undeclared identifier is reported only
> once for each function it appears in
> base/ib_md.c: In function 'uct_ib_mem_reg_internal':
> base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
> |^~~~
> |IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c: In function 'uct_ib_md_global_odp_init':
> base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 1449 |UCT_IB_MEM_ACCESS_FLAGS |
> IBV_ACCESS_ON_DEMAND,
> |
> ^~~~
> |
> IBV_EXP_ACCESS_ON_DEMAND
>
>
> Any
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Hi,
On 7.9 it's defined here:
$ rpm -qf /usr/include/infiniband/verbs.h
rdma-core-devel-22.4-6.el7_9.x86_64
$ grep IBV_ACCESS_ON_DEMAND /usr/include/infiniband/verbs.h
IBV_ACCESS_ON_DEMAND= (1<<6),
if you have MOFED it would be somewhere else, not sure.
Le jeu. 14 mars 2024, à 11 h 03, Joaquim Jornet Somoza
a écrit :
>
> Dear easybuilders,
>
> I am trying to install foss-2023a.eb on a RH7.7 servers, but when installing
> UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the following
> error:
> ...
> libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include
> -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
> -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
> -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
> -Wno-pointer-sign -Werror-implicit-function-declaration
> -Wno-format-zero-length -Wnested-externs -Wshadow
> -Werror=declaration-after-statement -O2 -ftree-vectorize -march=native
> -fno-math-errno -fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
> rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
> rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
> base/ib_md.c: In function 'uct_ib_md_access_flags':
> base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
> this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 638 | access_flags |= IBV_ACCESS_ON_DEMAND;
> | ^~~~
> | IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c:638:25: note: each undeclared identifier is reported only once
> for each function it appears in
> base/ib_md.c: In function 'uct_ib_mem_reg_internal':
> base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
> this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
> |^~~~
> |IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c: In function 'uct_ib_md_global_odp_init':
> base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
> this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 1449 |UCT_IB_MEM_ACCESS_FLAGS |
> IBV_ACCESS_ON_DEMAND,
> |
> ^~~~
> |
> IBV_EXP_ACCESS_ON_DEMAND
>
>
> Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND variable?
>
> Thank you !
>
> Best regards
> quim
> --
>
> Dr. Joaquim Jornet Somoza
> Técnico Superior de Cálculo Científico
> Servicios Generales a la Investigación (SGIker)
> Universidad del País Vasco (UPV/EHU)
> email: j.jornet.som...@gmail.com
> Edificio Joxe Maria Korta (Campus Gipuzkoa)
> Av. Tolosa 72, 4a planta
> 20018 Donostia-San Sebastián,
> Gipuzkoa, Spain
>
> External Collaborator.
> Nano-Bio Spectroscopy group
> Departamento de Física de Materiales
> Universidad del País Vasco (UPV/EHU)
> Donostia-San Sebastián, Gipuzkoa, Spain
>
> The Max Planck Institute for the Structure and Dynamics of Matter (MPSD)
> Bldg. 99 (CFEL)
> Luruper Chaussee 149
> 22761 Hamburg, Germany
--
Bart E. Oldeman, Ph.D. | bart.olde...@mcgill.ca | bart.olde...@calculquebec.ca
Scientific Analyst / Analyste scientifique
McGill HPC Centre / Centre de Calcul Haute Performance de McGill
Calcul Québec | https://www.calculquebec.ca
Digital Research Alliance of / Alliance de recherche numérique du
Canada | https://www.alliancecan.ca
T: 514-396-8926
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
lspci | grep -i mellanox
will show if you have any mellanox devices on the system, some of these could
be used as normal ethernet devices though
ibstatus
will show if any of those are running in Infiniband mode
If ibstatus doesn't exist then you probably don't have infiniband, or at least
lack the packages for using it.
From: easybuild-requ...@lists.ugent.be on
behalf of Joaquim Jornet Somoza
Sent: Friday, March 15, 2024 11:44
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
installing foss-2023a.eb
Dear Ake,
How can I check this?
Thank you!
El vie, 15 mar 2024, 7:58, Åke Sandgren
mailto:ake.sandg...@umu.se>> escribió:
No there is no bug there.
Which MOFED stack version are you using?
Or does your system lack Infiniband?
From: easybuild-requ...@lists.ugent.be<mailto:easybuild-requ...@lists.ugent.be>
mailto:easybuild-requ...@lists.ugent.be>> on
behalf of Joaquim Jornet Somoza
mailto:j.jornet.som...@gmail.com>>
Sent: Thursday, March 14, 2024 16:02
To: easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>
Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing
foss-2023a.eb
Dear easybuilders,
I am trying to install foss-2023a.eb on a RH7.7 servers, but when installing
UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the following error:
...
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include -O3
-g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
-Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
-Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
-Wno-pointer-sign -Werror-implicit-function-declaration -Wno-format-zero-length
-Wnested-externs -Wshadow -Werror=declaration-after-statement -O2
-ftree-vectorize -march=native -fno-math-errno -fPIC -MT
rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
base/ib_md.c: In function 'uct_ib_md_access_flags':
base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
638 | access_flags |= IBV_ACCESS_ON_DEMAND;
| ^~~~
| IBV_EXP_ACCESS_ON_DEMAND
base/ib_md.c:638:25: note: each undeclared identifier is reported only once for
each function it appears in
base/ib_md.c: In function 'uct_ib_mem_reg_internal':
base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
|^~~~
|IBV_EXP_ACCESS_ON_DEMAND
base/ib_md.c: In function 'uct_ib_md_global_odp_init':
base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
1449 |UCT_IB_MEM_ACCESS_FLAGS |
IBV_ACCESS_ON_DEMAND,
|
^~~~
|
IBV_EXP_ACCESS_ON_DEMAND
Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND variable?
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Dear Ake,
How can I check this?
Thank you!
El vie, 15 mar 2024, 7:58, Åke Sandgren escribió:
> No there is no bug there.
>
> Which MOFED stack version are you using?
> Or does your system lack Infiniband?
>
>
> From: easybuild-requ...@lists.ugent.be
> on behalf of Joaquim Jornet Somoza
> Sent: Thursday, March 14, 2024 16:02
> To: easybuild@lists.ugent.be
> Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when
> installing foss-2023a.eb
>
> Dear easybuilders,
>
> I am trying to install foss-2023a.eb on a RH7.7 servers, but when
> installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the
> following error:
> ...
> libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
> -I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
> -I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
> -I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include
> -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
> -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
> -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
> -Wno-pointer-sign -Werror-implicit-function-declaration
> -Wno-format-zero-length -Wnested-externs -Wshadow
> -Werror=declaration-after-statement -O2 -ftree-vectorize -march=native
> -fno-math-errno -fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
> rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
> rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
> base/ib_md.c: In function 'uct_ib_md_access_flags':
> base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 638 | access_flags |= IBV_ACCESS_ON_DEMAND;
> | ^~~~
> | IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c:638:25: note: each undeclared identifier is reported only
> once for each function it appears in
> base/ib_md.c: In function 'uct_ib_mem_reg_internal':
> base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
> |^~~~
> |IBV_EXP_ACCESS_ON_DEMAND
> base/ib_md.c: In function 'uct_ib_md_global_odp_init':
> base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use
> in this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
> 1449 |UCT_IB_MEM_ACCESS_FLAGS |
> IBV_ACCESS_ON_DEMAND,
> |
> ^~~~
> |
> IBV_EXP_ACCESS_ON_DEMAND
>
>
> Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND
> variable?
>
Re: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
No there is no bug there.
Which MOFED stack version are you using?
Or does your system lack Infiniband?
From: easybuild-requ...@lists.ugent.be on
behalf of Joaquim Jornet Somoza
Sent: Thursday, March 14, 2024 16:02
To: easybuild@lists.ugent.be
Subject: [easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing
foss-2023a.eb
Dear easybuilders,
I am trying to install foss-2023a.eb on a RH7.7 servers, but when installing
UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the following error:
...
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include -O3
-g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
-Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
-Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
-Wno-pointer-sign -Werror-implicit-function-declaration -Wno-format-zero-length
-Wnested-externs -Wshadow -Werror=declaration-after-statement -O2
-ftree-vectorize -march=native -fno-math-errno -fPIC -MT
rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1
base/ib_md.c: In function 'uct_ib_md_access_flags':
base/ib_md.c:638:25: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
638 | access_flags |= IBV_ACCESS_ON_DEMAND;
| ^~~~
| IBV_EXP_ACCESS_ON_DEMAND
base/ib_md.c:638:25: note: each undeclared identifier is reported only once for
each function it appears in
base/ib_md.c: In function 'uct_ib_mem_reg_internal':
base/ib_md.c:751:24: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
751 | if (access_flags & IBV_ACCESS_ON_DEMAND) {
|^~~~
|IBV_EXP_ACCESS_ON_DEMAND
base/ib_md.c: In function 'uct_ib_md_global_odp_init':
base/ib_md.c:1449:54: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in
this function); did you mean 'IBV_EXP_ACCESS_ON_DEMAND'?
1449 |UCT_IB_MEM_ACCESS_FLAGS |
IBV_ACCESS_ON_DEMAND,
|
^~~~
|
IBV_EXP_ACCESS_ON_DEMAND
Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND variable?
[easybuild] Failure in UCX-1.14.1-GCCcore-12.3.0.eb when installing foss-2023a.eb
Dear easybuilders,
I am trying to install foss-2023a.eb on a RH7.7 servers, but when
installing UCX-1.14.1-GCCcore-12.3.0.eb , the installation fails with the
following error:
*...*
*libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. "-DCPU_FLAGS=|avx"
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1
-I/dev/shm/easybuild/UCX/1.14.1/GCCcore-12.3.0/ucx-1.14.1/src
-I/software/easybuild/x86_64/software/numactl/2.0.16-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/zlib/1.2.13-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/pkgconf/1.9.5-GCCcore-12.3.0/include
-I/software/easybuild/x86_64/software/binutils/2.40-GCCcore-12.3.0/include
-O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers
-Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels
-Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
-Wno-pointer-sign -Werror-implicit-function-declaration
-Wno-format-zero-length -Wnested-externs -Wshadow
-Werror=declaration-after-statement -O2 -ftree-vectorize -march=native
-fno-math-errno -fPIC -MT rc/verbs/libuct_ib_la-rc_verbs_ep.lo -MD -MP -MF
rc/verbs/.deps/libuct_ib_la-rc_verbs_ep.Tpo -c rc/verbs/rc_verbs_ep.c -o
rc/verbs/libuct_ib_la-rc_verbs_ep.o >/dev/null 2>&1base/ib_md.c: In
function 'uct_ib_md_access_flags':base/ib_md.c:638:25: error:
'IBV_ACCESS_ON_DEMAND' undeclared (first use in this function); did you
mean 'IBV_EXP_ACCESS_ON_DEMAND'? 638 | access_flags |=
IBV_ACCESS_ON_DEMAND; | ^~~~
| IBV_EXP_ACCESS_ON_DEMANDbase/ib_md.c:638:25:
note: each undeclared identifier is reported only once for each function it
appears inbase/ib_md.c: In function
'uct_ib_mem_reg_internal':base/ib_md.c:751:24: error:
'IBV_ACCESS_ON_DEMAND' undeclared (first use in this function); did you
mean 'IBV_EXP_ACCESS_ON_DEMAND'? 751 | if (access_flags &
IBV_ACCESS_ON_DEMAND) { |^~~~
|IBV_EXP_ACCESS_ON_DEMANDbase/ib_md.c: In
function 'uct_ib_md_global_odp_init':base/ib_md.c:1449:54: error:
'IBV_ACCESS_ON_DEMAND' undeclared (first use in this function); did you
mean 'IBV_EXP_ACCESS_ON_DEMAND'? 1449 |
UCT_IB_MEM_ACCESS_FLAGS | IBV_ACCESS_ON_DEMAND, |
^~~~ |
IBV_EXP_ACCESS_ON_DEMAND*
Any hint on how to fix it? Is there a bug with IBV_ACCESS_ON_DEMAND
variable?
Thank you !
Best regards
quim
--
*Dr. Joaquim Jornet Somoza*
*Técnico Superior de Cálculo Científico *
Servicios Generales a la Investigación (*SGIker*)
Universidad del País Vasco (*UPV/EHU*)
email: j.jornet.som...@gmail.com
Edificio Joxe Maria Korta (Campus Gipuzkoa)
Av. Tolosa 72, 4a planta
20018 Donostia-San Sebastián,
Gipuzkoa, Spain
*External Collaborator.*
Nano-Bio Spectroscopy group
Departamento de Física de Materiales
Universidad del País Vasco (UPV/EHU)
Donostia-San Sebastián, Gipuzkoa, Spain
The Max Planck Institute for the Structure and Dynamics of Matter (MPSD)
Bldg. 99 (CFEL)
Luruper Chaussee 149
22761 Hamburg, Germany
