from libopen-pal.la :
dependency_libs=' -lrdmacm -libverbs -lscif -lnuma -ldl -lrt -lnsl
-lutil -lm'
i confirm mpicc fails linking
but FWIT, using libtool does work (!)
could the bug come from the mpicc (and other) wrappers ?
Gilles
$ gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c
-I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath
-Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib
-Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte
-lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi
$ /tmp/install/ompi.noromio/bin/mpicc -g -O0 -o hw -show ~/hw.c
gcc -g -O0 -o hw /csc/home1/gouaillardet/hw.c
-I/tmp/install/ompi.noromio/include -pthread -L/usr/lib64 -Wl,-rpath
-Wl,/usr/lib64 -Wl,-rpath -Wl,/tmp/install/ompi.noromio/lib
-Wl,--enable-new-dtags -L/tmp/install/ompi.noromio/lib -lmpi -lopen-rte
-lopen-pal -lm -lnuma -libverbs -lscif -lrdmacm -ldl -llustreapi
[gouaillardet@soleil build]$ /tmp/install/ompi.noromio/bin/mpicc -g -O0
-o hw ~/hw.c
/tmp/install/ompi.noromio/lib/libmpi.a(fbtl_posix_ipwritev.o): In
function `mca_fbtl_posix_ipwritev':
fbtl_posix_ipwritev.c:(.text+0x17b): undefined reference to `aio_write'
fbtl_posix_ipwritev.c:(.text+0x237): undefined reference to `aio_write'
fbtl_posix_ipwritev.c:(.text+0x3f4): undefined reference to `aio_write'
fbtl_posix_ipwritev.c:(.text+0x48e): undefined reference to `aio_write'
/tmp/install/ompi.noromio/lib/libopen-pal.a(opal_pty.o): In function
`opal_openpty':
opal_pty.c:(.text+0x1): undefined reference to `openpty'
/tmp/install/ompi.noromio/lib/libopen-pal.a(event.o): In function
`event_add_internal':
event.c:(.text+0x288d): undefined reference to `clock_gettime'
$ /bin/sh ./static/libtool --silent --tag=CC --mode=compile gcc
-std=gnu99 -I/tmp/install/ompi.noromio/include -c ~/hw.c
$ /bin/sh ./static/libtool --silent --tag=CC --mode=link gcc
-std=gnu99 -o hw hw.o -L/tmp/install/ompi.noromio/lib -lmpi
$ ldd hw
linux-vdso.so.1 => (0x00007fff7530d000)
librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x00007f0ed541e000)
libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x00007f0ed5210000)
libscif.so.0 => /usr/lib64/libscif.so.0 (0x0000003b9c600000)
libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x0000003ba5600000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003b9be00000)
librt.so.1 => /lib64/librt.so.1 (0x0000003b9ca00000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003bae200000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003bac600000)
libm.so.6 => /lib64/libm.so.6 (0x0000003b9ba00000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003b9c200000)
libc.so.6 => /lib64/libc.so.6 (0x0000003b9b600000)
/lib64/ld-linux-x86-64.so.2 (0x0000003b9b200000)
On 2014/08/05 7:56, Ralph Castain wrote:
> My thought was to post initially as a blocker, pending a discussion with Jeff
> at tomorrow's telecon. If he thinks this is something we can fix in some
> central point (thus catching it everywhere), then it could be quick and worth
> doing. However, I'm skeptical as I tried to do that in the most obvious
> place, and it failed (could be operator error).
>
> Will let you know tomorrow. Truly appreciate your digging on this!
> Ralph
>
> On Aug 4, 2014, at 3:50 PM, Paul Hargrove <[email protected]> wrote:
>
>> Ralph and Jeff,
>>
>> I've been digging and find the problem is wider than just the one library
>> and has manifestations specific to FreeBSD, NetBSD and Solaris. I am adding
>> new info to the ticket as I unearth it.
>>
>> Additionally, it appears this existed in 1.8, 1.8.1 and in the 1.7 series as
>> well.
>> So, would suggest this NOT be a blocker for a 1.8.2 release.
>>
>> Of course I am willing to provide testing if you still want to push for a
>> quick resolution.
>>
>> -Paul
>>
>>
>> On Mon, Aug 4, 2014 at 1:27 PM, Ralph Castain <[email protected]> wrote:
>> Okay, I filed a blocker on this for 1.8.2 and assigned it to Jeff. I took a
>> crack at fixing it, but came up short :-(
>>
>>
>> On Aug 3, 2014, at 10:46 PM, Paul Hargrove <[email protected]> wrote:
>>
>>> I've identified the difference between the platform that does link libutil
>>> and the one that does not.
>>>
>>> 1) libutil is linked (as an OMPI dependency) only on the working system:
>>>
>>> Working system:
>>> $ grep 'checking for .* LIBS' configure.out
>>> checking for OPAL LIBS... -lm -lpciaccess -ldl
>>> checking for ORTE LIBS... -lm -lpciaccess -ldl -ltorque
>>> checking for OMPI LIBS... -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil
>>>
>>> NON-working system:
>>> $ grep 'checking for .* LIBS' configure.out
>>> checking for OPAL LIBS... -lm -ldl
>>> checking for ORTE LIBS... -lm -ldl -ltorque
>>> checking for OMPI LIBS... -lm -ldl -ltorque
>>>
>>> So, the working system that does link libutil is doing so as an OMPI
>>> dependency.
>>> However it is also needed for opal (only caller of openpty is
>>> opal/util/open_pty.c).
>>>
>>> 2) Only the working system is building ROMIO:
>>>
>>> Comparing the 'checking if * can compile' lines of configure output shows
>>> only ONE difference:
>>>
>>> checking if MCA component fs:ufs can compile... yes
>>> checking if MCA component fs:pvfs2 can compile... no
>>> checking if MCA component io:ompio can compile... yes
>>> -checking if MCA component io:romio can compile... no
>>> +checking if MCA component io:romio can compile... yes
>>> checking if MCA component mpool:grdma can compile... yes
>>> checking if MCA component mpool:sm can compile... yes
>>> checking if MCA component mpool:udreg can compile... no
>>>
>>> So, it appears that *if* ROMIO is configured in, then "-lutil" gets added
>>> to OMPI_WRAPPER_EXTRA_LIBS.
>>> This masks the fact that it is missing from OPAL_WRAPPER_EXTRA_LIBS.
>>>
>>>
>>> I have confirmed that I can reproduce the static linking failure by adding
>>> --disable-io-romio to the configure options of the system that worked
>>> previously.
>>>
>>> So, I update my report (and the email subject line) to:
>>> Static linking fails on Linux when not building ROMIO
>>>
>>> -Paul
>>>
>>>
>>>
>>> On Sun, Aug 3, 2014 at 6:22 PM, Paul Hargrove <[email protected]> wrote:
>>> Hmm,
>>>
>>> On a different Linux/x86-64 host things work as expected with '-lutil'
>>> linked explicitly:
>>>
>>> $ ./INST/bin/mpicc -showme BLD/examples/hello_c.c
>>> pgcc BLD/examples/hello_c.c
>>> -I/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/include
>>> -L/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib
>>> -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath
>>> -Wl,/opt/torque/4.2.7.h1/lib -Wl,-rpath -Wl,/opt/torque/4.2.7.h1/lib
>>> -Wl,-rpath
>>> -Wl,/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib
>>>
>>> -L/scratch/scratchdirs/hargrove/OMPI/openmpi-1.8.2rc3-linux-x86_64-pgi-14.1/INST/lib
>>> -lmpi -lopen-rte -lopen-pal -lm -lpciaccess -ldl -ltorque -lrt -lnsl -lutil
>>>
>>> Searching for relevant differences now...
>>>
>>> -Paul
>>>
>>>
>>> On Sun, Aug 3, 2014 at 4:58 PM, Paul Hargrove <[email protected]> wrote:
>>>
>>> I've configured the 1.8.2rc3 tarball with "--enable-static
>>> --disable-shared" on a fairly standard Linux/x86-64 platform. While there
>>> are no problems on the same platform w/o these configure flags, with them I
>>> cannot link any application codes.
>>>
>>> $ mpicc -g hello_c.c -o hello_c
>>> /global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib/libopen-pal.a(opal_pty.o):
>>> In function `opal_openpty':
>>> opal_pty.c:(.text+0x1): undefined reference to `openpty'
>>>
>>> I checked "make openpty" and the manpage says to link with '-lutil'.
>>> The '-showme' does not show libutil:
>>>
>>> $ mpicc -showme hello_c.c
>>> gcc hello_c.c
>>> -I/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/include
>>> -pthread -L/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath
>>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath
>>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath
>>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath
>>> -Wl,/usr/syscom/opt/torque/4.1.4/lib -Wl,-rpath
>>> -Wl,/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib
>>> -Wl,--enable-new-dtags
>>> -L/global/homes/h/hargrove/GSCRATCH/OMPI/openmpi-1.8.2rc3-linux-x86_64-static/INST/lib
>>> -lmpi -lopen-rte -lopen-pal -lm -ldl -ltorque -libverbs -lrdmacm
>>>
>>>
>>> It looks like configure is doing the right thing on some level, but failing
>>> to add '-lutil' to the appropriate list of libs (OPAL_WRAPPER_EXTRA_LIBS?):
>>>
>>> ============================================================================
>>> == Library and Function tests
>>> ============================================================================
>>> checking if we need -lutil for openpty... yes
>>> checking for openpty... yes
>>>
>>>
>>> -Paul
>>>
>>> --
>>> Paul H. Hargrove [email protected]
>>> Future Technologies Group
>>> Computer and Data Sciences Department Tel: +1-510-495-2352
>>> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
>>>
>>>
>>>
>>> --
>>> Paul H. Hargrove [email protected]
>>> Future Technologies Group
>>> Computer and Data Sciences Department Tel: +1-510-495-2352
>>> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
>>>
>>>
>>>
>>> --
>>> Paul H. Hargrove [email protected]
>>> Future Technologies Group
>>> Computer and Data Sciences Department Tel: +1-510-495-2352
>>> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
>>> _______________________________________________
>>> devel mailing list
>>> [email protected]
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/devel/2014/08/15492.php
>>
>> _______________________________________________
>> devel mailing list
>> [email protected]
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post:
>> http://www.open-mpi.org/community/lists/devel/2014/08/15501.php
>>
>>
>>
>> --
>> Paul H. Hargrove [email protected]
>> Future Technologies Group
>> Computer and Data Sciences Department Tel: +1-510-495-2352
>> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
>> _______________________________________________
>> devel mailing list
>> [email protected]
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post:
>> http://www.open-mpi.org/community/lists/devel/2014/08/15503.php
>
>
>
> _______________________________________________
> devel mailing list
> [email protected]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/08/15504.php
