Re: [OMPI devel] Open MPI (not quite) on Cray XC30

On Wed, 30 Jan 2013 10:09:09 -0800 Ralph Castain wrote: > I'm afraid that optimized-common doesn't stand alone - it requires other > modules as well. If you look at optimized-lustre, you'll see that we include > cray-common, optimized-common, and gnu-common, and then add the

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Use the optimized-lustre. optimized-common is not intended to be used on its own. -Nathan On Wed, Jan 30, 2013 at 06:31:09PM +0100, Jure Pe??ar wrote: > On Wed, 30 Jan 2013 08:48:02 -0700 > Nathan Hjelm wrote: > > > Try configuring with > >

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

I'm afraid that optimized-common doesn't stand alone - it requires other modules as well. If you look at optimized-lustre, you'll see that we include cray-common, optimized-common, and gnu-common, and then add the lustre flags. You need all three of those modules to build successfully. If you

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

On Wed, 30 Jan 2013 08:48:02 -0700 Nathan Hjelm wrote: > Try configuring with > --with-platform=contrib/platform/lanl/cray_xe6/optimized-nopanasas . That > might work. If it doesn't try the optimized-lustre file from the same > directory in trunk. > I used optimized-common.

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Try configuring with --with-platform=contrib/platform/lanl/cray_xe6/optimized-nopanasas . That might work. If it doesn't try the optimized-lustre file from the same directory in trunk. -Nathan On Wed, Jan 30, 2013 at 04:28:14PM +0100, Jure Pe??ar wrote: > On Mon, 28 Jan 2013 12:28:34 -0800 >

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

On Mon, 28 Jan 2013 12:28:34 -0800 Paul Hargrove wrote: > I will be happy to retest on both the XC30 and XE6 at NERSC from a nightly > tarball with the fixes. > Please give me a heads up when that is available. I'd be interested in that too. Hello all, sorry to hijack a

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

I will be happy to retest on both the XC30 and XE6 at NERSC from a nightly tarball with the fixes. Please give me a heads up when that is available. -Paul On Mon, Jan 28, 2013 at 7:52 AM, Ralph Castain wrote: > The key was to --enable-static --disable-shared. That's the

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

The key was to --enable-static --disable-shared. That's the only way to generate the problem. Brian was already aware of it and fixed it this weekend. I tested the fix and it works fine. Waiting for Jeff to review it before committing to the trunk. On Jan 28, 2013, at 7:45 AM, Nathan Hjelm

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Try building static. Lots of errors due to missing libraries in libs_static. -Nathan On Fri, Jan 25, 2013 at 04:09:16PM -0800, Ralph Castain wrote: > FWIW: I can build it fine without setting any of the CC... flags on LANL's > Cray XE6, and mpicc worked just fine for me once built that way. >

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Ralph, Again our results differ. I did NOT need the additional #include to link a simple test program. I am going to try on our XE6 shortly. I suspect you are right about something in the configury being different. I am willing to try a few more nightly tarballs if somebody thinks they have the

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

On Jan 25, 2013, at 5:12 PM, Paul Hargrove wrote: > Ralph, > > Those are the result of the missing -lnuma that Nathan already identified > earlier as missing in BOTH 1.7 and trunk. > I see MORE missing symbols, which include ones from libxpmem and libugni. Alright, let me

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Ralph, Those are the result of the missing -lnuma that Nathan already identified earlier as missing in BOTH 1.7 and trunk. I see MORE missing symbols, which include ones from libxpmem and libugni. -Paul On Fri, Jan 25, 2013 at 4:59 PM, Ralph Castain wrote: > > On Jan 25,

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

In contrast to what Ralph just described, I found that when using gcc/g++/gfortran instead of cc/CC/ftn, I had to edit ${prefix}/share/openmpi/mpicc-wrapper-data.txt. To get a successful link, I had to add the following to libs_static: -lnuma -lxpmem -lugni To linker_flags I had to add the

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

I found the problem - it is an error in hwloc. It's missing a required include . I'll fix and push it upstream. On Jan 25, 2013, at 4:59 PM, Ralph Castain wrote: > > On Jan 25, 2013, at 4:53 PM, Ralph Castain wrote: >> The repeated libs is something we

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Ralph, LANL's platform file for the XE6 has enabled shared libs, which I must disable on our XC30 which is not setup for them. So, I believe Nathan's text A number of libraries (-lxpmem, -lugni, etc) are missing from libs_static. is the "borkedness" in mpicc, and shared-vs-static is probably

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Here is what my mpicc looks like: ct-fe1 /users/rcastain> mpicc --showme gcc -I/usr/aprojects/hpctools/rhc/build/include/openmpi/opal/mca/hwloc/hwloc151/hwloc/include -I/usr/aprojects/hpctools/rhc/build/include/openmpi/opal/mca/event/libevent2019/libevent

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

FWIW: I can build it fine without setting any of the CC... flags on LANL's Cray XE6, and mpicc worked just fine for me once built that way. So I'm not quite sure I understand the "mpicc is completely borked in the trunk". Can you elaborate? On Jan 25, 2013, at 3:59 PM, Paul Hargrove

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Nathan, The 2nd and 3rd non-blank lines of my original post: > Given that it is INTENDED to be API-compatible with the XE series, I began > configuring with > CC=cc CXX=CC FC=ftn --with-platform=lanl/cray_xe6/optimized-nopanasas So, I am surprised that nobody objected before now to my use

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Hmm, I see mpicc in there. It will use the compiler directly instead of Cray's wrappers. We didn't want Open MPI's wrapper linking in MPT afterall ;). mpicc is completely borked in the trunk. If you want to use the Cray wrappers with Open MPI I can give you a module file that sets up the

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Adding --without-lustre to my configure args allowed me to compile and link ring_c. I am in the queue now and will report later on run results. -Paul On Fri, Jan 25, 2013 at 2:13 PM, Paul Hargrove wrote: > Still having problems on the Cray XC30, but now they are when

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Nathan, Cray's "cc" wrapper is adding xpmem, ugni, pmi, alps and others already: $ cc -v hello.c 2>&1 | grep collect > /opt/gcc/4.7.2/snos/libexec/gcc/x86_64-suse-linux/4.7.2/collect2 > --sysroot= -m elf_x86_64 -static -u pthread_mutex_trylock -u > pthread_mutex_destroy -u pthread_create

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Something is wrong with the wrappers. A number of libraries (-lxpmem, -lugni, etc) are missing from libs_static. Might be a similar issue with eh missing -llustreapi. Going to create a critical bug to track this issue. Works in 1.7 :-/ ... If you add -lnuma to libs_static in

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Still having problems on the Cray XC30, but now they are when linking an MPI app: $ ./INSTALL/bin/mpicc -o ring_c examples/ring_c.c > fs_lustre_file_open.c:(.text+0x130): undefined reference to > `llapi_file_create' > fs_lustre_file_open.c:(.text+0x17e): undefined reference to >

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Thanks Paul I'm currently tracking down a problem on the Cray XE6 - it appears that recent OS release changed the way alps stores allocation info :-( Will hopefully have it running soon. On Jan 25, 2013, at 10:50 AM, Paul Hargrove wrote: > I was able to compile with

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

I was able to compile with openmpi-1.9a1r27905.tar.bz I'll report again when I've had an opportunity to run something like ring_c. Thanks, -Paul On Tue, Jan 22, 2013 at 6:08 PM, Ralph Castain wrote: > I went ahead and removed the duplicate code, so this should work now.

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

I went ahead and removed the duplicate code, so this should work now. The problem is that we re-factored the ompi_info/orte-info code, but didn't complete the job - specifically, the orte-info tool didn't get updated. It's about to get revamped yet again when the ompi-rte branch gets committed

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

I am using the openmpi-1.9a1r27886 tarball and I still see an error for one of the two duplicate symbols: CCLD orte-info ../../../orte/.libs/libopen-rte.a(orte_info_support.o): In function `orte_info_show_orte_version': ../../orte/runtime/orte_info_support.c:(.text+0xe10): multiple

Re: [OMPI devel] Open MPI (not quite) on Cray XC30

Luckily for us all the definitions contain the same constant (orte). r27864 should fix this. George. On Jan 18, 2013, at 06:21 , Paul Hargrove wrote: > My employer has a nice new Cray XC30 (aka Cascade), and I thought I'd give > Open MPI a quick test. > > Given that

[OMPI devel] Open MPI (not quite) on Cray XC30

My employer has a nice new Cray XC30 (aka Cascade), and I thought I'd give Open MPI a quick test. Given that it is INTENDED to be API-compatible with the XE series, I began configuring with CC=cc CXX=CC FC=ftn --with-platform=lanl/cray_xe6/optimized-nopanasas However, since this is Intel h/w,