Hi, On Wednesday 23 June 2010, Todd Gamblin wrote: > Hi, > > I'm doing research on scalable performance tools at Lawrence Livermore and > my group is very enthusiastic about switching to CMake for most of our > development. > > I've been trying to get a proof of concept port from autotools to CMake > done for a simple project, and things seem to work ok for our Linux systems > but not for our BlueGene/P systems. I'm trying to get a BlueGene/P > platform file together, and while there seem to be posts on this subject to > this list before, I couldn't find anything that worked. BG/P supports > shared libraries with two compiler toolchains, so this is a bit more > involved than the BlueGene/L file that's already in CMake. BG/P also has a > crazier system library layout, and I'm not sure that there is sufficient > support in CMake for this or for the way we typically install packages for > the BG compute nodes. > > I'm listing the issues I've discovered below. Are there ways around these > that I'm missing? > > > 1. Bootstrap script doesn't seem to work quite right on the frontend. > > Building CMake was a bit hairy on the Power6 front end node on our BlueGene > systems. The machine runs Linux and has both GNU and IBM XL toolchains for > the frontend itself. The CMake bootstrap script detected cc for the C > compiler and xlC for the C++ compiler, and linking failed on something 50% > through the build because it was passing -dynamic to xlC, which is the > wrong flag. It would be nice if CMake would prefer one or the other tool > chain. > > I was able to build with either if I just manually specified the compilers and linker like this: > > env CC=gcc C++=g++ LD=ld ./bootstrap --prefix=${CMAKE_HOME} > > > > env CC=xlc C++=xlC LD=xlC ./bootstrap --prefix=${CMAKE_HOME} > > But It would've been nice if I didn't have to do all that. Is PPC Linux > not used many other places anymore? I guess this is a sles10/ppc64 > environment, so maybe that is throwing the script off.
This doesn't seem to be related to the cross compiling part. Bill usually chimes in on such issues... > 2. The FindMPI module's library search breaks because the the cross-compiler name on BlueGene/P contains -l. Specifically, the GNU compute node compilers on BlueGene/P are named like this: > > powerpc-bgp-linux-g++ > > And the FindMPI module claims that it can't find the '-inux-g++' library, > or something similar. I was able to fix this by modifying FindMPI's search > to look for -l only when it's followed by a space. This is a little > worrisome because I think that is a standard gnu naming convention, so > other platforms with compilers named blah-linux are going to have the same > bug. Please add a bug report for this on http://public.kitware.com/Bug/ > 3. Finding system libraries in directories not named 'lib' > > CMake cross compile support doesn't seem robust enough for platforms where > there is more than a simple system environment directory (specified with > CMAKE_FIND_ROOT_PATH) and a custom user directory where platform binaries > will go (also included in CMAKE_FIND_ROOT_PATH). All the online > documentation and the latest edition of the book seem to assume a system > where there are a very limited number of places where backend libraries > might be. > > To start my port, I've tried to set up a BlueGene/P GNU tool chain file like this, based on the advice on the webpage: > > # the name of the target operating system > > set(CMAKE_SYSTEM_NAME BlueGeneP) > > > > # set the compiler > > set(CMAKE_C_COMPILER > > /bgsys/drivers/ppcfloor/gnu-linux/bin/powerpc-bgp-linux-gcc ) > > set(CMAKE_CXX_COMPILER > > /bgsys/drivers/ppcfloor/gnu-linux/bin/powerpc-bgp-linux-g++ ) > > > > # set the search path for the environment coming with the compiler > > # and a directory where you can install your own compiled software > > set(CMAKE_FIND_ROOT_PATH > > /bgsys/drivers/ppcfloor/ > > /bgsys/drivers/ppcfloor/gnu-linux/powerpc-bgp-linux/ > > /bgsys/drivers/ppcfloor/runtime/SPI > > /bgsys/drivers/ppcfloor/comm/ > > /bgsys/drivers/ppcfloor/comm/default/ > > /bgsys/drivers/ppcfloor/comm/sys/ > > ) > > > > # adjust the default behaviour of the FIND_XXX() commands: > > # search headers and libraries in the target environment, search > > # programs in the host environment > > set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) > > set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) > > set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) > > There are three problems here. > > 3a. First, using this setup, FindMPI fails because the last library it > needs is in /bgsys/drivers/ppcfloor/runtime/SPI, not > /bgsys/drivers/ppcfloor/runtime/SPI/lib. CMAKE_FIND_ROOT_PATH seems to > assume that its elements are just above a lib directory, but that's not how > things are structured on BG/P (you can blame IBM). So, if I just use the > root path for searching, find_library fails for libs in runtime/SPI. I guess you need a special Platform/BlueGeneP.cmake file, which not simply includes UnixPaths.cmake, but sets some special directories. Please add a bug report for this on http://public.kitware.com/Bug/ too. > 3b. I tried passing the root path elements as locations in a HINTS to > find_library command instead. Well, the first problem with this is that if > you try this with CMAKE_FIND_ROOT_PATH_MODE_LIBRARY set to ONLY as advised, > it just ignores the HINTS parameters. I then tried BOTH, and when I do > that, find_library seems to find the libraries in > /bgsys/drivers/ppcfloor/runtime/SPI correctly. However, it fails to find > other libraries correctly because now it searches system directories on the > front end node in addition to the backend library paths I've given above. > The result is that other libraries (like librt) resolve to their frontend > versions when they should resolve to the backend versions. If I build this > way, I'll either get me a link error or a nasty runtime error. Maybe we can solve this with 3a) > 3c. The third problem I see here is that while this particular project is > self-contained, we have a lot of preinstalled libraries for the BlueGene > backend that don't live in one nice, self-contained place. The webpage for > cross compiling with CMake suggests that I have *one* directory where all > my binaries for the backend should go, and that I put this directory last > in my CMAKE_FIND_ROOT_PATH. > > This isn't realistic for any of the large supercomputer installations I > know of. LLNL, Argonne, and ORNL all have prebuilt packages for the > backends in a lot of different places, and I need a mechanism to pass those > locations to the find_library command. Can you give some examples where the things are installed ? Alex _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake