Re: [gmx-users] compiling on different architecture than the compute nodes architecture
On Thu, Feb 7, 2013 at 10:00 AM, Broadbent, Richard richard.broadben...@imperial.ac.uk wrote: Dear Silard, cmake -DGMX_CPU_ACCELERATION=AVX_256 -DGMX_DEFAULT_SUFFIX=OFF -DGMX_BINARY_SUFFIX=_avx -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/include -DMKL_LIBRARIES=$MKLROOT/lib/intel64/libmkl_core.so;$MKLROOT/lib/intel64/l ibmkl_intel_lp64.so;$MKLROOT/lib/intel64/libmkl_sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/libs/gromacs Compiled, linked and installed without warning using intel-suite/2013, mpi/intel-3.1, and cmake/2.8.9 I was compiling release-4-6 out of the git repository ( 06935659a3125d9b44b84c57e67cb6788fda42c1 ). The job using that executable is still in the queue, however, another one built using the sse2 kernels ran a minimisation on a single core without problems. How much faster fftw3 is than mkl for gromacs, is the difference likely to be on the scale of 1-2% or 10%? The former, I would expect. Naturally it depends on your hardware, how you compiled FFTW, and whether/how you use PME in GROMACS. If your PME-PP load balance has too much work for the PP nodes, then FFT performance doesn't matter. The .log file prints lots of analysis about this, and mdrun tries by default to do what it can to balance. Recent versions of icc and MKL do make compiling for MKL much easier, so we'll probably add that for 5.0 Mark Thanks, Richard On 07/02/2013 02:17, Szilárd Páll szilard.p...@cbr.su.se wrote: On Wed, Feb 6, 2013 at 6:03 PM, Richard Broadbent richard.broadben...@imperial.ac.uk wrote: Dear All, I would like to compile gromacs 4.6 to run with the correct acceleration on the compute nodes on our local cluster. Some of the nodes have intel sandy-bridge whilst others only have sse4.1 and some (including the login and single core job nodes) are still stuck on ssse3 (gmx would use sse2 acceleration here). Installing several versions is not a problem however, I'm not sure how to make cmake build a version of the code that is not using the acceleration for the system on which the code is being compiled. Restrictions on job sizes makes running the compilation on the sandy-bridge nodes almost impossible. Can anyone let me know which flags cmake needs to enable avx-256 acceleration? my standard cmake line is: $ CC=mpiicc CXX=mpiicpc ; cmake -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/**include -DMKL_LIBRARIES=$MKLROOT/lib/**intel64/libmkl_core.so;$** MKLROOT/lib/intel64/libmkl_**intel_lp64.so;$MKLROOT/lib/**intel64/libmkl_ sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/**libs/gromacs ../ Note that MKL (without the fftw wrappers) is known to not work out of the box. Making it work requires a fairly simple workaround described here: http://redmine.gromacs.org/issues/1110#note-3 Additionally, note that FFTW is in most cases faster than MKL (but if you find the contrary do let us know). -- Szilárd Thanks, Richard -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/**mailman/listinfo/gmx-users http://lists.gromac s.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/** Support/Mailing_Lists/Search http://www.gromacs.org/Support/Mailing_Lists /Searchbefore posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists http://www.gromacs.org/Sup port/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] compiling on different architecture than the compute nodes architecture
Dear Silard, cmake -DGMX_CPU_ACCELERATION=AVX_256 -DGMX_DEFAULT_SUFFIX=OFF -DGMX_BINARY_SUFFIX=_avx -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/include -DMKL_LIBRARIES=$MKLROOT/lib/intel64/libmkl_core.so;$MKLROOT/lib/intel64/l ibmkl_intel_lp64.so;$MKLROOT/lib/intel64/libmkl_sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/libs/gromacs Compiled, linked and installed without warning using intel-suite/2013, mpi/intel-3.1, and cmake/2.8.9 I was compiling release-4-6 out of the git repository ( 06935659a3125d9b44b84c57e67cb6788fda42c1 ). The job using that executable is still in the queue, however, another one built using the sse2 kernels ran a minimisation on a single core without problems. How much faster fftw3 is than mkl for gromacs, is the difference likely to be on the scale of 1-2% or 10%? Thanks, Richard On 07/02/2013 02:17, Szilárd Páll szilard.p...@cbr.su.se wrote: On Wed, Feb 6, 2013 at 6:03 PM, Richard Broadbent richard.broadben...@imperial.ac.uk wrote: Dear All, I would like to compile gromacs 4.6 to run with the correct acceleration on the compute nodes on our local cluster. Some of the nodes have intel sandy-bridge whilst others only have sse4.1 and some (including the login and single core job nodes) are still stuck on ssse3 (gmx would use sse2 acceleration here). Installing several versions is not a problem however, I'm not sure how to make cmake build a version of the code that is not using the acceleration for the system on which the code is being compiled. Restrictions on job sizes makes running the compilation on the sandy-bridge nodes almost impossible. Can anyone let me know which flags cmake needs to enable avx-256 acceleration? my standard cmake line is: $ CC=mpiicc CXX=mpiicpc ; cmake -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/**include -DMKL_LIBRARIES=$MKLROOT/lib/**intel64/libmkl_core.so;$** MKLROOT/lib/intel64/libmkl_**intel_lp64.so;$MKLROOT/lib/**intel64/libmkl_ sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/**libs/gromacs ../ Note that MKL (without the fftw wrappers) is known to not work out of the box. Making it work requires a fairly simple workaround described here: http://redmine.gromacs.org/issues/1110#note-3 Additionally, note that FFTW is in most cases faster than MKL (but if you find the contrary do let us know). -- Szilárd Thanks, Richard -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/**mailman/listinfo/gmx-usershttp://lists.gromac s.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/** Support/Mailing_Lists/Searchhttp://www.gromacs.org/Support/Mailing_Lists /Searchbefore posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/**Support/Mailing_Listshttp://www.gromacs.org/Sup port/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] compiling on different architecture than the compute nodes architecture
Dear All, I would like to compile gromacs 4.6 to run with the correct acceleration on the compute nodes on our local cluster. Some of the nodes have intel sandy-bridge whilst others only have sse4.1 and some (including the login and single core job nodes) are still stuck on ssse3 (gmx would use sse2 acceleration here). Installing several versions is not a problem however, I'm not sure how to make cmake build a version of the code that is not using the acceleration for the system on which the code is being compiled. Restrictions on job sizes makes running the compilation on the sandy-bridge nodes almost impossible. Can anyone let me know which flags cmake needs to enable avx-256 acceleration? my standard cmake line is: $ CC=mpiicc CXX=mpiicpc ; cmake -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/include -DMKL_LIBRARIES=$MKLROOT/lib/intel64/libmkl_core.so;$MKLROOT/lib/intel64/libmkl_intel_lp64.so;$MKLROOT/lib/intel64/libmkl_sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/libs/gromacs ../ Thanks, Richard -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] compiling on different architecture than the compute nodes architecture
Hi, On Feb 6, 2013, at 6:03 PM, Richard Broadbent richard.broadben...@imperial.ac.uk wrote: Dear All, I would like to compile gromacs 4.6 to run with the correct acceleration on the compute nodes on our local cluster. Some of the nodes have intel sandy-bridge whilst others only have sse4.1 and some (including the login and single core job nodes) are still stuck on ssse3 (gmx would use sse2 acceleration here). Installing several versions is not a problem however, I'm not sure how to make cmake build a version of the code that is not using the acceleration for the system on which the code is being compiled. Restrictions on job sizes makes running the compilation on the sandy-bridge nodes almost impossible. Can anyone let me know which flags cmake needs to enable avx-256 acceleration? Use -DGMX_CPU_ACCELERATION=AVX_256 Try ccmake to have a look at the variables you can define and what values they can be set to. Carsten my standard cmake line is: $ CC=mpiicc CXX=mpiicpc ; cmake -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/include -DMKL_LIBRARIES=$MKLROOT/lib/intel64/libmkl_core.so;$MKLROOT/lib/intel64/libmkl_intel_lp64.so;$MKLROOT/lib/intel64/libmkl_sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/libs/gromacs ../ Thanks, Richard -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] compiling on different architecture than the compute nodes architecture
On Wed, Feb 6, 2013 at 6:03 PM, Richard Broadbent richard.broadben...@imperial.ac.uk wrote: Dear All, I would like to compile gromacs 4.6 to run with the correct acceleration on the compute nodes on our local cluster. Some of the nodes have intel sandy-bridge whilst others only have sse4.1 and some (including the login and single core job nodes) are still stuck on ssse3 (gmx would use sse2 acceleration here). Installing several versions is not a problem however, I'm not sure how to make cmake build a version of the code that is not using the acceleration for the system on which the code is being compiled. Restrictions on job sizes makes running the compilation on the sandy-bridge nodes almost impossible. Can anyone let me know which flags cmake needs to enable avx-256 acceleration? my standard cmake line is: $ CC=mpiicc CXX=mpiicpc ; cmake -DGMX_OPENMP=ON -DGMX_MPI=ON -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/**include -DMKL_LIBRARIES=$MKLROOT/lib/**intel64/libmkl_core.so;$** MKLROOT/lib/intel64/libmkl_**intel_lp64.so;$MKLROOT/lib/**intel64/libmkl_sequential.so -DCMAKE_INSTALL_PREFIX=$HOME/**libs/gromacs ../ Note that MKL (without the fftw wrappers) is known to not work out of the box. Making it work requires a fairly simple workaround described here: http://redmine.gromacs.org/issues/1110#note-3 Additionally, note that FFTW is in most cases faster than MKL (but if you find the contrary do let us know). -- Szilárd Thanks, Richard -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/**mailman/listinfo/gmx-usershttp://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/** Support/Mailing_Lists/Searchhttp://www.gromacs.org/Support/Mailing_Lists/Searchbefore posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/**Support/Mailing_Listshttp://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
