[gmx-users] problem with icc compiler
Hi, i observed the following problem. if i simulate water (spc or tip4p) with gromacs 4.0.5 i get with v-rescale or berendsen thermostat the wrong temperature (ref_t = 300K -> average around 425K, in about 1-2ps), but only in serial, not in parallel runs. non-water molecules or nose-hoover thermostat make no problems. see also http://lists.gromacs.org/pipermail/gmx-users/2010-March/049248.html for mdp and log file. gromacs was compiled with the following comands: and in the file 'configure' all '-lmkl' were deleted (don't ask me why, i don't really understand that stuff, the command were from our previous phd student). ./configure CC="icc" CPPFLAGS="-I/share/apps/intel/mkl/10.0.011/include" LDFLAGS="-L/share/apps/intel/mkl/10.0.011/lib/em64t -lmkl_solver_lp64_sequential -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread" --with-fft=mkl --prefix="/share/apps/gromacs/4.0.5" make make install make clean ./configure CC="icc" CPPFLAGS="-I/share/apps/intel/mkl/10.0.011/include" LDFLAGS="-L/share/apps/intel/mkl/10.0.011/lib/em64t -lmkl_solver_lp64_sequential -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread" --with-fft=mkl --prefix="/share/apps/gromacs/4.0.5" --enable-mpi --disable-nice --program-suffix=_mpi make mdrun make install-mdrun for gromacs 4.0.5 i used the icc 9.1.046 compiler. i also tried gromacs 4.0.7 with icc 9.1.046 and icc 10.1.008 with spc water, v-rescale thermostat. -> serial: too high temperature 425K iso 300K -> parallel: no problems with non-water (mesitylene) i have no problem in serial. the problem does not come from grompp because i can use same tpr-file for serial and parallel runs with the above results. if someone needs more informations about this please tell me. greetings thomas -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/mailing_lists/users.php
Re: [gmx-users] problem with icc compiler
Gromacs was compiled on a xeon (woodcrest). for the simulations i used an old xeon (don't know what chip, but also 64bit system) and a i7. About static / dynamic libraries: I used there default settings. At the end of the configure command it tells me the following: * On most platforms you can save 10X space with dynamic libraries, although the binaries might be less portable. Why not try --enable-shared? So i think the libraries are static. I tried Mark's suggestion and compiled a new version, where i change '--with-fft=mkl' with '--enable-fft=fftpack' (the restr of the configure command was the same then before). With that, the error didn't appear. Does that mean that the linking to mkl did not work for mdrun, but worked for mdrun_mpi (because there the temperature was right)? One thing for the temperature jump: Temperature starts at around 300 K (from 'gen_temp') then goes in 1-2ps up to around 425 K and then stays there. the simulation was 100ps long, in the end i had an average value of about 425 K (from log file). Here are the first 20ps from g_energy 0.00 305.240509 1.00 381.614166 2.00 410.572906 3.00 434.954956 4.00 414.660400 5.00 394.799591 6.00 389.087128 7.00 414.893982 8.00 449.444183 9.00 417.877472 10.00 442.470306 11.01 446.170258 12.01 448.844666 13.01 412.847473 14.01 454.549744 15.01 447.908478 16.00 404.607422 17.00 404.629944 18.00 441.559448 19.00 396.328400 20.00 421.386017 and: Energy Average RMSD Fluct. Drift Tot-Drift Temperature 424.62521.764521.6696 0.07032017.03215 On 10/03/2010 12:21 AM, Thomas Schlesier wrote: Could anybody reproduce that error or has an idea what is happening? Or i am alone with that problem? Nothing looks obviously wrong, but it's hard to be sure in the absence of information about your hardware. The most likely issue is some kind of dynamically-linked library mismatch. This can happen if your execution environment differs from your linking environment. Try forcing linking to static versions of the libraries, which will prevent this. Also try disabling things until you get sensible behaviour in all cases, like --enable-fft=fftpack. That would reveal that the problem was with linking to mkl. Also 1-2ps is a bit too short to expect convergence of temperature - check the plot of T against t with g_energy. Mark Date: Fri, 5 Mar 2010 23:11:45 +0100 From: Thomas Schlesier Subject: [gmx-users] problem with icc compiler To: "gmx-users@gromacs.org" Message-ID: <4b9181a1.7060...@uni-mainz.de> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Hi, i observed the following problem. if i simulate water (spc or tip4p) with gromacs 4.0.5 i get with v-rescale or berendsen thermostat the wrong temperature (ref_t = 300K -> average around 425K, in about 1-2ps), but only in serial, not in parallel runs. non-water molecules or nose-hoover thermostat make no problems. see also http://lists.gromacs.org/pipermail/gmx-users/2010-March/049248.html for mdp and log file. gromacs was compiled with the following comands: and in the file 'configure' all '-lmkl' were deleted (don't ask me why, i don't really understand that stuff, the command were from our previous phd student). ./configure CC="icc" CPPFLAGS="-I/share/apps/intel/mkl/10.0.011/include" LDFLAGS="-L/share/apps/intel/mkl/10.0.011/lib/em64t -lmkl_solver_lp64_sequential -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread" --with-fft=mkl --prefix="/share/apps/gromacs/4.0.5" make make install make clean ./configure CC="icc" CPPFLAGS="-I/share/apps/intel/mkl/10.0.011/include" LDFLAGS="-L/share/apps/intel/mkl/10.0.011/lib/em64t -lmkl_solver_lp64_sequential -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread" --with-fft=mkl --prefix="/share/apps/gromacs/4.0.5" --enable-mpi --disable-nice --program-suffix=_mpi make mdrun make install-mdrun for gromacs 4.0.5 i used the icc 9.1.046 compiler. i also tried gromacs 4.0.7 with icc 9.1.046 and icc 10.1.008 with spc water, v-rescale thermostat. -> serial: too high temperature 425K iso 300K -> parallel: no problems with non-water (mesitylene) i have no problem in serial. the problem does not come from grompp because i can use same tpr-file for serial and parallel runs with the above results. if someone needs more informations about this please tell me. greetings thomas -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archiv
Re: [gmx-users] problem with icc compiler
On 11/03/2010 3:04 AM, Thomas Schlesier wrote: Gromacs was compiled on a xeon (woodcrest). for the simulations i used an old xeon (don't know what chip, but also 64bit system) and a i7. Well, don't do that. IMO, dynamic linking should work in that kind of scenario, but something is clearly broken. About static / dynamic libraries: I used there default settings. At the end of the configure command it tells me the following: * On most platforms you can save 10X space with dynamic libraries, although the binaries might be less portable. Why not try --enable-shared? So i think the libraries are static. Your internal GROMACS linking is static, which you could change with --enable-shared. You will need to kick the linker harder than that to force it to link statically to system libraries also. I tried Mark's suggestion and compiled a new version, where i change '--with-fft=mkl' with '--enable-fft=fftpack' (the restr of the configure command was the same then before). With that, the error didn't appear. Does that mean that the linking to mkl did not work for mdrun, but worked for mdrun_mpi (because there the temperature was right)? Yep, that is strong evidence for my hypothesis. Either compile GROMACS on the target execution system (even submit a cluster job for the compilation!), or (if the former doesn't work) get the MKL documentation and read it about how to enforce static linking. There may be some cunning linker option for that, or you may need to explicitly cite the static versions of the libraries. Or, get FFTW installed and link to that. I have found near-negligible performance differences between MKL and FFTW on my machine. One thing for the temperature jump: Temperature starts at around 300 K (from 'gen_temp') then goes in 1-2ps up to around 425 K and then stays there. the simulation was 100ps long, in the end i had an average value of about 425 K (from log file). Here are the first 20ps from g_energy 0.00 305.240509 1.00 381.614166 2.00 410.572906 3.00 434.954956 4.00 414.660400 5.00 394.799591 6.00 389.087128 7.00 414.893982 8.00 449.444183 9.00 417.877472 10.00 442.470306 11.01 446.170258 12.01 448.844666 13.01 412.847473 14.01 454.549744 15.01 447.908478 16.00 404.607422 17.00 404.629944 18.00 441.559448 19.00 396.328400 20.00 421.386017 That's broken all right! Mark and: Energy Average RMSD Fluct. Drift Tot-Drift Temperature 424.625 21.7645 21.6696 0.0703201 7.03215 On 10/03/2010 12:21 AM, Thomas Schlesier wrote: Could anybody reproduce that error or has an idea what is happening? Or i am alone with that problem? Nothing looks obviously wrong, but it's hard to be sure in the absence of information about your hardware. The most likely issue is some kind of dynamically-linked library mismatch. This can happen if your execution environment differs from your linking environment. Try forcing linking to static versions of the libraries, which will prevent this. Also try disabling things until you get sensible behaviour in all cases, like --enable-fft=fftpack. That would reveal that the problem was with linking to mkl. Also 1-2ps is a bit too short to expect convergence of temperature - check the plot of T against t with g_energy. Mark Date: Fri, 5 Mar 2010 23:11:45 +0100 From: Thomas Schlesier Subject: [gmx-users] problem with icc compiler To: "gmx-users@gromacs.org" Message-ID: <4b9181a1.7060...@uni-mainz.de> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Hi, i observed the following problem. if i simulate water (spc or tip4p) with gromacs 4.0.5 i get with v-rescale or berendsen thermostat the wrong temperature (ref_t = 300K -> average around 425K, in about 1-2ps), but only in serial, not in parallel runs. non-water molecules or nose-hoover thermostat make no problems. see also http://lists.gromacs.org/pipermail/gmx-users/2010-March/049248.html for mdp and log file. gromacs was compiled with the following comands: and in the file 'configure' all '-lmkl' were deleted (don't ask me why, i don't really understand that stuff, the command were from our previous phd student). ./configure CC="icc" CPPFLAGS="-I/share/apps/intel/mkl/10.0.011/include" LDFLAGS="-L/share/apps/intel/mkl/10.0.011/lib/em64t -lmkl_solver_lp64_sequential -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread" --with-fft=mkl --prefix="/share/apps/gromacs/4.0.5" make make install make clean ./configure CC="icc" CPPFLAGS="-I/share/apps/intel/mkl/10.0.011/include" LDFLAGS="-L/share/apps/intel/mkl/10.0.011/lib/em64t -lmkl_solver_lp64_sequential -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl
Re: [gmx-users] problem with icc compiler
Hi, I have reported this problem to Bugzilla some months ago: http://bugzilla.gromacs.org/show_bug.cgi?id=378 I don't exactly know what the problem is, but it is definitely related to the MKL FFTs, which can be seen by switching from PME to cutoff, and the problem should go away. I have reproduced this behavior on several computers, as I have written in the bugzilla report. / Erik Brandt tor 2010-03-11 klockan 01:34 +0100 skrev Mark Abraham: > On 11/03/2010 3:04 AM, Thomas Schlesier wrote: > > Gromacs was compiled on a xeon (woodcrest). for the simulations i used > > an old xeon (don't know what chip, but also 64bit system) and a i7. > > Well, don't do that. IMO, dynamic linking should work in that kind of > scenario, but something is clearly broken. > > > About static / dynamic libraries: > > I used there default settings. At the end of the configure command it > > tells me the following: > > * On most platforms you can save 10X space with dynamic libraries, > > although the binaries might be less portable. Why not try --enable-shared? > > So i think the libraries are static. > > Your internal GROMACS linking is static, which you could change with > --enable-shared. You will need to kick the linker harder than that to > force it to link statically to system libraries also. > > > I tried Mark's suggestion and compiled a new version, where i change > > '--with-fft=mkl' with '--enable-fft=fftpack' (the restr of the configure > > command was the same then before). With that, the error didn't appear. > > Does that mean that the linking to mkl did not work for mdrun, but > > worked for mdrun_mpi (because there the temperature was right)? > > Yep, that is strong evidence for my hypothesis. Either compile GROMACS > on the target execution system (even submit a cluster job for the > compilation!), or (if the former doesn't work) get the MKL documentation > and read it about how to enforce static linking. There may be some > cunning linker option for that, or you may need to explicitly cite the > static versions of the libraries. Or, get FFTW installed and link to > that. I have found near-negligible performance differences between MKL > and FFTW on my machine. > > > One thing for the temperature jump: > > Temperature starts at around 300 K (from 'gen_temp') then goes in 1-2ps > > up to around 425 K and then stays there. the simulation was 100ps long, > > in the end i had an average value of about 425 K (from log file). > > > > Here are the first 20ps from g_energy > > 0.00 305.240509 > > 1.00 381.614166 > > 2.00 410.572906 > > 3.00 434.954956 > > 4.00 414.660400 > > 5.00 394.799591 > > 6.00 389.087128 > > 7.00 414.893982 > > 8.00 449.444183 > > 9.00 417.877472 > > 10.00 442.470306 > > 11.01 446.170258 > > 12.01 448.844666 > > 13.01 412.847473 > > 14.01 454.549744 > > 15.01 447.908478 > > 16.00 404.607422 > > 17.00 404.629944 > > 18.00 441.559448 > > 19.00 396.328400 > > 20.00 421.386017 > > That's broken all right! > > Mark > > > and: > > Energy Average RMSD Fluct. Drift Tot-Drift > > > > Temperature 424.625 21.7645 21.6696 0.0703201 7.03215 > > > > > > > >> > >> On 10/03/2010 12:21 AM, Thomas Schlesier wrote: > >>> Could anybody reproduce that error or has an idea what is happening? > >>> Or i am alone with that problem? > >> > >> Nothing looks obviously wrong, but it's hard to be sure in the absence > >> of information about your hardware. The most likely issue is some kind > >> of dynamically-linked library mismatch. This can happen if your > >> execution environment differs from your linking environment. Try forcing > >> linking to static versions of the libraries, which will prevent this. > >> > >> Also try disabling things until you get sensible behaviour in all cases, > >> like --enable-fft=fftpack. That would reveal that the problem was with > >> linking to mkl. > >> > >> Also 1-2ps is a bit too short to expect convergence of temperature - > >> check the plot of T against t with g_energy. > >> > >> Mark > >> > >>>> Date: Fri, 5 Mar 2010 23:11:45 +0100 > >>>> From: Thomas Schlesier > >>>> Subject: [gmx-users] problem with icc compiler > >>>> To: "gmx-u
