转发:[gmx-users] gromacs on GPU
Dear gromcas users, Can anyone tell me how to make a mdp file for gromacs mdrun on NVIDIA GPU card, the following is a mdp file which runs well on cpu, but when I add the option -testverlet to mdrun in order to run it on GPU, it returns error Fatal error:User non-bonded potentials are not (yet) supported with the Verlet schemeFor more information and tips for troubleshooting. Could you check the mdp file for me to see how to modefy it to make it suitable to GPU? Thanks in advance! integrator = mdnsteps = 5000dt = 0.002constraints = all-bondsnstcomm = 100ns_type = gridrlist = 1.2rcoulomb = 1.0 rcoulomb-switch = 0.9 rvdw =1.0vdwtype = shiftrvdw-switch = 0.9coulombtype = PME-Switch;Tcoupl = v-rescaletau_t = 0.1 0.1tc-grps = protein non-proteinref_t = 300 300Pcoupl = parrinello-rahmanPcoupltype = isotropictau_p = 0.5compressibility = 4.5e-5ref_p = 1.0gen_vel = nogen_temp = 310gen_seed = -1nstxout = 500nstenergy = 100energygrps = chA chClincs-iter = 2DispCorr = EnerPresoptimize_fft = yes Yeping Sun -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: 转发:[gmx-users] gromacs on GPU
On Sun, Jun 23, 2013 at 3:30 PM, sunyep...@aliyun.com wrote: Dear gromcas users, Can anyone tell me how to make a mdp file for gromacs mdrun on NVIDIA GPU card, the following is a mdp file which runs well on cpu, but when I add the option -testverlet to mdrun in order to run it on GPU, it returns error Fatal error:User non-bonded potentials are not (yet) supported with the Verlet schemeFor more information and tips for troubleshooting. Thanks, I can now use a copy-paste to see that this message should not be generated by your .mdp file below. I have filed a bug report http://redmine.gromacs.org/issues/1289. In the meantime, I suggest you use normal PME. FYI, there is an ongoing debate elsewhere on Redmine that PME-Switch might be a bad idea anyway... Next time you post an .mdp file please make the line wrapping readable. Mark Could you check the mdp file for me to see how to modefy it to make it suitable to GPU? Thanks in advance! integrator = mdnsteps = 5000dt = 0.002constraints = all-bondsnstcomm = 100ns_type = gridrlist = 1.2rcoulomb = 1.0 rcoulomb-switch = 0.9 rvdw =1.0vdwtype = shiftrvdw-switch = 0.9coulombtype = PME-Switch;Tcoupl = v-rescaletau_t = 0.1 0.1tc-grps = protein non-proteinref_t = 300 300Pcoupl = parrinello-rahmanPcoupltype = isotropictau_p = 0.5compressibility = 4.5e-5ref_p = 1.0gen_vel = nogen_temp = 310gen_seed = -1nstxout = 500nstenergy = 100energygrps = chA chClincs-iter = 2DispCorr = EnerPresoptimize_fft = yes Yeping Sun -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] GROMACS on GPU
Dear gromacs users, Can anyone tell me how to make a mdp file for gromacs mdrun on NVIDIA GPU card, the following is a mdp file which runs well on cpu, but when I add the option -testverlet to mdrun in order to run it on GPU, it returns error nonbond potiential is not supported. Could you check the mdp file for me to see how to modefy it to make it suitable to GPU? Thanks in advance! Yeping Sun -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] gromacs on GPU
Can anyone tell me how to make a mdp file for gromacs mdrun on NVIDIA GPU card, the following is a mdp file which runs well on cpu, but when I add the option -testverlet to mdrun in order to run it on GPU, it returns error nonbond potiential is not supported. Could you check the mdp file for me to see how to modefy it to make it suitable to GPU? Thanks in advance! integrator = mdnsteps = 5000dt = 0.002constraints = all-bondsnstcomm = 100ns_type = gridrlist = 1.2rcoulomb = 1.0 rcoulomb-switch = 0.9 rvdw =1.0vdwtype = shiftrvdw-switch = 0.9coulombtype = PME-SwitchTcoupl = v-rescaletau_t = 0.1 0.1tc-grps = protein non-proteinref_t = 300 300Pcoupl = parrinello-rahmanPcoupltype = isotropictau_p = 0.5compressibility = 4.5e-5ref_p = 1.0gen_vel = nogen_temp = 310gen_seed = -1nstxout = 1nstenergy = 5000energygrps = chA chClincs-iter = 2DispCorr = EnerPresoptimize_fft = yes Yeping Sun -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On Fri, Jun 21, 2013 at 8:33 AM, sunyep...@aliyun.com wrote: Can anyone tell me how to make a mdp file for gromacs mdrun on NVIDIA GPU card, the following is a mdp file which runs well on cpu, but when I add the option -testverlet to mdrun in order to run it on GPU, it returns error nonbond potiential is not supported. Please copy and paste actual error messages, not what you think was important about it. -testverlet is not an all-purpose magic bullet, and there are algorithm combinations that are implemented differently in the group and verlet schemes. See manual 7.3.10-11 Mark Could you check the mdp file for me to see how to modefy it to make it suitable to GPU? Thanks in advance! integrator = mdnsteps = 5000dt = 0.002constraints = all-bondsnstcomm = 100ns_type = gridrlist = 1.2rcoulomb = 1.0 rcoulomb-switch = 0.9 rvdw =1.0vdwtype = shiftrvdw-switch = 0.9coulombtype = PME-SwitchTcoupl = v-rescaletau_t = 0.1 0.1tc-grps = protein non-proteinref_t = 300 300Pcoupl = parrinello-rahmanPcoupltype = isotropictau_p = 0.5compressibility = 4.5e-5ref_p = 1.0gen_vel = nogen_temp = 310gen_seed = -1nstxout = 1nstenergy = 5000energygrps = chA chClincs-iter = 2DispCorr = EnerPresoptimize_fft = yes Yeping Sun -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] Gromacs 4.6 GPU on Nvidia 250 GTS?
Dear Gromacs users, Is it possible to use GROMACS 4.6 GPU Mode on Nvidia 250 GTS? Thanks in advance, Hovakim -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs 4.6 GPU on Nvidia 250 GTS?
On Sun, Apr 7, 2013 at 6:39 AM, Hovakim Grabski hovakim_grab...@yahoo.comwrote: Dear Gromacs users, Is it possible to use GROMACS 4.6 GPU Mode on Nvidia 250 GTS? No, the minimum compute capability is 2.0. Your card has 1.1. https://developer.nvidia.com/cuda-gpus -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On Thu, Jan 10, 2013 at 7:25 AM, James Starlight jmsstarli...@gmail.comwrote: Szilárd , thanks again for explanation! Today I've performed some tests on my calmodulin in water system with different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8 respectually) Below you can see that the highest performance was in case of 0.8 cut-offs all cut-offs 1.0 Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305 NOTE: The GPU has 20% more load than the CPU. This imbalance causes performance loss, consider using a shorter cut-off and a finer PME grid. Core t (s) Wall t (s)(%) Time: 1313.420 464.035 283.0 (ns/day)(hour/ns) Performance:9.3102.578 Finished mdrun on node 0 Thu Jan 10 09:39:23 2013 all cut-offs 0.9 Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059 Core t (s) Wall t (s)(%) Time: 2414.930 856.179 282.1 (ns/day)(hour/ns) Performance: 10.0922.378 Finished mdrun on node 0 Thu Jan 10 10:09:52 2013 all cut-offs 0.8 Force evaluation time GPU/CPU: 4.001 ms/4.659 ms = 0.859 Core t (s) Wall t (s)(%) Time: 1166.390 413.598 282.0 (ns/day)(hour/ns) Performance: 10.4452.298 Finished mdrun on node 0 Thu Jan 10 09:50:33 2013 Also I've noticed that 2-4 CPU cores usage in 2 and 3rd case was only 67%. Is there any other ways to increase performance by means of neighboor search parameters ( e.g nstlist etc) ? You can tweak nstlist and it often helps to increase it with GPUs, especially in parallel. However, as increasing nstlist requires larger rlist and more non-bonded calculations, this will not help you. You can try to decrease it to 10-15 which will increase the NS cost but decrease the GPU time, but it won't change the performance dramatically. What's strange is that your Core time/Wall time = (%) is quite low. If you're running on four threads on an otherwise empty machine, you should get close to 400 if the threads are not idling, e.g waiting for the GPU. For instance in the rc=0.8 case you can see that the GPU/CPU balance is 1.0 meaning that the GPU has less work than the CPU, case in which there should be no idling and you should be getting (%) = 400. Long story short: are you sure you're not running anything else on the computer while simulating? What do you get if you run on CPU only? Might such reduced cut-off be used with the force fields ( e,g charmm) where initially usage of longest cut-offs have given better results (e,g in charmm27 and gromos56 I always use 1.2 and 1.4 nm for rvdw, respectually) ? No, at least not without *carefully* checking whether a shorter LJ cut-off makes sense and that it does not break the physics of your simulation. Although we advise you to consider decreasing your cut-off - mostly because these days a large number of simulations are carried out with overly long cut-off chosen by the rule of thumb or folclore -, you should always either make sure that this makes sense before doing it or not do it at all. Cheers, -- Szilárd James 2013/1/10 Szilárd Páll szilard.p...@cbr.su.se: Hi James, The build looks mostly fine except that you are using fftw3 compiled with AVX which is slower than with only SSE (even on AVX-capable CPUs) - you should have been warned about this at configure-time. Now, performance-wise everything looks fine except that with a 1.2 nm cut-off your GPU is not able to keep up with the CPU and finish the non-bonded work before the CPU is done with Bonded + PME. That's why you see the Wait GPU taking 20% of the total time and that's also why you see some cores idling (because for 20% of the run-time thread 0 on core 0 is blocked waiting for the GPU while the rest idle). As the suggestion at the end of the log file point out, you can consider using a shorter cut-off which will push more work back to the PME on the CPU, but whether you can do this it depends on your very problem. There is one more alternative of running two MPI processes on the GPU (mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will execute part of the nonbonded on the CPU, but this might not help. Cheers, -- Szilárd On Wed, Jan 9, 2013 at 8:27 PM, James Starlight jmsstarli...@gmail.com wrote: Dear Szilárd, thanks for help again! 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: There could be, but I/we can't well without more information on what and how you compiled and ran. The minimum we need is a log file. I've compilated gromacs 4.6-3 beta via simple cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 make sudo make install I have not added any special params to the grompp or mdrun. After that I've run tested simulation of the
Re: [gmx-users] gromacs on GPU
Szilárd, There are no any others cpu-usage tasks. Below you can see log from the TOP. 26553 own 20 0 28.4g 106m 33m S 285.6 0.7 2263:57 mdrun 1611 root 20 0 171m 65m 24m S 3.0 0.4 7:43.05 Xorg 29647 own 20 0 381m 22m 17m S 3.0 0.1 0:01.77 mate-system-mon 2344 own 20 0 358m 17m 11m S 1.3 0.1 0:33.76 mate-terminal 29018 root 20 0 000 S 0.3 0.0 0:04.99 kworker/0:0 29268 root 20 0 000 S 0.3 0.0 0:00.22 kworker/u:2 29705 root 20 0 000 S 0.3 0.0 0:00.03 kworker/3:0 29706 own 20 0 23284 1648 1188 R 0.3 0.0 0:00.05 top 1 root 20 0 8584 872 736 S 0.0 0.0 0:02.34 init 2 root 20 0 000 S 0.0 0.0 0:00.02 kthreadd 3 root 20 0 000 S 0.0 0.0 0:00.57 ksoftirqd/0 6 root rt 0 000 S 0.0 0.0 0:00.00 migration/0 7 root rt 0 000 S 0.0 0.0 0:00.17 watchdog/0 8 root rt 0 000 S 0.0 0.0 0:00.00 migration/1 10 root 20 0 000 S 0.0 0.0 0:00.43 ksoftirqd/1 12 root rt 0 000 S 0.0 0.0 0:00.17 watchdog/1 13 root rt 0 000 S 0.0 0.0 0:00.00 migration/2 15 root 20 0 000 S 0.0 0.0 0:00.37 ksoftirqd/2 16 root rt 0 000 S 0.0 0.0 0:00.16 watchdog/2 17 root rt 0 000 S 0.0 0.0 0:00.00 migration/3 19 root 20 0 000 S 0.0 0.0 0:00.38 ksoftirqd/3 20 root rt 0 000 S 0.0 0.0 0:00.16 watchdog/3 21 root 0 -20 000 S 0.0 0.0 0:00.00 cpuset 22 root 0 -20 000 S 0.0 0.0 0:00.00 khelper 23 root 20 0 000 S 0.0 0.0 0:00.00 kdevtmpfs Usually I run my simulations by means of simple mdrun -v -deffnm md. Should I specify number of cores manually by means of -nt (or -ntmpi) flagg? Also I notice that -pinht flagg could give me Hyper-Threading support. Does it reasonable in the simulation on cpu+gpu ? What another possible options of md_run should I consider ? Finally is it possible that problems due to openMP (4.7.2) or open-mpi (1.4.5) drivers ? Thanks for help James 2013/1/10 Szilárd Páll szilard.p...@cbr.su.se: On Thu, Jan 10, 2013 at 7:25 AM, James Starlight jmsstarli...@gmail.comwrote: Szilárd , thanks again for explanation! Today I've performed some tests on my calmodulin in water system with different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8 respectually) Below you can see that the highest performance was in case of 0.8 cut-offs all cut-offs 1.0 Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305 NOTE: The GPU has 20% more load than the CPU. This imbalance causes performance loss, consider using a shorter cut-off and a finer PME grid. Core t (s) Wall t (s)(%) Time: 1313.420 464.035 283.0 (ns/day)(hour/ns) Performance:9.3102.578 Finished mdrun on node 0 Thu Jan 10 09:39:23 2013 all cut-offs 0.9 Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059 Core t (s) Wall t (s)(%) Time: 2414.930 856.179 282.1 (ns/day)(hour/ns) Performance: 10.0922.378 Finished mdrun on node 0 Thu Jan 10 10:09:52 2013 all cut-offs 0.8 Force evaluation time GPU/CPU: 4.001 ms/4.659 ms = 0.859 Core t (s) Wall t (s)(%) Time: 1166.390 413.598 282.0 (ns/day)(hour/ns) Performance: 10.4452.298 Finished mdrun on node 0 Thu Jan 10 09:50:33 2013 Also I've noticed that 2-4 CPU cores usage in 2 and 3rd case was only 67%. Is there any other ways to increase performance by means of neighboor search parameters ( e.g nstlist etc) ? You can tweak nstlist and it often helps to increase it with GPUs, especially in parallel. However, as increasing nstlist requires larger rlist and more non-bonded calculations, this will not help you. You can try to decrease it to 10-15 which will increase the NS cost but decrease the GPU time, but it won't change the performance dramatically. What's strange is that your Core time/Wall time = (%) is quite low. If you're running on four threads on an otherwise empty machine, you should get close to 400 if the threads are not idling, e.g waiting for the GPU. For instance in the rc=0.8 case you can see that the GPU/CPU balance is 1.0 meaning that the GPU has less work than the CPU, case in which there should be no idling and you should be getting (%) = 400. Long story short: are you sure you're not running anything else on the computer while simulating? What do you get if you run on CPU only? Might such reduced cut-off be used with the force
Re: [gmx-users] gromacs on GPU
Hi, On Thu, Jan 10, 2013 at 8:30 PM, James Starlight jmsstarli...@gmail.comwrote: Szilárd, There are no any others cpu-usage tasks. Below you can see log from the TOP. 26553 own 20 0 28.4g 106m 33m S 285.6 0.7 2263:57 mdrun This still shows that the average CPU utilization is only 285.6 iso 400 and that matches with what mdrun's log shows. Try to run with a very short cut-off, one which leads to =1 GPU/CPU balance (i.e no waiting) and if you still don't get 400, something weird is going on. 1611 root 20 0 171m 65m 24m S 3.0 0.4 7:43.05 Xorg 29647 own 20 0 381m 22m 17m S 3.0 0.1 0:01.77 mate-system-mon 2344 own 20 0 358m 17m 11m S 1.3 0.1 0:33.76 mate-terminal 29018 root 20 0 000 S 0.3 0.0 0:04.99 kworker/0:0 29268 root 20 0 000 S 0.3 0.0 0:00.22 kworker/u:2 29705 root 20 0 000 S 0.3 0.0 0:00.03 kworker/3:0 29706 own 20 0 23284 1648 1188 R 0.3 0.0 0:00.05 top 1 root 20 0 8584 872 736 S 0.0 0.0 0:02.34 init 2 root 20 0 000 S 0.0 0.0 0:00.02 kthreadd 3 root 20 0 000 S 0.0 0.0 0:00.57 ksoftirqd/0 6 root rt 0 000 S 0.0 0.0 0:00.00 migration/0 7 root rt 0 000 S 0.0 0.0 0:00.17 watchdog/0 8 root rt 0 000 S 0.0 0.0 0:00.00 migration/1 10 root 20 0 000 S 0.0 0.0 0:00.43 ksoftirqd/1 12 root rt 0 000 S 0.0 0.0 0:00.17 watchdog/1 13 root rt 0 000 S 0.0 0.0 0:00.00 migration/2 15 root 20 0 000 S 0.0 0.0 0:00.37 ksoftirqd/2 16 root rt 0 000 S 0.0 0.0 0:00.16 watchdog/2 17 root rt 0 000 S 0.0 0.0 0:00.00 migration/3 19 root 20 0 000 S 0.0 0.0 0:00.38 ksoftirqd/3 20 root rt 0 000 S 0.0 0.0 0:00.16 watchdog/3 21 root 0 -20 000 S 0.0 0.0 0:00.00 cpuset 22 root 0 -20 000 S 0.0 0.0 0:00.00 khelper 23 root 20 0 000 S 0.0 0.0 0:00.00 kdevtmpfs Usually I run my simulations by means of simple mdrun -v -deffnm md. Should I specify number of cores manually by means of -nt (or -ntmpi) If you just want to run on the full machine, simply running like that should in most cases still be the optimal run configuration or very close to the optimal, i.e. in your case: mdrun = mdrun -ntmpi 1 -ntomp 4 -gpu_id 0 -pinht flagg? Also I notice that -pinht flagg could give me Hyper-Threading support. Does it reasonable in the simulation on cpu+gpu ? What Correctly using HT is also fully automatic and optimal as long as you are using the full machine. another possible options of md_run should I consider ? Finally is it possible that problems due to openMP (4.7.2) or open-mpi (1.4.5) drivers ? No, you are using the latest version of compilers which is good. Other than my earlier suggestions, there isn't much you can do to eliminate the idling on the CPU (I assume that's what bugs you) - except getting a faster GPU. Btw, have you tried the hybrid GPU-CPU mode (although I expect it to not be faster)? Cheers, -- Szilárd Thanks for help James 2013/1/10 Szilárd Páll szilard.p...@cbr.su.se: On Thu, Jan 10, 2013 at 7:25 AM, James Starlight jmsstarli...@gmail.com wrote: Szilárd , thanks again for explanation! Today I've performed some tests on my calmodulin in water system with different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8 respectually) Below you can see that the highest performance was in case of 0.8 cut-offs all cut-offs 1.0 Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305 NOTE: The GPU has 20% more load than the CPU. This imbalance causes performance loss, consider using a shorter cut-off and a finer PME grid. Core t (s) Wall t (s)(%) Time: 1313.420 464.035 283.0 (ns/day)(hour/ns) Performance:9.3102.578 Finished mdrun on node 0 Thu Jan 10 09:39:23 2013 all cut-offs 0.9 Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059 Core t (s) Wall t (s)(%) Time: 2414.930 856.179 282.1 (ns/day)(hour/ns) Performance: 10.0922.378 Finished mdrun on node 0 Thu Jan 10 10:09:52 2013 all cut-offs 0.8 Force evaluation time GPU/CPU: 4.001 ms/4.659 ms = 0.859 Core t (s) Wall t (s)(%) Time: 1166.390 413.598 282.0 (ns/day)(hour/ns) Performance: 10.4452.298 Finished mdrun on node 0 Thu Jan 10 09:50:33 2013 Also I've noticed that 2-4 CPU cores usage in
Re: [gmx-users] gromacs on GPU
Szilárd, the regime with 4 cores + cut-offs 0.8 has been best still. PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 1652 own 20 0 28.4g 135m 33m R 288.8 0.8 4:30.33 mdrun Force evaluation time GPU/CPU: 5.257 ms/5.187 ms = 1.013 For optimal performance this ratio should be close to 1! Core t (s) Wall t (s)(%) Time: 494.240 171.719 287.8 (ns/day)(hour/ns) Performance: 10.0642.385 Finished mdrun on node 0 Fri Jan 11 09:38:38 2013 I've tried to use compination of the different core numbers but results was the same ( below example with the 2 cores +gpu) PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 1578 own 20 0 28.3g 163m 33m R 170.7 1.0 1:50.68 mdrun Also cut-offs lower that 0.8 produced the same results. When I've used cut-off 0.1 the simulation have been crushed :) Finally increassing of nstlist up to 50 also gave slightly better results ( cpu usage up to 295) but I'm not sure about influence os such large cutoofs on other aspects of simulation. On that tests I'm using cpu CPU Intel Core i5-3570 3.4 ГГц / 4core / SVGA HD Graphics 2500 / 1+6Мб / as well as GPU GeForce GTX 670. Also I want to point out that all simulation have been run from debian GE GNOME desktope. Should I run simulation from console mode only to kill all hidden cpu-dependent processes? By the way I wounder to know is it possible to use 2 gpu at the same time (in the SLI mode) ? How It might increase overasl performance ? In future I'd like to built new work-station on 8 cores i7 CPU + 2 GPU. What the performance of such work-station will be? ( in comparison to the typical cluster from several nodes of 8-12 cpu ) ? Thanks for suggestions, James 2013/1/11 Szilárd Páll szilard.p...@cbr.su.se: Hi, On Thu, Jan 10, 2013 at 8:30 PM, James Starlight jmsstarli...@gmail.comwrote: Szilárd, There are no any others cpu-usage tasks. Below you can see log from the TOP. 26553 own 20 0 28.4g 106m 33m S 285.6 0.7 2263:57 mdrun This still shows that the average CPU utilization is only 285.6 iso 400 and that matches with what mdrun's log shows. Try to run with a very short cut-off, one which leads to =1 GPU/CPU balance (i.e no waiting) and if you still don't get 400, something weird is going on. 1611 root 20 0 171m 65m 24m S 3.0 0.4 7:43.05 Xorg 29647 own 20 0 381m 22m 17m S 3.0 0.1 0:01.77 mate-system-mon 2344 own 20 0 358m 17m 11m S 1.3 0.1 0:33.76 mate-terminal 29018 root 20 0 000 S 0.3 0.0 0:04.99 kworker/0:0 29268 root 20 0 000 S 0.3 0.0 0:00.22 kworker/u:2 29705 root 20 0 000 S 0.3 0.0 0:00.03 kworker/3:0 29706 own 20 0 23284 1648 1188 R 0.3 0.0 0:00.05 top 1 root 20 0 8584 872 736 S 0.0 0.0 0:02.34 init 2 root 20 0 000 S 0.0 0.0 0:00.02 kthreadd 3 root 20 0 000 S 0.0 0.0 0:00.57 ksoftirqd/0 6 root rt 0 000 S 0.0 0.0 0:00.00 migration/0 7 root rt 0 000 S 0.0 0.0 0:00.17 watchdog/0 8 root rt 0 000 S 0.0 0.0 0:00.00 migration/1 10 root 20 0 000 S 0.0 0.0 0:00.43 ksoftirqd/1 12 root rt 0 000 S 0.0 0.0 0:00.17 watchdog/1 13 root rt 0 000 S 0.0 0.0 0:00.00 migration/2 15 root 20 0 000 S 0.0 0.0 0:00.37 ksoftirqd/2 16 root rt 0 000 S 0.0 0.0 0:00.16 watchdog/2 17 root rt 0 000 S 0.0 0.0 0:00.00 migration/3 19 root 20 0 000 S 0.0 0.0 0:00.38 ksoftirqd/3 20 root rt 0 000 S 0.0 0.0 0:00.16 watchdog/3 21 root 0 -20 000 S 0.0 0.0 0:00.00 cpuset 22 root 0 -20 000 S 0.0 0.0 0:00.00 khelper 23 root 20 0 000 S 0.0 0.0 0:00.00 kdevtmpfs Usually I run my simulations by means of simple mdrun -v -deffnm md. Should I specify number of cores manually by means of -nt (or -ntmpi) If you just want to run on the full machine, simply running like that should in most cases still be the optimal run configuration or very close to the optimal, i.e. in your case: mdrun = mdrun -ntmpi 1 -ntomp 4 -gpu_id 0 -pinht flagg? Also I notice that -pinht flagg could give me Hyper-Threading support. Does it reasonable in the simulation on cpu+gpu ? What Correctly using HT is also fully automatic and optimal as long as you are using the full machine. another possible options of md_run should I consider ? Finally is it possible that problems due to openMP (4.7.2) or open-mpi (1.4.5) drivers ? No, you are using the latest version of compilers which is good. Other than
Re: [gmx-users] gromacs on GPU
As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). I've seen that erorr in other posts in the mailing list but could not found any possible sollutions. by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.comwrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
I've solved previous problem but havent been able to launch mdrun-openmm. Below you can find mdrun's output Back Off! I just backed up md_test.log to ./#md_test.log.3# Reading file md_test.tpr, VERSION 4.6-beta3 (single precision) Changing nstlist from 10 to 40, rlist from 1 to 1.146 Compiled acceleration: None (Gromacs could use AVX_256 on this machine, which is better) 1 GPU detected: #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC: no, stat: compatible 1 GPU auto-selected to be used for this run: #0 Program mdrun-openmm, VERSION 4.6-beta3 Source code file: /home/own/gromacs-4.6-beta3/src/kernel/openmm_wrapper.cpp, line: 1367 Fatal error: OpenMM exception caught while initializating: Error setting device flags cannot set while device is active in this process For more information and tips for troubleshooting, please check the GROMACS website at http://www.gromacs.org/Documentation/Errors I'm still using cuda-5-toolkit with the 304.54 driver. Also my question about openCL is still valid. I've browesed manual but didnt found how I can compile gromacs with openCL instead of Cuda. On openMM group developers told me that openCL would give much performance for gpu-accelerated simulation in comparison to the CUDA. James 2013/1/9 James Starlight jmsstarli...@gmail.com: As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). I've seen that erorr in other posts in the mailing list but could not found any possible sollutions. by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.comwrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On Wed, Jan 9, 2013 at 3:17 AM, James Starlight jmsstarli...@gmail.comwrote: As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? No the new build in GPU implementation and openMM are two different things. The Gromacs-OpenMM interface isn't actively maintained and thus not recommended. also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine. by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. GMX_GPU doesn't support openCL. Roland James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com wrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- ORNL/UT Center for Molecular Biophysics cmb.ornl.gov 865-241-1537, ORNL PO BOX 2008 MS6309 -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Roland, indeed the error was that I'have compilate mdrun-openmm which is not the native gpu. now I've made mdrun via cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 and obtain workable gromacs. My test system consist of calmodulin (charmm27) solvated in tip3p water. total size of the system: 67752 atoms. I have set below values on the mdp file as grompp told me nstlist = 20 cutoff-scheme = Verlet ; !not quite sure whai is this! vdwtype = cut-off Now on the mdrun output I have Using 1 MPI thread Using 4 OpenMP threads 1 GPU detected: #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC: no, stat: compatible 1 GPU auto-selected to be used for this run: #0 does it mean that all 4 cores of my CPU + gpu are used at same time ? is there any other ways to increase performance ? ( e.g I'm not quite sure if open_mpi is used with that build because during compilation cmake asked me only about fftw3 libs). Is there any ways to monitor total performance ( e.g separate cpu and gpu usage ) ? Thanks for suggestions, James 2013/1/9 Roland Schulz rol...@utk.edu: On Wed, Jan 9, 2013 at 3:17 AM, James Starlight jmsstarli...@gmail.comwrote: As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? No the new build in GPU implementation and openMM are two different things. The Gromacs-OpenMM interface isn't actively maintained and thus not recommended. also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine. by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. GMX_GPU doesn't support openCL. Roland James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com wrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- ORNL/UT Center for Molecular Biophysics cmb.ornl.gov 865-241-1537, ORNL PO BOX 2008 MS6309 -- gmx-users mailing listgmx-users@gromacs.org
Re: [gmx-users] gromacs on GPU
On Wed, Jan 9, 2013 at 9:17 AM, James Starlight jmsstarli...@gmail.comwrote: As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? The manual and wiki guides are meant to be read. Although they are a bit dated, the main body of information is still valid. Please read the documentation; aka RT(F)M: http://www.gromacs.org/gpu#Compiling_and_installation_of_GROMACS-GPU_from_source also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). I've seen that erorr in other posts in the mailing list but could not found any possible sollutions. It surely works with the (admittedly dated) versions the docs suggest, you mileage may vary with other versions, but in principle everything should work with the latest compilers, CUDA, and OpenMM (myself and others have built and ran successfully). by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. No, RTM. OpenMM does have an OpenCL plugin, but that's not supported in GROMACS, if you want to use it you'll have to write your own code around the OpenMM library. We have no resources to implement it ATM, but as GROMACS is OSS anyone is welcome to contribute it. And let me reiterate, if you're running explicit solvent you should be using the native GPU acceleration as you've been told before. Cheers, Szilard James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com wrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On Wed, Jan 9, 2013 at 10:27 AM, James Starlight jmsstarli...@gmail.comwrote: I've solved previous problem but havent been able to launch mdrun-openmm. Below you can find mdrun's output Back Off! I just backed up md_test.log to ./#md_test.log.3# Reading file md_test.tpr, VERSION 4.6-beta3 (single precision) Changing nstlist from 10 to 40, rlist from 1 to 1.146 Compiled acceleration: None (Gromacs could use AVX_256 on this machine, which is better) Have you set the CPU acceleration manually to None? If you did *don't* do that, let the build system pick what's best for you. However, if you did not do that, there is some issue with our auto-detection; a log file as a start would possibly help in identifying the issue. 1 GPU detected: #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC: no, stat: compatible 1 GPU auto-selected to be used for this run: #0 Program mdrun-openmm, VERSION 4.6-beta3 Source code file: /home/own/gromacs-4.6-beta3/src/kernel/openmm_wrapper.cpp, line: 1367 Fatal error: OpenMM exception caught while initializating: Error setting device flags cannot set while device is active in this process For more information and tips for troubleshooting, please check the GROMACS website at http://www.gromacs.org/Documentation/Errors I'm still using cuda-5-toolkit with the 304.54 driver. Also my question about openCL is still valid. I've browesed manual but didnt found how I can compile gromacs with openCL instead of Cuda. On openMM group developers told me that openCL would give much performance for gpu-accelerated simulation in comparison to the CUDA. James 2013/1/9 James Starlight jmsstarli...@gmail.com: As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). I've seen that erorr in other posts in the mailing list but could not found any possible sollutions. by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com wrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to
Re: [gmx-users] gromacs on GPU
Dear James, On Wed, Jan 9, 2013 at 6:17 PM, James Starlight jmsstarli...@gmail.comwrote: Roland, indeed the error was that I'have compilate mdrun-openmm which is not the native gpu. now I've made mdrun via cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 and obtain workable gromacs. My test system consist of calmodulin (charmm27) solvated in tip3p water. total size of the system: 67752 atoms. I have set below values on the mdp file as grompp told me nstlist = 20 cutoff-scheme = Verlet ; !not quite sure whai is this! Again RTM: http://www.gromacs.org/Documentation/Cut-off_schemeshttp://www.gromacs.org/Documentation/Cut-off_schemes?highlight=verlet+scheme vdwtype = cut-off Now on the mdrun output I have Using 1 MPI thread Using 4 OpenMP threads 1 GPU detected: #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC: no, stat: compatible 1 GPU auto-selected to be used for this run: #0 does it mean that all 4 cores of my CPU + gpu are used at same time ? Yes. is there any other ways to increase performance ? ( e.g I'm not quite sure if open_mpi is used with that build because during compilation cmake asked me only about fftw3 libs). There could be, but I/we can't well without more information on what and how you compiled and ran. The minimum we need is a log file. Is there any ways to monitor total performance ( e.g separate cpu and gpu usage ) ? See the performance table and summary at the end of the log file. For further CPU monitoring you can use top, for GPUs the nvidia-smi tool that comes together with the driver has some monitoring capabilities. Finally, I would strongly suggest that you check the links I posted in the first mail. There is plenty of documentation and general description on the heterogeneous (CPU+GPU) acceleration on the wiki and mailing list; the google results of the searches I hinted will get you exactly that and more. Let me reiterate: while questions and general interest are appreciated, the *minimum* expectation is that you at least read the relevant part of the documentation and related mailing list posts, especially if this has been already suggested to you. Please do us and yourself a favor and browse through the available resources. Cheers, -- Szilárd Thanks for suggestions, James 2013/1/9 Roland Schulz rol...@utk.edu: On Wed, Jan 9, 2013 at 3:17 AM, James Starlight jmsstarli...@gmail.com wrote: As I understood that gromacs version already has included openMM so the installation of the external openMM sources is not needed, isnt it ? No the new build in GPU implementation and openMM are two different things. The Gromacs-OpenMM interface isn't actively maintained and thus not recommended. also I wounder to know what exactly CUDA version is needed ? For example I've tried lattest cuda-5.0 but with that version i've obtain error from mdrun-openmm that platform cuda was not detected (gromacs was compilated without any errors). with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine. by the way is it possible to compilate gromacs-4.6 agains other platgorm ( e.g openCL) ? I have no problems with the compatibility of the openCL and openMM. GMX_GPU doesn't support openCL. Roland James 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.com wrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing
Re: [gmx-users] gromacs on GPU
Dear Szilárd, thanks for help again! 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: There could be, but I/we can't well without more information on what and how you compiled and ran. The minimum we need is a log file. I've compilated gromacs 4.6-3 beta via simple cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 make sudo make install I have not added any special params to the grompp or mdrun. After that I've run tested simulation of the calmodulin in explicit water ( 60k atoms ) 100ps and obtain next output Host: starlight pid: 21028 nodeid: 0 nnodes: 1 Gromacs version:VERSION 4.6-beta3 Precision: single MPI library:thread_mpi OpenMP support: enabled GPU support:enabled invsqrt routine:gmx_software_invsqrt(x) CPU acceleration: AVX_256 FFT library:fftw-3.3.2-sse2-avx Large file support: enabled RDTSCP usage: enabled Built on: Wed Jan 9 20:44:51 MSK 2013 Built by: own@starlight [CMAKE] Build OS/arch: Linux 3.2.0-2-amd64 x86_64 Build CPU vendor: GenuineIntel Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Build CPU family: 6 Model: 58 Stepping: 9 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3 C compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG C++ compiler: /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3 C++ compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG CUDA compiler: nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2012 NVIDIA Corporation;Built on Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0, V0.2.1221 CUDA driver:5.0 CUDA runtime: 5.0 Core t (s) Wall t (s)(%) Time: 2770.700 1051.927 263.4 (ns/day)(hour/ns) Performance:8.2142.922 full log can be found here http://www.sendspace.com/file/inum84 Finally when I check CPU usage I notice that only 1 CPU was full loaded ( 100%) and 2-4 cores were loaded on only 60% but gave me strange results that GPU is not used (I've only monitored temperature of video card and noticed increase of the temperature up to 65 degrees ) +--+ | NVIDIA-SMI 4.304.54 Driver Version: 304.54 | |---+--+--+ | GPU Name | Bus-IdDisp. | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===+==+==| | 0 GeForce GTX 670 | :02:00.0 N/A | N/A | | 38% 63C N/A N/A / N/A | 9% 174MB / 2047MB | N/A Default | +---+--+--+ +-+ | Compute processes: GPU Memory | | GPU PID Process name Usage | |=| |0Not Supported | +-+ Thanks for help again, James -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Hi, is this an implicit water calculation? If so it shouldn't use PME. Roland On Wed, Jan 9, 2013 at 2:27 PM, James Starlight jmsstarli...@gmail.comwrote: Dear Szilárd, thanks for help again! 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: There could be, but I/we can't well without more information on what and how you compiled and ran. The minimum we need is a log file. I've compilated gromacs 4.6-3 beta via simple cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 make sudo make install I have not added any special params to the grompp or mdrun. After that I've run tested simulation of the calmodulin in explicit water ( 60k atoms ) 100ps and obtain next output Host: starlight pid: 21028 nodeid: 0 nnodes: 1 Gromacs version:VERSION 4.6-beta3 Precision: single MPI library:thread_mpi OpenMP support: enabled GPU support:enabled invsqrt routine:gmx_software_invsqrt(x) CPU acceleration: AVX_256 FFT library:fftw-3.3.2-sse2-avx Large file support: enabled RDTSCP usage: enabled Built on: Wed Jan 9 20:44:51 MSK 2013 Built by: own@starlight [CMAKE] Build OS/arch: Linux 3.2.0-2-amd64 x86_64 Build CPU vendor: GenuineIntel Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Build CPU family: 6 Model: 58 Stepping: 9 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3 C compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG C++ compiler: /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3 C++ compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG CUDA compiler: nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2012 NVIDIA Corporation;Built on Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0, V0.2.1221 CUDA driver:5.0 CUDA runtime: 5.0 Core t (s) Wall t (s)(%) Time: 2770.700 1051.927 263.4 (ns/day)(hour/ns) Performance:8.2142.922 full log can be found here http://www.sendspace.com/file/inum84 Finally when I check CPU usage I notice that only 1 CPU was full loaded ( 100%) and 2-4 cores were loaded on only 60% but gave me strange results that GPU is not used (I've only monitored temperature of video card and noticed increase of the temperature up to 65 degrees ) +--+ | NVIDIA-SMI 4.304.54 Driver Version: 304.54 | |---+--+--+ | GPU Name | Bus-IdDisp. | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===+==+==| | 0 GeForce GTX 670 | :02:00.0 N/A | N/A | | 38% 63C N/A N/A / N/A | 9% 174MB / 2047MB | N/A Default | +---+--+--+ +-+ | Compute processes: GPU Memory | | GPU PID Process name Usage | |=| |0Not Supported | +-+ Thanks for help again, James -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- ORNL/UT Center for Molecular Biophysics cmb.ornl.gov 865-241-1537, ORNL PO BOX 2008 MS6309 -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Hi James, The build looks mostly fine except that you are using fftw3 compiled with AVX which is slower than with only SSE (even on AVX-capable CPUs) - you should have been warned about this at configure-time. Now, performance-wise everything looks fine except that with a 1.2 nm cut-off your GPU is not able to keep up with the CPU and finish the non-bonded work before the CPU is done with Bonded + PME. That's why you see the Wait GPU taking 20% of the total time and that's also why you see some cores idling (because for 20% of the run-time thread 0 on core 0 is blocked waiting for the GPU while the rest idle). As the suggestion at the end of the log file point out, you can consider using a shorter cut-off which will push more work back to the PME on the CPU, but whether you can do this it depends on your very problem. There is one more alternative of running two MPI processes on the GPU (mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will execute part of the nonbonded on the CPU, but this might not help. Cheers, -- Szilárd On Wed, Jan 9, 2013 at 8:27 PM, James Starlight jmsstarli...@gmail.comwrote: Dear Szilárd, thanks for help again! 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: There could be, but I/we can't well without more information on what and how you compiled and ran. The minimum we need is a log file. I've compilated gromacs 4.6-3 beta via simple cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 make sudo make install I have not added any special params to the grompp or mdrun. After that I've run tested simulation of the calmodulin in explicit water ( 60k atoms ) 100ps and obtain next output Host: starlight pid: 21028 nodeid: 0 nnodes: 1 Gromacs version:VERSION 4.6-beta3 Precision: single MPI library:thread_mpi OpenMP support: enabled GPU support:enabled invsqrt routine:gmx_software_invsqrt(x) CPU acceleration: AVX_256 FFT library:fftw-3.3.2-sse2-avx Large file support: enabled RDTSCP usage: enabled Built on: Wed Jan 9 20:44:51 MSK 2013 Built by: own@starlight [CMAKE] Build OS/arch: Linux 3.2.0-2-amd64 x86_64 Build CPU vendor: GenuineIntel Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Build CPU family: 6 Model: 58 Stepping: 9 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3 C compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG C++ compiler: /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3 C++ compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG CUDA compiler: nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2012 NVIDIA Corporation;Built on Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0, V0.2.1221 CUDA driver:5.0 CUDA runtime: 5.0 Core t (s) Wall t (s)(%) Time: 2770.700 1051.927 263.4 (ns/day)(hour/ns) Performance:8.2142.922 full log can be found here http://www.sendspace.com/file/inum84 Finally when I check CPU usage I notice that only 1 CPU was full loaded ( 100%) and 2-4 cores were loaded on only 60% but gave me strange results that GPU is not used (I've only monitored temperature of video card and noticed increase of the temperature up to 65 degrees ) +--+ | NVIDIA-SMI 4.304.54 Driver Version: 304.54 | |---+--+--+ | GPU Name | Bus-IdDisp. | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===+==+==| | 0 GeForce GTX 670 | :02:00.0 N/A | N/A | | 38% 63C N/A N/A / N/A | 9% 174MB / 2047MB | N/A Default | +---+--+--+ +-+ | Compute processes: GPU Memory | | GPU PID Process name Usage | |=| |0Not Supported | +-+
Re: [gmx-users] gromacs on GPU
Szilárd , thanks again for explanation! Today I've performed some tests on my calmodulin in water system with different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8 respectually) Below you can see that the highest performance was in case of 0.8 cut-offs all cut-offs 1.0 Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305 NOTE: The GPU has 20% more load than the CPU. This imbalance causes performance loss, consider using a shorter cut-off and a finer PME grid. Core t (s) Wall t (s)(%) Time: 1313.420 464.035 283.0 (ns/day)(hour/ns) Performance:9.3102.578 Finished mdrun on node 0 Thu Jan 10 09:39:23 2013 all cut-offs 0.9 Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059 Core t (s) Wall t (s)(%) Time: 2414.930 856.179 282.1 (ns/day)(hour/ns) Performance: 10.0922.378 Finished mdrun on node 0 Thu Jan 10 10:09:52 2013 all cut-offs 0.8 Force evaluation time GPU/CPU: 4.001 ms/4.659 ms = 0.859 Core t (s) Wall t (s)(%) Time: 1166.390 413.598 282.0 (ns/day)(hour/ns) Performance: 10.4452.298 Finished mdrun on node 0 Thu Jan 10 09:50:33 2013 Also I've noticed that 2-4 CPU cores usage in 2 and 3rd case was only 67%. Is there any other ways to increase performance by means of neighboor search parameters ( e.g nstlist etc) ? Might such reduced cut-off be used with the force fields ( e,g charmm) where initially usage of longest cut-offs have given better results (e,g in charmm27 and gromos56 I always use 1.2 and 1.4 nm for rvdw, respectually) ? James 2013/1/10 Szilárd Páll szilard.p...@cbr.su.se: Hi James, The build looks mostly fine except that you are using fftw3 compiled with AVX which is slower than with only SSE (even on AVX-capable CPUs) - you should have been warned about this at configure-time. Now, performance-wise everything looks fine except that with a 1.2 nm cut-off your GPU is not able to keep up with the CPU and finish the non-bonded work before the CPU is done with Bonded + PME. That's why you see the Wait GPU taking 20% of the total time and that's also why you see some cores idling (because for 20% of the run-time thread 0 on core 0 is blocked waiting for the GPU while the rest idle). As the suggestion at the end of the log file point out, you can consider using a shorter cut-off which will push more work back to the PME on the CPU, but whether you can do this it depends on your very problem. There is one more alternative of running two MPI processes on the GPU (mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will execute part of the nonbonded on the CPU, but this might not help. Cheers, -- Szilárd On Wed, Jan 9, 2013 at 8:27 PM, James Starlight jmsstarli...@gmail.comwrote: Dear Szilárd, thanks for help again! 2013/1/9 Szilárd Páll szilard.p...@cbr.su.se: There could be, but I/we can't well without more information on what and how you compiled and ran. The minimum we need is a log file. I've compilated gromacs 4.6-3 beta via simple cmake CMakeLists.txt -DGMX_GPU=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0 make sudo make install I have not added any special params to the grompp or mdrun. After that I've run tested simulation of the calmodulin in explicit water ( 60k atoms ) 100ps and obtain next output Host: starlight pid: 21028 nodeid: 0 nnodes: 1 Gromacs version:VERSION 4.6-beta3 Precision: single MPI library:thread_mpi OpenMP support: enabled GPU support:enabled invsqrt routine:gmx_software_invsqrt(x) CPU acceleration: AVX_256 FFT library:fftw-3.3.2-sse2-avx Large file support: enabled RDTSCP usage: enabled Built on: Wed Jan 9 20:44:51 MSK 2013 Built by: own@starlight [CMAKE] Build OS/arch: Linux 3.2.0-2-amd64 x86_64 Build CPU vendor: GenuineIntel Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Build CPU family: 6 Model: 58 Stepping: 9 Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3 C compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG C++ compiler: /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3 C++ compiler flags: -mavx -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG CUDA compiler: nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2012 NVIDIA
Re: [gmx-users] gromacs on GPU
So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On Tue, Jan 8, 2013 at 3:22 PM, James Starlight jmsstarli...@gmail.comwrote: So could someone provide me more about gpu-accelerated MD implemented in the 4.6 gromacs ? Does it require openMM (what version is supported FYI, if nobody can, trust G: http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration http://lmgtfy.com/?q=gromacs+4.6+installation+instructions The wiki and mailing list contains quite extensive information (indexed by G). Otherwise, release notes (not final): http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x Install guide is at the expected location: http://www.gromacs.org/Documentation/Installation_Instructions Cheers, -- Szilárd for that gromacs release ?) installed? By the way at present time I force with the problem of compilation 4.1.1 openMM (i need to compile openMM because of cuda-5.0 ). If someone have done it (openMM 4.11 +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know. James 2013/1/7 James Starlight jmsstarli...@gmail.com: Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Hi James, Don't use the OpenMM accelerated mdrun for explicit solvent simulations. For such runs any modern CPU with 4-8 cores will probably be as fast or faster than most GPUs. However, do consider the 4.6 *native* heterogeneous GPU-accelerated code. This new efficient code can provide 2-4x speedup and uses both CPU and GPU. To get the latest 4.6 version use the 4.6-beta3 release (http://goo.gl/XFPoe) or even better, get the code from git (git clone git:// git.gromacs.org/gromacs -b release-4-6). Let us know how things worked out! Cheers, -- Szilárd On Mon, Jan 7, 2013 at 8:05 AM, James Starlight jmsstarli...@gmail.comwrote: Today I've tried to test gpu performance on the benchmark system ( explicit solvent with pme ) grompp-gpu -f md_sd.mdp -c conf -p topol.top -o md_test mdrun-gpu -device OpenMM:platform=Cuda,DeviceID=1,Memtest=15,force-device=no -v -deffnm md_test that produce the error Program mdrun-gpu, VERSION 4.5.5 Source code file: /home/own/gromacs_gpu/src/kernel/openmm_wrapper.cpp, line: 1365 Fatal error: OpenMM exception caught while initializating: cudaMemcpyToSymbol: SetSim copy to cSim failed invalid device symbol For more information and tips for troubleshooting, please check the GROMACS website at http://www.gromacs.org/Documentation/Errors In the mailing list archive I've found the same error with the same card posted some time ago http://article.gmane.org/gmane.science.biology.gromacs.user/55577/match=openmm+exception+caught+while+initializating+cudaMemcpyToSymbol+setsim+copy+cSim+failed+invalid+device+symbol From that topic I've understood that 670 graphic card is not supported yet in Gromacs (4.55). I've checked the manual for the 4.6 beta version and does not found anything about supporting of that card in the lattest gromacs realeses as well. IS there any ways to solve that problem ? Could that error be linked with some others ? E.g I've installed nvidia drivers + cuda first and openMM 4.01 than. During gromacs compilation I've not forced with any errors. Thanks for help James 2013/1/6 Justin Lemkul jalem...@vt.edu: On 1/6/13 1:53 PM, James Starlight wrote: OK! I've compilated gromacs-gpu from the source using that tutorial for the Debian OS http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html The only thing that it lack is the installation mdrun_d-gpu but i'm not sure that double precission can be used with gpu. Could someone provide me with some tutorial which would show me basic mdrun options with the calculation on my gpu as well as some testing system? Everything you need to know (including benchmark systems) is posted at http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs. The invocation of mdrun-gpu is like any other mdrun, with GPU-specific information available by reading mdrun-gpu -h. -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Hi Szilárd! As I understood you correctly gromacs-4.6 have specific algorithm (independent on openMM?) for gpu-based calculations havent ? If it true how I should compilate such new gpu-based gromacs? In the gromacs-4.6-beta-3 folder I've found instructuon for the standart installation via cmake cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF James 2013/1/7 Szilárd Páll szilard.p...@cbr.su.se: Szilárd -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] gromacs on GPU
Dear Gromacs Users! I want to install gromacs 4.55 on my Debian OS with the GPU support (I'd like to make some calculations on the GeForce 670 gpu). I have cuda-toolkit as well as openMM 4.01 installed from the binaries as well as gromac's sources. Could you provide me with some tutorial where I could found such step-by-step installation of the Gromacs with gpu. Thanks for help James -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On 1/6/13 4:48 AM, James Starlight wrote: Dear Gromacs Users! I want to install gromacs 4.55 on my Debian OS with the GPU support (I'd like to make some calculations on the GeForce 670 gpu). I have cuda-toolkit as well as openMM 4.01 installed from the binaries as well as gromac's sources. Could you provide me with some tutorial where I could found such step-by-step installation of the Gromacs with gpu. http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs#Installing_and_running_GROMACS-GPU -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Have someone tried to use gromacs-4.5-gpu binaries ? Would that packages work on debian or its better to build from sources ? :) James 2013/1/6 Justin Lemkul jalem...@vt.edu: On 1/6/13 4:48 AM, James Starlight wrote: Dear Gromacs Users! I want to install gromacs 4.55 on my Debian OS with the GPU support (I'd like to make some calculations on the GeForce 670 gpu). I have cuda-toolkit as well as openMM 4.01 installed from the binaries as well as gromac's sources. Could you provide me with some tutorial where I could found such step-by-step installation of the Gromacs with gpu. http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs#Installing_and_running_GROMACS-GPU -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On 1/6/13 7:41 AM, James Starlight wrote: Have someone tried to use gromacs-4.5-gpu binaries ? Would that packages work on debian or its better to build from sources ? :) The binaries online are very outdated. Compiling from source is rather easy, and the website below has all the steps listed. -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
OK! I've compilated gromacs-gpu from the source using that tutorial for the Debian OS http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html The only thing that it lack is the installation mdrun_d-gpu but i'm not sure that double precission can be used with gpu. Could someone provide me with some tutorial which would show me basic mdrun options with the calculation on my gpu as well as some testing system? James 2013/1/6 Justin Lemkul jalem...@vt.edu: On 1/6/13 7:41 AM, James Starlight wrote: Have someone tried to use gromacs-4.5-gpu binaries ? Would that packages work on debian or its better to build from sources ? :) The binaries online are very outdated. Compiling from source is rather easy, and the website below has all the steps listed. -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
On 1/6/13 1:53 PM, James Starlight wrote: OK! I've compilated gromacs-gpu from the source using that tutorial for the Debian OS http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html The only thing that it lack is the installation mdrun_d-gpu but i'm not sure that double precission can be used with gpu. Could someone provide me with some tutorial which would show me basic mdrun options with the calculation on my gpu as well as some testing system? Everything you need to know (including benchmark systems) is posted at http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs. The invocation of mdrun-gpu is like any other mdrun, with GPU-specific information available by reading mdrun-gpu -h. -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] gromacs on GPU
Today I've tried to test gpu performance on the benchmark system ( explicit solvent with pme ) grompp-gpu -f md_sd.mdp -c conf -p topol.top -o md_test mdrun-gpu -device OpenMM:platform=Cuda,DeviceID=1,Memtest=15,force-device=no -v -deffnm md_test that produce the error Program mdrun-gpu, VERSION 4.5.5 Source code file: /home/own/gromacs_gpu/src/kernel/openmm_wrapper.cpp, line: 1365 Fatal error: OpenMM exception caught while initializating: cudaMemcpyToSymbol: SetSim copy to cSim failed invalid device symbol For more information and tips for troubleshooting, please check the GROMACS website at http://www.gromacs.org/Documentation/Errors In the mailing list archive I've found the same error with the same card posted some time ago http://article.gmane.org/gmane.science.biology.gromacs.user/55577/match=openmm+exception+caught+while+initializating+cudaMemcpyToSymbol+setsim+copy+cSim+failed+invalid+device+symbol From that topic I've understood that 670 graphic card is not supported yet in Gromacs (4.55). I've checked the manual for the 4.6 beta version and does not found anything about supporting of that card in the lattest gromacs realeses as well. IS there any ways to solve that problem ? Could that error be linked with some others ? E.g I've installed nvidia drivers + cuda first and openMM 4.01 than. During gromacs compilation I've not forced with any errors. Thanks for help James 2013/1/6 Justin Lemkul jalem...@vt.edu: On 1/6/13 1:53 PM, James Starlight wrote: OK! I've compilated gromacs-gpu from the source using that tutorial for the Debian OS http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html The only thing that it lack is the installation mdrun_d-gpu but i'm not sure that double precission can be used with gpu. Could someone provide me with some tutorial which would show me basic mdrun options with the calculation on my gpu as well as some testing system? Everything you need to know (including benchmark systems) is posted at http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs. The invocation of mdrun-gpu is like any other mdrun, with GPU-specific information available by reading mdrun-gpu -h. -Justin -- Justin A. Lemkul, Ph.D. Research Scientist Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! * Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs 4.5.5 GPU Error
Hi, Could you compile in Debug mode and run mdurn-gpu in gdb, it will tell more about the location and type of exception. -- Szilárd On Mon, Mar 12, 2012 at 8:47 AM, TH Chew teong...@gmail.com wrote: Hi all, I am trying to get the GPU version of gromacs running. I manage to install OpenMM, compile and install mdrun-gpu. However when I ran mdrun-gpu on any of the benchmark files, I got this: ./run.sh: line 2: 14224 Floating point exception/usr/local/gromacs-4.5.5-cuda/bin/mdrun-gpu -v Any idea what goes wrong here? -- Regards, THChew -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] Gromacs 4.5.5 GPU Error
Hi all, I am trying to get the GPU version of gromacs running. I manage to install OpenMM, compile and install mdrun-gpu. However when I ran mdrun-gpu on any of the benchmark files, I got this: ./run.sh: line 2: 14224 Floating point exception/usr/local/gromacs-4.5.5-cuda/bin/mdrun-gpu -v Any idea what goes wrong here? -- Regards, THChew -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs on GPU
Sorry for the delay, but I used gcc version 4.1.2 20080704 to compile our version of GPU enabled gromacs. Matt On Sat, Jan 28, 2012 at 11:32 AM, Benjamin Hall benjamin.a.h...@ucl.ac.uk wrote: Justin A. Lemkul wrote: We get junk output with 4.5.4 if the box size is set to zero, for some odd reason, though in principle it should be ignored with pbc = no in the .mdp file. Using any non-zero box results in usable results. Our OpenMM version is the same, but Cuda is in the 3.x series (can't remember off the top of my head, sorry). I've tried setting the box size in the gro file but that doesn't seem to work (100x100x100nm). Which compiler did you use? I realise that this is an unlikely source of trouble but it's another variable to eliminate in testing. I'll update issue 568 on redmine, which has been closed but appears to be this problem. Thanks best wishes Ben -- Dr Benjamin A Hall Centre for Computational Science, Department of Chemistry, UCL benjamin.a.h...@ucl.ac.uk -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs on GPU
I should add that while we had runs that completed, all we got were NaN for the coordinates and energies. Personally, I have attributed our inability to execute the GPU code cleanly to our C2050 Teslas [they are double precision]. (As I had the GPUized code working with an earlier card, although the options were pretty limited) Matt On Sat, Jan 28, 2012 at 11:32 AM, Benjamin Hall benjamin.a.h...@ucl.ac.uk wrote: Justin A. Lemkul wrote: We get junk output with 4.5.4 if the box size is set to zero, for some odd reason, though in principle it should be ignored with pbc = no in the .mdp file. Using any non-zero box results in usable results. Our OpenMM version is the same, but Cuda is in the 3.x series (can't remember off the top of my head, sorry). I've tried setting the box size in the gro file but that doesn't seem to work (100x100x100nm). Which compiler did you use? I realise that this is an unlikely source of trouble but it's another variable to eliminate in testing. I'll update issue 568 on redmine, which has been closed but appears to be this problem. Thanks best wishes Ben -- Dr Benjamin A Hall Centre for Computational Science, Department of Chemistry, UCL benjamin.a.h...@ucl.ac.uk -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs on GPU
Benjamin Hall wrote: Justin A. Lemkul wrote: We get junk output with 4.5.4 if the box size is set to zero, for some odd reason, though in principle it should be ignored with pbc = no in the .mdp file. Using any non-zero box results in usable results. Our OpenMM version is the same, but Cuda is in the 3.x series (can't remember off the top of my head, sorry). I've tried setting the box size in the gro file but that doesn't seem to work (100x100x100nm). Which compiler did you use? I realise that this is an unlikely source of trouble but it's another variable to eliminate in testing. Ours is gcc-4.4.6 on a local machine using Cuda 3.2. On our university's GPU cluster, the installation was done with gcc-4.3.4 and Cuda 3.1. I have not tested Cuda 4.0, though it has recently become available to us so it might be worth a shot to see if this is another issue. -Justin I'll update issue 568 on redmine, which has been closed but appears to be this problem. Thanks best wishes Ben -- Dr Benjamin A Hall Centre for Computational Science, Department of Chemistry, UCL benjamin.a.h...@ucl.ac.uk mailto:benjamin.a.h...@ucl.ac.uk -- Justin A. Lemkul Ph.D. Candidate ICTAS Doctoral Scholar MILES-IGERT Trainee Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] Gromacs on GPU
Hi I've been attempting to use the GPU enabled version of gromacs to run implicit solvent simulations, but I've run into an odd problem. I have downloaded and can run the benchmark simulations from the gromacs website, and reproduce the speeds reported there. However, when I edit the MDP files to write out data both the xtc and trr files contain junk data- the first frame is correct but every coordinate in the following frames is the same. These same MDP files work on the GPU disabled (default) gromacs and produce reasonable trajectories. As the simulations aren't crashing I assume that the issue relates to writing specifically; has anyone observed the same problem, and is there a workaround or patch? The versions of each piece of software used are gromacs 4.5.5 openmm 3.1.1 cuda 4.0 and all compiled using intel compiler version 11.1/072 Thanks in advance Ben -- Dr Benjamin A Hall Centre for Computational Science, Department of Chemistry, UCL benjamin.a.h...@ucl.ac.uk -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs on GPU
Sorry to hear that, as I have had the same problem for over four months (OpenMM 3+, Gromacs 4.5.4+, Cuda 4, etc.). No one ever got back to me, and I just abandoned Gromacs for Amber (who's GPU accelerated code base compiles and works). I would be interested to hear if the intervening time, if a plan had been hatched to make the GPU code production quality. If that has indeed happened. Matthew On Fri, Jan 27, 2012 at 9:18 AM, Ben Hall benjamin.a.h...@ucl.ac.uk wrote: Hi I've been attempting to use the GPU enabled version of gromacs to run implicit solvent simulations, but I've run into an odd problem. I have downloaded and can run the benchmark simulations from the gromacs website, and reproduce the speeds reported there. However, when I edit the MDP files to write out data both the xtc and trr files contain junk data- the first frame is correct but every coordinate in the following frames is the same. These same MDP files work on the GPU disabled (default) gromacs and produce reasonable trajectories. As the simulations aren't crashing I assume that the issue relates to writing specifically; has anyone observed the same problem, and is there a workaround or patch? The versions of each piece of software used are gromacs 4.5.5 openmm 3.1.1 cuda 4.0 and all compiled using intel compiler version 11.1/072 Thanks in advance Ben -- Dr Benjamin A Hall Centre for Computational Science, Department of Chemistry, UCL benjamin.a.h...@ucl.ac.uk -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] Gromacs on GPU
Matthew Lardy wrote: Sorry to hear that, as I have had the same problem for over four months (OpenMM 3+, Gromacs 4.5.4+, Cuda 4, etc.). No one ever got back to me, and I just abandoned Gromacs for Amber (who's GPU accelerated code base compiles and works). I would be interested to hear if the intervening time, if a plan had been hatched to make the GPU code production quality. If that has indeed happened. I don't see a corresponding issue posted to redmine.gromacs.org - has there been one filed? If not, reports often get lost on the mailing list, so the only way to ensure action is to post a bug report. We get junk output with 4.5.4 if the box size is set to zero, for some odd reason, though in principle it should be ignored with pbc = no in the .mdp file. Using any non-zero box results in usable results. Our OpenMM version is the same, but Cuda is in the 3.x series (can't remember off the top of my head, sorry). -Justin -- Justin A. Lemkul Ph.D. Candidate ICTAS Doctoral Scholar MILES-IGERT Trainee Department of Biochemistry Virginia Tech Blacksburg, VA jalemkul[at]vt.edu | (540) 231-9080 http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] GROMACS with GPU
Hi! If I may ask, but what is the gromacs version that is planned on the roadmap to begin to work with the newer version of openmm? Specially interested due to ATI support Regards Jones On Thu, Aug 18, 2011 at 10:01 PM, Szilárd Páll szilard.p...@cbr.su.sewrote: Hi, Gromacs 4.5.x works only with OpenMM 2.x. You might be able to use CUDA 4.0, but you will probably have to recompile OpenMM from source. Cheers, -- Szilárd On Fri, Aug 19, 2011 at 2:23 AM, Park, Jae Hyun nmn pa...@ornl.gov wrote: Dear GMX users, I am installing GMX 4.5.3 with GPU. But, the CUDA in my GPU machine is ver 4.0. Can I use this version of CUDA and openmm-3.1.1? Or, do I have to re-install CUDA 3.2 and openmm-2.0 as described in manual? Thank you in advance for your kind instruction. Jae H. Park === Jae Hyun Park, Ph. D. Physics Division Oak Ridge National Laboratory PO. Box 2008, MS-6372 Oak Ridge, TN 37831 Phone (865) 241-1482 E-mail pa...@ornl.gov -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] GROMACS with GPU
Dear GMX users, I am installing GMX 4.5.3 with GPU. But, the CUDA in my GPU machine is ver 4.0. Can I use this version of CUDA and openmm-3.1.1? Or, do I have to re-install CUDA 3.2 and openmm-2.0 as described in manual? Thank you in advance for your kind instruction. Jae H. Park === Jae Hyun Park, Ph. D. Physics Division Oak Ridge National Laboratory PO. Box 2008, MS-6372 Oak Ridge, TN 37831 Phone (865) 241-1482 E-mail pa...@ornl.gov -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
Re: [gmx-users] GROMACS with GPU
Hi, Gromacs 4.5.x works only with OpenMM 2.x. You might be able to use CUDA 4.0, but you will probably have to recompile OpenMM from source. Cheers, -- Szilárd On Fri, Aug 19, 2011 at 2:23 AM, Park, Jae Hyun nmn pa...@ornl.gov wrote: Dear GMX users, I am installing GMX 4.5.3 with GPU. But, the CUDA in my GPU machine is ver 4.0. Can I use this version of CUDA and openmm-3.1.1? Or, do I have to re-install CUDA 3.2 and openmm-2.0 as described in manual? Thank you in advance for your kind instruction. Jae H. Park === Jae Hyun Park, Ph. D. Physics Division Oak Ridge National Laboratory PO. Box 2008, MS-6372 Oak Ridge, TN 37831 Phone (865) 241-1482 E-mail pa...@ornl.gov -- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] Gromacs on GPU: GTX or Tesla?
Hi, We want to start using Gromacs on GPU and we are debating whether to purchase GeForce GTX card or Tesla platform. Looking at the published data it seems GTX580 has much impressive performance compared to Tesla C2050, and of course it is much cheaper. Is there a reason to prefer Tesla over GTX for running Gromacs in the short or long term? Thanks, Efrat -- gmx-users mailing listgmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
[gmx-users] GROMACS and GPU (and PS3)
Hi everybody, I wonder if GROMACS will eventually support running on the graphics cards, especially since there have been several announcements of extremely powerful cards like AMD's FireStream recently. I know that the question has repeatedly been asked some time ago, but hey, the Stanford guys from [EMAIL PROTECTED] did manage to adapt GROMACS for GPUs, and they don't seem to regret it :) I'd also like to know if anyone has actually tried running GROMACS on PlayStation 3, seems like this Cell CPU could perform really well for moderately sized systems (so that you don't run into the 512 MB memory limitation)... Best regards, Vasilii ___ gmx-users mailing listgmx-users@gromacs.org http://www.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to [EMAIL PROTECTED] Can't post? Read http://www.gromacs.org/mailing_lists/users.php